Back to Community
Confusion regarding zipline functions in research environment and local, stand-alone notebook

There appear to be inconsistencies regarding the use of the zipline functions 'symbol' and 'symbols' between the research environment and using them in a local, stand-alone IPython Notebook, as I have done before the research environment was available (and may still want to do to use apps/functions not available or black-listed in the research environment).

As this notebook shows, importing zipline function 'symbols' results in unexpected result. I know that there is no need to import it as it already exists in the research environment, but blocking it might be a good idea to avoid confusion. Some zipline functions need to be imported at times and most seem ok, but again it might be worthwhile to block any that won't work properly in the research environment.

Also, should the functions 'symbol' and/or 'symbols' work in a local, standalone notebook (Anaconda 3 32-bit with python 2.7 OR python 3?

Loading notebook preview...
Notebook previews are currently unavailable.
15 responses

@Dave

I think the primary confusion here is that the symbols method that's pre-populated into the Research namespace and the symbols function that you imported from zipline.api are actually two entirely different functions, which unfortunately happen to have the same name.

The symbols function that's in the namespace by default is a function specially-written for research, which allows for ad-hoc querying of symbol information. It's designed to always work, and internally it's querying a Quantopian symbol resolution database.

zipline.api is a much older module that's part of our standalone Zipline package. It provides a bunch of functions that are only intended to be executed during calls to TradingAlgorithm.run(). Most of these functions are things like "order", or "schedule_function", which only really make sense in the context of a running TradingAlgorithm. zipline.api.symbols is a bit of an exception here, since you could plausibly imagine querying for symbol resolution outside the context of a running algorithm. The specific semantics of that method, however, mean that zipline.api.symbols actually does depend on a currently-running TradingAlgorithm in a few subtle ways:

  1. It uses the algorithm's current date to resolve symbols (though I actually consider this a bug, documented here: https://github.com/quantopian/zipline/issues/632 ).
  2. Zipline provides APIs for users to specify their own symbol-mapping data. zipline.api.symbols uses this data if it's available.

I know that there is no need to import it as it already exists in the research environment, but blocking it might be a good idea to avoid confusion.

This isn't quite true. These are two different functions with two different purposes: you should use the built-in symbols when you're doing ad-hoc querying outside the backtester. You should use zipline.api.symbols if you want to refer to a specific asset by symbol during execution of a TradingAlgorithm. Using the built-in symbols during a TradingAlgorithm run would cause you to make a web request to our database on every tick of your simulation, which will be quite slow.

I think the confusion here is exacerbated by the fact that we pre-populate a bunch of names into the Research Environment namespace. If users were required to do something like from quantopian.api import symbols, get_pricing, get_backtest, I think it would alleviate the issue.

-Scott

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

thanks for this Scott.

@Dave no problem. I'd be curious to hear your thoughts on my proposal to require imports of the pre-populated names in the Research Environment. While it's initially convenient to not have to do the imports at the top of the file, the Python language requires imports precisely to avoid these sorts of confusions where you have functions with the same names in two different modules, and I suspect that we've sacrificed too much clarity in the name of convenience here.

If I could, I'd propose requiring imports in the backtester as well, but breaking three years' worth of shared algorithms is a tough sell.

The research environment, at least, is still in beta, so if we're going to make this change, it's better to make it now rather than later.

@Scott I'm in favor of yr suggestion. It's a research environment, after all, and most developers using it will be importing other stuff anyway. I understand the need to make the environment as friendly as possible, but I think adding some imports to older shared algos is a small price to pay if it reduces confusion.

What happened to zipline.api.symbols? It has disappeared.

Does this line of code not work?

from zipline.api import symbols

https://github.com/quantopian/zipline/blob/4770c5956c350eab384d3f6cfba297cc3a468a05/zipline/algorithm.py#L758

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

No, James it doesn't. And I can't find "symbols" in zipline.api either. Here is the result of running the import statement:

In [28]: runfile('C:/Users/Anthony FJ Garner/Documents/Python Scripts/untitled0.py', wdir='C:/Users/Anthony FJ Garner/Documents/Python Scripts')  
Traceback (most recent call last):

  File "<ipython-input-28-3599d41217a3>", line 1, in <module>  
    runfile('C:/Users/Anthony FJ Garner/Documents/Python Scripts/untitled0.py', wdir='C:/Users/Anthony FJ Garner/Documents/Python Scripts')

  File "C:\Anaconda\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 685, in runfile  
    execfile(filename, namespace)

  File "C:\Anaconda\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 71, in execfile  
    exec(compile(scripttext, filename, 'exec'), glob, loc)

  File "C:/Users/Anthony FJ Garner/Documents/Python Scripts/untitled0.py", line 26, in <module>  
    from zipline.api import symbols

ImportError: cannot import name symbol  

I loaded with anaconda. My algorithm.py does NOT include:

@api_method  
    def symbols(self, *args):  
        """  
        Default symbols lookup for any source that directly maps the  
        symbol to the Asset (e.g. yahoo finance).  
        """  
        return [self.symbol(identifier) for identifier in args]

I run zipline using anaconda but have a separate installation of zipline through Source Tree which has the full github. In the github version of algorithm.py, this @api method IS included. But I can not get zipline running through this separate installation outside of Anaconda.

I am trying to port systems between Quantopian and zipline and as you can imagine, the absence of the "symbols" method in the Anaconda installation is making life difficult.

I used conda install -c Quantopian zipline and tried updating. I'm told I have the latest files....but?

Anthony, is the pip install working for you? I would suggest to head to the Zipline Google group to see how others are using conda and pip: https://groups.google.com/forum/#!forum/zipline

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

I have just reinstalled a new system (ubuntu 15.04)

sudo pip install zipline works well and installed 0.8RC1 ...
from zipline.api import * and from zipline.api import symbols works without problem.

I did compile, from their github repo, separately cython, numpy and scipy but I believe pip installed different versions anyways.

Many thanks Florent. I'm going to try and load the github version in Development Mode within Anaconda. I can understand the total lack of support by Quantopian for Zipline on the other hand I have no intention of using Quantopian as a sole development environment in case their venture does not work out.

Generous of the Quantopian team to open source Zipline in any case although no doubt that also has its benefits for them.

Many thanks Florent. I'm going to try and load the github version in Development Mode within Anaconda. I can understand the total lack of support by Quantopian for Zipline on the other hand I have no intention of using Quantopian as a sole development environment in case their venture does not work out.

Generous of the Quantopian team to open source Zipline in any case although no doubt that also has its benefits for them.

Many thanks Florent. I'm going to try and load the github version in Development Mode within Anaconda. I can understand the total lack of support by Quantopian for Zipline on the other hand I have no intention of using Quantopian as a sole development environment in case their venture does not work out.

Generous of the Quantopian team to open source Zipline in any case although no doubt that also has its benefits for them.