Back to Community
Iterating Through Every SID to Screen all Stocks

Can I run a for loop from 0 to 45526, then plug each value in as an SID, check if that stock meets my requirements, if so append to a list, if not move to the next stock? Is there a better way of doing this?

8 responses

This should be possible, but it isn't. Both sid(N) and symbol(SYM) require a constant literal, and throw an error if they get a variable or expression.

Shoot, there goes my plan to backtest all possible pairs trading combinations.... (which I am sure has been done by Wall Street quants).

Tristan,

I think Simon was searching for all pairs trading but gave up some time ago (there is a thread somewhere...). I suspect finding the needle in the haystack is harder than it appears.

As for symbols (in research), Grant had originally used the following:

import zipline.api

s = range(1,100000)  
stocks = zipline.api.symbols(s)  
sids = [stock.sid for stock in stocks]  

But this currently fails for me.

Doing get_pricing for all sids kills the server, as you might expect. Doing 100 at a time can work though.

So no one has been able to check all stocks? Can any Quantopian employee comment?

It's a computing/server limitation
(I believe the limit is up from 200 to 400 now?)

You will indeed run into server limitations if you try to run this is a notebook. We're working to make this easier; to unlock a new slew of analyses and algorithms by screening and filtering criteria. This is a teaser of what's to come. This is active on our radar.

And you can now trade 500 securities in your algorithm, up from the earlier 200 limit.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

What happened to zipline.api.symbols?

In the research platform, something like this is doable:

https://www.quantopian.com/posts/analysis-of-entire-quantopian-database-using-quantopian-research-platform

It should be feasible, then, to copy-and-paste a subset to securities into the backtester (or you could run the backtest right in the research platform). The actual limit is greater than 500 securities. Somewhere, I showed that the limit is actually 500 added per day, using update_universe(). So, within 5 days, 2500 securities could be in your universe.