EDIT: The updated version of this algorithm uses Q1500US, one of the pipeline's built-in filters. Lesson 11 from the Pipeline tutorial introduces these built-in filters and provides a brief explanation on how they are used to specify a downsized base universe. More detail on the selection criteria of these filters can be found here.
About a week ago Scott Sanderson posted about Pipeline's new support for string data, and included a notebook demonstrating how to narrow down the trading universe. These filters get rid of a lot of the equities that an algorithm generally shouldn't trade, like non-primary shares.
Specifically, there are nine filters, checking that the equity:
1. is a common stock
2. doesn't have a name indicating it's a limited partnership (LP)
3. doesn't have a company reference entry indicating it's a limited partnership
4. has fundamental data associated with it in Morningstar (isn't an ETF)
5. isn't over-the-counter
6. isn't When Issued
7. isn't a depository receipt
8. is primary share
9. has a high dollar volume
Pipeline is a powerful tool that opens up access to the full range of 8000+ equities in Quantopian's database. This includes a wide variety of types of equities, including ETFs, ADRs, non-primary shares, etc. Different types of equities can behave differently or have different data available. For example, ETFs don't have fundamental data. As a result, a model can be heavily dependent on the types of equities that it's using. These filters can help select the types of equities that you want to trade.
We wanted to draw some attention to these filters, since this type of security selection is a best practice we recommend. I've attached an example mean-reversion algorithm that makes use of the filters, and selects from a generic set of liquid and common stocks. Clone this algorithm and try using these filters in your own strategy!
|Returns||1 Month||3 Month||6 Month||12 Month|
|Alpha||1 Month||3 Month||6 Month||12 Month|
|Beta||1 Month||3 Month||6 Month||12 Month|
|Sharpe||1 Month||3 Month||6 Month||12 Month|
|Sortino||1 Month||3 Month||6 Month||12 Month|
|Volatility||1 Month||3 Month||6 Month||12 Month|
|Max Drawdown||1 Month||3 Month||6 Month||12 Month|
""" Utilizes the filters for a good trading universe laid out by Scott Sanderson: https://www.quantopian.com/posts/string-columns-now-available-in-pipeline """ from quantopian.algorithm import attach_pipeline, pipeline_output from quantopian.pipeline import Pipeline from quantopian.pipeline.data import morningstar as mstar from quantopian.pipeline.data.builtin import USEquityPricing from quantopian.pipeline.factors import AverageDollarVolume, SimpleMovingAverage from quantopian.pipeline.filters.morningstar import IsPrimaryShare def initialize(context): # Equity numbers for the mean reversion algorithm. context.num_securities = 20 context.num_short = context.num_securities // 2 context.num_long = context.num_securities - context.num_short schedule_function(my_rebalance, date_rules.week_start(), time_rules.market_open(hours=1)) schedule_function(my_record_vars, date_rules.every_day(), time_rules.market_close()) attach_pipeline(my_pipeline(context), 'my_pipeline') def my_pipeline(context): pipe = Pipeline() """ 9 filters: 1. common stock 2 & 3. not limited partnership - name and database check 4. database has fundamental data 5. not over the counter 6. not when issued 7. not depository receipts 8. primary share 9. high dollar volume Check Scott's notebook for more details. """ common_stock = mstar.share_class_reference.security_type.latest.eq('ST00000001') not_lp_name = ~mstar.company_reference.standard_name.latest.matches('.* L[\\. ]?P\.?$') not_lp_balance_sheet = mstar.balance_sheet.limited_partnership.latest.isnull() have_data = mstar.valuation.market_cap.latest.notnull() not_otc = ~mstar.share_class_reference.exchange_id.latest.startswith('OTC') not_wi = ~mstar.share_class_reference.symbol.latest.endswith('.WI') not_depository = ~mstar.share_class_reference.is_depositary_receipt.latest primary_share = IsPrimaryShare() # Combine the above filters. tradable_filter = (common_stock & not_lp_name & not_lp_balance_sheet & have_data & not_otc & not_wi & not_depository & primary_share) high_volume_tradable = (AverageDollarVolume(window_length=21, mask=tradable_filter).percentile_between(70, 100)) # The example algorithm - mean reversion. Note the tradable filter used as a mask. sma_10 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=10, mask=high_volume_tradable) sma_30 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=30, mask=high_volume_tradable) rel_diff = (sma_10 - sma_30) / sma_30 top_rel_diff = rel_diff.top(context.num_short) pipe.add(top_rel_diff, 'top_rel_diff') bottom_rel_diff = rel_diff.bottom(context.num_long) pipe.add(bottom_rel_diff, 'bottom_rel_diff') return pipe # Get the pipeline output and specify which equities to trade. def before_trading_start(context, data): context.output = pipeline_output('my_pipeline') context.short_set = set(context.output[context.output['top_rel_diff']].index) context.long_set = set(context.output[context.output['bottom_rel_diff']].index) context.security_set = context.long_set.union(context.short_set) # Rebalance weekly. def my_rebalance(context,data): for stock in context.security_set: if data.can_trade(stock): if stock in context.long_set: order_target_percent(stock, 1. / context.num_securities) elif stock in context.short_set: order_target_percent(stock, -1. / context.num_securities) for stock in context.portfolio.positions: if stock not in context.security_set and data.can_trade(stock): order_target_percent(stock, 0) # Record variables. def my_record_vars(context, data): shorts = longs = 0 for position in context.portfolio.positions.itervalues(): if position.amount < 0: shorts += 1 elif position.amount > 0: longs += 1 record(leverage=context.account.leverage, short_count=shorts, long_count=longs)