Back to Community
sorting entire universe according to 12 month momentum

i was wondering whether it is somehow possible to rank all stocks in the entire quantopian universe according to (some kind of) momentum? my understanding is that to do so, I would use the history function, which would however limit me to the 200 securities (or a couple more, depending whether the approach presented by MT in this threads: https://www.quantopian.com/posts/using-both-fundamental-and-technical-analysis is not only a temporary bug)?

could someone confirm that there is no other workaround for this issue than exploting the bug mentioned above (which is probably only a temporary solution anyway)?

thanks

8 responses

I'd love to see more universe-implementations too. Not just DollarVolumeUniverse.

One short term tact you could take is using Market Cap and Shares Outstanding from get_fundamentals() as a daily value in before_trading_start() for the full universe of available stocks from get_fundamentals()

So if "some kind" of momentum is a day-over-day momentum, then maybe you could take that tact. The 500 stock universe restrictions do not apply in before_trading_start() because you're not accessing pricing data.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Hello Ueli,

In the Quantopian research platform, it is possible to loop over all the the securities in the entire Quantopian database. Here's a rough example I posted awhile back:

https://www.quantopian.com/posts/analysis-of-entire-quantopian-database-using-quantopian-research-platform

So, perhaps you would write a custom function, get_momentum(), and just feed it every security in a loop. As you go, stick the results in your favorite sortable data structure, and you should be all set. There are some limitations in getting data out of the research platform, which are built in to make it difficult to steal stock data sets from Quantopian, but you can probably copy-and-paste a limited amount of data into the backtester to try out an idea.

Grant

This isn't currently as easy as it could be. We are aware of the issue and working on a solution to help you do exactly this. Here is a recording of a webinar where we discussed the new approach we are planning. If you have feedback, we'd love to hear it.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Ueli,

I'd add to Karen's note above, that in the meantime you can also take advantage of the 2x+ raise to the universe cap that we rolled out recently.

Here's a sample 12 month momentum algo that sorts a 5% universe based on the trailing 12 month price change, buys the top 10% (~40 stocks) and goes short the bottom 10%, rebalancing on the first trading day of each month.

Hope this helps you out while we keep working on making the full universe screen a reality!
Best, Jess

Clone Algorithm
90
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
'''
    This algorithm defines a market neutral portfolio based on momentum of returns and 
    rebalances it at a user-specified frequency. As defined initially we are computing the 12 month 
    price change across all stocks in our tradable universe. Then we are creating an equal weight long 
    portfolio of the 10% of stocks with BEST returns over the past year, and an equal weight short portfolio
    of the 10% of stocks with the WORST returns over the past year. 
    
    NOTE: This algo is intended to run in minute-mode simulation and is compatible with LIVE TRADING.

'''
# Import the libraries we will use here
import numpy as np

def initialize(context):
    # The initialize function is the place to set your tradable universe and any variables 
    # or parameter values that you'll use in your algorithm. 

    # Select the top 5% (from the 95th percentile to the 100th)  of stocks by average daily trading volume 
    # as our tradeable universe.
    set_universe(universe.DollarVolumeUniverse(95, 100))
    
    # Set execution cost assumptions. For live trading with Interactive Brokers we will assume a $1.00 minimum
    # per trade fee, with a per share cost of $0.0075. 
    set_commission(commission.PerShare(cost=0.0075, min_trade_cost=1.00))
    
    # Set market impact assumptions. We will limit our simulation to allow us to be up to 2.5% of 
    # the traded volume for any one minute, and that our price impact constant is 0.1. 
    set_slippage(slippage.VolumeShareSlippage(volume_limit=0.025, price_impact=0.10))
    
    # Define any algo variables
    context.long_leverage = 0.5
    context.short_leverage = -0.5
    context.lower_percentile = 10
    context.upper_percentile = 90
    context.returns_lookback = 252
           
    # In this example we will use schedule_function() to set a monthly rebalance to start 1 hour and 30 minutes 
    # after market open (11am ET) on the first trading day of each week.
    schedule_function(rebalance, 
                      date_rules.month_start(days_offset=0),
                      time_rules.market_open(hours = 1, minutes = 30))  
    
    # Each day at market close we will check our leverage, market exposure, and portfolio size and add them to
    # our custom time series plot using the record() function.
    schedule_function(record_daily_values,
                      date_rules.every_day(),
                      time_rules.market_close(hours=0,minutes=1))        
    
def handle_data(context,data):
    pass
    
def record_daily_values(context,data):
    
    # The record() function allows you to create a custom time-series plot of any variable in your simulation.
    # In this example we will record and plot the market exposure and leverage of our portfolio over time. 
    record(leverage = context.account.leverage)
    record(market_exposure=context.account.net_leverage)

    # We also want to monitor the number of long and short positions in our portfolio over time. This
    # loop will check our positition sizes and add the count of longs and shorts to our custom time-series plot.
    longs = shorts = 0
    for position in context.portfolio.positions.itervalues():
        if position.amount > 0:
            longs += 1
        if position.amount < 0:
            shorts += 1
            
    record(long_count=longs, short_count=shorts)        
            
def rebalance(context,data):
    # The rebalance() function defined here will be called according to our schedule_function settings. 
    # When it is time to rebalance, we first get the last N days of prices for every stock in our universe.
    prices = history(context.returns_lookback, '1d', 'price')
    
    # Calculate the trailing N day return for all instruments and remove any NaNs if there are missing prices
    returns = (prices.iloc[-1] - prices.iloc[0]) / prices.iloc[0]
    returns = returns.dropna()
    
    # Exclude any securities that have current open orders, since we rebalance weekly
    # it's safe to assume an open order means that security is no longer tradable.
    open_orders = get_open_orders()
    if open_orders:
        eligible_secs = [sec for sec in data if sec not in open_orders]
        returns = returns[eligible_secs]
    
    # Get the cutoff return percentiles for the long and short portfolios
    lower, upper = np.percentile(returns, [context.lower_percentile,
                                           context.upper_percentile])
    
    # Select the X% best performing securities to go long
    long_secs = returns[returns >= upper] 
    
    # Select the Y% worst performing securities to short
    short_secs = returns[returns <= lower]
    
    # Set the allocations to even weights in each portfolio
    long_weight = context.long_leverage / len(long_secs)
    short_weight = context.short_leverage / len(short_secs)
    
    for security in data:
        
        # Buy/rebalance securities in the long leg of our portfolio
        if security in long_secs:
            order_target_percent(security, long_weight)
            log.info("[Long] %s @ %s" % (security.symbol, data[security].price))
            
        # Sell/rebalance securities in the short leg of our portfolio
        elif security in short_secs:
            order_target_percent(security, short_weight)
            log.info("[Short] %s @ %s" % (security.symbol, data[security].price))
            
        # Close any positions that fell out of the list of securities to long or short.
        # Note that orders for 0 shares are garbage collected for you.
        else:
            order_target(security, 0)
            log.info("[Closed] %s @ %s"%(security.symbol, data[security].price))
 
There was a runtime error.
Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

I really appreciate the hints and suggestions. Unfortunately, also being limited to a 2x universe is still too much of a limitation.

Regarding the mentioned webinar: I really like the idea of a making a high level API available. Unfortunately it looks like this high level API is more of a workaround trying to allow as much data analysis as possible without exposing too much raw data (which I guess has been prohibited by Morningstar). Being restricted to this high level API in order to use an extended dataset however takes away a lot of the flexibility of the initial pure event based architecture. Especially the idea of using factors and filters reminds me of the Factor and Screen abstractions used by Bloomberg (FTST and EQS) which I consider more or less useless due to their lack of flexibility. As with every abstraction, it is never good enough to cover every usecase. Not having a powerful low level API to access the same data in these cases is very discouraging because there is always the risk of hitting a wall after having implemented 95% of the algorithm.

Just my two cents

Hi Ueli,
We continue to work on increasing the universe size. Our next goal is 1000 securities.

Did you watch the webinar to the part where we talked about customizing your factors, filters and classifiers? That will give you access to all of the data to with it whatever you like. The slides to the webinar are here (or will be shortly, I'm on a plane and they are uploading...) if that is an easier way for you to view the custom examples.

KR

Ueli,
The pipeline API was launched today and should help with these types of strategies. You can read all about it here.

I've also attached an example that calculates two factors (one momentum based, the other liquidity) and then ranks the universe by those factors. It longs 5% and shorts 5% of the resulting universe.

I believe we make all of the raw data available to you with this API. You have the ability to create custom factors using any piece of data from our pricing data or the Morningstar fundamentals data (and soon the Quantopian Store.) However, if there is something you would like to do and cannot, I would love to know about it. Our goal is flexibility and performance.

Clone Algorithm
944
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
"""
This example comes from a request in the forums. 
The post can be found here: https://www.quantopian.com/posts/ranking-system-based-on-trading-volume-slash-shares-outstanding

The request was: 

I am stuck trying to build a stock ranking system with two signals:
1. Trading Volume/Shares Outstanding.
2. Price of current day / Price of 60 days ago.
Then rank Russell 2000 stocks every month, long the top 5%, short the bottom 5%.

"""

from quantopian.algorithm import attach_pipeline, pipeline_output
from quantopian.pipeline import Pipeline
from quantopian.pipeline import CustomFactor
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.pipeline.data import morningstar


# Create custom factor #1 Trading Volume/Shares Outstanding
class Liquidity(CustomFactor):   
    
    # Pre-declare inputs and window_length
    inputs = [USEquityPricing.volume, morningstar.valuation.shares_outstanding] 
    window_length = 1
    
    # Compute factor1 value
    def compute(self, today, assets, out, volume, shares):       
        out[:] = volume[-1]/shares[-1]

# Create custom factor #2 Price of current day / Price of 60 days ago.        
class Momentum(CustomFactor):   
    
    # Pre-declare inputs and window_length
    inputs = [USEquityPricing.close] 
    window_length = 60
    
    # Compute factor2 value
    def compute(self, today, assets, out, close):       
        out[:] = close[-1]/close[0]
        
# Create custom factor to calculate a market cap based on yesterday's close
# We'll use this to get the top 2000 stocks by market cap
class MarketCap(CustomFactor):   
    
    # Pre-declare inputs and window_length
    inputs = [USEquityPricing.close, morningstar.valuation.shares_outstanding] 
    window_length = 1
    
    # Compute market cap value
    def compute(self, today, assets, out, close, shares):       
        out[:] = close[-1] * shares[-1]
        

def initialize(context):
    pipe = Pipeline()
    attach_pipeline(pipe, 'ranked_2000')
       
    # Add the two factors defined to the pipeline
    liquidity = Liquidity()
    pipe.add(liquidity, 'liquidity')
    
    momentum = Momentum()
    pipe.add(momentum, 'momentum')
    
    # Create and apply a filter representing the top 2000 equities by MarketCap every day
    # This is an approximation of the Russell 2000
    mkt_cap = MarketCap()
    top_2000 = mkt_cap.top(2000)
    
    # Rank factor 1 and add the rank to our pipeline
    liquidity_rank = liquidity.rank(mask=top_2000)
    pipe.add(liquidity_rank, 'liq_rank')
    
    # Rank factor 2 and add the rank to our pipeline
    momentum_rank = momentum.rank(mask=top_2000)
    pipe.add(momentum_rank, 'mom_rank')
    
    # Take the average of the two factor rankings, add this to the pipeline
    combo_raw = (liquidity_rank+momentum_rank)/2
    pipe.add(combo_raw, 'combo_raw') 
    
    # Rank the combo_raw and add that to the pipeline
    pipe.add(combo_raw.rank(mask=top_2000), 'combo_rank')
    
    # Set a screen to ensure that only the top 2000 companies by market cap 
    # with a momentum factor greater than 0 are returned
    pipe.set_screen(top_2000 & (momentum>0))
            
    # Scedule my rebalance function
    schedule_function(func=rebalance, 
                      date_rule=date_rules.month_start(days_offset=0), 
                      time_rule=time_rules.market_open(hours=0,minutes=30), 
                      half_days=True)
    
    # set my leverage
    context.long_leverage = 0.50
    context.short_leverage = -0.50
    
            
def before_trading_start(context, data):
    # Call pipelive_output to get the output
    context.output = pipeline_output('ranked_2000')
      
    # Narrow down the securities to only the top 200 & update my universe
    context.long_list = context.output.sort(['combo_rank'], ascending=False).iloc[:100]
    context.short_list = context.output.sort(['combo_rank'], ascending=False).iloc[-100:]   
    
    update_universe(context.long_list.index.union(context.short_list.index)) 


def handle_data(context, data):  
    
     # Record and plot the leverage of our portfolio over time. 
    record(leverage = context.account.leverage)
    
    print "Long List"
    log.info("\n" + str(context.long_list.sort(['combo_rank'], ascending=True).head(10)))
    
    print "Short List" 
    log.info("\n" + str(context.short_list.sort(['combo_rank'], ascending=True).head(10)))

# This rebalancing is called according to our schedule_function settings.     
def rebalance(context,data):
    
    long_weight = context.long_leverage / float(len(context.long_list))
    short_weight = context.short_leverage / float(len(context.short_list))

    
    for long_stock in context.long_list.index:
        if long_stock in data:
            log.info("ordering longs")
            log.info("weight is %s" % (long_weight))
            order_target_percent(long_stock, long_weight)
        
    for short_stock in context.short_list.index:
        if short_stock in data:
            log.info("ordering shorts")
            log.info("weight is %s" % (short_weight))
            order_target_percent(short_stock, short_weight)
        
    for stock in context.portfolio.positions.iterkeys():
        if stock not in context.long_list.index and stock not in context.short_list.index:
            order_target(stock, 0)
We have migrated this algorithm to work with a new version of the Quantopian API. The code is different than the original version, but the investment rationale of the algorithm has not changed. We've put everything you need to know here on one page.
There was a runtime error.