Back to Community
An implementation of the Robust Asset Allocation Strategy from Alpha Architects

The attached backtest illustrates an implementation of the RAA strategy from AlphaArchitects (described in this post). The strategy requires an implementation of both a Value and Momentum strategy. To ease the implementation I created a framework for incorporating multiple strategies ( I adapted some of the ideas from these posts, modular framework for trading algorithms, and abstracting an algorithm from one stock to many)

I thought it may be useful to the community so I’ve posted it here.

A couple of notes on the implementation,

  • all strategies are required to implement a robust asset allocation function
  • as a simplification the framework assumes all existing assets in a strategy are sold before new assets are ranked and then bought
  • the implementation ensures that strategies that do fundamental queries are executed on distinct days so that a maximum number of securities relevant to the particular strategy are examined (i.e. the 500 securities in 'data' are selected by the specific strategy and there is no overlap with other strategies)
  • Since stocks in the Q universe are generally restricted to those that trade on a US exchange, no distinct allocation is made to domestic and international equities, rather fundamental queries are allowed to invest in ADRs, and in securities not necessarily domiciled in the US
  • a Gold etf (GSG) is used to approximate an allocation to commodities. This is done to allow for a longer backtest.
  • the Value strategy implemented is the Magic formula
  • the momentum strategy is Time Series Momentum with a 12 month lookback
  • the strategy simply goes to cash when out of the market instead of investing in treasuries. An improvement to the framework would be to add a Portfolio Manager class to manage all trades from all strategies and put any cash in an appropriate alternative

Regards,
Mark

Clone Algorithm
467
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
##########################################################################################
# This is an attempt to implement the Robust Asset Allocation (RAA) strategy described here,
#    http://blog.alphaarchitect.com/2014/12/02/the-robust-asset-allocation-raa-solution/
# and described in detail in 'DIY Financial Advisor' by W Gray, J Vogel, D Foulke
#
# Notes on implementation
#    - all strategies are required to implement a robust asset allocation function
#    - as a simplification the framework assumes all existing assets in a strategy are sold before new assets are ranked and then bought
#    - the implementation ensures that strategies that do fundamental queries are executed on distinct days so that a maximum number of securities relevant to the particular strategy are examined (i.e. the 500 securities in 'data' are selected by the specific strategy and there is no overlap with other strategies)
#    -Since stocks in the Q universe are generally restricted to those that trade on a US exchange, no distinct allocation is made to domestic and international equities, rather fundamental queries are allowed to invest in ADRs, and in securities not necessarily domiciled in the US
#     -a Gold etf (GSG) is used to approximate an allocation to commodities. This is done to allow for a longer backtest.
#    - the Value strategy implemented is the Magic formula
#    - the momentum strategy is Time Series Momentum with a 12 month lookback
#    - the strategy simply goes to cash when out of the market instead of investing in treasuries. An improvement to the framework would be to add a Portfolio Manager class to manage all trades from all strategies and put any cash in an alternative
###########################################################################################

from datetime import timedelta
from sqlalchemy import or_
from scipy import stats
import statsmodels.api as sm
import talib 
import pandas as pd
import numpy as np
import time


#Base Class for all strategies
class Strategy(object):
    
    def __init__(self, pct_alloc, risk_free_asset, ma_threshold,sell_offset, buy_offset):
        self.alloc = pct_alloc                 #percent of portfolio to allocate to strategy
        self.risk_free_asset = risk_free_asset #risk free asset
        self.ma_threshold = ma_threshold       #moving average theshold
        self.sell_offset = sell_offset         #offset for scheduling when strategy will sell
        self.buy_offset = buy_offset           #offset for scheduling when strategy will buy
        self.sell_day = False                  
        self.buy_day = False    
        self.bought = []                       #maintain list of securites that were bought by strategy
        self.lookback = 252                    #set default lookback period to 1 year (252 days)
 

    def getAlloc(self):
        return self.alloc

    
    def setAlloc(self, alloc):
        self.alloc = alloc
    
    
    #Implement robust allocation algo for strategy
    def robust(self,context,data):
        raise NotImplementedError("implement robust method in Subclass")
    
    
    #Show relevant params at beginning of execution
    def show_params(self):
        raise NotImplementedError("implement show_params method in Subclass")
    
    
    #Implement fundamental queries here
    def handle_data(self,context,data):
        raise NotImplementedError("implement handle_data method in Subclass")
    
    
    #Implement buy, sell and ranking functions here
    def before_trading_start(self,context,data): 
        raise NotImplementedError("implement before_trading_start method in Subclass")
    
    
#Class for ETF strategy   - currently limited to holding 1 ETF
class ETF_Strat(Strategy):
    
    def __init__(self, pct_alloc, risk_free_asset, ma_threshold, etf, sell_offset, buy_offset):
        Strategy.__init__(self,pct_alloc,risk_free_asset,ma_threshold,sell_offset, buy_offset)
        
        self.etf = etf    #set ETF for strategy

        
    #calculate excess return over risk free asset    
    def get_excess_return(self,context,data):
        
        h = history( self.lookback,'1d','price')[[self.etf,self.risk_free_asset]]
        h = h.resample('M',how='last')
        
        pct_change =h.iloc[[0,-1]].pct_change()[1:]
        
        pct_change = pct_change.dropna(axis=1)
        
        ex_return = pct_change[self.etf]-pct_change[self.risk_free_asset]
        return ex_return[0]

    
    #implement RAA strategy 
    def robust(self,context,data):
        r = 0.0
    
        #if etf time series MOM returns are greater than risk free asset then invest 1/2 weight if etf price > moving avg then invest 1/2 weight. If both then invest full weight
        ex_return = self.get_excess_return(context,data)
        if ex_return > 0.0:
            r += 0.5
        if data[self.etf].price > data[self.etf].mavg(self.ma_threshold):
            r += 0.5

        return r
    
    
    #buy etf based on RAA allocation    
    def exec_buy(self,context,data):
        open_orders = get_open_orders()
        r = self.robust(context,data)
        w = r * self.alloc #RAA adjusted weight
        
        if self.etf in data and self.etf not in open_orders and w > 0.0:
            order_target_percent(self.etf,w)
            self.bought.append(self.etf)
       
        log.info(str(type(self).__name__) + " bought " + str(self.etf.symbol) + " w=" + str(w))

        
    #sell etf
    def exec_sell(self,context,data):
        open_orders = get_open_orders()
        
        #if self.etf in context.portfolio.positions and self.etf in data and self.etf in self.bought and self.etf not in open_orders:
        if self.etf in context.portfolio.positions and self.etf in self.bought and self.etf not in open_orders:
            order_target_percent(self.etf,0)
            self.bought.remove(self.etf)
        
    
    def set_sell_day(self,context,data):
        self.sell_day = True
        
    
    def set_buy_day(self,context,data):
        self.buy_day = True
        
        
    def handle_data(self,context,data):
        if self.sell_day:
            self.exec_sell(context,data)
            self.sell_day = False
        
        if self.buy_day:
            self.exec_buy(context,data)
            self.buy_day = False
        

    def before_trading_start(self,context,data): 
        pass
    
    
    def show_params(self,c):
        log.info("************************")
        log.info(" ETF params ")
        log.info("************************")
        log.info(" etf name: " + str(self.etf.symbol))
        log.info(" risk free asset: " + str(self.risk_free_asset.symbol))
        log.info(" market ma threshold: " + str(self.ma_threshold))
        log.info(" lookback: " + str(self.lookback))
        log.info(" pct alloc: " + str(self.alloc))
        log.info(" sell_offset: " + str(self.sell_offset))
        log.info(" buy_offset: " + str(self.buy_offset))

        
#Base class for Equity Strategies
class Equity_Strat(Strategy):
    
    def __init__(self, pct_alloc,risk_free_asset,ma_threshold, index, sell_offset, buy_offset):
        Strategy.__init__(self,pct_alloc,risk_free_asset,ma_threshold,sell_offset, buy_offset)
        
        self.index = index #set 

        
    def robust(self,context,data):
        r = 0.0     
        
        ex_return = self.get_excess_return(context,data)
        if ex_return > 0.0:
            r += 0.5
        if data[self.index].price > data[self.index].mavg(self.ma_threshold):
            r += 0.5
        
        return r
    
    
    def get_excess_return(self,context,data):
        h = history( self.lookback,'1d','price')[[self.index,self.risk_free_asset]]
        h = h.resample('M',how='last')

        pct_change =h.iloc[[0,-1]].pct_change()[1:]
        pct_change = pct_change.dropna(axis=1)
        
        ex_return = pct_change[self.index]-pct_change[self.risk_free_asset]
        
        return ex_return[0]


#Momentum Strategy
# -- Sort equities on best 1 year performance
class Mom_Strat(Equity_Strat):
   
    def __init__(self, pct_alloc, risk_free_asset, ma_threshold, index, sell_offset, buy_offset):
        Equity_Strat.__init__(self,pct_alloc,risk_free_asset,ma_threshold,index,sell_offset, buy_offset)
        
        self.lower_market_cap = 2000e6     #lower market cap for candidate selection
        self.limit = 500                   #max number of stocks to select for fundamental query
        self.num_positions = 30            #max number of positions the strategy will select
        
        
    def set_sell_day(self,context,data):
        self.sell_day = True
        
    
    def set_buy_day(self,context,data):
        self.buy_day = True
        
   
    def before_trading_start(self,context,data): 
        
        if self.buy_day:
            self.get_candidates()
            update_universe(self.candidates)
            
        
    def handle_data(self,context,data):
        if self.sell_day:
            self.exec_sell(context,data)
            self.sell_day = False
        
        if self.buy_day:
            self.exec_rank(context, data)
            self.exec_buy(context,data)
            self.buy_day = False

        
    def get_candidates(self):
        df_fundamentals = get_fundamentals(
            query(
                fundamentals.valuation.shares_outstanding,
                fundamentals.valuation.market_cap
            )
            #.filter(fundamentals.company_reference.country_id == "USA")
            .filter(or_(fundamentals.company_reference.primary_exchange_id == "NAS", fundamentals.company_reference.primary_exchange_id == "NYSE"))
            .filter(fundamentals.valuation.market_cap != None)
            .filter(fundamentals.valuation.market_cap > self.lower_market_cap ) 
            .filter(fundamentals.valuation.shares_outstanding != None)
            #.filter(fundamentals.share_class_reference.is_depositary_receipt == False)
            .filter(fundamentals.share_class_reference.is_primary_share == True)
            .filter(fundamentals.company_reference.primary_exchange_id != "OTCPK")
            .order_by(fundamentals.valuation.market_cap.desc())
            .limit(self.limit)
        )
    
        self.candidates=df_fundamentals.columns
   
    
    def exec_rank(self,context,data):
        
        #get TMOM
        h = history( self.lookback,'1d','price')[self.candidates]
        h = h.resample('M',how='last') #get monthly returns
        
        pct_change =h.iloc[[0,-1]].pct_change()[1:]
        pct_change = pct_change.dropna(axis=1)
        
        #convert dataframe to series for sorting. Then sort in descending order
        self.longs =  pct_change.squeeze().order(ascending=False)
        self.longs = self.longs[self.longs > 0.0]
        self.longs = self.longs[:self.num_positions]
    
 
    def exec_buy(self,context,data):
        r= self.robust(context,data)
        open_orders = get_open_orders()
            
        l = len(self.longs)
        w = (r*self.alloc)/l if l > 0 else 0 #robust adjusted weight
        
        for s in self.longs.index:
            if s in data and s not in open_orders:
                order_target_percent(s,w)
                if w > 0.0:
                    self.bought.append(s)
        
        log.info(str(type(self).__name__) + " longs list len " + str(len(self.longs)))
        log.info(str(type(self).__name__) + " bought list len: " + str(len(self.bought)))    
    
    def exec_sell(self,context,data):

        open_orders = get_open_orders()
        
        for s in context.portfolio.positions :
            if s in data and s not in open_orders :
                if s in self.bought:
                    order_target_percent(s,0)
                    self.bought.remove(s)
                
        log.info(str(type(self).__name__) + " bought list len: " + str(len(self.bought)))
        
    
    
    def show_params(self,c):
        log.info("************************")
        log.info(" Momentum params ")
        log.info("************************")
        log.info(" lower market cap: " + str(self.lower_market_cap))
        log.info(" stock search limit: " + str(self.limit))
        log.info(" risk free asset: " + str(self.risk_free_asset.symbol))
        log.info(" number of positions: " + str(self.num_positions))
        log.info(" moving average threshold: " + str(self.ma_threshold))
        log.info(" pct alloc: " + str(self.alloc))
        log.info(" sell_offset: " + str(self.sell_offset))
        log.info(" buy_offset: " + str(self.buy_offset))
        log.info(" lookback: " + str(self.lookback))
       
        

class Value_Strat(Equity_Strat):
    
    def __init__(self, pct_alloc, risk_free_asset, ma_threshold, index, sell_offset, buy_offset):
        
        Equity_Strat.__init__(self,pct_alloc,risk_free_asset,ma_threshold,index,sell_offset, buy_offset)
    
        self.lower_market_cap = 2000e6
        self.limit = 500
        self.num_positions = 30
        self.rebalance_month = 6 #June
        self.longs = []
        #self.alloc = pct_alloc
        #self.w = self.pct_alloc/self.num_positions

    
    def set_sell_day(self,context,data):
         today = get_datetime()
         if today.month == self.rebalance_month:
             self.sell_day = True
        
    
    def set_buy_day(self,context,data):
        today = get_datetime()
        if today.month == self.rebalance_month:
            self.buy_day = True
        
   
    def before_trading_start(self,context,data): 
        
        if self.sell_day:
            update_universe(self.bought)

        if self.buy_day:
            self.get_candidates()
            update_universe(self.candidates)
    
    
    def handle_data(self,context,data):
        
        if self.sell_day:
            self.exec_sell(context,data)
            self.sell_day = False
        
        if self.buy_day:
            self.exec_rank(context,data)
            self.exec_buy(context,data)
            self.buy_day = False
 
    
    def get_candidates(self):
        excluded_sectors = [103, 207]
        sector_code = fundamentals.asset_classification.morningstar_sector_code
        fundamental_df = get_fundamentals(
            query(
                sector_code,
                fundamentals.valuation.market_cap,
                fundamentals.valuation.enterprise_value,
                fundamentals.cash_flow_statement.capital_expenditure,
                fundamentals.operation_ratios.roic,
                fundamentals.income_statement.ebit,
                fundamentals.income_statement.ebitda,
                fundamentals.balance_sheet.total_assets,            
        )
        #.filter(fundamentals.company_reference.country_id == "USA")
        .filter(or_(fundamentals.company_reference.primary_exchange_id == "NAS", fundamentals.company_reference.primary_exchange_id == "NYSE"))
        .filter(fundamentals.share_class_reference.is_primary_share == True)
        .filter(fundamentals.company_reference.primary_exchange_id != "OTCPK")
        #.filter(fundamentals.share_class_reference.is_depositary_receipt == False)
        .filter(~sector_code.in_(excluded_sectors))
        .filter(fundamentals.valuation.market_cap > self.lower_market_cap ) 
        .filter(fundamentals.valuation_ratios.ev_to_ebitda > 0)
        .order_by(fundamentals.valuation.market_cap.desc())
        .limit(self.limit)
        )
        
        self.candidates = fundamental_df
        #self.longs = self.mf_rank(fundamental_df)
        #self.longs = self.longs[:self.num_positions]

        
    def exec_rank(self,context,data):
    
        #get MF components
        earnings_yield = self.candidates.ix['ebit'] / self.candidates.ix['enterprise_value']
        roic = self.candidates.ix['roic'].copy()
    
        #sort MF components
        earnings_yield.sort(ascending=False)
        roic.sort(ascending=False)

        #get ranks for components
        earnings_yield_ranking = pd.Series(range(len(earnings_yield)), index=earnings_yield.index)
        roic_ranking = pd.Series(range(len(roic)), index=roic.index) 
    
        #get combined ranking - ranks = earnings_yield_ranking + roic_ranking
        ranks = {}
        for s in self.candidates.columns:
            v_rank = earnings_yield_ranking.loc[s.sid]
            q_rank = roic_ranking.loc[s.sid]
            ranks[s] = v_rank+q_rank
    
        #convert to df and sort
        mf_ranks = pd.Series(ranks,name='Symbol')
        mf_ranks.sort(ascending = True)
        
        self.longs = mf_ranks.index.tolist()
        self.longs = self.longs[:self.num_positions]
        
        #return mf_ranks.index.tolist()

     
    def exec_buy(self,context,data):
        
        r= self.robust(context,data)
        open_orders = get_open_orders()
        
        l = len(self.longs)
        w = (r*self.alloc)/self.num_positions if l > 0 else 0
            
        for s in self.longs:
            if s in data and s not in open_orders:
                order_target_percent(s,w)
                if w > 0.0:
                    self.bought.append(s)
            
            
        log.info(str(type(self).__name__) + " longs list len " + str(len(self.longs)))
        log.info(str(type(self).__name__) + " bought " + str(len(self.bought)) + " stocks")

        
    def exec_sell(self,context,data):
        open_orders = get_open_orders()
        
        for s in context.portfolio.positions:
            if s in data and s in self.bought and s not in open_orders:
                order_target_percent(s,0)
                self.bought.remove(s)
            
        log.info(str(type(self).__name__) + " bought list len: " + str(len(self.bought)))

        
    def show_params(self,c):
        log.info("************************")
        log.info(" Value params ")
        log.info("************************")
        log.info(" lower market cap: " + str(self.lower_market_cap))
        log.info(" stock search limit: " + str(self.limit))
        log.info(" risk free asset: " + str(self.risk_free_asset.symbol))
        log.info(" number of positions: " + str(self.num_positions))
        log.info(" moving average threshold: " + str(self.ma_threshold))
        log.info(" pct alloc: " + str(self.alloc))
        log.info(" sell_offset: " + str(self.sell_offset))
        log.info(" buy_offset: " + str(self.buy_offset))        
        log.info(" rebalance month: " + str(self.rebalance_month))

        

def initialize(context):
    set_commission(commission.PerTrade(cost=0.00))  
    set_slippage(slippage.FixedSlippage(spread=0.00))
    #Common vars
    c = context
    c.lastYear = 2015
    c.lastMonth = 12
    c.index = symbol('SPY')
    c.risk_free_asset = symbol('SHY')
    c.ma_threshold = 200
    
    #register Strategies
    #IVY_4 allocations - no allocation to world equities
    # replace GSG with Gold to get longer backtest
    '''
    c.strategies = [ETF_Strat(0.25,c.risk_free_asset,c.ma_threshold,symbol('SPY'),1,2),
                    ETF_Strat(0.25,c.risk_free_asset,c.ma_threshold,symbol('IYR'),1,2),
                    ETF_Strat(0.25,c.risk_free_asset,c.ma_threshold,symbol('GLD'),1,2),
                    ETF_Strat(0.25,c.risk_free_asset,c.ma_threshold,symbol('IEF'),1,2)]
    '''
   
    #RAA Balanced - 40% allocation to equities
    '''
    c.strategies = [Value_Strat(0.2,c.risk_free_asset, c.ma_threshold, c.index,11,13),
                    Mom_Strat(0.2, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ,
                    ETF_Strat(0.2,c.risk_free_asset,c.ma_threshold,symbol('IYR'),1,2),
                    ETF_Strat(0.2,c.risk_free_asset,c.ma_threshold,symbol('GLD'),1,2),
                    ETF_Strat(0.2,c.risk_free_asset,c.ma_threshold,symbol('IEF'),1,2)]
    '''    
    
    #RAA Moderate Tilt - 60% allocation to equities
    '''
    c.strategies = [Value_Strat(0.3,c.risk_free_asset, c.ma_threshold, c.index,11,13),
                    Mom_Strat(0.3, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ,
                    ETF_Strat(0.1,c.risk_free_asset,c.ma_threshold,symbol('IYR'),1,2),
                    ETF_Strat(0.1,c.risk_free_asset,c.ma_threshold,symbol('GLD'),1,2),
                    ETF_Strat(0.2,c.risk_free_asset,c.ma_threshold,symbol('IEF'),1,2)]
    '''
        
    
    #RAA Aggressive Tilt - 80% allocation to equities
    
    c.strategies = [Value_Strat(0.4,c.risk_free_asset, c.ma_threshold, c.index,11,13),
                    Mom_Strat(0.4, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ,
                    ETF_Strat(0.05,c.risk_free_asset,c.ma_threshold,symbol('IYR'),1,2),
                    ETF_Strat(0.05,c.risk_free_asset,c.ma_threshold,symbol('GLD'),1,2),
                    ETF_Strat(0.1,c.risk_free_asset,c.ma_threshold,symbol('IEF'),1,2)]
    
    #Mom only
    #c.strategies = [Mom_Strat(1.0, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ]
    #Val only
    #c.strategies = [Value_Strat(1.0,c.risk_free_asset, c.ma_threshold, c.index,11,13) ]
    #Mom/Val 50/50
    #c.strategies = [Value_Strat(0.5,c.risk_free_asset, c.ma_threshold, c.index,11,13),
    #                Mom_Strat(0.5, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ]
    
    #show parameters of all strategies
    show_params(c)
    
    for strat in c.strategies:
        schedule_function(strat.set_sell_day,date_rules.month_start(days_offset=strat.sell_offset),
            time_rules.market_open(minutes=120)) 
        schedule_function(strat.set_buy_day,date_rules.month_start(days_offset=strat.buy_offset),
            time_rules.market_open(minutes=120)) 
    
    #as a last step check final state of portfolio. In particular check total value of unfullfilled orders
    schedule_function(check_positions,date_rules.month_end(days_offset=0),
                      time_rules.market_open(minutes=240))
    

def check_positions(context,data):

    today = get_datetime()
    
    if today.year == context.lastYear and today.month == context.lastMonth:
        #log current account
        log.info(" account leverage: " + str(context.account.leverage))
        log.info(" portfolio value: " + str(context.portfolio.portfolio_value))
        log.info(" portfolio cash: " + str(context.portfolio.cash))
        

        #check number and value of unfullfilled trades
        # unfullfilled trades are often the cause of a strategy appearing to use leverage when in reality it would not.
        # often the cause of an unfullfilled trade is a company merging or getting bought out
        
        open_orders = get_open_orders()
        #update_universe(open_order_sids.keys())
        
        
        c = 0
        tot_val = 0
        for s in context.portfolio.positions:
            if s in open_orders and s.end_date.year < context.lastYear and s.end_date.month < context.lastMonth:
                shrs = context.portfolio.positions[s].amount
                cost = context.portfolio.positions[s].cost_basis
                current = data[s].price
                c += 1
                tot_val +=  (shrs * current)
        
        log.info(" len open orders: " + str(len(open_orders)))
        log.info(" number of unfullfilled trades: " + str(c) + " value: " + str(tot_val))
    

def show_params(c):
    
    log.info("************************")
    log.info(" Algo params ")
    log.info("************************")
    
    for strat in c.strategies:
        strat.show_params(c)
   
    
def before_trading_start(context,data): 
    for strat in context.strategies:
        strat.before_trading_start(context, data)

            
def handle_data(context, data):

    #Call each individual algorithm's handle_data method.
    for strat in context.strategies:
        strat.handle_data(context, data)
    
    record(leverage = context.account.leverage)
    
There was a runtime error.
17 responses

I tried to run this framework in minute mode of backtest. I have a running error as below. Does anyone success to get it done?

AttributeError: 'Mom_Strat' object has no attribute 'candidates'
USER ALGORITHM:260, in exec_rankGo to IDE
h = history( self.lookback,'1d','price')[self.candidates]

Hi Adam,

thanks for pointing this out. The framework assumes that before_trading_start executes before handle_data, that is not the case when running in minute mode. The reason before_trading_start needs to execute before handle_data is to ensure candidates are available before they are ranked. To ensure that is the case in minute mode I added a check to ensure that get_candidates is executed,

def before_trading_start(self,context,data):

    if self.buy_day:  
        self.get_candidates()  
        if len(self.candidates) > 0:  
            **self.candidates_available = True** -code added  
            update_universe(self.candidates)  


def handle_data(self,context,data):  
    if self.sell_day:  
        self.exec_sell(context,data)  
        self.sell_day = False  

    if self.buy_day and **self.candidates_available:** -code added  
        self.exec_rank(context, data)  
        self.exec_buy(context,data)  
        self.buy_day = False  
        **self.candidates_available = False** --code added

This addresses the problem you ran into. However it leads to another problem which is that the algorithm fails because of a MemoryError. I get the following message,

There was a runtime error.
MemoryError
Algorithm used too much memory. Need to optimize your code for better performance. Learn More

This happens the first time the algorithm tries to calculate the moving average for the index,

def robust(self,context,data):
r = 0.0

    ex_return = self.get_excess_return(context,data)  
    if ex_return > 0.0:  
        r += 0.5  
    if data[self.index].price > **data[self.index].mavg(self.ma_threshold)**:  -see here  
        r += 0.5  

    return r

Since it is happening the very first time this code is executed I'm not sure why its running out of memory, when it is not in daily mode. So, I'm stuck on how to address the issue. Perhaps someone from Quantopian can comment?

Alisa said use .mean() instead of .mavg, here and indeed it is far faster.

I implemented the .mean() to def robust(self,context,data) function as suggested.

# if data[self.etf].price > data[self.etf].mavg(self.ma_threshold):  
    if data[self.etf].price > history(self.ma_threshold, '1d', 'price')[self.etf].mean():  

It helps execution speed a lot. Unfortunately, I still can not get rid of memory error problem. Any suggestion please?

MemoryError
Algorithm used too much memory. Need to optimize your code for better performance.

Hi Adam,

I think the code you want to change is in the Equity_Strat class,

if data[self.index].price > data[self.index].mavg(self.ma_threshold):

    if data[self.index].price > history(self.ma_threshold, '1d', 'price')[self.index].mean():

I've made that change and the strat is currently running without hitting the memory error yet.

Hi Mark,

My algorithm seems not work in minutes mode still. Do you mind sharing the minute mode of backtest result to the community?

Thanks,
Adam

This backtest works for 2015, essentially you need to replace any calls to mavg with a call to numpy.mean, as noted above.

For some reason in minute mode certain trades in the value strategy result in a key error, not sure why, since they work in monthly mode. Will have to look into that at some point.

Regards,
Mark

Clone Algorithm
467
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
##########################################################################################
# This is an attempt to implement the Robust Asset Allocation (RAA) strategy described here,
#    http://blog.alphaarchitect.com/2014/12/02/the-robust-asset-allocation-raa-solution/
# and described in detail in 'DIY Financial Advisor' by W Gray, J Vogel, D Foulke
#
# Notes on implementation
#    - all strategies are required to implement a robust asset allocation function
#    - as a simplification the framework assumes all existing assets in a strategy are sold before new assets are ranked and then bought
#    - the implementation ensures that strategies that do fundamental queries are executed on distinct days so that a maximum number of securities relevant to the particular strategy are examined (i.e. the 500 securities in 'data' are selected by the specific strategy and there is no overlap with other strategies)
#    -Since stocks in the Q universe are generally restricted to those that trade on a US exchange, no distinct allocation is made to domestic and international equities, rather fundamental queries are allowed to invest in ADRs, and in securities not necessarily domiciled in the US
#     -a Gold etf (GSG) is used to approximate an allocation to commodities. This is done to allow for a longer backtest.
#    - the Value strategy implemented is the Magic formula
#    - the momentum strategy is Time Series Momentum with a 12 month lookback
#    - the strategy simply goes to cash when out of the market instead of investing in treasuries. An improvement to the framework would be to add a Portfolio Manager class to manage all trades from all strategies and put any cash in an alternative
###########################################################################################

from datetime import timedelta
from sqlalchemy import or_
from scipy import stats
import statsmodels.api as sm
import talib 
import pandas as pd
import numpy as np
import time


#Base Class for all strategies
class Strategy(object):
    
    def __init__(self, pct_alloc, risk_free_asset, ma_threshold,sell_offset, buy_offset):
        self.alloc = pct_alloc                 #percent of portfolio to allocate to strategy
        self.risk_free_asset = risk_free_asset #risk free asset
        self.ma_threshold = ma_threshold       #moving average theshold
        self.sell_offset = sell_offset         #offset for scheduling when strategy will sell
        self.buy_offset = buy_offset           #offset for scheduling when strategy will buy
        self.sell_day = False                  
        self.buy_day = False    
        self.bought = []                       #maintain list of securites that were bought by strategy
        self.lookback = 252                    #set default lookback period to 1 year (252 days)
 

    def getAlloc(self):
        return self.alloc

    
    def setAlloc(self, alloc):
        self.alloc = alloc
    
    
    #Implement robust allocation algo for strategy
    def robust(self,context,data):
        raise NotImplementedError("implement robust method in Subclass")
    
    
    #Show relevant params at beginning of execution
    def show_params(self):
        raise NotImplementedError("implement show_params method in Subclass")
    
    
    #Implement fundamental queries here
    def handle_data(self,context,data):
        raise NotImplementedError("implement handle_data method in Subclass")
    
    
    #Implement buy, sell and ranking functions here
    def before_trading_start(self,context,data): 
        raise NotImplementedError("implement before_trading_start method in Subclass")
    
    
#Class for ETF strategy   - currently limited to holding 1 ETF
class ETF_Strat(Strategy):
    
    def __init__(self, pct_alloc, risk_free_asset, ma_threshold, etf, sell_offset, buy_offset):
        Strategy.__init__(self,pct_alloc,risk_free_asset,ma_threshold,sell_offset, buy_offset)
        
        self.etf = etf    #set ETF for strategy

        
    #calculate excess return over risk free asset    
    def get_excess_return(self,context,data):
        
        h = history( self.lookback,'1d','price')[[self.etf,self.risk_free_asset]]
        h = h.resample('M',how='last')
        
        pct_change =h.iloc[[0,-1]].pct_change()[1:]
        
        pct_change = pct_change.dropna(axis=1)
        
        ex_return = pct_change[self.etf]-pct_change[self.risk_free_asset]
        return ex_return[0]

    
    #implement RAA strategy 
    def robust(self,context,data):
        r = 0.0
    
        #if etf time series MOM returns are greater than risk free asset then invest 1/2 weight if etf price > moving avg then invest 1/2 weight. If both then invest full weight
        ex_return = self.get_excess_return(context,data)
        if ex_return > 0.0:
            r += 0.5
        #if data[self.etf].price > data[self.etf].mavg(self.ma_threshold):
        if data[self.etf].price > history(self.ma_threshold, '1d', 'price')[self.etf].mean(): 
            r += 0.5

        return r
    
    
    #buy etf based on RAA allocation    
    def exec_buy(self,context,data):
        open_orders = get_open_orders()
        r = self.robust(context,data)
        w = r * self.alloc #RAA adjusted weight
        
        if self.etf in data and self.etf not in open_orders and w > 0.0:
            order_target_percent(self.etf,w)
            self.bought.append(self.etf)
       
        log.info(str(type(self).__name__) + " bought " + str(self.etf.symbol) + " w=" + str(w))

        
    #sell etf
    def exec_sell(self,context,data):
        open_orders = get_open_orders()
        
        #if self.etf in context.portfolio.positions and self.etf in data and self.etf in self.bought and self.etf not in open_orders:
        if self.etf in context.portfolio.positions and self.etf in self.bought and self.etf not in open_orders:
            order_target_percent(self.etf,0)
            self.bought.remove(self.etf)
        
    
    def set_sell_day(self,context,data):
        self.sell_day = True
        
    
    def set_buy_day(self,context,data):
        self.buy_day = True
        
        
    def handle_data(self,context,data):
        if self.sell_day:
            self.exec_sell(context,data)
            self.sell_day = False
        
        if self.buy_day:
            self.exec_buy(context,data)
            self.buy_day = False
        

    def before_trading_start(self,context,data): 
        pass
    
    
    def show_params(self,c):
        log.info("************************")
        log.info(" ETF params ")
        log.info("************************")
        log.info(" etf name: " + str(self.etf.symbol))
        log.info(" risk free asset: " + str(self.risk_free_asset.symbol))
        log.info(" market ma threshold: " + str(self.ma_threshold))
        log.info(" lookback: " + str(self.lookback))
        log.info(" pct alloc: " + str(self.alloc))
        log.info(" sell_offset: " + str(self.sell_offset))
        log.info(" buy_offset: " + str(self.buy_offset))

        
#Base class for Equity Strategies
class Equity_Strat(Strategy):
    
    def __init__(self, pct_alloc,risk_free_asset,ma_threshold, index, sell_offset, buy_offset):
        Strategy.__init__(self,pct_alloc,risk_free_asset,ma_threshold,sell_offset, buy_offset)
        self.candidates_available = False
        self.index = index #set 

        
    def robust(self,context,data):
        r = 0.0     
        
        ex_return = self.get_excess_return(context,data)
        if ex_return > 0.0:
            r += 0.5
        #if data[self.index].price > data[self.index].mavg(self.ma_threshold):
        if data[self.index].price > history(self.ma_threshold, '1d', 'price')[self.index].mean():  
            r += 0.5
        
        return r
    
    
    def get_excess_return(self,context,data):
        h = history( self.lookback,'1d','price')[[self.index,self.risk_free_asset]]
        h = h.resample('M',how='last')

        pct_change =h.iloc[[0,-1]].pct_change()[1:]
        pct_change = pct_change.dropna(axis=1)
        
        ex_return = pct_change[self.index]-pct_change[self.risk_free_asset]
        
        return ex_return[0]


#Momentum Strategy
# -- Sort equities on best 1 year performance
class Mom_Strat(Equity_Strat):
   
    def __init__(self, pct_alloc, risk_free_asset, ma_threshold, index, sell_offset, buy_offset):
        Equity_Strat.__init__(self,pct_alloc,risk_free_asset,ma_threshold,index,sell_offset, buy_offset)
        
        self.lower_market_cap = 2000e6     #lower market cap for candidate selection
        self.limit = 500                   #max number of stocks to select for fundamental query
        self.num_positions = 30           #max number of positions the strategy will select
        
        
    def set_sell_day(self,context,data):
        self.sell_day = True
        
    
    def set_buy_day(self,context,data):
        self.buy_day = True
        
   
    def before_trading_start(self,context,data): 
        
        if self.buy_day:
            self.get_candidates()
            if len(self.candidates) > 0:
                self.candidates_available = True
                update_universe(self.candidates)
            
        
    def handle_data(self,context,data):
        if self.sell_day:
            self.exec_sell(context,data)
            self.sell_day = False
        
        if self.buy_day and self.candidates_available:
            self.exec_rank(context, data)
            self.exec_buy(context,data)
            self.buy_day = False
            self.candidates_available = False

        
    def get_candidates(self):
        df_fundamentals = get_fundamentals(
            query(
                fundamentals.valuation.shares_outstanding,
                fundamentals.valuation.market_cap
            )
            #.filter(fundamentals.company_reference.country_id == "USA")
            .filter(or_(fundamentals.company_reference.primary_exchange_id == "NAS", fundamentals.company_reference.primary_exchange_id == "NYSE"))
            .filter(fundamentals.valuation.market_cap != None)
            .filter(fundamentals.valuation.market_cap > self.lower_market_cap ) 
            .filter(fundamentals.valuation.shares_outstanding != None)
            #.filter(fundamentals.share_class_reference.is_depositary_receipt == False)
            .filter(fundamentals.share_class_reference.is_primary_share == True)
            .filter(fundamentals.company_reference.primary_exchange_id != "OTCPK")
            .order_by(fundamentals.valuation.market_cap.desc())
            .limit(self.limit)
        )
    
        self.candidates=df_fundamentals.columns
   
    
    def exec_rank(self,context,data):
        
        #get TMOM
        h = history( self.lookback,'1d','price')[self.candidates]
        h = h.resample('M',how='last') #get monthly returns
        
        pct_change =h.iloc[[0,-1]].pct_change()[1:]
        pct_change = pct_change.dropna(axis=1)
        
        #convert dataframe to series for sorting. Then sort in descending order
        self.longs =  pct_change.squeeze().order(ascending=False)
        self.longs = self.longs[self.longs > 0.0]
        self.longs = self.longs[:self.num_positions]
    
 
    def exec_buy(self,context,data):
        
        r= self.robust(context,data)
        open_orders = get_open_orders()
            
        l = len(self.longs)
        w = (r*self.alloc)/l if l > 0 else 0 #robust adjusted weight
        
        for s in self.longs.index:
            if s in data and s not in open_orders:
                order_target_percent(s,w)
                if w > 0.0:
                    self.bought.append(s)
        
        log.info(str(type(self).__name__) + " longs list len " + str(len(self.longs)))
        log.info(str(type(self).__name__) + " bought list len: " + str(len(self.bought)))    
    
    def exec_sell(self,context,data):

        open_orders = get_open_orders()
        
        for s in context.portfolio.positions :
            if s not in open_orders :
                if s in self.bought:
                    order_target_percent(s,0)
                    self.bought.remove(s)
                
        log.info(str(type(self).__name__) + " bought list len: " + str(len(self.bought)))
        
    
    
    def show_params(self,c):
        log.info("************************")
        log.info(" Momentum params ")
        log.info("************************")
        log.info(" lower market cap: " + str(self.lower_market_cap))
        log.info(" stock search limit: " + str(self.limit))
        log.info(" risk free asset: " + str(self.risk_free_asset.symbol))
        log.info(" number of positions: " + str(self.num_positions))
        log.info(" moving average threshold: " + str(self.ma_threshold))
        log.info(" pct alloc: " + str(self.alloc))
        log.info(" sell_offset: " + str(self.sell_offset))
        log.info(" buy_offset: " + str(self.buy_offset))
        log.info(" lookback: " + str(self.lookback))
       
        

class Value_Strat(Equity_Strat):
    
    def __init__(self, pct_alloc, risk_free_asset, ma_threshold, index, sell_offset, buy_offset):
        
        Equity_Strat.__init__(self,pct_alloc,risk_free_asset,ma_threshold,index,sell_offset, buy_offset)
    
        self.lower_market_cap = 2000e6
        self.limit = 500
        self.num_positions = 30
        self.rebalance_month = 1 
        self.longs = []
        
    
    def set_sell_day(self,context,data):
         today = get_datetime()
         if today.month == self.rebalance_month:
             self.sell_day = True
        
    
    def set_buy_day(self,context,data):
        today = get_datetime()
        if today.month == self.rebalance_month:
            self.buy_day = True
        
   
    def before_trading_start(self,context,data): 
        
        if self.sell_day:
            update_universe(self.bought)

        if self.buy_day:
            self.get_candidates()
            self.candidates_available = True
            update_universe(self.candidates)
    
    
    def handle_data(self,context,data):
        
        if self.sell_day:
            self.exec_sell(context,data)
            self.sell_day = False
        
        if self.buy_day and self.candidates_available:
            self.exec_rank(context,data)
            self.exec_buy(context,data)
            self.buy_day = False
            self.candidates_available = False
 
    
    def get_candidates(self):
        excluded_sectors = [103, 207]
        sector_code = fundamentals.asset_classification.morningstar_sector_code
        fundamental_df = get_fundamentals(
            query(
                sector_code,
                fundamentals.valuation.market_cap,
                fundamentals.valuation.enterprise_value,
                fundamentals.cash_flow_statement.capital_expenditure,
                fundamentals.operation_ratios.roic,
                fundamentals.income_statement.ebit,
                fundamentals.income_statement.ebitda,
                fundamentals.balance_sheet.total_assets,            
        )
        #.filter(fundamentals.company_reference.country_id == "USA")
        .filter(or_(fundamentals.company_reference.primary_exchange_id == "NAS", fundamentals.company_reference.primary_exchange_id == "NYSE"))
        .filter(fundamentals.share_class_reference.is_primary_share == True)
        .filter(fundamentals.company_reference.primary_exchange_id != "OTCPK")
        #.filter(fundamentals.share_class_reference.is_depositary_receipt == False)
        .filter(~sector_code.in_(excluded_sectors))
        .filter(fundamentals.valuation.market_cap > self.lower_market_cap ) 
        .filter(fundamentals.valuation_ratios.ev_to_ebitda > 0)
        .order_by(fundamentals.valuation_ratios.ev_to_ebitda.asc())
        .limit(self.limit)
        )
        
        self.candidates = fundamental_df
        
        
    def exec_rank(self,context,data):
    
        #get MF components
        earnings_yield = self.candidates.ix['ebit'] / self.candidates.ix['enterprise_value']
        roic = self.candidates.ix['roic'].copy()
    
        #sort MF components
        earnings_yield.sort(ascending=False)
        roic.sort(ascending=False)

        #get ranks for components
        earnings_yield_ranking = pd.Series(range(len(earnings_yield)), index=earnings_yield.index)
        roic_ranking = pd.Series(range(len(roic)), index=roic.index) 
    
        #get combined ranking - ranks = earnings_yield_ranking + roic_ranking
        ranks = {}
        for s in self.candidates.columns:
            v_rank = earnings_yield_ranking.loc[s.sid]
            q_rank = roic_ranking.loc[s.sid]
            ranks[s] = v_rank+q_rank
    
        #convert to df and sort
        mf_ranks = pd.Series(ranks,name='Symbol')
        mf_ranks.sort(ascending = True)
        
        self.longs = mf_ranks.index.tolist()
        self.longs = self.longs[:self.num_positions]

     
    def exec_buy(self,context,data):

        r= self.robust(context,data)
        open_orders = get_open_orders()
        
        l = len(self.longs)
        w = (r*self.alloc)/self.num_positions if l > 0 else 0
            
        for s in self.longs:
            if s not in open_orders and s.symbol != 'MUR' and s.symbol != 'NFX':
                order_target_percent(s,w)
                if w > 0.0:
                    self.bought.append(s)
            
        log.info(str(type(self).__name__) + " longs list len " + str(len(self.longs)))
        log.info(str(type(self).__name__) + " bought " + str(len(self.bought)) + " stocks")

        
    def exec_sell(self,context,data):
        open_orders = get_open_orders()
        
        for s in context.portfolio.positions:
            if s in data and s in self.bought and s not in open_orders:
                order_target_percent(s,0)
                self.bought.remove(s)
            
        log.info(str(type(self).__name__) + " bought list len: " + str(len(self.bought)))

        
    def show_params(self,c):
        log.info("************************")
        log.info(" Value params ")
        log.info("************************")
        log.info(" lower market cap: " + str(self.lower_market_cap))
        log.info(" stock search limit: " + str(self.limit))
        log.info(" risk free asset: " + str(self.risk_free_asset.symbol))
        log.info(" number of positions: " + str(self.num_positions))
        log.info(" moving average threshold: " + str(self.ma_threshold))
        log.info(" pct alloc: " + str(self.alloc))
        log.info(" sell_offset: " + str(self.sell_offset))
        log.info(" buy_offset: " + str(self.buy_offset))        
        log.info(" rebalance month: " + str(self.rebalance_month))

        

def initialize(context):
    #set_commission(commission.PerTrade(cost=0.00))  
    #set_slippage(slippage.FixedSlippage(spread=0.00))
    #Common vars
    c = context
    c.lastYear = 2015
    c.lastMonth = 12
    c.index = symbol('SPY')
    c.risk_free_asset = symbol('SHY')
    c.ma_threshold = 200
    
    #register Strategies
    #IVY_4 allocations - no allocation to world equities
    # replace GSG with Gold to get longer backtest
    '''
    c.strategies = [ETF_Strat(0.25,c.risk_free_asset,c.ma_threshold,symbol('SPY'),1,2),
                    ETF_Strat(0.25,c.risk_free_asset,c.ma_threshold,symbol('IYR'),1,2),
                    ETF_Strat(0.25,c.risk_free_asset,c.ma_threshold,symbol('GLD'),1,2),
                    ETF_Strat(0.25,c.risk_free_asset,c.ma_threshold,symbol('IEF'),1,2)]
    '''
   
    #RAA Balanced - 40% allocation to equities
    '''
    c.strategies = [Value_Strat(0.2,c.risk_free_asset, c.ma_threshold, c.index,11,13),
                    Mom_Strat(0.2, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ,
                    ETF_Strat(0.2,c.risk_free_asset,c.ma_threshold,symbol('IYR'),1,2),
                    ETF_Strat(0.2,c.risk_free_asset,c.ma_threshold,symbol('GLD'),1,2),
                    ETF_Strat(0.2,c.risk_free_asset,c.ma_threshold,symbol('IEF'),1,2)]
    '''    
    
    #RAA Moderate Tilt - 60% allocation to equities
    '''
    c.strategies = [Value_Strat(0.3,c.risk_free_asset, c.ma_threshold, c.index,11,13),
                    Mom_Strat(0.3, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ,
                    ETF_Strat(0.1,c.risk_free_asset,c.ma_threshold,symbol('IYR'),1,2),
                    ETF_Strat(0.1,c.risk_free_asset,c.ma_threshold,symbol('GLD'),1,2),
                    ETF_Strat(0.2,c.risk_free_asset,c.ma_threshold,symbol('IEF'),1,2)]
    '''
        
    
    #RAA Aggressive Tilt - 80% allocation to equities
    
    c.strategies = [Value_Strat(0.4,c.risk_free_asset, c.ma_threshold, c.index,5,6),
                    Mom_Strat(0.4, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ,
                    ETF_Strat(0.05,c.risk_free_asset,c.ma_threshold,symbol('IYR'),1, 2),
                    ETF_Strat(0.05,c.risk_free_asset,c.ma_threshold,symbol('GLD'),1,2),
                    ETF_Strat(0.1,c.risk_free_asset,c.ma_threshold,symbol('IEF'),1,2)]
    
    #Mom only
    #c.strategies = [Mom_Strat(1.0, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ]
    #Val only
    #c.strategies = [Value_Strat(1.0,c.risk_free_asset, c.ma_threshold, c.index,5,6) ]
    #Mom/Val 50/50
    #c.strategies = [Value_Strat(0.5,c.risk_free_asset, c.ma_threshold, c.index,21,22),
    #                Mom_Strat(0.5, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ]
    
    #show parameters of all strategies
    show_params(c)
    
    for strat in c.strategies:
        schedule_function(strat.set_sell_day,date_rules.month_start(days_offset=strat.sell_offset),
            time_rules.market_close()) 
        schedule_function(strat.set_buy_day,date_rules.month_start(days_offset=strat.buy_offset),
            time_rules.market_close()) 
    
    #as a last step check final state of portfolio. In particular check total value of unfullfilled orders
    schedule_function(check_positions,date_rules.month_end(days_offset=0),
                      time_rules.market_open(minutes=240))
    

def check_positions(context,data):

    today = get_datetime()
    
    if today.year == context.lastYear and today.month == context.lastMonth:
        #log current account
        log.info(" account leverage: " + str(context.account.leverage))
        log.info(" portfolio value: " + str(context.portfolio.portfolio_value))
        log.info(" portfolio cash: " + str(context.portfolio.cash))
        

        #check number and value of unfullfilled trades
        # unfullfilled trades are often the cause of a strategy appearing to use leverage when in reality it would not.
        # often the cause of an unfullfilled trade is a company merging or getting bought out
        
        open_orders = get_open_orders()
        
        c = 0
        tot_val = 0
        for s in context.portfolio.positions:
            if s in open_orders and s.end_date.year < context.lastYear and s.end_date.month < context.lastMonth:
                shrs = context.portfolio.positions[s].amount
                #cost = context.portfolio.positions[s].cost_basis
                current = data[s].price
                c += 1
                tot_val +=  (shrs * current)
        
        log.info(" len open orders: " + str(len(open_orders)))
        log.info(" number of unfulfilled trades: " + str(c) + " value: " + str(tot_val))
    

def show_params(c):
    
    log.info("************************")
    log.info(" Algo params ")
    log.info("************************")
    
    for strat in c.strategies:
        strat.show_params(c)
   
    
def before_trading_start(context,data): 
    for strat in context.strategies:
        strat.before_trading_start(context, data)

            
def handle_data(context, data):

    #Call each individual algorithm's handle_data method.
    for strat in context.strategies:
        strat.handle_data(context, data)
    
    record(leverage = context.account.leverage)
    
There was a runtime error.

Hi all,

I refactored the code to work in minute mode, see attached backtest.

Clone Algorithm
94
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
##########################################################################################
# This is an attempt to implement the Robust Asset Allocation (RAA) strategy described here,
#    http://blog.alphaarchitect.com/2014/12/02/the-robust-asset-allocation-raa-solution/
# and described in detail in 'DIY Financial Advisor' by W Gray, J Vogel, D Foulke
#
# Notes on implementation
#    - all strategies are required to implement a robust asset allocation function
#    - as a simplification the framework assumes all existing assets in a strategy are sold before new assets are ranked and then bought
#    - the implementation ensures that strategies that do fundamental queries are executed on distinct days so that a maximum number of securities relevant to the particular strategy are examined (i.e. the 500 securities in 'data' are selected by the specific strategy and there is no overlap with other strategies)
#    -Since stocks in the Q universe are generally restricted to those that trade on a US exchange, no distinct allocation is made to domestic and international equities, rather fundamental queries are allowed to invest in ADRs, and in securities not necessarily domiciled in the US
#     -a Gold etf (GSG) is used to approximate an allocation to commodities. This is done to allow for a longer backtest.
#    - the Value strategy implemented is the Magic formula
#    - the momentum strategy is Time Series Momentum with a 12 month lookback
#    - the strategy simply goes to cash when out of the market instead of investing in treasuries. An improvement to the framework would be to add a Portfolio Manager class to manage all trades from all strategies and put any cash in an alternative
###########################################################################################

from datetime import timedelta
from sqlalchemy import or_
from scipy import stats
import statsmodels.api as sm
import talib 
import pandas as pd
import numpy as np
import time


#Base Class for all strategies
class Strategy(object):
    
    def __init__(self, pct_alloc, risk_free_asset, ma_threshold,sell_offset, buy_offset):
        self.alloc = pct_alloc                 #percent of portfolio to allocate to strategy
        self.risk_free_asset = risk_free_asset #risk free asset
        self.ma_threshold = ma_threshold       #moving average theshold
        self.sell_offset = sell_offset         #offset for scheduling when strategy will sell
        self.buy_offset = buy_offset           #offset for scheduling when strategy will buy
        self.sell_day = False                  
        self.buy_day = False    
        self.bought = []                       #maintain list of securites that were bought by strategy
        self.lookback = 252                    #set default lookback period to 1 year (252 days)
 

    def getAlloc(self):
        return self.alloc

    
    def setAlloc(self, alloc):
        self.alloc = alloc
    
    
    #Implement robust allocation algo for strategy
    def robust(self,context,data):
        raise NotImplementedError("implement robust method in Subclass")
    
    
    #Show relevant params at beginning of execution
    def show_params(self):
        raise NotImplementedError("implement show_params method in Subclass")
       
    #Implement sell function
    def exec_sell(self,context,data):
        raise NotImplementedError("implement exec_sell method in Subclass")
    
    #Implement rank and buy function here
    def exec_buy(self,context,data):
        raise NotImplementedError("implement exec_buy method in Subclass")
    
    #Implement fundamental queries here
    def before_trading_start(self,context,data): 
        raise NotImplementedError("implement before_trading_start method in Subclass")
    
    
#Class for ETF strategy   - currently limited to holding 1 ETF
class ETF_Strat(Strategy):
    
    def __init__(self, pct_alloc, risk_free_asset, ma_threshold, etf, sell_offset, buy_offset):
        Strategy.__init__(self,pct_alloc,risk_free_asset,ma_threshold,sell_offset, buy_offset)
        
        self.etf = etf    #set ETF for strategy

        
    #calculate excess return over risk free asset    
    def get_excess_return(self,context,data):
        
        h = history( self.lookback,'1d','price')[[self.etf,self.risk_free_asset]]
        h = h.resample('M',how='last')
        
        pct_change =h.iloc[[0,-1]].pct_change()[1:]
        
        pct_change = pct_change.dropna(axis=1)
        
        ex_return = pct_change[self.etf]-pct_change[self.risk_free_asset]
        return ex_return[0]

    
    #implement RAA strategy 
    def robust(self,context,data):
        r = 0.0
    
        #if etf time series MOM returns are greater than risk free asset then invest 1/2 weight if etf price > moving avg then invest 1/2 weight. If both then invest full weight
        ex_return = self.get_excess_return(context,data)
        if ex_return > 0.0:
            r += 0.5
        #if data[self.etf].price > data[self.etf].mavg(self.ma_threshold):
        if data[self.etf].price > history(self.ma_threshold, '1d', 'price')[self.etf].mean(): 
            r += 0.5

        return r
    
    
    #buy etf based on RAA allocation    
    def exec_buy(self,context,data):
        open_orders = get_open_orders()
        r = self.robust(context,data)
        w = r * self.alloc #RAA adjusted weight
        
        if self.etf in data and self.etf not in open_orders and w > 0.0:
            order_target_percent(self.etf,w)
            self.bought.append(self.etf)
       
        log.info(str(type(self).__name__) + " bought " + str(self.etf.symbol) + " w=" + str(w))

        
    #sell etf
    def exec_sell(self,context,data):
        open_orders = get_open_orders()
        
        if self.etf in context.portfolio.positions and self.etf in self.bought and self.etf not in open_orders:
            order_target_percent(self.etf,0)
            self.bought.remove(self.etf)
        
    
    def before_trading_start(self,context,data): 
        pass
    
    
    def show_params(self,c):
        log.info("************************")
        log.info(" ETF params ")
        log.info("************************")
        log.info(" etf name: " + str(self.etf.symbol))
        log.info(" risk free asset: " + str(self.risk_free_asset.symbol))
        log.info(" market ma threshold: " + str(self.ma_threshold))
        log.info(" lookback: " + str(self.lookback))
        log.info(" pct alloc: " + str(self.alloc))
        log.info(" sell_offset: " + str(self.sell_offset))
        log.info(" buy_offset: " + str(self.buy_offset))

        
#Base class for Equity Strategies
class Equity_Strat(Strategy):
    
    def __init__(self, pct_alloc,risk_free_asset,ma_threshold, index, sell_offset, buy_offset):
        Strategy.__init__(self,pct_alloc,risk_free_asset,ma_threshold,sell_offset, buy_offset)
        self.index = index #set 

    
        
    def robust(self,context,data):
        r = 0.0     
        
        ex_return = self.get_excess_return(context,data)
        if ex_return > 0.0:
            r += 0.5
        if data[self.index].price > history(self.ma_threshold, '1d', 'price')[self.index].mean():  
            r += 0.5
        
        return r
    
    
    def get_excess_return(self,context,data):
        h = history( self.lookback,'1d','price')[[self.index,self.risk_free_asset]]
        h = h.resample('M',how='last')

        pct_change =h.iloc[[0,-1]].pct_change()[1:]
        pct_change = pct_change.dropna(axis=1)
        
        ex_return = pct_change[self.index]-pct_change[self.risk_free_asset]
        
        return ex_return[0]


#Momentum Strategy
# -- Sort equities on best 1 year performance
class Mom_Strat(Equity_Strat):
   
    def __init__(self, pct_alloc, risk_free_asset, ma_threshold, index, sell_offset, buy_offset):
        Equity_Strat.__init__(self,pct_alloc,risk_free_asset,ma_threshold,index,sell_offset, buy_offset)
        
        self.lower_market_cap = 4000e6     #lower market cap for candidate selection
        self.limit = 500                   #max number of stocks to select for fundamental query
        self.num_positions = 20           #max number of positions the strategy will select
        
        
    def update_universe_sell(self,context,data):
        self.sell_day = True
        
    
    def update_universe_buy(self,context,data):
        self.buy_day = True
        
   
    def before_trading_start(self,context,data): 
        
        if self.buy_day:
            self.get_candidates()
            if len(self.candidates) > 0:
                self.candidates_available = True
                update_universe(self.candidates)
                log.info(str(type(self).__name__) + " buy universe updated on: " + str(get_datetime()))  
            
        
    def get_candidates(self):
        df_fundamentals = get_fundamentals(
            query(
                fundamentals.valuation.shares_outstanding,
                fundamentals.valuation.market_cap
            )
            #.filter(fundamentals.company_reference.country_id == "USA")
            .filter(or_(fundamentals.company_reference.primary_exchange_id == "NAS", fundamentals.company_reference.primary_exchange_id == "NYSE"))
            .filter(fundamentals.valuation.market_cap != None)
            .filter(fundamentals.valuation.market_cap > self.lower_market_cap ) 
            .filter(fundamentals.valuation.shares_outstanding != None)
            #.filter(fundamentals.share_class_reference.is_depositary_receipt == False)
            .filter(fundamentals.share_class_reference.is_primary_share == True)
            .filter(fundamentals.company_reference.primary_exchange_id != "OTCPK")
            .order_by(fundamentals.valuation.market_cap.desc())
            .limit(self.limit)
        )
    
        self.candidates=df_fundamentals.columns
   
        
    def get_rank(self,context,data):
        
        #get TMOM
        h = history( self.lookback,'1d','price')[self.candidates]
        h = h.resample('M',how='last') #get monthly returns
        
        pct_change =h.iloc[[0,-1]].pct_change()[1:]
        pct_change = pct_change.dropna(axis=1)
        
        #convert dataframe to series for sorting. Then sort in descending order
        self.longs =  pct_change.squeeze().order(ascending=False)
        self.longs = self.longs[self.longs > 0.0]
        self.longs = self.longs[:self.num_positions]
    
 
    def exec_buy(self,context,data):
        if self.buy_day:
            self.buy_day = False
            r= self.robust(context,data)
        
            if r > 0.0:
        
                self.get_rank(context,data)
            
                l = len(self.longs)
                w = (r*self.alloc)/l if l > 0 else 0 #robust adjusted weight
        
                open_orders = get_open_orders()
                for s in self.longs.index:
                    if s in data and s not in open_orders:
                        order_target_percent(s,w)
                        if w > 0.0:
                            self.bought.append(s)
            
                log.info(str(type(self).__name__) + " longs list len " + str(len(self.longs)))
                log.info(str(type(self).__name__) + " bought list len: " + str(len(self.bought)))    
    
    
    def exec_sell(self,context,data):
        if self.sell_day:
            self.sell_day = False
            open_orders = get_open_orders()
        
            for s in context.portfolio.positions :
                if s not in open_orders :
                    if s in self.bought:
                        order_target_percent(s,0)
                        self.bought.remove(s)
                
            log.info(str(type(self).__name__) + " bought list len: " + str(len(self.bought)))
        
    
    def show_params(self,c):
        log.info("************************")
        log.info(" Momentum params ")
        log.info("************************")
        log.info(" lower market cap: " + str(self.lower_market_cap))
        log.info(" stock search limit: " + str(self.limit))
        log.info(" risk free asset: " + str(self.risk_free_asset.symbol))
        log.info(" number of positions: " + str(self.num_positions))
        log.info(" moving average threshold: " + str(self.ma_threshold))
        log.info(" pct alloc: " + str(self.alloc))
        log.info(" sell_offset: " + str(self.sell_offset))
        log.info(" buy_offset: " + str(self.buy_offset))
        log.info(" lookback: " + str(self.lookback))
       
        

class Value_Strat(Equity_Strat):
    
    def __init__(self, pct_alloc, risk_free_asset, ma_threshold, index, sell_offset, buy_offset):
        
        Equity_Strat.__init__(self,pct_alloc,risk_free_asset,ma_threshold,index,sell_offset, buy_offset)
    
        self.lower_market_cap = 4000e6
        self.limit = 500
        self.num_positions = 20
        self.rebalance_month = 6 
        self.longs = []
        
    
    def update_universe_sell(self,context,data):
         today = get_datetime()
         if today.month == self.rebalance_month:
             self.sell_day = True
        
    
    def update_universe_buy(self,context,data):
        today = get_datetime()
        if today.month == self.rebalance_month:
            self.buy_day = True
        
   
    def before_trading_start(self,context,data): 
        
        if self.sell_day:
            update_universe(self.bought)
            log.info(str(type(self).__name__) + " sell universe updated on: " + str(get_datetime()))  

        if self.buy_day:
            self.get_candidates()
            update_universe(self.candidates)
            log.info(str(type(self).__name__) + " buy universe updated on: " + str(get_datetime()))  
    
    
    def get_candidates(self):
        excluded_sectors = [103, 207]
        sector_code = fundamentals.asset_classification.morningstar_sector_code
        fundamental_df = get_fundamentals(
            query(
                sector_code,
                fundamentals.valuation.market_cap,
                fundamentals.valuation.enterprise_value,
                fundamentals.cash_flow_statement.capital_expenditure,
                fundamentals.operation_ratios.roic,
                fundamentals.income_statement.ebit,
                fundamentals.income_statement.ebitda,
                fundamentals.balance_sheet.total_assets,            
        )
        #.filter(fundamentals.company_reference.country_id == "USA")
        .filter(or_(fundamentals.company_reference.primary_exchange_id == "NAS", fundamentals.company_reference.primary_exchange_id == "NYSE"))
        .filter(fundamentals.share_class_reference.is_primary_share == True)
        .filter(fundamentals.company_reference.primary_exchange_id != "OTCPK")
        #.filter(fundamentals.share_class_reference.is_depositary_receipt == False)
        .filter(~sector_code.in_(excluded_sectors))
        .filter(fundamentals.valuation.market_cap > self.lower_market_cap ) 
        .filter(fundamentals.valuation_ratios.ev_to_ebitda > 0)
        .order_by(fundamentals.valuation_ratios.ev_to_ebitda.asc())
        .limit(self.limit)
        )
        
        self.candidates = fundamental_df
        
        
    def get_rank(self,context,data):
    
        #get MF components
        earnings_yield = self.candidates.ix['ebit'] / self.candidates.ix['enterprise_value']
        roic = self.candidates.ix['roic'].copy()
    
        #sort MF components
        earnings_yield.sort(ascending=False)
        roic.sort(ascending=False)

        #get ranks for components
        earnings_yield_ranking = pd.Series(range(len(earnings_yield)), index=earnings_yield.index)
        roic_ranking = pd.Series(range(len(roic)), index=roic.index) 
    
        #get combined ranking - ranks = earnings_yield_ranking + roic_ranking
        ranks = {}
        for s in self.candidates.columns:
            v_rank = earnings_yield_ranking.loc[s.sid]
            q_rank = roic_ranking.loc[s.sid]
            ranks[s] = v_rank+q_rank
    
        #convert to df and sort
        mf_ranks = pd.Series(ranks,name='Symbol')
        mf_ranks.sort(ascending = True)
        
        self.longs = mf_ranks.index.tolist()
        self.longs = self.longs[:self.num_positions]

     
    def exec_buy(self,context,data):
        if self.buy_day:
            self.buy_day = False
            
            r= self.robust(context,data)
            if r > 0.0:
                
                self.get_rank(context,data)
                
                l = len(self.longs)
                w = (r*self.alloc)/self.num_positions if l > 0 else 0
        
                open_orders = get_open_orders()
                for s in self.longs:
                    if s in data and s not in open_orders:
                        order_target_percent(s,w)
                        if w > 0.0:
                            self.bought.append(s)
                    
                log.info(str(type(self).__name__) + " longs list len " + str(len(self.longs)))
                log.info(str(type(self).__name__) + " bought " + str(len(self.bought)) + " stocks")
                log.info(str(type(self).__name__) + " buys completed on: " + str(get_datetime()))  

        
    def exec_sell(self,context,data):
         if self.sell_day:
            self.sell_day = False
            
            open_orders = get_open_orders()
        
            for s in context.portfolio.positions:
                if s in self.bought and s not in open_orders:
                    order_target_percent(s,0)
                    self.bought.remove(s)
        
            log.info(str(type(self).__name__) + " bought list len: " + str(len(self.bought)))
            log.info(str(type(self).__name__) + " sells completed on: " + str(get_datetime()))  

        
    def show_params(self,c):
        log.info("************************")
        log.info(" Value params ")
        log.info("************************")
        log.info(" lower market cap: " + str(self.lower_market_cap))
        log.info(" stock search limit: " + str(self.limit))
        log.info(" risk free asset: " + str(self.risk_free_asset.symbol))
        log.info(" number of positions: " + str(self.num_positions))
        log.info(" moving average threshold: " + str(self.ma_threshold))
        log.info(" pct alloc: " + str(self.alloc))
        log.info(" sell_offset: " + str(self.sell_offset))
        log.info(" buy_offset: " + str(self.buy_offset))        
        log.info(" rebalance month: " + str(self.rebalance_month))

        

def initialize(context):
    #set_commission(commission.PerTrade(cost=0.00))  
    #set_slippage(slippage.FixedSlippage(spread=0.00))
    #Common vars
    c = context
    c.lastYear = 2015
    c.lastMonth = 12
    c.index = symbol('SPY')
    c.risk_free_asset = symbol('SHY')
    c.ma_threshold = 200
    
    #register Strategies
    #IVY_4 allocations - no allocation to world equities
    # replace GSG with Gold to get longer backtest
    '''
    c.strategies = [ETF_Strat(0.25,c.risk_free_asset,c.ma_threshold,symbol('SPY'),1,2),
                    ETF_Strat(0.25,c.risk_free_asset,c.ma_threshold,symbol('IYR'),1,2),
                    ETF_Strat(0.25,c.risk_free_asset,c.ma_threshold,symbol('GLD'),1,2),
                    ETF_Strat(0.25,c.risk_free_asset,c.ma_threshold,symbol('IEF'),1,2)]
    '''
   
    #RAA Balanced - 40% allocation to equities
    '''
    c.strategies = [Value_Strat(0.2,c.risk_free_asset, c.ma_threshold, c.index,11,13),
                    Mom_Strat(0.2, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ,
                    ETF_Strat(0.2,c.risk_free_asset,c.ma_threshold,symbol('IYR'),1,2),
                    ETF_Strat(0.2,c.risk_free_asset,c.ma_threshold,symbol('GLD'),1,2),
                    ETF_Strat(0.2,c.risk_free_asset,c.ma_threshold,symbol('IEF'),1,2)]
    '''    
    
    #RAA Moderate Tilt - 60% allocation to equities
    '''
    c.strategies = [Value_Strat(0.3,c.risk_free_asset, c.ma_threshold, c.index,11,13),
                    Mom_Strat(0.3, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ,
                    ETF_Strat(0.1,c.risk_free_asset,c.ma_threshold,symbol('IYR'),1,2),
                    ETF_Strat(0.1,c.risk_free_asset,c.ma_threshold,symbol('GLD'),1,2),
                    ETF_Strat(0.2,c.risk_free_asset,c.ma_threshold,symbol('IEF'),1,2)]
    '''
        
    
    #RAA Aggressive Tilt - 80% allocation to equities
    c.strategies = [Value_Strat(0.4,c.risk_free_asset, c.ma_threshold, c.index,5,6),
                    Mom_Strat(0.4, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ,
                    ETF_Strat(0.05,c.risk_free_asset,c.ma_threshold,symbol('IYR'),1, 2),
                    ETF_Strat(0.05,c.risk_free_asset,c.ma_threshold,symbol('GLD'),1,2),
                    ETF_Strat(0.1,c.risk_free_asset,c.ma_threshold,symbol('IEF'),1,2)]
    
    #Mom only
    #c.strategies = [Mom_Strat(1.0, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ]
    #Val only
    #c.strategies = [Value_Strat(1.0,c.risk_free_asset, c.ma_threshold, c.index,5,6) ]
    #Mom/Val 50/50
    #c.strategies = [Value_Strat(0.5,c.risk_free_asset, c.ma_threshold, c.index,5,6),
    #                Mom_Strat(0.5, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ]
    
    #show parameters of all strategies
    show_params(c)
    
    for strat in c.strategies:
        #schedule universe update functions for Equity strategies
        # this turns on the flag to updat the universe, the universe is updated the next day before the market starts
        #trading (in the before_trading_start method)
        if type(strat).__name__ != 'ETF_StratProxy':
            schedule_function(strat.update_universe_sell,date_rules.month_start(days_offset=strat.sell_offset),
                              time_rules.market_close(minutes=30)) 
            schedule_function(strat.update_universe_buy,date_rules.month_start(days_offset=strat.buy_offset),
                              time_rules.market_close(minutes=30)) 
        
        #schedule buy and sell functions
        #note these functions are scheduled to run one day after the update universe functions
        #which means they actually run on the same day the universe is updated
        schedule_function(strat.exec_sell,date_rules.month_start(days_offset=(strat.sell_offset+1)),
            time_rules.market_close(minutes=30)) 
        schedule_function(strat.exec_buy,date_rules.month_start(days_offset=(strat.buy_offset+1)),
            time_rules.market_close(minutes=30)) 
    
    #as a last step check final state of portfolio. In particular check total value of unfullfilled orders
    schedule_function(check_positions,date_rules.month_end(days_offset=0),
                      time_rules.market_open(minutes=240))
    #turn on this function to eliminate trades that will go unfulfilled (i.e. will not be sellable).
    #schedule_function(handle_unfulfilled_trades,date_rules.every_day(),
    #                  time_rules.market_open(minutes=15))

#Sell any securities before they become unfulfillable   
def handle_unfulfilled_trades(context,data):
    open_orders = get_open_orders()
    today = get_datetime()
    
    for s in open_orders:
        if s in context.portfolio.positions:
            if (today + timedelta(days=10)) > s.end_date:
                order_target_percent(s,0.0)
    
def check_positions(context,data):

    today = get_datetime()
    
    if today.year == context.lastYear and today.month == context.lastMonth:
        #log current account
        log.info(" account leverage: " + str(context.account.leverage))
        log.info(" portfolio value: " + str(context.portfolio.portfolio_value))
        log.info(" portfolio cash: " + str(context.portfolio.cash))
        

        #check number and value of unfullfilled trades
        # unfullfilled trades are often the cause of a strategy appearing to use leverage when in reality it would not.
        # often the cause of an unfullfilled trade is a company merging or getting bought out
        
        open_orders = get_open_orders()
        
        c = 0
        tot_val = 0
        for s in context.portfolio.positions:
            if s in open_orders and s.end_date.year < context.lastYear and s.end_date.month < context.lastMonth:
                shrs = context.portfolio.positions[s].amount
                #cost = context.portfolio.positions[s].cost_basis
                current = data[s].price
                c += 1
                tot_val +=  (shrs * current)
        
        log.info(" len open orders: " + str(len(open_orders)))
        log.info(" number of unfulfilled trades: " + str(c) + " value: " + str(tot_val))
    

def show_params(c):
    
    log.info("************************")
    log.info(" Algo params ")
    log.info("************************")
    
    for strat in c.strategies:
        strat.show_params(c)
   
    
def before_trading_start(context,data): 
    for strat in context.strategies:
        strat.before_trading_start(context, data)

            
def handle_data(context, data):

    record(leverage = context.account.leverage)
    
There was a runtime error.

@Mark: Is the idea to check for a 'buy day' one day, and then trade the next if the condition is met? or should you instead be trading on the day when the check is made? Just curious!

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Mark,

In line 459 you declared 'SHY' as risk-free asset.
It will be good to use it instead of cash.

@Jamie:

Hi Jamie, are you referring to the fact that the exec_buy/exec_sell functions are scheduled one day after the update_universe_buy/ update_universe_sell functions? If so this is because the universe has to be updated before trading starts, so what happens is the update universe function executes which causes the universe to be updated before start of trading the next day, then the buy/sell orders are executed. So candidates are actually analyzed on the same day they are bought.

Hope that helps,
Mark

Hey there, not able to get this to run in Minute mode. (Mark Segal on 1/12/16)

Clone Algorithm
1
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
##########################################################################################
# This is an attempt to implement the Robust Asset Allocation (RAA) strategy described here,
#    http://blog.alphaarchitect.com/2014/12/02/the-robust-asset-allocation-raa-solution/
# and described in detail in 'DIY Financial Advisor' by W Gray, J Vogel, D Foulke
#
# Notes on implementation
#    - all strategies are required to implement a robust asset allocation function
#    - as a simplification the framework assumes all existing assets in a strategy are sold before new assets are ranked and then bought
#    - the implementation ensures that strategies that do fundamental queries are executed on distinct days so that a maximum number of securities relevant to the particular strategy are examined (i.e. the 500 securities in 'data' are selected by the specific strategy and there is no overlap with other strategies)
#    -Since stocks in the Q universe are generally restricted to those that trade on a US exchange, no distinct allocation is made to domestic and international equities, rather fundamental queries are allowed to invest in ADRs, and in securities not necessarily domiciled in the US
#     -a Gold etf (GSG) is used to approximate an allocation to commodities. This is done to allow for a longer backtest.
#    - the Value strategy implemented is the Magic formula
#    - the momentum strategy is Time Series Momentum with a 12 month lookback
#    - the strategy simply goes to cash when out of the market instead of investing in treasuries. An improvement to the framework would be to add a Portfolio Manager class to manage all trades from all strategies and put any cash in an alternative
###########################################################################################

from datetime import timedelta
from sqlalchemy import or_
from scipy import stats
import statsmodels.api as sm
import talib 
import pandas as pd
import numpy as np
import time


#Base Class for all strategies
class Strategy(object):
    
    def __init__(self, pct_alloc, risk_free_asset, ma_threshold,sell_offset, buy_offset):
        self.alloc = pct_alloc                 #percent of portfolio to allocate to strategy
        self.risk_free_asset = risk_free_asset #risk free asset
        self.ma_threshold = ma_threshold       #moving average theshold
        self.sell_offset = sell_offset         #offset for scheduling when strategy will sell
        self.buy_offset = buy_offset           #offset for scheduling when strategy will buy
        self.sell_day = False                  
        self.buy_day = False    
        self.bought = []                       #maintain list of securites that were bought by strategy
        self.lookback = 252                    #set default lookback period to 1 year (252 days)
 

    def getAlloc(self):
        return self.alloc

    
    def setAlloc(self, alloc):
        self.alloc = alloc
    
    
    #Implement robust allocation algo for strategy
    def robust(self,context,data):
        raise NotImplementedError("implement robust method in Subclass")
    
    
    #Show relevant params at beginning of execution
    def show_params(self):
        raise NotImplementedError("implement show_params method in Subclass")
    
    
    #Implement fundamental queries here
    def handle_data(self,context,data):
        raise NotImplementedError("implement handle_data method in Subclass")
    
    
    #Implement buy, sell and ranking functions here
    def before_trading_start(self,context,data): 
        raise NotImplementedError("implement before_trading_start method in Subclass")
    
    
#Class for ETF strategy   - currently limited to holding 1 ETF
class ETF_Strat(Strategy):
    
    def __init__(self, pct_alloc, risk_free_asset, ma_threshold, etf, sell_offset, buy_offset):
        Strategy.__init__(self,pct_alloc,risk_free_asset,ma_threshold,sell_offset, buy_offset)
        
        self.etf = etf    #set ETF for strategy

        
    #calculate excess return over risk free asset    
    def get_excess_return(self,context,data):
        
        h = history( self.lookback,'1d','price')[[self.etf,self.risk_free_asset]]
        h = h.resample('M',how='last')
        
        pct_change =h.iloc[[0,-1]].pct_change()[1:]
        
        pct_change = pct_change.dropna(axis=1)
        
        ex_return = pct_change[self.etf]-pct_change[self.risk_free_asset]
        return ex_return[0]

    
    #implement RAA strategy 
    def robust(self,context,data):
        r = 0.0
    
        #if etf time series MOM returns are greater than risk free asset then invest 1/2 weight if etf price > moving avg then invest 1/2 weight. If both then invest full weight
        ex_return = self.get_excess_return(context,data)
        if ex_return > 0.0:
            r += 0.5
        if data[self.etf].price > data[self.etf].mavg(self.ma_threshold):
            r += 0.5

        return r
    
    
    #buy etf based on RAA allocation    
    def exec_buy(self,context,data):
        open_orders = get_open_orders()
        r = self.robust(context,data)
        w = r * self.alloc #RAA adjusted weight
        
        if self.etf in data and self.etf not in open_orders and w > 0.0:
            order_target_percent(self.etf,w)
            self.bought.append(self.etf)
       
        log.info(str(type(self).__name__) + " bought " + str(self.etf.symbol) + " w=" + str(w))

        
    #sell etf
    def exec_sell(self,context,data):
        open_orders = get_open_orders()
        
        #if self.etf in context.portfolio.positions and self.etf in data and self.etf in self.bought and self.etf not in open_orders:
        if self.etf in context.portfolio.positions and self.etf in self.bought and self.etf not in open_orders:
            order_target_percent(self.etf,0)
            self.bought.remove(self.etf)
        
    
    def set_sell_day(self,context,data):
        self.sell_day = True
        
    
    def set_buy_day(self,context,data):
        self.buy_day = True
        
        
    def handle_data(self,context,data):
        if self.sell_day:
            self.exec_sell(context,data)
            self.sell_day = False
        
        if self.buy_day:
            self.exec_buy(context,data)
            self.buy_day = False
        

    def before_trading_start(self,context,data): 
        pass
    
    
    def show_params(self,c):
        log.info("************************")
        log.info(" ETF params ")
        log.info("************************")
        log.info(" etf name: " + str(self.etf.symbol))
        log.info(" risk free asset: " + str(self.risk_free_asset.symbol))
        log.info(" market ma threshold: " + str(self.ma_threshold))
        log.info(" lookback: " + str(self.lookback))
        log.info(" pct alloc: " + str(self.alloc))
        log.info(" sell_offset: " + str(self.sell_offset))
        log.info(" buy_offset: " + str(self.buy_offset))

        
#Base class for Equity Strategies
class Equity_Strat(Strategy):
    
    def __init__(self, pct_alloc,risk_free_asset,ma_threshold, index, sell_offset, buy_offset):
        Strategy.__init__(self,pct_alloc,risk_free_asset,ma_threshold,sell_offset, buy_offset)
        
        self.index = index #set 

        
    def robust(self,context,data):
        r = 0.0     
        
        ex_return = self.get_excess_return(context,data)
        if ex_return > 0.0:
            r += 0.5
        if data[self.index].price > data[self.index].mavg(self.ma_threshold):
            r += 0.5
        
        return r
    
    
    def get_excess_return(self,context,data):
        h = history( self.lookback,'1d','price')[[self.index,self.risk_free_asset]]
        h = h.resample('M',how='last')

        pct_change =h.iloc[[0,-1]].pct_change()[1:]
        pct_change = pct_change.dropna(axis=1)
        
        ex_return = pct_change[self.index]-pct_change[self.risk_free_asset]
        
        return ex_return[0]


#Momentum Strategy
# -- Sort equities on best 1 year performance
class Mom_Strat(Equity_Strat):
   
    def __init__(self, pct_alloc, risk_free_asset, ma_threshold, index, sell_offset, buy_offset):
        Equity_Strat.__init__(self,pct_alloc,risk_free_asset,ma_threshold,index,sell_offset, buy_offset)
        
        self.lower_market_cap = 2000e6     #lower market cap for candidate selection
        self.limit = 500                   #max number of stocks to select for fundamental query
        self.num_positions = 30            #max number of positions the strategy will select
        
        
    def set_sell_day(self,context,data):
        self.sell_day = True
        
    
    def set_buy_day(self,context,data):
        self.buy_day = True
        
   
    def before_trading_start(self,context,data): 
        
        if self.buy_day:
            self.get_candidates()
            update_universe(self.candidates)
            
        
    def handle_data(self,context,data):
        if self.sell_day:
            self.exec_sell(context,data)
            self.sell_day = False
        
        if self.buy_day:
            self.exec_rank(context, data)
            self.exec_buy(context,data)
            self.buy_day = False

        
    def get_candidates(self):
        df_fundamentals = get_fundamentals(
            query(
                fundamentals.valuation.shares_outstanding,
                fundamentals.valuation.market_cap
            )
            #.filter(fundamentals.company_reference.country_id == "USA")
            .filter(or_(fundamentals.company_reference.primary_exchange_id == "NAS", fundamentals.company_reference.primary_exchange_id == "NYSE"))
            .filter(fundamentals.valuation.market_cap != None)
            .filter(fundamentals.valuation.market_cap > self.lower_market_cap ) 
            .filter(fundamentals.valuation.shares_outstanding != None)
            #.filter(fundamentals.share_class_reference.is_depositary_receipt == False)
            .filter(fundamentals.share_class_reference.is_primary_share == True)
            .filter(fundamentals.company_reference.primary_exchange_id != "OTCPK")
            .order_by(fundamentals.valuation.market_cap.desc())
            .limit(self.limit)
        )
    
        self.candidates=df_fundamentals.columns
   
    
    def exec_rank(self,context,data):
        
        #get TMOM
        h = history( self.lookback,'1d','price')[self.candidates]
        h = h.resample('M',how='last') #get monthly returns
        
        pct_change =h.iloc[[0,-1]].pct_change()[1:]
        pct_change = pct_change.dropna(axis=1)
        
        #convert dataframe to series for sorting. Then sort in descending order
        self.longs =  pct_change.squeeze().order(ascending=False)
        self.longs = self.longs[self.longs > 0.0]
        self.longs = self.longs[:self.num_positions]
    
 
    def exec_buy(self,context,data):
        r= self.robust(context,data)
        open_orders = get_open_orders()
            
        l = len(self.longs)
        w = (r*self.alloc)/l if l > 0 else 0 #robust adjusted weight
        
        for s in self.longs.index:
            if s in data and s not in open_orders:
                order_target_percent(s,w)
                if w > 0.0:
                    self.bought.append(s)
        
        log.info(str(type(self).__name__) + " longs list len " + str(len(self.longs)))
        log.info(str(type(self).__name__) + " bought list len: " + str(len(self.bought)))    
    
    def exec_sell(self,context,data):

        open_orders = get_open_orders()
        
        for s in context.portfolio.positions :
            if s in data and s not in open_orders :
                if s in self.bought:
                    order_target_percent(s,0)
                    self.bought.remove(s)
                
        log.info(str(type(self).__name__) + " bought list len: " + str(len(self.bought)))
        
    
    
    def show_params(self,c):
        log.info("************************")
        log.info(" Momentum params ")
        log.info("************************")
        log.info(" lower market cap: " + str(self.lower_market_cap))
        log.info(" stock search limit: " + str(self.limit))
        log.info(" risk free asset: " + str(self.risk_free_asset.symbol))
        log.info(" number of positions: " + str(self.num_positions))
        log.info(" moving average threshold: " + str(self.ma_threshold))
        log.info(" pct alloc: " + str(self.alloc))
        log.info(" sell_offset: " + str(self.sell_offset))
        log.info(" buy_offset: " + str(self.buy_offset))
        log.info(" lookback: " + str(self.lookback))
       
        

class Value_Strat(Equity_Strat):
    
    def __init__(self, pct_alloc, risk_free_asset, ma_threshold, index, sell_offset, buy_offset):
        
        Equity_Strat.__init__(self,pct_alloc,risk_free_asset,ma_threshold,index,sell_offset, buy_offset)
    
        self.lower_market_cap = 2000e6
        self.limit = 500
        self.num_positions = 30
        self.rebalance_month = 6 #June
        self.longs = []
        #self.alloc = pct_alloc
        #self.w = self.pct_alloc/self.num_positions

    
    def set_sell_day(self,context,data):
         today = get_datetime()
         if today.month == self.rebalance_month:
             self.sell_day = True
        
    
    def set_buy_day(self,context,data):
        today = get_datetime()
        if today.month == self.rebalance_month:
            self.buy_day = True
        
   
    def before_trading_start(self,context,data): 
        
        if self.sell_day:
            update_universe(self.bought)

        if self.buy_day:
            self.get_candidates()
            update_universe(self.candidates)
    
    
    def handle_data(self,context,data):
        
        if self.sell_day:
            self.exec_sell(context,data)
            self.sell_day = False
        
        if self.buy_day:
            self.exec_rank(context,data)
            self.exec_buy(context,data)
            self.buy_day = False
 
    
    def get_candidates(self):
        excluded_sectors = [103, 207]
        sector_code = fundamentals.asset_classification.morningstar_sector_code
        fundamental_df = get_fundamentals(
            query(
                sector_code,
                fundamentals.valuation.market_cap,
                fundamentals.valuation.enterprise_value,
                fundamentals.cash_flow_statement.capital_expenditure,
                fundamentals.operation_ratios.roic,
                fundamentals.income_statement.ebit,
                fundamentals.income_statement.ebitda,
                fundamentals.balance_sheet.total_assets,            
        )
        #.filter(fundamentals.company_reference.country_id == "USA")
        .filter(or_(fundamentals.company_reference.primary_exchange_id == "NAS", fundamentals.company_reference.primary_exchange_id == "NYSE"))
        .filter(fundamentals.share_class_reference.is_primary_share == True)
        .filter(fundamentals.company_reference.primary_exchange_id != "OTCPK")
        #.filter(fundamentals.share_class_reference.is_depositary_receipt == False)
        .filter(~sector_code.in_(excluded_sectors))
        .filter(fundamentals.valuation.market_cap > self.lower_market_cap ) 
        .filter(fundamentals.valuation_ratios.ev_to_ebitda > 0)
        .order_by(fundamentals.valuation.market_cap.desc())
        .limit(self.limit)
        )
        
        self.candidates = fundamental_df
        #self.longs = self.mf_rank(fundamental_df)
        #self.longs = self.longs[:self.num_positions]

        
    def exec_rank(self,context,data):
    
        #get MF components
        earnings_yield = self.candidates.ix['ebit'] / self.candidates.ix['enterprise_value']
        roic = self.candidates.ix['roic'].copy()
    
        #sort MF components
        earnings_yield.sort(ascending=False)
        roic.sort(ascending=False)

        #get ranks for components
        earnings_yield_ranking = pd.Series(range(len(earnings_yield)), index=earnings_yield.index)
        roic_ranking = pd.Series(range(len(roic)), index=roic.index) 
    
        #get combined ranking - ranks = earnings_yield_ranking + roic_ranking
        ranks = {}
        for s in self.candidates.columns:
            v_rank = earnings_yield_ranking.loc[s.sid]
            q_rank = roic_ranking.loc[s.sid]
            ranks[s] = v_rank+q_rank
    
        #convert to df and sort
        mf_ranks = pd.Series(ranks,name='Symbol')
        mf_ranks.sort(ascending = True)
        
        self.longs = mf_ranks.index.tolist()
        self.longs = self.longs[:self.num_positions]
        
        #return mf_ranks.index.tolist()

     
    def exec_buy(self,context,data):
        
        r= self.robust(context,data)
        open_orders = get_open_orders()
        
        l = len(self.longs)
        w = (r*self.alloc)/self.num_positions if l > 0 else 0
            
        for s in self.longs:
            if s in data and s not in open_orders:
                order_target_percent(s,w)
                if w > 0.0:
                    self.bought.append(s)
            
            
        log.info(str(type(self).__name__) + " longs list len " + str(len(self.longs)))
        log.info(str(type(self).__name__) + " bought " + str(len(self.bought)) + " stocks")

        
    def exec_sell(self,context,data):
        open_orders = get_open_orders()
        
        for s in context.portfolio.positions:
            if s in data and s in self.bought and s not in open_orders:
                order_target_percent(s,0)
                self.bought.remove(s)
            
        log.info(str(type(self).__name__) + " bought list len: " + str(len(self.bought)))

        
    def show_params(self,c):
        log.info("************************")
        log.info(" Value params ")
        log.info("************************")
        log.info(" lower market cap: " + str(self.lower_market_cap))
        log.info(" stock search limit: " + str(self.limit))
        log.info(" risk free asset: " + str(self.risk_free_asset.symbol))
        log.info(" number of positions: " + str(self.num_positions))
        log.info(" moving average threshold: " + str(self.ma_threshold))
        log.info(" pct alloc: " + str(self.alloc))
        log.info(" sell_offset: " + str(self.sell_offset))
        log.info(" buy_offset: " + str(self.buy_offset))        
        log.info(" rebalance month: " + str(self.rebalance_month))

        

def initialize(context):
    set_commission(commission.PerTrade(cost=0.00))  
    set_slippage(slippage.FixedSlippage(spread=0.00))
    #Common vars
    c = context
    c.lastYear = 2015
    c.lastMonth = 12
    c.index = symbol('SPY')
    c.risk_free_asset = symbol('SHY')
    c.ma_threshold = 200
    
    #register Strategies
    #IVY_4 allocations - no allocation to world equities
    # replace GSG with Gold to get longer backtest
    '''
    c.strategies = [ETF_Strat(0.25,c.risk_free_asset,c.ma_threshold,symbol('SPY'),1,2),
                    ETF_Strat(0.25,c.risk_free_asset,c.ma_threshold,symbol('IYR'),1,2),
                    ETF_Strat(0.25,c.risk_free_asset,c.ma_threshold,symbol('GLD'),1,2),
                    ETF_Strat(0.25,c.risk_free_asset,c.ma_threshold,symbol('IEF'),1,2)]
    '''
   
    #RAA Balanced - 40% allocation to equities
    '''
    c.strategies = [Value_Strat(0.2,c.risk_free_asset, c.ma_threshold, c.index,11,13),
                    Mom_Strat(0.2, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ,
                    ETF_Strat(0.2,c.risk_free_asset,c.ma_threshold,symbol('IYR'),1,2),
                    ETF_Strat(0.2,c.risk_free_asset,c.ma_threshold,symbol('GLD'),1,2),
                    ETF_Strat(0.2,c.risk_free_asset,c.ma_threshold,symbol('IEF'),1,2)]
    '''    
    
    #RAA Moderate Tilt - 60% allocation to equities
    '''
    c.strategies = [Value_Strat(0.3,c.risk_free_asset, c.ma_threshold, c.index,11,13),
                    Mom_Strat(0.3, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ,
                    ETF_Strat(0.1,c.risk_free_asset,c.ma_threshold,symbol('IYR'),1,2),
                    ETF_Strat(0.1,c.risk_free_asset,c.ma_threshold,symbol('GLD'),1,2),
                    ETF_Strat(0.2,c.risk_free_asset,c.ma_threshold,symbol('IEF'),1,2)]
    '''
        
    
    #RAA Aggressive Tilt - 80% allocation to equities
    
    c.strategies = [Value_Strat(0.4,c.risk_free_asset, c.ma_threshold, c.index,11,13),
                    Mom_Strat(0.4, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ,
                    ETF_Strat(0.05,c.risk_free_asset,c.ma_threshold,symbol('IYR'),1,2),
                    ETF_Strat(0.05,c.risk_free_asset,c.ma_threshold,symbol('GLD'),1,2),
                    ETF_Strat(0.1,c.risk_free_asset,c.ma_threshold,symbol('IEF'),1,2)]
    
    #Mom only
    #c.strategies = [Mom_Strat(1.0, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ]
    #Val only
    #c.strategies = [Value_Strat(1.0,c.risk_free_asset, c.ma_threshold, c.index,11,13) ]
    #Mom/Val 50/50
    #c.strategies = [Value_Strat(0.5,c.risk_free_asset, c.ma_threshold, c.index,11,13),
    #                Mom_Strat(0.5, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ]
    
    #show parameters of all strategies
    show_params(c)
    
    for strat in c.strategies:
        schedule_function(strat.set_sell_day,date_rules.month_start(days_offset=strat.sell_offset),
            time_rules.market_open(minutes=120)) 
        schedule_function(strat.set_buy_day,date_rules.month_start(days_offset=strat.buy_offset),
            time_rules.market_open(minutes=120)) 
    
    #as a last step check final state of portfolio. In particular check total value of unfullfilled orders
    schedule_function(check_positions,date_rules.month_end(days_offset=0),
                      time_rules.market_open(minutes=240))
    

def check_positions(context,data):

    today = get_datetime()
    
    if today.year == context.lastYear and today.month == context.lastMonth:
        #log current account
        log.info(" account leverage: " + str(context.account.leverage))
        log.info(" portfolio value: " + str(context.portfolio.portfolio_value))
        log.info(" portfolio cash: " + str(context.portfolio.cash))
        

        #check number and value of unfullfilled trades
        # unfullfilled trades are often the cause of a strategy appearing to use leverage when in reality it would not.
        # often the cause of an unfullfilled trade is a company merging or getting bought out
        
        open_orders = get_open_orders()
        #update_universe(open_order_sids.keys())
        
        
        c = 0
        tot_val = 0
        for s in context.portfolio.positions:
            if s in open_orders and s.end_date.year < context.lastYear and s.end_date.month < context.lastMonth:
                shrs = context.portfolio.positions[s].amount
                cost = context.portfolio.positions[s].cost_basis
                current = data[s].price
                c += 1
                tot_val +=  (shrs * current)
        
        log.info(" len open orders: " + str(len(open_orders)))
        log.info(" number of unfullfilled trades: " + str(c) + " value: " + str(tot_val))
    

def show_params(c):
    
    log.info("************************")
    log.info(" Algo params ")
    log.info("************************")
    
    for strat in c.strategies:
        strat.show_params(c)
   
    
def before_trading_start(context,data): 
    for strat in context.strategies:
        strat.before_trading_start(context, data)

            
def handle_data(context, data):

    #Call each individual algorithm's handle_data method.
    for strat in context.strategies:
        strat.handle_data(context, data)
    
    record(leverage = context.account.leverage)
    
There was a runtime error.

Hi Corey,

I just cloned the algorithm and ran the attached backtest, seems to run fine.

Clone Algorithm
83
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
##########################################################################################
# This is an attempt to implement the Robust Asset Allocation (RAA) strategy described here,
#    http://blog.alphaarchitect.com/2014/12/02/the-robust-asset-allocation-raa-solution/
# and described in detail in 'DIY Financial Advisor' by W Gray, J Vogel, D Foulke
#
# Notes on implementation
#    - all strategies are required to implement a robust asset allocation function
#    - as a simplification the framework assumes all existing assets in a strategy are sold before new assets are ranked and then bought
#    - the implementation ensures that strategies that do fundamental queries are executed on distinct days so that a maximum number of securities relevant to the particular strategy are examined (i.e. the 500 securities in 'data' are selected by the specific strategy and there is no overlap with other strategies)
#    -Since stocks in the Q universe are generally restricted to those that trade on a US exchange, no distinct allocation is made to domestic and international equities, rather fundamental queries are allowed to invest in ADRs, and in securities not necessarily domiciled in the US
#     -a Gold etf (GSG) is used to approximate an allocation to commodities. This is done to allow for a longer backtest.
#    - the Value strategy implemented is the Magic formula
#    - the momentum strategy is Time Series Momentum with a 12 month lookback
#    - the strategy simply goes to cash when out of the market instead of investing in treasuries. An improvement to the framework would be to add a Portfolio Manager class to manage all trades from all strategies and put any cash in an alternative
###########################################################################################

from datetime import timedelta
from sqlalchemy import or_
from scipy import stats
import statsmodels.api as sm
import talib 
import pandas as pd
import numpy as np
import time


#Base Class for all strategies
class Strategy(object):
    
    def __init__(self, pct_alloc, risk_free_asset, ma_threshold,sell_offset, buy_offset):
        self.alloc = pct_alloc                 #percent of portfolio to allocate to strategy
        self.risk_free_asset = risk_free_asset #risk free asset
        self.ma_threshold = ma_threshold       #moving average theshold
        self.sell_offset = sell_offset         #offset for scheduling when strategy will sell
        self.buy_offset = buy_offset           #offset for scheduling when strategy will buy
        self.sell_day = False                  
        self.buy_day = False    
        self.bought = []                       #maintain list of securites that were bought by strategy
        self.lookback = 252                    #set default lookback period to 1 year (252 days)
 

    def getAlloc(self):
        return self.alloc

    
    def setAlloc(self, alloc):
        self.alloc = alloc
    
    
    #Implement robust allocation algo for strategy
    def robust(self,context,data):
        raise NotImplementedError("implement robust method in Subclass")
    
    
    #Show relevant params at beginning of execution
    def show_params(self):
        raise NotImplementedError("implement show_params method in Subclass")
       
    #Implement sell function
    def exec_sell(self,context,data):
        raise NotImplementedError("implement exec_sell method in Subclass")
    
    #Implement rank and buy function here
    def exec_buy(self,context,data):
        raise NotImplementedError("implement exec_buy method in Subclass")
    
    #Implement fundamental queries here
    def before_trading_start(self,context,data): 
        raise NotImplementedError("implement before_trading_start method in Subclass")
    
    
#Class for ETF strategy   - currently limited to holding 1 ETF
class ETF_Strat(Strategy):
    
    def __init__(self, pct_alloc, risk_free_asset, ma_threshold, etf, sell_offset, buy_offset):
        Strategy.__init__(self,pct_alloc,risk_free_asset,ma_threshold,sell_offset, buy_offset)
        
        self.etf = etf    #set ETF for strategy

        
    #calculate excess return over risk free asset    
    def get_excess_return(self,context,data):
        
        h = history( self.lookback,'1d','price')[[self.etf,self.risk_free_asset]]
        h = h.resample('M',how='last')
        
        pct_change =h.iloc[[0,-1]].pct_change()[1:]
        
        pct_change = pct_change.dropna(axis=1)
        
        ex_return = pct_change[self.etf]-pct_change[self.risk_free_asset]
        return ex_return[0]

    
    #implement RAA strategy 
    def robust(self,context,data):
        r = 0.0
    
        #if etf time series MOM returns are greater than risk free asset then invest 1/2 weight if etf price > moving avg then invest 1/2 weight. If both then invest full weight
        ex_return = self.get_excess_return(context,data)
        if ex_return > 0.0:
            r += 0.5
        #if data[self.etf].price > data[self.etf].mavg(self.ma_threshold):
        if data[self.etf].price > history(self.ma_threshold, '1d', 'price')[self.etf].mean(): 
            r += 0.5

        return r
    
    
    #buy etf based on RAA allocation    
    def exec_buy(self,context,data):
        open_orders = get_open_orders()
        r = self.robust(context,data)
        w = r * self.alloc #RAA adjusted weight
        
        if self.etf in data and self.etf not in open_orders and w > 0.0:
            order_target_percent(self.etf,w)
            self.bought.append(self.etf)
       
        log.info(str(type(self).__name__) + " bought " + str(self.etf.symbol) + " w=" + str(w))

        
    #sell etf
    def exec_sell(self,context,data):
        open_orders = get_open_orders()
        
        if self.etf in context.portfolio.positions and self.etf in self.bought and self.etf not in open_orders:
            order_target_percent(self.etf,0)
            self.bought.remove(self.etf)
        
    
    def before_trading_start(self,context,data): 
        pass
    
    
    def show_params(self,c):
        log.info("************************")
        log.info(" ETF params ")
        log.info("************************")
        log.info(" etf name: " + str(self.etf.symbol))
        log.info(" risk free asset: " + str(self.risk_free_asset.symbol))
        log.info(" market ma threshold: " + str(self.ma_threshold))
        log.info(" lookback: " + str(self.lookback))
        log.info(" pct alloc: " + str(self.alloc))
        log.info(" sell_offset: " + str(self.sell_offset))
        log.info(" buy_offset: " + str(self.buy_offset))

        
#Base class for Equity Strategies
class Equity_Strat(Strategy):
    
    def __init__(self, pct_alloc,risk_free_asset,ma_threshold, index, sell_offset, buy_offset):
        Strategy.__init__(self,pct_alloc,risk_free_asset,ma_threshold,sell_offset, buy_offset)
        self.index = index #set 

    
        
    def robust(self,context,data):
        r = 0.0     
        
        ex_return = self.get_excess_return(context,data)
        if ex_return > 0.0:
            r += 0.5
        if data[self.index].price > history(self.ma_threshold, '1d', 'price')[self.index].mean():  
            r += 0.5
        
        return r
    
    
    def get_excess_return(self,context,data):
        h = history( self.lookback,'1d','price')[[self.index,self.risk_free_asset]]
        h = h.resample('M',how='last')

        pct_change =h.iloc[[0,-1]].pct_change()[1:]
        pct_change = pct_change.dropna(axis=1)
        
        ex_return = pct_change[self.index]-pct_change[self.risk_free_asset]
        
        return ex_return[0]


#Momentum Strategy
# -- Sort equities on best 1 year performance
class Mom_Strat(Equity_Strat):
   
    def __init__(self, pct_alloc, risk_free_asset, ma_threshold, index, sell_offset, buy_offset):
        Equity_Strat.__init__(self,pct_alloc,risk_free_asset,ma_threshold,index,sell_offset, buy_offset)
        
        self.lower_market_cap = 4000e6     #lower market cap for candidate selection
        self.limit = 500                   #max number of stocks to select for fundamental query
        self.num_positions = 20           #max number of positions the strategy will select
        
        
    def update_universe_sell(self,context,data):
        self.sell_day = True
        
    
    def update_universe_buy(self,context,data):
        self.buy_day = True
        
   
    def before_trading_start(self,context,data): 
        
        if self.buy_day:
            self.get_candidates()
            if len(self.candidates) > 0:
                self.candidates_available = True
                update_universe(self.candidates)
                log.info(str(type(self).__name__) + " buy universe updated on: " + str(get_datetime()))  
            
        
    def get_candidates(self):
        df_fundamentals = get_fundamentals(
            query(
                fundamentals.valuation.shares_outstanding,
                fundamentals.valuation.market_cap
            )
            #.filter(fundamentals.company_reference.country_id == "USA")
            .filter(or_(fundamentals.company_reference.primary_exchange_id == "NAS", fundamentals.company_reference.primary_exchange_id == "NYSE"))
            .filter(fundamentals.valuation.market_cap != None)
            .filter(fundamentals.valuation.market_cap > self.lower_market_cap ) 
            .filter(fundamentals.valuation.shares_outstanding != None)
            #.filter(fundamentals.share_class_reference.is_depositary_receipt == False)
            .filter(fundamentals.share_class_reference.is_primary_share == True)
            .filter(fundamentals.company_reference.primary_exchange_id != "OTCPK")
            .order_by(fundamentals.valuation.market_cap.desc())
            .limit(self.limit)
        )
    
        self.candidates=df_fundamentals.columns
   
        
    def get_rank(self,context,data):
        
        #get TMOM
        h = history( self.lookback,'1d','price')[self.candidates]
        h = h.resample('M',how='last') #get monthly returns
        
        pct_change =h.iloc[[0,-1]].pct_change()[1:]
        pct_change = pct_change.dropna(axis=1)
        
        #convert dataframe to series for sorting. Then sort in descending order
        self.longs =  pct_change.squeeze().order(ascending=False)
        self.longs = self.longs[self.longs > 0.0]
        self.longs = self.longs[:self.num_positions]
    
 
    def exec_buy(self,context,data):
        if self.buy_day:
            self.buy_day = False
            r= self.robust(context,data)
        
            if r > 0.0:
        
                self.get_rank(context,data)
            
                l = len(self.longs)
                w = (r*self.alloc)/l if l > 0 else 0 #robust adjusted weight
        
                open_orders = get_open_orders()
                for s in self.longs.index:
                    if s in data and s not in open_orders:
                        order_target_percent(s,w)
                        if w > 0.0:
                            self.bought.append(s)
            
                log.info(str(type(self).__name__) + " longs list len " + str(len(self.longs)))
                log.info(str(type(self).__name__) + " bought list len: " + str(len(self.bought)))    
    
    
    def exec_sell(self,context,data):
        if self.sell_day:
            self.sell_day = False
            open_orders = get_open_orders()
        
            for s in context.portfolio.positions :
                if s not in open_orders :
                    if s in self.bought:
                        order_target_percent(s,0)
                        self.bought.remove(s)
                
            log.info(str(type(self).__name__) + " bought list len: " + str(len(self.bought)))
        
    
    def show_params(self,c):
        log.info("************************")
        log.info(" Momentum params ")
        log.info("************************")
        log.info(" lower market cap: " + str(self.lower_market_cap))
        log.info(" stock search limit: " + str(self.limit))
        log.info(" risk free asset: " + str(self.risk_free_asset.symbol))
        log.info(" number of positions: " + str(self.num_positions))
        log.info(" moving average threshold: " + str(self.ma_threshold))
        log.info(" pct alloc: " + str(self.alloc))
        log.info(" sell_offset: " + str(self.sell_offset))
        log.info(" buy_offset: " + str(self.buy_offset))
        log.info(" lookback: " + str(self.lookback))
       
        

class Value_Strat(Equity_Strat):
    
    def __init__(self, pct_alloc, risk_free_asset, ma_threshold, index, sell_offset, buy_offset):
        
        Equity_Strat.__init__(self,pct_alloc,risk_free_asset,ma_threshold,index,sell_offset, buy_offset)
    
        self.lower_market_cap = 4000e6
        self.limit = 500
        self.num_positions = 20
        self.rebalance_month = 6 
        self.longs = []
        
    
    def update_universe_sell(self,context,data):
         today = get_datetime()
         if today.month == self.rebalance_month:
             self.sell_day = True
        
    
    def update_universe_buy(self,context,data):
        today = get_datetime()
        if today.month == self.rebalance_month:
            self.buy_day = True
        
   
    def before_trading_start(self,context,data): 
        
        if self.sell_day:
            update_universe(self.bought)
            log.info(str(type(self).__name__) + " sell universe updated on: " + str(get_datetime()))  

        if self.buy_day:
            self.get_candidates()
            update_universe(self.candidates)
            log.info(str(type(self).__name__) + " buy universe updated on: " + str(get_datetime()))  
    
    
    def get_candidates(self):
        excluded_sectors = [103, 207]
        sector_code = fundamentals.asset_classification.morningstar_sector_code
        fundamental_df = get_fundamentals(
            query(
                sector_code,
                fundamentals.valuation.market_cap,
                fundamentals.valuation.enterprise_value,
                fundamentals.cash_flow_statement.capital_expenditure,
                fundamentals.operation_ratios.roic,
                fundamentals.income_statement.ebit,
                fundamentals.income_statement.ebitda,
                fundamentals.balance_sheet.total_assets,            
        )
        #.filter(fundamentals.company_reference.country_id == "USA")
        .filter(or_(fundamentals.company_reference.primary_exchange_id == "NAS", fundamentals.company_reference.primary_exchange_id == "NYSE"))
        .filter(fundamentals.share_class_reference.is_primary_share == True)
        .filter(fundamentals.company_reference.primary_exchange_id != "OTCPK")
        #.filter(fundamentals.share_class_reference.is_depositary_receipt == False)
        .filter(~sector_code.in_(excluded_sectors))
        .filter(fundamentals.valuation.market_cap > self.lower_market_cap ) 
        .filter(fundamentals.valuation_ratios.ev_to_ebitda > 0)
        .order_by(fundamentals.valuation_ratios.ev_to_ebitda.asc())
        .limit(self.limit)
        )
        
        self.candidates = fundamental_df
        
        
    def get_rank(self,context,data):
    
        #get MF components
        earnings_yield = self.candidates.ix['ebit'] / self.candidates.ix['enterprise_value']
        roic = self.candidates.ix['roic'].copy()
    
        #sort MF components
        earnings_yield.sort(ascending=False)
        roic.sort(ascending=False)

        #get ranks for components
        earnings_yield_ranking = pd.Series(range(len(earnings_yield)), index=earnings_yield.index)
        roic_ranking = pd.Series(range(len(roic)), index=roic.index) 
    
        #get combined ranking - ranks = earnings_yield_ranking + roic_ranking
        ranks = {}
        for s in self.candidates.columns:
            v_rank = earnings_yield_ranking.loc[s.sid]
            q_rank = roic_ranking.loc[s.sid]
            ranks[s] = v_rank+q_rank
    
        #convert to df and sort
        mf_ranks = pd.Series(ranks,name='Symbol')
        mf_ranks.sort(ascending = True)
        
        self.longs = mf_ranks.index.tolist()
        self.longs = self.longs[:self.num_positions]

     
    def exec_buy(self,context,data):
        if self.buy_day:
            self.buy_day = False
            
            r= self.robust(context,data)
            if r > 0.0:
                
                self.get_rank(context,data)
                
                l = len(self.longs)
                w = (r*self.alloc)/self.num_positions if l > 0 else 0
        
                open_orders = get_open_orders()
                for s in self.longs:
                    if s in data and s not in open_orders:
                        order_target_percent(s,w)
                        if w > 0.0:
                            self.bought.append(s)
                    
                log.info(str(type(self).__name__) + " longs list len " + str(len(self.longs)))
                log.info(str(type(self).__name__) + " bought " + str(len(self.bought)) + " stocks")
                log.info(str(type(self).__name__) + " buys completed on: " + str(get_datetime()))  

        
    def exec_sell(self,context,data):
         if self.sell_day:
            self.sell_day = False
            
            open_orders = get_open_orders()
        
            for s in context.portfolio.positions:
                if s in self.bought and s not in open_orders:
                    order_target_percent(s,0)
                    self.bought.remove(s)
        
            log.info(str(type(self).__name__) + " bought list len: " + str(len(self.bought)))
            log.info(str(type(self).__name__) + " sells completed on: " + str(get_datetime()))  

        
    def show_params(self,c):
        log.info("************************")
        log.info(" Value params ")
        log.info("************************")
        log.info(" lower market cap: " + str(self.lower_market_cap))
        log.info(" stock search limit: " + str(self.limit))
        log.info(" risk free asset: " + str(self.risk_free_asset.symbol))
        log.info(" number of positions: " + str(self.num_positions))
        log.info(" moving average threshold: " + str(self.ma_threshold))
        log.info(" pct alloc: " + str(self.alloc))
        log.info(" sell_offset: " + str(self.sell_offset))
        log.info(" buy_offset: " + str(self.buy_offset))        
        log.info(" rebalance month: " + str(self.rebalance_month))

        

def initialize(context):
    #set_commission(commission.PerTrade(cost=0.00))  
    #set_slippage(slippage.FixedSlippage(spread=0.00))
    #Common vars
    c = context
    c.lastYear = 2015
    c.lastMonth = 12
    c.index = symbol('SPY')
    c.risk_free_asset = symbol('SHY')
    c.ma_threshold = 200
    
    #register Strategies
    #IVY_4 allocations - no allocation to world equities
    # replace GSG with Gold to get longer backtest
    '''
    c.strategies = [ETF_Strat(0.25,c.risk_free_asset,c.ma_threshold,symbol('SPY'),1,2),
                    ETF_Strat(0.25,c.risk_free_asset,c.ma_threshold,symbol('IYR'),1,2),
                    ETF_Strat(0.25,c.risk_free_asset,c.ma_threshold,symbol('GLD'),1,2),
                    ETF_Strat(0.25,c.risk_free_asset,c.ma_threshold,symbol('IEF'),1,2)]
    '''
   
    #RAA Balanced - 40% allocation to equities
    '''
    c.strategies = [Value_Strat(0.2,c.risk_free_asset, c.ma_threshold, c.index,11,13),
                    Mom_Strat(0.2, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ,
                    ETF_Strat(0.2,c.risk_free_asset,c.ma_threshold,symbol('IYR'),1,2),
                    ETF_Strat(0.2,c.risk_free_asset,c.ma_threshold,symbol('GLD'),1,2),
                    ETF_Strat(0.2,c.risk_free_asset,c.ma_threshold,symbol('IEF'),1,2)]
    '''    
    
    #RAA Moderate Tilt - 60% allocation to equities
    '''
    c.strategies = [Value_Strat(0.3,c.risk_free_asset, c.ma_threshold, c.index,11,13),
                    Mom_Strat(0.3, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ,
                    ETF_Strat(0.1,c.risk_free_asset,c.ma_threshold,symbol('IYR'),1,2),
                    ETF_Strat(0.1,c.risk_free_asset,c.ma_threshold,symbol('GLD'),1,2),
                    ETF_Strat(0.2,c.risk_free_asset,c.ma_threshold,symbol('IEF'),1,2)]
    '''
        
    
    #RAA Aggressive Tilt - 80% allocation to equities
    c.strategies = [Value_Strat(0.4,c.risk_free_asset, c.ma_threshold, c.index,5,6),
                    Mom_Strat(0.4, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ,
                    ETF_Strat(0.05,c.risk_free_asset,c.ma_threshold,symbol('IYR'),1, 2),
                    ETF_Strat(0.05,c.risk_free_asset,c.ma_threshold,symbol('GLD'),1,2),
                    ETF_Strat(0.1,c.risk_free_asset,c.ma_threshold,symbol('IEF'),1,2)]
    
    #Mom only
    #c.strategies = [Mom_Strat(1.0, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ]
    #Val only
    #c.strategies = [Value_Strat(1.0,c.risk_free_asset, c.ma_threshold, c.index,5,6) ]
    #Mom/Val 50/50
    #c.strategies = [Value_Strat(0.5,c.risk_free_asset, c.ma_threshold, c.index,5,6),
    #                Mom_Strat(0.5, c.risk_free_asset, c.ma_threshold, c.index, 1, 2) ]
    
    #show parameters of all strategies
    show_params(c)
    
    for strat in c.strategies:
        #schedule universe update functions for Equity strategies
        # this turns on the flag to updat the universe, the universe is updated the next day before the market starts
        #trading (in the before_trading_start method)
        if type(strat).__name__ != 'ETF_StratProxy':
            schedule_function(strat.update_universe_sell,date_rules.month_start(days_offset=strat.sell_offset),
                              time_rules.market_close(minutes=30)) 
            schedule_function(strat.update_universe_buy,date_rules.month_start(days_offset=strat.buy_offset),
                              time_rules.market_close(minutes=30)) 
        
        #schedule buy and sell functions
        #note these functions are scheduled to run one day after the update universe functions
        #which means they actually run on the same day the universe is updated
        schedule_function(strat.exec_sell,date_rules.month_start(days_offset=(strat.sell_offset+1)),
            time_rules.market_close(minutes=30)) 
        schedule_function(strat.exec_buy,date_rules.month_start(days_offset=(strat.buy_offset+1)),
            time_rules.market_close(minutes=30)) 
    
    #as a last step check final state of portfolio. In particular check total value of unfullfilled orders
    schedule_function(check_positions,date_rules.month_end(days_offset=0),
                      time_rules.market_open(minutes=240))
    #turn on this function to eliminate trades that will go unfulfilled (i.e. will not be sellable).
    #schedule_function(handle_unfulfilled_trades,date_rules.every_day(),
    #                  time_rules.market_open(minutes=15))

#Sell any securities before they become unfulfillable   
def handle_unfulfilled_trades(context,data):
    open_orders = get_open_orders()
    today = get_datetime()
    
    for s in open_orders:
        if s in context.portfolio.positions:
            if (today + timedelta(days=10)) > s.end_date:
                order_target_percent(s,0.0)
    
def check_positions(context,data):

    today = get_datetime()
    
    if today.year == context.lastYear and today.month == context.lastMonth:
        #log current account
        log.info(" account leverage: " + str(context.account.leverage))
        log.info(" portfolio value: " + str(context.portfolio.portfolio_value))
        log.info(" portfolio cash: " + str(context.portfolio.cash))
        

        #check number and value of unfullfilled trades
        # unfullfilled trades are often the cause of a strategy appearing to use leverage when in reality it would not.
        # often the cause of an unfullfilled trade is a company merging or getting bought out
        
        open_orders = get_open_orders()
        
        c = 0
        tot_val = 0
        for s in context.portfolio.positions:
            if s in open_orders and s.end_date.year < context.lastYear and s.end_date.month < context.lastMonth:
                shrs = context.portfolio.positions[s].amount
                #cost = context.portfolio.positions[s].cost_basis
                current = data[s].price
                c += 1
                tot_val +=  (shrs * current)
        
        log.info(" len open orders: " + str(len(open_orders)))
        log.info(" number of unfulfilled trades: " + str(c) + " value: " + str(tot_val))
    

def show_params(c):
    
    log.info("************************")
    log.info(" Algo params ")
    log.info("************************")
    
    for strat in c.strategies:
        strat.show_params(c)
   
    
def before_trading_start(context,data): 
    for strat in context.strategies:
        strat.before_trading_start(context, data)

            
def handle_data(context, data):

    record(leverage = context.account.leverage)
    
There was a runtime error.

So much to learn. Im new to Python. Thank you so much.

Mark. Is there any chance you have version Q2 for this. Im spending 2 days trying to get it to work with Q2 but Ii could. This is so good framework for me to work with. But i guess I'm too new to Quantopian as well as Python. So if you have time to port i would really appreciated it.
Thanks

  • could not i meant

Can anyone fix this?

There was a runtime error.
AttributeError: 'Mom_Strat' object has no attribute 'candidates'
... USER ALGORITHM:260, in exec_rank
h = history( self.lookback,'1d','price')[self.candidates]