Back to Community
Robotus: Buying and Selling Based on Market Sentiment

Hi all,

This is my first algo, let call it Robotus. It buys and sells SPY depending on market sentiment (recession, VIX, etc).
Drawdown is acceptable howerver there is a bias because history takes some time to collect bars and benchmark starts from the first day.

  • There is no possibility to backtest before 01/01/2008, I have no idea what is wrong with the code.
  • Reducing drawdown could be achieve with buying and selling per slice of 25% of the protfolio value using the order_percent function set to 0.25 as long signal is true but the value of the positions open becomes over portfolio (it must be the normal behavior so there is a need to implement a limit to avoid this).

Advise and comment are welcome,

Florent

Clone Algorithm
500
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import numpy as np
import talib

@batch_transform(window_length=201, refresh_period=0)
def _history(data):
    return data

#Rename columns
###############

def rename_col(df):
    
    df = df.rename(columns={'Value': 'price'})
    df = df.fillna(method='ffill')
    # Correct look-ahead bias in mapping data to times   
    df = df.tshift(1, freq='b')
    log.info(' \n %s ' % df.head())
    return df

def initialize(context):
    
    #Fetch recession data
    #####################
    
    fetch_csv('http://www.quandl.com/api/v1/datasets/FRED/RECPROUSM156N.csv?trim_start=2002-01-01',
              date_column='Date',
              symbol='recession',
              post_func=rename_col,
              date_format='%Y-%m-%d')

    #Fetch VIX data
    ###############
    
    url ='http://www.quandl.com/api/v1/datasets/YAHOO/INDEX_VIX.csv?request_source=python&request_version=2&sort_order=asc&trim_start=2002-01-01'
    fetch_csv(url, 
              symbol='VIX', 
              date_column='Date')
    
    #Initialize SPY
    ###############    
    
    set_symbol_lookup_date('2014-3-5')
    context.SPY = symbol('SPY')
    context.HYG = symbol('HYG')

    #Initialize variable
    ###############
    
    context.position = 0
    context.recession = False
    context.short = False
    context.order = None
    context.market = 0
                
def handle_data(context, data):

    #Record account info
    ####################
    
    record_account_info(context, data)
    
    #Compute data
    #############
    
    price_history = history(bar_count=41, 
                                frequency='1d', 
                                field='price')
    
    histSPY = price_history[context.SPY]
    histHYG = price_history[context.HYG]

    ROCP30SPY = talib.ROCP(histSPY, 30)*100  
    ROCP20SPY = talib.ROCP(histSPY, 20)*100
    ROCP10SPY = talib.ROCP(histSPY, 10)*100   
    ROCP5SPY = talib.ROCP(histSPY, 5)*100 
    
    ROCP5HYG = talib.ROCP(histHYG, 5)*100
    ROCP2HYG = talib.ROCP(histHYG, 2)*100
    ROCP10HYG = talib.ROCP(histHYG, 10)*100
    ROCP20HYG = talib.ROCP(histHYG, 20)*100
    ROCP1HYG = talib.ROCP(histHYG, 1)*100
    
    #Compute CSV data
    #################
    
    hist = _history(data)
    
    # Return if there is not enough data yet 
    if hist is None:
        return
    
    # Use the close VIX value because the price is also the close
    rec = hist['price']['recession'].values
    REC = hist['price']['recession'].values[-1]
    vix = hist['Close']['VIX'].values
    VIX = hist['Close']['VIX'].values[-1]
    
    ROCP80VIX = talib.ROCP(vix, 80)*100 
    ROCP40VIX = talib.ROCP(vix, 40)*100 
    ROCP30VIX = talib.ROCP(vix, 30)*100    
    ROCP20VIX = talib.ROCP(vix, 20)*100    
    ROCP10VIX = talib.ROCP(vix, 10)*100    
    ROCP5VIX = talib.ROCP(vix, 5)*100    
    ROCP2VIX = talib.ROCP(vix, 2)*100   
    ROCP1VIX = talib.ROCP(vix, 1)*100
    
    MA200VIX = talib.MA(vix, 200)
    MA110VIX = talib.MA(vix, 110)
    MA100VIX = talib.MA(vix, 100)
    EMA40VIX = talib.EMA(vix, 40)
    WMA32VIX = talib.WMA(vix, 32)

    # If there is a recession short the market
    ##########################################
    
    if rec[-1] > 3 : 
        context.recession = True
    else :     
        context.recession = False
        
    if context.recession == True :
        
        if context.short == False :    
            order_target_percent(context.SPY, -1)
            context.position = -20
            context.short = True
    
        elif context.short == True :
            
            if MA100VIX[-1] < MA110VIX[-1] > 30 :
                order_target_percent(context.SPY, 1)
                context.short = None
                context.position = 20
    
       
    # If there is no recession
    ##########################

    if context.recession == False and data :
        
        # BUY signals
        #############
        
        #1 Buy when VIX is over it's MA200 and fall !
        if EMA40VIX[-1] > MA200VIX[-1] and WMA32VIX[-1] < EMA40VIX[-1] :
            order_target_percent(context.SPY, 1)
            context.position = 25
        
        #2 Buy when VIX rise 100% in the last 80 days !
        elif ROCP80VIX[-1] > 100 :
            order_target_percent(context.SPY, 1)
            context.position = 5
        
        #3 Buy when VIX rise 100% in the last 40 days !
        elif ROCP40VIX[-1] > 100 and ROCP2VIX[-1] < 0:
            order_target_percent(context.SPY, 1)
            context.position = 10
            
        #4 Buy when HYG signal a recovery during bear market
        elif ROCP10HYG[-1] > 3 and ROCP20SPY[-1] < -5 and ROCP2HYG[-1] > 0.5:
            order_target_percent(context.SPY, 1)
            context.position = 15
        
        #5 Buy when HYG and SPY progress
        elif ROCP5SPY[-1] > 2.4 and ROCP5HYG[-1] > 2.4:
            order_target_percent(context.SPY, 1)
            context.position = 20
        
        # SELL signals
        ##############
        
        #1 Sell when VIX fall 
        if ROCP20VIX[-1] < -40 :
            order_target_percent(context.SPY, -1)
            context.position = -5
              
        #2 Sell when HYG signal a correction during bull market
        elif ROCP20SPY[-1] > 4.5 and ROCP10HYG[-1] < -0.5 : # and ROCP5HYG[-1] < -1 :
            order_target_percent(context.SPY, -1)
            context.position = -10
            
    record(    #ROCP1VIX=ROCP1VIX[-1],
               #ROCP2VIX=ROCP2VIX[-1],
               #ROCP40VIX=ROCP40VIX[-1],
               #EMA40VIX=EMA40VIX[-1],
               #MA200VIX=MA200VIX[-1],
               #ROCP20VIX=ROCP20VIX[-1],
               #ROCP20SPY=ROCP20SPY[-1],
               #ROCP10HYG=ROCP10HYG[-1],
               #ROCP5HYG=ROCP5HYG[-1],
               #ROCP2HYG=ROCP2HYG[-1],
               #market=context.market,
               pos=context.position)
    
def record_account_info(context, data):
    
    P = context.portfolio
    #market_value = sum(data[i].price * abs(P.positions[i].amount) for i in data)
    record(#leverage=market_value / P.portfolio_value,
           positions_value=P.positions_value,
           portfolio_value=P.portfolio_value)  

            
There was a runtime error.
21 responses

I believe you can't backtest before 2008 due to HYG not beginning trading until 4/4/2007.

Info: http://www.ishares.com/us/products/239565/ishares-iboxx-high-yield-corporate-bond-etf

Correct but Quantopian has data begining from mid- 2007

Let me clone and take a look

Florent,

Great try...but...

the 'recession' data you used from 'Smoothed US Recession Probabilities' has a built-in two-month lag. Because one of the series included in the DFMS model is real manufacturing and trade sales, produced by the U.S. Census Bureau. This series is available only after a two month lag. So although the data updated with the most recent data of October 2014, it is still two-month lag because of the one indicator's built-in lagging feature (In your data, you used 'July 2014' as the most recent one...).

With the imperfect set of data, how can we apply your algo to a real-money trading in a real-time market?

I'm not attacking you but want to point out that one of data you used has a little problem to use in a real-time market.
My advice is that if you really want to use some sort of 'recession'-related indicators or economic indicators, you better use other data (e.g. TED Spread,.. etc.) that has a completely up-to-dated or something that you create with all other data.

But it's completely OK to discard my thought if this is still OK for you.
Thanks.

Hi Kyu,

Thank's for your comment and suggestion.

Two-months lag for one of the sub-components of the indicator does not gives false signal as the others three are up-to-date. The Smoothed Recession Probability is calculated from a dynamic-factor Markov-switching (DFMS) model applied to four monthly coincident variables: non-farm payroll employment, the index of industrial production, real personal income excluding transfer payments, and real manufacturing and trade sales.

Also there is 2-months lag for everyone in this world (or almost) and for this reason I think the information is still relevant even after a delay. For example it rise to 30% end of Dec. 2007 some days before the market enter risk-off mode and 9 months before it enter in crash mode.

I have a look at the TED Spread which also give good signal begining 15-august 2007. Good recommandation. In my opinion these two indicators can be used to reduce exposure to the market or even go short.

Do you think there is a posibility the benchmark starts after the first order is given ? the comparison would be more accurate.
Thank's.

Hey Kyu,

In this backtest I changed the indicator that uses the 'recession' data to base the trading decision off of the value from four months ago.

Does this fix the problem you talked about, or is the lagged data used somewhere else that I haven't found?

-Jeff

Clone Algorithm
81
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import numpy as np
import talib

@batch_transform(window_length=201, refresh_period=0)
def _history(data):
    return data

def rename_col(df):
    
    df = df.rename(columns={'Value': 'price'})
    df = df.fillna(method='ffill')
    # Correct look-ahead bias in mapping data to times   
    df = df.tshift(1, freq='b')
    log.info(' \n %s ' % df.head())
    return df

def initialize(context):
    
    #Fetch recession data
    #####################
    
    fetch_csv('http://www.quandl.com/api/v1/datasets/FRED/RECPROUSM156N.csv?trim_start=2002-01-01',
              date_column='Date',
              symbol='recession',
              post_func=rename_col,
              date_format='%Y-%m-%d')

    #Fetch VIX data
    ###############
    
    url ='http://www.quandl.com/api/v1/datasets/YAHOO/INDEX_VIX.csv?request_source=python&request_version=2&sort_order=asc&trim_start=2002-01-01'
    fetch_csv(url, 
              symbol='VIX', 
              date_column='Date')
    
    #Initialize SPY
    ###############    
    
    set_symbol_lookup_date('2014-3-5')
    context.SPY = symbol('SPY')
    context.HYG = symbol('HYG')

    #Initialize variable
    ###############
    
    context.position = 0
    context.recession = False
    context.short = False
    context.order = None
    context.market = 0
                
def handle_data(context, data):

    #Record account info
    ####################
    
    record_account_info(context, data)
    
    #Compute data
    #############
    
    price_history = history(bar_count=41, 
                                frequency='1d', 
                                field='price')
    
    histSPY = price_history[context.SPY]
    histHYG = price_history[context.HYG]

    ROCP30SPY = talib.ROCP(histSPY, 30)*100  
    ROCP20SPY = talib.ROCP(histSPY, 20)*100
    ROCP10SPY = talib.ROCP(histSPY, 10)*100   
    ROCP5SPY = talib.ROCP(histSPY, 5)*100 
    
    ROCP5HYG = talib.ROCP(histHYG, 5)*100
    ROCP2HYG = talib.ROCP(histHYG, 2)*100
    ROCP10HYG = talib.ROCP(histHYG, 10)*100
    ROCP20HYG = talib.ROCP(histHYG, 20)*100
    ROCP1HYG = talib.ROCP(histHYG, 1)*100
    
    #Compute CSV data
    #################
    
    hist = _history(data)
    
    # Return if there is not enough data yet 
    if hist is None:
        return
    
    # Use the close VIX value because the price is also the close
    rec = hist['price']['recession'].values
    REC = hist['price']['recession'].values[-1]
    vix = hist['Close']['VIX'].values
    VIX = hist['Close']['VIX'].values[-1]
    
    ROCP80VIX = talib.ROCP(vix, 80)*100 
    ROCP40VIX = talib.ROCP(vix, 40)*100 
    ROCP30VIX = talib.ROCP(vix, 30)*100    
    ROCP20VIX = talib.ROCP(vix, 20)*100    
    ROCP10VIX = talib.ROCP(vix, 10)*100    
    ROCP5VIX = talib.ROCP(vix, 5)*100    
    ROCP2VIX = talib.ROCP(vix, 2)*100   
    ROCP1VIX = talib.ROCP(vix, 1)*100
    
    MA200VIX = talib.MA(vix, 200)
    MA110VIX = talib.MA(vix, 110)
    MA100VIX = talib.MA(vix, 100)
    EMA40VIX = talib.EMA(vix, 40)
    WMA32VIX = talib.WMA(vix, 32)

    # If there is a recession short the market
    ##########################################
    
    if rec[-4] > 3 : 
        context.recession = True
    else :     
        context.recession = False
        
    if context.recession == True :
        
        if context.short == False :    
            order_target_percent(context.SPY, -1)
            context.short = True
    
        elif context.short == True :
            
            if MA100VIX[-1] < MA110VIX[-1] > 30 :
                order_target_percent(context.SPY, 1)
                context.short = None    
       
    # If there is no recession
    ##########################

    if context.recession == False and data :
        
        # BUY signals
        #############
        
        #1 Buy when VIX is over it's MA200 and fall !
        if EMA40VIX[-1] > MA200VIX[-1] and WMA32VIX[-1] < EMA40VIX[-1] :
            order_target_percent(context.SPY, 1)
        
        #2 Buy when VIX rise 100% in the last 80 days !
        elif ROCP80VIX[-1] > 100 :
            order_target_percent(context.SPY, 1)
        
        #3 Buy when VIX rise 100% in the last 40 days !
        elif ROCP40VIX[-1] > 100 and ROCP2VIX[-1] < 0:
            order_target_percent(context.SPY, 1)
            
        #4 Buy when HYG signal a recovery during bear market
        elif ROCP10HYG[-1] > 3 and ROCP20SPY[-1] < -5 and ROCP2HYG[-1] > 0.5:
            order_target_percent(context.SPY, 1)
        
        #5 Buy when HYG and SPY progress
        elif ROCP5SPY[-1] > 2.4 and ROCP5HYG[-1] > 2.4:
            order_target_percent(context.SPY, 1)
        
        # SELL signals
        ##############
        
        #1 Sell when VIX fall 
        if ROCP20VIX[-1] < -40 :
            order_target_percent(context.SPY, -1)
              
        #2 Sell when HYG signal a correction during bull market
        elif ROCP20SPY[-1] > 4.5 and ROCP10HYG[-1] < -0.5 : # and ROCP5HYG[-1] < -1 :
            order_target_percent(context.SPY, -1)            
    
def record_account_info(context, data):
    
    P = context.portfolio
    #market_value = sum(data[i].price * abs(P.positions[i].amount) for i in data)
    record(#leverage=market_value / P.portfolio_value,
           positions_value=P.positions_value,
           portfolio_value=P.portfolio_value)  

            
There was a runtime error.

Hi Paul, Kiu,

Here is a backtest with the feature 'recession detection' deactivated. Sell signals are efficient and still give excellent result. As you can see there is +9% in favor of the benchmark (SPY) at the begining of the backtest but Robotus v0.1 skyrocket soon after !

Some cleaning in the code and I will try to update the algo and release a Robotus v0.2 with more sell signals to reduce drowdown.

Florent

Clone Algorithm
500
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import numpy as np
import talib

@batch_transform(window_length=201, refresh_period=0)
def _history(data):
    return data

#Rename columns
###############

def rename_col(df):
    
    df = df.rename(columns={'Value': 'price'})
    df = df.rename(columns={'Index Put Volume': 'price'})
    df = df.fillna(method='ffill')
    # Correct look-ahead bias in mapping data to times   
    df = df.tshift(1, freq='b')
    log.info(' \n %s ' % df.head())
    return df

def initialize(context):
    
    #Fetch data
    ###########
    
    fetch_csv('http://www.quandl.com/api/v1/datasets/FRED/RECPROUSM156N.csv?trim_start=2002-01-01',
              date_column='Date',
              symbol='recession',
              post_func=rename_col,
              date_format='%Y-%m-%d')
    
    fetch_csv('http://www.quandl.com/api/v1/datasets/CBOE/INDEX_PC.csv?trim_start=2002-01-01',
              date_column='Date',
              symbol='PUT',
              post_func=rename_col,
              date_format='%Y-%m-%d')   
    
    url ='http://www.quandl.com/api/v1/datasets/YAHOO/INDEX_VIX.csv?request_source=python&request_version=2&sort_order=asc&trim_start=2002-01-01'
    fetch_csv(url, 
              symbol='VIX', 
              date_column='Date')
    
    #Initialize SPY
    ###############    
    
    set_symbol_lookup_date('2014-3-5')
    context.SPY = symbol('SPY')
    context.HYG = symbol('HYG')

    #Initialize variable
    ###############
    
    context.position = 0
    context.recession = False
    context.short = False
                
def handle_data(context, data):

    #Record account info
    ####################
    
    record_account_info(context, data)
    
    #Compute data
    #############
    
    price_history = history(bar_count=41, 
                                frequency='1d', 
                                field='price')
    
    histSPY = price_history[context.SPY]
    histHYG = price_history[context.HYG]

    ROCP30SPY = talib.ROCP(histSPY, 30)*100  
    ROCP20SPY = talib.ROCP(histSPY, 20)*100
    ROCP10SPY = talib.ROCP(histSPY, 10)*100   
    ROCP5SPY = talib.ROCP(histSPY, 5)*100 
    
    ROCP5HYG = talib.ROCP(histHYG, 5)*100
    ROCP2HYG = talib.ROCP(histHYG, 2)*100
    ROCP10HYG = talib.ROCP(histHYG, 10)*100
    ROCP20HYG = talib.ROCP(histHYG, 20)*100
    ROCP1HYG = talib.ROCP(histHYG, 1)*100
    
    #Compute CSV data
    #################
    
    hist = _history(data)
    
    # Return if there is not enough data yet 
    if hist is None:
        return
    
    # Use the close VIX value because the price is also the close
    rec = hist['price']['recession'].values
    vix = hist['Close']['VIX'].values
    put = hist['Close']['PUT'].values
    
    ROCP80VIX = talib.ROCP(vix, 80)*100 
    ROCP40VIX = talib.ROCP(vix, 40)*100 
    ROCP30VIX = talib.ROCP(vix, 30)*100    
    ROCP20VIX = talib.ROCP(vix, 20)*100    
    ROCP10VIX = talib.ROCP(vix, 10)*100    
    ROCP5VIX = talib.ROCP(vix, 5)*100    
    ROCP2VIX = talib.ROCP(vix, 2)*100   
    ROCP1VIX = talib.ROCP(vix, 1)*100
    
    MA200VIX = talib.MA(vix, 200)
    MA110VIX = talib.MA(vix, 110)
    MA100VIX = talib.MA(vix, 100)
    EMA40VIX = talib.EMA(vix, 40)
    WMA32VIX = talib.WMA(vix, 32)
    #MA20PUT = talib.ROCP(put, 1)
    #MA10PUT = talib.ROCP(put, 1)
    
    # If there is a recession short the market
    ##########################################
    
    if rec[-1] > 200 : 
        context.recession = True
    else :     
        context.recession = False
        
    if context.recession == True :
        
        if context.short == False :    
            order_target_percent(context.SPY, -1)
            context.position = -20
            context.short = True
    
        elif context.short == True :
            
            if MA100VIX[-1] < MA110VIX[-1] > 30 :
                order_target_percent(context.SPY, 1)
                context.short = None
                context.position = 20
    
       
    # If there is no recession
    ##########################

    if context.recession == False :
        
        # BUY signals
        #############
        
        #1 Buy when VIX is over it's MA200 and fall !
        if EMA40VIX[-1] > MA200VIX[-1] and WMA32VIX[-1] < EMA40VIX[-1] :
            order_target_percent(context.SPY, 1)
            context.position = 25
        
        #2 Buy when VIX rise 100% in the last 80 days !
        elif ROCP80VIX[-1] > 100 :
            order_target_percent(context.SPY, 1)
            context.position = 5
        
        #3 Buy when VIX rise 100% in the last 40 days !
        elif ROCP40VIX[-1] > 100 and ROCP2VIX[-1] < 0:
            order_target_percent(context.SPY, 1)
            context.position = 10
            
        #4 Buy when HYG signal a recovery during bear market
        elif ROCP10HYG[-1] > 3 and ROCP20SPY[-1] < -5 and ROCP2HYG[-1] > 0.5:
            order_target_percent(context.SPY, 1)
            context.position = 15
        
        #5 Buy when HYG and SPY progress
        elif ROCP5SPY[-1] > 2.4 and ROCP5HYG[-1] > 2.4:
            order_target_percent(context.SPY, 1)
            context.position = 20
        
        # SELL signals
        ##############
        
        #1 Sell when VIX fall 
        if ROCP20VIX[-1] < -40 :
            order_target_percent(context.SPY, -1)
            context.position = -5
              
        #2 Sell when HYG does not follow SPY
        elif ROCP20SPY[-1] > 4.3 and ROCP10HYG[-1] < -0.5 :
            order_target_percent(context.SPY, -1)
            context.position = -10
            
    record(    #ROCP1VIX=ROCP1VIX[-1],
               #ROCP2VIX=ROCP2VIX[-1],
               #ROCP40VIX=ROCP40VIX[-1],
               #EMA40VIX=EMA40VIX[-1],
               #MA200VIX=MA200VIX[-1],
               #ROCP20VIX=ROCP20VIX[-1],
               #ROCP20SPY=ROCP20SPY[-1],
               #ROCP10HYG=ROCP10HYG[-1],
               #ROCP5HYG=ROCP5HYG[-1],
               #ROCP2HYG=ROCP2HYG[-1],
               #market=context.market,
               #put10=MA10PUT[-1],
               #put20=MA20PUT[-1],
               pos=context.position)
    
def record_account_info(context, data):
    
    P = context.portfolio
    #market_value = sum(data[i].price * abs(P.positions[i].amount) for i in data)
    record(#leverage=market_value / P.portfolio_value,
           positions_value=P.positions_value,
           portfolio_value=P.portfolio_value)  

            
There was a runtime error.

hi Florent, why would you turn off the recession detection?

Peter

Hi Peter, it was to verify how other signals were doing.

Hi florent
your algorithm seems quite good, but it produces very few trade signals (one or two per year).
Is there any way to add more signals by changing the algorithm parameters for example?
By the way, can we force the algorithm to start trading the first day (using past data), and not waiting untel having enough data?

Thanks
Idiane

Hi Florent,

It would be great if you could give us a little explanation about the rationale behind that Robotus. It seems to be a very high potential model, but yet, I do not completely understand the metrics for buying and selling; and why using HYG as one of the traded securities along with SPY?

Thanks

Vinicius

Hi Florent,

Thanks for sharing this algo. One observation when looking at the code is that you may have "curve-fitted." There appears to be a large number of "tunable" parameters which is a red flag in my book.

I made some quick changes by simply rounding some of your threshold values to the nearest whole number (e.g 4.3 to 4) and the results while still not bad are very different from your last post.

Clone Algorithm
22
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import numpy as np
import talib

@batch_transform(window_length=201, refresh_period=0)
def _history(data):
    return data

#Rename columns
###############

def rename_col(df):
    
    df = df.rename(columns={'Value': 'price'})
    df = df.rename(columns={'Index Put Volume': 'price'})
    df = df.fillna(method='ffill')
    # Correct look-ahead bias in mapping data to times   
    df = df.tshift(1, freq='b')
    log.info(' \n %s ' % df.head())
    return df

def initialize(context):
    
    #Fetch data
    ###########
    
    fetch_csv('http://www.quandl.com/api/v1/datasets/FRED/RECPROUSM156N.csv?trim_start=2002-01-01',
              date_column='Date',
              symbol='recession',
              post_func=rename_col,
              date_format='%Y-%m-%d')
    
    fetch_csv('http://www.quandl.com/api/v1/datasets/CBOE/INDEX_PC.csv?trim_start=2002-01-01',
              date_column='Date',
              symbol='PUT',
              post_func=rename_col,
              date_format='%Y-%m-%d')   
    
    url ='http://www.quandl.com/api/v1/datasets/YAHOO/INDEX_VIX.csv?request_source=python&request_version=2&sort_order=asc&trim_start=2002-01-01'
    fetch_csv(url, 
              symbol='VIX', 
              date_column='Date')
    
    #Initialize SPY
    ###############    
    
    set_symbol_lookup_date('2014-3-5')
    context.SPY = symbol('SPY')
    context.HYG = symbol('HYG')

    #Initialize variable
    ###############
    
    context.position = 0
    context.recession = False
    context.short = False
                
def handle_data(context, data):

    #Record account info
    ####################
    
    record_account_info(context, data)
    
    #Compute data
    #############
    
    price_history = history(bar_count=41, 
                                frequency='1d', 
                                field='price')
    
    histSPY = price_history[context.SPY]
    histHYG = price_history[context.HYG]

    ROCP30SPY = talib.ROCP(histSPY, 30)*100  
    ROCP20SPY = talib.ROCP(histSPY, 20)*100
    ROCP10SPY = talib.ROCP(histSPY, 10)*100   
    ROCP5SPY = talib.ROCP(histSPY, 5)*100 
    
    ROCP5HYG = talib.ROCP(histHYG, 5)*100
    ROCP2HYG = talib.ROCP(histHYG, 2)*100
    ROCP10HYG = talib.ROCP(histHYG, 10)*100
    ROCP20HYG = talib.ROCP(histHYG, 20)*100
    ROCP1HYG = talib.ROCP(histHYG, 1)*100
    
    #Compute CSV data
    #################
    
    hist = _history(data)
    
    # Return if there is not enough data yet 
    if hist is None:
        return
    
    # Use the close VIX value because the price is also the close
    rec = hist['price']['recession'].values
    vix = hist['Close']['VIX'].values
    put = hist['Close']['PUT'].values
    
    ROCP80VIX = talib.ROCP(vix, 80)*100 
    ROCP40VIX = talib.ROCP(vix, 40)*100 
    ROCP30VIX = talib.ROCP(vix, 30)*100    
    ROCP20VIX = talib.ROCP(vix, 20)*100    
    ROCP10VIX = talib.ROCP(vix, 10)*100    
    ROCP5VIX = talib.ROCP(vix, 5)*100    
    ROCP2VIX = talib.ROCP(vix, 2)*100   
    ROCP1VIX = talib.ROCP(vix, 1)*100
    
    MA200VIX = talib.MA(vix, 200)
    MA110VIX = talib.MA(vix, 110)
    MA100VIX = talib.MA(vix, 100)
    EMA40VIX = talib.EMA(vix, 40)
    WMA32VIX = talib.WMA(vix, 32)
    #MA20PUT = talib.ROCP(put, 1)
    #MA10PUT = talib.ROCP(put, 1)
    
    # If there is a recession short the market
    ##########################################
    
    if rec[-1] > 200 : 
        context.recession = True
    else :     
        context.recession = False
        
    if context.recession == True :
        
        if context.short == False :    
            order_target_percent(context.SPY, -1)
            context.position = -20
            context.short = True
    
        elif context.short == True :
            
            if MA100VIX[-1] < MA110VIX[-1] > 30 :
                order_target_percent(context.SPY, 1)
                context.short = None
                context.position = 20
    
       
    # If there is no recession
    ##########################

    if context.recession == False :
        
        # BUY signals
        #############
        
        #1 Buy when VIX is over it's MA200 and fall !
        if EMA40VIX[-1] > MA200VIX[-1] and WMA32VIX[-1] < EMA40VIX[-1] :
            order_target_percent(context.SPY, 1)
            context.position = 25
        
        #2 Buy when VIX rise 100% in the last 80 days !
        elif ROCP80VIX[-1] > 100 :
            order_target_percent(context.SPY, 1)
            context.position = 5
        
        #3 Buy when VIX rise 100% in the last 40 days !
        elif ROCP40VIX[-1] > 100 and ROCP2VIX[-1] < 0:
            order_target_percent(context.SPY, 1)
            context.position = 10
            
        #4 Buy when HYG signal a recovery during bear market
        elif ROCP10HYG[-1] > 3 and ROCP20SPY[-1] < -5 and ROCP2HYG[-1] > 1:
            order_target_percent(context.SPY, 1)
            context.position = 15
        
        #5 Buy when HYG and SPY progress
        elif ROCP5SPY[-1] > 2 and ROCP5HYG[-1] > 2:
            order_target_percent(context.SPY, 1)
            context.position = 20
        
        # SELL signals
        ##############
        
        #1 Sell when VIX fall 
        if ROCP20VIX[-1] < -40 :
            order_target_percent(context.SPY, -1)
            context.position = -5
              
        #2 Sell when HYG does not follow SPY
        elif ROCP20SPY[-1] > 4 and ROCP10HYG[-1] < -1 :
            order_target_percent(context.SPY, -1)
            context.position = -10
            
    record(    #ROCP1VIX=ROCP1VIX[-1],
               #ROCP2VIX=ROCP2VIX[-1],
               #ROCP40VIX=ROCP40VIX[-1],
               #EMA40VIX=EMA40VIX[-1],
               #MA200VIX=MA200VIX[-1],
               #ROCP20VIX=ROCP20VIX[-1],
               #ROCP20SPY=ROCP20SPY[-1],
               #ROCP10HYG=ROCP10HYG[-1],
               #ROCP5HYG=ROCP5HYG[-1],
               #ROCP2HYG=ROCP2HYG[-1],
               #market=context.market,
               #put10=MA10PUT[-1],
               #put20=MA20PUT[-1],
               pos=context.position)
    
def record_account_info(context, data):
    
    P = context.portfolio
    #market_value = sum(data[i].price * abs(P.positions[i].amount) for i in data)
    record(#leverage=market_value / P.portfolio_value,
           positions_value=P.positions_value,
           portfolio_value=P.portfolio_value)  

            
There was a runtime error.

@Vinicius, Hi, HYG is sometime leading the markets, it is the reason why I use it.
@Rakesh, Hi, it is true there are lot's of tunnable paremeters in the algo and instead of "curve-fitted" I would say there values are optimized :-) Then they are linked to several scenarios for buy & sell orders so I think it is not really a problem.

@Rakesh have try to modify the sell condition (detection of HYG divergence with SPX) by using moving average crossing instead of rate of change variation but I cannot obtain similar result. The parameters rate of change you have tunned are very sensitive and step for modification must be 0.1

@Florent, I have a question. When I run backtest on your algorithm, the last day transaction won't be shown until 1 day later. For example, if I test it with end date as today, I see the last date of transaction as 2014-11-25. However, if I run the test with end date as 2014-11-25, the latest transaction won't be shown unless I change the end date to 2014-11-26 or later. Why is it behaving like that?

@Daniel, I also had strange behaviors with start & end date using this code. It must be updated by collecting value of fetcher in an array with the method proposed by Seong Lee and used by Richard Diehl here https://www.quantopian.com/posts/method-to-get-historic-values-from-fetcher-data

@Florent, another question for you. Why is HYG being an important factor in your algorithm?

Very cool algorithm! As you said in your original post "there is a bias because history takes some time to collect bars and benchmark starts from the first day"

I added a line to remove that bias, backtest attached.

Clone Algorithm
64
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import numpy as np
import talib

@batch_transform(window_length=201, refresh_period=0)
def _history(data):
    return data

#Rename columns
###############

def rename_col(df):
    
    df = df.rename(columns={'Value': 'price'})
    df = df.fillna(method='ffill')
    # Correct look-ahead bias in mapping data to times   
    df = df.tshift(1, freq='b')
    log.info(' \n %s ' % df.head())
    return df

def initialize(context):
    
    #Fetch recession data
    #####################
    
    fetch_csv('http://www.quandl.com/api/v1/datasets/FRED/RECPROUSM156N.csv?trim_start=2002-01-01',
              date_column='Date',
              symbol='recession',
              post_func=rename_col,
              date_format='%Y-%m-%d')

    #Fetch VIX data
    ###############
    
    url ='http://www.quandl.com/api/v1/datasets/YAHOO/INDEX_VIX.csv?request_source=python&request_version=2&sort_order=asc&trim_start=2002-01-01'
    fetch_csv(url, 
              symbol='VIX', 
              date_column='Date')
    
    #Initialize SPY
    ###############    
    
    set_symbol_lookup_date('2014-3-5')
    context.SPY = symbol('SPY')
    context.HYG = symbol('HYG')

    #Initialize variable
    ###############
    
    context.position = 0
    context.recession = False
    context.short = False
    context.order = None
    context.market = 0
                
def handle_data(context, data):
    
        

    #Record account info
    ####################
    
    record_account_info(context, data)
    
    #Compute data
    #############
    
    price_history = history(bar_count=41, 
                                frequency='1d', 
                                field='price')
    
    histSPY = price_history[context.SPY]
    histHYG = price_history[context.HYG]

    ROCP30SPY = talib.ROCP(histSPY, 30)*100  
    ROCP20SPY = talib.ROCP(histSPY, 20)*100
    ROCP10SPY = talib.ROCP(histSPY, 10)*100   
    ROCP5SPY = talib.ROCP(histSPY, 5)*100 
    
    ROCP5HYG = talib.ROCP(histHYG, 5)*100
    ROCP2HYG = talib.ROCP(histHYG, 2)*100
    ROCP10HYG = talib.ROCP(histHYG, 10)*100
    ROCP20HYG = talib.ROCP(histHYG, 20)*100
    ROCP1HYG = talib.ROCP(histHYG, 1)*100
    
    #Compute CSV data
    #################
    
    hist = _history(data)
    
    # Return if there is not enough data yet 
    if hist is None:
        order_target_percent(context.SPY, 1) 
        #ensure 100% in the benchmark while gathering data
        return
    
    # Use the close VIX value because the price is also the close
    rec = hist['price']['recession'].values
    REC = hist['price']['recession'].values[-1]
    vix = hist['Close']['VIX'].values
    VIX = hist['Close']['VIX'].values[-1]
    
    ROCP80VIX = talib.ROCP(vix, 80)*100 
    ROCP40VIX = talib.ROCP(vix, 40)*100 
    ROCP30VIX = talib.ROCP(vix, 30)*100    
    ROCP20VIX = talib.ROCP(vix, 20)*100    
    ROCP10VIX = talib.ROCP(vix, 10)*100    
    ROCP5VIX = talib.ROCP(vix, 5)*100    
    ROCP2VIX = talib.ROCP(vix, 2)*100   
    ROCP1VIX = talib.ROCP(vix, 1)*100
    
    MA200VIX = talib.MA(vix, 200)
    MA110VIX = talib.MA(vix, 110)
    MA100VIX = talib.MA(vix, 100)
    EMA40VIX = talib.EMA(vix, 40)
    WMA32VIX = talib.WMA(vix, 32)

    # If there is a recession short the market
    ##########################################
    
    if rec[-1] > 3 : 
        context.recession = True
    else :     
        context.recession = False
        
    if context.recession == True :
        
        if context.short == False :    
            order_target_percent(context.SPY, -1)
            context.position = -20
            context.short = True
    
        elif context.short == True :
            
            if MA100VIX[-1] < MA110VIX[-1] > 30 :
                order_target_percent(context.SPY, 1)
                context.short = None
                context.position = 20
    
       
    # If there is no recession
    ##########################

    if context.recession == False and data :
        
        # BUY signals
        #############
        
        #1 Buy when VIX is over it's MA200 and fall !
        if EMA40VIX[-1] > MA200VIX[-1] and WMA32VIX[-1] < EMA40VIX[-1] :
            order_target_percent(context.SPY, 1)
            context.position = 25
        
        #2 Buy when VIX rise 100% in the last 80 days !
        elif ROCP80VIX[-1] > 100 :
            order_target_percent(context.SPY, 1)
            context.position = 5
        
        #3 Buy when VIX rise 100% in the last 40 days !
        elif ROCP40VIX[-1] > 100 and ROCP2VIX[-1] < 0:
            order_target_percent(context.SPY, 1)
            context.position = 10
            
        #4 Buy when HYG signal a recovery during bear market
        elif ROCP10HYG[-1] > 3 and ROCP20SPY[-1] < -5 and ROCP2HYG[-1] > 0.5:
            order_target_percent(context.SPY, 1)
            context.position = 15
        
        #5 Buy when HYG and SPY progress
        elif ROCP5SPY[-1] > 2.4 and ROCP5HYG[-1] > 2.4:
            order_target_percent(context.SPY, 1)
            context.position = 20
        
        # SELL signals
        ##############
        
        #1 Sell when VIX fall 
        if ROCP20VIX[-1] < -40 :
            order_target_percent(context.SPY, -1)
            context.position = -5
              
        #2 Sell when HYG signal a correction during bull market
        elif ROCP20SPY[-1] > 4.5 and ROCP10HYG[-1] < -0.5 : # and ROCP5HYG[-1] < -1 :
            order_target_percent(context.SPY, -1)
            context.position = -10
            
    record(    #ROCP1VIX=ROCP1VIX[-1],
               #ROCP2VIX=ROCP2VIX[-1],
               #ROCP40VIX=ROCP40VIX[-1],
               #EMA40VIX=EMA40VIX[-1],
               #MA200VIX=MA200VIX[-1],
               #ROCP20VIX=ROCP20VIX[-1],
               #ROCP20SPY=ROCP20SPY[-1],
               #ROCP10HYG=ROCP10HYG[-1],
               #ROCP5HYG=ROCP5HYG[-1],
               #ROCP2HYG=ROCP2HYG[-1],
               #market=context.market,
               pos=context.position)
    
def record_account_info(context, data):
    
    P = context.portfolio
    #market_value = sum(data[i].price * abs(P.positions[i].amount) for i in data)
    record(#leverage=market_value / P.portfolio_value,
           positions_value=P.positions_value,
           portfolio_value=P.portfolio_value)  

            
There was a runtime error.

@Wayne thanks but isn't it hazardous to open position without having neither buy nor sell signal ?

My suggestion to improve this algo woud be first to build a separate function for each of the buy n sell scenario. It is also better to differentiate HYG risk-off signal over a long period (slow divergence, i.e. x months) and over a short period of time (fast divergence, i.e. x days). Risk-off signals generated over short-term could lead to enter short position while signals generated over long-term period could lead the algo to enter crash protection mode, for example by closing or reducing long positions in order to limit exposure.
- After a buy or sell order is generated another improvement could be to optimize the timing to enter the market, for example by measuring market strenght & trend and also CBOE Equity Put/Call ratio and CBOE Equity Put volume.

The change simply serves to eliminate the back testing bias present in the original post by going long 100% benchmark. The algo runs with the benchmark until enough history is available at which point, it makes the same decisions as before. I want to emphasize that I did not make any strategy changes. I simply modified the code such that the algo performance line is now comparable with the benchmark performance.