Back to Community
Live results vs. backtest results at a glance

In lieu of a more detailed implementation shortfall analysis (that I'll try to get up in blog post form soon), I wanted to share an updated screenshot of the actual market performance of the equal-weighted sector ETF strategy we have been running the past few months compared with backtest results for the same time period.

There are few possible places for discrepancies between these two:

-time frame is slightly off, the backtest runs from 9:30am on 1/28 through 4pm yesterday (4/24) while the live trading results are from 1pm on 1/28 through just now.

-back-tester estimates slippage and commissions while live trading has the actual costs as incurred via our broker.

In general though we have seen good fidelity between our backtested results and live trading. Look for a more detailed study on this to come soon.

live screencapture

Clone Algorithm
113
Loading...
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
'''
    This algorithm defines a target long-only equal weight portfolio and rebalances it at a user-specified frequency 
    NOTE: This algo is intended to run in minute-mode simulation and is compatible with LIVE TRADING.

'''
import datetime
import pytz
import pandas as pd

# The initialize method will be called once to set up context variables
def initialize(context):
    # Define the stocks in your target portfolio:
    context.secs =   [ sid(19662),  # XLY Consumer Discrectionary SPDR Fund   
                       sid(19656),  # XLF Financial SPDR Fund  
                       sid(19658),  # XLK Technology SPDR Fund  
                       sid(19655),  # XLE Energy SPDR Fund  
                       sid(19661),  # XLV Health Care SPRD Fund  
                       sid(19657),  # XLI Industrial SPDR Fund  
                       sid(19659),  # XLP Consumer Staples SPDR Fund   
                       sid(19654),  # XLB Materials SPDR Fund  
                       sid(19660) ] # XLU Utilities SPRD Fund

    context.SECURITIES_SET = set(context.secs)
    
    context.rebalance_date = None
    context.Rebalance_Days = 21
    context.weights = 0.99/len(context.secs)
    context.rebalance_hour_start = 10
    context.rebalance_hour_end = 15
    
    # set commissions and slippage to 0 since they will be handled by IB.
    set_commission(commission.PerTrade(cost=1.0))
    set_slippage(slippage.FixedSlippage(spread=0.01))

def handle_data(context, data):
    # Make sure we have no external positions.
    check_invalid_positions(context, context.SECURITIES_SET) 

    # Get the current exchange time, in local timezone 
    exchange_time = pd.Timestamp(get_datetime()).tz_convert('US/Eastern')

    # If it's rebalance day (defined by user params) then rebalance:
    if context.rebalance_date == None or exchange_time > context.rebalance_date + datetime.timedelta(days=context.Rebalance_Days):
        
        # Do nothing if there are open orders:
        if has_orders(context):
            print('has open orders')
            return
        
        rebalance(context, data, exchange_time)  
       
def rebalance(context, data, exchange_time):  
    # Only rebalance if we are in the user specified rebalance window
    if exchange_time.hour < context.rebalance_hour_start or exchange_time.hour > context.rebalance_hour_end:
       return
    
    for sec in context.secs:
        order_target_percent(sec, context.weights, limit_price=None, stop_price=None)
        
    context.rebalance_date = exchange_time
    log.info("Rebalanced to target portfolio weights at %s" % str(exchange_time))
    
def has_orders(context):
    # Return true if there are pending orders.
    has_orders = False
    for sec in context.secs:
        orders = get_open_orders(sec)
        if orders:
            for oo in orders:                  
                message = 'Open order for {amount} shares in {stock}'  
                message = message.format(amount=oo.amount, stock=sec)  
                log.info(message)
                
            has_orders = True
    return has_orders

def check_invalid_positions(context, securities):
    for sid, position in context.portfolio.positions.iteritems():
        if sid not in securities and position.amount != 0:
            errmsg = \
            "Invalid position found: {sid} amount = {amt} on {date}".format(
                sid=position.sid,
                amt=position.amount,
                date=get_datetime())
            raise Exception(errmsg)
There was a runtime error.
Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

1 response

Thanks Jess, I've been seeing different kinds of behavior in a higher frequency strategy between live trading and backtest, but it is, of course, very time consuming to have the live trading algo running and develop some data on that vs. the backtest.

In this algo, there were three trades in all of March. Four in Feb. It would be interesting to see how a higher frequency strategy compares between backtest and live trading; my guess is there would be more discrepancies in executions for low volume stocks, and so on.