Back to Community
ASDASD

ASDASD

7 responses

Derek, this looks very promising. From just eyeing it, I think all the major dips are during major downturns in the market as a whole. I wish I had the time to iterate on this right now but I at least wanted to get my gut feel suggestion to you tonight. This might sound too simple but maybe get out of positions when the market is below like a 200 day SMA or maybe use MACD to do something similar to sell off? Just a suggestion, I know it sounds crude so take it or leave it.

Derek,

When I clone this and run it, it buys no stocks? A total flat line. Not sure why? I've been going through it, but not sure why?

Best,
Tom

Performance is often deceptive when its a single continuous return series.

I mean just running the algo for the past 5 years or so demonstrates that. The algo is down ~30% against a benchmark of +70%.

Clone Algorithm
15
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Derek M Tishler
# Value Stocks with Buffett/Graham Quick Ranking
# 9-15-2015

import datetime
import pytz
import pandas as pd
import numpy as np
import scipy.stats as stats

def before_trading_start(context, data): 
    # Query for securities based on PE ratio and their economic sector
    fundamental_df = get_fundamentals(
        # Retrieve data so we can rank by PE*PB
        query(
            fundamentals.valuation_ratios.pe_ratio,
            fundamentals.valuation_ratios.pb_ratio
        )
        .filter(fundamentals.valuation_ratios.pb_ratio <= 0.75)
        .filter(fundamentals.valuation_ratios.pcf_ratio <= 15.0)
        .filter(fundamentals.operation_ratios.current_ratio >= 1.0)
        .filter(fundamentals.valuation.market_cap >= 100e6)
        .order_by(fundamentals.valuation_ratios.pe_ratio.desc()).limit(500)
    )
    context.fundamental_df = fundamental_df
    
    update_universe(context.fundamental_df.columns.values)

def initialize(context):
    
    # create the trading guard to avoid over-leveraged ETFs 
    set_do_not_order_list(security_lists.leveraged_etf_list)
    
    # Ensure we see the correct symbols for our backtest range(Dont think this is doing anythting in this algo cause fundamental universe selection?)
    set_symbol_lookup_date('2002-01-03') #YMD
    
    set_benchmark(symbol('SPY'))
    
    # Careful adding bias... but you can remove troublemakers by listing them here by symbol.
    context.flagged_stocks = []
    
    # Commented out to use defaults
    #set_commission(commission.PerShare(cost=0.005, min_trade_cost=1.0))
    #set_slippage(slippage.VolumeShareSlippage(volume_limit=0.25, price_impact=0.1))
    
    # Schedule timmed function
    schedule_function(end_of_day, date_rules.every_day(), time_rules.market_close(minutes=1))
    
    # Change this variable if ou want to rebalance less frequently
    context.Rebalance_Days = 60  #20
    
    # Rebalance at 10:15AM EST
    context.rebalance_date       = None
    context.rebalance_hour_start = 10
    context.rebalance_hour_end   = 15
    
    context.portfolio_size = 20
    
    context.has_created_portfolio = False
    
    context.orders_to_carry_over = {}
    
def handle_data(context, data):
    
    # Get the current exchange time, in the exchange timezone 
    context.exchange_time = pd.Timestamp(get_datetime()).tz_convert('US/Eastern')
    
    # We want end of day legerage, so we put this line in the end_of_day helper function.
    #record(leverage=context.account.leverage)
    
    #Check that our portfolio does not  contain any invalid/external positions/securities
    check_invalid_positions(context, data)
    
    #Continue orders from previous day, DONT USE IT, will fight with something(order_targer_percent?) after orders are cancled and then resumed and it drive some order ammounts to infinity using leverage.
    """if len(context.orders_to_carry_over) > 0:
        need_to_return = False
        for stock in context.orders_to_carry_over:
            if check_if_no_conflicting_orders(stock):
                log.info("Resuming Order for %s of %d shares."%(stock.symbol, context.orders_to_carry_over[stock]))
                order_target(stock, context.orders_to_carry_over[stock])
                need_to_return = True
        context.orders_to_carry_over.clear()
        if need_to_return:
            return"""
    
    # If it's a rebalance day (defined in intialize()) then rebalance:
    if context.rebalance_date == None or context.exchange_time > context.rebalance_date + datetime.timedelta(days=context.Rebalance_Days):

        # Do nothing if there are open orders:
        if has_orders(context, data):
            print('has open orders - doing nothing!')
            return
        
        if not context.has_created_portfolio:
            create_portfolio(context, data, context.exchange_time)
        else:
            rebalance(context, data, context.exchange_time)
        
def create_portfolio(context, data, exchange_time):  
    # Only create portfolio if we are in the user specified rebalance time-of-day window
    if exchange_time.hour < context.rebalance_hour_start or exchange_time.hour > context.rebalance_hour_end:
       return        

    past_volume = history(bar_count=125, frequency='1d', field='volume')#.dropna()
    
    # Rank Stocks Based on PE*PB, ascending
    stocks          = []
    fundamental_score = []
    
    # Lets loop through all the items we have fundamental data on
    for item in context.fundamental_df:
        # Dont include benchmark, or compare it to itself, or allow leveraged etf, ensure we can place orders(is it in data>), filter out low liquidity
        if item.symbol == 'SPY' or item.symbol in context.flagged_stocks or item in security_lists.leveraged_etf_list or item not in data or past_volume[item].mean() < 1e5:
            continue
        
        fundamental_score.append(context.fundamental_df[item]['pe_ratio'] * context.fundamental_df[item]['pb_ratio'])
        
        stocks.append(item)
       
    
    log.debug("Selecting from Universe of size: %d"%(len(stocks)))
    stocks                 = np.array(stocks)
    fundamental_score      = np.array(fundamental_score)
    
    # sort by fundamental score(Smallest to largest, to flip you could slice by [::-1])
    f_score_sorted_indx    = np.argsort(fundamental_score)
    
    stocks                 = stocks[f_score_sorted_indx]
    fundamental_score      = fundamental_score[f_score_sorted_indx]

    #Build both sides of the portfolio
    portfolio_long_stocks  = stocks[0:context.portfolio_size]
    portfolio_long_f_score = fundamental_score[0:context.portfolio_size]
        
    #Build portfolio
    for stock in portfolio_long_stocks:
        order_target_percent(stock, 0.99/len(portfolio_long_stocks))#, limit_price=None, stop_price=None)

    #Print the portforlio to log.
    if len(portfolio_long_stocks) > 0:
        log_stock = []
        log_volume = []
        for stock in portfolio_long_stocks:
            log_stock.append(stock.symbol)
            log_volume.append(past_volume[stock].mean())
        log.debug(log_stock)
        log.debug(log_volume)
        log.debug(portfolio_long_f_score)
        log.debug("")
        
        context.has_created_portfolio = True
        context.rebalance_date = exchange_time
        #log.info("Rebalanced to target portfolio weights at %s" % str(exchange_time))
        log.info("Created portfolio at %s" % str(exchange_time))
        log.debug("")
        log.debug("")
        
    else:
        log.debug("Failed to form a long side of the portfolio at %s" % str(exchange_time))
      

# Same reblanace method from ETF example on Quantopian Help    
def rebalance(context, data, exchange_time):  
    # Only rebalance if we are in the user specified rebalance time-of-day window
    if exchange_time.hour < context.rebalance_hour_start or exchange_time.hour > context.rebalance_hour_end:
       return

    # Do the rebalance. Loop through each of the stocks and order to the target
    # percentage.  If already at the target, this command doesn't do anything.
    # A future improvement could be to set rebalance thresholds.
    for sec in context.portfolio.positions:
        order_target_percent(sec, 0.99/len(context.portfolio.positions), limit_price=None, stop_price=None)

    context.rebalance_date = exchange_time
    log.debug("")
    log.info("Rebalanced to target portfolio weights at %s" % str(exchange_time))
    #log.debug("")

#########################################################################################

# misc helper functions

def check_if_no_conflicting_orders(stock):
    # Check that we are not already trying to move this stock
    open_orders = get_open_orders()
    safeToMove  = True
    if open_orders:
        for security, orders in open_orders.iteritems():
            for oo in orders:
                if oo.sid == stock.sid:
                    if oo.amount != 0:
                        safeToMove = False
    return safeToMove
    #
def check_invalid_positions(context, securities):
    # Check that the portfolio does not contain any broken positions
    # or external securities
    for sid, position in context.portfolio.positions.iteritems():
        if sid not in securities and position.amount != 0:
            errmsg = \
                "Invalid position found: {sid} amount = {amt} on {date}"\
                .format(sid=position.sid,
                        amt=position.amount,
                        date=get_datetime())
            raise Exception(errmsg)    
def end_of_day(context, data):
    
    record(leverage=context.account.leverage)
    
    # cancle any order at the end of day. Do it ourselves so we can see slow moving stocks.
    open_orders = get_open_orders()
    
    #if open_orders:# or context.portfolio.positions_value > 0.:
        #log.info("")
        #log.info("*** EOD: Stoping Orders & Printing Held ***")

    """# Print what positions we are holding overnight
    for stock in data:
        if context.portfolio.positions[stock.sid].amount != 0:
            log.info("{0:s} has remaining {1:,d} Positions worth ${2:,.2f}"\
                     .format(stock.symbol,
                             context.portfolio.positions[stock.sid].amount,
                             context.portfolio.positions[stock.sid].cost_basis\
                             *context.portfolio.positions[stock.sid].amount))"""
    # Cancle any open orders ourselves(In live trading this would be done for us, soon in backtest too)
    context.orders_to_carry_over.clear()
    if open_orders:  
        # Cancle any open orders ourselves(In live trading this would be done for us, soon in backtest too)
        for security, orders in open_orders.iteritems():
            for oo in orders:
                log.info("X CANCLED {0:s} with {1:,d} / {2:,d} filled"\
                                     .format(security.symbol,
                                             oo.filled,
                                             oo.amount))
                context.orders_to_carry_over[security] = oo.amount - oo.filled
                cancel_order(oo)
    #
        log.debug('')
    
def has_orders(context, data):
    # Return true if there are pending orders.
    has_orders = False
    for sec in data:
        orders = get_open_orders(sec)
        if orders:
            for oo in orders:                  
                message = 'Open order for {amount} shares in {stock}'  
                message = message.format(amount=oo.amount, stock=sec)  
                log.info(message)

            has_orders = True
    return has_orders
    
There was a runtime error.

When you backtest from 2010 that will have big difference performance, why have these issue ?

Clone Algorithm
21
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Derek M Tishler
# Value Stocks with Buffett/Graham Quick Ranking
# 9-15-2015

import datetime
import pytz
import pandas as pd
import numpy as np
import scipy.stats as stats

def before_trading_start(context, data): 
    # Query for securities based on PE ratio and their economic sector
    fundamental_df = get_fundamentals(
        # Retrieve data so we can rank by PE*PB
        query(
            fundamentals.valuation_ratios.pe_ratio,
            fundamentals.valuation_ratios.pb_ratio
        )
        .filter(fundamentals.valuation_ratios.pb_ratio <= 0.75)
        .filter(fundamentals.valuation_ratios.pcf_ratio <= 15.0)
        .filter(fundamentals.operation_ratios.current_ratio >= 1.0)
        .filter(fundamentals.valuation.market_cap >= 100e6)
        .order_by(fundamentals.valuation_ratios.pe_ratio.desc()).limit(500)
    )
    context.fundamental_df = fundamental_df
    
    update_universe(context.fundamental_df.columns.values)

def initialize(context):
    
    # create the trading guard to avoid over-leveraged ETFs 
    set_do_not_order_list(security_lists.leveraged_etf_list)
    
    # Ensure we see the correct symbols for our backtest range(Dont think this is doing anythting in this algo cause fundamental universe selection?)
    set_symbol_lookup_date('2002-01-03') #YMD
    
    set_benchmark(symbol('SPY'))
    
    # Careful adding bias... but you can remove troublemakers by listing them here by symbol.
    context.flagged_stocks = []
    
    # Commented out to use defaults
    #set_commission(commission.PerShare(cost=0.005, min_trade_cost=1.0))
    #set_slippage(slippage.VolumeShareSlippage(volume_limit=0.25, price_impact=0.1))
    
    # Schedule timmed function
    schedule_function(end_of_day, date_rules.every_day(), time_rules.market_close(minutes=1))
    
    # Change this variable if ou want to rebalance less frequently
    context.Rebalance_Days = 60  #20
    
    # Rebalance at 10:15AM EST
    context.rebalance_date       = None
    context.rebalance_hour_start = 10
    context.rebalance_hour_end   = 15
    
    context.portfolio_size = 20
    
    context.has_created_portfolio = False
    
    context.orders_to_carry_over = {}
    
def handle_data(context, data):
    
    # Get the current exchange time, in the exchange timezone 
    context.exchange_time = pd.Timestamp(get_datetime()).tz_convert('US/Eastern')
    
    # We want end of day legerage, so we put this line in the end_of_day helper function.
    #record(leverage=context.account.leverage)
    
    #Check that our portfolio does not  contain any invalid/external positions/securities
    check_invalid_positions(context, data)
    
    #Continue orders from previous day, DONT USE IT, will fight with something(order_targer_percent?) after orders are cancled and then resumed and it drive some order ammounts to infinity using leverage.
    """if len(context.orders_to_carry_over) > 0:
        need_to_return = False
        for stock in context.orders_to_carry_over:
            if check_if_no_conflicting_orders(stock):
                log.info("Resuming Order for %s of %d shares."%(stock.symbol, context.orders_to_carry_over[stock]))
                order_target(stock, context.orders_to_carry_over[stock])
                need_to_return = True
        context.orders_to_carry_over.clear()
        if need_to_return:
            return"""
    
    # If it's a rebalance day (defined in intialize()) then rebalance:
    if context.rebalance_date == None or context.exchange_time > context.rebalance_date + datetime.timedelta(days=context.Rebalance_Days):

        # Do nothing if there are open orders:
        if has_orders(context, data):
            print('has open orders - doing nothing!')
            return
        
        if not context.has_created_portfolio:
            create_portfolio(context, data, context.exchange_time)
        else:
            rebalance(context, data, context.exchange_time)
        
def create_portfolio(context, data, exchange_time):  
    # Only create portfolio if we are in the user specified rebalance time-of-day window
    if exchange_time.hour < context.rebalance_hour_start or exchange_time.hour > context.rebalance_hour_end:
       return        

    past_volume = history(bar_count=125, frequency='1d', field='volume')#.dropna()
    
    # Rank Stocks Based on PE*PB, ascending
    stocks          = []
    fundamental_score = []
    
    # Lets loop through all the items we have fundamental data on
    for item in context.fundamental_df:
        # Dont include benchmark, or compare it to itself, or allow leveraged etf, ensure we can place orders(is it in data>), filter out low liquidity
        if item.symbol == 'SPY' or item.symbol in context.flagged_stocks or item in security_lists.leveraged_etf_list or item not in data or past_volume[item].mean() < 1e5:
            continue
        
        fundamental_score.append(context.fundamental_df[item]['pe_ratio'] * context.fundamental_df[item]['pb_ratio'])
        
        stocks.append(item)
       
    
    log.debug("Selecting from Universe of size: %d"%(len(stocks)))
    stocks                 = np.array(stocks)
    fundamental_score      = np.array(fundamental_score)
    
    # sort by fundamental score(Smallest to largest, to flip you could slice by [::-1])
    f_score_sorted_indx    = np.argsort(fundamental_score)
    
    stocks                 = stocks[f_score_sorted_indx]
    fundamental_score      = fundamental_score[f_score_sorted_indx]

    #Build both sides of the portfolio
    portfolio_long_stocks  = stocks[0:context.portfolio_size]
    portfolio_long_f_score = fundamental_score[0:context.portfolio_size]
        
    #Build portfolio
    for stock in portfolio_long_stocks:
        order_target_percent(stock, 0.99/len(portfolio_long_stocks))#, limit_price=None, stop_price=None)

    #Print the portforlio to log.
    if len(portfolio_long_stocks) > 0:
        log_stock = []
        log_volume = []
        for stock in portfolio_long_stocks:
            log_stock.append(stock.symbol)
            log_volume.append(past_volume[stock].mean())
        log.debug(log_stock)
        log.debug(log_volume)
        log.debug(portfolio_long_f_score)
        log.debug("")
        
        context.has_created_portfolio = True
        context.rebalance_date = exchange_time
        #log.info("Rebalanced to target portfolio weights at %s" % str(exchange_time))
        log.info("Created portfolio at %s" % str(exchange_time))
        log.debug("")
        log.debug("")
        
    else:
        log.debug("Failed to form a long side of the portfolio at %s" % str(exchange_time))
      

# Same reblanace method from ETF example on Quantopian Help    
def rebalance(context, data, exchange_time):  
    # Only rebalance if we are in the user specified rebalance time-of-day window
    if exchange_time.hour < context.rebalance_hour_start or exchange_time.hour > context.rebalance_hour_end:
       return

    # Do the rebalance. Loop through each of the stocks and order to the target
    # percentage.  If already at the target, this command doesn't do anything.
    # A future improvement could be to set rebalance thresholds.
    for sec in context.portfolio.positions:
        order_target_percent(sec, 0.99/len(context.portfolio.positions), limit_price=None, stop_price=None)

    context.rebalance_date = exchange_time
    log.debug("")
    log.info("Rebalanced to target portfolio weights at %s" % str(exchange_time))
    #log.debug("")

#########################################################################################

# misc helper functions

def check_if_no_conflicting_orders(stock):
    # Check that we are not already trying to move this stock
    open_orders = get_open_orders()
    safeToMove  = True
    if open_orders:
        for security, orders in open_orders.iteritems():
            for oo in orders:
                if oo.sid == stock.sid:
                    if oo.amount != 0:
                        safeToMove = False
    return safeToMove
    #
def check_invalid_positions(context, securities):
    # Check that the portfolio does not contain any broken positions
    # or external securities
    for sid, position in context.portfolio.positions.iteritems():
        if sid not in securities and position.amount != 0:
            errmsg = \
                "Invalid position found: {sid} amount = {amt} on {date}"\
                .format(sid=position.sid,
                        amt=position.amount,
                        date=get_datetime())
            raise Exception(errmsg)    
def end_of_day(context, data):
    
    record(leverage=context.account.leverage)
    
    # cancle any order at the end of day. Do it ourselves so we can see slow moving stocks.
    open_orders = get_open_orders()
    
    #if open_orders:# or context.portfolio.positions_value > 0.:
        #log.info("")
        #log.info("*** EOD: Stoping Orders & Printing Held ***")

    """# Print what positions we are holding overnight
    for stock in data:
        if context.portfolio.positions[stock.sid].amount != 0:
            log.info("{0:s} has remaining {1:,d} Positions worth ${2:,.2f}"\
                     .format(stock.symbol,
                             context.portfolio.positions[stock.sid].amount,
                             context.portfolio.positions[stock.sid].cost_basis\
                             *context.portfolio.positions[stock.sid].amount))"""
    # Cancle any open orders ourselves(In live trading this would be done for us, soon in backtest too)
    context.orders_to_carry_over.clear()
    if open_orders:  
        # Cancle any open orders ourselves(In live trading this would be done for us, soon in backtest too)
        for security, orders in open_orders.iteritems():
            for oo in orders:
                log.info("X CANCLED {0:s} with {1:,d} / {2:,d} filled"\
                                     .format(security.symbol,
                                             oo.filled,
                                             oo.amount))
                context.orders_to_carry_over[security] = oo.amount - oo.filled
                cancel_order(oo)
    #
        log.debug('')
    
def has_orders(context, data):
    # Return true if there are pending orders.
    has_orders = False
    for sec in data:
        orders = get_open_orders(sec)
        if orders:
            for oo in orders:                  
                message = 'Open order for {amount} shares in {stock}'  
                message = message.format(amount=oo.amount, stock=sec)  
                log.info(message)

            has_orders = True
    return has_orders
    
There was a runtime error.

Thanks. You're right. I had reset to daily data to try to speed up the backtest and forgot about it. In terms of seeing huge differences, this will happen with a) fewer stocks and b) lower turnover. This is one reason that 'automated' rolling backtests with a large number of start dates can help a lot. But, the backtester is already so slow as to be unusable for large chunks of the day.... so not sure Quant can build this now even if they want to. I'd love to see them speed the site up. Or limit people's 'tests'... or something.

does not do well during the recession

This algorithm doesn't work because it is filtering the universe and choosing the top 20 securities only once (at least that's what's happening in the migrated version).

The op got good results because starting in the date he started, the algorithm picks 20 securities that perform well afterwards.

Please correct me if I'm wrong...