Back to Community
Sentdex's Quantopian Tutorials Updated (by me) for Quantopian 2: Algorithm 3 Videos 8-11

This is the third in my series updating Sentdex's Tutorial for Quantopian 2. It covers parts 8-11 of Sentdex's Tutorial.

(Here's the first.) (Here's the second.)

If you attempted this part of the tutorial recently, it probably gave you a handful of headaches- I know it gave me a few.

First, I was not able to use the fetcher for the link that he provided- it would stall out every time. So I had to download the data and upload it to my dropbox. I made three different batch sizes: small (about 3 months of data), medium (about a year of data) and large (the full amount that Sentdex uses in the video). Still, I could only get the small and medium to load. If you figure out what's going on please let me know.

Note, if you want to do something similar yourself, take a look at the docs- but you're free to just use the urls posted in the backtest.

Second, two securities were missing from the security list that Sentdex used despite the set_symbol_lookup_date being set to the same date he used. The symbols were 'ACE' and 'MHFI' and are excluded in the security list below.

Thirdly, and most importantly- Sentdex is wrong about the Position object. If you are short the asset, Position.amount will be a negative number. If you want to prove this to yourself, I suggest you set the size='small' in initialize and uncomment the log.info call in my_record_vars. You will see your proof in the logs.

If your a Quantopian employee- maybe this should be made explicit in the documents even though I know it is very intuitive.

And finally a few things that have been covered before-

  • I update for stock in data to the correct new syntax of if data.can_trade(stock)
  • I update data[stock].mavg(days) to the correct new syntax of
    data.history(stock,'price',days,'1d').mean()
  • I clean up the order logic syntax to be much cleaner
    And finally, I am 95% certain he incorrectly uses the stop_order logic. (see my explanation in my post on the second algorithm)

Note, the order logic is much more complicated than the first two algorithms because there are separate, distinct rules for each of entering longs, exiting longs, entering shorts, and exiting shorts. If you have trouble following along in the code, don't hesitate to ask.

Clone Algorithm
31
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
"""
This is a template algorithm on Quantopian for you to adapt and fill in.
"""
from quantopian.algorithm import attach_pipeline, pipeline_output
from quantopian.pipeline import Pipeline
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.pipeline.factors import AverageDollarVolume


def preview(df):
    log.info(df.head())
    return df


def initialize(context):
    """
    Called once at the start of the algorithm.
    """   
    set_symbol_lookup_date('2012-10-01')
    context.security_list = symbols('AAPL', 'MCD', 'FB', 'GME', 'INTC', 'SBUX', 'T', 'MGM', 'SHLD', 'NKE', 'NFLX', 'PFE', 'GS', 'TGT', 'NOK', 'SNE', 'TXN', 'JNJ', 'KO', 'VZ', 'XOM', 'WMT', 'MCO', 'TWTR', 'URBN', 'MCP', 'MSFT', 'HD', 'KSS', 'AMZN', 'S', 'BA', 'F', 'JPM', 'QCOM', 'TSLA', 'YHOO', 'BBRY', 'GM', 'IBM', 'C', 'ZNGA', 'BAC', 'DIS', 'SCHW', 'UA', 'CSCO', 'ORCL', 'SYMC', 'WFC', 'TM', 'EBAY', 'SCHL', 'MS', 'NDAQ', 'TIF', 'AIG', 'DAL', 'JCP', 'MRK', 'CA', 'SIRI', 'AMD', 'CVX', 'FSLR', 'LMT', 'P', 'CBS', 'TWX', 'PEP', 'LNKD', 'CMG', 'NVDA', 'BBY', 'TWC', 'M', 'RHT', 'ACN', 'CRM', 'PETS', 'CELG', 'BLK', 'GD', 'DOW', 'YUM', 'GE', 'MA', 'DTV', 'DDD', 'CAT', 'FDX', 'GRPN', 'BK', 'GILD', 'V', 'DUK', 'FFIV', 'WFM', 'CVS', 'UNH', 'LUV', 'CBG', 'AFL', 'CHK', 'BRCM', 'HPQ', 'LULU', 'ATVI', 'RTN', 'EMC', 'NOC', 'MAR', 'X', 'BMY', 'LOW', 'COST', 'HON', 'SPLS', 'BKS', 'AA', 'AXP', 'AMGN', 'GPS', 'MDT', 'LLY', 'CME', 'MON', 'WWWW', 'MU', 'DG', 'TRIP', 'HAL', 'COH', 'WYNN', 'PCLN', 'HTZ', 'CLF', 'DD', 'ACI', 'FCX', 'AON', 'GMCR', 'CSX', 'ADBE', 'PRU', 'PG', 'MYL', 'STT', 'PPG', 'EXPE', 'KORS', 'JNPR', 'UTX', 'HOT', 'SNDK', 'CCL', 'DRI', 'BIIB', 'BBT', 'APA', 'A', 'TDC', 'ANF', 'MTB', 'PPL', 'ABT', 'GNW', 'KMI', 'MET', 'FE', 'DVA', 'ETFC', 'GLW', 'NRG', 'INTU', 'KR', 'ARNA', 'VALE', 'MSI', 'EOG', 'AET', 'MAT', 'HST', 'COP', 'MO', 'IVZ', 'HUM', 'NUE', 'CI')
    
    # Rebalance every day, 1 hour after market open.
    schedule_function(my_rebalance, date_rules.every_day(), time_rules.market_open())
    schedule_function(my_record_vars, date_rules.every_day(), time_rules.market_close())
    
    links = {'small': 'https://dl.dropboxusercontent.com/s/gefqjbc38t11jo0/sentdex_signal_sample.csv?dl=0',
             'medium': 'https://dl.dropboxusercontent.com/s/loby2u6l7hkhnqy/sentdex_signal_medium.csv?dl=0',
             'large': 'https://dl.dropboxusercontent.com/s/0k8tg07ece3bkkd/sentdex_signal_full.csv?dl=0'}
    
    # adjust this variable to adjust the size of the backtest
    # and make sure to adjust date range on the backtest setup:
    # small ends at 01/09/2013
    # medium ends at 10/05/2013
    # large ends at 06/17/2015
    size = 'medium'
    fetch_csv(links[size], pre_func = preview)    
    

def before_trading_start(context, data):
    """
    Called every day before market open.
    """
    
    sentiments = data.current(context.security_list, 'sentiment_signal')
    
    positions = context.portfolio.positions
    context.longs = set()
    context.shorts = set()
    
    # find which positions that are long and which are short
    for position in positions.values():
        if position.amount > 0:
            context.longs.add(position.sid)
        elif position.amount < 0:
            context.shorts.add(position.sid)
        else:
            raise Exception('why is an asset with 0 amount in positions')
            
    # setting up action sets
    context.signal = {k:v for k,v in sentiments.iteritems()}
    context.close_longs = set()
    context.cover_shorts = set()
    context.buys = set()
    short_candidates = set() # entering new shorts requires a regime confirmation
    
    for stock,signal in context.signal.items():
        # logic for closing longs
        if (stock in context.longs) and signal < -1:
            context.close_longs.add(stock)
        
        # logic for closing shorts
        elif (stock in context.shorts) and signal >= -1:
            context.cover_shorts.add(stock)
            
        # logic for entering longs
        elif (stock not in positions) and signal > 5:
            context.buys.add(stock)
        
        # logic for entering shorts
        elif (stock not in positions) and signal <= -3:
            short_candidates.add(stock)
        
        else:
            pass
    
    # getting moving averages for identifying bear regimes
    prices300 = data.history(list(short_candidates), 'price', 300, '1d')
    prices100 = prices300[-100:]
    
    ma1 = prices100.mean()
    ma2 = prices300.mean()
    trending_down = (ma1 < ma2).to_dict()
    
    # logic for entering shorts
    context.sells = set(stock for stock,trend in trending_down.items()
                        if  trend and (stock not in positions))
    
    context.target_weight = calc_weight(context)
    
def calc_weight(context):
    # equally weighted portfolio with a max of 10% in any one asset
    
    # find number of securities in target portfolio
    # = (# of current positions) + (# of positions being added) - (# positions being closed)
    n = len(context.portfolio.positions) + len(context.buys) + len(context.sells) - len(context.close_longs) - len(context.cover_shorts)
    
    # no more than 10% of portfolio in any one stock
    return min(0.10, 1./n) if (n>0) else 0
    
def my_rebalance(context,data):
    """
    Execute orders according to our schedule_function() timing. 
    """
    # buy new longs
    for asset in context.buys:
        if data.can_trade(asset):
            order_target_percent(asset, context.target_weight)
            log.info("Entering long position in {}".format(asset.symbol))
    
    # close longs
    for asset in context.close_longs:
        if data.can_trade(asset):
            order_target(asset,0)
            log.info("Closing long position in {}".format(asset.symbol))
    
    # open new shorts
    for asset in context.sells:
        if data.can_trade(asset):
            order_target_percent(asset, -context.target_weight)
            log.info("Entering short position in {}".format(asset.symbol))
            
    # cover shorts
    for asset in context.cover_shorts:
        if data.can_trade(asset):
            order_target(asset,0)        
            log.info("Closing short position in {}".format(asset.symbol))
 
def my_record_vars(context, data):
    """
    Plot variables at the end of each day.
    """
    longs, shorts = 0,0
    for position in context.portfolio.positions.values():
        if position.amount < 0:
            #log.info("PROOF: Have a position of {} in {}".format(position.amount, position.sid.symbol))
            shorts += 1
        else:
            longs += 1
            
    record(leverage=context.account.leverage,
           longs = longs,
           shorts = shorts)
There was a runtime error.