Seeking Help Complete First Algorithm

Hi Quantopians,

I am new to this platform and so far I am loving it. I am currently an intern and one of my bosses has asked me to make a model on Excel for a certain stock behavior. It's pretty simple.
1. At the end of the day, I look at the top 5 gainers of the top 500 stocks and buy them all at 20% of the portfolio.
2. Following day, if the stock goes up 1% then sell it off, if it doesn't reach 1% then sell 5minutes before closing bell.
3. Buy top 5 stocks of the day and repeat.
I've been doing it manually for a couple of weeks but after having worked on my python whole summer I thought it would be neat if I could build an algorithm that would do the work for me. I have gotten to a certain extent but I can't seem to be able to complete it.
I've attached the notebook with pipeline that gets's me the top 5 equity of the day.
I couldn't attach the backtests since no backtests are working but here what I currently have has my code. I would like the code to work if I decide to change the top 5 to the top 10 or the top 20 and this is why I defined a weight function to work in any cases.

Thanks for you time and your help!

import quantopian.algorithm as algo
from quantopian.algorithm import attach_pipeline,pipeline_output
from quantopian.pipeline import Pipeline, CustomFactor
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.pipeline.filters import Q500US

def initialize(context):
schedule_function(my_rebalance,date_rules.every_day(),time_rules.market_close(minutes=1))
schedule_function(sell_off,date_rules.every_day(),time_rules.market_close(minutes=5))
schedule_function(sell,date_rules.every_day())
my_pipe = make_pipeline()
attach_pipeline(my_pipe,'my_pipeline')
def my_compute_weights(context):
if len(context.longs)==0:
long_weight = 0
else:
long_weight = 1 / len(context.longs)
return long_weight
context.output = pipeline_output('my_pipeline')
# LONG
context.longs = context.output['close_minus_open'].index.tolist()
context.long_weight = my_compute_weights(context)
def my_rebalance(context,data):
for security in context.longs:
order_target_percent(security,context.long_weight)
def sell_off(context,data):
for security in context.portfolio.positions:
order_target_percent(security,0)
def sell(context,data):
for security in context.portfolio.positions:
entry_price = context.portfolio.positions[security].cost_basis
pct = 1.01
exit_price = pct*entry_price
price_history = data.history(security, 'price', 2, '1m')
current_price = price_history[-1]
if current_price >= exit_price:
if get_open_orders(security):
continue
order_target_percent(security, 0)
class Close_Open_Delta(CustomFactor):
# Define inputs
inputs = [USEquityPricing.open, USEquityPricing.close]
window_length = 1
def compute(self, today, assets, out, open, close):
out[:] = (close - open)/open

def make_pipeline():
"""
Function to create a pipeline with high, low, open,and the Close_Open_Delta custom factors
"""
# Factors for the latest pricing data can be easily created using the "latest" method
# from the desired dataset
open = USEquityPricing.open.latest
high = USEquityPricing.high.latest
low = USEquityPricing.low.latest
close = USEquityPricing.close.latest
# Custom factors need to be instantiated explicitly
close_minus_open = Close_Open_Delta()
# Create a pipeline and add the factors to it
p = Pipeline()
# Create filters to select and sort securities based upon factors
# Always good to use an initial "universe" filter to start with a realistic set of tradable securities
my_universe = Q500US()    # built in universe of 500 larger stocks

# Create a "top gainers" filter. Use a mask to ensure only qualified securities get counted.
# Set the number to however many "off the top" one wishes to purchase (in this case 5)
top_gainers = close_minus_open.top(5, mask = (my_universe))
# Set a screen for our pipeline. Don't really need the my_universe and other filters because they were
# included in the top_gainers mask
p.set_screen(top_gainers)
return p

0
9 responses

@Laurent, What kind of errors are you getting when you run the backtests? You may want to start with a working backtest, rip out the parts you don't want, insert your own code for the parts that have to be different. Of course do that one step at a time making sure that each change doesn't break it.

I get this:
KeyError: 'longs'
There was a runtime error on line 32.

Not related to this specific error. But you should probably change the line about long_weight to:
long_weight = 1.0/len(context.longs)

Take a look at https://www.quantopian.com/posts/new-and-need-help-with-something-basic-i-hope
It appears to have already been working except for the above fix. It may provide some clues as to what you are missing in regards to 'longs'.

Still need help on this, can't seem to know how to fix it.

That KeyError for longs means it does not exist yet. So, either:

Add this to initialize section:
context.longs = {}

or

Add something similar to this to

# Gets our pipeline output every day.
context.output = pipeline_output('value_momo_pipeline')

# The low p/e stocks that we want to long.
context.longs = context.output[context.output['longs']].index.tolist()


Here is the correct algo that I have. Looking at transactions it seems like the only problem I have is selling the stock if it goes up by 1% during the day.

11
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
import quantopian.algorithm as algo
from quantopian.algorithm import attach_pipeline,pipeline_output
from quantopian.pipeline import Pipeline, CustomFactor
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.pipeline.filters import Q500US

def initialize(context):

schedule_function(my_rebalance,date_rules.every_day(),time_rules.market_close(minutes=20))
schedule_function(sell_off,date_rules.every_day(),time_rules.market_close(minutes=30))
schedule_function(sell,date_rules.every_day())

my_pipe = make_pipeline()
attach_pipeline(my_pipe,'my_pipeline')

# def my_compute_weights(context):
#     if len(context.longs)==0:
#        long_weight = 0
#     else:
#        long_weight = 1 / len(context.longs)
#     return long_weight

context.output = pipeline_output('my_pipeline')

# LONG
context.longs = context.output[context.output['longs']].index.tolist()

context.long_weight = 0.2

def my_rebalance(context,data):
for security in context.longs:
order_target_percent(security,context.long_weight)
def sell_off(context,data):
for security in context.portfolio.positions:
order_target_percent(security,0)
def sell(context,data):
for security in context.portfolio.positions:
entry_price = context.portfolio.positions[security].cost_basis
pct = 1.01
exit_price = pct*entry_price
price_history = data.history(security, 'price', 2, '1m')
current_price = price_history[-1]
if current_price >= exit_price:
order_target_percent(security, 0)
else:
pass

class Close_Open_Delta(CustomFactor):
# Define inputs
inputs = [USEquityPricing.open, USEquityPricing.close]
window_length = 1

def compute(self, today, assets, out, open, close):
out[:] = (close - open)/open

def make_pipeline():
"""
Function to create a pipeline with high, low, open,and the Close_Open_Delta custom factors
"""

# Factors for the latest pricing data can be easily created using the "latest" method
# from the desired dataset

open = USEquityPricing.open.latest
high = USEquityPricing.high.latest
low = USEquityPricing.low.latest
close = USEquityPricing.close.latest

# Custom factors need to be instantiated explicitly
close_minus_open = Close_Open_Delta()
longs = close_minus_open > 0
# Create a pipeline and add the factors to it
p = Pipeline()

# Create filters to select and sort securities based upon factors
# Always good to use an initial "universe" filter to start with a realistic set of tradable securities
my_universe = Q500US()    # built in universe of 500 larger stocks

# Create a "top gainers" filter. Use a mask to ensure only qualified securities get counted.
# Set the number to however many "off the top" one wishes to purchase (in this case 5)
top_gainers = close_minus_open.top(5, mask = (my_universe))

# Set a screen for our pipeline. Don't really need the my_universe and other filters because they were
# included in the top_gainers mask
p.set_screen(top_gainers)

return p
There was a runtime error.

The algorithm seems to sell at the end of the day. It also buys evenly the 5 stocks afterwards. All I am missing is to trigger a sell of a position if the stock rises up 1% the next day.

Easiest would be limit orders at start of day like stp_lmt() here, although they aren't allowed in the contest or with Optimize.
You can see the ordering in the logging window. Or particular stocks can be set, in 'symbols' list.

7
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
import quantopian.algorithm as algo
from quantopian.algorithm import attach_pipeline,pipeline_output
from quantopian.pipeline  import Pipeline, CustomFactor
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.pipeline.filters import Q500US

def initialize(context):
context.lmt_ratio = 1.01
context.stp_ratio =  .98
context.number_of_stocks = 5
schedule_function(rebalance, date_rules.every_day(), time_rules.market_close(minutes=20))
schedule_function(sell_off,  date_rules.every_day(), time_rules.market_close(minutes=30))
schedule_function(stp_lmt,   date_rules.every_day(), time_rules.market_open())
#schedule_function(sell,      date_rules.every_day(), time_rules.market_open())

pipe = make_pipeline(context)
attach_pipeline(pipe,'pipeline')

for i in range(1, 391):
schedule_function(track_orders, date_rules.every_day(), time_rules.market_open(minutes=i))

# def compute_weights(context):
#     if len(context.longs)==0:
#        long_weight = 0
#     else:
#        long_weight = 1 / len(context.longs)
#     return long_weight

context.output = pipeline_output('pipeline')
context.longs  = context.output[context.output['longs']].index.tolist()
context.long_weight = 1.0 / len(context.longs)  #0.2

if 'log_pipe_done' not in context:       # show pipe info once
log_pipe(context, data, context.output, 4) #, details=['open', 'high', 'low' ...])

def rebalance(context,data):
for security in context.longs:
if not data.can_trade(security): continue
order_target_percent(security, context.long_weight)

def sell_off(context,data):
cncl_oos(context, data)    # cancel open limit orders mainly
for security in context.portfolio.positions:
if not data.can_trade(security): continue
order_target(security, 0)

def stp_lmt(context,data):
for s in context.portfolio.positions:
if not data.can_trade(s): continue

# limits
order_target(s, 0, style=LimitOrder(
context.lmt_ratio * context.portfolio.positions[s].cost_basis
))
# stops
#order_target(s, 0, style=StopOrder(
#        context.stp_ratio * context.portfolio.positions[s].cost_basis
#    ))

def sell(context,data):
for security in context.portfolio.positions:
if not data.can_trade(security): continue
entry_price = context.portfolio.positions[security].cost_basis
pct = 1.01
exit_price = pct * entry_price
#price_history = data.history(security, 'price', 2, '1m')
current_price = data.current(security, 'price')  #price_history[-1]
if current_price >= exit_price:
order_target(security, 0)

class Close_Open_Delta(CustomFactor):    # Define inputs
inputs = [USEquityPricing.open, USEquityPricing.close]
window_length = 1
def compute(self, today, assets, out, open, close):
out[:] = (close - open) / open

def make_pipeline(context):
''' Function to create a pipeline with high, low, open,and the Close_Open_Delta custom factors

Factors for the latest pricing data can be easily created using the "latest" method
from the desired dataset
'''

# Create filters to select and sort securities based upon factors
# Always good to use an initial "universe" filter to start with a realistic set of tradable securities
universe  = Q500US()    # built in universe of 500 larger stocks

open  = USEquityPricing.open.latest
high  = USEquityPricing.high.latest
low   = USEquityPricing.low.latest
close = USEquityPricing.close.latest

# Custom factors need to be instantiated explicitly
longs = close_minus_open > 0
universe &= longs  # adding to mask

# Create a pipeline and add the factors to it
p = Pipeline()

# Create a "top gainers" filter. Use a mask to ensure only qualified securities get counted.
# Set the number to however many "off the top" one wishes to purchase (in this case 5)
top_gainers = close_minus_open.top(context.number_of_stocks, mask=universe)

# Set a screen for the pipeline. Don't really need the universe and other filters because they were
# included in the top_gainers mask
p.set_screen(top_gainers)

return p

def cncl_oos(context, data):   # Primarily to prevent the logging of unfilled orders at end of day
oo = get_open_orders()     #   Can also be use at any time to limit partial fills.
for s in oo:               #   Here, canceling limit orders before closing in sell_off()
for o in oo[s]:
cancel_order(o.id)

def track_orders(context, data):
'''  Show orders when made and filled.
Info: https://www.quantopian.com/posts/track-orders
'''
c = context
if 'trac' not in c:
c.t_opts = {        # __________    O P T I O N S    __________
'symbols'     : [],   # List of symbols to filter for, like ['TSLA', 'SPY']
'log_neg_cash': 1,    # Show cash only when negative.
'log_cash'    : 1,    # Show cash values in logging window or not.
'log_ids'     : 1,    # Include order id's in logging window or not.
'log_unfilled': 1,    # When orders are unfilled. (stop & limit excluded).
'log_cancels' : 0,    # When orders are canceled.
}    # Move these to initialize() for better efficiency.
c.trac = {}
c.t_dates  = {  # To not overwhelm the log window, start/stop dates can be entered.
'active': 0,
'start' : [],   # Start dates, option like ['2007-05-07', '2010-04-26']
'stop'  : []    # Stop  dates, option like ['2008-02-13', '2010-11-15']
}
from pytz import timezone as _tz  # Python only does once, makes this portable.
#   Move to top of algo for better efficiency.
# If 'start' or 'stop' lists have something in them, triggers ...
if c.t_dates['start'] or c.t_dates['stop']:
_date = str(get_datetime().date())
if   _date in c.t_dates['start']:    # See if there's a match to start
c.t_dates['active'] = 1
elif _date in c.t_dates['stop']:     #   ... or to stop
c.t_dates['active'] = 0
else: c.t_dates['active'] = 1           # Set to active b/c no conditions.
if c.t_dates['active'] == 0: return     # Skip if not active.
def _minute():   # To preface each line with the minute of the day.
bar_dt = get_datetime().astimezone(_tz('US/Eastern'))
return (bar_dt.hour * 60) + bar_dt.minute - 570 # (-570 = 9:31a)
def _trac(to_log):      # So all logging comes from the same line number,
log.info(' {}   {}'.format(str(_minute()).rjust(3), to_log))  # for vertical alignment in the logging window.

for oid in c.trac.copy():               # Existing known orders
o = get_order(oid)
if o.dt == o.created: continue        # No chance of fill yet.
cash = ''
prc  = data.current(o.sid, 'price') if data.can_trade(o.sid) else c.portfolio.positions[o.sid].last_sale_price
if (c.t_opts['log_neg_cash'] and c.portfolio.cash < 0) or c.t_opts['log_cash']:
cash = str(int(c.portfolio.cash))
if o.status == 2:                     # Canceled
do = 'Buy' if o.amount > 0 else 'Sell' ; style = ''
if o.stop:
style = ' stop {}'.format(o.stop)
if o.limit: style = ' stop {} limit {}'.format(o.stop, o.limit)
elif o.limit: style = ' limit {}'.format(o.limit)
if not c.t_opts['symbols'] or (c.t_opts['symbols'] and o.sid.symbol in c.t_opts['symbols']):
if c.t_opts['log_cancels']:
_trac('  Canceled {} {} {}{} at {}   {}  {}'.format(do, o.amount,
o.sid.symbol, style, prc, cash, o.id[-4:] if c.t_opts['log_ids'] else ''))
del c.trac[o.id]
elif o.filled:                        # Filled at least some.
filled = '{}'.format(o.amount)
filled_amt = 0
if o.status == 1:                   # Complete
if 0 < c.trac[o.id] < o.amount:
filled   = 'all {}/{}'.format(o.filled - c.trac[o.id], o.amount)
filled_amt = o.filled
else:                                    # c.trac[o.id] value is previously filled total
filled_amt = o.filled - c.trac[o.id]   # filled this time, can be 0
c.trac[o.id] = o.filled                # save fill value for increments math
filled = '{}/{}'.format(filled_amt, o.amount)
if filled_amt:
now = ' ({})'.format(c.portfolio.positions[o.sid].amount) if c.portfolio.positions[o.sid].amount else ' _'
pnl = ''  # for the trade only
amt = c.portfolio.positions[o.sid].amount ; style = ''
if (amt - o.filled) * o.filled < 0:    # Profit-taking scenario including short-buyback
cb = c.portfolio.positions[o.sid].cost_basis
if cb:
pnl  = -filled_amt * (prc - cb)
sign = '+' if pnl > 0 else '-'
pnl  = '  ({}{})'.format(sign, '%.0f' % abs(pnl))
if o.stop:
style = ' stop {}'.format(o.stop)
if o.limit: style = ' stop () limit {}'.format(o.stop, o.limit)
elif o.limit: style = ' limit {}'.format(o.limit)
if o.filled == o.amount: del c.trac[o.id]
if not c.t_opts['symbols'] or (c.t_opts['symbols'] and o.sid.symbol in c.t_opts['symbols']):
_trac('   {} {} {}{} at {}{}{}'.format(
'Bot' if o.amount > 0 else 'Sold', filled, o.sid.symbol, now,
'%.2f' % prc, pnl, style).ljust(52) + '  {}  {}'.format(cash, o.id[-4:] if c.t_opts['log_ids'] else ''))
elif c.t_opts['log_unfilled'] and not (o.stop or o.limit):
if not c.t_opts['symbols'] or (c.t_opts['symbols'] and o.sid.symbol in c.t_opts['symbols']):
_trac('      {} {}{} unfilled  {}'.format(o.sid.symbol, o.amount,
' limit' if o.limit else '', o.id[-4:] if c.t_opts['log_ids'] else ''))

oo = get_open_orders().values()
if not oo: return                       # Handle new orders
cash = ''
if (c.t_opts['log_neg_cash'] and c.portfolio.cash < 0) or c.t_opts['log_cash']:
cash = str(int(c.portfolio.cash))
for oo_list in oo:
for o in oo_list:
if o.id in c.trac: continue         # Only new orders beyond this point
prc = data.current(o.sid, 'price') if data.can_trade(o.sid) else c.portfolio.positions[o.sid].last_sale_price
c.trac[o.id] = 0 ; style = ''
now  = ' ({})'.format(c.portfolio.positions[o.sid].amount) if c.portfolio.positions[o.sid].amount else ' _'
if o.stop:
style = ' stop {}'.format(o.stop)
if o.limit: style = ' stop {} limit {}'.format(o.stop, o.limit)
elif o.limit: style = ' limit {}'.format(o.limit)
if not c.t_opts['symbols'] or (c.t_opts['symbols'] and o.sid.symbol in c.t_opts['symbols']):
_trac('{} {} {}{} at {}{}'.format('Buy' if o.amount > 0 else 'Sell',
o.amount, o.sid.symbol, now, '%.2f' % prc, style).ljust(52) + '  {}  {}'.format(cash, o.id[-4:] if c.t_opts['log_ids'] else ''))

def log_pipe(context, data, df, num, details=None):
''' Log info about pipeline output or any DataFrame or Series (df)
'''

# Options
log_nan_only = 0
show_sectors = 0
show_sorted_details = 1

if not len(df):
log.info('Empty')
return

# Series ......
context.log_pipe_done = 1 ; padmax = 6 ; content = ''
if 'Series' in str(type(df)):    # is Series, not DataFrame
nan_count = len(df[df != df])
nan_count = 'NaNs {}/{}'.format(nan_count, len(df)) if nan_count else ''
if (log_nan_only and nan_count) or not log_nan_only:
pad = max(6, len(str(df.max())))
log.info('{}{}{}   Series {}  len {}'.format('min' .rjust(pad+5),
))
return

# DataFrame ......
content_min_max = [ ['','min','mean','max',''] ]
for col in df.columns:
if col == 'sector' and not show_sectors: continue
nan_count = len(df[col][df[col] != df[col]])
nan_count = 'NaNs {}/{}'.format(nan_count, len(df)) if nan_count else ''
padmax    = max( padmax, max(6, len(str(df[col].max()))) )
content_min_max.append([col, str(df[col] .min()), str(df[col].mean()), str(df[col] .max()), nan_count])
if log_nan_only and nan_count or not log_nan_only:
content = 'Rows: {}  Columns: {}'.format(df.shape[0], df.shape[1])
if len(df.columns) == 1: content = 'Rows: {}'.format(df.shape[0])

paddings = [6 for i in range(4)]
for lst in content_min_max:    # set max lengths
i = 0
for val in lst[:4]:    # value in each sub-list
i += 1
content += ('\n{}{}{}{}{}'.format(
''
))
for lst in content_min_max[1:]:    # populate content using max lengths
content += ('\n{}{}{}{}     {}'.format(
lst[4],
))
log.info(content)

if not show_sorted_details: return
if len(df.columns) == 1:    return    # skip detail if only 1 column
if details == None:
details = df.columns
for detail in details:
if detail == 'sector': continue
hi = df[details].sort_values(by=detail, ascending=False).head(num)
lo = df[details].sort_values(by=detail, ascending=False).tail(num)
content  = ''
content += ('_ _ _   {}   _ _ _'  .format(detail))
content += ('\n\t... {} highs\n{}'.format(detail, str(hi)))
content += ('\n\t... {} lows \n{}'.format(detail, str(lo)))
if log_nan_only and not len(lo[lo[detail] != lo[detail]]):
continue  # skip if no nans
log.info(content)


There was a runtime error.

You've added a lot to my already existing algo. This is a lot to try and absorb but I really like the result. This will be a great learning experience!