Counter trend (limit orders, Zipline)

This is an example of my learning to use limit and stop orders. In normal operation, there is a limit order to buy at a lower price and a limit order to sell at a higher price. Here is the description:

Long-only counter-trend algorithm that adjusts every X multiple of the standard deviation of price over a lookback period (using limit and stop orders). It allows one price step per bar and increases the long position according to the bar's down price steps. The position is reduced back to a single long if a bar with an up price step occurs before downSteps exceeds downLimit. When downLimit is exceeded, a reset closes all positions. To try to avoid being long during downturns, trading after a reset is restarted when a dual moving average state becomes positive (fast > slow). The algorithm checks if the position size has exceeded cash, or if the bet is too big, although the share size is calculated to try to be fully invested when downSteps equals downLimit. The last trade price, from which steps are calculated, follows price steps up while holding the single long position.

It doesn't use some of the latest features for Quantopian since it is also trying to be compatible with Zipline, and I am hesitant to put it forward as recommended practice (the blind leading the blind). It was also an excuse to throw stuff in to see how things work between Quantopian and Zipline. Nevertheless, there should be some useful info in there for other beginners.

Best regards,

Bob Schmidt

117
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
#!/usr/bin/env python
# -*- coding: utf-8 -*-

'''
from __future__ import division
import pylab as pl

from zipline.finance import trading, commission, slippage
from zipline.api import order, record, symbol, get_open_orders, get_order
from zipline.api import cancel_order
'''

# code above is outside Quantopian in Zipline
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# code below is inside Quantopian

"""
Long-only counter-trend algorithm that adjusts every X multiple of the
standard deviation of price over a lookback period (using limit and stop
orders).  It allows one price step per bar and increases the long position
according to the bar's down price steps.  The position is reduced back
to a single long if a bar with an up price step occurs before downSteps
exceeds downLimit.  When downLimit is exceeded, a reset closes all
positions.  To try to avoid being long during downturns, trading after
a reset is restarted when a dual moving average state becomes positive
(fast > slow).  The algorithm checks if the position size has exceeded
cash, or if the bet is too big, although the share size is calculated
to try to be fully invested when downSteps equals downLimit.  The last
trade price, from which steps are calculated, follows price steps up
while holding the single long position.

The pure martingale doubles the bet on a loss with no limit on loss runs.

Alternately, we can reset after a limited run of losses and set the
betSize to a different function of downSteps to reduce the risk(reward).
"""

from math import sqrt
from datetime import datetime
import pytz
import numpy as np
import pandas as pd
from collections import deque as moving_window

def initialize(context):
context.stocks = [symbol('SPY'),  # S+P 500 ETF
symbol('XLY'),  # Consumer Discrectionary SPDR Fund
symbol('XLF'),  # Financial SPDR Fund
symbol('XLK'),  # Technology SPDR Fund
symbol('XLE'),  # Energy SPDR Fund
symbol('XLV'),  # Health Care SPDR Fund
symbol('XLI'),  # Industrial SPDR Fund
symbol('XLP'),  # Consumer Staples SPDR Fund
symbol('XLB'),  # Materials SPDR Fund
symbol('XLU'),  # Utilities SPDR Fund
symbol('TLT')]  # Long term treasury bond ETF
context.symbol = context.stocks[0]  # try some different stocks

minuteBars = False  # to match the setting in Quantopian
if minuteBars:
minuteBarMult = 60 * 6.5  # minutes in trading days
minuteStdevMult = sqrt(minuteBarMult)
else:  # daily bar data or in Zipline
minuteStdevMult = 1
minuteBarMult = 1

context.stdevSteps = 6 * minuteStdevMult  # trade steps at multiples of standard deviation
context.downLimit = 1  # reset the system if exceeded by downSteps
context.fastBars = 5 * minuteBarMult  # number of bars for fast moving average window
context.slowBars = int(round(context.fastBars * 2))  # slow mov avg bars
context.lookBackBars = int(round(252/6)) * minuteBarMult  # lookback period for std dev

context.showOpenOrders = False  # display open orders each bar
context.showWaitBars = False  # display fast-slow while waiting to trade each bar
context.showOrderIDs = False  # display order id strings

commissionPerShare = 0.006  # set all three to zero for ideal result
minCommissionPerTrade = 1.25  # set all three to zero for ideal result
bidAskSpread = 0.03  # set all three to zero for ideal result
context.minSize = 100  # minimum number of shares to trade
context.shareSize = context.minSize  # calculated to match cash amount
context.downSteps = 0  # counts the accumulated down steps
context.tradingEnabled = True  # else wait for mov avg signal after reset
context.startupCount = 0  # count the bars in the lookback period at startup
context.fastWindow = moving_window(maxlen=context.fastBars)
context.slowWindow = moving_window(maxlen=context.slowBars)
context.lookBack = moving_window(maxlen=context.lookBackBars)
context.openOrders = []

try:  # check for Quantopian
set_commission(commission.PerShare(cost=commissionPerShare,
set_benchmark(context.symbol)
context.symb = context.symbol.symbol
context.runningQuantopian = True

except:  # running Zipline (or my error above)
context.runningQuantopian = False
context.set_commission(commission.PerShare(cost=commissionPerShare,
context.capital_base = 1e5
context.symb = context.symbol
context.startDate = datetime(2008, 1, 1, 0, 0, 0, 0, pytz.utc)
context.endDate = datetime(2008, 6, 1, 0, 0, 0, 0, pytz.utc)

# differences are small for small values of downLimit
def calc_betSize(dnstep=0):
#return 2**dnstep  # double your bet each loss (max risk)
#return dnstep + 1  # increase bet by one each loss
#return round(dnstep**0.7)  # increase bet by < 1 each loss
return 1  # increase long with constant bet size each loss

# calculate the size to match downLimit number of down steps before reset
def calc_shareSize(context, data):
cost = 0
currPrice = data[context.symbol].price
for step in xrange(context.downLimit + 1):
betsiz = calc_betSize(step)
price = currPrice
price /= (1 + context.stdevSteps * context.stdev / currPrice)**step
cost += betsiz * context.shareSize * price
ratio = context.shareSize * context.portfolio.portfolio_value / cost
context.shareSize = context.minSize * int(0.95 * ratio / context.minSize)

# place a limit or a stop order and log the action
# often get warnings from volatile daily data versus a small price step
# shouldn't happen with minute data that changes more slowly
def limit_stop_order(context, data, amnt=0, price=0, style='', stepstr=''):
currPrice = data[context.symbol].price
if style == 'limit':
if amnt > 0 and price > currPrice:
print('request to buy at limit above price')
elif amnt < 0 and price < currPrice:
print('request to sell at limit below price')
orderID = order(context.symbol, amnt, limit_price=price)
elif style == 'stop':
if amnt > 0 and price < currPrice:
print('request to buy at stop below price')
elif amnt < 0 and price > currPrice:
print('request to sell at stop above price')
orderID = order(context.symbol, amnt, stop_price=price)
context.openOrders.append(orderID)
orderID = orderID if context.showOrderIDs else ''
orderstr = stepstr + str(context.downSteps) + ' place '
orderstr += style + ': ' + str(amnt) + ' ' + str(context.symb)
orderstr += ' @ ' + str(price) + ' ' + str(orderID)
print(orderstr)

def handle_data(context, data):
currPrice = data[context.symbol].price
context.fastWindow.append(currPrice)
context.slowWindow.append(currPrice)
context.lookBack.append(currPrice)
position = context.portfolio.positions[context.symbol].amount
netValue = context.portfolio.portfolio_value
netValue -= context.portfolio.starting_cash

else:

fast = np.mean(context.fastWindow)
slow = np.mean(context.slowWindow)
record(fast=fast, slow=slow)

if context.startupCount < context.lookBackBars:  # fill moving avg windows
# daily standard deviation
context.stdev = np.std(context.lookBack)/sqrt(context.lookBackBars)
context.startupCount += 1
return

# the price delta of a price step
priceStep = round(context.stdevSteps * context.stdev, 2)

if context.runningQuantopian:
datestr = ""  # Quantopian has built-in date string printing
else:
datestr = algo.get_datetime().strftime('%Y-%m-%d ')
datestr += str(currPrice) + ' ' + str(priceStep)

filledOrder = False
for ID in context.openOrders:  # check for a filled limit or stop order
ordr = get_order(ID)
if ordr:
#print(ordr.keys)
if ordr.filled:
filledOrder = True
orderID = ordr.id if context.showOrderIDs else ''
if ordr.limit_reached:
stylestr = 'limit'
elif ordr.stop_reached:
stylestr = 'stop'
if context.showTrades and fast < slow and not context.showWaitBars:
print("Waiting for positive moving average to start trading.")
symb = str(ordr.sid.symbol) if context.runningQuantopian else ordr.sid
message = datestr
message += ' fill {style}: {amnt} {symb} @ {price} {ID}'
message = message.format(style=stylestr, amnt=ordr.amount,
ID=orderID)
message += ' position: ' + str(position)
message += ' netpnl: ' + str(round(netValue))
print(message)
context.openOrders.remove(ID)
if ordr.amount > 0:
context.downSteps += 1
elif ordr.amount < 0:
context.downSteps = 0

if filledOrder:  # cancel the other order
for ID in context.openOrders:
ordr = get_order(ID)
if ordr.limit:
stylestr = 'limit'
cancelPrice = ordr.limit
elif ordr.stop:
stylestr = 'stop'
cancelPrice = ordr.stop
symb = str(ordr.sid.symbol) if context.runningQuantopian else ordr.sid
orderID = ordr.id if context.showOrderIDs else ''
message = datestr + ' cancel {style}: {amnt} {symb} @ {price} {ID}'
message = message.format(style=stylestr, amnt=ordr.amount,
symb=symb, price=cancelPrice,
ID=orderID)
print(message)
cancel_order(ordr)
context.openOrders.remove(ID)
# calculate change to priceStep only when no open orders
context.stdev = np.std(context.lookBack)/sqrt(context.lookBackBars)

foundOpenOrders = False  # check to see if there are any open orders
open_orders = get_open_orders()
if open_orders:
for security, orders in open_orders.iteritems():
for oo in orders:
orderID = oo.id if context.showOrderIDs else ''
if oo.limit:
foundOpenOrders = True
stylestr = 'limit'
orderPrice = oo.limit
elif oo.stop:
foundOpenOrders = True
stylestr = 'stop'
orderPrice = oo.stop
message = datestr
message += ' open {style}: {amnt} {symb} @ {price} {ID}'
message = message.format(style=stylestr, amnt=oo.amount,
symb=oo.sid, price=orderPrice,
ID=orderID)
if context.showOpenOrders:
print(message)

# wait for positive mov avg after stop loss reset before resuming trading
if fast > slow:
else:  # don't trade until fast > slow
startstr = datestr + ' ' + str(position) + ' @ ' + str(currPrice)
startstr += ' waiting for restart: ' + str(round(fast - slow, 3))
print(startstr)
return

# inital start or resuming after a reset
upstr = ' up'
downstr = ' dn'
if position == 0:
# don't change stdev calc when there are open orders
context.stdev = np.std(context.lookBack)/sqrt(context.lookBackBars)
betSize = calc_betSize(context.downSteps)
calc_shareSize(context, data)  # rebalance when entering
tradeSize = max(1, betSize) * context.shareSize
order(context.symbol, tradeSize)  # market order to go long
orderstr = ' place market: ' + str(tradeSize) + ' '
orderstr += str(context.symb) + ' @ ' + str(currPrice)
print(datestr + orderstr)

# move the tradePrice up if an up step while holding a single long
elif context.downSteps == 0 and currPrice > tradePrice + priceStep:
betSize = calc_betSize(context.downSteps)
tradeSize = max(1, betSize) * context.shareSize
stepstr = upstr + ' step: ' + str(position) + ' @ '
stepstr += str(round(tradePrice, 2)) + ' netpnl: ' + str(netValue)
print(datestr + stepstr)
for ID in context.openOrders:  # cancel old buy order
ordr = get_order(ID)
symb = str(ordr.sid.symbol) if context.runningQuantopian else ordr.sid
orderID = ordr.id if context.showOrderIDs else ''
message = ' cancel limit: {amnt} {symb} @ {price} {ID}'
message = message.format(amnt=ordr.amount, symb=symb,
price=ordr.limit, ID=orderID)
print(datestr + message)
cancel_order(ordr)
context.openOrders.remove(ID)
datestr + downstr)

crashstr = ''
if context.downSteps < context.downLimit:  # limit not exceeded
betSize = calc_betSize(context.downSteps)
tradeSize = max(1, betSize) * context.shareSize
if context.portfolio.cash < 0:  # total position is too big
crashstr = " RESET (CASH) "
elif tradeSize > int(context.portfolio.cash / currPrice):
crashstr = " RESET (BET SIZE) "  # betSize is too big
betSize = calc_betSize(context.downSteps)
tradeSize = max(1, betSize) * context.shareSize
# limit order to buy on down step
'limit', datestr + downstr)
# stop order to reset:  sell all positions on down step
limit_stop_order(context, data, -position,
crashstr + datestr + downstr)
if round(position - context.shareSize) > 0:  # shares to sell
calc_shareSize(context, data)  # rebalance on every sale
# limit order to sell all except one long on up step
if round(position - context.shareSize) > 0:  # check rebalance
limit_stop_order(context, data,
-round(position - context.shareSize),
datestr + upstr)

#  code above is inside Quantopian
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
#  code below is outside Quantopian in Zipline

'''
def show_results(algo, data, results):
#bm_returns = br[(br.index >= start) & (br.index <= end)]
#results['benchmark_returns'] = (1 + bm_returns).cumprod().values
#results['algo_returns'] = (1 + results.returns).cumprod()
#sharpe = [risk['sharpe'] for risk in algo.risk_report['one_month']]
#print("Monthly Sharpe ratios: {0}".format(sharpe))

initstr = 'trade at ' + str(algo.stdevSteps) + ' std dev price moves, '
initstr += 'down step limit is ' + str(algo.downLimit)
print(initstr)

#idealret = results.ideal / algo.portfolio.starting_cash
actual = results.portfolio_value / algo.portfolio.starting_cash - 1
#resultstr = "ideal return: " + str(round(100 * idealret[-1], 2)) + '%'
results['draw_down'] = (pd.expanding_max(actual) - actual).values
resultstr = "total return: " + str(round(100 * actual[-1], 2)) + '%, '
resultstr += 'worst case draw down: '
resultstr += str(round(100 * np.max(results['draw_down']), 2)) + '%'
print(resultstr)
#print(results.tail())
#print(dir(algo.sim_params))
#print(dir(algo.portfolio))

fig = pl.figure(1, figsize=(8, 10))

#results[['algo_returns', 'benchmark_returns']].plot(ax=ax1, sharex=True)
#(1 + idealret).plot(ax=ax1)
(results.portfolio_value/algo.portfolio.starting_cash).plot(ax=ax1)
(1 - results['draw_down']).plot(ax=ax1)
pl.setp(ax1.get_xticklabels(), visible=False)
pl.legend(loc=0)

bmret = data[algo.symbol] / data[algo.symbol][0]
results['bm_draw_down'] = (pd.expanding_max(bmret) - bmret).values
bmret.plot(ax=ax2, color='blue')
(1 - results['bm_draw_down']).plot(ax=ax2, color='magenta')
(results.fast / results.fast[0]).plot(ax=ax2, color='green')
(results.slow / results.slow[0]).plot(ax=ax2, color='tan')
pl.setp(ax2.get_xticklabels(), visible=False)
pl.legend(loc=0)

results.position.plot(ax=ax3, color='blue')
pl.legend(loc=0)

pl.gcf().set_size_inches(18, 8)
pl.show()

if __name__ == '__main__':
start=algo.startDate, end=algo.endDate).dropna()
results = algo.run(data)
show_results(algo, data, results)
'''
There was a runtime error.
5 responses

Thanks Bob, long example but well put together with a lot of info and comments. Everything here is still up-to-date and working obviously; I think the new features would just make the code more compact. Still a useful example for sure, thanks for sharing!

Gus

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Bob, Cool...Zipline and Quantopian. I like to use Visual Studio with PTVS. Does Python not have a compiler directive/ options so that you can define something like:

#if QUANTOPIAN
...do this
#else
...do this.


instead of the try-catch and uncommenting shows a mismatch between proper overriding/ wrapping/ inheritance of Quantopian on top of ZipLine...a simple copy past of code from zipline into quantopian would be the ultimate goal...

Quant Trader, Yes, when I tried to use an if statement, the builder said that the imports in the Zipline code weren't in the whitelist. That's way beyond my pay grade, lol. My simple solution was the comment blocks to display the Zipline code in Quantopian. Normally, when I have the uncommented Zipline file open in Spyder, I can copy/paste just the initialize and handle_data functions between them and the try/except block makes it work pretty well. I think the recent changes that the devs have made makes it alot easier to switch between them and they are probably not done yet.

Best regards,

Bob

Very cool. Good to see people writing zipline compatible algorithms. Two quick suggestions, you can use the new symbols() feature:

    context.stocks = symbols('SPY',  # S+P 500 ETF
'XLY',  # Consumer Discrectionary SPDR Fund
'XLF',  # Financial SPDR Fund
'XLK',  # Technology SPDR Fund
'XLE',  # Energy SPDR Fund
'XLV',  # Health Care SPDR Fund
'XLI',  # Industrial SPDR Fund
'XLP',  # Consumer Staples SPDR Fund
'XLB',  # Materials SPDR Fund
'XLU',  # Utilities SPDR Fund
'TLT')  # Long term treasury bond ETF


And, you can use history() since zipline will have support for it in the upcoming 0.7 release.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Hi Bob,
You enspired me to update zipline to latest version and see if I can get your code running as well. I have more experience with C# and Microsoft IDE. I am using: http://pytools.codeplex.com/ . Which has also the Package Console Manager with Python integrated. So you can run "pip install ..." directly from the IDE..
J.