OLMAR with Universe

This is an update on the OLMAR algorithm Grant and I have been working on over the last couple of months. We've been continuously updating and fixing bugs. I now went over the whole logic in detail and am highly confident that the implementation is correct.

The idea of the algorithm is that if a price of a stock diverged from its mavg, it will eventually revert to it (i.e. mean reversion). The portfolio is rebalanced in accordance to that. In more detail, the algorithm is finding the optimal portfolio weighting to maximize profits under the mean-reversion assumption. By doing so, the algo essentially implements a follow-the-loser strategy.

Secondly, this now uses the new set_universe() feature to remove selection bias. The algorithm needed to be changed a little bit to adjust for the changes in portfolio size which adds some complexity. Another problem is that the portfolio does not take into account that the sids can change. Maybe for each quarter the portfolio should be equally rebalanced? In any case though, this is a great feature as one can't just chose AAPL and go to town. I think it leads to more honest backtesting.

The paper describing the algorithm, which is one of my favorites so far, can be found here:
http://icml.cc/2012/papers/168.pdf
After thinking long and hard about it, there is actually a very nice intuitive understanding of what each term does. This previous paper goes into more detail about the intutions (the algorithm is not the same but the ideas are very similar):
http://www.cais.ntu.edu.sg/~chhoi/paper_pdf/PAMR_ML_final.pdf

There is a previous thread with discussions with the author of that paper here:
https://www.quantopian.com/posts/olmar-implementation-fixed-bug (note that this implementation contained a bug).

Ideas for improvements:
* Change mavg() to vwap(): high-volume stocks tend to be more volatile and the algorithm is prone to invest more into highly fluctuating stock.
* Change eps parameter: This is a critical parameter that adjusts how aggressively the portfolio gets rebalanced. Values closer to 1 are more conservative while a value like 5 leads to an allocation that puts everything into one basket.

231
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
import numpy as np
import datetime

def initialize(context):
context.eps = 2  #change epsilon here
context.init = False
context.counter = 0
context.stocks = []
set_slippage(slippage.VolumeShareSlippage(volume_limit=0.25, price_impact=0, delay=datetime.timedelta(minutes=0)))
set_commission(commission.PerShare(cost=0))
set_universe(universe.DollarVolumeUniverse(floor_percentile=98.0, ceiling_percentile=100.0))

def handle_data(context, data):
context.counter += 1
if context.counter <= 5:
return

context.stocks = [sid for sid in data]
m = len(context.stocks)

if not context.init:
context.b_t = np.ones(m) / m
rebalance_portfolio(context, data, context.b_t)
context.init = True
return

if len(context.b_t) > m:
# need to decrease portfolio vector
context.b_t = context.b_t[:m]
elif len(context.b_t) < m:
# need to grow portfolio vector
len_bt = len(context.b_t)
context.b_t = np.concatenate([context.b_t, np.ones(m-len_bt) / m])

assert len(context.b_t) == m

x_tilde = np.zeros(m)

b = np.zeros(m)

# find relative moving average price for each security
for i, stock in enumerate(context.stocks):
price = data[stock].price
x_tilde[i] = data[stock].mavg(5) / price

###########################
# Inside of OLMAR (algo 2)
x_bar = x_tilde.mean()

# market relative deviation
mark_rel_dev = x_tilde - x_bar

# Expected return with current portfolio
exp_return = np.dot(context.b_t, x_tilde)
log.debug("Expected Return: {exp_return}".format(exp_return=exp_return))
weight = context.eps - exp_return
log.debug("Weight: {weight}".format(weight=weight))
variability = (np.linalg.norm(mark_rel_dev))**2
log.debug("Variability: {norm}".format(norm=variability))
# test for divide-by-zero case
if variability == 0.0:
step_size = 0 # no portolio update
else:
step_size = max(0, weight/variability)
log.debug("Step-size: {size}".format(size=step_size))
log.debug("Market relative deviation:")
log.debug(mark_rel_dev)
log.debug("Weighted market relative deviation:")
log.debug(step_size*mark_rel_dev)
b = context.b_t + step_size*mark_rel_dev
b_norm = simplex_projection(b)
#np.testing.assert_almost_equal(b_norm.sum(), 1)

rebalance_portfolio(context, data, b_norm)

# Predicted return with new portfolio
pred_return = np.dot(b_norm, x_tilde)
log.debug("Predicted return: {pred_return}".format(pred_return=pred_return))

# Make sure that we actually optimized our objective
#assert exp_return-.001 <= pred_return, "{new} <= {old}".format(new=exp_return, old=pred_return)
# update portfolio
context.b_t = b_norm

def rebalance_portfolio(context, data, desired_port):
print 'desired'
print desired_port
desired_amount = np.zeros_like(desired_port)
current_amount = np.zeros_like(desired_port)
prices = np.zeros_like(desired_port)

if context.init:
positions_value = context.portfolio.starting_cash
else:
positions_value = context.portfolio.positions_value + context.portfolio.cash

for i, stock in enumerate(context.stocks):
current_amount[i] = context.portfolio.positions[stock].amount
prices[i] = data[stock].price

desired_amount = np.round(desired_port * positions_value / prices)
diff_amount = desired_amount - current_amount
for i, stock in enumerate(context.stocks):
order(stock, diff_amount[i]) #order_stock

def simplex_projection(v, b=1):
"""Projection vectors to the simplex domain

Implemented according to the paper: Efficient projections onto the
l1-ball for learning in high dimensions, John Duchi, et al. ICML 2008.
Implementation Time: 2011 June 17 by [email protected] AT pmail.ntu.edu.sg
Optimization Problem: min_{w}\| w - v \|_{2}^{2}
s.t. sum_{i=1}^{m}=z, w_{i}\geq 0

Input: A vector v \in R^{m}, and a scalar z > 0 (default=1)
Output: Projection vector w

:Example:
>>> proj = simplex_projection([.4 ,.3, -.4, .5])
>>> print proj
array([ 0.33333333, 0.23333333, 0. , 0.43333333])
>>> print proj.sum()
1.0

Original matlab implementation: John Duchi ([email protected])
Python-port: Copyright 2012 by Thomas Wiecki ([email protected]).
"""

v = np.asarray(v)
p = len(v)

# Sort v into u in descending order
v = (v > 0) * v
u = np.sort(v)[::-1]
sv = np.cumsum(u)

rho = np.where(u > (sv - b) / np.arange(1, p+1))[0][-1]
theta = np.max([0, (sv[rho] - b) / (rho+1)])
w = (v - theta)
w[w<0] = 0
return w
This backtest was created using an older version of the backtester. Please re-run this backtest to see results using the latest backtester. Learn more about the recent changes.
There was a runtime error.
Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

10 responses

Thanks Thomas,

I would also add, as a potential improvement, the so-called "BAH(OLMAR)" modification described in:

http://icml.cc/2012/papers/168.pdf

Effectively, it removes a potential bias of picking a fixed trailing window length (e.g. above, you use mavg(5)).

Additionally, a weighted moving average could be considered (e.g. http://en.wikipedia.org/wiki/Moving_average#Weighted_moving_average), so that more recent prices are favored over older ones.

Grant

Looks really good!

The first question that came to mind is: why does the performance trail off? It makes a huge gain in the first 6 months (2008 was not a good year) which is then never seen again.

Hello Thomas,

I see the same problem with this one (see my comment)...as soon as you start trading, the CASH should remain ~\$0, but it is not. Also, did you modify the algorithm intentionally, so that you no longer keep all of your money in the security portfolio (market)? My interpretation of the OLMAR algorithm, as published, is that it should re-allocate only amongst the (hopefully mean-reverting) securities, and not go to cash. It appears that you allow the algorithm to pull some/all of the money out of the market...right?

Grant

Thomas,

I tried running your code above and got this error:

9 Error Runtime exception: TypeError: init() got an unexpected keyword argument 'delay'

Grant

Grant, I think the latest and greatest OLMAR implementation is OLMAR 3.0

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Thanks Dan,

I just wanted to flag an apparent backwards compatibility issue with the slippage settings. Has 'delay' been removed?

Grant

I'm frankly not sure when that stopped working - I don't think it was ever a part of the documented slippage function. Thomas, of course, has insider knowledge and sometimes uses functions that aren't out and supported.

If you'd like to have a delay in your slippage, you can do a custom slippage model, too.

I do remember implementing this but I'm not sure what happened to it, sorry.

In any case, you should be OK just removing that option. Personally I like to start with a backtest that has all the complexities of order execution removed for debugging purposes, that's why it's there.

Runtime exception: TypeError: init() got an unexpected keyword argument 'delay' (Line 9)

Isaac you'll get better progress with the most recent version of the OLMAR implementation at OLMAR 3.0