Minimum Variance Portfolio on selected stocks

I used Wayne's Minimum Variance Portfolio on some selected stocks, and the results looks quite good initially. However, after I ran a dividend re-invested buy&hold strategy on this stock selection, the returns are even better than the returns using Minimum Variance algorithm. Although the algorithm tries to minimize variance, the volatility of the Minimum Variance returns are even larger than the buy&hold strategy. Can anyone help to explain?

54
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
import pandas as pd
import numpy as np
import numpy.linalg as la
import math

def initialize(context):
#parameters
context.nobs = 252
context.recalibrate = 126 #re-estimate every so often (in days)
context.leverage= 1

#setup the identifiers and data storage
#context.tickers = ['xlf', 'xle', 'xlu', 'xlk', 'xlb', 'xlp', 'xly','xli', 'xlv']
#context.sids = [ sid(19656), sid(19655),
#                   sid(19660), sid(19658),
#                   sid(19654), sid(19659),
#                   sid(19662), sid(19657),
#                   sid(19661) ]
context.sids = [ sid(24), sid(3496), sid(4954), sid(6928),
sid(62), sid(23998), sid(22140), sid(3735), sid(6653),
sid(368), sid(1900), sid(3766), sid(5061), sid(20940),
sid(16841), sid(4799), sid(3951), sid(5692),sid(25010),
sid(679), sid(23112), sid(4151), sid(5729), sid(7883),
sid(698), sid(2190), sid(5885),
sid(700), sid(3149), sid(4283), sid(5923),
sid(980), sid(3212), sid(4707), sid(5938), sid(8229),
sid(1335), sid(4758), sid(8347),
sid(1582), sid(20088), sid(4922), sid(6295)]
context.tickers = [int(str(e).split(' ')[0].strip("Security(")) for e in context.sids]
context.data = pd.DataFrame({ k : pd.Series() for k in context.tickers } )
context.daysToRecalibration = 0
context.onevec = np.asmatrix(np.ones((len(context.tickers), 1)))

def handle_data(context, data):
if context.portfolio.starting_cash == context.portfolio.cash:
#buy into the benchmark while we build the starting data set
order(sid(8554), math.floor(context.portfolio.starting_cash/data[sid(8554)].close_price) )

if len(context.data.index) < context.nobs:
#still recording data
newRow = pd.DataFrame({k:float(data[e].returns()) for k,e in zip(context.tickers, context.sids) },index=[0])
context.data = context.data.append(newRow, ignore_index = True)
else:
newRow = pd.DataFrame({k:float(data[e].returns()) for k,e in zip(context.tickers, context.sids) },index=[0])
context.data = context.data.append(newRow, ignore_index = True)
context.data = context.data[1:len(context.data.index)]

if context.portfolio.positions[sid(8554)].amount != 0:
#data gathering time is done, get out of the benchmark
order(sid(8554), -1.0*context.portfolio.positions[sid(8554)].amount)
#wait a day for the trades to clear before placing the new trades.
return

if context.daysToRecalibration == 0:
context.daysToRecalibration = context.recalibrate
#recalibrate
log.info('recalibrating...')

#calculate the minimum variance portfolio weights;
precision = np.asmatrix(la.inv(context.data.cov()))
pimv = precision*context.onevec / (context.onevec.T*precision*context.onevec)
pimv = { e:pimv[i,0] for i,e in enumerate(context.tickers) }

#open all positions:
startingCash = (context.portfolio.starting_cash+context.portfolio.pnl)*context.leverage
for i, e in enumerate(context.sids):
currentPosition = context.portfolio.positions[e].amount
newPosition = math.floor(startingCash*pimv[context.tickers[i]]/data[e].price)
order(e, newPosition - currentPosition)
else:
context.daysToRecalibration -= 1

record(c = context.portfolio.positions_value)


This backtest was created using an older version of the backtester. Please re-run this backtest to see results using the latest backtester. Learn more about the recent changes.
There was a runtime error.
5 responses

Below is the dividend re-invested buy&hold returns

37
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
import numpy as np
import datetime

def initialize(context):

# context.stocks = [sid(19662),sid(19659),sid(19655),sid(19656),sid(19661),sid(19657),sid(19654),sid(19658),sid(19660), sid(8554)]
#context.stocks = [sid(8554)]
context.stocks = [ sid(24), sid(3496), sid(4954), sid(6928),
sid(62), sid(23998), sid(22140), sid(3735), sid(6653),
sid(368), sid(1900), sid(3766), sid(5061), sid(20940),
sid(16841), sid(4799), sid(3951), sid(5692),sid(25010),
sid(679), sid(23112), sid(4151), sid(5729), sid(7883),
sid(698), sid(2190), sid(5885),
sid(700), sid(3149), sid(4283), sid(5923),
sid(980), sid(3212), sid(4707), sid(5938), sid(8229),
sid(1335), sid(4758), sid(8347),
sid(1582), sid(20088), sid(4922), sid(6295)]
context.m = len(context.stocks)
context.b_t = np.ones(context.m) / context.m
context.eps = 1  #change epsilon here
context.init = True
context.counter = 0

set_slippage(slippage.VolumeShareSlippage(volume_limit=0.25, price_impact=0))
set_commission(commission.PerShare(cost=0))

def handle_data(context, data):
context.counter += 1
if context.counter == 1:
else:
ReInvest(context, data)
record(cash = context.portfolio.cash, portofolio = context.portfolio.positions_value)

for i, stock in enumerate(context.stocks):
prices = np.zeros_like(context.b_t)
desired_amount = np.zeros_like(prices)
prices[i] = data[stock].price
desired_amount[i] = np.round(context.portfolio.starting_cash / context.m / prices[i])
log.info("initially Bought {ticker} @ {price} for {amount} shares".format(ticker = stock, price = prices[i], amount = desired_amount[i]))
order(stock, desired_amount[i])

def ReInvest(context, data):
for i, stock in enumerate(context.stocks):
prices = np.zeros_like(context.b_t)
desired_amount = np.zeros_like(prices)
prices[i] = data[stock].price
desired_amount[i] = np.round(context.portfolio.cash / context.m / prices[i])
if desired_amount[i] >0:
log.info("reinvest Bought {ticker} @ {price} for {amount} shares".format(ticker = stock, price = prices[i], amount = desired_amount[i]))
log.info("cash: {cash}".format(cash = context.portfolio.cash))
order(stock, desired_amount[i])
This backtest was created using an older version of the backtester. Please re-run this backtest to see results using the latest backtester. Learn more about the recent changes.
There was a runtime error.

Hi Huapu, To address your question directly, visually it may appear that the volatility of the MVP is higher than that of the buy and hold strategy but a quantitative measure would be more interesting. If you record the returns of both portfolios in a list then take the standard deviation of the list that could prove to be a nice one number comparison. I would not be surprised with either algorithm having the higher volatility but volatility can be deceiving visually. Another point to note is that in the second algorithm you have chosen to use an equal weighted portfolio. It is my feeling that comparing the MVP to a cap weighted buy and hold strategy could be more of what you meant to do. Equal weighted is actually really quite good and there has been quite a bit of research done on this. Edit: I found the reference I was looking for here

One pitfall of this Markowitz type of analysis is the curse of dimensionality. You have 43 stocks which means that you are estimating the covariance matrix containing 43*42/2+43 = 9,073 parameters utilizing just 252 observations. This estimator is going to have a high degree of variability. You are going to want to increase the length of your observation window to tighten the standard errors on the covariance estimates. 90,000 days would probably be sufficient for nice tight estimation with 43 assets. Obviously that size of estimation window is probably unreasonable which is why the next step would be to implement dimension reduction techniques. An exogenous factor model could work nicely, the principal components approach is another methodology or, the factors on demand methodology of Meucci. Asymptotic principal components by Connor and Korajczyk could be an excellent solution with a very large number of assets selected using the universe feature of quantopian.

Hope this helps.

Short version: Use dimension reduction techniques and compare to a capitalization weighted portfolio

Hi Wayne,

Thank you very much for your very detailed explanation. If you can forgive my ignorance in finance, I would like to ask you a few more question.

1. Regarding cap weighted portfolio, does it mean that I assign the weight according to the market cap of a stock? I don't know an easy way of implementing it in the Quantopian framework as it is not obvious to get time varying market cap value in Quantopian. I appreciate it if you have any suggestion on this.

2. If I understand your algo correctly, your algo is different from calculating the efficient frontier as described in the common Modern Portfolio Theory, like the one described in Wiki: http://en.wikipedia.org/wiki/Modern_portfolio_theory. The common MPT tries to maximize the difference between expected return and variance by calculating the tangent line of the efficient frontier with an intercept of risk free return, and only positive weight are allowed for each component in the portfolio. However, your algo calculates the min variance, and your calculation results allows negative weights. So you are hedging to get min variance. Is there a reference you can provide to your algo so that we can better understand the theory behind it?

Thanks a lot

Regarding cap weighted portfolio, does it mean that I assign the weight according to the market cap of a stock? I don't know an easy way of implementing it in the Quantopian framework as it is not obvious to get time varying market cap value in Quantopian. I appreciate it if you have any suggestion on this.

That is absolutely correct, market cap data may be retrieved via quandl and the fetch_csv functionality. Most equity indexes are market cap weighted including the S&P 500 with the glaring exception of the Dow which is price weighted.

If I understand your algo correctly, your algo is different from calculating the efficient frontier as described in the common Modern Portfolio Theory, like the one described in Wiki: http://en.wikipedia.org/wiki/Modern_portfolio_theory. The common MPT tries to maximize the difference between expected return and variance by calculating the tangent line of the efficient frontier with an intercept of risk free return, and only positive weight are allowed for each component in the portfolio. However, your algo calculates the min variance, and your calculation results allows negative weights. So you are hedging to get min variance. Is there a reference you can provide to your algo so that we can better understand the theory behind it?

This algorithm is not designed to calculate the efficient frontier just the minimum variance portfolio which is the "nose of the bullet" of the efficient frontier so to speak. The minimum variance portfolio is solved by a simple lagrange multiplier based approach that we (re)derived ( this is a classic result ) last year http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2331085 page 10 appendix B.

Edit: Grant posted a very nice write up on the subject see page 8 of this written up by Eric Zivot from the University of Washington's Economics department.

I would like to use same on Indian market with following yahoo tickers, and bench mark is "NIFTY". Please guide
LT.NS BRITANNIA.NS HDFCBANK.BO INDUSINDBK.NS ONGC.NS AXISBANK.NS CAIRN.NS TATAMOTOR.NS BAJAJFINSV.NS TECHM.NS FEDERALBNK.NS BANKBAROD.NS BPCL.NS HINDPETRO.BO KEC.NS JKLAKSHMI.NS CESC.NS ICICIBANK.NS SBIN.NS IDFC.NS