Back to Community
long-only minimum variance portfolio using scipy.optimize.minimize

Here's an example of a long-only minimum variance portfolio using scipy.optimize.minimize. Securities are:

    context.stocks = [ sid(19662),  # XLY Consumer Discrectionary SPDR Fund  
                       sid(19656),  # XLF Financial SPDR Fund  
                       sid(19658),  # XLK Technology SPDR Fund  
                       sid(19655),  # XLE Energy SPDR Fund  
                       sid(19661),  # XLV Health Care SPRD Fund  
                       sid(19657),  # XLI Industrial SPDR Fund  
                       sid(19659),  # XLP Consumer Staples SPDR Fund  
                       sid(19654),  # XLB Materials SPDR Fund  
                       sid(19660),  # XLU Utilities SPRD Fund  
                       sid(33652)]  # BND Vanguard Total Bond Market ETF  

Grant

Clone Algorithm
300
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import numpy as np
from pytz import timezone
import scipy

trading_freq = 20

def initialize(context):
    
    context.stocks = [ sid(19662),  # XLY Consumer Discrectionary SPDR Fund
                       sid(19656),  # XLF Financial SPDR Fund
                       sid(19658),  # XLK Technology SPDR Fund
                       sid(19655),  # XLE Energy SPDR Fund
                       sid(19661),  # XLV Health Care SPRD Fund
                       sid(19657),  # XLI Industrial SPDR Fund
                       sid(19659),  # XLP Consumer Staples SPDR Fund
                       sid(19654),  # XLB Materials SPDR Fund
                       sid(19660),  # XLU Utilities SPRD Fund
                       sid(33652)]  # BND Vanguard Total Bond Market ETF
    
    context.x0 = 1.0*np.ones_like(context.stocks)/len(context.stocks)

    set_commission(commission.PerShare(cost=0.013, min_trade_cost=1.3))
    
    context.day_count = -1

def handle_data(context, data):
     
    # Trade only once per day
    loc_dt = get_datetime().astimezone(timezone('US/Eastern'))
    if loc_dt.hour == 16 and loc_dt.minute == 0:
        context.day_count += 1
        pass
    else:
        return
    
    # Limit trading frequency
    if context.day_count % trading_freq != 0.0:
        return
    
    prices = history(21,'1d','price').as_matrix(context.stocks)
    ret = np.diff(prices,axis=0) # daily returns
    
    bnds = ((0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1))
    cons = ({'type': 'eq', 'fun': lambda x:  np.sum(x)-1.0})
    
    res= scipy.optimize.minimize(variance, context.x0, args=ret,method='SLSQP',constraints=cons,bounds=bnds)
    
    allocation = res.x
    allocation[allocation<0]=0
    denom = np.sum(allocation)
    if denom != 0:
        allocation = allocation/denom
        
    context.x0 = allocation
        
    record(stocks=np.sum(allocation[0:-1]))
    record(bonds=allocation[-1])
    
    for i,stock in enumerate(context.stocks):
        order_target_percent(stock,allocation[i])
        
def variance(x,*args):
    
    p = np.asarray(args)
    Acov = np.cov(p.T)
    
    return np.dot(x,np.dot(Acov,x))
There was a runtime error.
13 responses

Hey Grant,

I cloned your trade and tried it on levered ETFs. I need to find some with a longer price history or maybe try using levered equity ETFs with unlevered bond ETFs to get something that includes the financial crisis.

Thanks for sharing.

JW

Clone Algorithm
50
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import numpy as np
from pytz import timezone
import scipy

trading_freq = 20

def initialize(context):
    
    context.stocks = [ sid(33263),  # UYM Proshares Ultra Basic Materials
                       sid(33269),  # UGE Proshares Ultra Consumer Goods
                       sid(33268),  # UCC Proshares Ultra Consumer Services
                       sid(33265),  # UTG Proshares Ultra Financials
                       sid(33270),  # RXL Proshares Ultra Healthcare
                       sid(33261),  # UXI Proshares Ultra Industrials
                       sid(25960),  # DIG Proshares Ultra Oil&Gas
                       sid(33262),  # ROM Proshares Ultra Technology
                       sid(35968),  # LTL Proshares Ultra Telecommunications
                       sid(33267)]  # UPW Proshares Ultra Utilities
    
    context.x0 = 1.0*np.ones_like(context.stocks)/len(context.stocks)

    set_commission(commission.PerShare(cost=0.013, min_trade_cost=1.3))
    
    context.day_count = -1

def handle_data(context, data):
     
    # Trade only once per day
    loc_dt = get_datetime().astimezone(timezone('US/Eastern'))
    if loc_dt.hour == 16 and loc_dt.minute == 0:
        context.day_count += 1
        pass
    else:
        return
    
    # Limit trading frequency
    if context.day_count % trading_freq != 0.0:
        return
    
    prices = history(21,'1d','price').as_matrix(context.stocks)
    ret = np.diff(prices,axis=0) # daily returns
    
    bnds = ((0.01,1),(0.01,1),(0.01,1),(0.01,1),(0.01,1),(0.01,1),(0.01,1),(0.01,1),(0.01,1),(0.01,1))
    cons = ({'type': 'eq', 'fun': lambda x:  np.sum(x)-1.0})
    
    res= scipy.optimize.minimize(variance, context.x0, args=ret,method='SLSQP',constraints=cons,bounds=bnds)
    
    allocation = res.x
    allocation[allocation<0]=0
    denom = np.sum(allocation)
    if denom != 0:
        allocation = allocation/denom
        
    context.x0 = allocation
        
    record(stocks=np.sum(allocation[0:-1]))
    record(bonds=allocation[-1])
    
    for i,stock in enumerate(context.stocks):
        order_target_percent(stock,allocation[i])
        
def variance(x,*args):
    
    p = np.asarray(args)
    Acov = np.cov(p.T)
    
    return np.dot(x,np.dot(Acov,x))
There was a runtime error.

J.,

Glad you revived it! Have fun.

Grant

Very cool, I added a leverage parameter and made some small tweaks.

Clone Algorithm
88
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import numpy as np
from pytz import timezone
import scipy

trading_freq = 242/2 # twice a year 

def initialize(context):
    
    context.stocks = [ sid(19662),  # XLY Consumer Discrectionary SPDR Fund
                       sid(19656),  # XLF Financial SPDR Fund
                       sid(19658),  # XLK Technology SPDR Fund
                       sid(19655),  # XLE Energy SPDR Fund
                       sid(19661),  # XLV Health Care SPRD Fund
                       sid(19657),  # XLI Industrial SPDR Fund
                       sid(19659),  # XLP Consumer Staples SPDR Fund
                       sid(19654),  # XLB Materials SPDR Fund
                       sid(19660) ]  # XLU Utilities SPRD Fund
                       #sid(33652)]  # BND Vanguard Total Bond Market ETF
    
    context.leverage = 2
    
    context.x0 = 1.0*np.ones_like(context.stocks)/len(context.stocks)

    set_commission(commission.PerShare(cost=0.013, min_trade_cost=1.3))
    
    context.day_count = -1

def handle_data(context, data):
     
    # Trade only once per day
    loc_dt = get_datetime().astimezone(timezone('US/Eastern'))
    if loc_dt.hour == 16 and loc_dt.minute == 0:
        context.day_count += 1
        pass
    else:
        return
    
    # Limit trading frequency
    if context.day_count % trading_freq != 0.0:
        return
    
    prices = history(21,'1d','price').as_matrix(context.stocks)
    ret = np.diff(prices,axis=0) # daily returns
    
    bnds = [(0,1)]*len(context.stocks) 
    cons = ({'type': 'eq', 'fun': lambda x:  np.sum(x)-1.0})
    
    res= scipy.optimize.minimize(variance, context.x0, args=ret,method='SLSQP',constraints=cons,bounds=bnds)
    
    allocation = res.x
    allocation[allocation<0]=0
    denom = np.sum(allocation)
    if denom != 0:
        allocation = allocation/denom
        allocation = allocation * context.leverage
        
    context.x0 = allocation
    
    for i,stock in enumerate(context.stocks):
        order_target_percent(stock,allocation[i])
        
def variance(x,*args):
    
    p = np.asarray(args)
    Acov = np.cov(p.T)
    
    return np.dot(x,np.dot(Acov,x))
There was a runtime error.

Note that the minimum variance computation is a kind of trivial example, since a closed-form solution exists (e.g. http://faculty.washington.edu/ezivot/econ424/portfolioTheoryMatrix.pdf). Folks might consider problems that actually require an optimizer. My understanding is that some of the Python routines will accept inequality constraints, which might be of interest. --Grant

The code I posted is no longer running. I get the error:

Something went wrong. Sorry for the inconvenience. Try using the built-in debugger to analyze your code. If you would like help, send us an email.
ValueError: shapes (10,20,1) and (1,20,10) not aligned: 1 (dim 2) != 20 (dim 1)
There was a runtime error on line 48.

I suspect it is related to the recent update of numpy, etc. Please advise how to fix the code.

Thanks,

Grant

You got the daily returns (10,20) matrix ret that is passed into the variance function by scipy.optimize.minimize as (10,20,1) matrix arg.
I don't know why it does it, but here is a fix using numpy.squeeze:

Change like 64 for

p = np.squeeze(np.asarray(args))  

Here's the fixed version (thanks Alexandre), in case someone ends up cloning it. Odd that low-level numpy would be changed? Kinda disturbing...

Clone Algorithm
607
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# https://www.quantopian.com/posts/long-only-minimum-variance-portfolio-using-scipy-dot-optimize-dot-minimize

import numpy as np
from pytz import timezone
import scipy

trading_freq = 20

def initialize(context):
    
    context.stocks = [ sid(19662),  # XLY Consumer Discrectionary SPDR Fund
                       sid(19656),  # XLF Financial SPDR Fund
                       sid(19658),  # XLK Technology SPDR Fund
                       sid(19655),  # XLE Energy SPDR Fund
                       sid(19661),  # XLV Health Care SPRD Fund
                       sid(19657),  # XLI Industrial SPDR Fund
                       sid(19659),  # XLP Consumer Staples SPDR Fund
                       sid(19654),  # XLB Materials SPDR Fund
                       sid(19660),  # XLU Utilities SPRD Fund
                       sid(33652)]  # BND Vanguard Total Bond Market ETF
    
    context.x0 = 1.0*np.ones_like(context.stocks)/len(context.stocks)
    
    context.day_count = -1

def handle_data(context, data):
     
    # Trade only once per day
    loc_dt = get_datetime().astimezone(timezone('US/Eastern'))
    if loc_dt.hour == 16 and loc_dt.minute == 0:
        context.day_count += 1
        pass
    else:
        return
    
    # Limit trading frequency
    if context.day_count % trading_freq != 0.0:
        return
    
    prices = history(21,'1d','price').as_matrix(context.stocks)
    ret = np.diff(prices,axis=0) # daily returns
    
    bnds = ((0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1))
    cons = ({'type': 'eq', 'fun': lambda x:  np.sum(x)-1.0})
    
    res= scipy.optimize.minimize(variance, context.x0, args=ret,method='SLSQP',constraints=cons,bounds=bnds)
    
    allocation = res.x
    allocation[allocation<0]=0
    denom = np.sum(allocation)
    if denom != 0:
        allocation = allocation/denom
        
    context.x0 = allocation
        
    record(stocks=np.sum(allocation[0:-1]))
    record(bonds=allocation[-1])
    
    for i,stock in enumerate(context.stocks):
        order_target_percent(stock,allocation[i])
        
def variance(x,*args):
    
    p = np.squeeze(np.asarray(args))
    Acov = np.cov(p.T)
    
    return np.dot(x,np.dot(Acov,x))
We have migrated this algorithm to work with a new version of the Quantopian API. The code is different than the original version, but the investment rationale of the algorithm has not changed. We've put everything you need to know here on one page.
There was a runtime error.

You're welcome, Grant.
When I read your code, I saw that long-only meant that after optimization, you set short positions to zero and invest in bonds instead.
I think that if the optimizer found a minimum variance taking shorts into account and you take those shorts out, you're get a new variance. Would that be a minimum? So wouldn't be better to change the constraint from sum of weights equals to 1 to something that checks both sum of weights equal to 1 and every weight greater or equal to zero?

Well, go for it! See what you can come up with. Worst case, you might learn some Python (or maybe you're an expert), and it is cheaper than Lumosity. My hunch is that if you search, you'll find some literature on the topic...I just don't know. --Grant

Maybe I'm misunderstanding how the minimizer works, but it seems to be disregarding the bounds in this line:
bnds = ((0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1))

Shouldn't those bounds preclude shorts?

I've seen some discussion elsewhere about the SLSQP method disregarding bounds.

Might not be working properly, but I have gotten the bounds to be respected in other algos. If you sort it out, please post here. --Grant

@Grant, editing line 64 of the code will get the algo to compile again after the platform upgrade.

Changing it from,

    p = np.asarray(args)  

to,

p = np.squeeze(np.asarray(args))  
Clone Algorithm
51
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import numpy as np
from pytz import timezone
import scipy

trading_freq = 20

def initialize(context):
    
    context.stocks = [ sid(19662),  # XLY Consumer Discrectionary SPDR Fund
                       sid(19656),  # XLF Financial SPDR Fund
                       sid(19658),  # XLK Technology SPDR Fund
                       sid(19655),  # XLE Energy SPDR Fund
                       sid(19661),  # XLV Health Care SPRD Fund
                       sid(19657),  # XLI Industrial SPDR Fund
                       sid(19659),  # XLP Consumer Staples SPDR Fund
                       sid(19654),  # XLB Materials SPDR Fund
                       sid(19660),  # XLU Utilities SPRD Fund
                       sid(33652)]  # BND Vanguard Total Bond Market ETF
    
    context.x0 = 1.0*np.ones_like(context.stocks)/len(context.stocks)

    set_commission(commission.PerShare(cost=0.013, min_trade_cost=1.3))
    
    context.day_count = -1

def handle_data(context, data):
     
    # Trade only once per day
    loc_dt = get_datetime().astimezone(timezone('US/Eastern'))
    if loc_dt.hour == 16 and loc_dt.minute == 0:
        context.day_count += 1
        pass
    else:
        return
    
    # Limit trading frequency
    if context.day_count % trading_freq != 0.0:
        return
    
    prices = history(21,'1d','price').as_matrix(context.stocks)
    ret = np.diff(prices,axis=0) # daily returns
    
    bnds = ((0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1))
    cons = ({'type': 'eq', 'fun': lambda x:  np.sum(x)-1.0})
    
    res= scipy.optimize.minimize(variance, context.x0, args=ret,method='SLSQP',constraints=cons,bounds=bnds)
    
    allocation = res.x
    allocation[allocation<0]=0
    denom = np.sum(allocation)
    if denom != 0:
        allocation = allocation/denom
        
    context.x0 = allocation
        
    record(stocks=np.sum(allocation[0:-1]))
    record(bonds=allocation[-1])
    
    for i,stock in enumerate(context.stocks):
        order_target_percent(stock,allocation[i])
        
def variance(x, *args):
    
    p = np.squeeze(np.asarray(args))
    Acov = np.cov(p.T)
    
    return np.dot(x,np.dot(Acov,x))
There was a runtime error.
Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Errors....

ValueError: shapes (9,20,1) and (1,20,9) not aligned: 1 (dim 2) != 20 (dim 1)
------> There was a runtime error on line 48.