Back to Community
ChiPy FinSIG - Nov 2nd - Portfolio Optimization

Hi Chipy,

Here is a link to the presentation for tonight. Thanks again to RedShelf for providing the space and Quantopian for the pizza.

The first algorithm is the Single Factor Model with ordering done with the optimization engine. As the conversation turned to learning tools, a great way to learn is to clone and tweak other strategies. Quantopian forums are full of great stuff.

Clone Algorithm
Total Returns
Max Drawdown
Benchmark Returns
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
from quantopian.algorithm import attach_pipeline, pipeline_output, order_optimal_portfolio
from quantopian.pipeline import Pipeline
from quantopian.pipeline import CustomFactor
from import USEquityPricing
from quantopian.pipeline.filters import Q1500US
from import Fundamentals
from quantopian.pipeline.classifiers.fundamentals import Sector
import quantopian.optimize as opt
import numpy as np
import pandas as pd

#Fundamental Factors
class ev_ebitda(CustomFactor):
    window_length = 1
    inputs = [Fundamentals.ev_to_ebitda]
    def compute(self, today, assets, out, ev_to_ebitda):
        out[:] = np.where(ev_to_ebitda <= 0, 0, 1/ev_to_ebitda)

class PriceReturn(CustomFactor):  
    inputs = [USEquityPricing.close]
    window_length = 2
    def compute(self, today, assets, out, close):
        out[:] = close[-1]/close[0]
def initialize(context):
    Called once at the start of the algorithm.
    # Rebalance every day, 1 hour after market open.
    schedule_function(rebalance, date_rules.month_start(1), time_rules.market_open(hours=1))
    # Record tracking variables at the end of each day.
    schedule_function(my_record_vars, date_rules.every_day(), time_rules.market_close())
    # Create our dynamic stock selector.
    attach_pipeline(the_pipe(), 'factors')

def the_pipe():

    universe = Q1500US()
    pipe = Pipeline(
        'EV_EBITDA': ev_ebitda(mask=universe),
        #'PriceReturn': PriceReturn(mask=universe),
        'Sector': Sector(mask=universe)},
    return pipe

def before_trading_start(context, data):
    results = pipeline_output('factors').dropna()
    factor_name = 'EV_EBITDA'
    results['factor'] = results.groupby(['Sector'])[factor_name].apply(lambda x: x.fillna(x.median()))
    zscore = lambda x: (x - x.mean()) / x.std()
    results['factor_score'] = results.groupby(['Sector'])['factor'].transform(zscore)
    results['factor_adj'] = np.where(abs(results['factor_score']) > 3,np.nan,results['factor'])
    results['factor_adj'] = results.groupby(['Sector'])['factor_adj'].apply(lambda x: x.fillna(x.median()))
    results['factor_score_adj'] = results.groupby(['Sector'])['factor_adj'].transform(zscore)
    #results['factor_qtile'] = results.groupby(['Sector'])['factor_score_adj'].transform(lambda x: pd.cut(x, bins=5))
    results['factor_qtile'] = np.where(results['factor_score_adj'] > 2.5,5,3)
    results['factor_qtile'] = np.where(results['factor_score_adj'] <-2,1,results['factor_qtile'])
    results['factor_qtile'] = results['factor_qtile'].fillna(3)
    results['trade_type'] = np.where(results['factor_qtile']==1,'Short','No Action')
    results['trade_type'] = np.where(results['factor_qtile']==5,'Long', results['trade_type'])
    results['weight'] = results.groupby(['trade_type'])['factor_score_adj'].transform(lambda x: x/sum(x))
    results['weight'] = np.where(results['trade_type']=='Short',-1*results['weight'],results['weight'])
    context.output = results[['Sector',factor_name,'factor_score_adj','factor_qtile','trade_type','weight']]      
    context.security_list = results.index.tolist()

def my_record_vars(context, data):
    Plot variables at the end of each day.
    long_count = 0
    short_count = 0

    for position in context.portfolio.positions.itervalues():
        if position.amount > 0:
            long_count += 1
        if position.amount < 0:
            short_count += 1
    # Plot the counts
    record(num_long=long_count, num_short=short_count, leverage=context.account.leverage)

def rebalance(context, data):
    # Compute our portfolio weights.
    target_weights = context.output[context.output['trade_type'].isin(['Long','Short'])]['weight']
    W_max = 1
    min_weights = -.25
    max_weights = .2
    #objective = opt.MaximizeAlpha(context.output['factor_score_adj'])
    objective = opt.TargetWeights(target_weights)
    constraints = [opt.MaxGrossExposure(W_max),
                   opt.PositionConcentration.with_equal_bounds(min_weights, max_weights),
    order_optimal_portfolio(objective, constraints)
There was a runtime error.
9 responses

Here is the notebook with the discussion on portfolio optimization. The blog post from which this was derived is here.

There is probably some capacity for someone with a little better math background to explain the finer points of convex optimization.

Click to load notebook preview

And lastly, in case you are interested here is the original AlphaLens factor work for the EV/EBITDA factor. Watch out on this one, you can easily exceed the memory limit for notebooks.

Click to load notebook preview

Awesome post with some great tools to learn about alpha and optimization.


The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.


The Markowitz Bullet does not build when I change the 4 stocks for 25 S&P 500 stocks and 4 ETFs, (see attached). I want to replicate one of my academic papers - 'Modelling risk reduction in equity portfolios using exchange-traded funds with multiple variance-covariance (VCV) matrices."

Have you tried a Shrinkage method on the VCV matrix? Ledoit and Wolf (2004) state that it improves the accuracy of the tangency portfolio and the GMVP when compared to the Markowitz (1952) matrix or the Single Index Model (Sharpe, 1963) matrix. My example will be looking at minimising portfolio risk by using the GMVP coupled with a weighting in the risk-free rate. This will allow us to be along the Capital Allocation Line.


Please see these journals;
Ledoit, O. and Wolf, M. (2003) ‘Improved estimation of the covariance matrix of stock returns with an application to portfolio selection’, Journal of Empirical Finance 10 (5), 603-621.
Ledoit, O. and Wolf. M. (2004). ‘A-well conditioned estimator for large-dimensional covariance matrices’, Journal of Multivariate Analysis 88 (2), 365-411.
Ledoit, O. and Wolf. M. (2004) ‘Honey, I shrunk the sample covariance matrix’, Journal of Portfolio Management 30 (4), 110-119.
Markowitz, H. M. (1952) ‘Portfolio Selection’, Journal of Finance 7 (1), 77-91.
Markowitz, H. M. (1959) ‘Portfolio Selection: Efficient diversification of investment’, Edition 2.
Sharpe, W. F. (1963). “A Simplified Model for Portfolio Analysis,” Management Science 9 (2), 277-293.
Sharpe, W.F. (1964) Capital Asset Prices: A Theory of Market Equilibrium, Journal of Finance 19 (3), 425-442.

Click to load notebook preview

I ran a risk breakdown of the algorithm using Quantopian's new risk model. Looks like it's a prime candidate for tighter risk controls, which are discussed in this lecture -

Click to load notebook preview

Delaney wastes no time in showing of the new risk model. I love it. I like how the alpha calculated in zipline for the EV/EBITDA strategy shows up as spread between total and specific risk. I definitely agree about the risk controls. Spikes on serveral types of risk. The (high) bias towards value is expected given the factor used, but I was surprised at how consistently high (in absolute value) the volatility exposure was throughout. Additionally, the spikes in momentum risk are interesting, that is probably driven by the EV. I guess this is the content for my next Chipy talk - evaluating and constraining risk exposures.

Frank -
Sorry I meant to reply sooner. There are some NaN's in the covarience matrix when you add the additional assets. That causes the random portfolio generator to fail. You can fix that by dropping the offending asset or by setting the nan's in the matrix to a nominal value - of course that will hurt your ability to optimize, but it is an okay workaround. I'll share some code that does that. When I modified the covarience matrix, I still didn't get a great looking bullet, but that is because with 25 assets you now have many more possible portfolios. You could get a nicer looking scatter plot by ensuring the companies that have higher returns generally have higher weights - so not truly random weights.

Regarding the optimization, I have never worked with any other methods. The documents you share look interesting. I do think that there is some scope both in Q and for someone at Chipy (if anyone is reading this) to explore other methods of optimization. Certainly at higher speeds, the efficiency of the process would start to really matter. But for now, as a illustration of the process of optimization the cvx worked quick enough with enough accuracy to show the general process and value of optimization.

Awesome, we hope to have some new materials out shortly on how to detect risk exposure at the model level as well, as opposed to going all the way down to the portfolio level.

Now that the MaximizeAlpha objective has been in the wild for a while, I'm curious if it's still valuable to find optimal portfolio weights in the way you did in the portfolio optimization notebook example above. Are there any examples of algos on Quantopian using weights generated like that? I've only found examples in user posted notebooks and tutorial notebooks. Appreciate any insight! thanks so much for sharing this.

That's a good question. The optimization process demonstrated above was more to teach a concept than to find a best portfolio. In Quantopian's platform the best approach is probably to use Quantopian resources.