Back to Community
Quantopian Lecture Series: The Art of Not Following the Market

Algorithms that do not follow the market are very attractive to investors. We will discuss some approaches for reducing correlation to a benchmark and discuss why returns aren’t everything. Please find some algorithms demonstrating concepts in this notebook in the replies to this post.

In addition, we have released a notebook primer on linear regression here. Linear regression is used in determining beta for beta hedging, so those who are a little rusty may want to check it out first.

This is the first in Quantopian’s Summer Lecture Series. We are currently developing a quant finance curriculum and will be releasing clone-able notebooks and algorithms to teach key concepts. This notebook will be presented at this meetup. Stay tuned for more.

Credit for the notebooks goes to Jenny Nitishinskaya, and credit for the algorithms goes to David Edwards.

Loading notebook preview...
Notebook previews are currently unavailable.
Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

14 responses

Nice! I'd be curious to see how TSLA hedged using a Kalman filter beta estimator looks.

Here is an example of an algorithm which holds a long-only portfolio based on earnings yield, and does not implement any hedging or risk management. Notice its performance over the 08-09 period.

Clone Algorithm
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import pandas as pd
import numpy as np
import statsmodels.api as sm
import math


def initialize(context):
    use_beta_hedging = False # Set to False to trade unhedged
    # Initialize a universe of liquid assets
    schedule_function(buy_assets,
                      date_rule=date_rules.month_start(),
                      time_rule=time_rules.market_open())
    if use_beta_hedging:
        # Call the beta hedging function one hour later to
        # make sure all of our orders have gone through.
        schedule_function(hedge_portfolio,
                          date_rule=date_rules.week_start(),
                          time_rule=time_rules.market_open(hours=1))
    # trading days used for beta calculation
    context.lookback = 150
    # Used to aviod purchasing any leveraged ETFs 
    context.dont_buys = security_lists.leveraged_etf_list
    # Current allocation per asset
    context.pct_per_asset = 0
    context.index = symbol('SPY')
    
def before_trading_start(context):
    # Number of stocks to find
    num_stocks = 100
    fundamental_df = get_fundamentals(
        query(
            # To add a metric. Start by typing "fundamentals."
            fundamentals.valuation_ratios.earning_yield,
            fundamentals.valuation.market_cap,
        )
        .filter(fundamentals.valuation.market_cap > 1e9)
        .order_by(fundamentals.valuation_ratios.earning_yield.desc())
        .limit(num_stocks)
    )
    update_universe(fundamental_df)
    
def buy_assets(context, data):
    all_prices = history(1, '1d', 'price', ffill=True)
    eligible_assets = [asset for asset in all_prices
                       if asset not in context.dont_buys 
                       and asset != context.index]
    pct_per_asset = 1.0 / len(eligible_assets)
    context.pct_per_asset = pct_per_asset
    for asset in eligible_assets:
        # Some assets might cause a key error due to being delisted 
        # or some other corporate event so we use a try/except statement
        try:
            if get_open_orders(sid=asset):
                continue
            order_target_percent(asset, pct_per_asset)
        except:
            log.warn("[Failed Order] asset = %s"%asset.symbol)
           
    
def hedge_portfolio(context, data):
    """
    This function places an order for "context.index" in the 
    amount required to neutralize the beta exposure of the portfolio.
    Note that additional leverage in the account is taken on, however,
    net market exposure is reduced.
    """
    factors = get_alphas_and_betas(context, data)
    beta_exposure = 0.0
    count = 0
    for asset in context.portfolio.positions:
        if asset in factors and asset != context.index:
            if not np.isnan(factors[asset].beta):
                beta_exposure += factors[asset].beta
                count += 1
    beta_hedge = -1.0 * beta_exposure / count
    dollar_amount = context.portfolio.portfolio_value * beta_hedge
    record(beta_hedge=beta_hedge)
    if not np.isnan(dollar_amount):
        order_target_value(context.index, dollar_amount)
    
def get_alphas_and_betas(context, data):
    """
    returns a dataframe of 'alpha' and 'beta' exposures 
    for each asset in the current universe.
    """
    prices = history(context.lookback, '1d', 'price', ffill=True)
    returns = prices.pct_change()[1:]
    index_returns = returns[context.index]
    factors = {}
    for asset in context.portfolio.positions:
        try:
            X = returns[asset]
            factors[asset] = linreg(X, index_returns)
        except:
            log.warn("[Failed Beta Calculation] asset = %s"%asset.symbol)
    return pd.DataFrame(factors, index=['alpha', 'beta'])
    
def linreg(x, y):
    # We add a constant so that we can also fit an intercept (alpha) to the model
    # This just adds a column of 1s to our data
    X = sm.add_constant(x)
    model = sm.OLS(y, X).fit()
    return model.params[0], model.params[1]

def handle_data(context, data):
    record(net_exposure=context.account.net_leverage,
           leverage=context.account.leverage)
    
There was a runtime error.

Here is the same long-only earnings yield strategy, but with a beta hedge implemented. Because we can never actually know beta, only estimate it, the reduction in beta is significant, but not great. The performance is better and it doesn't take the same hit as the first.

Clone Algorithm
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import pandas as pd
import numpy as np
import statsmodels.api as sm
import math


def initialize(context):
    use_beta_hedging = True # Set to False to trade unhedged
    # Initialize a universe of liquid assets
    schedule_function(buy_assets,
                      date_rule=date_rules.month_start(),
                      time_rule=time_rules.market_open())
    if use_beta_hedging:
        # Call the beta hedging function one hour later to
        # make sure all of our orders have gone through.
        schedule_function(hedge_portfolio,
                          date_rule=date_rules.week_start(),
                          time_rule=time_rules.market_open(hours=1))
    # trading days used for beta calculation
    context.lookback = 150
    # Used to aviod purchasing any leveraged ETFs 
    context.dont_buys = security_lists.leveraged_etf_list
    # Current allocation per asset
    context.pct_per_asset = 0
    context.index = symbol('SPY')
    
def before_trading_start(context):
    # Number of stocks to find
    num_stocks = 100
    fundamental_df = get_fundamentals(
        query(
            # To add a metric. Start by typing "fundamentals."
            fundamentals.valuation_ratios.earning_yield,
            fundamentals.valuation.market_cap,
        )
        .filter(fundamentals.valuation.market_cap > 1e9)
        .order_by(fundamentals.valuation_ratios.earning_yield.desc())
        .limit(num_stocks)
    )
    update_universe(fundamental_df)
    
def buy_assets(context, data):
    all_prices = history(1, '1d', 'price', ffill=True)
    eligible_assets = [asset for asset in all_prices
                       if asset not in context.dont_buys 
                       and asset != context.index]
    pct_per_asset = 1.0 / len(eligible_assets)
    context.pct_per_asset = pct_per_asset
    for asset in eligible_assets:
        # Some assets might cause a key error due to being delisted 
        # or some other corporate event so we use a try/except statement
        try:
            if get_open_orders(sid=asset):
                continue
            order_target_percent(asset, pct_per_asset)
        except:
            log.warn("[Failed Order] asset = %s"%asset.symbol)
           
    
def hedge_portfolio(context, data):
    """
    This function places an order for "context.index" in the 
    amount required to neutralize the beta exposure of the portfolio.
    Note that additional leverage in the account is taken on, however,
    net market exposure is reduced.
    """
    factors = get_alphas_and_betas(context, data)
    beta_exposure = 0.0
    count = 0
    for asset in context.portfolio.positions:
        if asset in factors and asset != context.index:
            if not np.isnan(factors[asset].beta):
                beta_exposure += factors[asset].beta
                count += 1
    beta_hedge = -1.0 * beta_exposure / count
    dollar_amount = context.portfolio.portfolio_value * beta_hedge
    record(beta_hedge=beta_hedge)
    if not np.isnan(dollar_amount):
        order_target_value(context.index, dollar_amount)
    
def get_alphas_and_betas(context, data):
    """
    returns a dataframe of 'alpha' and 'beta' exposures 
    for each asset in the current universe.
    """
    prices = history(context.lookback, '1d', 'price', ffill=True)
    returns = prices.pct_change()[1:]
    index_returns = returns[context.index]
    factors = {}
    for asset in context.portfolio.positions:
        try:
            X = returns[asset]
            factors[asset] = linreg(X, index_returns)
        except:
            log.warn("[Failed Beta Calculation] asset = %s"%asset.symbol)
    return pd.DataFrame(factors, index=['alpha', 'beta'])
    
def linreg(x, y):
    # We add a constant so that we can also fit an intercept (alpha) to the model
    # This just adds a column of 1s to our data
    X = sm.add_constant(x)
    model = sm.OLS(y, X).fit()
    return model.params[0], model.params[1]

def handle_data(context, data):
    record(net_exposure=context.account.net_leverage,
           leverage=context.account.leverage)
    
There was a runtime error.

Here is an example of the same earnings yield strategy implemented as long short equity. In this case we long the top 50 equities ranked by earnings yield and short the bottom 50. We are betting on the returns spread of our ranking and the beta in this strategy is very low.

Clone Algorithm
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import pandas as pd
import numpy as np
import statsmodels.api as sm
import math


def initialize(context):
    use_beta_hedging = False # Set to False to trade unhedged
    # Initialize a universe of liquid assets
    schedule_function(buy_assets,
                      date_rule=date_rules.month_start(),
                      time_rule=time_rules.market_open())
    if use_beta_hedging:
        # Call the beta hedging function one hour later to
        # make sure all of our orders have gone through.
        schedule_function(hedge_portfolio,
                          date_rule=date_rules.month_start(),
                          time_rule=time_rules.market_open(hours=1))
    # trading days used for beta calculation
    context.lookback = 250
    # Used to aviod purchasing any leveraged ETFs 
    context.dont_buys = security_lists.leveraged_etf_list
    # Current allocation per asset
    context.pct_per_asset = 0
    context.index = symbol('SPY')
    
def before_trading_start(context):
    # Number of stocks to find
    num_stocks = 100
    
    fundamental_df = get_fundamentals(
        query(
            # To add a metric. Start by typing "fundamentals."
            fundamentals.valuation_ratios.earning_yield,
            fundamentals.valuation.market_cap,
        )
        .filter(fundamentals.valuation.market_cap > 1e9)
        .order_by(fundamentals.valuation_ratios.earning_yield.desc())
    )
    
    fundamental_df
    yields = fundamental_df.ix['earning_yield']
    context.shorts = yields.tail(num_stocks/2)
    context.longs = yields.head(num_stocks/2)
    # fundamental_df = pd.concat([context.shorts, context.longs])
    all_assets = context.shorts.append(context.longs).index
    update_universe(all_assets)
    
def buy_assets(context, data):
    all_prices = history(1, '1d', 'price', ffill=True).dropna(axis=1)
    eligible_assets = [asset for asset in all_prices
                       if asset not in context.dont_buys 
                       and asset != context.index]
    pct_per_asset = 1.0 / max(len(eligible_assets), 1)
    context.pct_per_asset = pct_per_asset
    for asset in eligible_assets:
        # Some assets might cause a key error due to being delisted 
        # or some other corporate event so we use a try/except statement
        try:
            if get_open_orders(sid=asset):
                continue
            elif asset in context.shorts:
                order_target_percent(asset, -pct_per_asset)
            elif asset in context.longs:
                order_target_percent(asset, pct_per_asset)
        except:
            log.warn("[Failed Order] asset = %s"%asset.symbol)
    
def hedge_portfolio(context, data):
    """
    This function places an order for "context.index" in the 
    amount required to neutralize the beta exposure of the portfolio.
    """
    factors = get_alphas_and_betas(context, data)
    beta_exposure = 0.0
    count = 0
    for asset in context.portfolio.positions:
        if asset in factors and asset != context.index:
            position = context.portfolio.positions[asset].amount
            # Long the index if shorting the asset
            if not np.isnan(factors[asset].beta):
                beta_exposure += factors[asset].beta * np.copysign(1, position)
                count += 1
    beta_hedge = -1.0 * beta_exposure / max(count, 1)
    record(beta_hedge=beta_hedge)
    dollar_amount = context.portfolio.portfolio_value * beta_hedge
    if not np.isnan(dollar_amount):
        order_target_value(context.index, dollar_amount)
    
def get_alphas_and_betas(context, data):
    """
    returns a dataframe of 'alpha' and 'beta' exposures 
    for each asset in the current universe.
    """
    prices = history(context.lookback, '1d', 'price', ffill=True)
    returns = prices.pct_change()[1:]
    index_returns = returns[context.index]
    factors = {}
    for asset in context.portfolio.positions:
        try:
            X = returns[asset]
            factors[asset] = linreg(X, index_returns)
        except:
            log.warn("[Failed Beta Calculation] asset = %s"%asset.symbol)
    return pd.DataFrame(factors, index=['alpha', 'beta'])
    
def linreg(x, y):
    # We add a constant so that we can also fit an intercept (alpha) to the model
    # This just adds a column of 1s to our data
    X = sm.add_constant(x)
    model = sm.OLS(y, X).fit()
    return model.params[0], model.params[1]

def handle_data(context, data):
    record(net_exposure=context.account.net_leverage,
           leverage=context.account.leverage)
    
We have migrated this algorithm to work with a new version of the Quantopian API. The code is different than the original version, but the investment rationale of the algorithm has not changed. We've put everything you need to know here on one page.
There was a runtime error.

Finally, here is the same long short equity strategy but with beta hedging turned on. Because the beta is already so low, the impact is negligible.

Clone Algorithm
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import pandas as pd
import numpy as np
import statsmodels.api as sm
import math


def initialize(context):
    use_beta_hedging = True # Set to False to trade unhedged
    # Initialize a universe of liquid assets
    schedule_function(buy_assets,
                      date_rule=date_rules.month_start(),
                      time_rule=time_rules.market_open())
    if use_beta_hedging:
        # Call the beta hedging function one hour later to
        # make sure all of our orders have gone through.
        schedule_function(hedge_portfolio,
                          date_rule=date_rules.month_start(),
                          time_rule=time_rules.market_open(hours=1))
    # trading days used for beta calculation
    context.lookback = 250
    # Used to aviod purchasing any leveraged ETFs 
    context.dont_buys = security_lists.leveraged_etf_list
    # Current allocation per asset
    context.pct_per_asset = 0
    context.index = symbol('SPY')
    
def before_trading_start(context):
    # Number of stocks to find
    num_stocks = 100
    
    fundamental_df = get_fundamentals(
        query(
            # To add a metric. Start by typing "fundamentals."
            fundamentals.valuation_ratios.earning_yield,
            fundamentals.valuation.market_cap,
        )
        .filter(fundamentals.valuation.market_cap > 1e9)
        .order_by(fundamentals.valuation_ratios.earning_yield.desc())
    )
    
    fundamental_df
    yields = fundamental_df.ix['earning_yield']
    context.shorts = yields.tail(num_stocks/2)
    context.longs = yields.head(num_stocks/2)
    # fundamental_df = pd.concat([context.shorts, context.longs])
    all_assets = context.shorts.append(context.longs).index
    update_universe(all_assets)
    
def buy_assets(context, data):
    all_prices = history(1, '1d', 'price', ffill=True).dropna(axis=1)
    eligible_assets = [asset for asset in all_prices
                       if asset not in context.dont_buys 
                       and asset != context.index]
    pct_per_asset = 1.0 / max(len(eligible_assets), 1)
    context.pct_per_asset = pct_per_asset
    for asset in eligible_assets:
        # Some assets might cause a key error due to being delisted 
        # or some other corporate event so we use a try/except statement
        try:
            if get_open_orders(sid=asset):
                continue
            elif asset in context.shorts:
                order_target_percent(asset, -pct_per_asset)
            elif asset in context.longs:
                order_target_percent(asset, pct_per_asset)
        except:
            log.warn("[Failed Order] asset = %s"%asset.symbol)
    
def hedge_portfolio(context, data):
    """
    This function places an order for "context.index" in the 
    amount required to neutralize the beta exposure of the portfolio.
    """
    factors = get_alphas_and_betas(context, data)
    beta_exposure = 0.0
    count = 0
    for asset in context.portfolio.positions:
        if asset in factors and asset != context.index:
            position = context.portfolio.positions[asset].amount
            # Long the index if shorting the asset
            if not np.isnan(factors[asset].beta):
                beta_exposure += factors[asset].beta * np.copysign(1, position)
                count += 1
    beta_hedge = -1.0 * beta_exposure / max(count, 1)
    record(beta_hedge=beta_hedge)
    dollar_amount = context.portfolio.portfolio_value * beta_hedge
    if not np.isnan(dollar_amount):
        order_target_value(context.index, dollar_amount)
    
def get_alphas_and_betas(context, data):
    """
    returns a dataframe of 'alpha' and 'beta' exposures 
    for each asset in the current universe.
    """
    prices = history(context.lookback, '1d', 'price', ffill=True)
    returns = prices.pct_change()[1:]
    index_returns = returns[context.index]
    factors = {}
    for asset in context.portfolio.positions:
        try:
            X = returns[asset]
            factors[asset] = linreg(X, index_returns)
        except:
            log.warn("[Failed Beta Calculation] asset = %s"%asset.symbol)
    return pd.DataFrame(factors, index=['alpha', 'beta'])
    
def linreg(x, y):
    # We add a constant so that we can also fit an intercept (alpha) to the model
    # This just adds a column of 1s to our data
    X = sm.add_constant(x)
    model = sm.OLS(y, X).fit()
    return model.params[0], model.params[1]

def handle_data(context, data):
    record(net_exposure=context.account.net_leverage,
           leverage=context.account.leverage)
    
We have migrated this algorithm to work with a new version of the Quantopian API. The code is different than the original version, but the investment rationale of the algorithm has not changed. We've put everything you need to know here on one page.
There was a runtime error.

@Simon A Kalman filter is definitely a great choice to estimate any moving quantity like beta, sharpe ratio, or equity price. I'm sure that adding a Kalman filter instead of a point-in-time estimate would improve the quality of these strategies. We will definitely go over Kalman filters and parameter estimation during a future lecture, and maybe re-release versions of these algorithms with Kalman filters added.

Hello!

We will be hosting a live webinar for this first lecture in our summer series, "The Art of Not Following the Market" on July 9th at 12pm ET. You can register to attend here: http://bit.ly/DontFollowTheMarket. We will also share the recording with the community afterwards.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Hello,

The video from "The Art of Not Following the Market" is now available: http://bit.ly/DontFollowtheMarketVideo. Our next webinar in the summer lecture series, "The Good, the Bad, and the Correlated" is being held this Thursday, at 12pm EDT. If you would like to join, please RSVP here.

We will also share the recording in the community forum.

Kelly

@Delaney Granizo-Mackenzie
Hi, in the beta hedging algorithm, function get_alphas_and_betas, why do you use index_returns as y, asset_return as X? Thanks

Hi Ryan,
Good catch, there is no reason to use the asset returns as X, that's a mistake on my part for sure. Here is the updated algorithm.

Clone Algorithm
69
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import pandas as pd
import numpy as np
import statsmodels.api as sm
import math


def initialize(context):
    use_beta_hedging = True # Set to False to trade unhedged
    # Initialize a universe of liquid assets
    schedule_function(buy_assets,
                      date_rule=date_rules.month_start(),
                      time_rule=time_rules.market_open())
    if use_beta_hedging:
        # Call the beta hedging function one hour later to
        # make sure all of our orders have gone through.
        schedule_function(hedge_portfolio,
                          date_rule=date_rules.week_start(),
                          time_rule=time_rules.market_open(hours=1))
    # trading days used for beta calculation
    context.lookback = 150
    # Used to aviod purchasing any leveraged ETFs 
    context.dont_buys = security_lists.leveraged_etf_list
    # Current allocation per asset
    context.pct_per_asset = 0
    context.index = symbol('SPY')
    
def before_trading_start(context):
    # Number of stocks to find
    num_stocks = 100
    fundamental_df = get_fundamentals(
        query(
            # To add a metric. Start by typing "fundamentals."
            fundamentals.valuation_ratios.earning_yield,
            fundamentals.valuation.market_cap,
        )
        .filter(fundamentals.valuation.market_cap > 1e9)
        .order_by(fundamentals.valuation_ratios.earning_yield.desc())
        .limit(num_stocks)
    )
    update_universe(fundamental_df)
    
def buy_assets(context, data):
    all_prices = history(1, '1d', 'price', ffill=True)
    eligible_assets = [asset for asset in all_prices
                       if asset not in context.dont_buys 
                       and asset != context.index]
    pct_per_asset = 1.0 / len(eligible_assets)
    context.pct_per_asset = pct_per_asset
    for asset in eligible_assets:
        # Some assets might cause a key error due to being delisted 
        # or some other corporate event so we use a try/except statement
        try:
            if get_open_orders(sid=asset):
                continue
            order_target_percent(asset, pct_per_asset)
        except:
            log.warn("[Failed Order] asset = %s"%asset.symbol)
           
    
def hedge_portfolio(context, data):
    """
    This function places an order for "context.index" in the 
    amount required to neutralize the beta exposure of the portfolio.
    Note that additional leverage in the account is taken on, however,
    net market exposure is reduced.
    """
    factors = get_alphas_and_betas(context, data)
    beta_exposure = 0.0
    count = 0
    for asset in context.portfolio.positions:
        if asset in factors and asset != context.index:
            if not np.isnan(factors[asset].beta):
                beta_exposure += factors[asset].beta
                count += 1
    beta_hedge = -1.0 * beta_exposure / count
    dollar_amount = context.portfolio.portfolio_value * beta_hedge
    record(beta_hedge=beta_hedge)
    if not np.isnan(dollar_amount):
        order_target_value(context.index, dollar_amount)
    
def get_alphas_and_betas(context, data):
    """
    returns a dataframe of 'alpha' and 'beta' exposures 
    for each asset in the current universe.
    """
    prices = history(context.lookback, '1d', 'price', ffill=True)
    returns = prices.pct_change()[1:]
    index_returns = returns[context.index]
    factors = {}
    for asset in context.portfolio.positions:
        try:
            y = returns[asset]
            factors[asset] = linreg(index_returns, y)
        except:
            log.warn("[Failed Beta Calculation] asset = %s"%asset.symbol)
    return pd.DataFrame(factors, index=['alpha', 'beta'])
    
def linreg(x, y):
    # We add a constant so that we can also fit an intercept (alpha) to the model
    # This just adds a column of 1s to our data
    X = sm.add_constant(x)
    model = sm.OLS(y, X).fit()
    return model.params[0], model.params[1]

def handle_data(context, data):
    record(net_exposure=context.account.net_leverage,
           leverage=context.account.leverage)
    
There was a runtime error.

The algorithms in this thread may cease working briefly as we're doing maintenance during the switch to Q2. Please let me know if this causes any issues.

@Delaney Granizo-Mackenzie, David Edwards and Ryan Chen

I am testing the migrated code of long short equity strategy with beta hedging adding couple of changes

  1. Replacing order_by(fundamentals.valuation_ratios.earning_yield.desc()) with order_by(fundamentals.valuation_ratios.earning_yield.desc()).limit(num_stocks), in before_trading_start
  2. Selecting [1:] in the returns and index_returns to avoid the nan in the first position, in get_alphas_and_betas
  3. Adding an if asset != context.index: inside the for to make sure the linreg is not applied to the SPY position that is not included in factors, in get_alphas_and_betas
  4. Replacing X = returns linreg(X, index_returns,y) with y = returns linreg(index_returns,y), in get_alphas_and_betas , as noted by Ryan Chen and David Edwards

Now the beta is reduced to 0.07 from 0.29, but I still get tons of WARNs like
2008-03-03 get_alphas_and_betas:105 WARN [Failed Beta Calculation] asset = HFC
2008-03-03 get_alphas_and_betas:105 WARN [Failed Beta Calculation] asset = UNT
2008-03-03 get_alphas_and_betas:105 WARN [Failed Beta Calculation] asset = TCB
2008-03-03 get_alphas_and_betas:105 WARN [Failed Beta Calculation] asset = RGC
2008-03-03 get_alphas_and_betas:105 WARN [Failed Beta Calculation] asset = WCC

Some times this happens for almost all the assets in the portfolio

Why is this happening?
How does this affect the beta hedging?
How can this be avoided or corrected?
How is beta calculated in the backtester(zipline) and how is that beta related with the beta hedging factor?

def get_alphas_and_betas(context, data):  
  prices = data.history(context.all_assets, 'price', context.lookback, '1d')  
    returns = prices.pct_change()[1:]  
    index_returns = data.history(context.index, 'price', context.lookback, '1d').pct_change()[1:]  
    factors = {}  
    for asset in context.portfolio.positions :  
        if asset != context.index:  
            try:  
                y = returns[asset]  
                factors[asset] = linreg(index_returns,y)  
            except:  
                log.warn("[Failed Beta Calculation] asset = %s"%asset.symbol)  
    return pd.DataFrame(factors, index=['alpha', 'beta'])  
Clone Algorithm
61
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import pandas as pd
import numpy as np
import statsmodels.api as sm
import math


def initialize(context):
    use_beta_hedging = True # Set to False to trade unhedged
    # Initialize a universe of liquid assets
    schedule_function(buy_assets,
                      date_rule=date_rules.month_start(),
                      time_rule=time_rules.market_open())
    schedule_function(record_vars,
                      date_rule=date_rules.every_day(),
                      time_rule=time_rules.market_close())
    if use_beta_hedging:
        # Call the beta hedging function one hour later to
        # make sure all of our orders have gone through.
        schedule_function(hedge_portfolio,
                          date_rule=date_rules.month_start(),
                          time_rule=time_rules.market_open(hours=1))
    # trading days used for beta calculation
    context.lookback = 250
    # Used to aviod purchasing any leveraged ETFs 
    context.dont_buys = security_lists.leveraged_etf_list
    # Current allocation per asset
    context.pct_per_asset = 0
    context.index = symbol('SPY')
    context.all_assets = []
    
def before_trading_start(context, data):
    # Number of stocks to find
    num_stocks = 100    
    fundamental_df = get_fundamentals(
        query(
            # To add a metric. Start by typing "fundamentals."
            fundamentals.valuation_ratios.earning_yield,
            fundamentals.valuation.market_cap,
        )
        .filter(fundamentals.valuation.market_cap > 1e9)
        .order_by(fundamentals.valuation_ratios.earning_yield.desc()).limit(num_stocks)
    )   
    yields = fundamental_df.ix['earning_yield']
    context.shorts = yields.tail(num_stocks/2)
    context.longs = yields.head(num_stocks/2)
    # fundamental_df = pd.concat([context.shorts, context.longs])
    context.all_assets = context.shorts.append(context.longs).index
    
def buy_assets(context, data):
    all_prices = data.history(context.all_assets, 'price', 1, '1d').dropna(axis=1)
    eligible_assets = [asset for asset in all_prices
                       if asset not in context.dont_buys 
                       and asset != context.index]
    pct_per_asset = 1.0 / max(len(eligible_assets), 1)
    context.pct_per_asset = pct_per_asset
    for asset in eligible_assets:
        # Some assets might cause a key error due to being delisted 
        # or some other corporate event so we use a try/except statement
        try:
            if get_open_orders(asset):
                continue
            elif asset in context.shorts:
                order_target_percent(asset, -pct_per_asset)
            elif asset in context.longs:
               order_target_percent(asset, pct_per_asset)
        except:
            log.warn("[Failed Order] asset = %s"%asset.symbol)
    
def hedge_portfolio(context, data):
    """
    This function places an order for "context.index" in the 
    amount required to neutralize the beta exposure of the portfolio.
    """
    factors = get_alphas_and_betas(context, data)
    beta_exposure = 0.0
    count = 0
    for asset in context.portfolio.positions:
        if asset in factors and asset != context.index:
            position = context.portfolio.positions[asset].amount
            # Long the index if shorting the asset
            if not np.isnan(factors[asset].beta):
                beta_exposure += factors[asset].beta * np.copysign(1, position)
                count += 1
    beta_hedge = -1.0 * beta_exposure  / max(count, 1)
    record(beta_hedge=beta_hedge)
    dollar_amount = context.portfolio.portfolio_value * beta_hedge
    if not np.isnan(dollar_amount):
        order_target_value(context.index, dollar_amount)
    
def get_alphas_and_betas(context, data):
    """
    returns a dataframe of 'alpha' and 'beta' exposures 
    for each asset in the current universe.
    """
    prices = data.history(context.all_assets, 'price', context.lookback, '1d')
    returns = prices.pct_change()[1:]
    index_returns = data.history(context.index, 'price', context.lookback, '1d').pct_change()[1:]
    factors = {}
    for asset in context.portfolio.positions :
        if asset != context.index:
            try:
                y = returns[asset]
                factors[asset] = linreg(index_returns,y)
            except:
                log.warn("[Failed Beta Calculation] asset = %s"%asset.symbol)
    return pd.DataFrame(factors, index=['alpha', 'beta'])
    
def linreg(x, y):
    # We add a constant so that we can also fit an intercept (alpha) to the model
    # This just adds a column of 1s to our data
    X = sm.add_constant(x)
    model = sm.OLS(y, X).fit()
    return model.params[0], model.params[1]

def record_vars(context, data):
    # Check how many long and short positions we have.
    longs = shorts = 0
    for position in context.portfolio.positions.itervalues():
        if position.amount > 0:
            longs += 1
        if position.amount < 0:
            shorts += 1

    # Record and plot the leverage of our portfolio over time as well as the
    # number of long and short positions. Even in minute mode, only the end-of-day
    # leverage is plotted.  
    
    #record(long_count=longs, short_count=shorts) 
              
    record(leverage = context.account.leverage,
           net_exposure=context.account.net_leverage)
    
There was a runtime error.

Hello German,

Generally a failed beta calculation will occur when there are NaNs in the returns data. This is somewhat unavoidable as data sources will always have some degree of dirt in them. The hope with a strategy like this is that you are invested in enough positions that failing on a few positions doesn't affect the overall beta of the strategy. Also, it's generally a good idea to exit/not enter the positions if we can't compute a reasonable beta.

You can check out the beta calculation in the backtester source code here: https://github.com/quantopian/zipline/blob/master/zipline/finance/risk/cumulative.py

We are working on an updated version of this algo and hope to release it soon.

Thanks,
Delaney

For some reason, the algo is shorting 10X the portfolio value despite the input is .

order_target_percent(asset, 0.01)

The symbol is EEQ on 8/4 to 8/18

Can anyone confirm this?