Back to Community
Index problem when calling history()

Hi,

I'am relatively new in python programming and I'm facing a problem that I don't understand...

I'am trying to define a low volatility universe that will change at each rebalance time. First, I create universe with filtered fondamentals, in order to get only data with certains market cap and then I try to get historical prices for thoses securities in order to calcul volatility. Finally, I would like keep only context.m securities that have lowest volatility.

I post my entire script just below :

import pandas  
import numpy as np

def initialize(context):  
    schedule_function(  
                      trade,  
                      date_rule=date_rules.every_day(),  
                      time_rule=time_rules.market_close(minutes=15)  
                      )  
    context.m = 30 # screened stocks number  
    context.lookback = 30 # 252*n = n years  
    context.bmark = symbol('SPY')  
    context.days = 3  
    context.b_t = np.ones(context.m) / context.m  
    context.eps = 1 # change epsilon here  
    context.rebalance_days = 10  
    context.number_day = 0  
    context.w = np.zeros(context.m)  
    context.data = []

def handle_data(context, data):  
    cash = context.portfolio.cash  
    current_positions = context.portfolio.positions  
    current_price = history(1, '1d', 'price')  
     # If my portfolio is empty, I invest (equally weighted) on all stock of my univese  
    if len(current_positions) == 0:  
        allocate(context, data, context.b_t)  
    record(leverage = context.account.leverage)  
    print context.w  
    context.w = olmar(context, context.days)

def trade(context, data):  
    # Condition for rebalancing  
    if context.number_day == context.rebalance_days:  
        allocate(context, data, context.w)  
        context.number_day = 0  
    else:  
        pass

def olmar(context, window):  
    # In this case, no problem to get data...  
    p = history(context.days, '1d', 'price')  
    prices = p

    x_tilde = np.zeros(context.m)  
    x_bar = np.ones(context.m)  
    x_var = np.zeros(context.m)  
    mar_rel_dev = np.zeros(context.m)  
    for i, stock in enumerate(context.stocks):  
        x_tilde[i] = (np.sum(prices[stock])/context.days)/prices[stock][-1]

        x_var[i] = 1-np.var(np.asarray(p[stock][1:])/np.asarray(p[stock][0:-1]))

    x_bar = np.mean(x_tilde)  
    mar_rel_dev = (1-x_var)*x_tilde.mean()

    dot_prod = context.b_t*x_tilde  
    num = context.eps - dot_prod  
    denom = np.exp(np.absolute(x_tilde-mar_rel_dev))

    lam = num/denom  
    lam[lam<0] = 0  
    b = context.b_t + lam*(x_tilde-x_bar)  
    b = np.absolute(context.b_t-b)**2

    b_norm = simplex_projection(b)  
    return b_norm

def before_trading_start(context):  
    context.number_day +=1  
    mkt_cap = get_fundamentals(  
            query(fundamentals.valuation.market_cap)  
            .filter(fundamentals.company_reference.primary_exchange_id == 'NAS')  
            .filter(fundamentals.valuation.market_cap != None)  
            .filter(fundamentals.valuation.market_cap >= 2000000000)  
            .order_by(fundamentals.valuation.market_cap.desc()).limit(150))  
    update_universe(mkt_cap.columns.values)

    # First, universe filtered to keep only big companies  
    context.stocks = [stock for stock in mkt_cap]

    # And then, according to the universe, keep only context.m lowest volatility companies  
    low_vol = lowVol_universe(context, context.lookback)  
    update_universe(low_vol.columns.values)  
    context.stocks = [stock for stock in low_vol]  
def lowVol_universe(context, window):  
    # The problem comes from here !!!  
    # I don't understand why, because I use the same method than before...  
    histo = history(context.lookback, '1d', 'price')  
    x_sd = np.zeros(context.m)  
    for i, stock in enumerate(context.stocks):  
        x_sd = np.std(np.asarray(histo[stock][1:])/np.asarray(histo[stock][0:-1]))  
        x_sd = np.sort(x_sd).limit(context.m)  
    return x_sd  

def allocate(context, data, desired_port):  
    for i, stock in enumerate(context.stocks):  
        order_target_percent(stock, desired_port[i])  
    for stock in data:  
        if stock not in context.stocks:  
            order_target_percent(stock,0)  

def simplex_projection(v, b=1):  
    v = np.asarray(v)  
    p = len(v)

    u = np.sort(v)[::-1]  
    sv = (np.cumsum(u)-b)/(np.arange(1, p + 1, 1))

    rho = np.max(np.where(u > sv))  
    theta = ((np.cumsum(u)-b)/(rho+1))[rho]  
    w = (v - theta)  
    w[w<0] = 0  
    return w  

When I run the backtest, I get the error message :
IndexError: index 0 is out of bounds for axis 0 with size 0

Thanks

3 responses

I think it's because inside before_trading_start() it's not possible to use history function.

If it's true, it's problematic because universe can only be changed within before_trading_start statment. I guess I have to find another way to do it.

Hi Antoine,

The reason you're getting this error is indeed because the history() function cannot be called in before_trading_start(). From a brief glance at your code, it seems like this might be a good job for the Pipeline API. Pipeline lets you create custom factors for the entire universe of 8000+ securities on Quantopian. It also lets you filter this down to a subset that you can then set as your trading universe. There are also some sample Pipeline algorithms here.

Let me know if this helps.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Hi Jamie

Thanks you for the tip, I will investigate that further.

Antoine