Back to Community
Fetcher History - Calculate Moving Average

I know I can't use data.history with fetcher, but does anyone have a simple workaround? The most recently posted examples that I found have deprecated code and are very complex (I'm a beginner). The code below is trying to calculate a moving 60-day average of the 30-yr Treasury yields.

Alternatively, if I know a specific date of the data I want, how would I get that single date's value?


def preview(df):  
    log.info(' \n %s ' % df.head())  
    return df

def postview(df):  
    log.info(' \n %s ' % df.head())  
    return df

def initialize(context):  
    long_url='https://www.quandl.com/api/v3/datasets/YAHOO/INDEX_TYX.csv?api_key=r85Zyd1sc9d4gBebFSFy' #treasury yield  
    fetch_csv(long_url, pre_func=preview, post_func=postview, date_column = 'Date', symbol='tyx')  



def before_trading_start(context, data):  
    """  
    Called every day before market open.  
    """  
    hist = data.history('tyx', 'Close', 60, '1d')  
    print hist.mean()  
10 responses

There is not a simple workaround. I've posted a complicated workaround in the past, try searching for "history for fetch_csv"

Thanks, Simon. I did see those posts and have tried working through the code. Unfortunately, they have deprecated code or throw errors when I try running them. It's too bad that fetcher is not supported more fully. That would be a game changer.

This will get you your data:

def initialize(context):  
    context.day = 0  
    context.treasury = []  
    fetch_csv('https://www.quandl.com/api/v3/datasets/YAHOO/INDEX_TYX.csv?api_key=r85Zyd1sc9d4gBebFSFy',  
              date_column='Date',  
              symbol='tyx')  

def before_trading_start(context, data):  
    context.day += 1  
    val = data.current('tyx', 'Adjusted Close')  
    context.treasury.append(val)  
    if context.day == 60:  
        print context.treasury  

You will have to modify it a bit, but it's a good start.

Much better approach:

import pandas as pd

def preprocess(df):  
    mean = pd.rolling_mean(df['Adjusted Close'], 60)  
    df['mean'] = mean  
    return df

def initialize(context):  
    fetch_csv('https://www.quandl.com/api/v3/datasets/YAHOO/INDEX_TYX.csv?api_key=r85Zyd1sc9d4gBebFSFy',  
              pre_func=preprocess,  
              date_column='Date',  
              symbol='tyx')  

def before_trading_start(context, data):  
    print data.current('tyx', 'mean')  

This is great, zippy. Thank you! I noticed the mean is calculating the forward rolling mean but I think I can make some adjustments to get it to calculate the backward-looking mean. Big help. Thanks!

Hi John, wow...really on the forward rolling mean...I guess you always have to test this stuff multiple times ;-)

It is very easy to accidentally introduce future snooping bias when using pre/post-funcs and fetch_csv. That is why I went to so much effort to pack the historical data in only, and do all transformations in regular algo code.

Hi Simon, where does the future snooping bias creep in? When you calculate moving averages or shift the index on a dataframe? I have seen some posts mention mistakenly using an EOD value at opening, so it would be good to know where the mistakes are made.

Thanks!

Hey John, the yahoo data set has most recent values on top and needs to be flipped. This should do the trick:

def preprocess(df):  
    df = df.reindex(df.index[::-1])  
    mean = pd.rolling_mean(df['Adjusted Close'], 60)  
    df['mean'] = mean  
    return df

Brilliant! It works perfectly.