Back to Community
Forward Filling, Simple Question Regarding History

Hi all, just to make this clear,
given an algo trades at market open, which one of these returns the closing values while Excluding most recent value for the current day (meaning the day history is requested):

A data.history(context.stocks, "close", 123, "1d")[:-1]

or B data.history(context.stocks, "close", 123, "1d")

As i understand from docs, it should be B, since close prices are not forward filled
Thanks for your time

6 responses

Hi Darell -

Per the help page:

"price" is always forward-filled. The other fields ("open", "high", "low", "close", "volume") are never forward-filled.

So you are correct that close prices are not forward filled. However, I don't see how it is relevant. If you are looking to get a trailing window of daily closing prices, just drop the final row (your choice A).

See attached for an example.

Clone Algorithm
Total Returns
Max Drawdown
Benchmark Returns
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
def initialize(context):
    context.stocks = [sid(8554),sid(23921)]
def before_trading_start(context, data):
def show_history(context, data):
    prices = data.history(context.stocks,'close',5,'1d')[:-1]
    # prices = data.history(context.stocks,'close',5,'1d')[:-1]
    print get_datetime()
    print prices
    print data.current(context.stocks,'price')
    print '----------------------------'
There was a runtime error.

Thanks Grant,
I was asking since if Close is not forward filled, why is one required to remove the last row (since if its not forward filled, its not provided)

The forward-filling is done if historical trade data are not available for a given minute/day. You have to remove the last row, because the earliest you can call history is one minute after the open (at 9:31), and if you call it with 1d you'll get a trailing window of daily closing prices plus the most recent minutely close. For example, from the help page:

data.history(context.assets, "price", 6, "1d") returns the prices for the previous 5 days and the current price.

Instead of 'price' you can use 'close' if you don't want the forward-filling, but for a stock like SPY, there won't be difference, since it trades every minute. For stocks that don't necessarily trade every minute/day, I think 'close' returns a NaN (you should check if it matters in your algo), since filling is turned off. Either way, when using '1d' you have to drop the last row if you only want a trailing window of daily closing prices.

Thanks, i did not know that even when requesting 1d and close the last minute bar is also included. So to recap, if i want to get yesterdays close price at the open today, one needs to do: data.history(context.sid, "close", N, "1d")[:-1]

Am i correct in assuming then that, even if algo is triggered 30 min before the close, the history would still include the latest minute bar and hence [:-1] should be used to also exclude that from the series ?

If i may, i ask the same of Vix data from quandl via pipeline, is it pre-shifted, meaning i dont need to shift vix data at the open when asking for vix_close to get yesterdays vix close.

Thanks for your time

Yes, for '1d' you'll get a trailing window of daily values, plus the most recent minutely one. So, if you don't want the most recent minutely one, you'll need to drop it. I recommend simply fiddling with the code I provided to see the behavior for yourself.

Regarding VIX data, I don't know the answer.

Thank you Grant