Back to Community
add_history bug in research environment

Hello,
I have been running backtests manually in research environment, but since yesterday I got the error of: "cannot import name add_history"
As far as I know, having add_history is necessary to get prices from the database. I suppose it has something to do with Q2 upgrade.
I read the zipline code and I think it is because add_history is no longer an "api_method" in Algorithm. I tried to see what has been the workaround for this in Q2, but as I found out, it needs you to use DataPortal, but I don't think I can manually call the DataPortal constructor which gives me the right prices via Quantopian engine.
Any help would be appreciated, thanks

15 responses

Hi Peyman,
I believe you can just remove add_history. I testing on my algorithm and changed my inputs to the following,

from zipline import TradingAlgorithm  
from zipline.api import (order_target_percent, record, symbol, history, get_datetime, get_open_orders, get_order, order_target_value, order, order_target, sid)  
from zipline.finance.slippage import FixedSlippage  
Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Hi Karen,
Thank you for your quick reply.
I actually have checked what removing that line leads to.
Later when I run the algorithm, on the line calling history(20,'1d', 'close_price') function, I get the following error:
"Invalid field: close_price" And if I change the 'close_price' to 'close' or 'high' or ... the error will change to:
'DataPortal' object has no attribute '_equity_daily_aggregator'

I really appreciate your help.

Interesting. I just remembered that my algo doesn't use history, so it wasn't a very good test. The backtesting with zipline tutorial in the tutorials folder of your research env does, and it works. My next suggestion would be to look at that and see if you can figure out what is different.

Another possibility is to migrate it to the Q2 history API and see if you can get that to work.

Alternatively you can send it in to feedback (or share it here) and I am happy to take a look.

Hello, pls help

data history bug

def handle_data(context, data):
----> prices_close = history(context.lookback, context.charttype, 'close_price')
prices_close = list(prices_close.values.flatten())
-----> prices = history(context.lookback, context.charttype, 'price')[context.stock]
prices = history(context.lookback, context.charttype, 'price')[context.stock]

Karen,

The tutorial you mentioned breaks as soon as you switch the frequency to "minute" which is what Q2 solely does now on the IDE side. I've spent quite a bit of time going through the zipline code on github to find a workaround but I'm drawing a blank. Am I missing something very obvious?

Hi Peyman,
Thanks for pointing this oout we've found a bug and have a change working through the system now. I'll let you know when it's fixed on prod.

KR

Thank you very much Karen,
As far as I looked at the zipline code, I thought adding add_history to zipline api will help users to use history from Q1. And adding DataPortal to research env will allow users to use history from Q2. I really appreciate your help.

Hi, Karen Rubin

I have the same problem as Peyman Pejman. Do you know how many time may take this bug fix?

Best regards,
Maksim

Hi Karen,

I am running into same issue. Any idea when the fix will be available?

Thanks,
Soumya.

I have the same issue. I can neither use Quantopian research platform nor zipline since add_history has disappeared. Any workarounds?

bump

any fix for this?

bump, any fix?

bump, any fix?

Sorry - actually Karens suggestions worked for me ...depending on the algo you are working with, this quick workaround (...i think...seems to be working for me):

Attached is working notebook - took this guys notebook from here: https://www.quantopian.com/posts/replication-of-backtest-results-in-research-environment and then:

1) removed add_history from zipline.api import
2) Removed add_history from initialize
3) changed price_history = history(context.lookback, frequency="1d", field='price') to price_history = data.history(context.stocks, fields='price', bar_count = context.lookback, frequency="1d")

returns changed up a but I am chalking that up to Q1 to Q2 port changes (unlikely since commissions/slippage are specified...) please post something if someone finds out)

Loading notebook preview...
Notebook previews are currently unavailable.