Back to Community
Is backtesting in research notebooks impossible?

Hi Qunat people,

I need to run a loop of backtesting via Zipline in Research environment, but it’s not implementable even for a simple example as attachment (buy APPL and print 2 comments)
There isn’t unfortunately any Tutorial about it. The Link here seems to be deleted, which was a Notebook in folder “Tutorials and Documentation”.
Thank you in advance!

Loading notebook preview...
Notebook previews are currently unavailable.
11 responses

I have noticed that to run the zipline algorithms, either run_algorithm() must be used, whose importing raised an import error, or Ipython magic %%zipline in notebooks, however it seems that Research environment doesn’t support magic.

Does it mean that backtesting in research notebooks is impossible?

Loading notebook preview...
Notebook previews are currently unavailable.

No support from Quant people...
It is unacceptable!

Hi Ahmad,

I've gone ahead and answered your question in the Zipline Google Group as well.

You can run backtests in the research environment, but you should use the run method (like you do in your notebook in this post (https://www.quantopian.com/posts/is-backtesting-in-research-notebooks-impossible). Using Zipline in Quantopian Research is different from running Zipline on your own machine, using IPython Notebook.

It seems the error is coming from getting benchmark data, in zipline.data.benchmarks. I've opened a PR that should hopefully fix that.

If you want to use Zipline, you could either using Zipline in your own machine, in an IPython Notebook, or just use the backtester on quantopian.com/algorithms, if you're looking to use Pipeline, Optimize, or other Quantopian APIs.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

It seems that one should be able to take an algo that runs in the backtester/trading API, copy it as-is into the research platform, and run it there. Is this possible?

after seeing

Ahmad Hashemi 3 days ago
No support from Quant people...
It is unacceptable!

I was going to walk away (...surprised anyone else bothered replying), but attaching research backtests just because Grant K also requested

Loading notebook preview...
Notebook previews are currently unavailable.

after seeing

Ahmad Hashemi 3 days ago
No support from Quant people...
It is unacceptable!

I was going to walk away (...surprised anyone else bothered replying), but attaching research backtests just because Grant K also requested

Loading notebook preview...
Notebook previews are currently unavailable.

one notebook is for non-pipline, and the other is for with pipeline - hope it help. Would be great if someone combines them plus incorporates some the more latest features (optimize etc.)

Amazing that the Quant devs have not yet solved the problem, though the reason is completely clear.

Thanks to the People of Zipline group, I found three ways to solve it locally:

  1. use yahoo (in ../zipline/data/benchmarks.py) instead of 'google'

  2. https://github.com/pydata/pandas-datareader/pull/425/commits/60ad11c96cb08a2cc2b2f1ba07f16746eb067329

  3. https://github.com/pydata/pandas-datareader/commit/7d8803d9707b9bb710982397ad8984773295722b

Then post the fix here instead of your contemptuous comments

Amazing that the Quant devs have not yet solved the problem, though the reason is completely clear.

Hi Ahmad,

I did open a PR recently to fix the benchmark issue here. I believe I also mentioned this above as well as on GitHub and the Zipline Google Group :)

Hi Freddie

I fixed the local zipline by means of your pull, and implemented my Expose via zipline. It works errorless, thank you so much, however there is differences in pipelines output between local zipline and online research, thus the results of analysis via each one are different.

I want to know, weather it is normal, or it has something to do with your suggested pull namely benchmarking from IEX.
One question more, do you even recommend that I test the performance of Trading System based on results of local zipline on online IDE?

Happy new year
best regards