zipline.TradingAlgorithm Removed From Research and Backtesting

Today, we removed the ability to import and use the zipline.TradingAlgorithm class in Research and in Backtesting.

We had previously allowed community members to import and use TradingAlgorithm to run Zipline backtests in Research, but the feature was poorly supported: you couldn't run pipelines easily from your algorithm, and you had to pass a DataFrame or Panel of static data to the algorithm constructor. As part of our upcoming work for supporting international markets, we plan to make Zipline changes that would make backtesting in the Research environment even harder to support, and today we noticed a potential security issue that would have been difficult to fix without disrupting Zipline users, so we made the decision to remove TradingAlgorithm from the platform.

Unfortunately, this means that you won't be able to run Zipline simulations in the Research environment. If you need to run algorithms, we recommend that you move your work to the backtester.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

12 responses

Jamie, what is Quantopians recommendation for 'sequencing' backtests now? I've only recently joined and am currently implementing my strategies based on futures instruments which will usually have 2-4 parameters for which I'm interested in their best combination, let's say per sector.
Is there any alternative on Quantopian other than run, save the logs and restart the backtest algos manually?? Given the number of possible parameter combinations and different markets I hope the answer isn't 'yes' ;) Thanks, Sven

Hi Sven,

Other than futures pricing data, are you using any other data in your research? If you'd like you can email in to [email protected] to answer privately.

Cheers,
Jamie

Hi Jamie,
thx for quick turnaround.

No other data, my strategies are 100% price data based (which by the way is a challenge currently for futures with low tick sizes such as JY, here the daily data precision on Q is way too low, needed to compile my daily data out of minute data).

Coming to Quantopia I was planning to implement strategy cores in the backtester and do the fine tuning in research by optimizing the parameter levels. Currently I'm investing setting up a zipline environment locally in case that can't be done anymore.

Your advice on what options I have here is very welcome.

Cheers & Regards,
Sven

Hi Jamie -

From an architectural standpoint, this seems like a step backwards. I would think that improving integration of the backtesting engine with the research platform would be the way to go. For example, being able to launch parallel backtests parametrically from the research platform, and then analyze them in batches would seem to be a use case of interest to the community.

I think what Night Hawk is asking is how to launch N backtests to sample a parameter space, create a response surface, and optimize. This use case is not well-supported. One way to do it is to randomly sample the parameter space (with a random number generator in the code), and record the parameters with the record function, whose output is available in the research platform. Each backtest needs to be launched manually, by clicking a button in the browser. And then the backtest IDs need to be collected, and pasted into the research platform.

So, it is do-able, but for some reason, not well-supported, when it would seem to be a common use case for a platform that supports running an unlimited number of backtests in parallel. What are the impediments to supporting the use case? Or perhaps it would not be that useful to the broad community? I understand that running Zipline in a notebook is not the way to go, but not providing an alternative doesn't make sense to me.

Hi Grant, Jamie,

just adding that for my purposes hitting the backtest button manually is not do-able for practical reasons. Why? Typically you are addressing 30-40 different futures in different sectors that you are trading in parallel (A) with some strategies across the board and (B) with specific strategies that may be only suitable for some sectors (such as FX, agrar, metals, ...). You'd do that in order to smooth out drawdowns, increasing your return on account (ROA).

Now, given the number of your strategies (let's assume 3 across all sectors and 2 additional ones per sector) you'd have 40 futures times 5 strategies times # of your parameters/strategy. This is not going to work with manually kicking off backtests and then read out the logs in research with the backtest handles.

For this you'd need to run the inidivual tests batch-wise (through parameterized loops) and collect their test results in arrays or data frames for further analysis.

I'm not a great expert in equity based strategies (other than indices) but I'd be surprised if somebody working in the futures area would be able to test strategies without such feature.

I'd appreciate if such option would be made available here.

Thanks!

Hi Jamie,

If the 'zipline.TradingAlgorithm' is removed, that means one can't do the automated backtesting as described in the following link, right?

Jamie, running automated, parameterized backtests is a very important feature. Did you plan to support automations in backtester?

Hi,

How about the answer to my question? Can we still run the optimation as before?

Up! I am still waiting for the answer from Q.

So nobody found a way to backtest using research? Has anyone done this locally? I cant get mine to work

Cemal,
Quite a bit of setup, yet works well!
See @marketneutral, J.Larkin's Alphatools
https://github.com/marketneutral/alphatools
alan

Hey Alan,

Thank you for the link, ill check it out more thoroughly . I am travelling atm, but I will get back to you!

Kind regards,

Cemal