Back to Community
Walk Forward Optimization - Is there plan in the work?

Walk Forward Optimization is an optimization process that addresses the issue of curve fitting in strategy development.

Walk Forward Optimization segregates the data series into multiple segments, and each segment is divided into an in-sample (IS) portion and an out-of-sample (OOS) portion.

Parameter optimization for the strategy is performed using the IS portion of the first segment. The same parameters are then used to back test the strategy on the OOS portion of the same segment. The process is repeated for the remaining segments.

The OOS performance results from each of the segments are considered "real" instead of "curve-fit" because the parameters that produced the OOS results were generated from IS data.

2 responses

There are no doubt some who have built optimization wrappers around ZipLine (Quantopian's backtest engine) which would allow a strategy to optimize various internal parameters and you might ask the list for such code or their advice. Walkforward optimization is an even bigger programming endeavor than simple optimization. Persistence of intermediate optimization results, ranking of those results, best scenario selection, timeseries start/end time reselection, repeat (with about a dozen other steps not mentioned) is no easy task. TradeStation's variation shows considerable effort was involved in writing that chunk of code. But what do I know of the Q's internal feature ToDo list.

If Quantopian offered a web API where one could send in a JSON packet containing backtest parameters, including strategy variables (that plugged in and started a test) then maybe a HTML5 driver app could be written to execute 100 or 1000 simultaneous backtests (fully saturate Q's servers) collect the intermediate results, reformulate new backtest payloads based on the results and run the next date range of tests; if that was offered then maybe a walkforward solution could be forthcoming.

I wouldn't bet on it though.

Here's a blog post and discussion on the topic: http://blog.quantopian.com/parameter-optimization/. And another relevant one: http://blog.quantopian.com/zipline_in_the_cloud/.

My hunch is that it'll be awhile before it comes to the browser-based backtester, but if the Quantopian Research Environment figures out how to run parallel code, you'll basically have zipline and access to the Quantopian backtest data set. So, I don't see why you couldn't write your own walk-forward optimization code there (once Quantopian Research is opened to the user base).

Recently, Quantopian added before_trading_start(context), https://www.quantopian.com/help#api-before_trading_start. Presently, it times out at 60 seconds (see https://www.quantopian.com/posts/before-trading-start-does-it-time-out), but presumably the run time could be extended to allow for optimization on a daily basis.

One thing to consider is using fetcher to feed in optimization results on a daily basis, either from your own stand-alone optimizer or from Quantopian Research (assuming that a link is enabled, which presently does not exist, to my knowledge).