Back to Community
Feature request: Expose chunksize in Pipeline API to allow predictable backtesting of compute intensive Pipelines without timeout

If you create compute intensive Pipeline calculations, you've all probably run into this - Pipeline batches several days seemingly arbitrarily together, which can lead to timeouts if you're doing heavy calculations that would easily fit into the 5min window for a few days, but not if Pipeline starts batching several months into a single run during before_trading_starts()

According to this the Pipeline API has the ability to manually specify a chunksize in attach pipeline:

zipline.api.attach_pipeline(self, pipeline, name, chunks=None)  

However, currently the 'chunks' parameter is not recognized by the backtester.

Is there any way to expose this 'chunks' parameter to be used in backtesting? This would be immensely valuable to create a more predictable backtesting behavior with more compute intensive Pipeline calculations.