Hi All,
We've just implemented a fix that we believe will solve some of the issues with
before_trading_start (BTS) timeouts that have been reported in the community. From this point going forward, all pipeline computation will be done before BTS runs, and therefore all pipeline output for a given chunk will be cached prior to BTS getting invoked. There will now be a separate 10 minute timeout specifically for pipeline computation, which should allow you not only to complete longer pipeline computations, but also to make more use of the current 5 minute timeout in BTS, which will remain as is. It's important to note that there were no changes made to the API with this fix. All changes made were on the back-end, so there are no changes needed in your algocode. Calling pipeline_output will retrieve the output of your pipeline that was executed in the 10 minute computation window prior to before_trading_start. The following is an example pointing out the parts of an algorithm affected by this change:
from quantopian.algorithm import (
attach_pipeline,
pipeline_output,
)
from quantopian.pipeline import CustomFactor, Pipeline
from quantopian.pipeline.data.builtin import USEquityPricing
class TestFactor(CustomFactor):
window_length = 1
inputs = []
def compute(self, today, assets, out):
# previously, this computation would be included in BTS's 5 minute timeout,
# but now, it has its own separate 10 minute timeout
some_computation()
def initialize(context):
p = Pipeline()
test = TestFactor()
p.add(test, 'test')
attach_pipeline(p, "test")
def before_trading_start(context, data):
# this still has a 5 minute timeout, but pipeline_output has already been computed
# in its own 10 minute timeout
results = pipeline_output('test')
We hope this fix will make it easier for those with more time-intensive contest entries, and are looking forward to hearing your feedback! Feel free to reach out to us with any questions you may have.
