Well, I'm slowly converting the algorithm over to use pipelines. So far I've only converted STA and SNOA computations, I suspect I'll soon run into timeout issues. Especially when I start implementing the franchise power and margin factors. The pipeline API as set up doesn't tell us where it is spending compute cycles, so any sort of optimization is virtually impossible.
I considered storing data in the context object, but as you said, that just doesn't work for live trading.
Another alternative to adding dates to
get_fundamentals would be to allow specification of specific times for which we want to retrieve data for custom factors. Right now you have a very coarse
window_size argument, would be good to be able to generalize that, even with something as simple as taking a vector of indices rather than a single window size.
The pipeline API is a really nice feature, but it is currently very inefficient especially when you want pipeline outputs infrequently (rather than daily), and does not come with a particularly rich set of operations that can be run on pipeline factors.