A few weeks ago, we put out a call to our user base asking for examples of algorithms using the Pipeline API that were running slowly. Our primary goals were to:
- Better understand whether our users were effectively using Pipelines.
- Better understand where the underlying Pipeline infrastructure was encountering bottlenecks.
We received several helpful submissions, which we ran through an internal benchmarking harness to analyze where time was being spent, both in user code and in Quantopian code. We learned a lot from this experience, and I wanted to share some of our results. The attached notebook includes a summary of some of our findings as well as a detailed analysis of two of the submitted algorithms. Hopefully this provides an interesting view into our some of our development process.