Pipeline Recipes

The following code snippets define pipelines that can be run in Research or the IDE. As a reminder, use run_pipeline to run a pipeline in Research and use attach_pipeline in the IDE to attach a pipeline to your algorithm.

Running an empty pipeline

This example defines a pipeline with no columns. Each of columns, screen, and domain are optional arguments to the Pipeline constructor. The following code returns a MultiIndex DataFrame with no columns and an index made up of all equities that were listed on a supported exchange in the specified domain (US_EQUITIES), every day. Running this pipeline in Research is a good way to see all equities that the Pipeline API knows about in a particular domain, each day.

from quantopian.pipeline import Pipeline
from quantopian.pipeline.domain import US_EQUITIES

# Define a pipeline with no columns and no screen
# over the US_EQUITIES domain.
pipe = Pipeline(domain=US_EQUITIES)

Constructing a tradable universe

This example defines a pipeline that filters its output down to the QTradableStocksUS, which is required for all contest algorithms.

from quantopian.pipeline import Pipeline
from quantopian.pipeline.filters import QTradableStocksUS

qtu = QTradableStocksUS()

# Define a pipeline and filter the output down to the QTradableStocksUS.
pipe = Pipeline(
    screen=qtu
)

Using built-in factors

This example defines a pipeline using several built-in pipeline factors. Note that some built-in factors have default inputs and/or window_length values, while others do not. Some built-in factors may also have additional arguments (like factors.SimpleBeta). Be sure to check the Pipeline API Reference to learn about the arguments and defaults of a built-in factor.

from quantopian.pipeline import Pipeline

# Import built-in factors and filters.
from quantopian.pipeline.factors import DailyReturns, Returns, SimpleBeta

# Defaults for DailyReturns are inputs=[EquityPricing.close], window_length=2.
daily_returns = DailyReturns()

# Defaults for Returns are inputs=[EquityPricing.close], window_length not specified.
returns_1w = Returns(window_length=6)

# Only allowed_missing_percentage has a default value (0.25). We define target and
# regression_length and override the default value for allowed_missing_percentage.
beta_to_spy = SimpleBeta(
    target=symbols('SPY'),
    regression_length=252,
    allowed_missing_percentage=0.05
)


pipe = Pipeline(
    columns={
        'daily_returns': daily_returns,
        'returns_1w': returns_1w,
        'beta_to_spy': beta_to_spy,
    }
)

Note

This example used the symbols() function to define SPY as the target for our beta to spy computation. The pipeline code can be run in Research as written. To run it in the IDE, you will need to change the line target=symbols('SPY') to target=symbol('SPY').

Using a CustomFactor as input to another factor

Generally speaking, the inputs of a pipeline term (factor, filter, or classifier) can either be a BoundColumn or another pipeline term. The example below illustrates this behavior by defining a custom factor and passing it as input to a built-in factor. Note that in order to pass a term (custom factor or built-in term) to another term as input, we need to indicate to pipeline that the term is window_safe, meaning our factor is invariant across split adjustments. We can do this by setting window_safe=True in our custom factor defintion.

import numpy as np

from quantopian.pipeline import CustomFactor, Pipeline
from quantopian.pipeline.data.factset.estimates import PeriodicConsensus

class PercentChange(CustomFactor):
    """
    Computes the percent change between the oldest and newest value in
    the lookback window.
    """
    # With this line, we tell the pipeline engine that our custom
    # factor is invariant across corporate action adjustments.
    window_safe = True

    def compute(self, today, assets, out, values):
        out[:] = (values[-1] - values[0]) / values[0]

class StdDev(CustomFactor):
    """
    Computes the standard deviation of our input for each asset over the
    window length.
    """
    def compute(self, today, asset_ids, out, values):
        # Calculates the column-wise standard deviation, ignoring NaNs.
        out[:] = np.nanstd(values, axis=0)

fq1_eps_cons = PeriodicConsensus.slice('EPS', 'qf', 1)

fq1_eps_est_change_2w = PercentChange(inputs=[fq1_eps_cons.mean], window_length=10)

# We can pass our fq1_eps_est_change_2w term as input to StdDev (or any other pipeline
# term), since it is window_safe.
fq1_eps_est_change_stddev = StdDev(inputs=[fq1_eps_est_change_2w], window_length=63)

pipe = Pipeline(
    columns={
        'fq1_eps_est_change_2w': fq1_eps_est_change_2w,
        'fq1_eps_est_change_stddev': fq1_eps_est_change_stddev,
    }
)

Note

For a description of what window_safe means, see this comment in the Quantopian community forums. Common factors that are not window safe include price per share (including price rank or price zscore), earnings per share (including rank and zscore), and anything whose output value is not invariant across splits. Conversley, examples of factors that are window safe include returns and relative difference.

Slicing a factor

The following code snippet creates a Returns factor and slices it to extract the column for AAPL. It then feeds the AAPL returns into the compute function of a custom factor.

from quantopian.pipeline import CustomFactor
from quantopian.pipeline.factors import Returns

returns = Returns(window_length=30)
returns_aapl = returns[symbols('AAPL')]

class MyFactor(CustomFactor):
    inputs = [returns_aapl]
    window_length = 5

    # `returns_aapl` is a 5 x 1 numpy array of the last 5 days of
    # returns values for AAPL. For example, it might look like:
    # [[ .01],
    #  [ .02],
    #  [-.01],
    #  [ .03],
    #  [ .01]]

    def compute(self, today, assets, out, returns_aapl):
        # Do something
        pass

Find the SID of a non-US asset

The following example demonstrates the best way to lookup the SID of a non-US asset. Note that there are two slightly different solutions offered in the snippet. One is optimized for looking up a single SID, the other is optimized for looking up all symbol --> SID mappings in a domain.

from quantopian.pipeline import Pipeline
from quantopian.pipeline.domain import CA_EQUITIES
from quantopian.pipeline.factors import SimpleBeta

from quantopian.research import run_pipeline

pipe = Pipeline(
    columns={
    },
    domain=CA_EQUITIES,
)

df = run_pipeline(pipe, '2015-01-01', '2015-01-01')

# Look up a single ticker.
df[df.index.get_level_values(1).map(lambda x: x.symbol == 'XIU')]

# Look up all tickers and store mappings in a dict.
ticker_to_sid = {}

for equity in df.index.get_level_values(1):
    ticker_to_sid[equity.symbol] = equity.sid