Back to Community
New Feature: Multiple Output Pipeline Custom Factors

It's common when writing algorithms using pipeline to have factors that rely on the same underlying data. We've added the ability for custom factors to define multiple outputs so that you can do all your computations requiring the same data in one factor.

Here is an example which returns the high and low prices for a 100 day window. If you clone the algorithm below, you can see the output in the logs.

class MultipleOutputs(CustomFactor):  
    # Define inputs and outputs.  
    inputs = [USEquityPricing.close]

    # Specify and name the different outputs.  
    outputs = ['highs', 'lows']  
    window_length = 10

    def compute(self, today, assets, out, close):  
        highs = np.nanmax(close, axis=0)  
        lows = np.nanmin(close, axis=0)  
        # Write the desired return values into `out.<output_name>` for each output name in `self.outputs`.  
        out.highs[:] = highs  
        out.lows[:] = lows

This is a very simple example and these factors will become powerful tools for building regression and correlation factors. Stay tuned for more in this department.

Clone Algorithm
71
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5734dcfeaa074b0f86b86d34
There was a runtime error.
Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

7 responses

How can a multi-output custom factor be used for a filter?

I see they are just returning a tuple with all the values per each asset in pipeline's output, but instantiating the custom factor, in your example

custom_factor = MultipleOutputs()

and trying to access the tuple's values with [] operator as in

my_filter = custom_factor[1] < custom_factor[0]

yields

TypeError: 'MultipleOutputs' object does not support indexing

while trying to use the same syntax you showed in the example

my_filter = custom_factor.low < custom_factor.high

results in

BadBinaryOperator: Can't compute RecarrayField > instancemethod  

Hi Andrea,

I think the code you're looking for is custom_factor.lows < custom_factor.highs. Note the s's at the end of attributes: the sub-factors supplied by a multi-output factor have to exactly match the strings provided at class-definition time. This works as expected for me:

from quantopian.pipeline import CustomFactor  
from quantopian.pipeline.data.builtin import USEquityPricing  
​
​class MultipleOutputs(CustomFactor):
    inputs = [USEquityPricing.close]  
    outputs = ('lows', 'highs')  
    window_length = 10  
    def compute(self, today, assets, out, closes):  
        out.lows[:] = 5  
        out.highs[:] = 6  
f = MultipleOutputs()  
In[7]: f  
MultipleOutputs((USEquityPricing.close::float64,), window_length=10)

In [8]: f.lows < f.highs  
Out[8]: NumExprFilter(expr='x_0 < x_1', bindings={'x_0': RecarrayField((MultipleOutputs((USEquityPricing.close::float64,), window_length=10),), window_length=0), 'x_1': RecarrayField((MultipleOutputs((USEquityPricing.close::float64,), window_length=10),), window_length=0)})  

The scary expression in the output of that last cell says that (f.lows < f.highs) is a Filter which, when evaluated, will compute f and then compute the expression x_0 < x_1 with f.lows bound to x_0 and f.highs bound to x_1.

my_filter = custom_factor.low < custom_factor.high  

results in

BadBinaryOperator: Can't compute RecarrayField > instancemethod  

Hrm...I can't reproduce this error. If I execute the code listed above, I instead get this:

In [5]: custom_factor.low < custom_factor.high  
---------------------------------------------------------------------------  
AttributeError                            Traceback (most recent call last)  
<ipython-input-5-12d92f2a97d7> in <module>()  
----> 1 custom_factor.low < custom_factor.high

/home/ssanderson/quantopian/zipline_repo/zipline/pipeline/factors/factor.py in __getattr__(self, attribute_name)
   1211             raise AttributeError(  
   1212                 'Instance of {factor} has no output called {attr!r}.'.format(  
-> 1213                     factor=type(self).__name__, attr=attribute_name,  
   1214                 )  
   1215             )

AttributeError: Instance of MultipleOutputs has no output called 'low'.  

I've opened a PR in Zipline to make that error also include what the options so that it's easier to find issues like this in the future.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

That was a typo just in this message, I didn't adapt the actual error I was seeing to Karen's example. I confirm that in my case the attributes accessed are the same of the labels defined in the outputs attribute of the custom factor.

I'm trying to produce a SSCCE to track the error down now, while in the same factor it works but I'm using two difference custom factor' subclass instances and comparing their "fields".

Btw this dotted notation access is a bit odd, you should have gone with a simple key access like f.['low'] and f.['high']. What happens when someone tries to use 'my-beautiful-indicator' as label and the dash won't allow translating it to a proper python identifier?

edit: typos

My issue was due to the fact that one of my custom factors had an output field called 'bottom'. I spent quite a few hours trying to figure what the hell was coercing my value, supposedly a RecarrayField instance, to a method before realizing that there could be an actual attribute masking it, in fact quantopian.pipeline.factors.Factor has a bottom() method.

This falls into the same kind of issues I was referring to earlier, you'll have to state in docs what strings are allowed and whatnot for outputs, and detect non-valid strings at build time.

Andrea,

Apologies for any trouble this issue may have caused. You are correct that the bottom method was overshadowing the output field, which is almost always not the desired behavior. We've released a fix that will catch any naming collisions and do one of the following: (1) if the output name cannot be used, raise a helpful error message, or (2) if the output name conflicts with a method name, return the output instead of the method. For example, "bottom" can still be used as an output name, but accessing the bottom attribute of a multi-output factor will now give you the output. The reasoning behind this is that most of the methods on factors would not make sense when used on a multi-output factor anyway.

Hope this helps,
David

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

In my algorithm, I have a dispersion matrix (asset x asset) as output, but the custom factor does not like the matrix. Is there a smooth way to handle this issue?

thanks for this feature