Back to Community
add new column after pipeline run - deciles accounting for industries

Hi, I am quite new to python and dont fully understand limits and capabilities of the language.
I am trying to run a backtest where I look at my fundamental factor which is a ratio computed across Q1500 stocks within their industries and want to go short 10th deciles and long 1st deciles stocks.
It wouldnt be that complicated if I looked at the fundamentals across whole Q1500, but i want to account for industries because top/bottom decile might be skewed towards some particular industry.
I came up with this code but obviously it doesnt work, its breaking after return pipe.
Any idea/solution would be much appreciated, thanks.

    pipe = Pipeline(  
        columns={  
            'n1': NI1,  
            'n2': NI2,  
            'ts': TD,  
            'recent_indebt' : recent_indebt,  
            'sectors' : sectors  
        },  
        screen = (sector)  
        #mask = universe  
    )  
    return pipe

    dec = pipe.groupby(['sectors'])['ts'].transform(  
                     lambda x: pd.qcut(x, 10, labels=range(1,11)))  
    pipe.add(dec, 'dec')  
    low_returns = pipe['dec'].eq('10')  
    pipe.add(low_returns, 'low_returns')  
    pipe['high_returns'] = pipe['dec'].eq('1')  
    pipe.add(high_returns, 'high_returns')  
5 responses

You can easily compute the deciles from the factor itself (pipeline factors have some useful methods like .deciles) and then you can add the results to the pipeline itself:

from quantopian.pipeline.classifiers.morningstar import Sector

    pipe = Pipeline(  
        columns={  
            'n1': NI1,  
            'n2': NI2,  
            'ts': TD,  
            'ts_dec': TD.deciles(groupby=Sector()),  
            'recent_indebt' : recent_indebt,  
            'sectors' : sectors  
        },  
        screen = (sector)  
        #mask = universe  
    )  
    return pipe

thanks Luca, I get the below error. Am I missing something?

TypeError: deciles() got an unexpected keyword argument 'groupby'
...

Hello Peter,

The .deciles() attribute method does not have a 'groupby' parameter. An alternative would be to first zscore securities grouping them by sector, and then apply the deciles method. Here is an example using Luca's code from above:

from quantopian.pipeline.classifiers.morningstar import Sector

    pipe = Pipeline(  
        columns={  
            'n1': NI1,  
            'n2': NI2,  
            'ts': TD,  
            'ts_dec': TD.zscore(groupby=Sector()).deciles(),  
            'recent_indebt' : recent_indebt,  
            'sectors' : sectors  
        },  
        screen = (sector)  
        #mask = universe  
    )  
    return pipe  

This should give you a result close to what you are looking for. I hope it helps.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

oh, I was too optimistic, I expected .deciles method to have the same arguments as .zscore and .rank. Actually, I am not sure why 'groupby' is not in .deciles.
Thanks Ernesto for the correction and workaround.

Thanks both for the advice