Back to Community
Help with Factor Scaling

Hi Everyone,

I'm trying to scale factors (from within Pipeline) to values between 0 and 1, using sklearn's MinMaxScaler. However, I'm running into errors that I can't figure out unfortunately. If anyone could help I'd be very grateful.

Attached is a Notebook with the errors I'm getting. Not sure if I need to use a CustomFactor or not, but I can't get it to work either way. Many thanks in advance.

Loading notebook preview...
3 responses

Joakim, the error you are seeing is UserWarning: MinMaxScaler assumes floating point values as input, got bool. That's the clue. You are passing boolean values to the custom factor. Probably unintentional. Here's the offending line of code

factor1_scaled = FactorScaler(inputs=[factor1.notnull()], mask=universe )

The input you are passing factor1.notnull() is a filter (ie boolean values) it's not the factor stripped of nulls. What you want is this

factor1_scaled = FactorScaler(inputs=[factor1], mask=universe & factor1.isfinite() )  

Pass the factor but then filter that factor with the mask. Notice too that it's probably wiser to use the isfinite() method. This will not only catch the nans but also the infinite values. Infinite values don't play well with the MinMaxScaler method. Best to avoid those too.

You may also want to set copy=False (rather than True) to avoid the additional step of copying the input array.

This is actually a nice way to normalize factors and an alternate to the 'rank' or 'demean' approaches.

Good post.

Loading notebook preview...

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Hi Joakim, similar to what Dan mentioned above, here's a notebook I've used when exploring alternative scaling options aside from the usual ranking / demean / zscore.

Loading notebook preview...

Hi Dan and Daniel,

Super helpful, thanks so much!!