Custom Factor Issue

I am trying to create a custom factor that returns +1 if the trend is positive, -1 if the trend is negative, and 0 if there is no trend. Direction of the trend is calculated by a fast MA and a slow MA.

My plan is to use the result as a multiplier for a future calculation in the algorithm, so that I can return positive numbers when the trend is up, and negative numbers when the trend is down.

I am not experienced in Python, and am really struggling with Custom Factors. Am I on the right track here? I am receiving errors when I try to use the result in pipeline calculations.

Error Message: NonWindowSafeInput: Can't compute windowed expression trend_direction((NumExprFactor(expr='x_0 / x_1', bindings={'x_0': SimpleMovingAverage((USEquityPricing.close::float64,), window_length=9), 'x_1': SimpleMovingAverage((USEquityPricing.close::float64,), window_length=20)}),), window_length=1) with windowed input NumExprFactor(expr='x_0 / x_1', bindings={'x_0': SimpleMovingAverage((USEquityPricing.close::float64,), window_length=9), 'x_1': SimpleMovingAverage((USEquityPricing.close::float64,), window_length=20)}).

fast_ma = SimpleMovingAverage(inputs=[USEquityPricing.close],window_length=9)
slow_ma = SimpleMovingAverage(inputs=[USEquityPricing.close],window_length=20)

ma_ratio = fast_ma / slow_ma
class trend_direction(CustomFactor):
inputs = [ma_ratio]
window_length = 1
def compute(self, today, assets, out, highs, lows):
if ma_ratio > 1.0:
trend_direction = 1.0
elif ma_ratio < 1.0:
trend_direction = -1.0
else:
trend_direction = 0
def compute(self, today, assets, out, highs, lows):
if ma_ratio > 1.0:
trend_direction = 1.0
elif ma_ratio < 1.0:
trend_direction = -1.0
else:
trend_direction = 0

10 responses

I think I may be over-complicating this. Since I am already successfully calculating the ma_ratio, it seems that I could just subtract 1 and remove the decimal.

I have imported Numpy as np, and am trying to use the "around" function, but receiving errors. Any suggestions?

Again, the goal is to have the result be 1, -1, or 0

trend_direction = np.around(ma_ratio, 0)


Can you share the full notebook? Debugging snippets is very difficult - can't look at what ma_ratio is, the whitespace issues, stuff like that.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Thanks for looking at this Dan - I feel like I am close, but having issues with the "np.trunc" line

1
Notebook previews are currently unavailable.

Hi Dan S,
I am also very much a a "new struggler with python" and I was trying something similar to you, and also having problems.
Writing the line that didn't work in my code to the same context as yours, what I was trying was

trend_direction = np.sign(ma_ratio -1.0)

.... which didn't work for me either. I suspect the problem is in the "NonWindowSafeInput" part of the error message, if that helps at all.

Tony - thank you for pointing me out the numpy "sign" operator. That is perfect for what I am trying to accomplish.

It seems to me that I can't do this calculation within the pipeline. It needs to be done in a CustomFactor, then passed to the pipeline. I think I am getting close here - the results are properly formatted for a pandas pipeline output, but I don't seem to be getting accurate results for the "trend_direction".

The most complicated part has been trying to get two separate moving averages into one custom factor. Does it appear that I am doing this correctly?

class trend_direction(CustomFactor):
inputs = [USEquityPricing.close]
window_length = 20
#this is close, but it doesn't seem to be returning accurate data (trend_direction returns 1.0 for all stocks, instead of a -1.0 for stocks where the short MA is below the long MA)
def compute(self, today, assets, out, close):
fast_mavg = np.nanmean(close[-7:])
slow_mavg = np.nanmean(close[-15:])
out[:] = np.sign(fast_mavg / slow_mavg)


np.sign(fast_mavg - slow_mavg)

Blue - thanks for the input. I am not sure why I was previously seeking a ratio (with division) versus a simple subtraction.

When I change it to your code, I still receive 1.0 as the result in my pipeline for all stocks. Am I calculating the moving averages incorrectly (trying for a 7-day versus a 15-day)?

When remove the np.sign part of the out[:] statement, I get the exact same result for all stocks in the pipeline. It seems that I am not properly formatting this to get the result for all stocks.

Wow. I finally figured this out, and learned a lot about Custom Factors in the process. Thanks to Blue Seahawk and Tony Moreland who helped with a couple of simple things that weren't immediately obvious to me.

My final issue was that my custom factor was returning the same result for every stock. This was finally solved by adding axis = 0 to the moving average calculations.

Here is my final code for what turned out to be a pretty simple Custom Factor.:

#Custom Factor that returns a +1 if fast SMA is above slow SMA, -1 if they are inverted, and 0 if they are equal
class TrendDirection(CustomFactor):
inputs = [USEquityPricing.close]
window_length = 20 # be sure window exceeds longest MA
def compute(self, today, asset_ids, out, close):
fast_mavg = np.nanmean(close[-7:], axis=0) #7-day SMA - NEEDS AXIS!
slow_mavg = np.nanmean(close[-15:], axis=0) #15-day SMA - NEEDS AXIS!
out[:] = np.sign(fast_mavg - slow_mavg) #returns the sign (pos/neg) of the result


Nice going in figuring that out. In it you have a binary indicator and then another step could be to go with some of your earlier thinking as a sort of trend_strength as well, analog, producing positive and negative floats that Optimize API would be happy to turn into longs and short for you:

        out[:] = (fast_mavg / slow_mavg) - 1


And/or some parts of this, not intended as an answer, instead just food for thought in feeding those +/- signals to Optimize, where context.pipe.ma_ratio comes from the 'ma_ratio' pipeline column name added, something I had handy. Using that line above in your factor you could replace : ma, .... with .... : TrendDirection(), to use your factor and remove the SimpleMovingAverage() lines. If it goes south in a backtest try making that -TrendDirection() with a minus sign in front of it to flip the +/- signals upside-down and reverse long, short.

from quantopian.algorithm import attach_pipeline, pipeline_output
from quantopian.pipeline.factors import CustomFactor, SimpleMovingAverage
from quantopian.pipeline.filters import Q1500US, Q3000US
import quantopian.optimize as op

def initialize(context):
attach_pipeline(make_pipeline(context), 'p')

def make_pipeline(context):
m   = Sector().notnull() & Q3000US()    # mask
ma  = (ma2 / ma1) - 1

return Pipeline(
screen  = m,
columns = {
'ma_ratio': ma,
})

context.pipe = pipeline_output('p').dropna()

order_optimal_portfolio(
objective = op.MaximizeAlpha(context.pipe.ma_ratio),
constraints=[
op.MaxGrossExposure(1.0),
op.PositionConcentration.with_equal_bounds(-.015, .015),
op.DollarNeutral(),
op.NetGroupExposure.with_equal_bounds(
labels = c.pipe.sector,
min = -0.0001,
max =  0.0001,
),
],
)


This is very helpful, and along the lines of what I am trying to do next. I am trying to learn the language by re-creating interesting studies from other platforms.

My goal was to re-create the TrendQuality study provided by ThinkorSwim. The first few steps steps to the calculation are:

1. Create a binary indicator which tells us whether the fast_ma is above or below the slow_ma.
2. Determine the Cumulative Price Change of the stock since the cross of the two moving averages.
3. Calculate the moving average of the price change since the cross of the moving averages

There are further steps, but the end result is that you would have a final number, TrendQuality, that could be tested for alpha in Notebook and used in Algorithms.

Step 1 is complete, with my TrendDirection custom factor. I am stuck on Step 2, as I can't seem to reference the previous day's TrendDirection. I have tried to accomplish this as a Custom Factor, as well as within the pipeline itself, but no luck. Here was my attempt at a Custom Factor for cumulative price change after an MA cross:

#creating cumulative price change from a fixed point (reversal)
#should also be able to calculate "Days in Trend" (at +1 or -1)
class Cum_Price_Change(CustomFactor):
inputs = [USEquityPricing.close]
window_length = 252
#window_safe = True
def compute(self, today, asset_ids, out, close):
if TrendDirection[-1] != TrendDirection[0]:
price_change = 0
else:
price_change = (price_change[-1]) + (close - close[-1])
out[:] = price_change


The factor always seems to result in "True" for the first "if" statement. I am trying to ask:
1. "If today's TrendDirection is different than yesterday, that means it is a new trend, and we start with 0 as our price change"
2. "Otherwise, if today's TrendDirection is the same as yesterday, we must be continuing the previous trend, so we will add today's price change to yesterday's, and continue to add these totals together every day until the trend changes"

I suspect that there is circular logic in the second part of my if / else statement, but I can't get past the "if" to address the problems in the "else".

The thread How to create function that returns "time since last crossover" has some hints that this can be done, but when I try to adapt the logic in my pipeline with my custom factor TrendDirection as the equivalent of "only_signs", I can't get any results. I'm not even sure where to use this (CustomFactor, or within the pipeline).

Apologies for my confusion about what I think are probably some pretty basic fundamentals of the language.