Slope calculation

The idea is to come up with an average slope for a curve, or trend line for a given lookback window.
Would like -90 degrees to +90 degrees. Of course, just stretching a graph horizontally would change those numbers.
Edited 2017-06-17
Edit 2017-12-22 Backtest on this date below has an example without the loop, multiple stocks to history and slopes all at once.

'''
Slope calculation using statsmodels.api
'''
import statsmodels.api as sm

def initialize(context):
context.sids = symbols('TSLA', 'AAPL')

def handle_data(context, data):
for s in context.sids:
slp = slope(data.history(s, 'close', 60, '1m').dropna())
if slp > .05:
print slp
order_target_percent(s, .5)
elif slp < -.05:
order_target(s, 0)

def slope(in_):
return sm.OLS(in_, sm.add_constant(range(-len(in_) + 1, 1))).fit().params[-1]  # slope

20 responses

You might want to normalize the prices into cumulative percentage returns, so that you could interpret the slope as best-fit return-per-day. You might also want to fix the y-intercept at 0; that might be what you are attempting with the time, I'm not familiar off-hand with the various ways of calling OLS.

Rather than fitting, you might just sum the returns, as illustrated in the attached backtest.

22
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 56852f414465b7117c9df501
There was a runtime error.

Not to ignore the others, I was working with normalize, so here's that. The summary is that I can't trust the code I posted. (Is my normalization right, another question mark). Anything that takes a list like prices or portfolio and returns a slope analog would be fine.

Looking at prices between the two dates here, first column, lastval in the normalized list.

2015-05-20 slope_calc4:34 INFO lastval 75.18        slp 8.17
2015-05-21 slope_calc4:34 INFO lastval 73.55        slp 8.26
2015-05-22 slope_calc4:34 INFO lastval 90.78        slp 6.78
2015-05-26 slope_calc4:34 INFO lastval 84.53        slp 5.59
2015-05-27 slope_calc4:34 INFO lastval 75.04        slp 4.85
2015-05-28 slope_calc4:34 INFO lastval 100.00        slp 5.15
2015-05-29 slope_calc4:34 INFO lastval 89.66        slp 5.39
2015-06-01 slope_calc4:34 INFO lastval 69.52        slp 3.98
2015-06-02 slope_calc4:34 INFO lastval 55.34        slp 4.74
2015-06-03 slope_calc4:34 INFO lastval 65.45        slp 5.93
2015-06-04 slope_calc4:34 INFO lastval 21.35        slp 3.79


The code has slope at a positive value, 3.79 while the eye says downward and so does Excel.

9
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 56870778040d3d117058dc1e
There was a runtime error.

The GK code was doing a diff SPY to TLT. Extracting from it, simplifying, couple of questions:

1. Can this be used as a slope analog?
2. Is it possible to adjust values -90 to +90? (I think the answer is no)

Perf wasn't the point above so this is just a sidenote, the GK code appeared to have a consistent leverage around one, that's because it missed intraday, was actually -62% cash. To avoid that, track max intraday leverage, and at some point we might have an in-house context.account.max_leverage -ish to make that easy except the todo list is a mile long and new features take priority.

You can compare the benchmark curve and slope-ish values in the custom chart here. The lookback window is 11.
Should that really be 0 on Apr 18? Positive on Oct 24? I think the answer is no. If so, what's wrong?

1
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 56871009bce84311690a18b7
There was a runtime error.

Playing around starting with the Market Tech code, adding Grant K's slope analog and some of my tools.

15
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 56872f5cc15b711180c3e5d9
There was a runtime error.

Noticed there's some code that handles slope for all securities at once in a dataframe here, below is the general idea.

    long_window  = history(bar_count=250, frequency='1d', field='price')
short_window = history(bar_count=190, frequency='1d', field='price')
long_slopes  = reg_slopes(long_window)
short_slopes = reg_slopes(short_window)
for sec in data:
if long_slopes[sec] > 0:  # and long_now_prcnt > long_avg_prcnt * 1.1:
elif short_slopes[sec] < 0:  # and short_now_prcnt < short_avg_prcnt * .9:
# sell

def reg_slopes(df):
''' Returns the slope of the regression line for series '''
x = range(len(df))
slopes = {}
for col in df.columns:
slopes[col] = sm.OLS(x, p1).fit().params[col]
return pd.Series(slopes)



These are all wrong so far and will produce false signals.
One way or another we ought to be able to come up with slope analog values that can be trusted.

reg_slopes() above may be efficient however, now having taken a closer look, is not really taking the full window into account somehow. Try a short window like 10. You'll see negative values when the trend is clearly up, for example. Maybe someone who knows OLS can take a look. And maybe someone who knows pandas can try normalizing the prices for the one above.

Meanwhile here's an alternative for the top example that [I was surprised to find] returns similar numbers without the need for importing statsmodels, using the more common import, numpy. This might be more useful/efficient with some sets of code if it were taking in a pandas dataframe instead like reg_slopes().

import numpy as np

def slope_calc(in_list):
vals = []
for i in range(len(in_list) - 1):
vals.append(in_list[i+1] - in_list[i])
return np.mean(np.array(vals))


@garyha: Isn't this the same as (in_list[-1]-in_list[0])/(len(in_list)-1)?

Your vals is [in_list[1]-in_list[0], in_list[2]-in_list[1], ..., in_list[-1]-in_list[-2]]. Then np.mean will return sum(vals)/len(vals) == ((in_list[1]-in_list[0])+(in_list[2]-in_list[1])+...+(in_list[-1]-in_list[-2]))/(len(in_list)-1) == (-in_list[0]+in_list[-1])/(len(in_list)-1) == (in_list[-1]-in_list[0])/(len(in_list)-1), the slope of the line passing through the first and last point.

Now consider the sequence 1, 2, 3, ..., 1,000,000, -1,000,000. The regression slope should be just a little less than 1. Your "slope", based only on the first and last point (because intermediate points cancel out), would be about -1.

Slope as a pipeline factor something like this:

import statsmodels.api as sm

def make_pipeline(context):
pipe = Pipeline()

[...]
return pipe

class Slope(CustomFactor):
inputs = [USEquityPricing.close]
def compute(self, today, assets, out, closes):
out[:] = slope(closes)

def slope(in_):     # Return slope of regression line. Make sure this list contains no nans or screen its output later
return sm.OLS(in_, sm.add_constant(range(-len(in_) + 1, 1))).fit().params[-1]  # slope


Thanks for the share.

Thank you, Blue, Market Tech and everybody for the input. It is very useful info. Does anybody know if there is a way to calculate slope for MACD array?

Yes, like macd_slope = slope([list or ndarray of macd values]).

import statsmodels.api as sm
def slope(in_):     # Return slope of regression line. Make sure this list contains no nans or screen its output later
return sm.OLS(in_, sm.add_constant(range(-len(in_) + 1, 1))).fit().params[-1]  # slope


Edit 4/2018 Returns wild numbers if all inputs are equal

Thank you
Do you thign this would work?...

import statsmodels.api as sm

def make_pipeline(context):

pipe = Pipeline()

[...]
return pipe


class Slope(CustomFactor):
macd = talib.MACD(prices[stock], 12, 26, 9)[0][-1]
inputs = macd
def compute(self, today, assets, out, closes):
out[:] = slope(closes)

def slope(in_): # Return slope of regression line. Make sure this list contains no nans or screen its output later
return sm.OLS(in_, sm.add_constant(range(-len(in_) + 1, 1))).fit().params[-1] # slope

Outside of pipeline, forget about the class, just feed a list to the def, an output from talib.MACD, that's simple. Inside pipeline, maybe someone can find a way.

I attempted to get R2 in the same manner inside pipeline as well.

r2 = sm.OLS(in_, sm.add_constant(range(-len(in_) + 1, 1))).fit().rsquared

But it doesn't work for me.
Any suggestions

ValueError: shapes (60,8216) and (60,8216) not aligned: 8216 (dim 1) != 60 (dim 0)

Multiple stocks to history and obtaining slopes on them all at once. Then decided to feed those slopes to Optimize (the real market would react to such high volume momentum trading in a way backtesting cannot).

8
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5a3da8bf8f151a40a50d206d
There was a runtime error.

As a factor, for slope I've switched to this adapted from another member:

from scipy.stats import linregress
class Slp(CustomFactor):
inputs = [USEquityz.close] ; window_length = 80  # defaults. can override on calling
def compute(self, today, assets, out, z):
slopes = np.empty(len((z).T), dtype=np.float64)
x = np.arange(len(z))
i = -1
for col in np.log(z).T:
i += 1
if np.allclose(col, col[0]) or np.all(np.isnan(col)):
slopes[i] = 0
continue
slope, intercept, r_value, p_value, std_err = linregress(x, col)
slopes[i] = (np.power(np.exp(slope), self.window_length) - 1) * 100 * r_value**2
out[:] = slopes


Called something like this:

    m   = AnnualizedVolatility(mask=QTradableStocksUS()).top(90)
roa = Slp(inputs=[Fundamentals.roa], window_length=88, mask=m)  #.rank()


Hi Blue Seahawk,

I sent you a message regarding slope trading.

Also, it seems like there's sort of strange behavior in the algorithm, clearly I'm doing something wrong.
I'm really interested in this idea of triggering trades based off a positive and negative slope and the numbers around zero indicating time to enter or exit a position. I hope you are still working on this as I am too. I'm just not that great of a coder, so when I come to a roadblock, it takes me a bit to get over it.

Backtest with applicable code is attached. (I used your initial slope calculation system to get the slope data, it works for the first few minutes, but then I get this error message "ValueError: zero-size array to reduction operation maximum which has no identity
There was a runtime error on line 27."

I desperately want to figure out this system on the Quantopian platform. Can you help me integrate your post above within the algorithm I have attached? Is the error message the reason why you changed your method for getting the slope?

5
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5be1116b256637422531b04f
There was a runtime error.

Mainly add .values to the history call.

I had used these slope methods a lot, although not for awhile and coincidentally dug back into the first one today before seeing your post.

Then I sort of went to town on your backtest, try working with some of these things ...

8
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5be148a9256637422531bd10
There was a runtime error.

Hey Blue Seahawk,