Back to Community
An Empirical Algorithmic Evaluation of Technical Analysis

At a recent meeting of the Quantopian staff journal club, I presented a paper by Andrew Lo, Harry Mamaysky, and Jiang Wang called Foundations of Technical Analysis: Computational Algorithms, Statistical Inference, and Empirical Implementation (2000). In this paper, the authors utilize non-parametric kernel regression to smooth a stock's daily price time series to a point where the local minima and maxima that a human technical analyst would find relevant can be separated from noisier short-term price fluctuations. The authors then search these denoised local minima and maxima for the patterns commonly pursued by technical analysts. Once they've identified occurrences of particular patterns, the authors test their predictive power by observing the subsequent forward return on the stock.

For a fuller explanation of what is going on in this notebook, I encourage you to take a look at the original paper: https://www.cis.upenn.edu/~mkearns/teaching/cis700/lo.pdf

It is interesting to note that since this paper was written in 2000 and all the data used in my implementation is from 2003-2016, my results can be considered to be "out of sample" with respect to the authors' findings.

As I discuss in the notebook, one of my concerns with the author's methodology is the introduction of lookahead bias via the kernel regression. I'm eager to see how these technical patterns perform as predictors when implemented in an actual trading algorithm. (I'd love some help getting this analysis running as an algorithm on Quantopian.) I imagine we could use Pipeline to scan for patterns on a 40 day lookback window for a large universe of stocks. I'd also be interested to see how this pattern detection works when we smooth price timeseries using an exponential moving average instead of kernel regression.

If you're not familiar with research notebooks, what you need to do is click "Clone Noteook" (you need to be logged in first) and then you can page through the step-by-step of my analysis. You can also "run cell" from the beginning and reproduce all of the results that I did.

Loading notebook preview...
Notebook previews are currently unavailable.
Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

19 responses

This may help.

Wonderful effort Q. I applaud the vast array of interesting papers you guys have published.

Despite my deep scepticism I will certainly contribute to this once I have the time.

I believe an Event Study could give us much more information regarding those technical patterns. So I added an event study at the bottom of your NB. With this information it should be easier to build an algorithm as we can better decide when to enter/exit our positions once we detect a pattern.

P.s. Andrew, your posts are always interesting, thanks for your work.

Loading notebook preview...
Notebook previews are currently unavailable.

I was puzzling yesterday about currency conversion and back testing on patterns.

On trend following I have decided the "correct" way to deal with a multi currency Portfolio is to convert all time series to the base currency prior to running my daily orders (and of course for any backtesting). In my view there is little point in declaring an entry into the yen version if the Nikkei if you are a dollar investor - you need to take the currency into account to check the trade still makes sense I'm dollars.

Many backtesting programs work with native currencies and only convert the P&L back to the base currency.

While my method is fine for long term trends, I suspect it would create havoc with other types of pattern such as those being considered here.

you need to take the currency into account to check the trade still makes sense I'm dollars.

This is offtopic but you could also hedge the position on FX market so then index returns would then basically be in USD. If you trade yen based nikkei equity without hedging you are actually trading both the equity and yen at the same time (this might not be a problem at all but it's worth noting).

Mikko M
Agreed but in trend following that is exactly what I think one should do - trade the trend in the Nikkei as that trend is modified by the currency. There are two other alternatives - take signals just on the Nikkei trend and ignore the currency except in calculating the daily P&L, or hedge as you suggest.

There are very few hedged ETFs around - most trade in one currency (eg the USD) with an un-hedged investment in yen assets. The market ensures the USD price remains in line both with the movement of the Nikkei and the USD/JPY.

I don't have much difficulty with any of these approaches but in other forms of pattern recognition, I suspect logically a dollar investor would be best to base his signals on the unconverted JPY time series. There again I have done no testing whatsoever and am just raising a "note to myself" for when I come to look at this later.

If there is any validity to a "head and shoulders" (if it has any predictive power) I imagine that may disappear if you convert the time series to USD.

In trend following I don't mind doing that. I'm an investors rather than a trader and if an up-trend in the Nikkei is outweighed by a down trend in the yen, I don't want to take the trade.

Incidentally I don't hedge. I want the diversification.

@Luca That event study is a great addition to this analysis. Thank you for sharing!

Some takeaways:

1) I'd want the algo to trade on each of these technical signals. A larger "N" is going to lead to smaller error bars on our overall forward return.

2) Most of these signals are significantly decayed after 6 days. (I'm still concerned that the kernel regression has introduced lookahead bias). I'd be interested to see how many of these signals would have been missed if we provided only {1,2,3,4,5,6,7} days of trailing price data after the last local maximums identified with the full range of available data.

3) Inverse Head-and-Shoulders appears to be a particularly useful signal. It occurs often and has a large, slow decaying forward return.

Do you notice anything in particular?

@Andrew, here is an algorithm that uses those patterns.

There is a pipeline factor (PatternFactor) that detect those patterns and return a code (an integer) any time a pattern is found.

You can see the factor has a class member "indentification_lag" that is a limit on how many days ago a pattern can be recognized: ideally we would like to recognize a pattern the day after his last local minimum/maximum is found, but in practice we need more days to detect the last minimum/maximum. The "indentification_lag" parameter is there to let us test different configuration.

After changing "indentification_lag" we are suppose to update the number of days a position is held accordingly: the latter we detect a pattern the less days we can hold a position. The number of holding days per event can be configured in before_trading_start function.

This backtest used an indentification_lag of 4 days and a holding period 2 days (as Andrew correctly reported "Most of these signals are significantly decayed after 6 days" )

Clone Algorithm
133
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5743fd32866d69120adf32b3
There was a runtime error.

I'm glad we looked at this in an algo. Identifying the last maxima/minima in a pattern is clearly a key limitation. It'd be interesting to run these tests on various kernel regressor bandwidth parameter values to see if there is an identification lag/smoothness combination that produces superior returns.

I wonder if Kalman filter smoothing would look any different? I'll try swapping one into the notebook.

Here is a factor tear sheet of PatterFactor (the pipeline factor used in the previous algorithms to identify the technical patterns). There is a tear sheet for identification lags of 1, 2 and 3 days. I expect the results to be similar to what we found in the event study (even though in this NB we analyse a shorter time period 2013 - 2016, for technical limitation of research environment)

Edit: updated NB

Loading notebook preview...
Notebook previews are currently unavailable.

Summarizing what we have done up to know....

  • The event study shows the technical patterns have some signal that we can exploit

  • The signal decays after ~6 days, though Andrew is concern that the kernel regression introduces a lookahead bias: indeed we cannot predict a local minimum or maximum without having collected some data(daily close price) AFTER the local minimum/maximum. This means we cannot use the full 6 days signal but possibly less days of it and would be interested to know how many.

  • Created a pipeline factor, PatternFactor, that detects the same patterns analysed in the event study. PatternFactor is configurable in the number of identification lag days to detect a local minimum/maximum. Obviously the less days we configure the less local minimum/maximum it finds and also, less days could mean detect false positive.

  • Created an algorithm with PatternFactor and experimented with identification lag of 1,2,3,4 and 5 days (more is pointless as the signal decays after 6 days)

  • The algorithm is not profitable, this can be due to bugs in the code :) or Andrew's concern is well founded. To understand the problem we created a factor tear sheet for PatternFactor

  • Studied PatternFactor with the Factor Tear Sheet and discovered that if we don't give enough identification lag days to the factor it is not able to detect
    the same patterns we studied in the event study. Meaning it detect local maximum/minimum that are false positive (they wouldn't have been detected as local maximum/minimum if we had provided more data)

The last conclusion was not clear simply looking at the last two factor tear sheets posted above. So here is a new NB that shows PatternFactor with 1,2,3 and 10 days identification lag (I never tried 10 days lag before ). In this last NB I also filter out stocks whose price is less than 5 dollars because the signal returns are very low and the transaction cost would make those stocks untradable.

My opinion is that with identification lag of 10 days the factor is able to detect patterns very similar to the ones seen in the event study. There are little discrepancy but we have to keep in mind that we are not using the same universe in the tear sheet and in the event study (as the pipeline recalculate its stock universe every day while the event study has a fixed universe for the whole time period examined).

If we look at the results of PatternFactor with 1,2 or 3 days identification lag we can see the recognized patterns are not similar to the ones seen in the
event study. In particular, the more lag days we have the more similar the patterns are to the ones in the event study. In my opinion that should corroborate Andrew's thesis/fear of lookahead bias.

It would be nice to have my conclusion reviewed by someone else, as it is very likely that I missed something (and we can never exclude bugs presence in the code).

Another thing I noticed is that PatternFactor with identification lag of 1 day, while detecting different patterns than the event study, generates a signal that last 1 day. Is not very strong but we could try exploiting it.

Loading notebook preview...
Notebook previews are currently unavailable.

The algorithm has few shortcomings.... but just to show the signal generated from identification lag of 1 day.

Clone Algorithm
133
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 57601b77d151f611a4f5750e
There was a runtime error.

Thanks all for contributing to this post but I have one question:
What look ahead bias is being referred to? That is, Isn't the period used for the kernel regression plus lag to observe the signal roughly equal to the time that a trader would take to identify the various patterns?

The look ahead bias is in the Andres'NB and in the event study NB (not in the algorithm and the factor tear sheet) and it is due to the kernel regression being run for the full time period in one shot. In reality you would have to run the regression every day instead, like a rolling kernel regression on the past 30 days of data or so. Why this matter? Because the kernel regression doesn't give you good smoothed values at the end of a time series, there isn't just enough data to do it. This means that in reality you cannot trust kernel regression in the proximity of the current date, as you run a regression every day, and the last local minimum/maximum needed to identify a pattern might become evident only some days later. By the time a pattern is identified there might still be some alpha left or not, but to understand that we should rewrite the NB code so that it performs the pattern identification using a rolling kernel regression to simulate live trading behaviour, differently we cannot trust the results as they are biased.

Thanks Luca....I guess I assumed that the NB was replicating the paper exactly. So since the authors were using rolling regressions, they had no look ahead bias as Andrew implied at the beginning of this post:

As I discuss in the notebook, one of my concerns with the author's methodology is the introduction of lookahead bias via the kernel regression...

Well, In his NB Andrew takes into account the detection lag when he calculates the returns. He assumes a 4 days identification lag that decrease the hypothetical returns. But still the NB might be considering patterns that wouldn't have been recognized in real trading, with a rolling kernel regression
It's not that hard to modify the code so that it uses a rolling kernel regression though.

For all the experts on this forum, is the paper by Lo on this subject dated? (do not mean disrespect for the paper here). Are there there any best practices/standard libraries in identifying these basic technical patterns since it appears that most of the financial investment platforms offer screening for these patterns such as finviz.com and interactive broker(?). Thanks.

Out of curiosity I updated the NB, now it uses Alphalens

Loading notebook preview...
Notebook previews are currently unavailable.