Back to Community
Trend follow algo

This algo is a trend follow for major asset class ETFs. Since beta can be high when the stock is in clear up-trend of down-trend, this algo may not be suitable for the contest. I am sharing it because it may be useful for individual accounts. For that reason, the leverage is 0.9 to be safe.

Clone Algorithm
5608
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 56011b9feb228f0e27fb28c8
We have migrated this algorithm to work with a new version of the Quantopian API. The code is different than the original version, but the investment rationale of the algorithm has not changed. We've put everything you need to know here on one page.
There was a runtime error.
52 responses

thanks for sharing. I might trade this one.

Try it with some of the reverse ETFs like SDOW. 50% more return with same drawdown

Hi Chris. The script shorts stocks when it is in down trend, so adding SDOW (3x leveraged short DOW index) may be redundant. For example, when DOW index is going down, it will short DIA and long SDOW, and it will bet too much (4x) DOW falling compared to all other assets. On the other hand, it might make sense to add leveraged bond ETFs, even though they are not allowed in the contest, to make it volatile enough compared to equities.

Good clean algo. Useful frame work you have.

Use of regression.! I am loving my stay at Quantopian.

Naoki,

Thanks very much for sharing this. Agree that it's super helpful / education for me to see coding and how people are using this. Your code solved one of my questions about creating large universes of ETF's and then checking to see if they existed at the time or not, so that I can run max. length backtests. And all the code is very simple and clean to me (as a novice programmer). Thanks!

I like the linear regression for trend analysis also. I've seen that on other Quantopian systems, but hadn't seen that much on other platforms I was on. I would suggest that you use limit or vwap orders on the entries. Even with large ETF's, can have very volatility intraminute pricing (and sometimes huge movements) that can lose you a lot of money without that. A single bad trade could really hurt. But, that will depend on 'time decay' of the pattern.

I am curious if you have studied the 'signal decay' (how important is it to get really rapid fills for returns). If so, how do returns 'fall off' with only checking signals once every 3 hours, once a day, once a week, once a month. If the fall off is large, what causes it? Is it the entries being delayed or the stop loss amounts?

I am also curious about Universe bias (although the universe feels fair). How much optimization has gone into the selection of these ETF's? Have you run any backtests / studies on various universes that are non-correlated with one another, but tend to 'exhibit trend behaviors' to see if it's a general pattern. The 'ideal' would be to have Universe ETF's selected 'mechanically' based on the degree to which they have trended over the past X period. So, I would think about a much larger Universe of ETF's to start. I did run some tests with a lot more emerging stock market indices in the mix, and the performance was much worse (large DD since 7/2011 or so). In theory these markets should work well (China, Brazil, Russia, Taiwan, etc). So, that would be concerning.

(Some of the indices I mixed in:) 'FXI', # My addition below here
'EWJ',
'EWZ',
'EWA',
'IWM',
'IWC',
'EWW', # Mexico
'EWT', #Taiwan
'EWY', # South Korea
'RSX', #Russia

I am also curious about any stop-loss testing that you did. Your current rule is pretty cool. I don't get it 100%, but see that it's generally the slope of the regression line modified in some way by time period. How sensitive are the results to various stop loss amounts and formulas? I would likely look at a 'constraint' for the maximum stop loss. Have you looked at 'volatility based stops' based on the volatility of the position itself and/or of the overall market? I have sometimes found these effective in the past. Have you also just looked at flat 'percentage based' stop losses that are applied equally to all positions? Curious what you found.

And how sensitive is the system to slippage assumptions and or various rebalance periods?

There are some markets that I have heard CTA's say trend more and exhibit more trend like behaviors (i.e. Asian markets and energies), and it might be interesting to test on groups of just them as well - however most CTA's trade a lot of different strat's.

It will also be very cool to combine this with more position and risk management logic - i.e. a 'constraint based' risk management at the position, total asset class and 'market exposure' levels.
These could be done, for example, with an algo that matches the 'slope based' position sizing logical with some volatility (risk) constraints per position (so after the initial 'linear regression based position sizing is selected it is put into 'risk budget' analyzer and re-adjusted - so that it stays within any position size and asset class level constraints). As well as potentially constrained 'beta' for the long and short sides each week or whatever (so there is a max. long and short beta component).

Most of the pieces for building this exist on the site.

But it's gonna take me a few weeks to build it. But thanks a LOT for sharing.

If I get time (won't be till early next week), I will check and report back on some of this. I am also curious about 'general effectiveness' of the signal versus it only being effective on this basket. Anyway, those are just some initial thoughts.

Best,
Tom

I am also curious about some position size approaches that might work, such as:
a) divide the trend line slop by the Abs value of peak DD over the period to get the raw scores (so this is taken into account)
b) Creating Omega Ratio, Sortino and Information Ratio over the lookback period and then multiplying the trend line by the average of these - so that we have more risk adjusted 'trend strength.' One or several of these should work to give better position sizing results that are taking account of 'risk' not just 'slope'.
c) Calculate average trailing X period correlations of all 'longs' and all 'shorts.' Then adjust ('slope weights') * (1.5-the average pairwise correlation of the ETF with all other ETF's) / (peak DD over lookback). Something like this should work better than A above as it takes into account the recent correlation of the longs with the other longs being bought and the shorts with the other shorts.

Has anyone coded these 'basic ratios' yet? They may not actually work, as the rebalacing of the volatility may be a core piece of the returns and the system would then need vol to achieve best results. Just some ideas.

Great Job!

Thank you for sharing this interesting algo.

It seems very sensitive to various parameters:
- slope_min (BTW I don't get how you figure it corresponds to 7%)
- ETF selection: adding VEU, TLT, or SLV worsens the returns considerably

Also context.multiple does not seem to add much to the results versus context.maxlever alone and again, I don't really get what it does.
In my tests, the performance worsens considerably when I simulate at minute level suggesting maybe the algo is too quick to react?

Any thoughts?

Thank you for comments, every one. After thinking about your what you all told me, I have rewrote the script. Here you go.
Major changes;

Consider max drawdown - When entering into a position, the algo looks into max drawdown over lookback (thanks to Tom Austin). Drawdown works better than volatility because volatility could also go up when the stock is trending sharply, but not drawdowns.

Much broader universe - Departing from the original basic ETFs I chose arbitrary (thanks to Tom, Pierre, Chris), I now use top 5% trading volume universe. It is free from look ahead bias.

The result yields quite humble return for 10-year, but with impressive low drawdown, even through the financial crisis.

Clone Algorithm
5608
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5609bcf801929f0f2c794609
We have migrated this algorithm to work with a new version of the Quantopian API. The code is different than the original version, but the investment rationale of the algorithm has not changed. We've put everything you need to know here on one page.
There was a runtime error.

Hello Naoki,
What would you say are the "optimizeable" parameters in this algo? Could you possibly identify them?

Thanks!
Adam

Naoki,

Great script, elegant code. Thanks for sharing.

Does the algo only enter new positions on trend change? So if a trend is established already, it won't jump onto it until the trend changes? Just looking at this bit from the entry section, which requires a cross of price action and regression line to enter a position.

 if slope > slope_min and delta[-1] > 0 and delta[-2] < 0 and dd < context.maxdrawdown:  
                context.weights[s] = slope  
                context.drawdown[s] = slope_min  
                log.info('/     Long  a = %+.2f%% %3s - %s' %(slope*100, s.symbol, s.security_name),gain)  

Actually 'delta' is the distance between the current price and the trend line, and not the change in trend line slope.

For example (with long position) it does not enter into position when:

  • Trend is established, but price keep going up above and away from trend line (too late to enter)
  • Trend is established, the price crosses the trend line downward. (wait till it comes up)
  • Trend is established, the price is swinging a lot and crosses the trend line multiple times (because 'dd < context.maxdrawdown' will not be FALSE)

Thanks for clarifying the 'delta'. What I mean, is that if the algo is started today, we willl need to wait until trend changes to enter any positions, correct?

Naoki, thanks for sharing this algo. And thank you for making it easy to understand. I have learned a lot by reading your algo, and it has given me more ideas to try. At the very least, my Python skills are slowly improving! :)

BTW, I have copied your "get_gain" function to a Github community site that I have started. I hope you don't mind!

https://github.com/tristanbob/Quantopian-Community

Naoki

I'm really impressed with this algo for larger time frames. I was curious to see how it would perform on intraday trading, using minute bars, and obviously using different values for the context variables. I figured I would have to tweak it considerably to reduce the very high friction due to bid/ask spreads, but still expected to get moderately positive gains.

Unfortunately, I seem to be doing something very wrong, as it is buying and selling way beyond the limits of the portfolio. This despite your implementation of the "order_target" as opposed to "order" criteria.

I suspect the problem lies in the loop that runs every 15 minutes shown below.

 for i in range(total_minutes):  
        # Every 15 minutes run schedule  
        if i % 15 == 0:  
          # This will start at 9:31AM and will run every 15 minutes  
          schedule_function(regression, date_rules.every_day(),  time_rules.market_open(minutes=1))  
          schedule_function(trade, date_rules.every_day(),  time_rules.market_open(minutes=1))  
          schedule_function(trail_stop, date_rules.every_day(),  time_rules.market_open(minutes=1))  
          schedule_function(execute, date_rules.every_day(),  time_rules.market_open(minutes=1))  
    ```  
I will struggle with this further, but perhaps you are curious yourself and / or have already successfully dealt with this issue.

By the way, as you can see in the code, I intended to introduce a "limit" order as I have had bad experiences in day trading when dealing with market orders. I had to comment that part of code out because my limits were getting set to 0.00, which resulted in no orders ever getting placed. It's probably fairly trivial to decode that , so I'll get to that later.

I look forward to any hints/suggestions you may have.  
Clone Algorithm
69
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 566cf882334cb6115a91fa30
There was a runtime error.

You might have already figured it out, but you forgot to add the 'i' here.

          schedule_function(regression, date_rules.every_day(),  time_rules.market_open(minutes=i+1))  
          schedule_function(trade, date_rules.every_day(),  time_rules.market_open(minutes=i+1))  
          schedule_function(trail_stop, date_rules.every_day(),  time_rules.market_open(minutes=i+1))  
          schedule_function(execute, date_rules.every_day(),  time_rules.market_open(minutes=i+1))  

Noticed something that makes an opportunity for a sidenote tip: You can see in the code that the metrics, like those above the chart in the IDE (or full backtest too) are copyable, good for everyone to know. Now, taking that another step, can copy/paste the metrics and also apply a macro to them in an editor for like:
tr121.3% br118.8% a0.05 b0.29 sh0.72 s0.88 ir0.02 v0.10 d16.7%

Naoki,

Good catch. That did the trick. Of course, as I suspected it does not make money on intraday, but I want to isolate this further, because if you
eliminate all friction and trading costs, in theory it should be profitable, yet it is not. I'll play around with my variable settings and let you know how I fare.

Thanks again.

Serge

Thanks for sharing, Naoki.

What's the motivation of setting the weight to be slope(ontext.weights[s] = slope)? is it a magic number?

Hello and first of all Thanks for Sharing!

I am trying to analize the algo and maybe this can help someone, i wanted to visualize the value used as a momentum indicator ( slope )
So this is my first question: why you consider a/b and not just the slope a?

(just SPY in this first notebok )

Loading notebook preview...
Notebook previews are currently unavailable.

In this algo I used a/b to be the slope to normalize the price difference.

Stock X: If it is at $100 at day 0, went up average of 1% per day. a will be 1.0 and b will be 100. a/b will be 0.01.
Stock Y: If it is at $10 at day 0, went up average of 2% per day. a will be 0.2 and b will be 10. a/b will be 0.02.

From the slope = a/b, you can tell Stock Y is trending up twice more than Stock X. The algo then put more weight on Stock Y than Stock X by applying

context.weights[s] = slope  

This example is perfectly clear thank you very much -Giuseppe

well-written and brilliant algo! Thanks for sharing!

@Naoki - Great and simple code! Thanks for sharing! Have you tried trading it live?

@Naoki - I really enjoy your algo and the backtests look very promising. I am going to put it for live trading soon. However, my paper trading result shows it requires at least two months to "warm up" before it starts to place the first trade.
Do you have any suggestion how to reduce this period for this algo?
Thanks.

A way to "warm it up" beforehand is to run the backtest until the day before you run the live trade (say, Friday evening). Using the print function, write the current holding and the parameters (in this case, context.weights and context.drawdown.) Then, hard code the initial holdings in the 'initialize function.' (it will start off on Monday morning). You can check the current date and only output the holding on the last day.

hey all. does this algo work on Quantopian 2?

Very good algo! Thanks for sharing.

@Tybalt: If you clone the backtest at the top of this thread, you will get the Quantopian 2 version.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

thank you sir

the algorithm migrated to Quantopian 2 version, have the obviously difference performance compare to original version!
Could give the advice to check the reason?

Big kudos to Nagai-san. indeed very good investment thesis behind the algo.
3 source of profit: asset alloc, stock selection and market timing. Nagai-san's v1 and v2 touched 2 of the 3. Thank you.

Yes, When back testing in Quantopian 2 The algo does not go short during the 2008 Lehman bankruptcy it just takes no positions.. Why is this different?

Hi Naoki,

I cloned your algo and start the backtesting. I haven't changed any code. But the "Total Return" is very terrible. :-/

Clone Algorithm
9
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5774e60ff904011240035dd6
There was a runtime error.

It seems to be quite hard to write code that works in today's market. Perhaps all the quants have canceled all the information out of it?

Me too, the result is totally different now... someone may to explane why?
The code is Very well done for learning purpose for newbee as me.. thanks

It looks like the original algo was migrated to 2.0 incorrectly. See attached the up-to-date return of the original algo. Still need adjust the code to remove the depreciated functions.

Clone Algorithm
42
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 580d6fef640e79105f0c0391
There was a runtime error.

Right!
Then, this is my small contribution to the Q community: removed all deprecated code.
Any ideas how to improve it?

Clone Algorithm
171
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 580dd0478f5cc9104c987e94
There was a runtime error.

Thanks Michele for fixing the code. I think that another great improvement can be the integration of the volatility based position sizing, for example as described in Robert Carver's book Systematic Trading (http://qoppac.blogspot.ca/p/systematic-trading-book.html).
Simon Thornington tried to implement this logic in his algo here https://www.quantopian.com/posts/exponentially-weighted-moving-average-and-standard-deviation-in-pipeline
Unfortunately, his algo didn't work as intended. Maybe someone can try implementing this logic to adjust position sizing of the algo?

The code doesn't look fixed at all. The new performance does not match the original performance nor the adjusted performance after Naoki modified to avoid look-ahead bias.

A.W. , please note that the diffrence in the performance may come from the different initial capital $.
The most recent version of the algo run with $10K of the initial capital vs. $100K in the original algo

Here is my attempt to plug the position volatilty adjustment (credit to Simon's algo here https://www.quantopian.com/posts/exponentially-weighted-moving-average-and-standard-deviation-in-pipeline)
The algo adjusts the leverage to keep the same volatility. In some cases the leverage can go up to 3x+, so might not be appropriate for many retail brokerage accounts, but it achieves 2x performance of the previous algo with very similar drawdowns

Clone Algorithm
599
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5810b4250e428c11f886714f
There was a runtime error.

unbelievable stuff!

Just read the code. But the point is, the context.price is defined as none at first, so how to compare it with 0? I mean, if context.stopprice[s] < 0:, how does it work?

With a request of being pardoned for ignorance of python and statistics I have a silly question. I would appreciate an answer. If y=ax+b is the equation then 'a' is the slope and 'b' is the intercept. In the code segment I don't understand the part "slope = a / b * 252.0 # Daily return regression * 1 year"
+++++++++++++++++++++++++++++++++++++++++++++++

Run regression y = ax + b

    results = sm.OLS(Y,A).fit()  
    (b, a) =results.params  

    # Normalized slope  
    slope = a / b * 252.0        # Daily return regression * 1 year  

++++++++++++++++++++++++++++++++++++++++++++++++++++++

If 'a' is the slope what does a/b give, if a is not the slope what does the ols function return ?

Thanks!

Just ran a back test on this, is there a Robin Hood friendly version of this algo? this pumps through a ton of trades, you would very quickly be labeled a day trader on Robin Hood.

Let me know!

This algo was originally written with ETFs to be exposed to diversified asset classes, but now you can this on futures. Here is an example.

https://www.quantopian.com/posts/futures-trend-reversion-algo

That's awesome that you already have a futures version of the algorithm, Naoki. Thanks for posting it!

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

And....POOF....just like that, this algo went from looking GOOD....to not keeping pace with the benchmark.

Tried to run previously cloned algo TODAY and it gave me an error message on line 149. I thought OK, maybe I had change something, so i cloned it just now and a pop-up box came up and said the ALGO had been migrated to Q2......which seemed like a good idea, until I ran the algo....indeed the logic of the ALGO was still there, but the back test results are 'spectacularly' LOWER.....lower even than the benchmark.

I read the Q2 info and there could be a number of things that caused this, but in general, I am quite discouraged that even Q2 will not give accurate results as what may 'actually' occur in real time trading ?!

Hello Blaine,

We're constantly improving the accuracy of our backtesting system, one of the ways we do that is open sourcing our engine (amongst many other repos). https://github.com/quantopian/zipline

Algorithms are complex systems with serial state-dependent behavior, small changes can throw off algorithm. That said, good algorithms are generally more robust to small changes, and algorithms being massively different after a small change can imply (amongst other things) that
A: The algorithm was overfit and oversensitive to small parameter changes.
B: There was a bug in the backtesting system that was falsely causing the algorithm to look good.

This is more common than you might think. Overfitting is incredibly common and nobody is immune. As Marcos Lopez de Prado pointed out in his recent QuantCon keynote, backtest overfitting is likely the single biggest problem in finance right now (https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2308659). Part of the reason this occurs is that if you find an anomaly, you'll likely try to dial into it and take more advantage of that behavior in your data. A data bug or quirk of the data can look no different from a true returns inefficiency. We've also released some research about how common overfitting is here https://blog.quantopian.com/using-machine-learning-to-predict-out-of-sample-performance-of-trading-algorithms/.

Sometimes a small change to the algo is all that's needed to get it back on track after a code change like this. Sometimes what our improved backtest system revealed is that the algo was never going to work in the first place. Backtesting is always at best an estimate of what will happen, and that's why it's important to use many other techniques to determine whether your algorithm will be okay, and to determine where the failure point actually is. I discuss this more here: https://blog.quantopian.com/idea-algorithm/ In my opinion, backtesting should just be used to check if the algorithm survives real market conditions like slippage and trading costs. You should know your model is predictive before backtesting. That's what alphalens is for.

If you believe you've discovered a bug or inaccuracy, please let us know more about what's going on at [email protected]. It's incredibly helpful to us and everybody who uses Quantopian/Zipline to receive feedback that will make our systems better.

Lastly, for Quantopian's allocation purposes you shouldn't worry about beating the market. It's not a good benchmark unless that's precisely your target. What's interesting to us is algorithms with alpha -- returns which are not correlated with other risk factors or strategies. Even with low returns, an uncorrelated algorithm can be combined with other strategies or levered to the point of usefulness. In general if the strategy makes money and has well controlled risk exposures, it doesn't matter whether or not it beats the market. It's cheap to buy market exposure through ETFs, so the goal of a single strategy is to be an diversified investment.

Everyone has probably long moved on from the original code, but like Pierre Warnier, I'm having a lot of trouble understanding how a slope_min of 0.252 corresponds to yearly growth of 7%.

Slope = a/b * 252
Assume slope_min = 0.252
0.252 = a/b * 252
0.001 = a/b
Using the example from Naoki Nagai,
"Stock X: If it is at $100 at day 0, went up average of 1% per day. a will be 1.0 and b will be 100. a/b will be 0.01." Obviously in this case, a/b of 0.01 is greater than 0.001, so this 1% per day increase stock qualifies.
Now consider a hypothetical Stock Y: If it is at $100 at day 0, and went up 10% in a year (0.0378% per day average as this gives a 10% increase over 1 year). a will be 0.0378 and b will be 100. a/b will be 0.000378 which will be significantly less than 0.001 which means that it doesn't qualify as exceeding the slope_min, even though it rises 10% in a year which is over 7%.
Apologies for any confusion, but I still can't figure out how 0.252 slope_min corresponds to 7% increase per year.
Thanks.

It is interesting to read a thread on trend following, bearing in mind what has been happening to trend following CTAs over the last few years. Who knows why but a strategy which made hundreds of millions for the likes of JW Henry and Bill Dunn suddenly stopped working and may or may not "come back". At least so far as a diversified basket of futures is concerned.

Broadly speaking my belief is that successful "probability" and pattern trading requires the participant to be in the right place at the right time. It is about luck rather than skill; or at least given enough skill not to make a complete fool of yourself you will still get nowhere without luck.

It may be that trend following comes back into profit. It may be that it is too overcrowded.

Ironically of course all investment relies on trends. If a market does not trend upwards over time you will make no profit from it.

The question therefore becomes "to trade or not to trade".

If world economic growth continues over the next hundred years you can expect to profit from a simple buy and hold of an index. Preferably a world index and preferably an index which is leaning more towards equal weight that market cap.

The trouble is with this industry is that the vast majority of participants are wrongly diagnosing luck for skill.

Other than that the major problem is the incurable disease of overfitting a system to the data.