Back to Community
winning algo drops below $90K

Here's my winning algo that dropped below $90K today and was stopped, per the contest rules, by Quantopian. --Grant

Clone Algorithm
1590
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 55d653d3764c260db1956402
We have migrated this algorithm to work with a new version of the Quantopian API. The code is different than the original version, but the investment rationale of the algorithm has not changed. We've put everything you need to know here on one page.
There was a runtime error.
50 responses

Thanks for sharing!

Yes, thanks for sharing this algo and for all your numerous contributions to the community. It was a tough market to beat. I hope that your next contest winning algo draws a healthy profit.

@Grant: Congrats on a nice effort. We have all learned a lot from your experience and knowledge. (I enjoy reading old forum and blog posts from ~2012)

Any thoughts on why it didn't perform well?

Is it true that this algorithm would make trades everyday on every stock? $18/day * 20/(days in a month) * 6 months = $2,160 (around 2% of the portfolio). This algorithm would have lower commissions with weekly rebalancing, but would it perform better/worse?

@Grant
If you don't mind sharing this info...
What criteria did you use to select the securities that you used?
How did you select the 20 day history window utilized?
How did you select/optimize the value of epsilon?

I played with a copy of an OLMAR algo a little and felt that perhaps it was very sensitive to epsilon and even the characteristics of the selected securities. That concerned me. If that is in fact an issue, is it possible that the algo might be overoptimized?

I understand the need for the 10% drawdown rule used by Quantopian, especially since they don't actually know the mechanism and implementation details (eg. optimization, failsafes, etc.) of an algo. All they see is "the exhaust". Still, might even the most robust algo experience a 10% drawdown at some point in history due to a market anomaly which might not be visible outside the framework of the algo?

Grant, sharing your algorithm is so very helpful and the returns are most illuminating; your contribution on this alone is very admirable to this newbie. A couple of questions.

Was the inception date of live trading March 2? The month designation on the contest board is confusing to me with the change earlier in the week. I don't know why a simple inception date isn't indicated.

Is the displayed return series, starting at the beginning of 2013, the entire length of the backtest + live, the paper trading period + live or just a convenient starting point for illustrative purposes? It is notable that this period had few sustained drawdowns in the S&P500.

If March 2 was the inception date, did you notice the qualitative change in paper trading behavior Vs. live trade behavior immediately after inception date? For the month of February your algorithm outperformed the S&P500 by, jeesh 10% or more. However, the algorithm started experiencing much more volatility than the S&P500 and then suffered a sustained drawdown starting mid-June until it was terminated yesterday. Did you modify your algorithm during the past three months to address this anomaly? Is that even possible for contest winners?

Again, thank you Grant for your full disclosure, it is very helpful for learning. Someday I hope to be able to even make an algorithm like yours.

Grant,

Sharing the source of your algorithm was an incredibly classy move. Yet again you are propelling our community. Thank you so much for this and for all your contributions. I'm delighted that you were our first contest winner.

I see this first attempt as an investment in your education and our education. Please keep competing!

thanks,
fawce

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Thanks. Agree, classy move !!

Totally agree! It gives a really interesting insight.

Thanks Grant!

Great to see, I'm grateful.
That was OLMAR code, among 40+ here with various small tweaks, so it is popular. Not sure what makes OLMAR apparently sensitive, some experience, couldn't nail it down, whether eps, stock selection, window or what. Here's one that does well in 2008 vs 39% drawdown above. Another hit an incomplete sell and leapt into huge margin.

Some responses (more later):

Any thoughts on why it didn't perform well?

Long-only, with the possibility of concentrating in a handful of stocks. Also, for a flat or downward-trending market, I have a suspicion that it may tend to chase losers (backtesting and then pulling results into the research platform might shed some light on this).

Is it true that this algorithm would make trades everyday on every stock? $18/day * 20/(days in a month) * 6 months = $2,160 (around 2% of the portfolio). This algorithm would have lower commissions with weekly rebalancing, but would it perform better/worse?

Yes, it re-balances every day. You could run it with weekly rebalancing to see how it performs.

Hi Grant, thank you for sharing this implementation both in its original form and the winning algorithm. It has been extremely instructional to read the OLMAR papers shared by you and watch the implementation effort over time. Of course, I'll gladly pin it in the quantapolis library. Thanks again, and good hunt for alpha.

More responses (just let me know if I didn't cover something):

What criteria did you use to select the securities that you used?

I don't recall exactly, but stocks that performed well over the backtest period were picked. "Go with the trend" was the philosophy (which was sorta embedded in the contest rules at the time, to have any chance of winning).

How did you select the 20 day history window utilized?

Again, I don't recall the details. I think it was from some tinkering and not a formal optimization. You can try different values to see the effect.

How did you select/optimize the value of epsilon?

The value of epsilon was selected to minimize the drawdown, which trended up sharply after the minimum of epsilon at 1.0028. I ran a series of backtests and found the minimum (in Excel).

is it possible that the algo might be overoptimized?

Yes. But in the context of the contest (and the rules at the time), there was only upside to optimizing the backtest a bit.

Was the inception date of live trading March 2?

Yes.

If March 2 was the inception date, did you notice the qualitative change in paper trading behavior Vs. live trade behavior immediately after inception date?

No (I assume you mean between backtesting and live trading).

Did you modify your algorithm during the past three months to address this anomaly? Is that even possible for contest winners?

No, unless the rules have changed, algos cannot be modified once they are launched. I'm not sure what I would have done, though. Short SPY and pray? Not a bad strategy for the last couple days.

Whats the rationale behind choosing those particular stocks? Seems like discretionary value picks?

Brilliant,
Thanks for sharing, you have no clue how much it picks my brain to just go over the work of a pro. This is never about money but about learning how keep your gray matter alive.

Thanks again

What's the rationale behind choosing those particular stocks? Seems like discretionary value picks?

As I recall, I basically did some Google searches on the best performing stocks over the backtest period. You can take a look at what I picked. I think I tried to achieve diversification across various sectors, but it was not systematic--I just winged it! I didn't run lots of backtests over various portfolios to find the best. Also, I wanted to trade every day, so with only $100K, I figured around 20 stocks would be about the maximum the algo could sustain. I think that with fewer stocks, volatility starts to creep in (you could play around with the algo, to see what happens).

There might be the perception that I "cheated" by applying what is termed look-ahead bias to obtain good backtest results. But, of course, this sort of thing is done in the financial industry all the time (with the ever-present footnote that past results do not guarantee future results). So, in the spirit Fawce's Quantopian Manifesto (http://blog.quantopian.com/quantopian-manifesto/), the tables were turned. I got to gamble with Wall Street's money, rather than the other way around. But, as Fawce points out, it was really more of a learning (and marketing) exercise (although I'm sorta disappointed I couldn't get my picture taken with Fawce and one of those giant poster-board checks!).

Thanks for sharing, Grant! Keep competing!

Well, so far I'm #22 on the leaderboard for the February prize. The submitted algo is kinda goof-ball in my mind (not really what Q needs for their fund), but what the heck. Maybe I should just send them the code, and ask them if it is a waste of electricity powering the servers...

Thanks Grant, nice of you to share it.
I wonder why was the result removed from the winners page? I think for reference it should still be there with Stopped -10% ?

Lucas,

Hah! Those shameless Quantopian marketeers! I think they're applying survivorship bias to promote their contest. In ten years, they'll have a whole page of money-making algos. "You too could be a winner. See https://www.quantopian.com/winners--the odds are in your favor!"

Grant

Seems like all the algorithims are losing money.

I invite Quantopian to put up data related to this algorithm, should someone want to study it in detail. For example:

  1. OHLCV data for relevant securities, from their Nanex-derived live feed and history() running live (it turns out that live trading data is a merging of Nanex and backtest data, per Jonathan Kamens on https://www.quantopian.com/posts/march-prize-algorithm-restart, so there is an analysis subtlety here).
  2. Trading history from IB, including commissions and fees.
  3. Any relevant internal logs (e.g. I think latency is monitored, to make sure everything stays in synch).

I figure these data could be shared on the Quantopian research platform. I don't have time to go through the analyses, but I figure that out of 45,000 users, somebody might be motivated. Might make a nice little college project or a talk for QuantCon 2016.

When the algo was launched, I was provided a link to the algo dashboard (without the code visible, and with no control to stop the algo). I received no information on the goings-on at Interactive Brokers.

There were some "trading guards" added by Q to augment my algo, but I don't have a handy record of them (somewhere in my e-mail archive). Perhaps the Q folks could fill in this bit.

Here are screen shots of the dashboard provided to me by Q when the algo was launched:

http://imgur.com/f2TzNAI
http://imgur.com/VTJD2ID

And the log output:

2015-08-13 11:51SYSTEMINFO(IB) Updating cash in minute performance tracker from -602.303553 to -553.42
2015-08-13 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -588.756201 to -637.64
2015-08-06 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -976.018189 to -977.39
2015-07-29 15:52SYSTEMINFO(IB) Updating cash in minute performance tracker from -82.58347 to -81.92
2015-07-29 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -619.190561 to -619.85
2015-07-27 11:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -767.371851 to -724.59
2015-07-23 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -1303.84 to -1274.02
2015-07-22 14:22SYSTEMINFO(IB) Updating cash in minute performance tracker from -1304.335832 to -1303.84
2015-07-22 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -1696.834352 to -1727.15
2015-07-10 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -600.466099 to -600.21
2015-07-07 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -926.94 to -927.67
2015-07-06 12:29SYSTEMINFO(IB) Updating cash in minute performance tracker from -931.139454 to -926.94
2015-07-06 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -1064.103969 to -1068.3
2015-07-02 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -998.459783 to -995.42
2015-07-01 09:31SYSTEMINFO(IB) IB reports a holding of 46 shares of sid CC; adding to Quantopian blotter
2015-07-01 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -104.08 to -104.07
2015-06-30 14:09SYSTEMINFO(IB) Updating cash in minute performance tracker from -183.988704 to -104.08
2015-06-30 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -180.01 to -259.92
2015-06-29 13:50SYSTEMINFO(IB) Updating cash in minute performance tracker from -260.347953 to -259.92
2015-06-29 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -191.67 to -192.1
2015-06-25 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -1077.88 to -1068.38
2015-06-24 11:34SYSTEMINFO(IB) Updating cash in minute performance tracker from -1115.801077 to -1077.88
2015-06-24 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -605.749043 to -653.17
2015-06-18 12:25SYSTEMINFO(IB) Updating cash in minute performance tracker from -1140.664525 to -1113.52
2015-06-18 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from 554.900655 to 527.76
2015-06-12 11:22SYSTEMINFO(IB) Updating cash in minute performance tracker from -598.88 to -594.78
2015-06-12 10:33SYSTEMINFO(IB) Updating cash in minute performance tracker from -619.46 to -598.88
2015-06-12 10:30SYSTEMINFO(IB) Order of -1 shares of sid AMZN not transmitted to broker because the data feed is behind.
2015-06-12 10:30SYSTEMINFO(IB) Order of 43 shares of sid CSCO not transmitted to broker because the data feed is behind.
2015-06-12 10:30SYSTEMINFO(IB) Order of -13 shares of sid AMGN not transmitted to broker because the data feed is behind.
2015-06-12 10:30SYSTEMINFO(IB) Order of 18 shares of sid QCOM not transmitted to broker because the data feed is behind.
2015-06-12 10:30SYSTEMINFO(IB) Order of 1 shares of sid GOOG_L not transmitted to broker because the data feed is behind.
2015-06-12 10:30SYSTEMINFO(IB) Order of 9 shares of sid MSFT not transmitted to broker because the data feed is behind.
2015-06-12 10:30SYSTEMINFO(IB) Order of 11 shares of sid AAPL not transmitted to broker because the data feed is behind.
2015-06-12 10:30SYSTEMINFO(IB) Order of 30 shares of sid DD not transmitted to broker because the data feed is behind.
2015-06-12 10:30SYSTEMINFO(IB) Order of -18 shares of sid TRV not transmitted to broker because the data feed is behind.
2015-06-12 10:30SYSTEMINFO(IB) Order of -5 shares of sid MMM not transmitted to broker because the data feed is behind.
2015-06-12 10:30SYSTEMINFO(IB) Order of 40 shares of sid GE not transmitted to broker because the data feed is behind.
2015-06-12 10:30SYSTEMINFO(IB) Order of -12 shares of sid DIS not transmitted to broker because the data feed is behind.
2015-06-12 10:30SYSTEMINFO(IB) Order of -12 shares of sid NKE not transmitted to broker because the data feed is behind.
2015-06-12 10:30SYSTEMINFO(IB) Order of -8 shares of sid HD not transmitted to broker because the data feed is behind.
2015-06-12 10:30SYSTEMINFO(IB) Order of 18 shares of sid INTC not transmitted to broker because the data feed is behind.
2015-06-12 10:30SYSTEMINFO(IB) Order of -5 shares of sid UNH not transmitted to broker because the data feed is behind.
2015-06-12 10:30SYSTEMINFOThe algorithm is receiving out-of-date, stale data. Orders will not be placed until the data feed catches up.
2015-06-11 13:24SYSTEMINFO(IB) Updating cash in minute performance tracker from -655.419741 to -619.46
2015-06-11 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -771.329959 to -807.29
2015-06-05 13:23SYSTEMINFO(IB) Updating cash in minute performance tracker from -580.412012 to -562.24
2015-06-05 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -896.383978 to -914.56
2015-06-04 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -554.060574 to -554.71
2015-06-01 11:29SYSTEMINFO(IB) Updating cash in minute performance tracker from -479.512823 to -474.95
2015-06-01 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -305.249944 to -309.81
2015-05-19 10:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -921.20522 to -921.23
2015-05-14 12:49SYSTEMINFO(IB) Updating cash in minute performance tracker from -229.299857 to -214.22
2015-05-14 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from 303.238019 to 288.16
2015-05-06 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -303.933537 to -304.38
2015-04-22 13:22SYSTEMINFO(IB) Updating cash in minute performance tracker from -617.15 to -595.15
2015-04-22 12:27SYSTEMINFO(IB) Updating cash in minute performance tracker from -637.727526 to -617.15
2015-04-22 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -655.380935 to -697.96
2015-04-07 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -814.274109 to -814.68
2015-03-31 13:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from -217.083958 to -131.28
2015-03-31 09:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from -238.05 to -323.85
2015-03-30 10:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from -323.828198 to -323.85
2015-03-26 12:23SYSTEMDEBUG(IB) Updating cash in minute performance tracker from -312.384254 to -310.61
2015-03-26 09:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from -109.500121 to -111.27
2015-03-24 11:25SYSTEMDEBUG(IB) Updating cash in minute performance tracker from 143.073324 to 147.2
2015-03-24 09:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from 339.62313 to 335.5
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 153 shares of sid MSFT; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 40 shares of sid MMM; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 57 shares of sid AAPL; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 18 shares of sid QCOM; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 28 shares of sid AMGN; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 9 shares of sid NKE; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 578 shares of sid INTC; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 80 shares of sid DIS; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 35 shares of sid GE; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 147 shares of sid CSCO; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 42 shares of sid DD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 76 shares of sid GILD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 21 shares of sid HD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 96 shares of sid TRV; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 55 shares of sid CMCS_A; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 21 shares of sid AMZN; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 24 shares of sid UNH; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from 100000.0 to 1046.87
2015-03-31 13:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -217.083958 to -131.28
2015-03-31 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -238.05 to -323.85
2015-03-30 10:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from -323.828198 to -323.85
2015-03-26 12:23SYSTEMDEBUG(IB) Updating cash in minute performance tracker from -312.384254 to -310.61
2015-03-26 09:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from -109.500121 to -111.27
2015-03-24 11:25SYSTEMDEBUG(IB) Updating cash in minute performance tracker from 143.073324 to 147.2
2015-03-24 09:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from 339.62313 to 335.5
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 153 shares of sid MSFT; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 40 shares of sid MMM; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 57 shares of sid AAPL; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 18 shares of sid QCOM; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 28 shares of sid AMGN; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 9 shares of sid NKE; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 578 shares of sid INTC; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 80 shares of sid DIS; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 35 shares of sid GE; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 147 shares of sid CSCO; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 42 shares of sid DD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 76 shares of sid GILD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 21 shares of sid HD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 96 shares of sid TRV; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 55 shares of sid CMCS_A; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 21 shares of sid AMZN; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 24 shares of sid UNH; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from 100000.0 to 1046.87
2015-03-30 10:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -323.828198 to -323.85
2015-03-26 12:23SYSTEMDEBUG(IB) Updating cash in minute performance tracker from -312.384254 to -310.61
2015-03-26 09:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from -109.500121 to -111.27
2015-03-24 11:25SYSTEMDEBUG(IB) Updating cash in minute performance tracker from 143.073324 to 147.2
2015-03-24 09:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from 339.62313 to 335.5
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 153 shares of sid MSFT; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 40 shares of sid MMM; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 57 shares of sid AAPL; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 18 shares of sid QCOM; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 28 shares of sid AMGN; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 9 shares of sid NKE; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 578 shares of sid INTC; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 80 shares of sid DIS; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 35 shares of sid GE; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 147 shares of sid CSCO; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 42 shares of sid DD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 76 shares of sid GILD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 21 shares of sid HD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 96 shares of sid TRV; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 55 shares of sid CMCS_A; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 21 shares of sid AMZN; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 24 shares of sid UNH; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from 100000.0 to 1046.87
2015-03-26 12:23SYSTEMDEBUG(IB) Updating cash in minute performance tracker from -312.384254 to -310.61
2015-03-26 09:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from -109.500121 to -111.27
2015-03-24 11:25SYSTEMDEBUG(IB) Updating cash in minute performance tracker from 143.073324 to 147.2
2015-03-24 09:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from 339.62313 to 335.5
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 153 shares of sid MSFT; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 40 shares of sid MMM; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 57 shares of sid AAPL; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 18 shares of sid QCOM; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 28 shares of sid AMGN; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 9 shares of sid NKE; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 578 shares of sid INTC; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 80 shares of sid DIS; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 35 shares of sid GE; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 147 shares of sid CSCO; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 42 shares of sid DD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 76 shares of sid GILD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 21 shares of sid HD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 96 shares of sid TRV; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 55 shares of sid CMCS_A; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 21 shares of sid AMZN; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 24 shares of sid UNH; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from 100000.0 to 1046.87
2015-03-26 12:23SYSTEMINFO(IB) Updating cash in minute performance tracker from -312.384254 to -310.61
2015-03-26 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from -109.500121 to -111.27
2015-03-24 11:25SYSTEMDEBUG(IB) Updating cash in minute performance tracker from 143.073324 to 147.2
2015-03-24 09:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from 339.62313 to 335.5
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 153 shares of sid MSFT; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 40 shares of sid MMM; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 57 shares of sid AAPL; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 18 shares of sid QCOM; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 28 shares of sid AMGN; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 9 shares of sid NKE; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 578 shares of sid INTC; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 80 shares of sid DIS; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 35 shares of sid GE; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 147 shares of sid CSCO; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 42 shares of sid DD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 76 shares of sid GILD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 21 shares of sid HD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 96 shares of sid TRV; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 55 shares of sid CMCS_A; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 21 shares of sid AMZN; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 24 shares of sid UNH; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from 100000.0 to 1046.87
2015-03-24 11:25SYSTEMDEBUG(IB) Updating cash in minute performance tracker from 143.073324 to 147.2
2015-03-24 09:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from 339.62313 to 335.5
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 153 shares of sid MSFT; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 40 shares of sid MMM; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 57 shares of sid AAPL; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 18 shares of sid QCOM; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 28 shares of sid AMGN; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 9 shares of sid NKE; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 578 shares of sid INTC; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 80 shares of sid DIS; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 35 shares of sid GE; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 147 shares of sid CSCO; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 42 shares of sid DD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 76 shares of sid GILD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 21 shares of sid HD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 96 shares of sid TRV; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 55 shares of sid CMCS_A; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 21 shares of sid AMZN; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 24 shares of sid UNH; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from 100000.0 to 1046.87
2015-03-24 11:25SYSTEMINFO(IB) Updating cash in minute performance tracker from 143.073324 to 147.2
2015-03-24 09:31SYSTEMINFO(IB) Updating cash in minute performance tracker from 339.62313 to 335.5
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 153 shares of sid MSFT; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 40 shares of sid MMM; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 57 shares of sid AAPL; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 18 shares of sid QCOM; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 28 shares of sid AMGN; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 9 shares of sid NKE; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 578 shares of sid INTC; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 80 shares of sid DIS; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 35 shares of sid GE; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 147 shares of sid CSCO; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 42 shares of sid DD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 76 shares of sid GILD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 21 shares of sid HD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 96 shares of sid TRV; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 55 shares of sid CMCS_A; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 21 shares of sid AMZN; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 24 shares of sid UNH; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from 100000.0 to 1046.87
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 153 shares of sid MSFT; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 40 shares of sid MMM; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 57 shares of sid AAPL; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 18 shares of sid QCOM; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 28 shares of sid AMGN; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 9 shares of sid NKE; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 578 shares of sid INTC; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 80 shares of sid DIS; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 35 shares of sid GE; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 147 shares of sid CSCO; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 42 shares of sid DD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 76 shares of sid GILD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 21 shares of sid HD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 96 shares of sid TRV; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 55 shares of sid CMCS_A; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 21 shares of sid AMZN; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 24 shares of sid UNH; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from 100000.0 to 1046.87
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 153 shares of sid MSFT; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 40 shares of sid MMM; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 57 shares of sid AAPL; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 18 shares of sid QCOM; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 28 shares of sid AMGN; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 9 shares of sid NKE; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 578 shares of sid INTC; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 80 shares of sid DIS; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 35 shares of sid GE; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 147 shares of sid CSCO; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 42 shares of sid DD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 76 shares of sid GILD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 21 shares of sid HD; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 96 shares of sid TRV; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 55 shares of sid CMCS_A; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 21 shares of sid AMZN; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) IB reports a holding of 24 shares of sid UNH; adding to Quantopian blotter
2015-03-02 10:31SYSTEMDEBUG(IB) Updating cash in minute performance tracker from 100000.0 to 1046.87

A link to the MS Excel file used to estimate the best value for context.eps:

https://infotomb.com/k4p6e.xla

Thanks Grant very much. In your algo I saw several indicator such as : lam = max(0, num/denom) and dot_prod = np.dot(context.b_t, x_tilde)
Can you tell me if these indicator's inspiration from article ? I can not understand the meaning of these indicators such as multiply context.b_t and x_tilde.
Thank you very much.

Jiyuan,

Here's a link to the paper:

http://arxiv.org/ftp/arxiv/papers/1206/1206.4626.pdf

On Page 5, there is an outline of the algorithm. My code should follow it closely.

The "BAH(OLMAR)" averaging is described in the paper, as well, and the algo posted above uses it. The code accomplishes the weighted averaging by:

    sum_weighted_port = np.zeros(context.m)  
    sum_weights = 0  
    for n in range(0,len(prices[:,0])+1):  
        (weight,weighted_port) = get_weighted_port(data,context,prices,n)  
        sum_weighted_port += weighted_port  
        sum_weights += weight  
    allocation_optimum = sum_weighted_port/sum_weights  

I don't claim to have a full understanding of the underlying optimization theory, but if this is what your are after, have a look at the links in:

https://www.quantopian.com/posts/on-line-portfolio-selection-from-grant-k

Also, the OLMAR paper references:

Boyd, S. and Vandenberghe, L., Convex optimization, Cambridge University Press, New York, 2004
https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf

Grant

@ Grant Kiehne , Mr. Grant Kiehne thank you very much. Thanks for sharing too. I will read the paper carefully.

Here's an updated backtest, from the contest start until now.

Clone Algorithm
1590
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 56b3e924caf62f12bf0fee07
We have migrated this algorithm to work with a new version of the Quantopian API. The code is different than the original version, but the investment rationale of the algorithm has not changed. We've put everything you need to know here on one page.
There was a runtime error.

Here's a long-short version that I cooked up. Same securities and idea, but there is some code to select which stocks should be long and which short, and a position in SPY to keep beta near zero, if necessary.

Clone Algorithm
61
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 56b3ed40648d39117123cfe1
There was a runtime error.

Grant, after seeing some odd results in the contest, and searching for "gaming" and "contest" this is a post I found....

Are you saying that:

  1. You purposefully added securities to the algo which had the best performance over the period required for the backtest?
  2. You tweaked the code so that the contest parameters are affected in favor of winning the contest?
  3. Based on your results, Quantopian gave you 100K to trade and you lost 10% in a few months, so the algo was stopped?
  4. Quantopian does not seem to care that 1-3 above are true, and they did not vet or analyze your code before investing money in it?
  5. Quantopian has issued press releases where they say "amateur beats Wall Street" adn so forth, but did not qualify them with 1-4 above?

This is certainly not motivating. If, to put it lightly, "gamers" can get ahead in this manner, than all of this is for naught.

Hello BT,

Yes, the backtest was biased by adding securities that performed well. There was no reason to think, however, that they might not have continued to do well for the six months of real-money trading. In fact, for the one month of simulated trading, the algorithm did quite well. It was long-only, though, so when the market turned down, so did the strategy.

Regarding tweaking the code, I did systematically optimize one parameter ( context.eps); there turned out to be a sweet spot for minimizing drawdown, with a little boost in return. It was not a multi-parameter over-fitting. The other key parameters, such as the 20-day look-back and the 390 minute smoothing of prices are pretty generic. I didn't spend a lot of time fiddling with those.

The algo was stopped, per the rules, once it lost 10%. See attached for a backtest starting at the point when real money was applied.

Regarding your #4, I can't speak for Quantopian, but my sense is that they needed to start somewhere, and move up the learning curve. I suspect, too, that they were initially a bit fast-and-loose with the rules, because they needed to build crowd participation in the fund concept, which had just been introduced in the fall of the prior year. As now, they did not look at my code, and did not perform any more detailed analysis of its "exhaust" to determine its suitability for funding. However, keep in mind that this is a kind of promotional contest and not an investment. My understanding is that they are following a more rigorous process for fund inclusion.

Regarding the press releases, I did not share the algo with anyone until after it was stopped by Quantopian. As I recall, the press releases came out shortly after the algo was launched with real money. Quantopian was just promoting their contest and business; I wouldn't fault them.

It is important to note that there has been a dramatic evolution of the contest since its inception (the latest was a move from $1M to $10M in capital, per https://www.quantopian.com/posts/contest-20-rules-changes-$10m-capital-base-new-entry-required). One important difference is that winners are determined solely based on the 6-month paper trading performance, assuming I'm interpreting this correctly (https://www.quantopian.com/open/rules):

We will calculate an overall rank by averaging the Participant's rank in each criterion in paper trading.

So gaming won't help, although one can get lucky with a goof-ball backtest followed by good out-of-sample performance.

Clone Algorithm
19
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5791d73b76edc01290074ce7
There was a runtime error.

"There was no reason to think, however, that they might not have continued to do well for the six months of real-money trading."

Mean reversion, anyone?

OK, so all due respect and nothing personal, but if I were Quantopian, I would never invest with people who cheat and overfit on purpose. You seem to have deliberately chosen a set of scurities and then played with additional metrics to reduce drawdown in the backtest, i.e. you overfit on both accounts.

Wow, I thought that Quantopian was founded by some people who had enough industry expertese to detect this sort of stuff....the whole "my servers know your algo, but I don't" proposition is meaningless, because they can not vet and/or audit the cheating algos like yours, and they can not say they look at the code since they will lose some of the free intel....they are in a real pickle, and I am not sure who will give them money to manage until they reconcile the proposition. Either vet the full code, or let the thieves be the keepers.

BT -

Yeah, it'll be interesting to see how things play out. I gather that they are largely relying on a minimum of 6 months of simulated trading, and then comparing the results to a backtest. Then, they can wade in with real money. They can also ask questions of the algo owner, to get a sense for some strategy details. The algo owner can also reveal the code, and full details, if he chooses.

I could be wrong, but my bet is that in the end, Quantopian will end up hiring/contracting some of the hot shots, and will require the standard employee-company relationship, with IP belonging to the company. But this is just a guess. The stated goal is to get to $10B AUM, so at some point, there will be more than crowd-sourcing (recruiting). But it is just a hunch.

Here's a reference that may give some insight:

https://www.quantopian.com/posts/q-paper-all-that-glitters-is-not-gold-comparing-backtest-and-out-of-sample-performance-on-a-large-cohort-of-trading-algorithms

My conclusion was that most of the backtests are useless, in terms of predicting out-of-sample performance. This is consistent with your statements above, I believe.

@ BT

Regarding your comment about "looking at code" and actually understanding the nitty gritty details of users' algos and IP, this is already done to some extent. If you try to enter the contest with certain code elements, even if they are commented out, your entry will be disallowed. One could imagine taking this much further. For example, if a computer can win a Jeopardy! then one can be designed and taught to analyze algos just like a human could, but keep the secret-sauce IP to itself. The computer could just spit out an assessment. I don't know if Q is working on such a thing (sorta on the right track with the contest and tear sheets, I suppose), but would be one way to tackle the problem. Have a computer understand the strategy, but then only give a kind of thumbs up/down, overall figures-of-merit across various requirements and preferences. I don't see how they can scale to 80,000 users in any other way, if they are serious about the crowd-sourcing concept (at least my interpretation of it). Basically, they need to automate the traditional gated review process for product development, while protecting users' IP.

Hi Grant, I am new to quantopia but I have some experience in quantitative investing. Based on this thread, it sounds like Quantopia is rewarding overfitting of models through over optimization of the backtest. I don't think any algorithms should be able to benefit from forward bias like this. In the past, I have found that a good way to avoid overfit is to look at general robustness of an algorithm and have plenty of out of sample data to test the algorithm on. I have a lot of my own ideas that I think are very promising/robust but I am trying to learn as much as I can from the quantopia community to get new ideas and understand the current state of Quantitative Investing. I have been going through a lot of your posts the last couple days trying to learn but there are so many and I don't want to waste time. I would greatly appreciate if someone would direct me towards threads that they think are the most meaningful. I would also love to talk to anyone who is willing to talk to me about Quantitative Investing! A little bit of info about me: I am a 21 yo math major at Berkeley with some internship experience at hedge funds.

This is not just deliberate overfitting - it is plain and simple cheating. If I were granted (no pun intended) the funds based on such backtest and contest, I would have came clean and would have admitted to cheating, and I would have rejected the 100K allocation for the test fund.

It is utterly unbelieveable how people who have cheated are hinting in these public forums that they should be hired/contracted by Quantopian as fund managers. It's totally revolting and despicable. But, as I said in another post, there is nothing new on Wall Street, and yes, greed is bad.

Behavioral Trader

I don't think it is beneficial to the communiy to disparage one of them most selfless and generous contributors to this forum. Grant, is a stand up guy who is nothing but upfront. The rules were specifically defined and he played 100% by the rules. The algo was not selected merely on the backtest, but also some time of paper trading. All contestants had the exact same direction and same opportunity.

I would ask you to please refrain from such negative comments and take the higher road.

"The algo was not selected merely on the backtest, but also some time of paper trading. "

Rafael, read the posts above, and you shall find the truth. Better yet, contact Quantopian and you shall find out what happened. In the meantime, I would ask you to not play forum police.

Finally, being selfless and generous has nothing to do with being honest. If your hedge fund manager is also running a charity, then you better move your money elsewhere.

Hello Chris,

The direction Quantopian has been taking is to provide tools and guidance on how to put together algos that trade perhaps 50-200 stocks at a time, in a long-short fashion (possibly filtered from a much larger universe of stocks). The basic outline can be found on https://www.quantopian.com/fund and https://www.quantopian.com/open. Also, if you have a look at https://www.quantopian.com/leaderboard, you can get a sense for the kind of performance they are looking for. So, first I'd see if any of your ideas overlap with what might get funded.

If you haven't already, I'd do a deep-dive into the Quantopian pipeline API (I've just scratched the surface). There's a slew of posts by the Quantopian team that use it, so if you have the time, I'd work your way up the learning curve. It is only dealing with the mechanics of establishing a useful pool of stocks to work with, but you could have a look at https://www.quantopian.com/posts/the-tradeable500us-is-almost-here.

Gotta run...I suggest contacting Quantopian support directly. I'm sure they can give you a bunch of links to recent posts of interest.

Grant

Behavioral Trader -

For the first contest, the rules and judging were pretty lose compared to today. Long-only algos were acceptable, and Rafael is correct that both the backtest period and 1-month paper trading portion of the contest counted in the final ranking. It was not required by the rules to use a specific stock universe; it was up to each contestant. It was a reasonable choice to pick stocks that had done well recently, to give weight to the backtest and to give the best shot at a positive return over the 1 month of paper trading and 6 months of real-money trading. The algo returned 10% in the first month of paper trading, so apparently there was a trend that persisted. It was a bet that didn't play out well (because the market did not continue on its trend). Although you might disagree, I believe I participated in Quantopian's promotional contest fairly, with respect to other participants and to the guidelines laid out by Quantopian.

I would also note that I voluntarily shared my code, and fielded questions about it. I think this was in the spirit of what Quantopian is working to accomplish here. It certainly illustrates that a gamble on a market trend can end badly, and that hedging is one way to manage the risk (which became a contest requirement). I've learned something and perhaps others have, as well.

I suggest focusing your energy on assisting Mr. Atkeson, rather than continuing to criticize me and Quantopian. Do you have any suggestions for him? What do you have to offer to a young chap interested in this field? You seem experienced...anything to share?

Hi Grant,
Thank you for posting your code and your numerous contributions to this community.

In reading the OLMAR paper I noticed they used a Buy And Hold (BAH) version. In your investigation did you try BAH OLMAR vs. regular OLMAR?

Thanks for the help,
Peter

Hi Peter,

As I recall, the BAH version tends to reduce volatility of returns, compared to the regular OLMAR. You basically pick a max trailing window length (e.g. 20 days), and then compute the optimum portfolio vector and its expected return for a series of sub-windows (e.g. 3 days up to 20 days). The final portfolio is the weighted average of all of the sub-portfolios, weighted by their respective expected returns. So it is intuitive that it could lead to portfolio vector that more smoothly rotates in its multi-dimensional space, versus one that tends to jump around. At least this is the hope.

I'd also take note in the code:

prices = data.history(context.security_list, 'price',20*390,'1m')  
prices = pd.ewma(prices, span = 390).as_matrix(context.security_list)  

Rather than working with daily closing prices, for example, the algo uses smoothed minutely prices. This way, you aren't forecasting returns based on the prices from individual trades (i.e. you don't want the denominator of mean_price/prices[-1,i] to be a price from a single bar).

Hi Grant,

Thanks for sharing the algo and thanks for all your contributions. The amount of time you've put in Q over the years is definitely inspirational. I would love to dig into the OLMAR when time allows.

Regarding the winning of the first contest, I completely disagree with Behavioral Trader. In my view, overfitting has nothing to do with cheating. If you're overfitting too much, it will just eventually bite you in the a**, so you're really only cheating yourself. Back in the days, the contest ranking was a combination of the backtest score and the paper trading score. As long as the backtest score is part of the final score, it is only rational to overfit the backtest as much as possible while keeping the expected future stats to the desired levels. Nowadays, the backtest score is luckily irrelevant for the final score, probably leading to more out-of-sample robust algos. Yet, in my view, what Grant did was perfectly rational given the contest rules at the time.

Hi Frederik,

See https://www.quantopian.com/posts/on-line-portfolio-selection-from-grant-k for a list of references (Nov. 9, 2012 posting).

Grant

On this thread there were several deleted posts claiming that I had over-fit and "cheated" to win the first Quantopian contest. Regarding over-fitting, I admit to this quant crime (which apparently had enough momentum to win the 1-month out-of-sample live trading). However, it was all within the rules at the time.

deleted posts claiming that I had over-fit and "cheated" to win the first Quantopian contest. Regarding over-fitting, I admit to this quant crime

High bar. Sounds like even gathering information in trying different things is regarded as overfitting.

The Dangers of Overfitting - Quantopian Lecture 19
https://www.quantopian.com/lectures/the-dangers-of-overfitting

"just running two backtests is already overfitting"

Could you explain the simple projection part of the code in the initial algorithm you shared Grant?

Hi Mustafa,

It's called a "simplex projection" and is used in the solution to the optimization problem, but beyond that I don't have anything to add, other than some relevant links:

http://arxiv.org/ftp/arxiv/papers/1206/1206.4626.pdf
https://www.quantopian.com/posts/on-line-portfolio-selection-from-grant-k
https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf

Note that the same problem can be solved with CVXPY, which is available on Quantopian (although when I've tried it, it is slower than the code I posted above). As I understand, the Optimization API also uses CVXPY, by the way.

Thanks Grant that helped!