Back to Community
Full Winners' Returns Data Now Available

We get a lot of requests to see more information about the winners and their performance. I'm happy to report that we made a lot more information available today. If you check out the Quantopian Open FAQ, you can now find three different CSV downloads. The first download has the daily returns of the contest entries' 2-year backtests. The second download has the daily returns of the paper trading competition period. The third download is the daily returns of the real-money trading portion.

They are in a form that you can evaluate them with pyfolio, if you choose. You can look at them separately, or stitch them together and compare the in-sample performance (backtest) to the out-of-sample performance (paper and real-money trading). That analysis might turn out to be very interesting, and I hope the community comes up with insights that we haven't. We certainly have gotten insights from community analysis of the leaderboard, and look forward to more in the future.

We've also retired the old "winners" page. That page wasn't doing its job at converting new Quantopian visitors into community members and contestants. Instead, we have revised the Quantopian Open's homepage. The revised page more prominently displays the payments that we are making to algorithm writers. We're expecting by putting the contest benefits more clearly front-and-center that we can get more visitors (and existing community members) interested in competing in the Quantopian Open.

While the contest and the fund are different, they both result in checks being written to algorithm writers. The focus on the profits being earned by algorithm writers works for both the contest and the hedge fund.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

9 responses

I am a newbie but am surprised to see the returns to be so low. If I understand correctly, per-day returned dollars would be to multiply the return (say 0.0045) with the total capital invested.

For example if I sum up Grant Kiehne's real money returns till now, they show -0.10; Does this mean his algo had a 10% loss thus far? Thanks for the clarifications

Sa

Yes, his algo (and mine) were both stopped and liquidated when they hit $90k. Mine was subsequently restarted, since they decided it had been a mistake to stop it, but has since done poorly as well.

The take-away is that writing profitable trading algorithms is not easy, and from personal experience, it's worse if you cut corners during development that lead to backtest overfitting.

Simon.

Hello Sa,

I posted my algo:

https://www.quantopian.com/posts/winning-algo-drops-below-$90k

Aside from being high beta, long-only, there was probably some over-fitting combined with bias in the fixed universe of securities. The rules and administration of the contest have evolved, so this sort of thing is less likely to succeed down the road. I knew that the algo could do poorly in a flat or down market, so it was a bet that the market would continue to go up, and that the securities picked would continue to out-perform. Since I had nothing to lose, it seemed like a good bet.

Note that you can clone my algo, and see how it has performed to-date by running a backtest. You can also change the stock universe, and play around with the parameters, as well.

I would agree with Simon's statement "that writing profitable trading algorithms is not easy" and the Quantopian team has gotten much better at filtering out corner-cutting. On the other hand, they continue to up their game in terms of the tools provided for algo development, so everything seems pointed in the right direction. It's still gonna take a lot of work and luck to find a consistently profitable strategy that will scale to $5M-$10M.

Grant

This is what you are pointing to. I don't know who produced that. Someone pls explain how the average could be beneath the other values.

Ahh Garyha, the non-intuitive nature of compounded returns and geometric averages. For example, consider two days of returns; 100% on day one (I doubled my money!) and 50% loss on day 2 (I lost half my money!). The simple average is a 25% gain, but the compounded return is 0%. Or consider the reverse with a 50% gain followed by 100% loss; even though your average returns were only -25% you have no money left.

That is what is happening to the Q-fund. Even though the individual returns are now -10.2%, -8%, -7.9%, -8.3%, 3.1% and 0.2% (in order of funding) averaging about -5.2%, the overall portfolio is -9.2% if you compound the daily returns together; mostly due to the fact that the first four winners suffered large draw downs.

A couple of words of experience. Q can claim that they started a fund with a fixed amount of money (say $1,000,000) and that each $100,000 contest winner was given an allocation of the pool. This allows Q to claim that the overall performance of the entire fund is much better. This scheme works when you are losing money, the un-allocated capital brings the overall negative performance up. Of course, in a winning fund the un-allocated capital dilutes the performance and no ethical manager reports performance sans cash. Also, actual money management is constrained by investors unwilling to pay a manager for holding cash and the periodic subscriptions and redemptions every fund faces. Q did not, AFAIK, announce a set pool of contest winnings and so the only treatment that is fair (and real-world) is that new winners capital is treated as a subscription around the time of investment.

As an aside, the most critical decision a fund manager faces is their first few investments. It is hard to dig yourself out of a hole with compounding returns. Also, greenhousing performance, where you start multiple funds and then market based on the winning funds, is a big NO-NO.

YMMV

The contest is a fun promotion to encourage our community to develop high quality algorithms; it is not a fund. We are not making any claims about the contest winners. Claims are not necessary because we are disclosing complete data.

When I look at the contest, my primary concern is whether our selection has improved over time. Are we engaging the community to produce higher quality algorithms, and are we ranking them in a way that surfaces the highest quality algos? I'd love your help scrutinizing that.

The contest is a laboratory for Quantopian's research team to iterate and improve our evaluation tools (pyfolio, also opensource!) and our selection criteria for the first fund product. Overall, the algos winning for the past few months are better constructed (hedged, clear and intentional position building/reducing, tightly managed leverage, universe screening with fundamentals), and now we can all watch to see how they perform with real money.

I've paid for this learning by taking the trading losses incurred along the way. Because we are exclusively using Quantopian's money, we can approach the problem of finding high quality algorithms scientifically, and with a long view. The fund business is plagued by short-term goals that promote short-term thinking. The contest is a series of experiments that seeks to optimize long-term algorithm production rather than short-term returns.

We will continue to share our complete data on contest winner performance, so you may conduct your own analyses, and draw your own conclusions. I'd love to see your analysis, especially if it is in a research notebook :).

thanks,
fawce

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Hi Fawce,

Hmm? I have to wonder that if all of the contest winners' algos had performed splendidly, you'd say something else. "The winners represent the early stages of our crowd-sourced hedge fund. As their 6-month real-money out-of-sample trading wraps up, we will increase their capital levels and roll their strategies into our fund." In fact, you are implying that the recent batch of winners is just that: potentially fund-worthy algos funded at a seed level. So, I think that Sally has a good point. Until you actually put money into other algos as part of the fund, the contest is your fund. And the aggregate performance has been relatively poor. You can't just say it has been a learning exercise.

Grant

FYI the CSV files of the winner returns are totally broken (downloaded 2016/Sept/01). The rows all have different random numbers of columns, many cells are an empty quoted string instead of a number, one cell in the middle of the 3rd file even contains a URL (just weird).