We spent the last two days carefully reviewing the finish of the October Prize, using the processes and tools I described in last month's post.
We chose to disqualify one entry because it wasn't financially prudent to invest in. The second entry looks good, and was named the winner.
Algo 1: Inconsistent and Position Concentration
There were two aspects of this algorithm that concerned us. The first was that the returns weren't good when we looked at a long backtest. The algorithm performs well in the most recent two years, but it performs poorly in the years leading up. That's often an indicator of overfitting. We're concerned that this algorithm can't maintain it's recent success and will revert to the long-term trend. This algorithm was entered into the contest in late August of this year. You can see that it started to perform well about two years before that, the duration of the contest backtest. But before the summer of 2013, it lost money too consistently.
The second concern is that the algorithm holds only two securities. That's a lot of concentrated risk. If one of them has a bad event, it's a large risk. We would prefer to invest in algorithms that have more diversified holdings that reduce the exposure to a single stock.
This algo trades a book of several hundred stocks, presumably chosen using fundamental data. The returns were more consistent, and more positive.
The fact that it's using these baskets of long/short stocks was a good indicator of a solid investing strategy. We'd rather there was more out-of-sample testing, but the nature of the 1-month contest prevents that.
This, of course, is the last one-month duration contest. We're looking forward to having more out-of-sample data in the leaderboard, and we expect it will result in some great algorithms on top.