Olive Coyote: thanks a lot, I am working hard to have good out-of-sample procedures, and I'm happy it is working out for me.
Ryan: Thank you for all that helpful information, and for pointing us to that other thread.
Regarding my jump up in the rankings, like Dan pointed out in the other thread, there was a bug in the leaderboard that caused the rankings to be incorrect. I was in first place before the bug was introduced. My top performing entry then suddenly disappreared from the leaderboard for a few weeks. Following the first attempt at a bug-fix from Q staff, there was a strange situation where the metrics for some algorithms (including mine) were much worse on the leaderboard than when viewed on the live trading dashboard and stats page. For example, beta was reported around 0.2 for me on the leaderboard, but 0.01 on the stats page.
I noticed that backtests of copies of my algorithm gave results consistent with the leaderboard. Furthermore, I wrote my own code to compute the metrics, and found results consistent with both the backtests and the live trading stats/dashboard. At this point I was hopeful that the numbers on the leaderboard were incorrect, which indeed turned out to be the case.
I was in first place before the bug was introduced into the leaderboard, and returned to first place after it was resolved. I want to be clear that the bug was in the leaderboard, not in paper trading. The contest rules weren't changed, just the leaderboard was incorrect for a few weeks.
I can speak for myself that I do not post frequently on the forum, because I do not want my name associated with some of the discussions. There have been a lot of strange opinions about what Q staff could, should, or "have" to do. I would like to remind some people that Quantopian is a private company and can do as they wish. We are lucky that Q provides this amazing platform where we have all learned a lot about quantitative finance. Their main responsibilities are to uphold the mandate set by their board of directors, not to run a contest. I am worried that due to unprofessional behaviour by some users, they will decide that the contest is not worth their while and take it away from us.
I personally think of the Q staff as potential business partners, and try to treat them with the same respect I treat my colleagues and collaborators. A few weeks ago, I thought I was going to lose the competition due to a bug in the leaderboard. I assure you I didn't complain. In fact, when I found out about the issue, I told the staff that I would be disappointed, but would obviously accept whatever outcome they determined to be fair.
Besides making oneself happier by refusing to get worked up about these things, there is also a lot to be gained from professionalism, and a lot to lose from the opposite. I can say, from my own business decisions, that if a deal is borderline, I might choose to say "no thanks" if the person I would have to work with is unprofessional.
Finally, there is a lot to be said for confidence in these situations. I assure you, if I don't win contest 27, I will win some other contest -- it is only a matter of time at this point. For example, paper trading was restarted for contest 32, and yet I am back up to third place. Similarly Olive Coyote is back up to #2, and Ryan is also top ten. Victoria: your algorithm looks good. If you don't win this time, you will have a good chance in contest 28. Either way, I wish you the best of luck in future competitions, or in this one if it goes that way.
I am on vacation at the moment and am writing this ridiculously long post using my phone, because I felt the need to defend myself and Q. I will be hiking out into the wilderness soon, where there will be thankfully no cell reception, and certainly no discussions of leaderboards. Thanks again to the various people making constructive comments and positively contributing to the community and competition.