Back to Community
What happened to the Contest scoring system?

I just noticed that current scores and historical scores in the contest were changed overnight. Did you rescale the scores or change the way they're calculated?

10 responses

Is it possible that the algorithm just hit the 63 trading day mark? Scores are calculated based on the past 63 days only and I ran into this same situation in the past.

I noticed that as of April 5th scores were changed including scores from before. I have asked for explanation from support but don't have a satisfactory answer. In addition, the notebook they referred me to calculate the scores gives a different answer than what was posted. I heard and understand the 63 day trailing window but my algo has been around lot longer than that with no scoring problems until now.

That's interesting, is the algorithm's volatility relatively stable? If there's been a spike in volatility within the 63 day window, perhaps the scoring term max(trailing_63_day_volatility, 0.02) had been using 0.02 prior to April 5.

Well I heard many explanations. However, nothing that explains how a score from a prior date can be arbitrarily changed.
Also, the delta for a given date is always the difference in the scores between that date and the previous one. Except on April 5th
I am told that score for that day went up but overall score went down because the score from 63 day prior was dropped.
I find this hard to swallow because on every other day the delta was the difference between the scores on the day and one before.

And if you run the notebook they have which calculates scores (i think it is in the last lesson on contests) it comes with a higher and I think the correct answer for the score.

Rather annoying because I am currently number 2 on the leader board and will soon lose that position if this keeps up.

Are you referring to the Cumulative Scores at the end of the notebook? Do you know if this is adjusted for the trailing 63 day window? I just ran it on one of my contest algorithms that has recently dropped out of Top 10 on the leaderboard, and the score was significantly higher as well.

yes, i am under the impression it should generate the same sore posted online.

I opened a ticket with Cal from Q Support and will let you know when they get back to me. The score from the notebook was the same that the algo had before it suddenly dropped as well.

Cal is looking at my complaint as well - will keep you posted if I hear anything.

He mentions that the score in the Notebook is the cumulative score for the time period, without adjusting for a 63 day trailing window.

got most questions answered. However, no good explanation of why the delta reported on a day is usually the difference in score between that day and the previous, but on some cases (like April 5th) it is not.