Can someone explain me precisely how the ranking system works ? I’ve read that it’s based on TrueSkill, but that’s all I’ve found on the website. And there’s something I don’t understand.
I’ve submitted some code 3 days ago, and it keeps going up on the leaderboard. It moves quite slowly now, and I think it will reach a stable position very soon, but still, for now, it moves. That seems to be quite normal with any ELO-like system.
My problem is the following, if I try to improve the codebase until the last minute, and I submit something 20 minutes before final deadline, it may not have the time to reach its stable position. I fear that with a better code, I’d end at a lower rank. I’m not quite sure about how the system workds, could this happen ?
I used to play in the Google AI Challenge (another AI challenge, quite similar to the codingame platform), and they had a TrueSkill based algorithm too. After final submission date, they used to block new submissions, reset rankings and let all AI fight for 3 days, until everything was stable. Is there a plan like this on Codingame ?
Actually this would be really nice to know as the contest is approaching a close. Currently it’s extremely hard to even approach the top 10, even if I can beat many of them. A reset would ensure that everybody gets a fair chance at the tops without a select few having so stabilized scores that even multiple losses cannot nudge them.
I would also vote for the Google approach. Lock the submits, reset everything and run for a long while until the scores stabilize. It would also be a lot of fun to watch how the final scoreboard forms
Could someone from CodinGame staff please answer this thread? If there’s no reset, then we would need to submit at least a few hours to half a day prior to deadline to have stable results. On the other hand, if there is a reset, then we can take advantage of every last drop of time we have. Since it can be quite a deal breaker, it would be good to know in advance
I have the exact same problem so +1.
After submission and the 100 first matches, my AI reaches top 60. If others players send their AI and give me more matches, my AI needs at leat 4 hours (~300 games) to reach its rank (which was top 8-26 last 48 hours).
If there’s no reset, I think the number of games done after 1 submit should be increase to at least 250. Even if that means to set a limit of 1 submit / hour / coder.
Ahem… This is getting more urgent. Especially considering the way this competition unfolds. I would greatly appreciate an explanation. It should be a part of the official rules anyway.
So each time you enter the arena, your AI will fight 10 games in parallel against opponents taken from the whole leaderboard (from first to last). Based on the results of these 10 games you will be positioned into the arena at a given position (based on your TrueSkill mean for the 10 matches minus 3 times the deviation which is very high at this point => you will enter the Arena at a lower place than what you probably deserve).
Then 100 matches will be played sequentially against opponents not far from your rank (taken from the range [rank-10, rank+5]). After each of the 100 matches, your TrueSkill mean and deviation is recomputed and a new rank is computed before the new match is launched: that’s how you bubble up in the leaderboard.
After the 10+100 matches are played, you will play additional matches only against opponents re-submitting into the arena. Your mean, deviation and rank will be recomputed accordingly after each match.
At this point, for the Platinum Rift challenge we do not have the time to move the 100 limit up. We know we should as it is a bit small, but we will really not be able to make it.
But we heard you loud and clear, so at the end of the challenge for the top 100 players only, 200 additional matches will be played per player. So do not hesitate to push before the end of the challenge (it means at the very least 300 matches per player + all the matches you will be play against opponents). Read the full news in the updated Platinum Rift statement.
Last thing: when fighting with someone within the top 15, each match is replayed 5 times (with a new seed) before being fed to TrueSkill, it allows us to make sure (or be more certain) that if you beat someone in the top 15 or if someone from the top 15 beats you then it was not by chance. It means that players in the top 15 are likely to do more than > 500 matches per submissons. Goal being to have a more stable leaderboard at the top.
