For the CodeBusters contest we decided to improve the ranking stability. Indeed for the Smash the Code contest we all witnessed some pretty disturbing changes in the leaderboard close to the end of the ranking process and we needed to do something about it.
So the following will happen for CodeBusters:
During the contest, nothing changes: CodinGame uses a basic TrueSkill like system to rank players. The system is fast and good enough and provides a decent ranking. Our belief is that when players strengths are very close, TrueSkill will rank with a -5/+5 rank error margin. The only improvement we made is that the top five players no longer play systematically against a stronger player, these players can also play matches against players below them as it is the case for the other players.
Once the contest is closed, we wait until all the in-progress matches are done.
After that, we launch an additional batch of 200 matches for the players of the Legend league (limited to the first 150 players).
For these matches, we will rely on a modified version of TrueSkill : the score will be computed as the average of TrueSkill scores over the 200 matches rather than relying on the latest score. The rationale is that TrueSkill has a short memory and the last match result can influence way too much the ranking (e.g. if one player is lucky, he/she can move up by a few positions for a short period of time).
Additionally, when playing the 200 matches, each time a match is launched involving a player from the top 15, 5 sub-matches are launched between this player and the other player and only the global result of the matches is fed to TrueSkill. Again the goal is to provide TrueSkill with reliable results to lower the “luck” factor and short memory issues. It means that players in the top 15 will play at least 1000 matches of their own.
We have made some in-depth testing of this method and we are confident that it will yield better results than the one we used in the past.
Some of these changes were suggested by members of the CodinGame community and we thank them for their involvement.