Does it make sense to update the score of ranked players?

Platform Evolution
#1

Does it make sense to update the Trueskill score of ranked players when someone submits his bot?

Consider this extrem case: I am in gold league and I submit a bot which will be promoted to legend. Does it make sense to update any score of the gold league (I won’t even be part of that league)?

Why do I ask this question? When someone submits his code several times and is ranked around you, your last battles against him can be about 10% or 20% of all battles used to set your original rank. So your rank becomes a rank relative to one player instead of depending on a set of players in the same area. If you are strong against this player, you go up in the rankings (for example, you can go to the upper league without fighting the boss). If you are weak against this player, you go down in the rankings (but you can get your initial ranking back if you submit your code again - which proves that your ranking shouldn’t have been changed).

When you play a game in multiplayer mode, you just redesign the rankings depending on people which are strong or weak against you. In real life (where TrueSkill is used), I guess there is no such issue because:

  • It is unusual that a player resets his skills. A human player always progresses continuously and his skill only changes slightly. While a bot restarts from an unknown skill each time it is submitted.
  • Also if a strong human player suddendly shows up and defeats you several times, it means that you are an active player and you will immediately balance your score by doing other game sessions against other players. On Codingame, the ranked bots are not active. If an active bot battles with them, they won’t balance their ranking with other players in the same area.

Here are some ideas to improve the rankins (I don’t pretend to suggest solutions - this is a complex issue and I don’t know what the best way to fix it could be) (some suggestions are not related to the issue reported above but I take this opportunity to mention other ideas which could improve the rankins in general) (numbers are not priorities, they are here to help to refer to them in comments):

  1. Don’t update the score of ranked players.
  2. Trust the initial ten matches. This suggestion is to avoid to modify the rankings between 200th-100th positions if the player has the level of the 50th player. If the estimated rank is optimistic, the player will go quickly down. If it is pessimistic, it will be the same situation as currently. Currently we almost always start from the bottom of the rankings whatever strong is the bot. Probably because you reduce the initial Trueskill score by a conservative value which is too large in my opinion.
  3. Avoid the confidence to converge too fast. During the ranking process, the score of ranked players should be updated very slightly until the rank of the submitting player becomes accurate.
  4. At the end of the ranking process, battle with players close to the current rank. I observed that you can battle a player which is ten ranks below and if you lose (which can happen randomly) you can drop several positions on your last battle (probably the confidence decreases but the ranking process stops whatever the confidence is).
  5. Continue the ranking process until reaching a given confidence.
#2

I like Halite approach a bit more, although even there you have to resubmit 2-3 times occasionally after you left a broken bot on the ladder for too long.

But what’s great about it is that you start your games from around your current rank and don’t go from the very beginning of the ladder. Which makes an awful lot more sense.

This would have fixed a lot of issues in the last contest.

#3

Hello,
Without really taking position about this particular point, that sounds reasonable but I read very quickly, I think that globally speaking, some more clarity would be welcome about scores.
I needed some time to realize it was not necessarily contradictory to be 20000th in the 4th category while somebody is 50th in the 10th one.
And the scores are not always calculated on the same numbers, which can be confusing. Well, that does not prevent from improving one’s skills, but while you take care of displaying scores, it would be coherent they are clear.

#4

I feel, specially in contests, that bots are improving really fast, meaning if I’m let’s say 50th when I go to sleep. The next morning, my bot doesn’t deserve that rank anymore. I’m not sure how much does losing or winning against a ranking bot works, but it should be based on where the bots stabilize to determine how much you win or lose points.

But personally I dont see any problem with rank being affected by matches against ranking bots. It’s not rare that I submit a better bot after a while that in the end ranks lower than my old bot, not because my bot is worse, but because my old bot didnt deserve that rank anymore.