Recurrent question methinks 
)
BTW, in CvZ there are 3356 players with positive score
(had been there too ;))
1 Like
Yup I’ve seen your post earlier but it doesn’t solve everything. Maybe @thibaudCG could help us with that? I’m currently building a tool for stats-inclined person like myself, and I’d love to have exact formulas
1 Like
@CvxFous: well spotted! There was an issue with Code vs Zombies. Scores should be good tomorrow.
1 Like
You’re welcome
Hi,
I lost arround 1K points of challenges few days ago. Is it related to the issue with CvZ points?
Sorry to bring that old post to life, but I think that Coding Rank should reflect in some way the relative skill of a programmer, in a global leaderboard.
But after many observations, IMHO Contest and Multiplayer formulas doesn’t reflect at all the average position or skills of a coder.
The Problem
For some simplistic examples I’ll use the playercount from leaderboards, ignoring the Score >0 rule at will because it’s not important for the current issue.
Current Problem in Multiplayer:
Player A:
Coders Strike Back: Rank 2300th/20838
Score calculated: 1953 points
Player B:
Fantastic Bits: Rank 1st/923
HyperSonic: Rank 1st/997
Score calculated: 1920 points
Wait, whaaat? Are you telling me that a player on Silver League CSB is better than the top player on two other multiplayer games? This is completely insane.
And the same observations can be done on Challenges. In fact I filter the general leaderboard by Contest Points (https://www.codingame.com/leaderboards/global?column=codingpoints&value=contests)
and all top 10 players there share the same common challenge, well placed on Code vs Zombies (and maybe The Accountant).
In fact a fast calculation between Code vs Zombies against Codebusters (most and least played challenges recently) reflects that the 219th player on CvZ has more points that the 1st player on CodeBuster.
For me the main problem is assuming that the more players, the harder and more rewarding a game must be.
I think that player count is a matter of sample size on a statistical sample. Having more players in theory won’t move you a lot on your relative position. If I’m in the top 20% for 1000 players, I’ll be in the same 20% with a population of 20000 players (maybe with some confidence error, or whatever is called).
So the formula must add a % bonus for player count, but not have a base point based on player count.
Errors that bring using Player Count as base for points:
- On challenges it can make impossible for new players to ever reach veterans. If for any reason there was a challenge with 5000 players, and from now on all challenges are 2000 players, new players never ever can reach the same scoring than veterans (including a hat trick from a new player).
- The same can happen on the opposite, if suddenly CG have a reddit hug and a challenge is player by 10000 players, probably older scores will be worthless against a pool of 20k points for a single challenge.
- Multiplayer points are totally biased, as stated before. Some very hard games give little points whereas other multis are plenty of points. Getting into top 30 on any multiplayer is equally hard.
- A future problem will be the drift in the Multiplayer vs Challenges scoring. With more and more multiplayers coming in, and challenges limited to 3 best scores, the scoring slowly drift towards Multiplayer. There must be some way to make Challenge Total Scoring as a % of multiplayer total Scoring. This way the global score can be created as 15% Challenges+70% multis+Code Golf +Optimization, or something like that
Multiplayer Formula proposal
(Base_Multi * f(N) )^( (N-C+1) /N)
Where Base_Multi will be a fixed value, the base score for each Multiplayer, and f(N) is a function that gives a bonus based on Player Count (values [100%-115%]. The rest of the formula remains the same. So any top1 player from any multi would score something between Base_Multi and Base_Multi * 1.15
I don’t specify how f(N) should be calculated, but it must take into account possible variances on population and still be fair. Can be lineal, non-lineal, etc…
With the modified formulas (asuming Base_Multi as 3000 for example), the prior calculations were something like:
Player A:
Coders Strike Back: Rank 2300th/20838 I assume is the most played multi, bonus = 15%
Current Score: 1953 points --> New Calc: (3000*1.15)^( (20838-2300+1)/20838 )
New Score : 1404 points
Player B:
Fantastic Bits: Rank 1st/923
HyperSonic: Rank 1st/997 I assume both games are the least played multis, bonus = 0%
Current Score: 1920 points --> New Calc: (30001.00)^( (923-1+1)/923 ) + (30001.00)^( (997-1+1)/997 )
New Score : 6000 points
Now this is more logical and fair for FB and HS players.
Contest Formula proposal
(Base_Challenge * f(N) )^( (N-C+1) /N) * 2
The Base_Challenge Points should be based on the number of multiplayer games, this way it won’t drift on the future.
As best 3 challenges scores, Base_Challenge = Base_Multi * “number_of_multis” * 5%
This will assign a 15% value of the multiplayer to challenges (3 challenges * 5%). Obviously % are set by CG, depending on how important challenges are to the global score.
If you’ll have different Contest categories, then make some changes to the idea, and have 3 different Base_Challenges.
I really can’t predict how these formulas changes the current ranking, but I think it’s more fair to everybody.
What do you think?
17 Likes
Hi all, I don’t play to codingame sine one year and my profil slice from 3006 points to 582 points. All my points are coming from puzzle. What the explanation please? Lost all points from hard work isn’t fun at all. It seems like time spent.
Hello @Marchete ,
this seems like a decent proposal and I agree the current calculations have their flaws. I’ll see with the team what we can do.
@negstek1 losing your points doesn’t mean losing your skills FYI, what happened with your points: https://www.codingame.com/blog/careful-rework-in-progress/
I agree with everything Marchete said.
Very well argued. Thank you for this.
- danBhentschel
I have trouble with contest points calculation… If I follow the rules… these are my 3 best :
the accountant 974e / 6214 = 1584 CP
code4life 172e / 2366 = 1349CP
there is no spoon 70e / 1628 = 1190CP
total = 4122 x 2 = 8244CP… but in my info I see 2598CP
where is my mistake pls?
Still an issue almost 3 months later. In fact it’s gotten worse, since there’s now 25k players in CSB…
N number of players with a score > 0, C rank of the player.
Ok thx… but you should give this N on the historique information. Because actually informations given are useless to compare challenges to know how each one feed me in CP
1 Like
Hi @marchete and others
I’m sorry I’ve not come back on this old topic before.
-
What would you propose for f(N)? If we make it linear, more or less, it will make CSB 0,15% more important than others.
-
Why did you take 3000 for Base_Multi? Was it just random or you have something in mind for it?
-
Same question for contests making for 15% of multis available. Do you think it’s an OK ratio?
-
Shouldn’t we do the same for optimization puzzles? (I’m keeping CoC outside of the pb)
-
So each time we add a contest, all your contest points could change (because of f(N)), even if you don’t participate in it. Don’t you feel it could be confusing?
Hey, I’m playing a bit the devil’s advocate here. However, you must understand that, while it’s just a formula to change, it implies quite a lot of change for a lot of codingamers. And you know how many people are resistant to change. Even if the change is for good reasons, a lot of codingamers will complain. So I want to ensure we’ve got a really good formula.
Anyone, feel free to comment.
1 Like
I do not agree with @marchete on this point. f(N) should not impact the maximum rewarded points for a puzzle (contest, multiplayer puzzle or optimization puzzle).
The formula for every puzzles should be something like this:
BASE^((N - rank - 1) / N)
For the BASE, it could be something like this:
- Contest : 5000 (3 puzzles maximum; i don’t think we want to change it)
- Multiplayer puzzle : 2000 (or 2500)
- Optimization puzzle : 1000
- Golf puzzle : 200 (5 languages only, i don’t think we want to change it)
2 Likes
I agree with Marchete’s statement and the figures he used are a very good example.
Therefore BASE should be a constant since being 1st on a contest is equally hard, regardless of the number of contestants.
BUT there is one contest that had a very low number of contestants, in which being high ranked was a little bit easier : SF2442 (211 people)
This one needs to be treated differently, either by removing it from the CG Points computation, or giving it less points.
Perhaps stay with the actual formula, but lowering a lot the MIN cap to 500 instead of 5000 ?
(BASE*min(N/500,1))^((N-C+1)/N)
1 Like
In fact, after reviewing Magus and Neumann posts, I think the f(N) bonus should be removed, and the formula may be like Neumann proposes: (BASE*min(N/500,1))^((N-C+1)/N).
According to sample size calculators, for a margin of error of 5%, confidence 95% and population 780k, the minimum recommended sample size is 384 (see Sample Size Calculator). 500 samples will reduce the margin of error to 4.38%. It’s a round number (better than 384) that gives enough confidence, and it doesn’t need to be changed on the future (385 is the limit when population tends to infinite).
This removes the problem of really low played games, and simplify the calculation (as it will not be based on external variables).
The f(N) bonus isn’t useful and make the formula overly complicated.
Mean points of least and most played Multi: (1000+5000)/2=3000 at this time
No, I think it should be something between 24% and 33%, but I’m biased… Contests show some skills of players (deadline and stress management, coding speed, short term objetives…), and Multis reflect other skills (long term objectives, code optimization, perseverance, constancy). With enough luck, Contests are 3 weeks of work, while Multis are at least 16 weeks of work (and increasing).
Percentage ratio should be defined by Codingame Staff, according to the relevance they want to put to Challenges (or just a poll).
Ratio related to multis? Probably yes.There aren’t that many Optimization puzzles to be that relevant.
If you make Base_Challenge related to Base_Multi it will change too, a lot. It could be confusing, yes.
There is the option to make Base_Challenge constant, and assume there will be a drift in the future, maybe it will be noticeable in 4+ years.
Understood, and I prefer that the formula is agreed by the most people possible. The problem with the previous one is that there wasn’t enough unit tests for all extreme cases, where its flaws are noticeable. The new one must be tested with extreme cases (low playercount, extreme high player count, a lot of new multis coming in, etc etc) and check that is stable enough.
About points, to avoid excessive complains there is the option to create “inflation” on the CG Points. Like in economics, the best way to reduce money value without people getting furious is via inflation. If the most played multi has Base_Multi == 5000 then set it to 5000 for all multis. Then, if Codingame Staff decides that Challenges must be 40% of the Multi Score, then Base_Challenge = 15 Multis*5000 Base_Multi * 40% /3 challenges = 10000 points. This is similar to Magus proposal, just 2x the points. Maybe these points increase are even more confusing, I can’t know, but people react better to point increases.
P.S: Why we need to manually calculate CG Points? It’s a nonsense, prone to errors.
The profile page should give a CG Point breakdown. I have a page with complete XP details, but I don’t have CG Point details, There are some graphics but without any way to see how it’s calculated.
3 Likes
I don’t agree with Magus and Neumann’s view that being 1st is equally hard regardless of N. This is mathematically false. Counter example: if there is 1 person he will always be first. I’m pretty sure if you had a contest with a million people winning would be a little harder than if you had 1000.
I’m not sure about the best solution to take into account N though.
My best idea comes from Marchete when he said “If I’m in the top 20% for 1000 players, I’ll be in the same 20% with a population of 20000 players”. One way would be to say 1/500= top 0.2% and 1/5000=0.02% and therefore worth slightly more points simply by changing (N-C+1)/N to (N-C)/N.
Look at it this way: with the proposed system of base^(N-C+1)/N ranking 1st out of N players is independent of N but ranking 2nd isn’t.
2 Likes