As usual, we are going to increase the number of battles launched for the top ranked players to have a improved measure of the trueskill score.
At the end of the contest (Sunday, March 6 at 18:00 UTC, 2pm New York time, 20:00 Paris time), we will wait for all the current matches to play. Then we will relaunch 100 matches for the top 100 players. Once all the matches from this second wave are completed, the leaderboard will be final.
So do not hesitate to push close to the end of the contest!
Such relaunch has one problem. If I know about it, I can build AI which will be tuned against different opponents immediately after upload, and then will be reconfigured (with detection by time of game) against top AIs. Prediction of opponent behavior is very important, especially for top AIs.
Actually players can try to analyse opponent moves, but in case of awaiting relaunch this is not required.
Hi all,
Thanks a lot for this contest ! That was very fun !
Just a small question : I was 77th just before the second wave of 100 matches for the top 100 players. And now, I’m 101st. How can this be possible ?
Could there be a bias if matches are only against other top 100 players where the end of the ranking lose TrueSkill points and therefore finish lower than the ones that haven’t played the second wave of matches ?
My theory is that you rank after the submit is very dependant of when you submit. If you submit at the same time as someone else, you bot will most likely encounter that other bot.
For exemple at the end of the contest, Neumann, pb4608, Jeff06 and Owl, they all submit a new IA just before the end. Me, i was in the top 10 too at this moment, but my IA was submitted since the morning so i was here for something 12 hours. The result is that Neumann had a massive amount of games against pb4608, Jeff06 and Owl. But not me. So at the end he was under me. It’s clearly wrong because its IA is better than mine.
The 100 additional games force the IA to fight with all IA around.
Thanks for your answer. I didn’t know if the 100 additional games were with all IAs around or only with IAs in the top 100, thus creating a bias. That was the aim of my question.
There may still be a bias near the 100th rank mark but it must be less important than if matches were only among top 100 players. But as reducing this one would require an astronomical amount of work, it is logical that it is still there.
Edit:
After viewing my last battles, none were against not top100.
My theory is that you rank after the submit is very dependant of when
you submit. If you submit at the same time as someone else, you bot will
most likely encounter that other bot.
You’re absolutely right. This was my first contest here, but I was kind of shocked how bad the tournament structure is. Whole ranking should be scrapped at the end and a new generated. Sure, it would be push the results for another few (or more) hours, but at least it would be fair for everyone (and a little bit more exciting at the same time).
That being said, the task was great (but why no physics code!?) and I was really sad that I learned about the contest so late.
Eh… I mostly was only able to submit dummy code for this contest… as well, placing terribly is better than not placing at all.
First contest I was part of where I was kind of living out of my backpack all super homeless and stuff. Mostly relying on Public Terminals… and I think I could have done better if I managed my time a bit better. Also–really liking how much better I’ve gotten with actually making code for this stuff.
You are likely aware that I’m mostly doing stuff on this website to try to get myself back up to the level of coding excellence that I was at prior to… a bad bunch of stuff happening. I’m actually proud of the progress I am making… even though well–I could only get Dummy Code in this contest submitted.
Oh… and as per what that “Stack Underflow” message in my bot is about… well, I never actually got to the point of giving the bots instructions… so yeah… The Dummy Code literally just causes a “no instructions” error to show up. Which… apparently is enough to be better than 25 other people. Eh… I’ve had bots that don’t even compile do better than a couple hundred–so the competition is also getting better–which is freaking awesome as well.
I also love how more and more people keep joining these darn things… so that if I don’t really do that well… I don’t place that well xD
Completely agree. I have been suggesting this for the last 2 contests or so! Submission time bias should be removed and every AI should be evaluated from scratch.
