What is the progress on getting release mode enabled for submitted code? The debug mode has become a significant hindrance to some of the games, especially those that involve any kind of tree searching, filling, etc.
In my last stream I was hunting performance issues in my Hypersonic bot. I tried many things, and was able to come up with some awful code to speed it up. Yet it was completely useless compared to simply compiling in release mode (tested locally). All the bottlenecks were in fundamental language constructs that should have been nearly free. The debug mode had an execution time 30x longer than release mode.
Attempting to optimize in debug mode is counter-productive and results in questionable code quality. I presume CodinGame would prefer to promote clean coding skills.
Please enable release mode compilation.
Well, this is an old subject on CodinGame. Back in 2016: https://www.codingame.com/forum/t/c-and-the-o3-compilation-flag/1670
It’s time we take care about this issue. I’ll push for it.
Ok, I’ve put the topic back on the table.
Due to CG non-optimal technical architecture, adding optimization flags would greatly impact the duration of submits since we recompile your code for each battle. Granted, that sounds a bit weird, but it is how it is.
That being said, we’re willing to dive into the code and change this. I’m sure you’ll concur that this won’t be the easiest nor the safest fix of the year. It’ll take a bit of time. Considering our current priorities, I’m confident we’ll be able to tackle this during summer (vagueness intended).
Meanwhile, since there are only a few users coding in this language, we have activated release mode for Rust for submits in multiplayer games (only games with last battles).
Ah cool, that’ll be great for the show to have release mode. Thanks a lot.
Consider, with the time thing, that many languages offer intermediate optimizations modes, like
-O1 for GCC, that can sometimes compile faster than the unoptimized mode and gain most of the significant speed improvements (it removes the horrible layer of debug cruft).
that will be a huge fix if we just compile codes once
Quick question , is it just ‘on’ for all rust entries ‘submitted’ in muti-player league type games? What about when we do a test run in the ‘IDE’?
only submits in multiplayer games indeed. Plays in IDE remain in debug mode.
Will this also apply to contests?
Yes; that was the main objective
This is excellent news! Thanks for pushing for it!
Can anyone confirm they are seeing ‘production’ rust speed in multiplayer games. Based on non-debug rust being available I rewrote my MCTS Tron bot and submitted it. It is time limited in its main loop to 93ms, and, at the start of a 4 player game (longest rollouts) my figures are as follows:
Rollouts per 93ms:
Running on my desktop:(Intel® Core™ i5-3550 CPU @ 3.30GHz, 4 Core(s)))
Debug : 200-250 rollouts
Non-debug: 4,500 rollouts.
IDE mode : 230 rollouts.
Submit mode: 300 rollouts.
Just to confirm the code is all single threaded! I was hoping for the the 4k figure on a submitted bot! and then start performance tuning it, but there is no chance of using Monte Carlo Tree Search if I have to start tuning at 230 rollouts.
So is anyone seeing rust performance on a submitted bot close to what could reasonably be expected from non-debug code?
How do you know that you have 300 rollouts in submit mode? If you send the game parameters to your ide I suspect the program is run in debug mode again.
But I don’t think it is working as expected too. I am coding in the Ultimate Tic-Tac-Toe challange. When I test me code in the IDE I would suggest that I lose hard against those bots which are along my leaderboard position, because when my position in the leaderboard is calculated there would be at least 10x more computing power, since the code is compiled in release mode. But this is not the case.
Most multiplayer games let you print messages by appending them to your output (eg on CSB you print “x y thrust message”). You should also be able to view your debug on the replays of a submit if you click the “…” button.
Hopefully I’ll get a chance to test this soon. I’ve been moving for the past while and don’t have my streaming stuff setup (new equipment).
During, Code of Kutulu I used a Floyd–Warshall algorithm with some optimisation. The implementation timeout in IDE as expected, because it toke >1s on my desktop in debug mode. But it didn’t timeout (on same maze) during submission, and the printed duration are lower on submission.
So I can confirm the enabling of release mode for submission. The bad effect is that I can’t replay in IDE some games from submission because they timeout
Thanks for starting support of release mode.
I’m currently trying the rust language in the Ultimate Tic-Tac-Toe game but it seems that rust is in debug mode both in the IDE and in the last battles tab. I quickly tried this code and it always displays “Debugging enabled”. Moreover, a quick test gives me approximately the same execution times in both cases.
Did you revert the release mode in multiplayer games?
I tried again today and the release mode seems completely random. Sometimes it shows “Debugging enabled” and sometimes “Debugging disabled” and run 10x faster. Even by running the code twice in the IDE there are different behaviors. The release mode is quite buggy…
I can confirm that some matches in a submit are run with debug assertions on (debug mode, not release mode)
@TwoSteps Can you clarify what we should currently expect w.r.t Rust optimizations? For my own particular interest, what would be the expected behaviour in:
a) A typical contest with multiplayer environments, like Botters of Galaxy or MadMax or WondevWomen etc.
b) A contest like a*craft, where it’s competitive but the challenge environment itself isn’t “multiplayer”
c) The contests of a/b, but after contest completion and now in ‘game’ mode.
Thanks… and thanks for making the effort to improve the Rust experience. I realize it’s a relatively small share of total users (though positive to see that in the a*craft contest Rust implementations were at 20% of js implementations and 10% of c++!)