Can anyone confirm they are seeing ‘production’ rust speed in multiplayer games. Based on non-debug rust being available I rewrote my MCTS Tron bot and submitted it. It is time limited in its main loop to 93ms, and, at the start of a 4 player game (longest rollouts) my figures are as follows:
Rollouts per 93ms:
Running on my desktop:(Intel® Core™ i5-3550 CPU @ 3.30GHz, 4 Core(s)))
Debug : 200-250 rollouts
Non-debug: 4,500 rollouts.
Just to confirm the code is all single threaded! I was hoping for the the 4k figure on a submitted bot! and then start performance tuning it, but there is no chance of using Monte Carlo Tree Search if I have to start tuning at 230 rollouts.
So is anyone seeing rust performance on a submitted bot close to what could reasonably be expected from non-debug code?
How do you know that you have 300 rollouts in submit mode? If you send the game parameters to your ide I suspect the program is run in debug mode again.
But I don’t think it is working as expected too. I am coding in the Ultimate Tic-Tac-Toe challange. When I test me code in the IDE I would suggest that I lose hard against those bots which are along my leaderboard position, because when my position in the leaderboard is calculated there would be at least 10x more computing power, since the code is compiled in release mode. But this is not the case.
Most multiplayer games let you print messages by appending them to your output (eg on CSB you print “x y thrust message”). You should also be able to view your debug on the replays of a submit if you click the “…” button.
During, Code of Kutulu I used a Floyd–Warshall algorithm with some optimisation. The implementation timeout in IDE as expected, because it toke >1s on my desktop in debug mode. But it didn’t timeout (on same maze) during submission, and the printed duration are lower on submission.
So I can confirm the enabling of release mode for submission. The bad effect is that I can’t replay in IDE some games from submission because they timeout
I’m currently trying the rust language in the Ultimate Tic-Tac-Toe game but it seems that rust is in debug mode both in the IDE and in the last battles tab. I quickly tried this code and it always displays “Debugging enabled”. Moreover, a quick test gives me approximately the same execution times in both cases.
Did you revert the release mode in multiplayer games?
I tried again today and the release mode seems completely random. Sometimes it shows “Debugging enabled” and sometimes “Debugging disabled” and run 10x faster. Even by running the code twice in the IDE there are different behaviors. The release mode is quite buggy…
@TwoSteps Can you clarify what we should currently expect w.r.t Rust optimizations? For my own particular interest, what would be the expected behaviour in:
a) A typical contest with multiplayer environments, like Botters of Galaxy or MadMax or WondevWomen etc.
b) A contest like a*craft, where it’s competitive but the challenge environment itself isn’t “multiplayer”
c) The contests of a/b, but after contest completion and now in ‘game’ mode.
Thanks… and thanks for making the effort to improve the Rust experience. I realize it’s a relatively small share of total users (though positive to see that in the a*craft contest Rust implementations were at 20% of js implementations and 10% of c++!)
However, I am still a little disappointed with the performance of my code on UTTT. I have around 1100 simulations in round 2 in the IDE and 1700 in the arena. When I test the same code on my computer it gives me around 2100 simulations in debug and 37000 in release. That’s rather a huge difference! Do you specify any particular optimization level in release?
I see in the doc (https://www.mankier.com/1/rustc) that this equivalent to opt-level=2 and that what is called release mode in Cargo corresponds to opt-level=3.
Mmm, it doesn’t explain my difference. I tried with the opt-level=2 and it doesn’t change a lot. Do you also compile the libs with the -O flag? Using debug libs and opt-level=2 only for my code with rustc gives me around 3000 simulations, which is similar to the arena.
Mmm. Nope! We precompile the libs with a simple cargo build. So does this mean that we would need two versions of the libs one used when executing rust in the IDE (rustc -g) and one when submitting to the arena?
Or could we use the “release” libs all the time? But then what would be the impact from a debugging prespective?
IMHO It should be fine to have release libs all the time. I think the debug symbols are mainly useful for the code written in the IDE where most of the errors can occur. However, I don’t know all the implications that can happens and it is maybe safer to have 2 versions of the libs.
I would agree compiling the libs as release would be fine. Worst case ‘should’ be that the stack trace indicates the failure originating from the line that called to the lib, which should be a strong clue of miss-using a lib somehow. In my humble experience stack traces outside of main.rs are usually a form of cruft and the bug is clearly in main.
Yes opt level 3. But the improvement should come from compiling the libs in release mode. @WORST_DEV_EVER could you confirm that this is working as expected now?
