I noticed that on the first turn performance is about 10x slower than on other turns.
For instance, a simple sort + loop over 47 items in a list takes about 7 ms on first turn but the same code takes < 0.7ms on next turns.
I am using C# and I’m using the System.Diagnostics.Stopwatch to measure time.
I’ve tried several different code snippets and they consistently show a significant performance drop on first turn. For instance, another code path that takes 0.5ms on first turn, on next turns it takes 0.025ms.
I haven’t had time to try replicating this in other languages but in C# I’m quite confident this is indeed a problem.
To further expand on this, I’ve even tested simple operations such as allocating an array, or copying an array. Tested several times, the first turn every single operation is 10x to 15x slower than on next turns. This means that despite getting that extra time on the first turn, I can’t exactly perform much more computation. I still really only get about the same number of operations that I can perform.
ACCEPT-DEFEAT is an easter egg
And yes you can rely on legalActions, if you think it’s wrong, please send me a replay and the frame at which you think the problem is.
Are you sure your observation of time is not affecting the way time is measured? I’ve seen in C/C++ that calling clock_gettime at the wrong moment gives me weird results that got me thinking the beginning of each turn wasted about 20 ms, which they didn’t. My workaround is to sleep a bit (a few milliseconds) at the beginning of each turn and I don’t see this delay anymore.
Maybe there’s a better way to deal with it, but my point was you should not always trust your metrics, even if the code is very simple, especially when system calls are involved (such as clock queries).
Also, you shouldn’t be bothered by a few extra milliseconds at first turn because you have more time for it:
Response time per turn ≤ 50ms
Response time for the first turn ≤ 1000ms
Since you have 20× more time for the first turn than the others, I’m sure you can live with a 10× loss, granted it really happens.
I think ACCEPT-DEFEAT is itself a bug. You can accept defeat and still win. It should be called ACCEPT-THAT-I-HAVE-NO-LEGAL-ACTIONS-AND-I-VOLUNTARILY-RESIGN-MY-REMAINING-TURNS-BUT-I-STILL-HOPE-TO-WIN-IN-THE-END. Still looking for that Easter Egg.
vdevid I appreciate your reply but there are a few problems with this:
the whole point of having one turn with 1000ms is so that you can pre-calculate things; at 10-15x slowdown that essentially means that you’re only getting 100 to 60ms of actual run time.
I’ve even tried to sleep for 900ms prior to starting the main loop to let the system “settle” and then I’ve measured performance and it still shows the same slow down
the way I’m measuring is the same on first turn and all subsequent turns and I only see the slowdown on first turn
I’ve measure several operations, including: allocating arrays, copying an array, sorting an array and they all seem to run 10x slower on first turn
The code I’m using to measure goes like this:
// initialize stopwatch before the game loop
while(true) {
// after reading inputs
stopwatch.Reset(); stopwatch.Start();
// measure
long startTicks = stopwatch.ElapsedTicks;
Array.Copy(testArray, testArrayCopy, testArray.Length); // I've tried different things here
long endTicks = stopwatch.ElapsedTicks;
Console.Error.WriteLine("Time delta = {0}", (endTicks - startTicks));
// at game loop end
stopwatch.Stop();
}
This same code runs on each turn and no matter what I’ve tried the reported time delta on first turn is approximately 10-15x larger than on all next turns.
The whole point of having 1s for the first turn is to have compiled language not to timeout for the first turn because of bootstrap/start time/JIT stuff.
In this case, why do native languages and vm-based languages get the same performance constraints?
it seems things like: time per turn, time for first turn, memory limit etc. should be configured based on each language individually and things like max stack depth should be configured the same for all languages. Otherwise what is the purpose of allowing multiple languages for these contests?
Then maybe you can explain to me . I put a comand from the legal actions and I go over opponent’s unit. How come he does not do this? Even if he is puting near me. How could he know in the same turn where will I move?
Illedan, are you sure you can move after you play ACCEPT-DEFEAT? In this replay, I play ACCEPT-DEFEAT on the first turn and I can’t play any moves any longer, even when the opponent is within pushing distance.
If there is an exception to that case, I would love to see a replay.
Edit: Here is another replay when I played ACCEPT-DEFEAT naturally (when I had no legal moves). Same result.
Yeah, I know you can’t move afterwards, because the refree throws LostException on that action. But when you have 0 moves left, you should be able to push the enemy if he passes you…
If you can push, then you don’t have zero moves left; you have have the PUSH&BUILD move left. Do you mean you have zero MOVE&BUILD moves left? Or are you saying there is a move available not in the legal_actions list?
I’ve reviewed the debate on time-allowance on the other thread and yes, it’s a thorny subject with no clear resolution.
On C# specifically it would have been nice to be able to take advantage of some sort of “pre-warming” of the VM, or perhaps running some ngen utility to create a native image but at least now I’m aware of the limitations.
I did do some further benchmarking and it seems like once a code-path gets “warm” performance improves dramatically. It just goes to show the importance of using the 1000ms on first turn for “pre-warming”.
