Validation cases and optimisation puzzles

Hi there !

We have been discussing lately with some CG users about optimisation puzzles. At the moment there are only 4 puzzles in the category : Code of the Ring, Mars Lander, Code vs Zombies and the CG sponsored puzzle.

There is a very strong bias on the methods used depending on whether or not the validation tests are known to the players. It is (almost) no secret now that the top players on CotR and Mars Lander know the validation cases and have hard coded solutions. I won’t talk about the CG contest since I don’t know it that well. As for CvZ, all top players use a variant of el-didou’s solution (posted in the CvZ strategy topic) ie a Monte-Carlo approach.

Now it is problematic, if we want to implement state of the art optimisation methods, to be limited to 100ms. In cases like CvZ where some very neat approach could be used, the best solutions are random and require multiple submits to hit the highest scores (without talking about the submit spamming required to top the best players).

In Mars Lander and CotR however, knowing the validation cases allow players to test very neat optimisation algorithms. For instance, Mars Lander has been solved by some players using genetic algorithms, while others might have used simulated annealing + neural networks.

If reverse engineering the validation tests might be considered by some as an exploit, because “hardcoding is ugly” (statement that I totally agree with), on the other hand, having random solutions and having a submit script that spams until becoming first, is not ideal either. To be completely honest, I think the value of having validation tests for the optimisation problems is a big bonus, since it leads not only to real solvers but also very precise ones. The optimisation then really becomes optimisation and not “quick and dirty solution since you only have 100ms”.

Soooo, here is the question : Is it conceivable for CG to release officially the validation cases for CotR, CvZ and Mars Lander so that : 1- Every player plays with the same chances; 2- Avoid spamming and “dumb” methods to keep in a 100ms timeframe ?

Thanks a lot in advance :slight_smile:


Note: even if the validators are not rendered public, CvZ’s will be known and shared before next contest. I already have Ash & 12th first humans initial coordinates for all test cases (by dichotomy).

For Mars Lander at least, it would require making a new set of validators for the non-optimization puzzle.

1 Like

Dunno if that would be such a big problem :slight_smile:

1 Like

It’s not a problem, just a reminder for the staff in case your suggestion is accepted. :wink:

Actually I think that for optimization puzzles, the test cases could be replaced with the current validators, this way everybody would really be on the same level, even if they don’t have any way of testing their code locally.

There’s no need for secret validators for optimization puzzles.

And to keep the Force balanced, maybe the golfing puzzles could get some randomized validators? :stuck_out_tongue:


I would be all for random validators for golf puzzles.
And I totally agree for the test cases being replaced by the validators. That would be the easiest way of doing it ! If we have the same thing in the IDE and in the validators then we can concentrate on optimisation :slight_smile:

1 Like


Replacing IDE tests by validators is the best way to do it :slight_smile:

29 zombies coords to go :stuck_out_tongue:

PS : you have a zombie out of the ground : case 19 second one (499,9071)