I had a few exchanges in chat lately regarding this topic.
I believe that In some case, private validators are quite mandatory when the point is to devise an algorithm that is able to adapt to a kind of problem rather that to a specific set of data. Like NN image recognition contest and the like.
On the other hand, quite a lot of the optimization problems available in here are very specific cases that doesn’t have much if any direct generic applicability.
To be competitive in those optimization puzzles such as for instance CvZ, you just actually cannot compete if you don’t use the final validators as optimization target.
While Mars lander’s validators are given as test cases, CvZ validators are hidden and require a daunting amount of time to generate those out of some boring trial and error process.
I understand that you are not supposed to… However you just cannot ever be competitive if you don’t do so…
For not only you are competing from behind without much clues on the validators but you also have to do it in real time while every top ranked player likely ran their optimization algorithms for hours on the actual final validators…
And that is actually why I stopped working on optimization puzzle… Because the sheer amount of time to spend just to rebuild the validators in order to be competitive is plain boring.
I’m thinking that this keeps a lot of people from even competing.
Regarding this, in my opinion, every optimization puzzle validators should be made public.
Simply by just setting them as test case like in Mars Lander.
I would also add that regarding a “stabilization” of optimization puzzle in case of making validators public that this “stabilization” has already happened…
Beside the fact that you kinda have 2 ranking mixed into just one : The ranking for people who used final validators optimization and those who didn’t…
On a note, making those old puzzle validators public could also create some incentive for people into joining the competition