Bug in calculating puzzle score, got 100% while undeserved


#1

Hi,
I was solving the puzzle of the week:


in the IDE I got 7/8 test passed, so I decided to have a pause,
I submitted it and... WOW, I got a 100% ??
So my code passed all validators by chance, and, even not passing one of the test inside the IDE, I got 100%??

Apparently the submitted code is only checked vs the validators, and not vs the "official" tests? why? is just CG confident in honesty of developers ? :wink:


#2

That's not a bug of the platform, maybe those puzzle validators aren't enough to fully validate the code.
IDE tests aren't checked because is dead simple to hardcode them. You know the inputs and the expected output.


#3

Yes, agreed that the root cause of the problem is essentially that the validators in this puzzle are not enough exhaustive.

Still seems to me a so reasonable and low-cost feature for the platform to check also with IDE test in submission, just to prevent this weird occurence.
That I must think should be not so rare, since that occurred to me, without having done anything on purpose.


#4

If the problem is to not waste compute time on rerun tests, then I will propose a solution for, at the contrary, save computation time:

when submit, a validator is considered KO (without even run it) if the corresponding test in IDE was not already passed.

What about this proposal?