Clojure tests are slow

circular17 · July 13, 2021, 8:33am

In Clojure, execution time of a test is much longer than with most other languages. Thus it is quite difficult to finish a clash of code in time.

green-coder · August 31, 2021, 4:31pm

The tests could be running way faster if the code was interpreted instead of run in a new JVM instance each time.
It is possible to interpret it instantly (with no JVM starting wait time) using a library called SCI GitHub - borkdude/sci: Configurable Clojure interpreter suitable for scripting and Clojure DSLs.</ compiled using GraalVM.

TwoSteps · September 7, 2021, 2:09pm

Unfortunately, it’s not possible. That’s not how our environment works. Sorry

circular17 · September 8, 2021, 6:24am

What kind of constraints do you have?

green-coder · September 9, 2021, 3:15pm

Basically, they say that they don’t know how to do, because if they understood what I proposed they would not answer that.

Tiv0w · June 6, 2022, 4:58pm

I’m really pleased with Codingame and I find it super useful to train on problem solving. Now I’d like to improve in using functional programming languages and mostly Clojure, but right now the tests being slow limits quite a bit its use for Clash of Code.

I read that you don’t have a Clojure dev on the team, and I’m willing to help as much as I can to improve the support.

babashka has really demonstrated solid performances and stability, and is still improving quickly.

It might be possible (given your infrastructure constraints) to swap the “classic” Clojure compiler for babashka.

Possible solution

Complete assumptions

I’m assuming Codingame is running the user’s code in some form of container (Docker, LXC?). I’m also assuming that there’s a common template for running the code against the tests, namely something of the form:

interpreter user-code.txt < test1-input.txt

and then the output of the command is checked against the expected result.

And I suspect that for Clojure it’s something along the lines of

clojure user-code.clj < test1-input.txt

If all these assumptions are somewhat correct, it would then be relatively straightforward (possibly) to swap for babashka.

bb -i -o -f user-code.clj < test1-input.txt   # the binary for babashka is `bb`

Advantages

Babashka has been created as a fast starting alternative for Clojure.
No JVM slow startup time.
Most popular libraries have been “ported”, and the standard library is available. Since only the builtin libs can be used on CG, that’s not an issue.

I ran some tests to ensure that there’s a benefit to swapping, and I used my Clojure solution to the puzzle “MIME type” (measured it with hyperfine).

clojure test.clj < cloj-test

  Time (mean ± σ):     887.9 ms ±  26.0 ms

bb -i -o -f test.clj < cloj-test

  Time (mean ± σ):      13.4 ms ±   2.1 ms