Cg-brutaltester, because you love to be brutal


#1

Some of you remember my cg-arena tool to run multiple games in local. Forget it. cg-arena is dead.

Here come cg-brutaltester. Why creating cg-brutaltester instead of just improving cg-arena ? Because cg-arena has a big problem, the referee are coded in java directly in cg-arena. But now CodinGame gives the referee code for the contest. And there's no guarantee that the referees will always be coded in Java.

cg-brutaltester take the referee as an external process, exactly like the players codes. This say, cg-brutaltester can accept any runnable referee and players codes. In any langage. If you can run a code with a simple command line, cg-brutaltester can run it too.

It is also easier to create your own referee for cg-brutaltester. See the readme file for more informations.

Keep in mind that cg-brutaltester is slower than Agade Ghost in the Cell arena. Java will always be slower than C++. But cg-brutaltester is already prepared for the future ! If CodinGame give the next contest referee the same way they did for Ghost in the Cell, it will take something like 30 minutes to code a compatible referee for cg-brutaltester.


#2

Nice. A question.
The CG referee code you use for Ghost in the Cell is similar to the CG.
What are the steps for converting a CG referee on a "brutal" referee?

Just in case they release again the referee code.

Thanks


#3

The main steps are to code the main method.

https://github.com/dreignier/cg-referee-ghost-in-the-cell/blob/master/src/Referee.java In this file, you can look between lines 855 and 940.

I can't say if the next referee will have to be implemented the same way (we are not even sure that the next referee will be in Java).


#4

Not sure if I'm using it wrong or misunderstanding end result stats, but this doesn't look right to me:

$ java -jar cg-brutaltester.jar ... -t 1 -n 1
08:01:24,979 INFO  [com.magusgeek.brutaltester.GameThread] End of game 1: 0 1
08:01:24,980 INFO  [com.magusgeek.brutaltester.Main] *** End of games ***
08:01:24,980 INFO  [com.magusgeek.brutaltester.Main] *** Statistics for player 1:
08:01:24,980 INFO  [com.magusgeek.brutaltester.PlayerStats]  Position 1 : 1 (100%)
08:01:24,980 INFO  [com.magusgeek.brutaltester.PlayerStats]  Position 2 : 0 (0%)
08:01:24,980 INFO  [com.magusgeek.brutaltester.PlayerStats]  Position 3 : 0 (0%)
08:01:24,980 INFO  [com.magusgeek.brutaltester.PlayerStats]  Position 4 : 0 (0%)
08:01:24,980 INFO  [com.magusgeek.brutaltester.Main] *** Statistics for player 2:
08:01:24,980 INFO  [com.magusgeek.brutaltester.PlayerStats]  Position 1 : 0 (0%)
08:01:24,980 INFO  [com.magusgeek.brutaltester.PlayerStats]  Position 2 : 1 (100%)
08:01:24,980 INFO  [com.magusgeek.brutaltester.PlayerStats]  Position 3 : 0 (0%)
08:01:24,980 INFO  [com.magusgeek.brutaltester.PlayerStats]  Position 4 : 0 (0%)

Does each "game" here mean actually running every position combination? If no, then how can both players have 100% in some position? If yes, then how can 1-1 in positions lead to 0-1 end of game 1 result?


#5
End of game 1: 0 1

that means, player 1 wins, player 2 is second (0-based index). A "01" (without space) would be a tie, making both players 1st.

Statistics for player 1:  Position 1 : 1 (100%)

player 1 ranks first in 100% of his matches. Position isn't a good word for that.

As far as I understand, your player1 code will always run at player1 position.


#6

eulerscheZahl is right. In your results, you have only 1 game. Player 1 won the game (finish at position 1) and player 2 losed (finish at position 2).

At the moment brutaltester doesn't handle rotations. I may add it later it's not that difficult. But not during the contest :stuck_out_tongue:


#7

Thanks, makes sense now that you've described it, but maybe add a "NB: position != rotation" remark somewhere in readme :slight_smile:

Oh and a very cool tool in general, thanks!


#8

it works perfectly using 2 exe files
however when I try with a jar with a command like this one:

-r "java -jar cg-cotc.jar" -p1 "./solution1reshots.exe" -p2 "java Player.jar" -t 1 -n 10 -l "./logs/"

it fails with
NoSuchElementException: No line found
the log file states:
"Player 1 error: Errore: impossibile trovare o caricare la classe principale Player.jar"
(impossible to load main class Player.jar)

I'm actually using Intellij Idea community edition and i was unable to get a jar for the brutaltester, i'm receiving a
"java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory"
that i can't solve even inserting the apache jar in the project.
however, when i press the "play" button it actually works and i think its very good, i can set my pc to make 2 ais fight all night and see result on 10000 games.
(p.s. it seldom can crash one thread. when it happens, the program waits forever and won't give the final stats....)


#9

cg-brutaltester is build with maven. I don't know how to use maven in Intellij. But you don't have to build it yourself except if you want to modify it.

I published the runnable jar in the release page on github: https://github.com/dreignier/cg-brutaltester/releases


#10

thank you very much.
i also discovered my dumb error on the commandline

"java Player.jar" gotta be written as
"java -jar Player.jar"


#11

I've got an error:
java -jar cg-brutaltester.jar -r "java -jar cg-mm.jar" -p1 "MeanMax.exe" -p2 "MeanM
ax.exe" -p3 "./MeanMax.exe" -l "./logs/" -v
18:06:32,422 INFO [com.magusgeek.brutaltester.Main] Verbose mode activated
18:06:32,423 INFO [com.magusgeek.brutaltester.Main] Referee command line: java -jar cg-mm.jar
18:06:32,424 INFO [com.magusgeek.brutaltester.Main] Player 1 command line: MeanMax.exe
18:06:32,426 INFO [com.magusgeek.brutaltester.Main] Player 2 command line: MeanMax.exe
18:06:32,426 INFO [com.magusgeek.brutaltester.Main] Player 3 command line: ./MeanMax.exe
18:06:32,426 INFO [com.magusgeek.brutaltester.Main] Number of games to play: 1
18:06:32,427 INFO [com.magusgeek.brutaltester.Main] Number of threads to spawn: 1
18:06:32,524 ERROR [com.magusgeek.brutaltester.GameThread] Exception in game 1
java.util.NoSuchElementException: No line found
at java.util.Scanner.nextLine(Unknown Source) ~[?:1.8.0_151]
at com.magusgeek.brutaltester.GameThread.run(GameThread.java:86) [cg-brutaltester.jar:?]
18:06:32,528 INFO [com.magusgeek.brutaltester.Main] *** End of games ***
18:06:32,531 INFO [com.magusgeek.brutaltester.Main] *** Statistics for player 1:
18:06:32,533 INFO [com.magusgeek.brutaltester.Main] *** Statistics for player 2:
18:06:32,533 INFO [com.magusgeek.brutaltester.Main] *** Statistics for player 3:

Does anybody has an idea?


#12

This stack can appears for 2 reasons:

  • Your AI crash.
  • Your AI does not flush its stdout and stderr. On codingame our AI are started with an auto flush command line. That is not the case for brutaltester.

#13

You can also get that error if the referee doesn't run correctly - which is very common on cg-mm.jar as it's compiled with a very recent version of java so most people get a version error.


#14

Oh. Nice to know :smiley:


#15

Same here :

01:04:26,797 ERROR [com.magusgeek.brutaltester.GameThread] Exception in game 100
java.util.NoSuchElementException: No line found
at java.util.Scanner.nextLine(Unknown Source) ~[?:1.8.0_131]
at com.magusgeek.brutaltester.GameThread.run(GameThread.java:86) [cg-brutaltester-0.0.1.jar:?]

:"(


#16

A quick message just to say that brutaltester can now handle the "new referees". Just see the README.md file to know how to create the referees.

As always, i'm open to Pull Requests :stuck_out_tongue:


#17

Oh that's great, i just wrote my own UTTT referee for brutaltester in c++ yesterday.


#18

Problem seem's to be -l "./logs/" , if you run without it, it should work. I had the same problem.


#19

If anybody else was stuck trying to run UTTT referee with this cryptic error

13:32:31,563 ERROR [com.magusgeek.brutaltester.GameThread] Negative score during game 1

Then the solution is to provide -Dleague.level=2 to your tictactoe.jar referee.

Full command example:

java -jar cg-brutaltester-1.0.0-SNAPSHOT.jar -r "java -jar -Dleague.level=2 tictactoe.jar" -p1 "cmake-build-debug/uttt" -p2 "cmake-build-debug/uttt" -t 1 -n 1

And if you're wondering how would I know that, then it's buried deep deep in the CG game engine sources:


#20

Wow nice catch. I think i'll update my github to set the default league level at 2.
Thanks you.