Cg-brutaltester, because you love to be brutal

inoryy · March 25, 2018, 12:23pm

No problem, I wanted to do that and submit a PR, but couldn’t figure out a clean way to set it (as you can see they call System.getProperty every time)…

While we’re on the subject of updating UTTT referee, maybe change timeout to 1000ms globally as it seemed to crash for me with default 100ms (even though bot itself has strict 95ms limit).

_CG_Maxime · March 31, 2018, 7:06am

I had to change this for the logs:

--- a/src/main/java/com/magusgeek/brutaltester/GameThread.java
+++ b/src/main/java/com/magusgeek/brutaltester/GameThread.java
@@ -47,7 +47,7 @@ public class GameThread extends Thread {
                }

                if (logs != null) {
-                       command[playersCount * 2 + 2] = "-l";
+                       command[splitted.length + playersCount * 2] = "-l";
                }
        }

Magus · March 31, 2018, 10:01am

Oh yes i forgot to test logs with new referees

dbf · May 6, 2018, 9:48pm

Hi Magus,
I’ve updated game-ultimate-tictactoe and cg-brutaltester from your github and still have ‘Negative score during game 1’ running with
C:\Programs\JDK8\bin\java.exe -jar C:\Projects\cg-brutaltester\target\cg-brutaltester-1.0.0-SNAPSHOT.jar -r "java -jar C:\Projects\game-ultimate-tictactoe\target\tictactoe-1.0-SNAPSHOT.jar" -p1 test_p1.bat -p2 test_p2.bat -t 1 -n 1
I’ve also tried to pass -Dleague.level=2 to referee java call, but it doesn’t change anything. Is there something else that should be done?
Thanks.

Magus · May 7, 2018, 9:55am

First, don’t forget the -Dleague.level=2 in the referee command line.

Second, if you are on windows like me, the codingame SDK got a big problem to measure time. So most of the time the game is finished with a timeout at first round …

To avoid that i changed the timeouts constants in the referee. Just replace them with a very big number

nafesto · June 19, 2018, 10:00pm

I’m having almost the same issue using cg-brutaltester for the first time on the current contest Code of Kutulu.
I program in Groovy

java -jar cg-brutaltester.jar -r "java -jar cok-referee.jar" -p1 "src/CodeOfKutulu.groovy" -p2 "src/CodeOfKutulu.groovy" -p3 "src/CodeOfKutulu.groovy" -p4 "src/CodeOfKutulu.groovy" -n 1 -v

21:11:00,257 INFO [com.magusgeek.brutaltester.Main] Verbose mode activated
21:11:00,258 INFO [com.magusgeek.brutaltester.Main] Referee command line: java -jar cok-referee.jar
21:11:00,259 INFO [com.magusgeek.brutaltester.Main] Player 1 command line: src/CodeOfKutulu.groovy
21:11:00,260 INFO [com.magusgeek.brutaltester.Main] Player 2 command line: src/CodeOfKutulu.groovy
21:11:00,261 INFO [com.magusgeek.brutaltester.Main] Player 3 command line: src/CodeOfKutulu.groovy
21:11:00,261 INFO [com.magusgeek.brutaltester.Main] Player 4 command line: src/CodeOfKutulu.groovy
21:11:00,261 INFO [com.magusgeek.brutaltester.Main] Number of games to play: 1
21:11:00,264 INFO [com.magusgeek.brutaltester.Main] Number of threads to spawn: 1
21:11:00,364 ERROR [com.magusgeek.brutaltester.GameThread] Exception in game 1
java.util.NoSuchElementException
at java.util.Scanner.throwFor(Scanner.java:862) ~[?:1.8.0_112]
at java.util.Scanner.next(Scanner.java:1485) ~[?:1.8.0_112]
at java.util.Scanner.nextInt(Scanner.java:2117) ~[?:1.8.0_112]
at java.util.Scanner.nextInt(Scanner.java:2076) ~[?:1.8.0_112]
at com.magusgeek.brutaltester.GameThread.run(GameThread.java:85) [cg-brutaltester.jar:?]
21:11:00,369 ERROR [com.magusgeek.brutaltester.GameThread] If you want to replay and see this game, use the following command line:
21:11:00,371 ERROR [com.magusgeek.brutaltester.GameThread] java -jar cok-referee.jar -p1 src/CodeOfKutulu.groovy -p2 src/CodeOfKutulu.groovy -p3 src/CodeOfKutulu.groovy -p4 src/CodeOfKutulu.groovy -s
21:11:00,372 INFO [com.magusgeek.brutaltester.Main] *** End of games ***
±---------±---------±---------±---------±---------+
| Results | Player 1 | Player 2 | Player 3 | Player 4 |
±---------±---------±---------±---------±---------+
| Player 1 | | NaN% | NaN% | NaN% |
±---------±---------±---------±---------±---------+
| Player 2 | NaN% | | NaN% | NaN% |
±---------±---------±---------±---------±---------+
| Player 3 | NaN% | NaN% | | NaN% |
±---------±---------±---------±---------±---------+
| Player 4 | NaN% | NaN% | NaN% | |
±---------±---------±---------±---------±---------+

The bot works fine in the online CodinGame IDE.
I tried using the following in the beginning:

System.err.flush()
System.out.flush()

but that didn’t help.
Any idea how to fix this?

Edit: Tried using only Players from the tests here https://github.com/JohnnyGuye/code-of-kutulu/tree/master/codeOfKutulu/src/test/java/runner/tests

java -jar cg-brutaltester.jar -r "java -jar cok-referee.jar" -p1 "Random.java" -p2 "Waiter.java" -p3 "Yell.java" -p4 "Useplan.java"

but same error

fala · June 20, 2018, 10:09am

I managed to alter Code of Kutulu referee to run it with Cg-brutaltester. Not sure on the moral aspect of sharing the code during the contest (I can put it to github if some admin or Magus says its ok), but I can offer some clues that might be useful also for future contests:

Get the game running locally using the referee only with the CommandLineInterface from Cg-brutaltester.

Start by applying the steps on https://github.com/dreignier/cg-brutaltester
You can start with some test players if they are provided in tests for referee code.
Changing the logging level to ‘trace’ helps for debugging ("rootLogger.level = trace" in log4j2.properties of referee). You might want to revert it once it runs fine.
Some method/class names can change comparing to tictactoe in https://github.com/CodinGame/codingame-game-engine . Download it and compare.
The lack of stdout flushing of players is a thing to look out for. For python -u option is enough, e.g. python -u kt.py
Before trying to run Cg-brutaltester make sure you can run referee with and without -s. E.g. for CoK this should work (depending on your path to test classes):
java -jar codeOfKutulu-1.0.jar -p1 "java -cp . runner.tests.random.Player" -p2 "java -cp . runner.tests.random.Player" -p3 "java -cp . runner.tests.uselight.Player" -p4 "java -cp . runner.tests.yell.Player" -l ./logs/game1.json -s
Without -s it should give you the final score and seed only.

Then it should work with Cg-Brutaltester

I’m not a Java guy, but I was able to that without any Java IDE, just had to get maven to build the packages (go into the directory with pom.xml and run mvn package) and change a name here and there.

fala · June 20, 2018, 10:14am

As input to -p1, p2… you should provide a string that can be used to create a process. For groovy I guess it would be groovy src/CoK.groovy and for java you have to compile it into .class files and then run like java -cp . Waiter (mind the classpath).

Telokis · September 9, 2021, 5:03pm

After taking a look, I stumbled on your answer. It doesn’t work anymore. I tried running the referee manually like so but I’m still in league 1 because my first 4,4 move triggers an error.

The command I used is the following one:

java -Dleague.level=2 -jar ../referees/tictactoe.jar -l logs -p1 ../bench/codingame.bench_1-1.0.0.exe -p2 ../bench/codingame.bench_2-1.0.0.exe -d seed=2596996162

Not sure if it’s relevant but I’m on Windows 10 with Java 1.8.0_281.

fgsdt · May 4, 2022, 7:03pm

Hello!!

I have been delaying the use of brutaltester for some time, but at the end I would have found it useful for this latest contest, with pretty bad leadearboard submision times the last three days. So I decided to try to finally install it and test it.

I think I have followed the steps explained in the GitHub.

I already had the java jdk 1.8
I installed Maven
I compiled the brutal tester apparently ok, I got the .jar as expected without errors
I also compiled downloaded and compiled the referee (spring2022)

But when I try to execute it with my bots I get an error with the log4j, as class not found:

Exception in thread “main” java.lang.NoClassDefFoundError: org/apache/logging/log4j/util/ReflectionUtil
at org.apache.logging.log4j.jcl.LogAdapter.getContext(LogAdapter.java:39)
at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:47)
at org.apache.logging.log4j.jcl.LogFactoryImpl.getInstance(LogFactoryImpl.java:40)
at org.apache.logging.log4j.jcl.LogFactoryImpl.getInstance(LogFactoryImpl.java:55)
at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:655)
at com.magusgeek.brutaltester.Main.(Main.java:16)
Caused by: java.lang.ClassNotFoundException: org.apache.logging.log4j.util.ReflectionUtil
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)

I am not very good at java environments. I have seen a reference to the log4j in the pom.xml of the brutalforce project, but I do not know how to make it work.

Please, any idea?
Thanks

raxkin · May 6, 2022, 8:14am

I had the same problem for this challange. Removing this dependence from the brutalforce project fixed for me:

        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-jcl</artifactId>
            <version>2.4.1</version>
        </dependency>

Also make sure you are using a modified spring2022 project where it have the CommandLineInterface main. This is one already modified that i have been using:

fgsdt · May 6, 2022, 10:34am

Thanks a lot for the answer…

I have done as you said, and now I have a different problem. It “plays” all the games but for each one it gives me an error (and consequently 0,0% victories for everybody):

ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property ‘log4j2.debug’ to show Log4j2 internal initialization logging.

Just in case, these are the steps I have followed:

Download brutaltester from GitHub - dreignier/cg-brutaltester: A local arena for codingame multiplayer puzzles
delete in pom.xml the dependency you just said (there is still remaining another one refering to log4j)
Create .jar with maven and rename it to “cg-brutaltester.jar”.
Download the spring2022 from GitHub - johnpage-agixis/SpringChallenge2022
Create .jar with maven and rename it to “spider-attack-spring-2022.jar”.
In this spider-attack directory I copied the brutaltester.jar and the two .exe (c++ compiled) players.
I execute the brutaltester:

java -jar cg-brutaltester.jar -r “java -jar spider-attack-spring-2022.jar” -p1 “./spring2022.exe” -p2 “spring2022-3.5.exe” -t 2 -n 10

I must be missing something, but I do not know what. Any more ideas?

Thanks a lot

raxkin · May 6, 2022, 12:17pm

I think this error should not be a problem and you can ignore it. The problem might be that the players are not executing correctlly? Did you test executing manually and check if it is waiting for input? Maybe you can test the same command but with any starterAIs from spring2022, for example setting the player to “java Starter.java” to verify that is not a problem of the exe.

Other option is that the game is tied every time, and i think it does not calculate the winner by wild mana so there is no winner? But not sure about that.

Astrobytes · May 6, 2022, 1:21pm

If I remember correctly, this will fix it

fgsdt · May 6, 2022, 3:55pm

Ok. I tried with a python starter as the opponent. It gives him a 100%, so, something is failing on my c++ exe, but I cannot say what. I have executed apart the c++ exe and it waits for the input, and if the input is correct it returns something likable to be a right answer.

Note: I have tried to include in the command the -Dleague.level=2 that Astrobytes said. It doesn’t seem to do anything different.

I will continue searching, meanwhile thanks for the tips.

raxkin · May 6, 2022, 5:06pm

For the -Dleague.level=2, it’s to define the league rules, like wood 2 wood 1, etc… but the project i send you, already have it hardcoded to league lvl 3 so it’s bronze, so no need to change anything.

Maybe it’s something about pemisions of the executable? But at least you know where the problem is now.

elderlybeginner · May 7, 2022, 6:32pm

I’m getting the following error:

20:25:55,992 INFO  [com.magusgeek.brutaltester.Main] Referee command line: java -jar target/spider-attack-spring-2022-1.0-SNAPSHOT.jar
20:25:55,994 INFO  [com.magusgeek.brutaltester.Main] Player 1 command line: python3 sc2022.py
20:25:55,994 INFO  [com.magusgeek.brutaltester.Main] Player 2 command line: python3 sc2022.py
20:25:55,994 INFO  [com.magusgeek.brutaltester.Main] Number of games to play: 1
20:25:55,995 INFO  [com.magusgeek.brutaltester.Main] Number of threads to spawn: 1
20:26:01,341 ERROR [com.magusgeek.brutaltester.GameThread] Problem with referee output in game1. Output content:WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.

20:26:01,342 ERROR [com.magusgeek.brutaltester.GameThread] Error during game 1
20:26:01,343 ERROR [com.magusgeek.brutaltester.GameThread] ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property 'log4j2.debug' to show Log4j2 internal initialization logging.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$2 (file:(...)/SpringChallenge2022/target/spider-attack-spring-2022-1.0-SNAPSHOT.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$2
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

I’ve compiled referee from “johnpage-agixis” and ignored the part with ts and npm. I made no modifications. SC2021 works, but SC2022 don’t. Would you mind sharing any ideas about the problem?

edit:
If I misspell player file e.g:

-p1 "python3 starterAIs/starter.py" -p2 "starterAIs/sssstarter.py"

P1 wins, but with proper files names it’s 0-0

OldJohn · September 3, 2022, 4:15pm

The error of google.inject is from codingame source code… Still not corrected…

Is there is anyone that got success using cultists-wars referee with cg-brutaltester ?

OJ

Akadine · January 29, 2023, 8:36pm

Try my c++ 20 version that uses only standard libraries. I built everything else needed, and then recoded brutal tester pretty much exactly, no third party libraries needed!