I wrote Technology Explanation only in Japanese. I'm afraid but please translate by yourself.
I used 2 benchmarks for evaluating Egaroucid. The first one is The FFO endgame test suite. This test is for the speed of endgame complete search. The second one is the matches against old versions of Egaroucid and Edax 4.4. To test the strength of its evaluation function, I used no book, and used XOT for the starting positions.
The endgame search is evaluated by 3 features:
The most important feature for users is the search time. This feature is shown as the actual time (second) to solve The FFO endgame test suite #40 to #59. This value is good if it decreases.
To shorten the search time, we can do two things: decrease the number of nodes and increase the number of nodes visited in a unit time.
There are some graphs of results of The FFO endgame test suite on Core i9 13900K, SIMD version.
You can see that Egaroucid 7.4.0 takes longer time than 7.3.0. This is because of Intel microcode 0x129. In 7.3.0 measurement, 0x129 was not included, but in 7.4.0, 0x129 was included.
It is the best way to evaluate the strength of Othello AI that we have battles with some engines. The result of battles by each version of Egaroucid and Edax 4.4 is below.
To avoid same lines, I used XOT as the beginning board. Each battle is done in level 1 (lookahead depth is 1 for the midgame, 2 for the endgame).
Name | 7.4.0 | 7.3.0 | 7.2.0 | 7.1.0 | 7.0.0 | 6.5.X | 6.4.X | 6.3.X | 6.1.X | 6.0.X | Edax |
---|---|---|---|---|---|---|---|---|---|---|---|
Winning Rate | 0.5836 | 0.5532 | 0.5492 | 0.5471 | 0.5332 | 0.5094 | 0.4779 | 0.4441 | 0.4812 | 0.4178 | 0.4033 |
Version | 7.4.0 | 7.3.0 | 7.2.0 | 7.1.0 | 7.0.0 | 6.5.0 | 6.4.0 | 6.3.0 | 6.2.0 | 6.1.0 | 6.0.0 | 5.10.0 | 5.9.0 | 5.8.0 | 5.7.0 | 5.5.0/5.6.0 | 5.4.1 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Date | 2024/10/03 | 2024/08/16 | 2024/06/25 | 2024/06/06 | 2024/04/17 | 2023/10/25 | 2023/09/01 | 2023/07/09 | 2023/03/15 | 2022/12/23 | 2022/10/10 | 2022/06/08 | 2022/06/07 | 2022/05/13 | 2022/03/26 | 2022/03/16 | 2022/03/02 |