The FFO endgame test suite is a common benchmark for endgame searching. Computer completely solves each testcase, and find the best move. This benchmark evaluates the exact time for searching and the speed (NPS: Nodes Per Second). In this section, you can see the result of FFO#40-59.
AVX512 edition do not works with Core i9-13900K, so there are only SIMD, Generic, and x86 editions.
Egaroucid's result are here. As a comparison, I also show Edax 4.5.3's result. Although Edax 4.6 is the latest, but on my environment, I found that 4.5.3 is faster than 4.6.
| AI | Edition | Time (sec) | Nodes | NPS | File |
|---|---|---|---|---|---|
| Egaroucid | SIMD | 20.547 | 14035262270 | 683080852 | 000_ffo40_59_Core_i9-13900K_SIMD.txt |
| Egaroucid | Generic | 30.346 | 14290308977 | 470912442 | 001_ffo40_59_Core_i9-13900K_Generic.txt |
| Edax | x64_modern | 26.093 | 28087572364 | 1076440898 | 010_ffo40_59_Core_i9-13900K_edax_x64_modern.txt |
| Edax | x64 | 30.299 | 27886392112 | 920373349 | 011_ffo40_59_Core_i9-13900K_edax_x64.txt |
With Core i9-11900K, you can run AVX512 edition.
Egaroucid's result and Edax 4.5.3's result are:
| AI | Edition | Time (sec) | Nodes | NPS | File |
|---|---|---|---|---|---|
| Egaroucid | AVX512 | 34.272 | 13061469404 | 381111969 | 100_ffo40_59_Core_i9-11900K_AVX512.txt |
| Egaroucid | SIMD | 35.089 | 13275458500 | 378336757 | 101_ffo40_59_Core_i9-11900K_SIMD.txt |
| Egaroucid | Generic | 62.678 | 13403862190 | 213852742 | 102_ffo40_59_Core_i9-11900K_Generic.txt |
| Edax | x64_avx512 | 39.046 | 27252069199 | 697947785 | 110_ffo40_59_Core_i9-11900K_edax_x64_avx512.txt |
| Edax | x64_modern | 40.718 | 26747187985 | 656888550 | 111_ffo40_59_Core_i9-11900K_edax_x64_modern.txt |
| Edax | x64 | 48.263 | 26602214755 | 551192731 | 112_ffo40_59_Core_i9-11900K_edax_x64.txt |
Edax 4.6 is one of the best Othello AI in the world.
If I set the game from the very beginning, same line appears a lot. To avoid this, I set the game from many different near-draw lines.
If Egaroucid Win Ratio is over 0.5, then Egaroucid wins more than Edax do. In all conditions, Egaroucid is stronger than Edax.
The average number of discs earned indicates how many more discs Egaroucid was able to acquire than Edax on average. The higher this value, the bigger the victory over Edax.
I used XOT for its testcases. No opening books used.
| Level | Avg. Discs Earned | Winning Rate | Egaroucid Win | Draw | Edax Win |
|---|---|---|---|---|---|
| 1 | +11.27 | 0.742 | 734 | 15 | 251 |
| 5 | +8.37 | 0.793 | 781 | 25 | 194 |
| 10 | +1.72 | 0.633 | 590 | 87 | 323 |
| 15 | +0.89 | 0.614 | 138 | 31 | 81 |
| 21 | +0.45 | 0.565 | 50 | 13 | 37 |