Find out why AIs are stumped by some games

In nimThere are a limited number of optimal moves for a given board configuration. If you don’t play one of them, you essentially hand over control to your opponent, who can win by making nothing but optimal moves. And again, optimal moves can be identified by evaluating a mathematical parity function.

So there are reasons to think that the training process that worked for chess might not be effective for nim. The surprise is how bad it actually was. Zhou and Riis discovered that during a nim On a board with five rows, the AI ​​improved quite quickly and was still improving after 500 training iterations. However, adding just one more row caused the rate of improvement to slow down dramatically. And, for a seven-row board, performance improvements had virtually stopped by the time the AI ​​had played 500 times.

To better illustrate the problem, the researchers changed the subsystem that suggested potential moves to one that operated randomly. In seven rows nim dashboard, the performance of the trained and randomized versions was indistinguishable over 500 training gains. Basically, once the board became large enough, the system was unable to learn by observing the results of the game. The initial state of the seven-row setup has three potential moves that are consistent with an ultimate win. However, when their system’s trained motion evaluator was asked to verify all potential motions, he evaluated each of them as approximately equivalent.

The researchers conclude that Nim requires players to learn the parity function to play effectively. And the training procedure that works so well for chess and Go is unable to do so.

not only nim

One way to look at the conclusion is that nim (and by extension, all unbiased games) is just weird. But Zhou and Riis also found some signs that similar problems could also arise in chess-playing AIs that were trained this way. They identified several “incorrect” chess moves (those that failed a mate attack or caused an endgame) that were initially rated highly by the AI ​​board evaluator. It was only because the software took a series of additional branches in various moves into the future that it was able to avoid these errors.