Thoughts on AI, Chess & Quality Engineering

Recently I’ve been learning to play Chess. Unfortunately I’m not very good, but it has given me something to think about regarding Artificial Intelligence and Machine Learning (AI/ML) in Software Testing & Quality Engineering.

In this blog I wanted to cover a few topics;

·       What is chess?

·       How has AI/ML been applied to chess?

·       What has the impact of AI/ML been on chess?

·       How can AI/ML learnings from chess be applied to other domains?

What is chess?

Chess is a strategic board game played between two opponents on an 8x8 grid of alternating light and dark squares. Each player commands an army of 16 pieces, including pawns, knights, bishops, rooks, a queen, and a king, each with distinct movement rules. The objective is to checkmate the opponent's king, placing it under attack with no legal moves to escape. Chess demands foresight, calculation, and tactical prowess, making it a timeless test of intellect and skill.

Chess is a closed system with a relatively small number of squares and pieces on a chess board. However there are a between 10^111 & 10^123 total moves in chess (10^40 legal moves) which is more than the number of atoms in the universe.

How has AI/ML been applied to chess?

AI/ML and chess have a long history together. As far back as 1951 Alan Turing published a program that on paper was capable of playing a full game of chess. In the 1980’s the AI/ML rivalry really started to heat up. Chess world champion Garry Kasparov, considered by many to be the best chess player of all time, claimed that AI chess engines would never be able to beat top human players. Kasparov was right for a time. He was able to beat IBM’s Deep Thought, in 1989, and its successor, Deep Blue, in 1996, until in 1997 Deep Blue was controversially able to beat Kasparov. Kasparov claimed that IBM cheated in the match.

Fast forward to today and AI/ML chess engines are common and are dramatically stronger than any human player. Stockfish, a popular open source chess engine, has an estimated ELO (chess rating) of over 3,500, there are difficulties in assessing the true strength of engines so this is likely underrated, compared to the rating of Magnus Carlsen, another candidate for chess GOAT and highest rated player of all time, of 2,882. If we use the ELO scores to predict the outcomes we see that Stockfish has a ~97% chance of winning and a ~3% chance of drawing.

Stockfish uses a specific type of Machine Learning called Efficiently Updated Neural Networks (NNUE). A Neural Network is a type of machine learning model inspired by the way that human brains work. There are multiple layers which consists of nodes, or artificial neurons, which are trained with large amounts of training data. These individual nodes have different characteristics (e.g. weightings & threshold values) which are adjusted over time based on the training data. Neural Networks are a very powerful form of Machine Learning and are utilized in Large Language Models (LLMs) which have become very popular with tools such as ChatGPT. If you want to learn more about the NNUE that makes Stockfish different from other chess engines check this out: https://www.chessprogramming.org/Stockfish_NNUE

AI/ML based chess engines, like Stockfish, are now embedded into common chess platforms like lichess (an open source platform) and chess.com (a commercial platform). Beginners and grand masters alike us AI/ML engines alike use chess engines to help them evaluate their play, identifying mistakes or inaccuracies, and gaining insight on how to improve.

What has the impact of AI/ML been on chess?

Despite the fact that AI/ML based chess engines are now definitively better than even the best humans at chess. AI/ML has had some interesting impacts on the game.

AI/ML has increased the overall level of chess skill. From beginner to Super Grand Master the standard of chess has improved as AI/ML is an invaluable teaching aid. It allows players to rapidly evaluate the strength of a position and investigate possible lines of play, as well as to review their games. This is particularly true with opening theory where now, with the help of chess engines, players will often have “book” (the best) moves memorized for the first 15-20 moves.

AI/ML has counter-intuitively made chess more creative. Players will rely on chess tactics and positional play to optimize their chances of winning which can sometimes result in dull games at the top level. Magnus Carlsen said that after working with AlphaZero, another chess engine, that he was inspired to  adopt a more attacking style of play.

AI/ML has made studying chess more accessible and contributed to an increase in popularity. High level chess coaching can be expensive, but the invention of chess engines means that you have access to the analysis of a “player” stronger than that of any human players. This “coach” allows all players to continuously improve their play even getting recommendations for lessons based on mistakes during a game.

Unfortunately, AI/ML has not been all positive for chess. One of the biggest challenges that the game faces is the ease with which a player can cheat using an engine. At any player's fingertips they can get access to chess engines which are more powerful than any player giving them a huge advantage. There have been some very “interesting” accusations of cheating at the top levels as well as in the amateur ranks, particularly for online matches. What makes this even more challenging is that top level players can gain a large advantage with relatively minor assistance from a chess engine. Receiving guidance at only a critical spot can be the difference between winning or losing a game..

How can AI/ML learnings from chess be applied to other domains?

Firstly, we should recognize that chess has been a proving ground for AI/ML from the very early days of the field. Because chess has a well defined rule structure, it has been useful for researchers to use as a testing ground for different algorithms and machine learning approaches. We see approaches piloted in chess that are later applied to many different areas of the modern world including natural language processing, reinforcement learning, game theory etc.

Looking at the use of AI/ML, the biggest lesson that I take is that all of the best players use AI/ML in some way. This should be something that we all internalize. AI/ML is not going away and for us to be successful in our endeavors, whatever they may be, we need to determine the best places to use AI/ML. In software testing and quality engineering we can’t be afraid to use AI/ML tools. We should make use of all of the tools that we have at our disposal in order to be the best that we can possibly be.

Often the conversation around AI/ML is in how it will decrease human creativity, but as we see with AlphaZero the fact that AI/ML “thinks” in a different way to humans can give us insights that we otherwise may not have had. When used appropriately AI/ML can actually make us more creative.

We also need to be aware of the limitations of AI/ML. For example top chess players will often look at a position where a chess engine believes that there is a theoretical advantage to one player, but that assumes perfect play by the chess engine. In practice a player may not be able to capitalize on that advantage due to the complexity of the position and for practical purposes they may in fact be worse.

The use of AI/ML in Chess also highlights that AI/ML is not synonymous with GenAI or LLMs. Chess Engines use a related but different type of AI/ML than the LLMs which are currently very popular. There are many current real world AI/ML use cases including facial recognition, product recommendations on online stores, e-mail automation and spam-filtering, and many more. While GenAI and LLMs present a very interesting advancement in AI/ML we need to broaden our understanding to different kinds of AI/ML, and ensure that we select the most appropriate AI/ML for our use case.

Finally, for me the prevalence of AI/ML in chess is a sign of things to come in other areas of our lives. AI/ML will continue to improve with better access to training data, more complex models, and improved computational speed. Just like a top chess player using chess engines, whatever area we are operating in we need to find the right way to use AI/ML in our context and adapt.