Memory: The Key To Chess?

Examining the connections between chess and memory research

Magnus Carlsen remembers more chess games than you do. This is not surprising. What may be surprising, though, is the sheer extent of Carlsen’s recall. He remembers the moves and circumstances not only of his own games, but other players’ as well, going back decades. Is this then the secret of Magnus’ greatness: a vast internal library of chess games? Or is it the other way around: can he remember so much because he understands chess so well?

Chess is closely intertwined with the history of memory research. The chess master and psychologist Adriaan de Groot showed a chess position to players of varying levels for a few seconds and then asked them to reconstruct it from memory. The world champion Max Euwe was able to reconstruct the position perfectly. A master got most, but not all, of the pieces on the right squares. An average player got less than half the pieces.

Yet other experiments have found that being good at general memory tasks is not especially correlated with chess strength. And the memory feats of chess masters are limited to chess: when it comes to remembering other sorts of things, they are no better than average. (In an interview on 60 Minutes, Carlsen mentioned that he often loses his keys.)

In another experiment, researchers did a twist on de Groot’s recall task. They asked people to reconstruct a position after looking at it briefly, but rather than taking positions from real chess games, they arranged the pieces on the board completely randomly. In this case, chess expertise proved to be only a small advantage when it came to reconstructing the position. This and similar experiments have led scientists to believe that the ability of masters to remember chess positions depends on what they call “chunking.” A chunk is a pattern of related ideas. In chess, it could be an arrangement of pieces that occurs frequently. This is why the masters were much better at remembering real positions, but only a little better at the random positions. In the real positions they were able to divide the board into familiar chunks, in effect having to remember less things to reconstruct the position. In the random positions, the pieces were not in the familiar patterns, so they had to try to remember them one-by-one.

To see the power of chunking, imagine you had to memorize this fragment of a Shakespeare sonnet:

When to the sessions of sweet silent thought

I summon up remembrance of things past,

I sigh the lack of many a thing I sought,

And with old woes new wail my dear time’s waste …

You could probably commit it to memory in a few minutes without too much trouble. But now imagine you had to remember a completely random sequence of 177 characters (the number in the fragment). It would certainly be much harder. It’s much easier to remember the same amount of information if it’s grouped into familiar chunks with meaningful connections between them.

This principle is important to keep in mind when memorizing opening lines. If you’re going to memorize a line, you want it to be like a poem, not a random string of characters. That is, the moves should connect with one another to tell a story. If it’s just a random sequence of moves with no underlying order, you probably won’t remember it correctly, and even if you do, it won’t do you much good in a real game. As soon as you reach the end of the line or your opponent deviates, you’ll have no logic to fall back on.

The general theory of chunking makes sense, but it’s not entirely clear what these chess chunks would actually consist of. The psychologist Herbert Simon estimated that a grandmaster knows 1.8 million chunks. Presumably most of these would be operating subconsciously – I don’t know of any grandmasters who could articulate 1.8 million piece arrangements that they’re on the lookout for.

In a neural network chess engine like AlphaZero, information passes through a series of layers, which encode the information in increasingly higher levels of complexity or abstraction. It is tempting to think that the chunks that grandmasters learn could also be found in AlphaZero’s internal representations of chess positions. That would be nice, because while you can’t dissect a grandmaster’s brain – or at least you shouldn’t – you can examine what’s going on inside AlphaZero (after a fashion: there is no direct way to query what a neural network “knows,” but researchers worked out a way to show which squares “light up” in certain sections of the network). Still, we should be careful with such analogies. Neural networks are only loosely inspired by the brain: every detail of how they function is vastly different from a biological brain. Then there’s the matter of the representations themselves. When you visualize them, it turns out that most don’t correspond to any (known) chess concept. As the authors say:

“While some of these concepts are shown to be related to human chess concepts… some concepts may be something we don’t yet have a name for, or too complex for us to understand (yet).”

If you want to try your hand at decoding these mysterious patterns, you can peruse them online. The authors of the paper have requested descriptions of the patterns these blocks are describing. Looking through a few at random, I wasn’t able to make sense of them, but maybe you’ll have better luck.

In the video from the start of this post, the only position that David Howell manages to stump Magnus with is from Harry Potter. Magnus only gets it after David gives him a hint that it’s from the entertainment industry. Initially I thought Magnus was just guessing Harry Potter because it’s a prominent piece of media that features chess, but as he continues talking, it becomes clear that he remembers quite a bit about the game.

“Okay, so Black is down a queen… So from the first Harry Potter movie they played the Scandinavian and then they went knight c3, queen c6, so is it from that one?”

During that scene, most viewers were probably wondering whether the kids would succeed in stopping professor Snape from getting the sorcerer’s stone, or if Ron would regain consciousness after being thrown from a knight, but Magnus was focusing on the details of the position. Maybe this then is the secret of both Magnus’s amazing chess memory and his prodigious skill. Compared to a normal person, or even a normal grandmaster, he is much, much more interested in chess.