Chess plays as data for word2vec models

♟️ You shall know a piece by the company it keeps

Chess plays as data for word2vec models

Programmers, have you ever looked at a PGN file and thought: this looks like a language?

What if we trained word2vec on 5.4 million chess games, treating each move as a word and each game as a sentence? Would the vectors capture anything about chess strategy? Or would it be nonsense?

I asked exactly that. And the results are surprisingly… meaningful.

🔗 Original preprint: arXiv:2407.19600

🧩 The core idea

Word2vec relies on the distributional hypothesis: a word’s meaning is defined by its context (neighbouring words).

Orekhov’s move: apply the same to chess moves.

He built 18 different models from a corpus of ~5.4M high‑level games (≈840M moves).

Two main approaches:

• Type 1 – moves only (each move = a word, game = sentence)
• Type 2 – moves + board state (each “sentence” = current position + the move)

Then: train word2vec (skip‑gram / CBOW), get dense vectors (300‑500 dims), and run the usual NLP tricks – nearest neighbours, odd‑one‑out, analogies, t‑SNE visualisations.

🔍 What the model learned (without ever seeing a board)

📊 Over 68% of moves have at least 3 of their 10 nearest neighbours being the same piece moving to the same square. That’s not random – it’s structural.

🤯 The most beautiful visualisation

The model trained on games without queens produced a perfectly symmetric t‑SNE plot – white moves on one side, black moves mirrored.

The algorithm had no concept of “colour” as a feature. Yet the distributional signal was so strong that the two colour spaces separated like antipodal points on a sphere.

⚠️ The honest conclusion (refreshing for a paper)

“I don't see how this representation can be used productively. It's unlikely to help engines or humans choose the best move. But in a purely academic sense, it captures something important about the nature of the game.”

No overhyped “this will revolutionise chess AI”. Just a clean, curious application of NLP to a symbolic system – and it works.

🧰 For the hacker’s toolbox

• All models are public on HuggingFace: nevmenandr/w2v-chess
• Code & data: PGN corpus of 5.4M games (freely available online)
• Techniques used: word2vec, t‑SNE, cosine distance, analogies, “stemming” (removing source square), part‑of‑speech tags for captures

📚 Cite this work

APA

Orekhov, B. (2024). You shall know a piece by the company it keeps. Chess plays as a data for word2vec models. arXiv preprint. https://arxiv.org/abs/2407.19600

IEEE

B. Orekhov, “You shall know a piece by the company it keeps. Chess plays as a data for word2vec models,” 2024. [Online]. Available: arXiv:2407.19600

MLA

Orekhov, Boris. “You Shall Know a Piece by the Company It Keeps. Chess Plays as a Data for Word2vec Models.” arXiv, 2024, https://arxiv.org/abs/2407.19600.

BibTeX

@misc{orekhov2024shallknowpiececompany,
      title={You shall know a piece by the company it keeps. Chess plays as a data for word2vec models}, 
      author={Boris Orekhov},
      year={2024},
      eprint={2407.19600},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2407.19600}, 
}

👉 Would you run word2vec on Go games? Poker hands? Protein folding sequences? The method is language‑agnostic – and that’s the real lesson.