Project #10: Chess and The Elo System

In this DISCOVERY Data Science Project, you will do real data science in less than an hour and you will earn this project's card to your collection when you fully complete this project! 🎉

Data Source: Lichess.org Open Database

The website lichess.com is an open source and free platform for playing chess online. As part of their website, lichess provides the "lichess.org open database" that contains an absolutely massive number of chess games, puzzles, and engine evaluations. Billions of games are stored in the database with about a hundred million being added each month.

In this project, you will analyze a small collection of the data and explore using Python functions to evaluate the Elo system and check the accuracy and efficacy of the Elo rating. Let's nerd out! 🎉

Background Knowledge

To finish this project, we assume you already know:

Let's get started! :)

Part 1: Importing the lichess.com Dataset

As part of this project, we have selected the 121,114 games played in January 2013, provided for you in the dataset lichess_games_1-2013.csv (218 games were excluded due to missing data).

Load the lichess_games_1-2013.csv dataset into a DataFrame stored in the Python variable df:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

Data Dictionary: Understanding the Data

A chess game occurs between two players: one player who plays with the white chess pieces and a second player who plays with the black chess pieces. Each player has a skill rating, called their "Elo rating", that measures their skill at the game of chess.

  • The columns WhiteElo and BlackElo contain data about the ratings of each player before the game begins.

A chess match will always result in either the white pieces winning (represented by 1-0, meaning that "white scored one point"), the black pieces winning (represented by 0-1, meaning "black scored one point"), or a draw where neither player wins (represented by 1/2-1/2, meaning both players scored a half-point).

  • The column Result contains either 1-0, 0-1, or 1/2-1/2.

To ensure a game of chess does not last forever, every chess match is played with a "time control". The time control indicates the total number of seconds each player has for the entire game and how many additional seconds each player gets for each move played. If a player runs out of time, they forfeit the game due to running out of time (called a "Time forfeit").

  • The column TimeControl indicates the time control for the game. A time control of 60+1 indicates each player has 60 seconds to play the entire game with an additional 1 second for each moved played.
  • The column Termination contains "Normal" for a normal game ending, or "Time forfeit" if a player runs out of time.

Finally, the first several moves of a chess game determines the overall "structure" of the game in high-level chess. Each common sequence of initial moves are given a name (ex: "Queen's Gambit", "Scandinavian Defense", "Caro-Kann Defense", etc). The opening used in each game and the three character code from Encyclopaedia of Chess Openings for the opening is provided as part of this dataset.

Part 2: The Elo System

A player's Elo changes after each game, and the change depends on the Elo of the opponent. Someone who wins against a higher-rated player gains more Elo than if they win against a lower-rated player. This reflects the fact that a higher-rated player is considered more likely to win. But exactly how likely?

Suppose two players, A and B, are playing a game of chess. A simplified Elo formula for the probability that A wins is:

$$P(\text{A wins}) = \frac{1}{1 + 10^{(-r/400)}}$$
where
$$r = \text{A's Elo} - \text{B's Elo}$$

Example Calculation

For example, if A has an Elo of 1000 and B has an Elo of 1200:

$$r = \text{A's Elo} - \text{B's Elo}$$
$$r = \text{1000} - \text{1200}$$
$$r = -200$$

Then, solving for the probability:

$$P(\text{A wins}) = \frac{1}{1 + 10^{(-r/400)}}$$
$$P(\text{A wins}) = \frac{1}{1 + 10^{(-(-200)/400)}}$$
$$P(\text{A wins}) = 0.2403 = 24.03\%$$

It makes sense that A has a low chance of winning, since they are rated 200 points lower than B.

Simplified Formula Assumption

This formula assumes that there are only two results: A wins or B wins. A more complicated formula accounts for draws, but we won't use it here.

This means that for this project (and any time you cannot have a tie in an Elo system):

$$P(\text{B wins}) = 1 - P(\text{A wins})$$

Part 2.1: Probability of Winning Formula

Create a function p_A_wins that takes two arguments, elo_A and elo_B, and returns the probability that player A wins using the formula outlined earlier:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

Keep in mind that the formula from this section is a very rough approximation. In reality, the probability of winning is affected by many factors, such as whether the player is playing white or black, the time control, and whether their current Elo truly reflects their skill level.

⚙️ Test Case: Part 2.1: Probability of Winning Formula

Part 2.2: Applying Your Elo Formula

A major chess cheating scandal in 2022 had a previous World Champion, Magnus Carlsen, accusing his opponent Hans Niemann of cheating after losing to him.

At the time, Hans Niemann had an Elo of 2688, and Magnus Carlsen had an Elo of 2861. Use your function to estimate the probability of Hans Niemann winning and store that probability in the variable p_Hans_wins:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)
⚙️ Test Case: Part 2.2: Applying Your Elo Formula

Part 2.3: Your Elo

Let's take a look at your probabilities of winning against different opponents!

  • If you have a chess Elo, put it in the variable your_elo.
  • If you don't have an Elo, feel free to use 1200 or another reasonable value. (An Elo of 500 would represent someone who understands the rules of chess, but uses very little strategy; an Elo of 1200 might be someone who plays recreationally; and an Elo of 2000 begins to be range of players who play professionally.)
Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

The lowest possible Elo on Lichess is 400. Calculate the probability of winning against a player with that Elo (use the variable your_elo that you defined above).

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

Two equally rated players should have an exactly 50% chance of winning. Let's verify this by calculating the probability of winning when you play a game against someone with the exact same Elo rating as yourself:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

Part 2.4: Average Elo

Now calculate the average Elo of every player in the dataset. Make sure to include both white and black players.

Hint: There are exactly as many white players as black players in the dataset, so you can average the two means.

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

Now calculate the probability of winning against a player with the average Elo.

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

The highest Elo ever reached by a human was 2882 by Magnus Carlsen in 2014. Calculate the probability of winning against Magnus Carlsen at his peak.

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)
⚙️ Test Case: Part 2.3 and Part 2.4: Elo Calculations

Part 2.5: Plotting Your Winning Chances

We've looked at a few different win probabilities. Now let's plot a bunch of different ones to see how the probability changes with Elo!

Create a DataFrame called df_win_prob with the columns Elo and p_win.

  • You should consider all values of Elo from 400 to 2900 (inclusive).
  • The p_win column should contain the probability of you winning against a player with that Elo.

Creating this can be done in a way very similar to a simulation, where you are recording every possible Elo value (in our specified range) and are calculating the probability of winning against each Elo value given your own Elo value.

It may be useful to use the two-argument range function in Python. Traditionally, we usually use the one-argument range function that starts at 0 and stops at the number you specify. For example, range(100) starts at 0, stops at 100 and includes all the numbers 0...99.

for i in range(100):
  print(i)
# Result: 0  1  2  ...  98  99

Using the two argument range function, you specify the start and the stop. If we want to start at 42 instead of 0, the following code starts at 42 instead of the default of 0:

for i in range(42, 100):
  print(i)
# Result: 42  43  44  ...  98  99

It may be useful to do something like this for your Elo, similar to:

for Elo in range(10, 20):
  print(Elo)
# Result: 10  11  12  ...  18  19

Create a DataFrame called df_win_prob that considers the probability of you winning for every Elo in the range 400..2900, with the columns Elo and p_win:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

Using df_win_prob and plot.line, plot a line chart. The x-axis should be Elo, and the y-axis should be p_win.

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)
⚙️ Test Case: Part 2.5: Plotting Your Winning Chances

Part 3: Probability vs. Reality

Let's take everything we've done and put it together!

  • We have a function to calculate the probability of winning given two Elo ratings, and
  • We have a dataset of games where we know which side won the game.

Let's find out if the Elo system really works in practice.

Part 3.1: Decisive Games

Since our function doesn't allow for draws, filter the dataset to only include decisive (no draw) games. Store the result in a new DataFrame called df_decisive:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)
⚙️ Test Case: Part 3.1: Decisive Games

Part 3.2: Probability of White Winning

The code below creates a copy of your df_decisive DataFrame and adds a new column p_win_white that is the probability that white is expected to win the game given the Elo differences. This code uses the p_A_wins you defined earlier and the apply function of the DataFrame.

Run the code to add the column:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

Part 3.3: Unlikely Upsets

Let's explore games that are unlikely upsets. We'll define an unlikely upset to be a game where:

  • Black wins (0-1) when there is at least a 90% probability that white was to win, OR
  • White wins (1-0) when there is at least a 90% probability that black was to win

Find all the unlikely upsets in df_decisive and save it as a new DataFrame df_upsets:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

Part 3.4: Likely Wins

To see how common unlikely upsets are, we need to also find out how often likely wins occurs. We'll define an likely win to be a game where:

  • White wins when there is at least a 90% probability that white was to win, OR
  • Black wins when there is at least a 90% probability that black was to win

Find all the likely wins in df_decisive and save it as a new DataFrame df_likely:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

Part 3.5: Percentage of Likely Wins

If our Elo formula is an accurate representation of a player's skill, among games where there's a 90% or greater chance for one side to win, that side should actually win 90% of the time.

In the previous two parts, we've calculated all the games where there was a 90% or greater chance for one side to win. Now, calculate the probability that a "likely win" occurs instead of an "upset" by calculating the following probability:

$$ P(\text{Likely Win}) = \frac{ \#\text{ of Likely Win Games} }{ \#\text{ of Likely Win Games } + \#\text{ of Upset Games} } $$

Store the calculation of $P(\text{Likely Win})$ in the variable P_likely_win:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)
⚙️ Test Case: Part 3: Probability vs. Reality

JKLMNOP: Just Keep Letting Me Nerd Out Please

This project contains just 0.1% of the games played in just a single month in the lichess.org open database, and the dataset available is absolutely huge! If you enjoy the game of Chess, or looking for a technical puzzle, check out the database and nerd out with any of the billions of chess games available! :)

Validate and Complete This Project!

Congratulations on finishing this DISCOVERY Data Science Project! 🎉🎉

To validate your entire project, your entire code will run from top-to-bottom on this page and each test case will be validated one final time. If everything looks good, you'll earn the card for completing this project: