Project #9: FIFA Matches since 1872

In this DISCOVERY Data Science Project, you will do real data science in less than an hour and you will earn this project's card to your collection when you fully complete this project! 🎉

Data Source: FIFA International Soccer Match Data via GitHub

The Fédération Internationale de Football Association, or simply FIFA, is the largest governing organization of international football (soccer) competition, including the FIFA World Cup that occurs once every four years. The FIFA World Cup is likely the most popular sporting event in the world, drawing billions of viewers every tournament.

Mart Jürisoo (@martj42) maintains the @martj42/international_results GitHub repository that has collected over 45,000 "results of international football matches starting from the very first official match in 1872 [...]".

The matches range from FIFA World Cup to FIFI Wild Cup to regular friendly matches. The matches are strictly men's full internationals and the data does not include Olympic Games or matches where at least one of the teams was the nation's B-team, U-23 or a league select team.

In this project, you will find how to collect the raw URL for a GitHub-hosted dataset, group data using different variables on the dataset, and create visualizations highlighting over 150 years of FIFA matches! Let's nerd out! 🎉

Background Knowledge

To finish this project, we assume you already know:

Let's get started! :)

Part 1: Importing the FIFA Matches from GitHub

When a CSV file is hosted on a public GitHub repository, you can import the dataset directly into your notebook by using the raw-format URL. The "raw file" includes only the file's content, without anything added by GitHub. To find the raw file URL:

  1. Visit the @martj42/international_results here: https://github.com/martj42/international_results.
  2. Click the results.csv file.
  3. Click the Raw view on the top-right of the Github file viewer.
  4. Finally, use the URL of the page showing the raw CSV file as the URL for the dataset you're loading into the DataFrame named df.

In the following cell, import the dataset as a DataFrame into the Python variable named df by using the raw URL from GitHub:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

Data Cleaning: Remove Incomplete Data

The DataFrame you imported comes from a continuously-updated repository that has been updated over 200 times and contains over 49,000 FIFA matches. If a FIFA match was played yesterday, it may already be in the dataset! Occasionally, this dataset may contain incomplete rows of data that pandas represents as NaN or "Not a Number".

In this project, we will focus on all the games that we have complete data. To remove any incomplete rows, use df = df.dropna() to update df to a DataFrame that has dropped any rows with incomplete data. Update your dataset below:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

Organizing Data: Adding Columns year

In the DataFrame, a date column provides an ISO-formatted date string. However, it will be useful to also have a column containing only the year.

Run the following code to create a new column called year, which contains the first four characters of the date string as a numeric value (ex: "1872-11-30" is recorded as the numeric year 1872):

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)
⚙️ Test Case: Part 1: Importing the FIFA Matches from GitHub

Part 2: Calculating the Total Number of Goals Scored for Every Country

In an international FIFA match, teams representing two countries play a game of football (soccer) where a few points are usually scored. The dataset organizes the score of the match with columns home_team and home_score for the "home team" and away_team and away_score for the "away (visiting) team".

Each observation in our dataset is a single game. The first bit of exploratory data analysis (EDA) that would be interesting is to find the total number of goals scored by each country. To do this, we'll need to:

  • Find the number of goals scored by each country as the "home team",
  • Find the number of goals scored by each country as the "away team", and
  • Merge the two DataFrames together to combine the goals they scored while "home" or "away".

Part 2.1: Finding the Goals Scored for each "Home Team"

Create a new DataFrame named df_home that summarizes the entire DataFrame by each unique home_team and calculating the total number of home goals for each country. It is very useful to use numeric_only=True to make sure you sum only the numeric columns.

(See the DISCOVERY lecture "Grouping Data in Python" for a refresher on how to group data.)

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

In df_home, the values for any column except the country (home_team) and score (home_score) are meaningless. Remove any extra columns to ensure your DataFrame contains only the country and score:

(See the DISCOVERY guide "Removing Columns in a DataFrame" for a refresher on how to remove columns from a DataFrame.)

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)
⚙️ Test Case: Part 2.1: Finding the Goals Scored for each "Home Team"

Part 2.2: Finding the Goals Scored for each "Away Team"

How about for away goals? Do the same thing as you did in 2.1, but for away_team and away_score. Call this DataFrame df_away:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

Similar to what you did above, ensure sure you only have data for the columns about the country (away_team) and score (away_score). Remove any extra columns to ensure your DataFrame contains only the country and score:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)
⚙️ Test Case: Part 2.2: Finding the Goals Scored for each "Away Team"

Part 2.3: Setting Up the DataFrames To Merge

To merge two DataFrames together, the simplest technique is to set up two DataFrames where one column has an identical name in both DataFrames. Currently, in your DataFrame df_home, each country is in the column name home_team; in df_away, each country is in away_team.

To rename a column in a DataFrame, the rename() function allows you to specify the old and new column names in a Python dictionary. Specifically, the syntax to rename the home_team column to team is:

df_home = df_home.rename(columns = {"home_team": "team"})
#                                    ^^^^^^^^^    ^^^^
#                           (old column name)^    ^(new column name)

Using df_home.rename(...), rename your home_team column to team:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

Repeat the process for df_away, making sure away_team is renamed to team:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

Part 2.4: Merge Two DataFrames Into One!

You're all set up to merge two DataFrames together! You have two DataFrames which have exactly one column with an identical name -- both df_home and df_away have the column named team.

Read the DISCOVERY guide "Combining DataFrames by Merging" to learn how to merge two DataFrames together using pd.merge and create a merged DataFrame called df_scores.

  • A few countries have never played a home game, and a few other countries have never played an away game! This means that some teams do not exist in both df_home and df_away!
  • Make certain to keep all unmatched rows by reading the "Keeping Unmatched Rows" section in the guide linked above!
Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)
⚙️ Test Case: Part 2.4: Merge Two DataFrames Into One

Part 2.5: Fixing Missing Data

When merging your DataFrames using an "outer" join, some teams will have only played home games while other teams will have only played away games. Let's look at cells that do not have either a home_score or an away_score:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

As a data scientist, we have to make a decision each time we see missing data on how to best handle missing data in each dataset:

  1. Many times, missing data occurs when an observation is not collected and should be left as missing data. Python will skip over missing data when calculating statistics like the mean(), sum(), std(), etc.
  2. Other times, missing data indicates that there is no data. In cases where we're certain of this, we can replace the missing NaN markers with the correct numeric value indicating there's no data (ex: 0).
  3. Finally, sometimes missing data indicates a problem with the data, and the data can be excluded from the analysis as an incomplete record.

In our example, we know missing data appears in df_scores when a team has never played either a home or an away game. The correct value for these cells is that missing cells indicate that zero points have been scored. The DataFrame function fillna(...) will fill all rows with missing data with the value you specify. For example:

# Fills all missing values in the DataFrame `df` with the value 100.
df = df.fillna(100)
df

Update your df_scores DataFrame and fill in all missing values with the number 0 to indicate that they have scored zero points in either home or away games:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)
⚙️ Test Case: Part 2.5: Fixing Missing Data

Part 2.6: Creating a Data Visualization

Let's discover what country scored the most goals across the 45,000 FIFA matches! First, we'll want to organize the dataset by the total number of goals. Add a new column to your DataFrame df_scores called total_score that adds up the home_score and away_score for each team:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

Using df_scores, create a stacked bar chart that stacks the values of home_score and away_score on top of each other. To do this, use df_scores.plot.bar(...) with the following function options:

  • The parameter x sets the column of data to use for the x-axis data; set x to be the column name that contains the name of each country,
  • The parameter y sets the list of columns to use for y-axis data; set y equal to the list ["home_score", "away_score"].
  • The parameter stacked must be set to True to indicate a stacked bar chart (stacked=True).
  • The parameter title set to an informative title (ex: "Total Points Scored in FIFA International Matches"), or similar.

In the cell below, set your visualization equal to ax so we can also access it in the test case:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

Visualization Updates

That's a lot of teams!!

Modify your code above to display only the top 15 teams based on the value in total_score. This can be done by adding nlargest(15,"total_score") into your line of visualization code above. For example:

# Full Dataset
df_scores.plot.bar( ... )

# Largest Rows
df_scores.nlargest(15, "total_score").plot.bar( ... )
#         ^^^^^^^^^^^^^^^^^^^^^^^^^^^

Update your visualization in your code cell above! :)

⚙️ Test Case: Part 2.6: Creating a Data Visualization

Analysis

With this data visualization, there are several interesting questions we can ask about over 45,000 FIFA games:

  • What team scored the most points overall?
  • Did that team also score the most points at home, or did a different team do better in their home games?
  • Most teams seem to score more points at home than away. Is there any team that scored more points at away games than home games?

Part 3: FIFA World Cup

Returning to the full dataset df that contains one observation for each match played, the tournament column in df provides the tournament where the game was played.

World Cup games were the only international matches played on Nov. 30, 2022 (2022-11-30). Find the tournament string used for games during the World Cup by looking at games with that date during the World Cup:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

Part 3.1: Finding World Cup Games

Take note of the label used for World Cup games in this dataset. Now knowing how the World Cup games are labeled, create a new DataFrame df_worldcup that contains all World Cup games played in the dataset:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)
⚙️ Test Case: Part 3.1: World Cup Games

Part 3.2: Grouping Data by Year

Create a summary of all the World Cup games that calculates the average home_score and away_score each year and store this DataFrame in a variable called df_worldcup_years.

(See the DISCOVERY lecture "Grouping Data in Python" for a refresher on how to group data.)

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)
⚙️ Test Case: Part 3.2: Grouping Data by Year

Part 3.3: Creating a Multi-Line Data Visualization

Using df_worldcup_years, create a multi-line line chart that graphs both the home_score and away_score in a single line chart. To do this, use df_worldcup_years.plot.line(...). This is very similar to having two different variables visualized using a bar chart. Create a graph plot using the following options:

  • The parameter x sets the column of data to use for the x-axis data; set x to be the column name that contains the year,
  • The parameter y sets the list of columns to use for y-axis data; set y equal to the list ["home_score", "away_score"].
  • The parameter title set to an informative title (ex: "Average Points Scored in All World Cup Games Each Tournament"), or similar.

In the cell below, set your visualization equal to ax again so we can access it in the test case:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)
⚙️ Test Case: Part 3.3: Creating a Multi-Line Data Visualization

Analysis

In this visualization, you learned how to plot multiple variables on a single data visualization! In general, what is the trend of points scored per game in World Cup matches? (Are matches played today scoring more points, on average, than 80 years ago?)

Note that almost every World Cup match is a "Neutral" site game, so the "home_score" and "away_score" do not have much context when looking only at World Cup matches.

Validate and Complete This Project!

Congratulations on finishing this DISCOVERY Data Science Project! 🎉🎉

To validate your entire project, your entire code will run from top-to-bottom on this page and each test case will be validated one final time. If everything looks good, you'll earn the card for completing this project: