Project #3: Highest Mountains in the World

In this DISCOVERY Data Science Project, you will do real data science in less than an hour and you will earn this project's card to your collection when you fully complete this project! 🎉

Data Source: Wikipedia's "List of mountains by elevation"

Wikipedia is an ad-free, open source, and very reliable source of information about almost every topic you can imagine! In this project, you will explore how to easily import data from any Wikipedia tables into a DataFrame.

The Wikipedia article "List of mountains by elevation" (https://en.wikipedia.org/wiki/List_of_mountains_by_elevation) contains information on hundreds of mountains -- including Mount Everest (tallest in the world), Denali / Mount McKinley (tallest in the United States), and hundreds more!

  • Click the Wikipedia link above to view how the Wikipedia page looks in your web browser before we begin to work with it in Python!

Once you've taken a look at the data, let's nerd out with gathering data from Wikipedia!

Background Knowledge

To finish this project, we assume you already know how to:

With that knowledge, this project will guide you through nerding out with gathering data from Wikipedia and finding some facts about the tallest mountains in the world. Let's get started! :)

Part 1: Fetching Data From Wikipedia

The Wikipedia article "List of mountains by elevation" is organized into several tables of data:

  • One table for mountains that are at least 8,000m in height,
  • Another table for 7,000m mountains (7,000m - 7,999m),
  • Another table for 6,000m mountains (6,000m - 6,999m),
  • ...and so on...

The pd.read_html(...) function in the pandas library is designed to read data from tables found in webpages.

Part 1.1: Using pd.read_html

In the following cell, we'll use pd.read_html(...) to read all of the tables from the Wikipedia page "List of mountains by elevation" (https://en.wikipedia.org/wiki/List_of_mountains_by_elevation). Here's a brief overview of the pd.read_html function:

  • The read_html function is very similar to the commonly used read_csv.
  • Instead of returning a DataFrame from a CSV file, the read_html returns one DataFrame for each table on the website as a Python list of DataFrames. (This means we'll have 8 different DataFrames when there are eight different tables on the Wikipedia page.)
  • Just like read_csv, you only need to provide the URL of the data or an HTML file that you've downloaded! 🎉

Using an HTML file vs. Using a URL

Historically, many sites could be easily and automatically downloaded with Python and you can use the URL inside of pd.read_html. However, with the growth of AI agents scraping websites, many sites (including Wikipedia) are adding security to prevent automatic "scraping" of data. This prevents us from having Python directly download the website and extract the data.

To use data from a site, you can save a copy of the webpage as an HTML file. We have done that for the Wikipedia page and provided it for you as part of this project. The file is located at https://waf-server-01.cs.illinois.edu/static/List-of-mountains-by-elevation_Wikipedia.html.

In the cell below, load the HTML page using the URL above and fetch a list of DataFrames. Each DataFrame is a table from the Wikipedia page "List of mountains by elevation" and you will store those in the Python variable dfList:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

🔬 Checkpoint Tests 🔬

⚙️ Test Case: Part 1.1: Using pd.read_html

Part 1.2: Exploring the List of DataFrames

Your variable dfList contains a Python list of several DataFrames, one DataFrame for each table on the webpage. To use this as one complete dataset, we need to join the lists together into one large DataFrame.

Before we do that, let's explore the individual DataFrames. To look at the first item a list, we access the 0th index of the list by using the Python code:

# Accesses the first element (index 0) of a list called `myList`
myList[0]

Applying this to the variable dfList, the following code displays the first DataFrame stored in dfList. This first DataFrame contains the data from the first table on the Wikipedia page:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

The second DataFrame is accessed at index [1]:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

Continue to look at each index, until you find the very last DataFrame in the list that contains data about the mountains. (We'll need to know the last index for the next section.)

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

Part 1.3: Joining the individual DataFrames into one large DataFrame

Before we can do analysis on the whole dataset, we need to join the individual DataFrames together into one large DataFrame. When we join DataFrames end-to-end, where the last row of the previous DataFrame is followed by the first row of the next DataFrame, the operation is called concatenation.

Read the DISCOVERY guide to learn the syntax on "Combining DataFrames by Concatenation"

Use concatenation to create a single DataFrame df that contains data about every mountain found on the Wikipedia page:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

🔬 Checkpoint Tests 🔬

⚙️ Test Case: Part 1.3: Joining the individual DataFrames into one large DataFrame

Part 2: Mountains in the United States

Now that we have every mountain in a single DataFrame, we can do some analysis!

In the dataset, the Location and Notes column contains a human-written description of the location and other notes. For example, the notes about the mountain "Makalu" notes that the mountain is in "Nepal".

To do the next analysis, we want to select from all the mountains in the entire dataset df and find only the mountains located in the United States.

To do this, you'll need to do two things:

  1. First, look back at the Wikipedia page, or explore df here in Python, to find out all the different ways mountains in the United States might be labeled. (Hint: There's two different ways, take a look through the "Location and notes" column data!)
  2. Second, read the DISCOVERY guide to learn the syntax on "Selecting DataFrame Rows Based on String Contents" to identify how we can use the two different ways the United States is labeled:

Create a DataFrame of only the mountains in the United States and store it in the variable df_us in the cell below.

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

Part 2 Analysis: Percentage of Mountains in the Dataset in the United States?

What percentage of mountains in the entire dataset are found in the United States?

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

🔬 Checkpoint Tests 🔬

⚙️ Test Case: Part 2: Mountains in the United States

Part 3: Higher than the Highest Mountain in the United States

You have identified the highest mountains in the United States, and also have the data for the highest mountains across the world! 🎉

In the final puzzle for this project, create a DataFrame that contains ALL of the mountains that have a height that is higher than the highest mountain in the United States. Store the DataFrame in the Python variable df_higherThanUS:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

Part 3 Visualization: Heights of Various Mountains

A bar chart is a great way to visualize data that contains non-numeric data or categories. In this project, you have explored the height of various mountains -- let's visualize just how tall Mount Everest compared to other mountains in this dataset.

Since there are over 1,000 mountains in the dataset, our visualization will show a subset of all the mountains. Specifically, we'll visualize every 49th mountain -- indexes [0], [49], [98], [147], [196], etc.

Selecting only every 49th mountain can be done by selecting a range from your DataFrame with the following format:

# Selects every 49th row, starting with [0]:
df[::49]

Creating a bar chart from a DataFrame is done by using the general format:

# Generic format for a bar chart from a DataFrame:
df.plot.bar(x="data-column-name", y="data-column-name")

Combining these together, your bar chart can be created with the general format:

# Generic format for a bar chart, with every 49th row of a DataFrame:
df[::49].plot.bar(x="data-column-name", y="data-column-name")

Create Your Visualization

Create a bar chart below, using the mountain name for your x-axis data and the height (either feet or meters, your choice!) for the y-axis data:

Reset Code Run All to Here Python Output:
(Run your code to see your code result's here.)

🔬 Checkpoint Tests 🔬

⚙️ Test Case: Part 3: Higher than the Highest Mountain in the United States

Validate and Complete This Project!

Congratulations on finishing this DISCOVERY Data Science Project! 🎉🎉

To validate your entire project, your entire code will run from top-to-bottom on this page and each test case will be validated one final time. If everything looks good, you'll earn the card for completing this project: