Building a Scene Recognition Model from Video Frames

MicroProject Overview

Visual images are an important part of all media and Data Scientists are often using images as data sources. In this MicroProject, you will create a simple model to detect the amount of time spent in two different "scenes" we used when creating office-hour style videos for Data Science DISCOVERY.

Center for Innovation in Teaching and Learning

This MicroProject was inspired by a podcast that we recently recorded with the team from the Center for Innovation in Teaching and Learning who helped produce our video. To learn the background and hear from Karle and Wade about the journey of creating DISCOVERY, go over and listen to our episode on the "Teach Talk Listen Learn Podcast" where talk with TTLL host Bob Dignan and our CITL video producer Eric Schumacher: https://citl.illinois.edu/citl-101/teaching-learning/teach-talk-listen-learn

In addition, check out the Center for Innovation in Teaching and Learning at The University of Illinois to find all about what they do! :)

Data Science Skills

  • Using sklearn to load images
  • Finding the average pixel color of an image
  • Creating a simple classification function based on the average pixel color of an image
  • Iterating through building a classification function for multiple images

Let's nerd out!

MicroProject in Visual Studio Code
MicroProject in Visual Studio Code
Reading an image using skimage
Reading an image using skimage
Visual summary of classification results
Visual summary of classification results
Results of the classification function on Frame #0001
Results of the classification function on Frame #0001

First Time Doing a MicroProject?

Each MicroProject starts with a notebook that we provide to you to get started! You will need to configure a git repository to connect to our `microprojects` remote where we release the starter notebook.

Fetch the Initial Files

In your terminal, navigate to your GitHub repository and merge the initial files by running the following commands:

git fetch microprojects
git merge microprojects/microproject-video-frame-scene-recognition-model --allow-unrelated-histories -m "Merging initial files"

Complete the Notebook

If the commands above were successful, you have merged in the initial files to start on the MicroProject.

  • Find the new microproject-video-frame-scene-recognition-model folder.
  • Open microproject-video-frame-scene-recognition-model.ipynb and complete the MicroProject!

Commit and Grade Your Notebook

Once you have finished your notebook, you must use the built-in GitHub Action to preform automated grading of your MicroProject notebook! You will need to commit your work and then manually run the GitHub Action.

Commit Your Work

To commit your notebook, run the standard git commands in your terminal:

git add -u
git commit -m "microproject completed"
git push

Grade Your Notebook

To grade your notebook, you will need to visit your GitHub repository in your browser.

  • Visit your GitHub repository in your browser
  • Click on the "Actions" tab
  • Under "Workflows", find the workflow for this microproject
  • Click the "Run Workflow" in the blue box, and then the green "Run Workflow"
  • After about 10 seconds, you should see a new job that has started running
    • You can click on the job to watch it run in real-time
    • It will take ~1 minute to run and grade
  • Once the running is complete, the autograding summary will be available!