Skip to main content

Robotics: Vision Intelligence and Machine Learning

Provided by University of Pennsylvania (PennX)
8–10 hours
per week, for 12 weeks
Free

$349 USD for graded exams and assignments, plus a certificate

Learn how to design robot vision systems that avoid collisions, safely work with humans and understand their environment.

Before you start

  • College-level introductory linear algebra (vector spaces, linear systems, matrix decomposition)
  • College-level introductory calculus (partial derivatives, function gradients)
  • Basic knowledge of computer programming (variables, functions, control flow) is preferred, but students may also choose to learn it on their own. The class projects will be carried out MATLAB/Python, with C++ as an option.
Course opens: Jul 10, 2017
Course ends: Feb 2, 2020

What you will learn

  • The fundamentals of image filtering and tracking, and how to apply those principles to face detection, mosaicking and stabilization
  • How to use geometric transformations to determine 3D poses from 2D images for augmented reality tasks and visual odometry for robot localization
  • How to recognize objects and the basics of visual learning and neural networks for the purpose of classification

Week 1: Camera Geometry and Color Sensing

Week 2: Fourier Transforms, Image Convolution, Edge Detection
Week 3: Image Convolution and Edge Detection Part 2, Image Pyramids
Week 4: Feature Detection: Filters, SIFT, HOG
Week 5: Geometrical Transformation, Affine, Protective and Ransac
Week 6: Optical Flow Estimation
Week 7: Image Morphing
Week 8: Image Blending
Week 9: Image Carving
Week 10: Probability and Statistics, Regression and Classification
Week 11: SVM and Object Recognition
Week 12: Convolutional Neural Network

Overview

How do robots “see”, respond to and learn from their interactions with the world around them? This is the fascinating field of visual intelligence and machine learning. Visual intelligence allows a robot to “sense” and “recognize” the surrounding environment. It also enables a robot to “learn” from the memory of past experiences by extracting patterns in visual signals.

You will understand how Machine Learning extracts statistically meaningful patterns in data that support classification, regression and clustering. Then by studying Computer Vision and Machine Learning together you will be able to build recognition algorithms that can learn from data and adapt to new environments.

By the end of this course, part of the Robotics MicroMasters program, you will be able to program vision capabilities for a robot such as robot localization as well as object recognition using machine learning.

Projects in this course will utilize MATLAB and OpenCV and will include real examples of video stabilization, recognition of 3D objects, coding a classifier for objects, building a perceptron, and designing a convolutional neural network (CNN) using one of the standard CNN frameworks.

Meet your instructors

Jianbo Shi
Professor, Computer and Information Science, School of Engineering and Applied Science
University of Pennsylvania
Kostas Daniilidis
Professor, Computer and Information Science, School of Engineering and Applied Science
University of Pennsylvania
Dan Lee
Professor, Computer and Information Science and Electrical and Systems Engineering, School of Engineering and Applied Science
University of Pennsylvania

Who can take this course?

Unfortunately, learners from one or more of the following countries or regions will not be able to register for this course: Iran, Cuba and the Crimea region of Ukraine. While edX has sought licenses from the U.S. Office of Foreign Assets Control (OFAC) to offer our courses to learners in these countries and regions, the licenses we have received are not broad enough to allow us to offer this course in all locations. EdX truly regrets that U.S. sanctions prevent us from offering all of our courses to everyone, no matter where they live.

View Courses
This course is part of:

Earn a MicroMasters® Program Certificate in 1 year if courses are taken one at a time.

View the program
  1. 96–120 hours of effort

    Master the foundational math concepts that drive robotics and put them into practice using MATLAB.

  2. Robotics: Vision Intelligence and Machine Learning
  3. 96–120 hours of effort

    Learn how to design and engineer complex, dynamic robotic systems.

  4. 96–120 hours of effort

    Learn how to design, build, and program dynamical, legged robots that can operate in the real world.

Get started in computer science

Browse over 600 computer science courses
Of all edX learners:
73% are employed
Of all edX learners:
45% have children
Based on internal survey results
419,621 people are learning on edX today