Video Processing with MATLAB

Applications, examples, and techniques

Common Applications

Video applications present common but difficult challenges that require flexible analysis and processing functionality. Using MATLAB and Simulink products, you can develop solutions to common video processing challenges such as video stabilization, video mosaicking, target detection, and tracking.

Object Tracking

Object tracking is an essential part of many applications including pedestrian avoidance, security and surveillance, and augmented reality. This examples shows motion-based tracking of moving people in a video from a stationary camera.

Example: Motion-based multiple object tracking

Example: Motion-based multiple object tracking

Object Detection and Counting

Video processing can be used to detect and count objects that move in video sequences. In this case study, scientists in Australia are using video footage to estimate the wildlife population of waterbirds.

Advancing Wildlife Research: The Development of a Solution to Process Video Footage of Waterbirds

Case study: Advancing Wildlife Research: The Development of a Solution to Process Video Footage of Waterbirds

MATLAB Conference 2017 Australia Dr. Shannon Dundas, NSW Department of Primary Industries

Video Processing in MATLAB

MATLAB provides tools and algorithms that let you view, analyze, read, and write videos. Video processing can be useful in applications like:

Video Processing in 4 Easy Steps

Video processing in MATLAB involves the following steps:

  1. Reading the video
  2. Displaying the video
  3. Processing the video
  4. Writing the video

Step 1. Reading the Video

You can read video from files or directly from cameras.

A single MATLAB command lets you read in videos from a file:

>> vid = VideoReader('filename.avi')

MATLAB supports webcams for video processing, while Image Acquisition Toolbox enables live acquisition from many industrial and scientific cameras.

MATLAB lets you read video files using a variety of codecs including OS-specific codecs for Microsoft® Windows®, Mac, and Linux®.

Step 2. Displaying the Video

There are two methods for displaying video in MATLAB:

video-viewer-app

The Video Viewer app, which plays MATLAB movies, videos, or image sequences. The app lets you start, stop, and play video at different speeds, and jump to a section of the video.

Step 3. Processing the Video

A video is a sequence of individual video frames, or images. This means an algorithm designed to perform edge detection on an image can be quickly converted to perform edge detection on a video.

Read single image

 Read image frame from video

current_image = imread('flowers.png');
edge(current_image);

current_image = readFrame(vid);
edge(current_image);

Video processing can be very simple, as in the example using edge detection, or significantly more complex, such as tracking algorithms that must account for an object’s location in previous frames.

For more information on advanced video processing, see examples for:

Step 4. Writing the Video

After processing, you can write each frame of a video back to a file. You can create a video file with the function:

>> vid_w = VideoWriter('newfile.avi');
>> open(vid_w)

The variable vid_w can accumulate new frames to create a video.

A Complete MATLAB Example

Putting all the components together, let’s run through a complete example to show the steps of reading, displaying, processing, and writing video:

%% Read and process a video into MATLAB
 
% Setup: create Video Reader and Writer
videoFileReader = VideoReader('tilted_face.avi');
myVideo = VideoWriter('myFile.avi');

% Setup: create deployable video player and face detector
depVideoPlayer = vision.DeployableVideoPlayer;
faceDetector = vision.CascadeObjectDetector();
open(myVideo);

%% Detect faces in each frame
while hasFrame(videoFileReader)

	% read video frame
	videoFrame = readFrame(videoFileReader);
	% process frame
bbox = faceDetector(videoFrame);
videoFrame = insertShape(videoFrame, 'Rectangle', bbox); 

	% Display video frame to screen
	depVideoPlayer(videoFrame);

	% Write frame to final video file
	writeVideo(myVideo, videoFrame);
	pause(1/videoFileReader.FrameRate);

end
close(myVideo)

You can download this code on MATLAB Central.

Advanced Techniques

Video Processing Algorithms for Computer Vision

MATLAB algorithms that use temporal correlation for video processing are based on the concept of “state,” the idea that the algorithm is working on a current video frame but also uses previous frames to determine its output. This is crucial for object tracking algorithms, which rely on prior information to inform future action. A common example of tracking is the KLT algorithm, which tracks individual points in an object to keep track of an object’s location.

Developers of video processing algorithms can also use the vision-specific algorithms in Computer Vision System Toolbox. The algorithms let you read and view high-resolution videos in a rapid and memory-efficient way. The toolbox also includes algorithms for 3D point cloud processing, stereo vision, object detection, tracking and recognition, and other applications.

Learn More About Video Processing

Generate CUDA Code for a Fog Rectification Algorithm

Use GPU Coder to generate CUDA code from a fog rectification algorithm written in MATLAB.

Vision Processing for FPGA (5 videos)

Learn the considerations, workflow, and techniques for targeting a vision processing algorithm to FPGA hardware

Deep Learning for Computer Vision

Learn how MATLAB addresses common challenges encountered while developing object recognition systems and see new capabilities for deep learning, machine learning, and computer vision.

30-Day Free Trial

Get started