Classify Static Image Using Deep Learning on Raspberry Pi
This example shows you how to generate and deploy code for an image classification algorithm using MATLAB® Support Package for Raspberry Pi® Hardware. The algorithm uses the ResNet-50 neural network to identify the image that is passed as an input using the command line of Raspberry Pi.
When you generate code for prediction the Raspberry Pi support package builds the executable on the hardware. Using ResNet-50, the executable classifies images into 1000 object categories, such as keyboard, coffee mug, pencil, and many animals. The executable predicts the object and outputs a label for it along with the associated probability of object category.
Note: You cannot generate and deploy deep learning code on Raspberry Pi hardware using macOS.
Required Hardware
Raspberry Pi hardware (Model 4 recommended)
USB cable
Ethernet cable
A monitor connected to the Raspberry Pi hardware and a HDMI cable (optional)
Prerequisite
Configure the Raspberry Pi network using the Hardware Setup screen. During the hardware setup process, ensure that you download the MathWorks® Raspbian image for deep learning. Instead of the MathWorks Raspbian image, if you choose to customize an existing image on your hardware, ensure that you select the option to install the ARM Compute Library.
Step 1: Connect the Raspberry Pi Hardware
Tip: Before you start this example, we recommend you to complete the Getting Started with MATLAB Support Package for Raspberry Pi Hardware and Send Inputs to MATLAB Function from Command Line of Raspberry Pi examples.
Connect the micro end of the USB cable to the Raspberry Pi and the regular end of the USB cable to the computer. Wait until the PWR LED on the hardware starts blinking.
In the MATLAB® Command Window, create a connection to the Raspberry Pi hardware.
r = raspi
r = raspi with properties: DeviceAddress: '172.31.164.186' Port: 18734 BoardName: 'Raspberry Pi Model B+' AvailableLEDs: {'led0'} AvailableDigitalPins: [4,5,6,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27] AvailableSPIChannels: {'CE0','CE1'} AvailableI2CBuses: {'i2c-1'} AvailableWebcams: {} I2CBusSpeed: 100000 Supported peripherals
Step 2: Open the Object Classification Algorithm
In this example, the raspi_fileRead_resnet
function loads the ResNet-50 deep learning neural network to classify the input image. The function resizes the image using OpenCV to fit the size expected by the network. It then classifies the image into object categories and outputs a label for object predictions along with the probabilities for each of the object categories.
For more information on the function, enter this command in the MATLAB Command Window.
type raspi_fileRead_resnet
function raspi_fileRead_resnet(fileName) %#codegen % Copyright 2020 The MathWorks, Inc. assert(all(size(fileName) <= 50)); assert(isa(fileName,'char')); %Initialize DNN and the input size net = coder.loadDeepLearningNetwork('resnet50'); inputSize = [224, 224,3]; %net.Layers(1).InputSize; %Read the image imgSizeAdjusted = readResizedImg(fileName,inputSize); %Classify the input image [label,score] = net.classify(imgSizeAdjusted); labelStr = cellstr(label); maxScore = max(score); outputStr = sprintf('Label : %s \nScore : %f',labelStr{:},maxScore); %Print label and score to stdout fprintf('%s\n',outputStr); end
You can modify the function by using the edit
command.
edit raspi_fileRead_resnet
Step 3: Generate C++ Code for Image Classification on Raspberry Pi
Create a hardware configuration object by using the targetHardware
function in the MATLAB Command Window.
board = targetHardware('Raspberry Pi');
Verify the DeviceAddress
, Username
, and Password
properties listed in the output. If required, change the value of the properties by using the dot notation syntax. For example, to change the device address to 172.18.186.123
, enter,
board.DeviceAddress = '172.18.186.123'
Set the CoderConfig.GenCodeOnly
property of the board object to true
. This enables the support package to generate and run the image classification algorithm code only on the host computer.
board.CoderConfig.TargetLang = 'C++';
board.CoderConfig.GenCodeOnly = true;
Create an arm-compute
deep learning configuration object. Specify the architecture of Raspberry Pi.
dlcfg = coder.DeepLearningConfig('arm-compute'); dlcfg.ArmArchitecture = 'armv7';
Execute this command to display the version of ARM Compute Library on Raspberry Pi.
r.system('strings $ARM_COMPUTELIB/lib/libarm_compute.so | grep arm_compute_versio | cut -d\ -f 1')
ans = 'arm_compute_version=v20.02.1 '
Copy the version number and assign it to the ArmComputeVersion
property of the deep learning configuration object.
dlcfg.ArmComputeVersion = '20.02.1'
dlcfg = ARMNEONConfig with properties: ArmComputeVersion: '20.02.1' ArmArchitecture: 'armv7' DataType: 'fp32' CalibrationResultFile: '' TargetLib: 'arm-compute'
Set the DeepLearningConfig
property of the code generation configuration object to the deep learning configuration object.
board.CoderConfig.DeepLearningConfig = dlcfg
board = targetHardware with properties: Name: 'Raspberry Pi' DeviceAddress: '172.31.164.186' Username: 'pi' Password: '*********' BuildDir: '/home/pi' EnableRunOnBoot: 0 BuildAction: 'Build, load, and run' CoderConfig: [1×1 coder.CodeConfig]
Deploy the function on the hardware by using the deploy
function. The deploy
function initiates code generation of the raspi_fileRead_resnet
function. Once code generation is complete, MATLAB generates a code generation report. Use this report to debug the function for any errors and warnings in the generated code.
After successfully generating the code, update the main.cpp
file to accept command line arguments.
deploy(board,'raspi_fileRead_resnet')
Code generation successful: View report
Step 4: Edit C++ Main File to Accept Input from Command Line of Raspberry Pi
This section explains how to modify the main.cpp
file to provide inputs using the command line of Raspberry Pi. A premodified main.cpp
file is available for you to use. You can choose to use the premodified file or to update the file manually using the steps in this section.
To use the premodified main.cpp
file, enter this command in the MATLAB Command Window.
copyfile('main.cpp',[pwd,'\codegen\exe\raspi_fileRead_resnet']) copyfile([pwd,'\codegen\exe\raspi_fileRead_resnet\examples\main.h'],[pwd,'\codegen\exe\raspi_fileRead_resnet'])
To update the file manually, copy the generated main.cpp
and main.h
files to one level higher than the current level in the file hierarchy. Open the main.cpp
file from the GENERATED CODE section of the code generation report and update the file by adding code at the points labeled in this code.
Step 5: Deploy the Function as a Standalone Executable on the Raspberry Pi
To generate and deploy the code on the hardware, set the CoderConfig.GenCodeOnly
property of the board
object to false
and then use the deploy
function. Since the deploy
function overwrites the main.cpp
file that you edited in the previous section, disable the creation of the file before deployment.
board.CoderConfig.CustomSource = ['codegen',filesep,'exe',filesep,'raspi_fileRead_resnet',filesep,'main.cpp']; board.CoderConfig.GenerateExampleMain = 'DoNotGenerate'; board.CoderConfig.GenCodeOnly = false
board = targetHardware with properties: Name: 'Raspberry Pi' DeviceAddress: '172.31.164.186' Username: 'pi' Password: '*********' BuildDir: '/home/pi' EnableRunOnBoot: 0 BuildAction: 'Build, load, and run' CoderConfig: [1×1 coder.CodeConfig]
deploy(board,'raspi_fileRead_resnet')
Deploying code. This may take a few minutes. Code generation successful: View report
The deploy
function initiates code generation of the raspi_fileRead_resnet
function.
After successfully generating the code, the support package loads the object classification algorithm as a standalone executable on the hardware. The location of the executable is displayed in the MATLAB Command Window. Note the location for later use.
Step 6: Copy Image File to the Raspberry Pi
After the code generation report has been successfully generated, click View report on the MATLAB Command Window.
On the MATLAB Coder Report Viewer window, on the Build Logs tab, copy the location of the generated elf.
Copy the image file to the hardware for prediction, using
putFile.
In this example, coffeemug.png
is the image input to the executable and the location of the executable is same as the one you noted in the previous step.
4. You can provide the input image of your choice and copy it to the location of the generated elf using the following syntax:
r.putfile('<input image>','<target location of the generated elf copied from the code generation report>')
In this example, coffeemug.png
is the image input to the executable and the location of the executable is /home/pi/MATLAB_ws_R2021a/C/Users/Documents/MATLAB/Examples/raspberrypiio-ex31623069
.
r.putFile('coffeemug.png','/home/pi/MATLAB_ws/R2021a/C/Users/Documents/MATLAB/Examples/raspberrypiio-ex31623069')
Step 7: Run the Executable Program to Get Prediction Scores
To open an SSH terminal with Raspberry Pi, use this command in the MATLAB Command Window.
openShell(r)
In the terminal, log in by entering the user name and password for the Raspberry Pi.
Change the current directory to the directory where the executable was saved.
cd /home/pi/MATLAB_ws/R2021a/C/Users/Documents/MATLAB/Examples/raspberrypiio-ex31623069
Run the executable on the Raspberry Pi by providing the coffee.png
image as an input to the executable.
sudo ./raspi_fileRead_resnet.elf coffeemug.png
The executable starts classifying the object in the image file, then it predicts the object and displays its label and the probability associated with the object category.