Validate and Accelerate Video Algorithm with Cosimulation and FPGA-in-the-Loop

This example shows how to validate a video processing algorithm using cosimulation and accelerate this process using FPGA-in-the-loop (FIL). The process analyzes a simple system that sharpens an RGB video input at 24 frames per second (FPS). You enhance the RGB video feed using an FIR filter. Then, to validate the design, you leverage simulator-based and hardware-based verification features provided with HDL Verifier™ Toolbox.

Requirements and Prerequisites

• FPGA design software (AMD® Vivado® design suite or Intel® Quartus® Prime design software).

• One of the supported FPGA development boards and accessories. For more information about supported hardware, see Supported FPGA Devices for FPGA Verification.

• For connection using Ethernet: Gigabit Ethernet Adapter installed on host computer, Gigabit Ethernet crossover cable.

• For connection using JTAG: USB Blaster I or II cable and driver for Intel FPGA boards. JTAG cable and driver for AMD FPGA boards.

• For connection using PCI Express®: FPGA board installed into PCI Express slot of host computer.

• Vivado simulator from AMD.

MATLAB® and FPGA design software can either be locally installed on your computer or on a network accessible device. If you use software from the network, you will need a second network adapter installed in your computer to provide a private network to the FPGA development board. Consult the hardware and networking guides for your computer to learn how to install the network adapter.

Before using cosimulation or FIL, make sure your system environment is set up properly for accessing FPGA design software. You can use the hdlsetuptoolpath (HDL Coder) function to add AMD Vivado or Intel Quartus Prime to the system path for the current MATLAB session.

Open and Execute Simulink Model

Open the sim_videosharp model and run the simulation.

The simulation is not fluent due to the large quantity of data to process. The frame rate is 1.70 FPS. The performance can vary depending on the host machine. You improve the simulation speed later in the example with FIL.

Generate HDL Code

To generate HDL code for the Streaming 2-D FIR Filter subsystem, follow these steps.

a. Right-click on the block labeled DUT.

b. Select HDL Code > Generate HDL for Subsystem in the context menu.

Alternatively, you can generate HDL code by entering the following command in MATLAB.

makehdl('sim_videosharp/Streaming 2-D FIR Filter ')

You can also use the pre-generated HDL files located in the videosharp_hdlsrc folder.

Set Up FPGA Design Software

For AMD FPGA boards, run

hdlsetuptoolpath('ToolName','Xilinx Vivado','ToolPath','C:\Xilinx\Vivado\2023.1\bin');

This example assumes that the AMD Vivado executable is located in C:\Xilinx\Vivado\2023.1\bin. Substitute with the actual installation location if needed.

For Intel boards, run

hdlsetuptoolpath('ToolName','Altera Quartus II','ToolPath','C:\Intel\quartus\22.4\bin64');

This example assumes that the Intel Quartus Prime executable is located in C:\Intel\quartus\22.4\bin64. Substitute with the actual installation location if needed.

Validate Algorithm Using Cosimulation

This example provides the Framed_cosim_videosharp.slx template for cosimulation. It replaces the design under test (DUT) with the generated cosimulation block.

To generate a cosimulation block, use the Cosimulation Wizard tool.

In the Input/Output Ports pane, set Port Type for the input and output ports of your module as shown in this figure.

In the Clock/Reset Details pane, set the clock and reset parameter settings as shown in this figure.

For the step-by-step process of generating cosimulation block, and replacing DUT with this generated block, see Get Started with Simulink HDL Cosimulation.

Open the Framed_cosim_videosharp.slx model and replace the Framed_cosim_videosharp/Streaming 2-D FIR Filter/DUT subsystem with the cosimulation block you generated in the previous step. To set the cosimulation start time settings, double-click on the Streaming 2-D FIR Filter /Streaming_2_D_FIR_Filter subsystem block and navigate to the Clocks, Resets and Enables tab. Change the timing details as shown in this figure.

Now close the dialog box and run the model.

You see that the cosimulation block sharpens the video and this validates that your HDL files work same as your algorithm.

Accelerate Process with FIL

To generate the FIL block, use the FPGA-in-the-Loop Wizard tool. To generate the FIL block, follow steps 1 to 8 in the Block Generation with the FIL Wizard example. Note that the top-level HDL module for this example is Streaming 2-D FIR Filter.v.

After you complete these steps, you see that a FIL block named Streaming_2_D_FIR_Filter is generated in a new model. To complete the Simulink model and run FIL simulation, follow these steps.

Open and Complete Simulink Model for FIL

a. Open the FIL_videosharp.slx model.

b. Copy the previously generated FIL block to the FIL_videosharp.slx model in place of the Streaming 2-D FIR Filter/HDL DUT subsystem, where it says "Replace this with FIL block".

Configure FIL Block

a. Double-click the FIL block in the FIL_videosharp.slx model to open the block mask.

b. Click Load.

c. Click OK to close the block mask.

Run FIL Simulation

Run the simulation and observe the performance improvement.

You observe that frame rate is approximately 10.30 FPS, increased by almost 6 times as compared to sim_videosharp. The performance depends upon the interface and the hardware board. This example uses Ethernet interface and AMD Kintex®-7 FPGA KC705 Evaluation Kit.

Conclusion

In this example, you develop a Simulink model to process the RGB video input and enhance its quality. The inclusion of the cosimulation block validates the design and the inclusion of the FIL block accelerates the process. The frame rate increases by approximately 6 times, which is a significant improvement for any video processing algorithm dealing with large data. This performance can vary depending upon the interface and hardware board that you use.