Design Video Processing Algorithms for HDL in Simulink

This example uses:

This example shows how to design a hardware-targeted image filter using Vision HDL Toolbox™ blocks. It also uses Computer Vision Toolbox™ blocks.

The key features of a model for hardware-targeted video processing in Simulink® are:

Streaming pixel interface: Blocks in Vision HDL Toolbox use a streaming pixel interface. Serial processing is efficient for hardware designs, because less memory is required to store pixel data for computation. The serial interface allows the block to operate independently of image size and format and makes the design more resilient to video timing errors. For further information, see Streaming Pixel Interface (Vision HDL Toolbox).
Subsystem targeted for HDL code generation: Design a hardware-friendly pixel-streaming video processing model by selecting blocks from the Vision HDL Toolbox libraries. The part of the design targeted for HDL code generation must be in a separate subsystem.
Conversion to frame-based video: For verification, you can display frame-based video or compare the result of your hardware-compatible design with the output of a Simulink behavioral model. Vision HDL Toolbox provides a block that allows you to deserialize the output of your design.

Open Model Template

This tutorial uses a Simulink model template to get started.

Click the Simulink button, or type simulink at the MATLAB® command prompt. On the Simulink start page, find the Vision HDL Toolbox section, and click the Basic Model template.

The template creates a new model that you can customize. Save the model with a new name.

Import Data

The template includes a Video Source block that contains a 240p video sample. Each pixel is a scalar uint8 value representing intensity. A best practice is to design and debug your design using a small frame size for quick debug cycles, before scaling up to larger image sizes. You can use this 240p source to debug a design targeted for 1080p video.

Serialize Data

The Frame To Pixels block converts framed video to a stream of pixels and control structures. This block provides input for a subsystem targeted for HDL code generation, but it does not itself support HDL code generation.

The template includes an instance of this block. To simulate with a standard video format, choose a predefined video padding format to match your input source. To simulate with a custom-size image, choose the dimensions of the inactive regions that you want to surround the image with. This tutorial uses a standard video format.

Open the Frame To Pixels block dialog box to view the settings. The source video is in 240p grayscale format. A scalar integer represents the intensity value of each pixel. To match the input video, set Number of components to 1, and the Video format to 240p.

Note: The sample time of the video source must match the total number of pixels in the frame size you select in the Frame To Pixels block. Set the sample time to Total pixels per line × Total lines. In the InitFcn callback, the template creates a workspace variable, totalPixels, for the sample time of a 240p frame.

Design HDL-Compatible Model

Design a subsystem targeted for HDL code generation, by modifying the HDL Algorithm subsystem. The subsystem input and output ports use the streaming pixel format described in the previous section. Open the HDL Algorithm subsystem to edit it.

In the Simulink Library Browser, click Vision HDL Toolbox. You can also open this library by typing visionhdllib at the MATLAB command prompt.

Select an image processing block. This example uses the Image Filter (Vision HDL Toolbox) block from the Filtering sublibrary. You can also access this library by typing visionhdlfilter at the MATLAB command prompt. Add the Image Filter block to the HDL Algorithm subsystem and connect the ports.

Open Image Filter block and make the following changes:

Set Filter coefficients to ones(4,4)/16 to implement a 4×4 blur operation.
Set Padding method to Symmetric.
Set Line buffer size to a power of 2 that accommodates the active line size of the largest required frame format. This parameter does not affect simulation speed, so it does not need to be reduced when simulating with a small test image. The default, 2048, accommodates 1080p video format.
On the Data Types tab, under Data Type, set Coefficients to fixdt(0,1,4).

Design Behavioral Model

You can visually or mathematically compare your HDL-targeted design with a behavioral model to verify the hardware design and monitor quantization error. The template includes a Behavioral Model subsystem with frame-based input and output ports for this purpose. Double-click on the Behavioral Model to edit it.

For this tutorial, add the 2-D FIR Filter (Computer Vision Toolbox) block from Computer Vision Toolbox™. This block filters the entire frame at once.

Open the 2-D FIR Filter block and make the following changes to match the configuration of the Image Filter block from Vision HDL Toolbox:

Set Coefficients to ones(4,4)/16 to implement a 4×4 blur operation.
Set Padding options to Symmetric.
On the Data Types tab, under Data Type, set Coefficients to fixdt(0,2,4).

Deserialize Filtered Pixel Stream

Use the Pixels To Frame block included in the template to deserialize the data for display.

Open the Pixels To Frame block. Set the image dimension properties to match the input video and the settings you specified in the Frame To Pixels block. For this tutorial, the Number of components is set to 1 and the Video format is set to 240p. The block converts the stream of output pixels and control signals back to a matrix representing a frame.

Display Results and Compare to Behavioral Model

Use the Video Viewer blocks included in the template to compare the output frames visually. The validOut signal of the Pixels To Frame block is connected to the Enable port of the viewer. Run the model to display the results.

Generate HDL Code

Once your design is working in simulation, you can use HDL Coder™ to generate HDL code for the HDL Algorithm subsystem. See Generate HDL Code from Simulink (Vision HDL Toolbox).