Verify Generated C/C++ Code by Running Existing MATLAB Tests

Since R2025a

You can verify the behavior of generated C/C++ code by using MEX replacement test suites, which are tests that run existing MATLAB® tests and replace the calls to MATLAB functions with calls to MEX functions. MEX replacement test suites use the matlabtest.coder.MEXReplacementTestSuite class.

Running MEX replacement test suites uses the matlab.unittest framework and stores the results in matlab.unittest.TestResult objects. MEX replacement test suites have these advantages over verifying generated C/C++ code by using the coder.runTest (MATLAB Coder) function:

  • You can run tests from multiple test files.

  • You can collect coverage for the generated C/C++ code.

  • You can generate test and coverage reports from the results.

  • You can run MEX replacement test suites on continuous integration (CI) platforms.

  • You can run MEX replacement test suites and MATLAB test suites in the same run by executing a single command.

Note

You can only run MEX replacement test suites programmatically.

Alternatively, you can create and run equivalence tests that generate C/C++ code from MATLAB source code, execute the generated code, and verify that the execution matches the execution of the MATLAB source code. For more information, see Generate C/C++ Code and Test for Equivalence. Use equivalence tests if you need to run tests interactively or need test results that persist across sessions. For more information about running tests, see Run MATLAB Tests.

Generate MEX Function

Before you can create MEX replacement test suites, you must generate a MEX function from your MATLAB function by using the codegen (MATLAB Coder) function. For more information, see Generate C Code at the Command Line (MATLAB Coder).

This example code uses the ShortestPath project to generate a MEX function from the shortest_path function with the required input types and sizes.

openExample("matlabtest/ShortestPathExample")
cfg = coder.config("mex");
mtxType = coder.typeof(ones(100,100),[],1);
scalarDbl = coder.typeof(1);
codegen shortest_path -args {mtxType,scalarDbl,scalarDbl} -config cfg
Alternatively, you can interactively generate code by using the MATLAB Coder (MATLAB Coder) app. For more information, see Generate C Code by Using the MATLAB Coder App (MATLAB Coder).

Specify Code Generation Configuration for SIL or PIL Verification

With Embedded Coder®, you can verify generated C/C++ code by using software-in-the-loop (SIL) and processor-in-the-loop (PIL) execution. For more information about SIL and PIL execution, see Code Verification Through Software-in-the-Loop and Processor-in-the-Loop Execution (Embedded Coder).

To generate a MEX function for SIL or PIL verification, create a code generation configuration object for a LIB or DLL build type by using the coder.config (MATLAB Coder) function. Set the VerificationMode (MATLAB Coder) property to "SIL" or "PIL". For PIL verification, configure the Hardware (MATLAB Coder) property of your configuration parameter object for your processor hardware by using the coder.hardware (MATLAB Coder) function. Use the code generation configuration object during code generation.

This example code:

  • Creates a code generation configuration object for a static C library

  • Sets the verification mode to SIL

  • Generates code for a function called foo that does not take inputs and used the code configuration object

cfg = coder.config("lib");
cfg.VerificationMode = "SIL";
codegen foo -config cfg

Enable Collecting Coverage for Generated Code

To enable collecting coverage for the generated code when you run MEX replacement test suites, use Embedded Coder to generate a static C/C++ library that uses SIL verification:

  1. Create a code generation configuration object for a static library by using the coder.config (MATLAB Coder) function.

  2. Set the VerificationMode (MATLAB Coder) property to "SIL".

  3. Set the CodeCoverage (MATLAB Coder) property to true.

Use the code generation configuration object during code generation.

This example code creates a code generation configuration object that enables collecting coverage for the C code generated from the function foo.

cfg = coder.config("lib");
cfg.VerificationMode = "SIL";
cfg.CodeCoverage = true;
codegen foo -config cfg

Create Test Suites

To create MEX replacement test suites, use one of the static methods of the matlabtest.coder.MEXReplacementTestSuite class. Creating the test suite creates an array of matlab.unittest.Test objects that use the same tests that you use to test MATLAB code, but replace the calls to MATLAB functions with calls to MEX functions. You can create test suites that are based on:

  • A test file or class

  • A single test in a file

  • A folder, namespace, or project that contains tests

  • Tests that link to requirements

Tip

When you create the test suite, you must specify the MEX function to call. The default name for the generated MEX function combines the function name with _mex, followed by the platform-specific file extension. If you generate a MEX function for SIL or PIL verification, the MEX function name combines the function name with _sil or _pil, followed by the platform-specific file extension. To determine the file extension for the current platform, use the mexext function.

This example code uses the ShortestPath project to create a MEX replacement test suite based on tests from the graph_unit_tests.m test file and specifies the MEX file to call as shortest_path_mex.mexw64.

import matlabtest.coder.MEXReplacementTestSuite

testFile = "tests/graph_unit_tests.m";
mexFcn = which("shortest_path_mex.mexw64");
suite = MEXReplacementTestSuite.fromFile(testFile,mexFcn);

Run Test Suites

To run MEX replacement test suites, use the run method of matlab.unittest.TestSuite. You can only run MEX replacement test suites programmatically.

This example code runs a test suite stored in a variable called suite and stores the results in a variable.

results = run(suite);
Setting up MEXReplacementFixture
Done setting up MEXReplacementFixture: Replacing MATLAB function calls with the specified MEX functions
__________

Running graph_unit_tests
.......... ....
Done graph_unit_tests
__________

Collect Coverage

To collect coverage for the generated C/C++ code when you run MEX replacement test suites:

  1. Define the coverage format by creating an instance of matlab.unittest.plugins.codecoverage.CoverageResult.

  2. Create a plugin for the generated C/C++ code by creating an instance of matlabtest.coder.plugins.GeneratedCodeCoveragePlugin. Specify the desired coverage type for the plugin or use the default settings, which specify statement and function coverage.

  3. Create a test runner by using the testrunner function.

  4. Add the code coverage plugin to the test runner by using the addPlugin method.

  5. Run the test suite by using the run method of matlab.unittest.TestRunner. Store the test results in a variable.

This example code configures the test runner to collect coverage for the generated C code, runs the MEX replacement test suite that is stored in a variable called suite, and stores the test results in a variable.

import matlab.unittest.plugins.codecoverage.CoverageResult
import matlabtest.coder.plugins.GeneratedCodeCoveragePlugin

format = CoverageResult;
plugin = GeneratedCodeCoveragePlugin(Producing=format, ...
    MetricLevel="mcdc");
runner = testrunner("textoutput");
addPlugin(runner,plugin);
results = run(runner,suite);
Setting up MEXReplacementFixture
Done setting up MEXReplacementFixture: Replacing MATLAB function calls with the specified MEX functions
__________

Running graph_unit_tests
### Starting SIL execution for 'shortest_path'
    To terminate execution: clear shortest_path_sil
.......... ....
Done graph_unit_tests
__________

Tearing down MEXReplacementFixture
### Application stopped
### Stopping SIL execution for 'shortest_path'
    Code coverage report: getCodeCoverageData('shortest_path_sil')
Done tearing down MEXReplacementFixture
__________

View and Report Test Results

To view the test results programmatically, access the variable that contains the test results in the MATLAB Command Window. To generate a test report from the test results, use the generateHTMLReport method of the matlab.unittest.TestResult class.

This example code displays the test results stored in a variable called results, then generates an HTML test report called shortest_path_mex_results and saves it in the reports folder.

results
results = 
  1×14 TestResult array with properties:

    Name
    Passed
    Failed
    Incomplete
    Duration
    Details

Totals:
   14 Passed, 0 Failed, 0 Incomplete.
   1.4171 seconds testing time.

reportPath = fullfile("reports","shortest_path_mex_results.html");
generateHTMLReport(results,reportPath);
    Adding content to the test report.
    Writing test report to file.
Test report has been saved to:
 C:\Users\jdoe\MATLAB\reports\shortest_path_mex_results.html

View and Report Coverage Results

The Result property of the matlab.unittest.plugins.codecoverage.CoverageResult object contains the coverage results. To view the coverage results programmatically, use the coverageSummary method. To generate an HTML coverage report, use the generateHTMLReport method. For more information about the code coverage report, see Collect Code Coverage Metrics for MATLAB Source Code. Alternatively, to generate a standalone report as a single HTML file, use the generateStandaloneReport method.

This example code stores the coverage results in a variable, generates an HTML report from the results called shortest_path_mex_cov_results and stores it in the reports folder, then accesses the statement coverage results programmatically.

covResults = format.Result;
cReportPath = fullfile("reports","shortest_path_mex_cov_results.html");
generateHTMLReport(covResults,cReportPath);
covSummary = coverageSummary(covResults,"statement")

covSummary = 9×2
    39    44
    18    21
     0     2
     0     4
     1     4
    70    79
     2     2
     0    47
     2     2

See Also

Classes

Topics