How to generate code coverage and cobertura test reports for the same source code in R2022b
5 views (last 30 days)
Show older comments
I have used the following code to execute test prior to R2022b (R2018b-R2021b).
import matlab.unittest.TestRunner;
import matlab.unittest.TestSuite;
import matlab.unittest.plugins.CodeCoveragePlugin;
import matlab.unittest.plugins.XMLPlugin;
import matlab.unittest.plugins.codecoverage.CoberturaFormat;
import matlab.unittest.plugins.codecoverage.CoverageReport
import matlab.unittest.selectors.HasBaseFolder;
suite = testsuite(fullfile(pwd, 'tests'), 'IncludeSubfolders', true);
fprintf("Running %d tests.\n", length([{suite.ProcedureName}]));
runner = TestRunner.withTextOutput();
runner.addPlugin(XMLPlugin.producingJUnitFormat('results.xml'));
runner.addPlugin( ...
CodeCoveragePlugin.forFolder({'.'}, ...
'IncludingSubfolders', true, ...
'Producing', CoberturaFormat('coverage.xml') ...
) ...
);
runner.addPlugin( ...
CodeCoveragePlugin.forFolder({'.'}, ...
'IncludingSubfolders', true, ...
'Producing', CoverageReport('coverageReport') ...
) ...
);
results = runner.run(suite);
display(results.table);
assert( ...
~any([results.Failed]), ...
'At least one test failed in the test session.' ...
);
% Optional, opens the test results in the matlab browser
% open(fullfile("coverageReport","index.html"))
As of R2022b this code fails with the following error:
MATLAB:unittest:CoverageReport:CoverageSessionsWithOverlappingSourcesNotAllowed
Error using matlab.unittest.internal.coverage.CodeCoverageCollector/initialize
Simultaneously generating coverage reports for the same source files is not
supported.
I noticed in the R2022b docs a new section about generating coverage with two plugins that isn't in the R2018b-R2021b CodeCoveragePlugin document, but didn't notice any breaking change in the R2022b release notes to the CodeCoveragePlugin. Based on the R2022b documents it seems like we can't generate two reports for the same source code. Did the functionality change?
Thanks for any insights or help.
0 Comments
Accepted Answer
David Hruska
on 12 Apr 2023
Hi Alexander,
In releases prior to R2022a, running tests with more than one code coverage plugin added to the TestRunner could sometimes produce incorrect results for one or both of the reports.
As of R2022a, you can only use multiple CodeCoveragePlugin instances in a single test run if they report on different source files. I see that this is documented only on the CodeCoveragePlugin page (see the "Version History" section), but not in the release notes. I'm not sure if we can retroactively adjust the release notes for R2022a, but I'll check with our documentation team on that.
It's still possible to produce multiple reports at the same time by providing multiple formats to a single CodeCoveragePlugin. For example:
runner.addPlugin( ...
CodeCoveragePlugin.forFolder({'.'}, ...
'IncludingSubfolders', true, ...
'Producing', [CoberturaFormat('coverage.xml'), CoverageReport('coverageReport')] ...
) ...
);
Hope this helps,
David
2 Comments
David Hruska
on 14 Apr 2023
Edited: David Hruska
on 14 Apr 2023
Happy to help! Following up on the release notes question: I confirmed with our documentation team that we're no longer making any updates to the R2022a release notes at this point and the best source of information is the Version History on the CodeCoveragePlugin documentation page.
More Answers (0)
See Also
Categories
Find more on Run Unit Tests in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!