How to generate code coverage and cobertura test reports for the same source code in R2022b

10 views (last 30 days)
I have used the following code to execute test prior to R2022b (R2018b-R2021b).
import matlab.unittest.TestRunner;
import matlab.unittest.TestSuite;
import matlab.unittest.plugins.CodeCoveragePlugin;
import matlab.unittest.plugins.XMLPlugin;
import matlab.unittest.plugins.codecoverage.CoberturaFormat;
import matlab.unittest.plugins.codecoverage.CoverageReport
import matlab.unittest.selectors.HasBaseFolder;
suite = testsuite(fullfile(pwd, 'tests'), 'IncludeSubfolders', true);
fprintf("Running %d tests.\n", length([{suite.ProcedureName}]));
runner = TestRunner.withTextOutput();
runner.addPlugin(XMLPlugin.producingJUnitFormat('results.xml'));
runner.addPlugin( ...
CodeCoveragePlugin.forFolder({'.'}, ...
'IncludingSubfolders', true, ...
'Producing', CoberturaFormat('coverage.xml') ...
) ...
);
runner.addPlugin( ...
CodeCoveragePlugin.forFolder({'.'}, ...
'IncludingSubfolders', true, ...
'Producing', CoverageReport('coverageReport') ...
) ...
);
results = runner.run(suite);
display(results.table);
assert( ...
~any([results.Failed]), ...
'At least one test failed in the test session.' ...
);
% Optional, opens the test results in the matlab browser
% open(fullfile("coverageReport","index.html"))
As of R2022b this code fails with the following error:
MATLAB:unittest:CoverageReport:CoverageSessionsWithOverlappingSourcesNotAllowed
Error using matlab.unittest.internal.coverage.CodeCoverageCollector/initialize
Simultaneously generating coverage reports for the same source files is not
supported.
I noticed in the R2022b docs a new section about generating coverage with two plugins that isn't in the R2018b-R2021b CodeCoveragePlugin document, but didn't notice any breaking change in the R2022b release notes to the CodeCoveragePlugin. Based on the R2022b documents it seems like we can't generate two reports for the same source code. Did the functionality change?
Thanks for any insights or help.

Accepted Answer

David Hruska
David Hruska on 12 Apr 2023
Hi Alexander,
In releases prior to R2022a, running tests with more than one code coverage plugin added to the TestRunner could sometimes produce incorrect results for one or both of the reports.
As of R2022a, you can only use multiple CodeCoveragePlugin instances in a single test run if they report on different source files. I see that this is documented only on the CodeCoveragePlugin page (see the "Version History" section), but not in the release notes. I'm not sure if we can retroactively adjust the release notes for R2022a, but I'll check with our documentation team on that.
It's still possible to produce multiple reports at the same time by providing multiple formats to a single CodeCoveragePlugin. For example:
runner.addPlugin( ...
CodeCoveragePlugin.forFolder({'.'}, ...
'IncludingSubfolders', true, ...
'Producing', [CoberturaFormat('coverage.xml'), CoverageReport('coverageReport')] ...
) ...
);
Hope this helps,
David
  2 Comments
Alexander
Alexander on 12 Apr 2023
Hi David,
Thanks for the fast response and pointing out that producing would accept an array out outputs. Sorry I missed that, appreciate your help.
-Alexander
David Hruska
David Hruska on 14 Apr 2023
Edited: David Hruska on 14 Apr 2023
Happy to help! Following up on the release notes question: I confirmed with our documentation team that we're no longer making any updates to the R2022a release notes at this point and the best source of information is the Version History on the CodeCoveragePlugin documentation page.

Sign in to comment.

More Answers (0)

Products


Release

R2022b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!