matlab.unittest.plugins.DiagnosticsRecordingPlugin class

Package: matlab.unittest.plugins

Plugin to record diagnostics on test results

Description

The DiagnosticsRecordingPlugin enables programmatic access to the diagnostic information from unit tests.

This class creates a plugin to record diagnostics on test results. The TestRunner records these diagnostics as DiagnosticRecord arrays in the Details property of the TestResult object. Each element of the DiagnosticRecord array corresponds to an event in an individual test.

If you run tests with the runtests function, the testrunner function with no input, or the run method of TestSuite or TestCase, the test framework uses this plugin by default. Also, if you run performance tests with the runperf function or the run method of TimeExperiment, the test framework uses this plugin by default.

Construction

matlab.unittest.plugins.DiagnosticsRecordingPlugin creates a plugin to record diagnostics on test results. By default, the DiagnosticsRecordingPlugin records qualification failures and logged events.

matlab.unittest.plugins.DiagnosticsRecordingPlugin(Name,Value) creates a plugin with additional options specified by one or more Name,Value pair arguments. Name must appear inside single quotes (''). You can specify several name-value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

Input Arguments

Name-Value Pair Arguments

Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside quotes. You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

Example: matlab.unittest.plugins.DiagnosticsRecordingPlugin('IncludingPassingDiagnostics',true) creates a plugin that records passing diagnostics in addition to diagnostics for failing qualifications and logged events.

'IncludingPassingDiagnostics' — Indicator to record diagnostics from passing tests

false (default) | true

Whether to record diagnostics from passing tests, specified as false or true. By default the plugin does not record diagnostics from passing tests.

Data Types: logical

'LoggingLevel' — Maximum level at which logged diagnostics are recorded

1 (default) | 0 | 2 | 3 | 4 | matlab.unittest.Verbosity enumeration

Maximum level at which logged diagnostics are recorded by the plugin instance, specified as an integer value from 0 through 4, or as a matlab.unittest.Verbosity enumeration object. The plugin records diagnostics that are logged at this level and below. Integer values correspond to the members of the matlab.unittest.Verbosity enumeration.

By default the plugin records diagnostics logged at the matlab.unittest.Verbosity.Terse level (level 1). To exclude logged diagnostics, specify LoggingLevel as Verbosity.None (level 0).

Logged diagnostics are diagnostics that you supply to the testing framework with a call to the log (TestCase) or log (Fixture) method.

Numeric Representation	Enumeration Member Name	Verbosity Description
0	None	No information
1	Terse	Minimal information
2	Concise	Moderate amount of information
3	Detailed	Some supplemental information
4	Verbose	Lots of supplemental information

'OutputDetail' — Detail level for recorded events

3 (default) | 0 | 1 | 2 | 4 | matlab.unittest.Verbosity enumeration

Detail level for recorded events, specified as an integer value from 0 through 4, or as a matlab.unittest.Verbosity enumeration object. Integer values correspond to the members of the matlab.unittest.Verbosity enumeration.

The plugin records passing, failing, and logged events with the amount of detail specified by OutputDetail. By default the plugin records events at the matlab.unittest.Verbosity.Detailed level (level 3).

Numeric Representation	Enumeration Member Name	Verbosity Description
0	None	No information
1	Terse	Minimal information
2	Concise	Moderate amount of information
3	Detailed	Some supplemental information
4	Verbose	Lots of supplemental information

Properties

IncludePassingDiagnostics — Indicator if diagnostics for passing events are recorded

false (default) | true

This property is read-only.

Indicator if diagnostics for passing events are recorded, returned as false or true. This property is false by default. You can specify it as true during construction.

Data Types: logical

LoggingLevel — Maximum verbosity level for logged diagnostics

matlab.unittest.Verbosity enumeration object

This property is read-only.

Maximum verbosity level for logged diagnostics recorded by the plugin, returned as a matlab.unittest.Verbosity enumeration object. The plugin records diagnostics that are logged at this level and below. By default this property value is matlab.unittest.Verbosity.Terse. You can specify a different logging level during plugin construction.

Logged diagnostics are diagnostics that you supply to the testing framework with a call to the log (TestCase) or log (Fixture) method.

OutputDetail — Display level for event details

matlab.unittest.Verbosity enumeration object

This property is read-only.

Display level for event details, returned as a matlab.unittest.Verbosity enumeration object. The plugin displays passing, failing, and logged events with the amount of detail specified by OutputDetail. By default this property value is matlab.unittest.Verbosity.Detailed. You can specify a different output detail during plugin construction.

Copy Semantics

Handle. To learn how handle classes affect copy operations, see Copying Objects.

Examples

Record Diagnostics on Test Result

In your working folder, create a file, ExampleTest.m, containing the following test class. The intent of this test is to illustrate how to use the DiagnosticsRecordingPlugin plugin, and it is not intended to be a representative unit test.

classdef ExampleTest < matlab.unittest.TestCase  
    methods (Test)
        function testA(testCase)
            testCase.log(1,'Terse log message') 	% logs
            testCase.log(3,'Detailed log message') 	% logs
            testCase.verifyEqual(3+2,5)             % passes
            testCase.assumeTrue(true)               % passes
            testCase.verifyGreaterThan(5, 9)        % fails
            testCase.assertEqual(3.14,pi)           % fails/incomplete
        end
        function testB(testCase)
            % This test contains an intentional error - passing a character
            % instead of a variable to the ones function.
            a = [1 2];
            testCase.verifyEqual(ones('a'),[1 1]);  % errors
        end
    end
end

At the command prompt, create a test suite from the ExampleTest class.

suite   = testsuite('ExampleTest');

Create a test runner with no plugins. This code creates a silent runner and provides you with complete control over the installed plugins. Add a DiagnosticsRecordingPlugin to the test runner.

import matlab.unittest.TestRunner;
import matlab.unittest.plugins.DiagnosticsRecordingPlugin;

runner = TestRunner.withNoPlugins;
runner.addPlugin(DiagnosticsRecordingPlugin);

Run the tests.

results = runner.run(suite);

Display the result from the second test. The test fails and is incomplete.

results(2)

ans = 

  TestResult with properties:

          Name: 'ExampleTest/testB'
        Passed: 0
        Failed: 1
    Incomplete: 1
      Duration: 7.8912e-04
       Details: [1×1 struct]

Totals:
   0 Passed, 1 Failed, 1 Incomplete.
   0.00078912 seconds testing time.

Index into the diagnostic record to display more information.

results(2).Details.DiagnosticRecord

ans = 

  ExceptionDiagnosticRecord with properties:

                          Event: 'ExceptionThrown'
                     EventScope: TestMethod
                  EventLocation: 'ExampleTest/testB'
                      Exception: [1×1 MException]
    AdditionalDiagnosticResults: [1×0 matlab.unittest.diagnostics.DiagnosticResult]
                          Stack: [1×1 struct]
                         Report: 'Error occurred in ExampleTest/testB and it did not run to completion…'

The test throws an uncaught exception.

Collect the diagnostic records for the first test, testA.

testA_records = results(1).Details.DiagnosticRecord

testA_records = 

  1×3 heterogeneous DiagnosticRecord (LoggedDiagnosticRecord, QualificationDiagnosticRecord) array with properties:

    Event
    EventScope
    EventLocation
    Stack
    Report

View the events that the plugin recorded for testA.

{testA_records.Event}'

ans =

  3×1 cell array

    {'DiagnosticLogged'  }
    {'VerificationFailed'}
    {'AssertionFailed'   }

The plugin records the message logged at a Terse level of verbosity, and the verification and assertion failures.

Create a plugin that records messages at all verbosity levels and includes passing diagnostics. Rerun the tests and collect the diagnostic records for testA.

runner = TestRunner.withNoPlugins;
runner.addPlugin(DiagnosticsRecordingPlugin(...
    'IncludingPassingDiagnostics',true,'OutputDetail',4,'LoggingLevel',4));
results = runner.run(suite);
testA_records = results(1).Details.DiagnosticRecord;

View the events that the plugin recorded for testA.

{testA_records.Event}'

ans =

  6×1 cell array

    {'DiagnosticLogged'  }
    {'DiagnosticLogged'  }
    {'VerificationPassed'}
    {'AssumptionPassed'  }
    {'VerificationFailed'}
    {'AssertionFailed'   }

The plugin records diagnostic information for all the qualifications and calls to the log method.

Select all the records with failing event diagnostics.

failedRecords = selectFailed(testA_records)

failedRecords = 

  1×2 QualificationDiagnosticRecord array with properties:

    Event
    EventScope
    EventLocation
    TestDiagnosticResults
    FrameworkDiagnosticResults
    AdditionalDiagnosticResults
    Stack
    Report

Select all the records with passing event diagnostics and display the report for the first record.

passedRecords = selectPassed(testA_records);
passedRecords(1).Report

ans =

    'Verification passed in ExampleTest/testA.
     
         ---------------------
         Framework Diagnostic:
         ---------------------
         verifyEqual passed.
         --> The values are equal using "isequaln".
         
         Actual Value:
              5
         Expected Value:
              5
     
         ------------------
         Stack Information:
         ------------------
         In C:\work\ExampleTest.m (ExampleTest.testA) at 6'

Select all the records for incomplete events.

incompleteRecords = selectIncomplete(testA_records)

incompleteRecords = 

  QualificationDiagnosticRecord with properties:

                          Event: 'AssertionFailed'
                     EventScope: TestMethod
                  EventLocation: 'ExampleTest/testA'
          TestDiagnosticResults: [1×0 matlab.unittest.diagnostics.DiagnosticResult]
     FrameworkDiagnosticResults: [1×1 matlab.unittest.diagnostics.DiagnosticResult]
    AdditionalDiagnosticResults: [1×0 matlab.unittest.diagnostics.DiagnosticResult]
                          Stack: [1×1 struct]
                         Report: 'Assertion failed in ExampleTest/testA and it did not run to completion…'

Since this event is an assertion failure, the framework also returns this record with the failing diagnostics as failedRecords(2).

Select all the records with logged events and display the logged messages.

loggedRecords = selectLogged(testA_records);
{loggedRecords.Report}'

ans =

  2×1 cell array

    {'[Terse] Diagnostic logged (2018-04-12 13:15:23): Terse log message'      }
    {'[Detailed] Diagnostic logged (2018-04-12 13:15:23): Detailed log message'}