Unit/integration tests that involve closing/re-opening handles to objects

Hi all. I'm trying to set up a unit/integration testing environment for a set of MATLAB apps I'm developing. However, I'm having trouble managing handles to these apps in my testing environment. TL;DR: how can I set up handles to objects under test in a test class, and ensure that the handle can be overwritten when a test requires that the subservient object is deleted?

In my application, I'm trying to perform tests where e.g. the test class mimics pressing a button and determines the resulting flow of data and the outcome across several apps. Some of these tests result in objects being deleted as a desired outcome. Constructing and setting up the apps takes quite a bit of time, and so I'd like to avoid having to tear down and reconstruct several apps between each test as much as possible. Ideally, the test class should detect that a handle has been invalidated and replace it with a newly constructed object before continuing its tests. I thought I had found a way around this by creating properties for the test class and assigning handles to the apps under test to these properties. However, because the testCase object isn't shared between tests, once the handle to an object goes invalid there doesn't seem to be a way of restoring/replacing this handle, requiring a re-construction the object on every subsequent test even if the object does not get deleted in between.

For example, suppose I set up a test class like this:

classdef handle_test < matlab.unittest.TestCase & matlab.uitest.TestCase

%% Test properties.

properties

object_handle

end

%% Shared setup for the entire test class.

methods(TestClassSetup)

function testCase = setupOnce(testCase)

testCase.object_handle = SomeObjectConstructor(); % Imagine that this is a very time-consuming constructor, and singleton so it is not possible to construct several copies.

end

end

%% Setup for each test.

methods(TestMethodSetup)

function testCase = setup(testCase)

if ~isvalid(testCase.object_handle)

testCase.object_handle = SomeObjectConstructor(); % This should only be called once after a test that closes the object.

end

end

end

%% Shared teardown for the entire test class.

methods(TestClassTeardown)

function teardownOnce(testCase)

%

end

end

%% Teardown for each test.

methods(TestMethodTeardown)

function teardown(~)

%

end

end

%% Test methods

methods(Test)

% Perform Foo.

function testCase = testFoo(testCase)

actValue = testCase.object_handle.Foo();

testCase.verifyTrue(actValue, 'Foo did not true.');

end

% Close object.

function testCase = testClose(testCase)

testCase.object_handle.CloseObject();

actValue = isvalid(testCase.object_handle);

testCase.verifyFalse(actValue, 'Object not closed.');

end

% Perform Bar.

function testCase = testBar(testCase)

actValue = testCase.object_handle.Bar();

testCase.verifyTrue(actValue, 'Bar did not true.');

end

% Perform Baz.

function testCase = testBaz(testCase)

actValue = testCase.object_handle.Baz();

testCase.verifyTrue(actValue, 'Baz did not true.');

end

% ... % More tests.

end

In this example, the problem is as follows as the testrunner goes through the tests:

• Running testFoo does not require SomeObjectConstructor() because the object has already been constructed during the Once setup, and the handle object_handle is valid. This is fine.
• Running testClose now closes the object pointed to by object_handle and invalidates the handle. The test itself succeeds. This is fine.
• Running testBar next requires a call to SomeObjectConstructor() in the setup() method, because object_handle is invalid. This is fine.
• Running testBaz and any subsequent test next also require a call to SomeObjectConstructor() because the new valid handle that was generated during testBar()'s setup is not shared between subsequent test methods. This is what I'm trying to avoid.

As far as I can tell from the documentation of TestClass, it is not possible to somehow pass a modified testCase from one Test method to another, because the testrunner reserves the right to process its Test methods in any order and so only the testCase as it exists at the end of setupOnce() is passed to each setup() and subsequent test, with no flow of data in the other direction. This poses a problem when one the handles assigned during setupOnce() gets invalidated, because I couldn't find a way to then replace the invalid object_handle with a new, valid reference.

I imagine this problem occurs because I'm breaking some principle of unit/integration testing, but I am unsure how to restructure my test suite/class/configuration to avoid the problem and still have the benefits of organizing tests under the umbrella of a test class. Any suggestions or help is appreciated.

[Chi]

Please provide full code snippets for running the code above, including definition for SomeObjectConstructor class.

Answer 1

[Steven Lord]

What you're describing sounds like you could easily fall into Interacting Tests, a category of Erratic Tests.

If you had some way to reset the state of the app between tests (rather than creating a new instance) you could split the test cases between two or more test files, one for the test cases that don't delete the object and one or more for the methods that actually do delete them.