What happens when unittest fixtures fail

In unittest fixture syntax and flow reference, I only presented fixture methods and functions that threw no exceptions.
However, in real production code, it is entirely possible for something to go wrong when setting up test fixtures.

This post is simply do demonstrate exactly what happens to the flow of your test code when an exception is thrown in a fixture function.

And, while I’m at it, I may as well demo the normal control flow when a test fails, asserts, or throws an exception.

To make the demo code small, I’m going to re-use the code from the previous post, derive my test classes from class TestFixtures (also from that post), and only override one method at a time to show a failure.

If you want to follow along, you’ll want to put this at the top of your file:

I’m using DemoException to represent a failure in a fixture. However, an assert will be treated the same way.

Let’s look at the following:

General Concept

The general idea is that if a setUp type of method/function fails, then the enclosed tests are NOT run, and the matching tearDown is NOT run.
If a tearDown type of method/function fails, then the only difference from the good case is that the error is noted, and the test fails.

exception in setUp()

The module and class level fixtures are called, just as they would in good cases.
We see setUp being called for both tests.
Since setUp is failing, neither of the tests are run.
And the tearDown method is not run for either test.

Output

exception in tearDown()

The flow looks identical to a successful test case, but the tearDown exception causes both tests to fail.

Output

exception in setUpClass()

Module fixtures are run normally, but nothing else.

Output

exception in tearDownClass()

Full test flow as in normal good tests, but the error is noted, and the test fails.

Output

exception in setUpModule()

To demostrate failure in module functions, I found it easiest just to copy the original test_fixtures.py file, and modify the setUpModule (or tearDownModule).

Not surprisingly, once setUpModule() hits the exception, nothing else is run.
The tearDownModule() is also not run, and the test records one error, and fails the test.

Output

exception in tearDownModule()

Since tearDownModule() is the last thing to be called when just running one module of tests, we see that all of the flow looks just like the good case. However, unittest does recognize the error, and fails the overall result.

Output

failure in a test

For test failures, asserts, and exceptions, I just wanted to demonstrate that all of the test fixtures are run, regardless of the test method outcome.

Output

assert in a test

Output

exception in a test

Output

Feedback

As always, let me know if any of this info is incorrect or misleading.

Cheers.

Leave a Reply