Python unit testing: An introductory guide

Over my career, I’ve met several developers who were intimidated by unit testing. I used to be one myself. Once I had a chance to learn a little bit about unit testing, my intimidation went away. I’ve been fortunate enough to share what I know about unit testing with many of those developers.

Today, we’re going to talk about unit testing, and specifically Python unit testing. By the time we’re done with this post, you’ll find that there isn’t any reason to be intimidated by unit testing.

The people I’ve met who are intimidated by unit testing usually fall into one of three camps: they’re not sure what unit testing is, they’re not sure how to effectively get started, or they’re not sure how to do it the “right” way. We’ll talk about all three of those things in this post.

What is Python unit testing?

Unit testing is a technique by which developers write small tests that they can run automatically to verify that small pieces of code work. We call these small pieces of code “units.” Each individual test runs to test a single unit of code.

How small should a unit be? Generally, as small as possible. When I’m writing unit tests, I write a test for every branch of an if statement within a function. The idea is that you have one full test for each possible unit of behavior in your code.

How small should a unit be? Generally, as small as possible.

How does a unit test work?

Whether you’re writing for Python or any other language, you’re likely to structure each test similarly. Let’s go over the major steps of a unit test:

When you establish the context, you basically say, “This is how I expect the state of the world to be.”

Establish the context

This might take the shape of inserting rows into your testing database or mocking function behavior. The goal of a unit test is to test one specific unit of behavior. When you establish the context, you basically say, “This is how I expect the state of the world to be.”

Each unit test is testing only one unit of behavior. This differentiates it from a functional test or an end-to-end test. So, step one is to establish the expected state of the world.

If you find yourself in a situation where establishing the context is too complicated, that might be a sign that your code is too tightly coupled. When we talk about code being “coupled,” we mean that it can’t be independently worked without connecting to other parts of your code.

Finding out that your code is overly coupled is an extremely common side effect of improving your unit testing. If you’d like to learn more about the process of decoupling your software, you can check out this great post.

Define your input data

Step two of a unit test is to define the data you’ll send to the function you’re testing. The benefit of a unit test is that it directly calls the function you’re testing. This means that you need to pass a value for each valid parameter to the function when you test it.

For quite simple function parameters, like strings or integers, you might simply call the function with the parameters defined in line. For more complicated parameters, like full objects, you’ll probably want to define them on separate lines to keep your tests readable.

Whenever possible, you should try to vary your testing data as much as possible. Use data that looks very close to what you expect your software to work with, so that you know that the application will work in real-world scenarios.

However, it’s also a great idea to mix things up, and use data that looks very different from what you’re expecting. You want to do this, because it means that you can be certain your software will correctly handle incorrect inputs.

Many software bugs come from software that doesn’t fail gracefully when prompted with incorrect inputs. The process of ensuring that your software validates input before operating on it is an element of defensive programming.

You can leverage unit tests to ensure that you’re programming in a way that won’t cause unexpected problems when malicious or mistaken users send you data that doesn’t fit your expectations.

Define the expected result

This is just like defining your input data, except here, we’re defining what we expect the output to be. Just like with inputs, we might be OK defining the expected output directly in line with the assertion statement. For more complicated objects, you might want to define them before you test the output.

When you’re defining expected results, you want to be as specific as possible. It’s much better to expect that your outcome from running a function will exactly equal the value 5, instead of simply testing that the value is not null.

The reason for this is that specific tests provide instant alerts when new code breaks existing behaviors. If your code that was supposed to output 5 instead started outputting 5.0, or 6, or “five,” you would find that bug much more quickly if you were testing for an explicit value match, versus just checking for null.

Sometimes, you can’t say with exact certainty what value you’ll get back from a function.

A note about fuzzy values

Sometimes, you will work in environments where specificity is not possible in your expected outputs. One common problem set where specific values are difficult is when you’re relating to any kind of “time math.”

You might want to assert that when you schedule some behavior to happen, it’s scheduled to run, say, 1 day in the future. Except that your code takes time to process, which means that by the time you test things, it’s slightly less than 1 day in the future.

Some libraries exist to help with this kind of problem, for instance, freezegun and timemachine. It often makes sense to adopt a library like this when you’re working with time-based code.

But if you can’t, or you’re working with something else that’s “fuzzy,” it’s good to understand that you don’t need to make every expected value perfectly precise. Sometimes, you can’t say with exact certainty what value you’ll get back from a function.

When that’s the case, you want to bound your expected values as tightly as you can, and simply accept that your unit tests won’t be perfectly precise.

Assert that the function result matches what you expect

This is the last step of the unit test. You call the function and store the result in a variable. Then, you use the unit testing library’s assertion capability to assert that the function result equals what you expected. If it does, then when you run the test, the test will pass. If it doesn’t, or if your code throws an error, the test will fail.

How many assertions are right for a unit test?

If you’re new to writing unit tests, this can feel like a difficult question. After all, you can put as many assertions into a test as you want. What’s the right number to include in an individual test?

There’s no single correct answer. In general, the advice that I give to newer developers is to try to limit your unit tests to a single assertion, but sometimes you might need to break that rule. The reason I try to target a single assertion for a unit test is because the point of a unit test is to test a single unit of code. If you’re asserting more than one thing inside that test, it’s likely that you’re actually testing more than one unit of code.

If you’re working on something like a functional test or an integration test you might find yourself needing to assert more than one thing inside a test. That makes sense, as those are tests that test more of the code base than just a single unit.

If you do find yourself writing a unit test where you feel that you need to test more than perhaps 2 or 3 assertions, it’s probably a good sign that you should step back and rethink either your testing strategy or the way that you’ve organized this unit of code.

What does a unit test look like?

Let’s steal a couple examples from the Python unittest docs:

import unittest

class TestStringMethods(unittest.TestCase):

    def test_upper(self):
        self.assertEqual(‘foo’.upper(), ‘FOO’)

    def test_isupper(self):
        self.assertTrue(‘FOO’.isupper())
        self.assertFalse(‘Foo’.isupper())

    def test_split(self):
        s = ‘hello world’
        self.assertEqual(s.split(), [‘hello’, ‘world’])

        # check that s.split fails when the separator is not a string
        with self.assertRaises(TypeError):
            s.split(2)

These tests are each fairly simple, but they illustrate the process nicely. None needs to pre-define any context, and they show how we can define both our inputs and our expected outputs directly in line.

It’s important to remember that each unit test is a normal function. Anything you can do in normal code you can also do in a unit test!

How can I add unit testing to my project?

If you’re using the unittest library, adding unit tests to your project is extremely simple! In fact, if all you wanted to do was test out the basic functionality, you can do so in a single file, with just one import statement at the top of the file:

import unittest

That matches directly with our example code up above. It really is that simple: all you have to do is add the unittest library to a file, and make some assertions. Then, when you want to run the test file, you can use the following command in your command line, inside the directory for your code:

python -m unittest [path/to/test/module]

That command will run your file as a test file and provide feedback on what tests failed and what tests passed. That’s it, you’ve started unit testing!

However, that’s not really how you want to organize your unit tests. Sprinkling test files around your code base, with minimal structure and no naming conventions, will make things harder to manage as your code base grows.

Instead, you should organize your test files into a predictable directory structure, and either prefix or suffix each of your test file names with the word test. Adopting a couple of basic conventions simplifies test management within your repository and will help any new developers get up to speed quickly.

If you’re using a different library like Pytest, Nose2 or PyTest, the best way to understand how to set up a basic test file and run unit tests will be found directly on their project home pages. The unittest library is built directly into Python, so we covered it more comprehensively here.

What Python unit testing libraries should I use?

There are a variety of popular Python unit testing frameworks. All of them allow you to write useful tests quickly and run those tests automatically.

Let’s break down the most popular frameworks and talk about why you might use each one.

unittest

This is the built-in Python unit testing framework, and it will work out of the box without any additional packages. It’s very well-documented because it’s part of the standard library. There’s a lot to be said for sane, well-documented defaults!

unittest can also feel a little bit verbose. What we call the “developer experience” isn’t always particularly strong. There have been a couple of attempts to improve on the experience of unit testing within Python on the market, and we’ll walk through those options next.

Nose2

The developers of this package wrote it to improve the process of testing code with unittest. Nose2’s libraries offer functions you can leverage to reduce repetition within your unit testing code. To get the full picture, you should visit the docs, but the primary highlight is probably the domain-specific language that helps make your tests more readable.

Testify

The Testify library is a wrapper around both unittest and Nose. Much like Nose2, the goal is to provide developers with helpers that reduce the amount of code they need to write for each test.

PyTest

Unlike the other libraries on this list, PyTest is not based on unittest. It is an entirely separate unit testing library. If you’re working in an existing code base, there’s a good chance that you have at least a few PyTest tests. PyTest has wide adoption because the syntax that it uses for tests often feels more “Pythonic,” like the rest of the Python in your code base.

Each of these major libraries does the same basic things. Each test you write uses the same structure outlined above. If you’re trying to figure out which to use for your project, test them all out. See which one fits your needs and which one you like writing the most. The best unit tests are the ones you write.

Python unit testing best practices

To wrap up, let’s talk about a few best practices for unit testing in any language, and specifically for Python:

A common failure state for many teams is thinking that they don’t need to test some areas of functionality

Actually write tests

This is the most important best practice! No matter whether you have one test or 1,000, a good unit test adds value to your code base. A common failure state for many teams is thinking that they don’t need to test some areas of functionality.

While this is rarely true, in reality, it is a good idea to have unit tests integrated even with the simplest parts of your code base. You simply do not know when some seemingly innocuous code change will introduce a regression.

When that happens, you’ll eventually find out about the bug. If you have automated testing, you get to find out before you ship it out to your customers. When you don’t, you get to find out when your customers tell you about it.

IBM Writer Ian Smalley puts it best when he notes, “In the short term, unit tests facilitate a quicker development process by allowing for automated testing. Over the long haul, unit testing yields savings in labor costs because less debugging is needed later in the software development lifecycle (SDLC), when those costs are apt to be considerably higher.”

Avoid the biggest testing mistakes

Take a gander at the biggest software testing mistakes you want to avoid. All three of the major mistakes in that article are ones that are worth calling out. The author is absolutely correct that the best way to find and catch bugs is to prevent writing them in the first place.

In order to do that, both developers and testers need to understand the underlying request motivating a software change. Sometimes a bug happens not because the code was written wrong, but because we were writing the wrong code in the first place anyway.

The second and third mistakes are alike: they both recognize the value of keeping human beings in the testing loop. While test automation is extremely valuable and you should automate tests across your entire stack, it’s still useful to keep human beings in the loop.

Human beings look at things critically, not just following the same script over and over. Varying your testing data and ensuring that human beings are still involved in your loop ensures that you are getting the best of both worlds: immediate feedback when something breaks, but also the ability to think outside of a box and find errors you wouldn’t find otherwise.

Test the smallest units you can

It’s easy to try to do too much in each of your tests. This makes the tests more complicated and difficult to read or reason about, so try to make sure that each test tests only one thing.

One of the major benefits of comprehensive unit testing is that when something breaks, you know what thing broke, and you know about it immediately. When you expand the level of test coverage outside of a single unit of code, you muddy the waters when it comes to diagnosing failing tests.

That doesn’t mean that you shouldn’t have tests that cover larger areas of your codebase; we’ve already talked about testing approaches like functional and integration testing. But the point of unit tests in particular is to test the smallest units possible.

Automate running your tests

One of the big benefits of unit testing is running those tests on every commit. That ensures that the commit’s changes didn’t upset any applecarts within your existing code. Within mature software teams, failing even a single test fails automated software builds.

While you may not have full test coverage over your entire codebase, it benefits your customers and your team to identify issues before you ship those issues into production.

By automating your test runs and integrating those runs into your build process, you ensure that every test provides the maximum value. At Tricentis, we know a lot about test automation, and we can help you figure out how to automate your tests.

You don’t need to make sure that every single test is perfect

Don’t be intimidated by unit testing

Like we talked about at the start, many developers feel intimidated by unit tests. They’re afraid of doing them the “wrong way.” But as we’ve noted, the worst way to write unit tests is not to write them at all. Each unit test you write adds value to your code base.

Each unit test helps document your code and ensure that future changes don’t break your existing logic.

You don’t need to make sure that every single test is perfect. And if you’re just getting started, it might take you a little while to get comfortable. But just like the rest of your code, you can refactor tests to make them better in the future, too.

The best day to get started writing unit tests is today. And if you’re ready to go beyond the basics, explore Tricentis solutions and supercharge your Python unit testing today.

This post was written by Eric Boersma. Eric is a software developer and development manager who’s done everything from IT security in pharmaceuticals to writing intelligence software for the US government to building international development teams for non-profits. He loves to talk about the things he’s learned along the way, and he enjoys listening to and learning from others as well.