lesson 124 Testing & Code Quality · intermediate

Fixtures and test doubles

Tests need data and sometimes need to avoid slow or outside services. setUp gives every test fresh data, and a test double (a small fake or a Mock) stands in for a service and records how it was called.

You'll be able to

Explain why tests sharing one mutable object can pass or fail depending on their order
Build fresh test data in setUp
Pass a test double into the code under test, and check how it was called with Mock

The idea

Tests often need the same starting data: a basket with two items, a list of orders. Putting that data in one module-level variable that every test uses is tempting, and wrong when the code under test changes it: whatever one test does is still there when the next one starts, and results depend on the order tests run in.

unittest calls a TestCase's setUp method before every test method. Data built there is new for each test. (pytest calls such a function a *fixture*.) Store it on self, as self.basket = …, so the test methods can use it.

Run each example to see what it prints. Running examples doesn't count toward progress.

example-1.py · Shared data versus setUp

import unittest

def add_item(basket, sku, quantity):
    basket[sku] = basket.get(sku, 0) + quantity

def remove_item(basket, sku):
    del basket[sku]

SHARED = {"21329": 2}

class TestSharedBasket(unittest.TestCase):
    def test_empty_after_remove(self):
        remove_item(SHARED, "21329")
        self.assertEqual(SHARED, {})

    def test_more_after_add(self):
        add_item(SHARED, "21329", 1)
        self.assertEqual(SHARED["21329"], 3)

class TestFreshBasket(unittest.TestCase):
    def setUp(self):
        self.basket = {"21329": 2}

    def test_empty_after_remove(self):
        remove_item(self.basket, "21329")
        self.assertEqual(self.basket, {})

    def test_more_after_add(self):
        add_item(self.basket, "21329", 1)
        self.assertEqual(self.basket["21329"], 3)

def run(case):
    result = unittest.TestResult()
    unittest.defaultTestLoader.loadTestsFromTestCase(case).run(result)
    print("ran", result.testsRun, "| failures", len(result.failures), "| errors", len(result.errors))
    for test, details in result.failures + result.errors:
        print("  " + test.id().split(".", 1)[1], "->", details.strip().splitlines()[-1])

run(TestSharedBasket)
run(TestFreshBasket)

Some code depends on things a test shouldn't use: an exchange-rate service on the internet, a payment provider, the clock. Write such code so the dependency is passed in (as get_rate below). Then a test can pass a test double instead: a stand-in that returns a chosen answer instantly and records how it was called.

unittest.mock.Mock builds one in a line. Mock(return_value=1.17) can be called with any arguments and returns 1.17; Mock(side_effect=ConnectionError(...)) raises that error when called. Afterwards it can tell you how it was used: call_count, call_args_list, and assert_called_once_with(...), assert_not_called().

example-2.py · A fake exchange-rate service

import unittest
from unittest.mock import Mock

def total_in(currency, amount_gbp, get_rate):
    if currency == "GBP":
        return amount_gbp
    return round(amount_gbp * get_rate(currency), 2)

fake = Mock(return_value=1.17)
print(fake("EUR"), fake("USD"))
print(fake.call_count, fake.call_args_list)

class TestTotalIn(unittest.TestCase):
    def test_converts_with_the_rate(self):
        get_rate = Mock(return_value=1.17)
        self.assertEqual(total_in("EUR", 100.0, get_rate), 117.0)
        get_rate.assert_called_once_with("EUR")

    def test_pounds_need_no_rate(self):
        get_rate = Mock()
        self.assertEqual(total_in("GBP", 25.0, get_rate), 25.0)
        get_rate.assert_not_called()

    def test_rates_service_down(self):
        get_rate = Mock(side_effect=ConnectionError("rates service timed out"))
        with self.assertRaises(ConnectionError):
            total_in("USD", 10.0, get_rate)

def run(case):
    result = unittest.TestResult()
    unittest.defaultTestLoader.loadTestsFromTestCase(case).run(result)
    print("ran", result.testsRun, "| failures", len(result.failures), "| errors", len(result.errors))
    for test, details in result.failures + result.errors:
        print("  " + test.id().split(".", 1)[1], "->", details.strip().splitlines()[-1])

run(TestTotalIn)

When to use it: setUp whenever several tests start from the same data; a double for anything slow, random, external or costly (networks, payments, emails, the current time).

Mistakes to avoid:

Module-level test data that tests modify (order-dependent results, as above).
Replacing the code you meant to test. The double stands in for a *dependency*; the assertions are still about your function.
Asserting only on the double (get_rate was called) and never on the result.
Doubles that return values the real service never would, so the tests pass against a world that doesn't exist.

Review

setUp runs before every test: build fresh data there and store it on self.
Code that receives its dependencies can be tested with doubles instead of real services.
Mock(return_value=…) / Mock(side_effect=…) stand in for a call and record it: assert_called_once_with, assert_not_called, call_count.
Next: which inputs to test, so that a broken version is actually caught.

Practise this

Test restock alerts with a Mock