|$ curl https://forge-ai.dev/api/markdown?path=docs/python/testing
$cat docs/python-—-testing-(pytest/unittest).md
updated Recently·18 min read·published
Python — Testing (pytest/unittest)
◆Python◆Intermediate
Introduction
Testing is a critical practice in Python development. The standard library provides unittest, a xUnit-style framework, while the third-party pytest ecosystem offers a more expressive and feature-rich alternative. Both are widely used, and understanding each gives you flexibility across different projects.
This guide covers unittest fundamentals (TestCase, assertions, fixtures, test discovery), pytest (fixtures, parametrize, conftest.py, markers, monkeypatch), mocking with unittest.mock, test organization, coverage measurement, tox for multi-environment testing, and test-driven development patterns.
| Feature | unittest | pytest |
|---|---|---|
| Discovery | Built-in (test loader) | Auto-discovery by convention |
| Assertions | assertEqual, assertTrue, etc. | Plain assert (rich diff) |
| Fixtures | setUp/tearDown methods | Dependency injection via fixtures |
| Parametrize | subTest() or manual loops | @pytest.mark.parametrize |
| Plugins | Limited | Extensive plugin ecosystem |
| Stdlib | Built-in (no install) | Third-party (pip install) |
unittest — TestCase Basics
The unittest module provides a test framework inspired by Java's JUnit. Tests are defined by subclassing TestCase and naming methods with the test_ prefix. The test runner discovers and executes these methods automatically.
test_basic.py
Python
| 1 | import unittest |
| 2 | from typing import Any |
| 3 | |
| 4 | # Simple TestCase |
| 5 | def add(a: int, b: int) -> int: |
| 6 | return a + b |
| 7 | |
| 8 | class TestMath(unittest.TestCase): |
| 9 | """Tests for math utilities.""" |
| 10 | |
| 11 | def test_add_positive(self) -> None: |
| 12 | self.assertEqual(add(2, 3), 5) |
| 13 | |
| 14 | def test_add_negative(self) -> None: |
| 15 | self.assertEqual(add(-1, 1), 0) |
| 16 | |
| 17 | def test_add_zero(self) -> None: |
| 18 | self.assertEqual(add(0, 0), 0) |
| 19 | |
| 20 | # Running from command line: |
| 21 | # python -m unittest test_module.py |
| 22 | # python -m unittest test_module.TestMath |
| 23 | # python -m unittest test_module.TestMath.test_add_positive |
| 24 | |
| 25 | # Running with verbosity |
| 26 | # python -m unittest -v test_module.py |
| 27 | |
| 28 | # Test with expected failures |
| 29 | class TestEdgeCases(unittest.TestCase): |
| 30 | def test_float_precision(self): |
| 31 | # Use assertAlmostEqual for floats |
| 32 | self.assertAlmostEqual(0.1 + 0.2, 0.3, places=6) |
| 33 | |
| 34 | @unittest.expectedFailure |
| 35 | def test_known_bug(self): |
| 36 | # This test is expected to fail — won't count as failure |
| 37 | self.assertEqual(complex_operation(), 42) |
| 38 | |
| 39 | # Skipping tests |
| 40 | class TestSkip(unittest.TestCase): |
| 41 | @unittest.skip("not implemented yet") |
| 42 | def test_future(self): |
| 43 | pass |
| 44 | |
| 45 | @unittest.skipIf(sys.version_info < (3, 10), |
| 46 | "requires Python 3.10+") |
| 47 | def test_new_feature(self): |
| 48 | pass |
| 49 | |
| 50 | @unittest.skipUnless(sys.platform.startswith("linux"), |
| 51 | "Linux-only test") |
| 52 | def test_linux_only(self): |
| 53 | pass |
ℹ
info
Run python -m unittest discover to recursively discover all tests matching test_*.py in the current directory. Use the -v flag for verbose output showing individual test results.
unittest — Assert Methods
TestCase provides dozens of assert methods for different types of comparisons. These produce descriptive failure messages and are preferred over plain assertbecause they integrate with the test runner's reporting.
test_assertions.py
Python
| 1 | import unittest |
| 2 | |
| 3 | class TestAssertions(unittest.TestCase): |
| 4 | """Demonstrates common assert methods.""" |
| 5 | |
| 6 | def test_equality(self): |
| 7 | # Value equality (uses ==) |
| 8 | self.assertEqual(2 + 2, 4) |
| 9 | self.assertNotEqual(2 + 2, 5) |
| 10 | |
| 11 | def test_booleans(self): |
| 12 | self.assertTrue(True) |
| 13 | self.assertFalse(False) |
| 14 | self.assertTrue(isinstance("hi", str)) |
| 15 | |
| 16 | def test_comparison(self): |
| 17 | self.assertGreater(10, 5) |
| 18 | self.assertGreaterEqual(5, 5) |
| 19 | self.assertLess(3, 7) |
| 20 | self.assertLessEqual(7, 7) |
| 21 | |
| 22 | def test_collections(self): |
| 23 | self.assertIn("a", "abc") |
| 24 | self.assertNotIn("x", "abc") |
| 25 | self.assertCountEqual( # same elements, any order |
| 26 | [1, 2, 3], [3, 1, 2] |
| 27 | ) |
| 28 | self.assertListEqual( # ordered list comparison |
| 29 | [1, 2, 3], [1, 2, 3] |
| 30 | ) |
| 31 | self.assertDictEqual( |
| 32 | {"a": 1}, {"a": 1} |
| 33 | ) |
| 34 | self.assertTupleEqual( |
| 35 | (1, 2), (1, 2) |
| 36 | ) |
| 37 | |
| 38 | def test_none(self): |
| 39 | self.assertIsNone(None) |
| 40 | self.assertIsNotNone("hello") |
| 41 | |
| 42 | def test_identity(self): |
| 43 | a = [1, 2, 3] |
| 44 | b = a |
| 45 | c = [1, 2, 3] |
| 46 | self.assertIs(a, b) # same object |
| 47 | self.assertIsNot(a, c) # different objects |
| 48 | |
| 49 | def test_exceptions(self): |
| 50 | with self.assertRaises(ValueError): |
| 51 | int("not_a_number") |
| 52 | |
| 53 | with self.assertRaisesRegex(ValueError, "invalid"): |
| 54 | int("invalid") |
| 55 | |
| 56 | def test_floats(self): |
| 57 | self.assertAlmostEqual(0.1 + 0.2, 0.3) |
| 58 | self.assertNotAlmostEqual(0.1 + 0.2, 0.31) |
| 59 | |
| 60 | def test_warnings(self): |
| 61 | import warnings |
| 62 | with self.assertWarns(DeprecationWarning): |
| 63 | warnings.warn("old", DeprecationWarning) |
| 64 | |
| 65 | def test_logs(self): |
| 66 | import logging |
| 67 | with self.assertLogs("myapp", level="INFO") as log: |
| 68 | logging.getLogger("myapp").info("test msg") |
| 69 | self.assertIn("test msg", log.output[0]) |
| 70 | |
| 71 | # TestSuite — group and run tests programmatically |
| 72 | def suite() -> unittest.TestSuite: |
| 73 | suite = unittest.TestSuite() |
| 74 | suite.addTest(TestAssertions("test_equality")) |
| 75 | suite.addTest(TestAssertions("test_none")) |
| 76 | return suite |
| 77 | |
| 78 | if __name__ == "__main__": |
| 79 | runner = unittest.TextTestRunner(verbosity=2) |
| 80 | runner.run(suite()) |
| 81 | |
| 82 | # TestLoader — load tests by pattern |
| 83 | loader = unittest.TestLoader() |
| 84 | suite = loader.loadTestsFromTestCase(TestAssertions) |
| 85 | suite = loader.discover("tests", pattern="test_*.py") |
unittest — Fixtures (setUp / tearDown)
Fixtures in unittest are defined by overriding setUp() and tearDown() methods. These run before and after every test method. Class-level and module-level fixtures are also available via setUpClass, tearDownClass, setUpModule, and tearDownModule.
test_fixtures.py
Python
| 1 | import unittest |
| 2 | import tempfile |
| 3 | import os |
| 4 | from pathlib import Path |
| 5 | |
| 6 | # Module-level fixtures (run once per module) |
| 7 | def setUpModule(): |
| 8 | """Called once before any test in this module.""" |
| 9 | print("\nsetUpModule: module started") |
| 10 | |
| 11 | def tearDownModule(): |
| 12 | """Called once after all tests in this module.""" |
| 13 | print("\ntearDownModule: module done") |
| 14 | |
| 15 | |
| 16 | class TestDatabase(unittest.TestCase): |
| 17 | """Test with per-test fixtures.""" |
| 18 | |
| 19 | @classmethod |
| 20 | def setUpClass(cls): |
| 21 | """Called once before all tests in this class.""" |
| 22 | cls.db_path = tempfile.mktemp(suffix=".db") |
| 23 | print(f"creating database at {cls.db_path}") |
| 24 | |
| 25 | @classmethod |
| 26 | def tearDownClass(cls): |
| 27 | """Called once after all tests in this class.""" |
| 28 | if os.path.exists(cls.db_path): |
| 29 | os.remove(cls.db_path) |
| 30 | |
| 31 | def setUp(self): |
| 32 | """Called before every test method.""" |
| 33 | self.data: list[str] = [] |
| 34 | self.tmpdir = Path(tempfile.mkdtemp()) |
| 35 | print(f"setUp: test starting") |
| 36 | |
| 37 | def tearDown(self): |
| 38 | """Called after every test method, even on failure.""" |
| 39 | for f in self.tmpdir.iterdir(): |
| 40 | f.unlink() |
| 41 | self.tmpdir.rmdir() |
| 42 | print(f"tearDown: cleanup done") |
| 43 | |
| 44 | def test_insert(self): |
| 45 | self.data.append("value") |
| 46 | self.assertEqual(len(self.data), 1) |
| 47 | |
| 48 | def test_empty_by_default(self): |
| 49 | self.assertEqual(len(self.data), 0) |
| 50 | |
| 51 | # Mixin pattern — reusable fixture logic |
| 52 | class TempDirMixin: |
| 53 | """Mixin that provides a temporary directory.""" |
| 54 | tmpdir: Path |
| 55 | |
| 56 | def setUp(self): |
| 57 | super().setUp() # call parent setUp |
| 58 | self.tmpdir = Path(tempfile.mkdtemp()) |
| 59 | |
| 60 | def tearDown(self): |
| 61 | for f in self.tmpdir.iterdir(): |
| 62 | f.unlink() |
| 63 | self.tmpdir.rmdir() |
| 64 | super().tearDown() |
| 65 | |
| 66 | class TestFileIO(TempDirMixin, unittest.TestCase): |
| 67 | def test_write_read(self): |
| 68 | path = self.tmpdir / "test.txt" |
| 69 | path.write_text("hello") |
| 70 | self.assertEqual(path.read_text(), "hello") |
✓
best practice
Keep setUp and tearDown focused on the specific resource each test needs. Use setUpClass for expensive singleton resources (database connections, API clients). Always call super().setUp() and super().tearDown() when using inheritance.
pytest — Getting Started
pytest is a mature, feature-rich testing framework that uses plain assert statements with intelligent diff reporting. It auto-discovers tests by convention: files named test_*.py or *_test.py, and functions/classes prefixed with test.
test_pytest_basics.py
Python
| 1 | # Install: pip install pytest |
| 2 | # Run: pytest |
| 3 | # Run with details: pytest -v |
| 4 | # Run specific: pytest tests/test_math.py::test_add |
| 5 | # Stop on first failure: pytest -x |
| 6 | # Run last failed: pytest --lf |
| 7 | |
| 8 | # ----- test_math.py ----- |
| 9 | import pytest |
| 10 | from typing import Any |
| 11 | |
| 12 | def add(a: int, b: int) -> int: |
| 13 | return a + b |
| 14 | |
| 15 | # Plain functions — no class needed |
| 16 | def test_add_positive(): |
| 17 | assert add(2, 3) == 5 |
| 18 | |
| 19 | def test_add_negative(): |
| 20 | assert add(-1, 1) == 0 |
| 21 | |
| 22 | # Group tests in a class (no inheritance needed) |
| 23 | class TestMath: |
| 24 | def test_mul(self): |
| 25 | assert 3 * 4 == 12 |
| 26 | |
| 27 | def test_div(self): |
| 28 | assert 10 / 2 == 5.0 |
| 29 | |
| 30 | # Fixtures via dependency injection |
| 31 | @pytest.fixture |
| 32 | def sample_data() -> dict[str, int]: |
| 33 | return {"a": 1, "b": 2} |
| 34 | |
| 35 | def test_with_fixture(sample_data: dict[str, int]): |
| 36 | assert sample_data["a"] + sample_data["b"] == 3 |
| 37 | |
| 38 | # Markers — metadata for test selection |
| 39 | @pytest.mark.slow |
| 40 | def test_heavy_computation(): |
| 41 | import time |
| 42 | time.sleep(2) |
| 43 | assert True |
| 44 | |
| 45 | # Run: pytest -m slow (select slow tests) |
| 46 | # Run: pytest -m "not slow" (exclude slow tests) |
| 47 | |
| 48 | # Skip tests |
| 49 | @pytest.mark.skip(reason="not implemented") |
| 50 | def test_future(): |
| 51 | pass |
| 52 | |
| 53 | @pytest.mark.skipif( |
| 54 | __import__("sys").version_info < (3, 10), |
| 55 | reason="requires Python 3.10+" |
| 56 | ) |
| 57 | def test_new_feature(): |
| 58 | pass |
| 59 | |
| 60 | @pytest.mark.xfail(reason="known bug", strict=True) |
| 61 | def test_known_bug(): |
| 62 | assert complex_logic() == expected |
| 63 | |
| 64 | # Custom markers — register in pyproject.toml: |
| 65 | # [tool.pytest.ini_options] |
| 66 | # markers = [ |
| 67 | # "slow: marks tests as slow", |
| 68 | # "integration: marks integration tests", |
| 69 | # ] |
ℹ
info
pytest automatically captures stdout/stderr and shows it only on test failure (use -s to show all output). The -v flag increases verbosity, --tb=short shortens tracebacks, and -k "expression" filters test names by keyword expression.
pytest — Fixtures in Depth
pytest fixtures are functions decorated with @pytest.fixture that provide reusable test dependencies. They use dependency injection — a test or fixture declares the fixture name as a parameter. Fixtures have configurable scope (function, class, module, session) and built-in cleanup via yield.
test_fixtures_pytest.py
Python
| 1 | import pytest |
| 2 | import tempfile |
| 3 | from pathlib import Path |
| 4 | from typing import Iterator, Any |
| 5 | |
| 6 | # ----- Basic fixture ----- |
| 7 | @pytest.fixture |
| 8 | def db_connection(): |
| 9 | """Provides a fresh database connection per test.""" |
| 10 | conn = {"connected": True, "data": []} |
| 11 | yield conn # test runs here |
| 12 | conn["connected"] = False # teardown |
| 13 | |
| 14 | def test_db_insert(db_connection): |
| 15 | db_connection["data"].append("item") |
| 16 | assert len(db_connection["data"]) == 1 |
| 17 | |
| 18 | # Fixture with teardown (yield fixture) |
| 19 | @pytest.fixture |
| 20 | def tmp_project() -> Iterator[Path]: |
| 21 | """Creates a temp directory and cleans up after.""" |
| 22 | path = Path(tempfile.mkdtemp()) |
| 23 | yield path |
| 24 | import shutil |
| 25 | shutil.rmtree(path) # teardown after test |
| 26 | |
| 27 | def test_project_structure(tmp_project: Path): |
| 28 | (tmp_project / "src").mkdir() |
| 29 | assert (tmp_project / "src").exists() |
| 30 | |
| 31 | # ----- Fixture scopes ----- |
| 32 | @pytest.fixture(scope="function") # default — run per test |
| 33 | def per_test(): ... |
| 34 | |
| 35 | @pytest.fixture(scope="class") # once per test class |
| 36 | def per_class(): ... |
| 37 | |
| 38 | @pytest.fixture(scope="module") # once per module |
| 39 | def per_module(): ... |
| 40 | |
| 41 | @pytest.fixture(scope="session") # once per test run |
| 42 | def per_session(): ... |
| 43 | |
| 44 | # ----- Autouse fixtures ----- |
| 45 | @pytest.fixture(autouse=True) |
| 46 | def ensure_clean_env(): |
| 47 | """Runs for every test automatically.""" |
| 48 | import os |
| 49 | original = os.environ.copy() |
| 50 | yield |
| 51 | os.environ.clear() |
| 52 | os.environ.update(original) |
| 53 | |
| 54 | # ----- conftest.py (shared across test files) ----- |
| 55 | # Place shared fixtures in test/conftest.py: |
| 56 | # |
| 57 | # @pytest.fixture |
| 58 | # def api_client(): |
| 59 | # return MyAPIClient(base_url="https://test.example.com") |
| 60 | # |
| 61 | # Any test file under test/ can use api_client without importing. |
| 62 | |
| 63 | # ----- monkeypatch ----- |
| 64 | @pytest.fixture |
| 65 | def mock_env(monkeypatch: pytest.MonkeyPatch): |
| 66 | monkeypatch.setenv("DATABASE_URL", "sqlite:///test.db") |
| 67 | monkeypatch.setattr("os.cpu_count", lambda: 4) |
| 68 | yield |
| 69 | |
| 70 | def test_with_env(mock_env): |
| 71 | import os |
| 72 | assert os.environ["DATABASE_URL"] == "sqlite:///test.db" |
| 73 | |
| 74 | # ----- tmp_path (built-in fixture) ----- |
| 75 | def test_write_file(tmp_path: Path): |
| 76 | d = tmp_path / "subdir" |
| 77 | d.mkdir() |
| 78 | f = d / "file.txt" |
| 79 | f.write_text("hello") |
| 80 | assert f.read_text() == "hello" |
| 81 | assert len(list(tmp_path.iterdir())) == 1 |
ℹ
info
Use scope="session" sparingly — session-scoped fixtures persist across all tests and can cause test interdependence. Prefer scope="function" (default) for isolation. Use autouse=True for fixtures that must always run (e.g., env cleanup, patching).
pytest — Parametrize
@pytest.mark.parametrize allows running the same test with multiple inputs, generating separate test cases for each combination. This replaces manual loops and unittest.subTest(), providing clearer failure reports and better test selection.
test_parametrize.py
Python
| 1 | import pytest |
| 2 | from typing import Any |
| 3 | |
| 4 | # ----- Basic parametrize ----- |
| 5 | @pytest.mark.parametrize("a,b,expected", [ |
| 6 | (1, 2, 3), |
| 7 | (0, 0, 0), |
| 8 | (-1, 1, 0), |
| 9 | (100, -100, 0), |
| 10 | ]) |
| 11 | def test_add(a: int, b: int, expected: int): |
| 12 | assert a + b == expected |
| 13 | |
| 14 | # Each tuple becomes a separate test case: |
| 15 | # test_add[1-2-3] |
| 16 | # test_add[0-0-0] |
| 17 | # test_add[-1-1-0] |
| 18 | # test_add[100--100-0] |
| 19 | |
| 20 | # ----- Parametrize with named values ----- |
| 21 | @pytest.mark.parametrize("value,expected", [ |
| 22 | pytest.param("hello", True, id="non-empty-string"), |
| 23 | pytest.param("", False, id="empty-string"), |
| 24 | pytest.param(" ", True, id="whitespace"), |
| 25 | ]) |
| 26 | def test_is_not_empty(value: str, expected: bool): |
| 27 | assert bool(value.strip()) == expected |
| 28 | |
| 29 | # ----- Multiple parametrize decorators (cartesian product) ----- |
| 30 | @pytest.mark.parametrize("separator", [",", "|", "-"]) |
| 31 | @pytest.mark.parametrize("items", [ |
| 32 | ["a", "b"], |
| 33 | ["x"], |
| 34 | [], |
| 35 | ]) |
| 36 | def test_join(items: list[str], separator: str): |
| 37 | result = separator.join(items) |
| 38 | assert isinstance(result, str) |
| 39 | |
| 40 | # 3 × 3 = 9 test cases generated |
| 41 | |
| 42 | # ----- Fixtures + parametrize ----- |
| 43 | @pytest.fixture |
| 44 | def db_record() -> dict[str, Any]: |
| 45 | return {"id": 1, "name": "test"} |
| 46 | |
| 47 | @pytest.mark.parametrize("field,expected", [ |
| 48 | ("id", 1), |
| 49 | ("name", "test"), |
| 50 | ]) |
| 51 | def test_record_field(db_record, field: str, expected: Any): |
| 52 | assert db_record[field] == expected |
| 53 | |
| 54 | # ----- Parametrize with pytest_generate_tests ----- |
| 55 | # In conftest.py: |
| 56 | # def pytest_generate_tests(metafunc): |
| 57 | # if "config" in metafunc.fixturenames: |
| 58 | # metafunc.parametrize("config", [ |
| 59 | # {"env": "dev"}, |
| 60 | # {"env": "prod"}, |
| 61 | # ]) |
| 62 | |
| 63 | # ----- Combining parametrize with fixtures (indirect) ----- |
| 64 | @pytest.fixture |
| 65 | def user(request: pytest.FixtureRequest): |
| 66 | role = request.param |
| 67 | return {"name": "Alice", "role": role} |
| 68 | |
| 69 | @pytest.mark.parametrize("user", ["admin", "editor", "viewer"], indirect=True) |
| 70 | def test_user_permissions(user: dict[str, str]): |
| 71 | assert user["role"] in ("admin", "editor", "viewer") |
✓
best practice
Use meaningful pytest.param(..., id="name") IDs for complex parametrize cases — they appear in test output and make failure reports immediately understandable. Avoid too many parameters per test; split into multiple focused parametrize blocks if needed.
Mocking (unittest.mock)
The unittest.mock module provides tools for replacing real objects with mock objects during testing. Mock and MagicMock record call history and define return values. patch temporarily replaces attributes in modules or classes during a test scope.
test_mocking.py
Python
| 1 | from unittest.mock import Mock, MagicMock, patch, PropertyMock, sentinel, call |
| 2 | import pytest |
| 3 | from typing import Any |
| 4 | |
| 5 | # ----- Mock — basic usage ----- |
| 6 | mock = Mock() |
| 7 | mock.return_value = 42 |
| 8 | assert mock() == 42 |
| 9 | |
| 10 | # Record calls |
| 11 | mock("hello", key="val") |
| 12 | mock.call_count # → 1 |
| 13 | mock.assert_called_once_with("hello", key="val") |
| 14 | mock.assert_called() # at least once |
| 15 | mock.assert_called_once() # exactly once |
| 16 | |
| 17 | # Track call arguments |
| 18 | mock(1, 2, a=3) |
| 19 | mock.call_args # → call(1, 2, a=3) |
| 20 | mock.call_args_list # → [call("hello", key="val"), call(1, 2, a=3)] |
| 21 | |
| 22 | # ----- MagicMock — includes dunder methods ----- |
| 23 | mm = MagicMock() |
| 24 | mm.__len__.return_value = 5 |
| 25 | assert len(mm) == 5 |
| 26 | |
| 27 | mm.__str__.return_value = "magic" |
| 28 | assert str(mm) == "magic" |
| 29 | |
| 30 | mm.__getitem__.return_value = "item" |
| 31 | assert mm[0] == "item" |
| 32 | |
| 33 | # ----- PropertyMock ----- |
| 34 | class User: |
| 35 | @property |
| 36 | def name(self) -> str: |
| 37 | return "real" |
| 38 | |
| 39 | with patch("__main__.User.name", new_callable=PropertyMock) as mock_name: |
| 40 | mock_name.return_value = "mocked" |
| 41 | user = User() |
| 42 | assert user.name == "mocked" |
| 43 | |
| 44 | # ----- patch — context manager ----- |
| 45 | # Patch a module attribute |
| 46 | with patch("os.getcwd", return_value="/fake/path"): |
| 47 | import os |
| 48 | assert os.getcwd() == "/fake/path" |
| 49 | # Original restored after block |
| 50 | |
| 51 | # Patch as decorator |
| 52 | @patch("os.listdir", return_value=["a.txt", "b.txt"]) |
| 53 | def test_listdir(mock_listdir: Mock): |
| 54 | import os |
| 55 | assert os.listdir(".") == ["a.txt", "b.txt"] |
| 56 | mock_listdir.assert_called_once_with(".") |
| 57 | |
| 58 | # Patch object methods |
| 59 | @patch.object(User, "name", return_value="patched") |
| 60 | def test_user_name(mock_name: Mock): |
| 61 | assert User().name == "patched" |
| 62 | |
| 63 | # ----- sentinel — unique sentinel objects ----- |
| 64 | sentinel.SOME_VALUE # unique object used as placeholder |
| 65 | mock.method.return_value = sentinel.RESULT |
| 66 | assert mock.method() is sentinel.RESULT |
| 67 | |
| 68 | # ----- call — asserting sequences ----- |
| 69 | mock = Mock() |
| 70 | mock(1) |
| 71 | mock(2) |
| 72 | mock(3) |
| 73 | expected_calls = [call(1), call(2), call(3)] |
| 74 | assert mock.call_args_list == expected_calls |
| 75 | |
| 76 | # call with keyword args |
| 77 | mock(x=1) |
| 78 | mock.assert_has_calls([call(x=1)]) |
| 79 | |
| 80 | # ----- Mocking async functions ----- |
| 81 | async def fetch(): |
| 82 | return "data" |
| 83 | |
| 84 | mock_async = AsyncMock() |
| 85 | mock_async.return_value = "mocked data" |
| 86 | |
| 87 | # ----- patch.multiple — patch several at once ----- |
| 88 | with patch.multiple("os", getcwd=Mock(return_value="/x"), listdir=Mock(return_value=[])): |
| 89 | import os |
| 90 | assert os.getcwd() == "/x" |
| 91 | assert os.listdir() == [] |
| 92 | |
| 93 | # ----- Mock side_effect — dynamic behavior ----- |
| 94 | mock = Mock() |
| 95 | mock.side_effect = [1, 2, 3] # each call returns next value |
| 96 | assert mock() == 1 |
| 97 | assert mock() == 2 |
| 98 | assert mock() == 3 |
| 99 | |
| 100 | mock.side_effect = ValueError("error") |
| 101 | try: |
| 102 | mock() |
| 103 | except ValueError: |
| 104 | pass |
| 105 | |
| 106 | # mock.side_effect can be a callable: |
| 107 | mock.side_effect = lambda x: x * 2 |
| 108 | assert mock(5) == 10 |
ℹ
info
Prefer Mock over MagicMock unless you need dunder methods (like __len__ or __iter__). MagicMock defines default return values for all dunder methods, which can mask bugs. Always use assert_called_once_with over assert_called_with when you expect exactly one call.
Coverage & Multi-Environment Testing
coverage.py measures which lines of code are executed during testing, identifying untested paths. tox automates testing across multiple Python versions and environments. Together, they ensure your code works correctly and is thoroughly exercised.
test_coverage.py
Python
| 1 | # ----- coverage.py ----- |
| 2 | # Install: pip install coverage |
| 3 | # |
| 4 | # Run with pytest: |
| 5 | # coverage run -m pytest |
| 6 | # coverage report -m # terminal report with missing lines |
| 7 | # coverage html # HTML report (htmlcov/index.html) |
| 8 | # coverage xml # Cobertura XML (for CI tools) |
| 9 | # |
| 10 | # Configuration (pyproject.toml): |
| 11 | # [tool.coverage.run] |
| 12 | # source = ["src"] |
| 13 | # omit = ["*/tests/*", "*/migrations/*"] |
| 14 | # |
| 15 | # [tool.coverage.report] |
| 16 | # exclude_lines = [ |
| 17 | # "pragma: no cover", |
| 18 | # "def __repr__", |
| 19 | # "raise NotImplementedError", |
| 20 | # "if __name__ == .__main__.:", |
| 21 | # "pass", |
| 22 | # ] |
| 23 | # fail_under = 80 # CI fails if below 80% |
| 24 | |
| 25 | # ----- Example coverage gap ----- |
| 26 | def process_user(data: dict[str, Any]) -> str: |
| 27 | if data.get("role") == "admin": |
| 28 | return f"admin:{data['name']}" |
| 29 | elif data.get("role") == "editor": |
| 30 | return f"editor:{data['name']}" |
| 31 | else: |
| 32 | return f"viewer:{data['name']}" |
| 33 | # Test covers admin and viewer but NOT editor → uncovered branch |
| 34 | |
| 35 | # ----- tox — multi-environment testing ----- |
| 36 | # Install: pip install tox |
| 37 | # |
| 38 | # tox.ini (project root): |
| 39 | # [tox] |
| 40 | # envlist = py39, py310, py311, py312, lint |
| 41 | # isolated_build = True |
| 42 | # |
| 43 | # [testenv] |
| 44 | # deps = |
| 45 | # pytest |
| 46 | # coverage |
| 47 | # commands = |
| 48 | # coverage run -m pytest |
| 49 | # coverage report -m |
| 50 | # |
| 51 | # [testenv:lint] |
| 52 | # deps = ruff |
| 53 | # commands = ruff check src/ |
| 54 | # |
| 55 | # Run: tox # runs all environments |
| 56 | # Run specific: tox -e py311 |
| 57 | # Run parallel: tox -p auto |
| 58 | # Recreate: tox -r |
| 59 | |
| 60 | # ----- pytest --cov plugin ----- |
| 61 | # Install: pip install pytest-cov |
| 62 | # Run: pytest --cov=src --cov-report=term-missing |
| 63 | # Combines coverage with pytest directly |
| 64 | |
| 65 | # ----- GitHub Actions with tox ----- |
| 66 | # .github/workflows/test.yml: |
| 67 | # jobs: |
| 68 | # test: |
| 69 | # strategy: |
| 70 | # matrix: |
| 71 | # python-version: ["3.9", "3.10", "3.11", "3.12"] |
| 72 | # steps: |
| 73 | # - uses: actions/checkout@v4 |
| 74 | # - uses: actions/setup-python@v5 |
| 75 | # with: |
| 76 | # python-version: ${{ matrix.python-version }} |
| 77 | # - run: pip install tox |
| 78 | # - run: tox -e py # runs matching py version |
✓
best practice
Set a coverage floor (fail_under = 80) in CI to prevent coverage regression. Focus on covering critical business logic and edge cases rather than aiming for 100% — some code (like __repr__ or exception handlers) may not need explicit tests. Use # pragma: no cover sparingly.
Best Practices & TDD
Effective testing is about organization, discipline, and tooling. Following these patterns — from test file structure to test-driven development (TDD) — leads to maintainable, reliable test suites.
test_best_practices.py
Python
| 1 | # ----- Directory structure ----- |
| 2 | # project/ |
| 3 | # src/ |
| 4 | # myapp/ |
| 5 | # __init__.py |
| 6 | # core.py |
| 7 | # utils.py |
| 8 | # tests/ |
| 9 | # __init__.py # (optional, allows pytest namespace) |
| 10 | # conftest.py # shared fixtures |
| 11 | # test_core.py # mirrors src/myapp/core.py |
| 12 | # test_utils.py # mirrors src/myapp/utils.py |
| 13 | # integration/ |
| 14 | # test_api.py |
| 15 | # conftest.py # integration-specific fixtures |
| 16 | |
| 17 | # ----- Conftest organization ----- |
| 18 | # tests/conftest.py: |
| 19 | import pytest |
| 20 | from typing import Iterator, Any |
| 21 | |
| 22 | @pytest.fixture(scope="session") |
| 23 | def db_url() -> str: |
| 24 | return "sqlite:///:memory:" |
| 25 | |
| 26 | @pytest.fixture |
| 27 | def db_session(db_url: str): |
| 28 | """Per-test database session.""" |
| 29 | from sqlalchemy import create_engine |
| 30 | from sqlalchemy.orm import Session |
| 31 | engine = create_engine(db_url) |
| 32 | session = Session(engine) |
| 33 | yield session |
| 34 | session.close() |
| 35 | |
| 36 | # ----- pytest plugins (useful) ----- |
| 37 | # pip install: |
| 38 | # pytest-xdist — parallel test execution (-n auto) |
| 39 | # pytest-cov — coverage integration |
| 40 | # pytest-timeout — kill hanging tests (@pytest.mark.timeout(30)) |
| 41 | # pytest-sugar — better CLI output |
| 42 | # pytest-mock — streamlined mocking fixture |
| 43 | # pytest-asyncio — async test support |
| 44 | # pytest-randomly — randomize test order (catch interdependence) |
| 45 | # |
| 46 | # pytest.ini / pyproject.toml: |
| 47 | # [tool.pytest.ini_options] |
| 48 | # testpaths = ["tests"] |
| 49 | # addopts = "-v --strict-markers --tb=short" |
| 50 | # filterwarnings = ["error"] |
| 51 | |
| 52 | # ----- Test-Driven Development (TDD) cycle ----- |
| 53 | # 1. RED: Write a failing test first |
| 54 | def test_validate_email(): |
| 55 | from myapp.validation import validate_email |
| 56 | assert validate_email("user@example.com") is True |
| 57 | |
| 58 | # (Running pytest now fails — validate_email doesn't exist) |
| 59 | |
| 60 | # 2. GREEN: Write minimal code to pass |
| 61 | # src/myapp/validation.py: |
| 62 | # def validate_email(email: str) -> bool: |
| 63 | # return "@" in email |
| 64 | |
| 65 | # 3. REFACTOR: Improve code while keeping tests green |
| 66 | # def validate_email(email: str) -> bool: |
| 67 | # import re |
| 68 | # pattern = r"^[\w.+-]+@[\w-]+\.[\w.]+$" |
| 69 | # return bool(re.match(pattern, email)) |
| 70 | |
| 71 | # 4. Repeat for edge cases: |
| 72 | @pytest.mark.parametrize("email,expected", [ |
| 73 | ("user@example.com", True), |
| 74 | ("user.name+tag@example.co.uk", True), |
| 75 | ("", False), |
| 76 | ("not-an-email", False), |
| 77 | ("@domain.com", False), |
| 78 | ("user@", False), |
| 79 | ]) |
| 80 | def test_validate_email_parametrized(email: str, expected: bool): |
| 81 | from myapp.validation import validate_email |
| 82 | assert validate_email(email) is expected |
| 83 | |
| 84 | # ----- Arrange-Act-Assert (AAA) pattern ----- |
| 85 | def test_user_creation(): |
| 86 | # Arrange |
| 87 | data = {"name": "Alice", "role": "admin"} |
| 88 | # Act |
| 89 | user = create_user(data) |
| 90 | # Assert |
| 91 | assert user.name == "Alice" |
| 92 | assert user.role == "admin" |
| 93 | assert user.id is not None |
| 94 | |
| 95 | # ----- Avoid test interdependence ----- |
| 96 | # Bad — depends on previous test state |
| 97 | class TestBad: |
| 98 | items: list[str] = [] |
| 99 | def test_add(self): |
| 100 | self.items.append("a") |
| 101 | assert len(self.items) == 1 |
| 102 | def test_add_again(self): |
| 103 | self.items.append("b") |
| 104 | assert len(self.items) == 1 # Fails: length is 2 |
| 105 | |
| 106 | # Good — each test is isolated |
| 107 | class TestGood: |
| 108 | def test_add(self, fresh_list): |
| 109 | fresh_list.append("a") |
| 110 | assert len(fresh_list) == 1 |
| 111 | def test_add_again(self, fresh_list): |
| 112 | fresh_list.append("b") |
| 113 | assert len(fresh_list) == 1 # Passes: fresh list each test |
ℹ
info
Follow the FIRST principles for unit tests: Fast (run in ms), Isolated (no shared state), Repeatable (same result every run), Self-validating (pass/fail binary), and Thorough (cover edge cases and failure paths). Write tests before code in TDD — even small red-green-refactor cycles build confidence.
$Blueprint — Engineering Documentation·Section ID: PYTHON-TEST·Revision: 1.0