|$ curl https://forge-ai.dev/api/markdown?path=docs/python/magic-methods
$cat docs/python-—-magic-methods.md
updated Recently·18 min read·published
Python — Magic Methods
◆Python◆Intermediate to Advanced
Introduction
Magic methods (also called dunder methods for "double underscore") let you define how objects behave with Python's operators, built-in functions, and language constructs. They are the hooks that make your classes integrate seamlessly into Python's protocol-oriented design.
Every magic method is surrounded by double underscores — __init__, __str__, __add__, etc. When you write obj + other, Python looks for obj.__add__(other). When you call print(obj), Python calls obj.__str__().
ℹ
info
Always return NotImplemented (not raise NotImplementedError) from comparison and arithmetic methods when you don't know how to handle the other type. This lets Python try the reflected method on the other operand.
Object Construction & Destruction
These methods control the lifecycle of an object — creation, initialization, and cleanup.
| Method | Signature | Purpose | Called By |
|---|---|---|---|
| __new__ | cls, *args, **kwargs | Allocate & return a new instance | MyClass() |
| __init__ | self, *args, **kwargs | Initialize the instance (no return) | After __new__ |
| __del__ | self | Finalizer before garbage collection | del obj / GC |
construction.py
Python
| 1 | class Resource: |
| 2 | def __new__(cls, name: str): |
| 3 | print(f"Allocating {cls.__name__}") |
| 4 | instance = super().__new__(cls) |
| 5 | return instance |
| 6 | |
| 7 | def __init__(self, name: str): |
| 8 | print(f"Initializing Resource({name!r})") |
| 9 | self.name = name |
| 10 | self._handle = open(f"/tmp/{name}", "w") |
| 11 | |
| 12 | def __del__(self): |
| 13 | print(f"Finalizing Resource({self.name!r})") |
| 14 | self._handle.close() |
| 15 | |
| 16 | r = Resource("data") |
| 17 | # → Allocating Resource |
| 18 | # → Initializing Resource('data') |
| 19 | del r |
| 20 | # → Finalizing Resource('data') |
⚠
warning
There is no guarantee __del__ will be called promptly or at all. Use context managers (with blocks) or atexit handlers for deterministic cleanup of system resources like file handles and network connections.
String Representation
Control how your objects appear as strings — for display, debugging, formatting, and serialization.
| Method | Signature | Purpose |
|---|---|---|
| __str__ | self | Human-readable string for print() / str() |
| __repr__ | self | Unambiguous string for repr() / debugging |
| __format__ | self, format_spec | Custom formatting for f"{obj:spec}" |
| __bytes__ | self | Byte representation for bytes() |
representation.py
Python
| 1 | class Point: |
| 2 | def __init__(self, x: float, y: float): |
| 3 | self.x = x |
| 4 | self.y = y |
| 5 | |
| 6 | def __str__(self) -> str: |
| 7 | """Readable display""" |
| 8 | return f"({self.x}, {self.y})" |
| 9 | |
| 10 | def __repr__(self) -> str: |
| 11 | """Unambiguous — should recreate the object""" |
| 12 | return f"Point({self.x!r}, {self.y!r})" |
| 13 | |
| 14 | def __format__(self, spec: str) -> str: |
| 15 | """Custom format: f"{p:.2f}" → rounded coords""" |
| 16 | if spec: |
| 17 | parts = [f"{c:{spec}}" for c in (self.x, self.y)] |
| 18 | return f"({', '.join(parts)})" |
| 19 | return str(self) |
| 20 | |
| 21 | def __bytes__(self) -> bytes: |
| 22 | import struct |
| 23 | return struct.pack("dd", self.x, self.y) |
| 24 | |
| 25 | p = Point(3.14159, 2.71828) |
| 26 | print(str(p)) # → (3.14159, 2.71828) |
| 27 | print(repr(p)) # → Point(3.14159, 2.71828) |
| 28 | print(f"{p:.2f}") # → (3.14, 2.72) |
| 29 | print(bytes(p)) # → b'\x18...' |
✓
best practice
Always implement __repr__ for your classes. The convention is that __repr__ should return a string you could paste into Python to recreate the object, while __str__ should be readable for end users. If only __repr__ is defined, it also serves as __str__.
Comparison & Hashing
Enable rich comparison operators and make your objects hashable so they work in sets and as dictionary keys.
| Method | Operator | Default Fallback |
|---|---|---|
| __eq__ | == | Identity check (is) |
| __ne__ | != | Not __eq__ if __eq__ defined |
| __lt__ | < | None (error) |
| __le__ | <= | None (error) |
| __gt__ | > | None (error) |
| __ge__ | >= | None (error) |
| __hash__ | hash() | Identity-based hash |
comparison.py
Python
| 1 | from functools import total_ordering |
| 2 | |
| 3 | @total_ordering |
| 4 | class Version: |
| 5 | def __init__(self, major: int, minor: int, patch: int = 0): |
| 6 | self.major = major |
| 7 | self.minor = minor |
| 8 | self.patch = patch |
| 9 | |
| 10 | def __eq__(self, other: object) -> bool: |
| 11 | if not isinstance(other, Version): |
| 12 | return NotImplemented |
| 13 | return (self.major, self.minor, self.patch) == ( |
| 14 | other.major, other.minor, other.patch |
| 15 | ) |
| 16 | |
| 17 | def __lt__(self, other: "Version") -> bool: |
| 18 | if not isinstance(other, Version): |
| 19 | return NotImplemented |
| 20 | return (self.major, self.minor, self.patch) < ( |
| 21 | other.major, other.minor, other.patch |
| 22 | ) |
| 23 | |
| 24 | def __hash__(self) -> int: |
| 25 | return hash((self.major, self.minor, self.patch)) |
| 26 | |
| 27 | def __repr__(self) -> str: |
| 28 | return f"Version({self.major}, {self.minor}, {self.patch})" |
| 29 | |
| 30 | v1 = Version(2, 1, 0) |
| 31 | v2 = Version(3, 0, 0) |
| 32 | print(v1 < v2) # → True |
| 33 | print(v1 == Version(2, 1, 0)) # → True |
| 34 | |
| 35 | # Hashable — works in sets / dicts |
| 36 | versions = {v1, v2} |
| 37 | print(len(versions)) # → 2 |
🔥
pro tip
The @total_ordering decorator from functools lets you define only __eq__ and __lt__, and it fills in the other four comparison operators automatically. Whenever you define __eq__, consider whether __hash__ should also be defined — mutable objects should not be hashable.
Numeric Operators
Make your objects work with arithmetic, bitwise, and unary operators. Each operator has three variants: forward, reflected (when the left operand doesn't support the operation), and in-place.
| Category | Methods | Operators |
|---|---|---|
| Arithmetic | __add__, __sub__, __mul__, __truediv__, __floordiv__, __mod__, __pow__ | +, -, *, /, //, %, ** |
| Unary | __neg__, __pos__, __abs__, __invert__ | -, +, abs(), ~ |
| Bitwise | __and__, __or__, __xor__, __lshift__, __rshift__ | &, |, ^, <<, >> |
| Reflected | __radd__, __rsub__, __rmul__, ... | Same operators, swapped operands |
| In-place | __iadd__, __isub__, __imul__, ... | +=, -=, *=, /=, //=, %=, **=, &=, |=, ^= |
numeric.py
Python
| 1 | class Vector: |
| 2 | def __init__(self, x: float, y: float): |
| 3 | self.x = x |
| 4 | self.y = y |
| 5 | |
| 6 | # Forward arithmetic |
| 7 | def __add__(self, other: "Vector") -> "Vector": |
| 8 | if not isinstance(other, Vector): |
| 9 | return NotImplemented |
| 10 | return Vector(self.x + other.x, self.y + other.y) |
| 11 | |
| 12 | def __sub__(self, other: "Vector") -> "Vector": |
| 13 | if not isinstance(other, Vector): |
| 14 | return NotImplemented |
| 15 | return Vector(self.x - other.x, self.y - other.y) |
| 16 | |
| 17 | def __mul__(self, scalar: float) -> "Vector": |
| 18 | if not isinstance(scalar, (int, float)): |
| 19 | return NotImplemented |
| 20 | return Vector(self.x * scalar, self.y * scalar) |
| 21 | |
| 22 | # Reflected: scalar * vector |
| 23 | def __rmul__(self, scalar: float) -> "Vector": |
| 24 | return self.__mul__(scalar) |
| 25 | |
| 26 | # Unary |
| 27 | def __neg__(self) -> "Vector": |
| 28 | return Vector(-self.x, -self.y) |
| 29 | |
| 30 | def __abs__(self) -> float: |
| 31 | return (self.x ** 2 + self.y ** 2) ** 0.5 |
| 32 | |
| 33 | def __bool__(self) -> bool: |
| 34 | return self.x != 0 or self.y != 0 |
| 35 | |
| 36 | # In-place |
| 37 | def __iadd__(self, other: "Vector") -> "Vector": |
| 38 | if not isinstance(other, Vector): |
| 39 | return NotImplemented |
| 40 | self.x += other.x |
| 41 | self.y += other.y |
| 42 | return self |
| 43 | |
| 44 | def __repr__(self) -> str: |
| 45 | return f"Vector({self.x}, {self.y})" |
| 46 | |
| 47 | v = Vector(1, 2) |
| 48 | w = Vector(3, 4) |
| 49 | print(v + w) # → Vector(4, 6) |
| 50 | print(v * 3) # → Vector(3, 6) |
| 51 | print(3 * v) # → Vector(3, 6) [reflected] |
| 52 | print(-v) # → Vector(-1, -2) |
| 53 | print(abs(v)) # → 2.236... |
| 54 | v += w |
| 55 | print(v) # → Vector(4, 6) |
ℹ
info
In-place methods (__iadd__ etc.) should return self after mutation. If an in-place method is not defined, Python falls back to forward(self, other) and reassigns the result. Returning NotImplemented from forward triggers the reflected method on the other operand.
Container & Iteration Protocols
Make your objects behave like built-in containers — lists, dicts, sets — with indexing, membership, iteration, and length.
| Method | Purpose |
|---|---|
| __len__ | Return container length for len() |
| __getitem__ | Access element by key: obj[key] |
| __setitem__ | Assign element: obj[key] = value |
| __delitem__ | Delete element: del obj[key] |
| __contains__ | Membership test: value in obj |
| __iter__ | Return iterator for for x in obj |
| __next__ | Return next item from iterator (raise StopIteration) |
| __reversed__ | Reverse iteration for reversed() |
container.py
Python
| 1 | class RingBuffer: |
| 2 | """Fixed-size circular buffer that supports iteration.""" |
| 3 | |
| 4 | def __init__(self, size: int): |
| 5 | self._size = size |
| 6 | self._data = [None] * size |
| 7 | self._pos = 0 |
| 8 | self._count = 0 |
| 9 | |
| 10 | def __len__(self) -> int: |
| 11 | return self._count |
| 12 | |
| 13 | def __getitem__(self, idx: int): |
| 14 | if not isinstance(idx, int): |
| 15 | raise TypeError("Index must be int") |
| 16 | if idx < 0 or idx >= self._count: |
| 17 | raise IndexError("RingBuffer index out of range") |
| 18 | return self._data[(self._pos - self._count + idx) % self._size] |
| 19 | |
| 20 | def __setitem__(self, idx: int, value): |
| 21 | if idx < 0 or idx >= self._count: |
| 22 | raise IndexError("RingBuffer index out of range") |
| 23 | self._data[(self._pos - self._count + idx) % self._size] = value |
| 24 | |
| 25 | def append(self, value): |
| 26 | self._data[self._pos] = value |
| 27 | self._pos = (self._pos + 1) % self._size |
| 28 | self._count = min(self._count + 1, self._size) |
| 29 | |
| 30 | def __contains__(self, value) -> bool: |
| 31 | return any(v == value for v in self) |
| 32 | |
| 33 | def __iter__(self): |
| 34 | self._idx = 0 |
| 35 | return self |
| 36 | |
| 37 | def __next__(self): |
| 38 | if self._idx >= self._count: |
| 39 | raise StopIteration |
| 40 | val = self[self._idx] |
| 41 | self._idx += 1 |
| 42 | return val |
| 43 | |
| 44 | def __reversed__(self): |
| 45 | for i in range(self._count - 1, -1, -1): |
| 46 | yield self[i] |
| 47 | |
| 48 | buf = RingBuffer(3) |
| 49 | buf.append(10); buf.append(20); buf.append(30); buf.append(40) |
| 50 | print(len(buf)) # → 3 |
| 51 | print(buf[0]) # → 20 (oldest) |
| 52 | print(30 in buf) # → True |
| 53 | print(list(buf)) # → [20, 30, 40] |
| 54 | print(list(reversed(buf))) # → [40, 30, 20] |
🔥
pro tip
If you define __getitem__ with integer keys and raise IndexError for out-of-range, Python can provide __iter__ and __contains__ automatically, though explicit implementations are more efficient.
Callable Objects
Make any object callable like a function by implementing __call__. This is the foundation of function-like objects (functors), decorators, and closures.
| Method | Signature | Purpose |
|---|---|---|
| __call__ | self, *args, **kwargs | Make instance callable: obj() |
callable.py
Python
| 1 | from typing import Callable |
| 2 | import functools |
| 3 | |
| 4 | class Profiler: |
| 5 | """Callable decorator that profiles function calls.""" |
| 6 | |
| 7 | def __init__(self): |
| 8 | self.stats: dict[str, list[float]] = {} |
| 9 | |
| 10 | def __call__(self, func: Callable) -> Callable: |
| 11 | @functools.wraps(func) |
| 12 | def wrapper(*args, **kwargs): |
| 13 | import time |
| 14 | start = time.perf_counter() |
| 15 | result = func(*args, **kwargs) |
| 16 | elapsed = time.perf_counter() - start |
| 17 | self.stats.setdefault(func.__name__, []).append(elapsed) |
| 18 | return result |
| 19 | return wrapper |
| 20 | |
| 21 | def report(self) -> str: |
| 22 | lines = [] |
| 23 | for name, times in self.stats.items(): |
| 24 | avg = sum(times) / len(times) |
| 25 | total = sum(times) |
| 26 | lines.append( |
| 27 | f"{name}: {len(times)} calls, " |
| 28 | f"avg {avg*1000:.2f}ms, total {total*1000:.2f}ms" |
| 29 | ) |
| 30 | return " |
| 31 | ".join(lines) |
| 32 | |
| 33 | profiler = Profiler() |
| 34 | |
| 35 | @profiler |
| 36 | def slow_add(a: int, b: int) -> int: |
| 37 | return a + b |
| 38 | |
| 39 | slow_add(1, 2) |
| 40 | slow_add(3, 4) |
| 41 | print(profiler.report()) |
| 42 | # → slow_add: 2 calls, avg ...ms, total ...ms |
Context Managers
Context managers let you use the with statement to set up and tear down resources deterministically. Both synchronous and async versions are supported.
| Method | Signature | Purpose |
|---|---|---|
| __enter__ | self | Enter context; return value bound to as |
| __exit__ | self, exc_type, exc_val, exc_tb | Exit context; handle exceptions |
| __aenter__ | self | Enter async context (async with) |
| __aexit__ | self, exc_type, exc_val, exc_tb | Exit async context; handle exceptions |
context_managers.py
Python
| 1 | import time |
| 2 | |
| 3 | class Timer: |
| 4 | """Context manager that measures execution time.""" |
| 5 | |
| 6 | def __enter__(self): |
| 7 | self.start = time.perf_counter() |
| 8 | return self |
| 9 | |
| 10 | def __exit__(self, exc_type, exc_val, exc_tb): |
| 11 | self.elapsed = time.perf_counter() - self.start |
| 12 | # Return False (or None) to propagate exceptions, |
| 13 | # return True to suppress them |
| 14 | return False |
| 15 | |
| 16 | def __str__(self) -> str: |
| 17 | return f"{self.elapsed*1000:.2f}ms" |
| 18 | |
| 19 | class ManagedFile: |
| 20 | """Context manager for file-like resources.""" |
| 21 | |
| 22 | def __init__(self, path: str, mode: str = "r"): |
| 23 | self.path = path |
| 24 | self.mode = mode |
| 25 | |
| 26 | def __enter__(self): |
| 27 | self._file = open(self.path, self.mode) |
| 28 | return self._file |
| 29 | |
| 30 | def __exit__(self, exc_type, exc_val, exc_tb): |
| 31 | self._file.close() |
| 32 | if exc_type is FileNotFoundError: |
| 33 | print(f"File not found: {self.path}") |
| 34 | return True # suppress |
| 35 | return False |
| 36 | |
| 37 | # Usage |
| 38 | with Timer() as t: |
| 39 | total = sum(range(10_000_000)) |
| 40 | print(f"Computation took {t}") # → Computation took 123.45ms |
ℹ
info
Context managers are the preferred way to manage resources in Python. Use contextlib.contextmanager to turn a generator function into a context manager without writing a class. For async, use contextlib.asynccontextmanager.
Attribute Access
Intercept attribute access, assignment, deletion, and listing. These methods enable lazy loading, computed attributes, proxies, and ORMs.
| Method | Signature | When Called |
|---|---|---|
| __getattr__ | self, name | When normal lookup fails (obj.missing) |
| __getattribute__ | self, name | On every attribute access (unconditional) |
| __setattr__ | self, name, value | On obj.attr = value |
| __delattr__ | self, name | On del obj.attr |
| __dir__ | self | On dir(obj) — list available attributes |
attribute_access.py
Python
| 1 | class LazyRecord: |
| 2 | """Loads attributes lazily from a dict on first access.""" |
| 3 | |
| 4 | def __init__(self, data: dict): |
| 5 | self._data = data |
| 6 | self._cache: dict = {} |
| 7 | |
| 8 | def __getattr__(self, name: str): |
| 9 | """Called only when normal lookup fails.""" |
| 10 | if name.startswith("_"): |
| 11 | raise AttributeError(name) |
| 12 | if name in self._cache: |
| 13 | return self._cache[name] |
| 14 | if name not in self._data: |
| 15 | raise AttributeError( |
| 16 | f"LazyRecord has no attribute {name!r}" |
| 17 | ) |
| 18 | print(f"Loading {name}...") |
| 19 | value = self._data[name] |
| 20 | self._cache[name] = value |
| 21 | return value |
| 22 | |
| 23 | def __setattr__(self, name: str, value): |
| 24 | if name.startswith("_"): |
| 25 | super().__setattr__(name, value) |
| 26 | else: |
| 27 | self._cache[name] = value |
| 28 | |
| 29 | def __delattr__(self, name: str): |
| 30 | if name in self._cache: |
| 31 | del self._cache[name] |
| 32 | elif name in self._data: |
| 33 | del self._data[name] |
| 34 | else: |
| 35 | raise AttributeError(name) |
| 36 | |
| 37 | def __dir__(self): |
| 38 | return sorted(set( |
| 39 | list(self._data.keys()) |
| 40 | | list(self._cache.keys()) |
| 41 | | {"_data", "_cache"} |
| 42 | )) |
| 43 | |
| 44 | record = LazyRecord({"name": "Alice", "role": "Engineer"}) |
| 45 | print(record.name) # → Loading... |
| 46 | Alice |
| 47 | print(record.name) # → Alice (cached, no "Loading") |
| 48 | record.role = "Senior Engineer" |
| 49 | print(dir(record)) # → ['_cache', '_data', 'name', 'role'] |
⚠
warning
Be careful with __getattribute__ — it intercepts every attribute access including those inside the method itself, which causes infinite recursion. Always call super().__getattribute__(name) to access the actual attribute. Use __getattr__ (which only fires on failure) when possible.
Type Conversion
Define how your objects convert to built-in types like integers, floats, booleans, and complex numbers.
| Method | Signature | Purpose |
|---|---|---|
| __int__ | self | Convert to int |
| __float__ | self | Convert to float |
| __bool__ | self | Truthiness: bool(obj), if obj |
| __complex__ | self | Convert to complex |
| __index__ | self | Convert to int for indexing / bin() / hex() |
conversion.py
Python
| 1 | class Rational: |
| 2 | """A rational number with true conversion methods.""" |
| 3 | |
| 4 | def __init__(self, num: int, den: int = 1): |
| 5 | if den == 0: |
| 6 | raise ZeroDivisionError("denominator cannot be 0") |
| 7 | self._num = num |
| 8 | self._den = den |
| 9 | |
| 10 | def __int__(self) -> int: |
| 11 | return self._num // self._den |
| 12 | |
| 13 | def __float__(self) -> float: |
| 14 | return self._num / self._den |
| 15 | |
| 16 | def __bool__(self) -> bool: |
| 17 | return self._num != 0 |
| 18 | |
| 19 | def __complex__(self) -> complex: |
| 20 | return complex(float(self), 0.0) |
| 21 | |
| 22 | def __index__(self) -> int: |
| 23 | if self._den != 1: |
| 24 | raise ValueError( |
| 25 | "Cannot use non-integer rational as index" |
| 26 | ) |
| 27 | return self._num |
| 28 | |
| 29 | def __repr__(self) -> str: |
| 30 | return f"Rational({self._num}, {self._den})" |
| 31 | |
| 32 | r = Rational(3, 4) |
| 33 | print(float(r)) # → 0.75 |
| 34 | print(int(r)) # → 0 |
| 35 | print(bool(r)) # → True |
| 36 | print(complex(r)) # → (0.75+0j) |
| 37 | |
| 38 | r2 = Rational(5, 1) |
| 39 | items = ["a", "b", "c", "d", "e"] |
| 40 | print(items[r2]) # → e (via __index__) |
Summary
Magic methods are the backbone of Python's protocol-oriented design. By implementing the right dunder methods, your classes can participate in every language construct — iteration, arithmetic, context management, attribute access, and more.
| Category | Key Methods | Python Feature |
|---|---|---|
| Construction | __new__, __init__, __del__ | Object lifecycle |
| Representation | __str__, __repr__, __format__ | String display |
| Comparison | __eq__, __lt__, __hash__ | Sorting, sets, dict keys |
| Numeric | __add__, __mul__, __iadd__ | Arithmetic operators |
| Container | __getitem__, __iter__, __len__ | Indexing, iteration |
| Callable | __call__ | Function-like objects |
| Context | __enter__, __exit__ | with statement |
| Attribute | __getattr__, __setattr__ | Dynamic attributes |
| Conversion | __int__, __float__, __bool__ | Type casting |
A good rule of thumb: implement __repr__ on every class, __eq__ and __hash__ together when value equality matters, and __enter__/__exit__ whenever your class manages external resources. The rest depend on what behaviors your domain objects naturally support.
$Blueprint — Engineering Documentation·Section ID: PYTHON-MM·Revision: 1.0