lesson 183 Profiling & Optimization · advanced

Caching: remember results instead of recomputing

When a function is called again with the same arguments and always returns the same result, it can remember the answer. functools.cache and lru_cache do it in one line, and cache_info proves it works.

You'll be able to

Apply functools.cache and lru_cache(maxsize=…) and read cache_info()
Explain how a bounded cache evicts the least recently used entry
Recognise when caching is unsafe: impure functions, unhashable arguments, stale data

The idea

“Recursion trade-offs: depth and repeated work” showed ways(n) recomputing the same subproblems exponentially often. If a function's result depends only on its arguments (a pure function, the Functional & Dynamic Python module), it can store each result in a dictionary keyed by the arguments and look it up next time. functools.cache is a decorator that does exactly that. func.cache_info() reports hits (answers from the cache), misses (real calls) and the current size.

Run each example to see what it prints. Running examples doesn't count toward progress.

example-1.py · The staircase, cached

from functools import cache

calls = {"n": 0}

@cache
def ways(steps):
    calls["n"] += 1
    if steps <= 1:
        return 1
    return ways(steps - 1) + ways(steps - 2)

print(ways(40), calls["n"])
print(ways.cache_info())
print(ways(40), calls["n"], ways.cache_info().hits)

functools.cache never forgets, which is fine for a small set of arguments but a memory leak for unbounded ones. lru_cache(maxsize=n) keeps at most n results and, when full, evicts the least recently used one, so arguments asked for often stay cached.

example-2.py · Exchange rates with a two-entry cache

from functools import lru_cache

fetches = []

@lru_cache(maxsize=2)
def rate(currency):
    fetches.append(currency)
    return {"EUR": 0.92, "GBP": 0.79, "JPY": 151.3}[currency]

for currency in ["EUR", "GBP", "EUR", "JPY", "GBP", "EUR"]:
    rate(currency)
print(fetches)
print(rate.cache_info())

example-3.py · Arguments must be hashable

from functools import cache

@cache
def basket_total(prices):
    return sum(prices)

print(basket_total([7.49, 31.0]))

When to use it: expensive pure functions called repeatedly with the same arguments: recursive computations, parsing the same values, lookups that don't change during a run. When not to: functions with side effects (a cached send_email sends once), results that depend on something besides the arguments (the clock, a database, a global that changes: the cache keeps returning the stale answer unless you call cache_clear()), and arguments that are rarely repeated (the cache only costs memory). Methods are a special case: caching on self keeps every object alive (L36), so cache module-level functions or use a per-object dictionary.

Review

@cache and @lru_cache(maxsize=n) remember results keyed by the (hashable) arguments.
cache_info() shows hits, misses and size; a bounded cache evicts the least recently used entry.
Cache only pure functions; for data that changes, use expiry or cache_clear().
Next: making file input and output efficient.

Practise this