lesson 181 Profiling & Optimization · advanced

timeit: measuring small pieces of code fairly

One stopwatch reading is mostly noise. timeit runs code thousands of times, in several rounds, and the best round per call is the number worth comparing.

You'll be able to

Use timeit.timeit and timeit.repeat with a function or lambda and a number of runs
Turn repeat results into a per-call time using the minimum, and explain why
Compare two implementations by a ratio, on the same input, and not trust tiny differences

The idea

Timing a single call is unreliable: it may take microseconds, the clock has limited resolution, and the computer is doing other things. timeit solves this by running the code number times in a loop and returning the total seconds. timeit.repeat(..., repeat=5) does that several times and returns a list of totals.

Take the minimum of those totals: the fastest round is the one least disturbed by other work, so it is the best estimate of the code's own cost. Divide by number for a per-call time. Then compare implementations by their ratio: a 30× difference survives any machine; a 5% one may not even survive a second run.

Run each example to see what it prints. Running examples doesn't count toward progress.

example-1.py · Looking up a SKU: list against set

import timeit

codes = [f"SKU-{n:05}" for n in range(5000)]
as_list = codes
as_set = set(codes)
list_time = min(timeit.repeat(lambda: "SKU-04999" in as_list, number=2000, repeat=3))
set_time = min(timeit.repeat(lambda: "SKU-04999" in as_set, number=2000, repeat=3))
print(list_time > set_time, list_time / set_time > 20)

A fair benchmark measures the same work on the same input, keeps setup (building the data) outside the timed code, and repeats enough to see past noise. Wrapping that in a function makes it reusable. Passing the clock in as a parameter (timer=time.perf_counter) is the the Advanced OOP & Design module dependency-injection idea: the real program uses the real clock, and a test can pass a fake one.

example-2.py · A reusable benchmark helper

import time
import timeit

def best_per_call(func, number, repeat, timer=time.perf_counter):
    totals = timeit.Timer(func, timer=timer).repeat(repeat=repeat, number=number)
    return min(totals) / number

def dedupe_slow(items):
    kept = []
    for item in items:
        if item not in kept:
            kept.append(item)
    return kept

def dedupe_fast(items):
    return list(dict.fromkeys(items))

skus = [f"SKU-{n % 400}" for n in range(2000)]
print(dedupe_slow(skus) == dedupe_fast(skus))
slow = best_per_call(lambda: dedupe_slow(skus), number=5, repeat=3)
fast = best_per_call(lambda: dedupe_fast(skus), number=5, repeat=3)
print(slow / fast > 10)

When to use it: to decide between two ways of writing a small, hot piece of code, or to confirm that an optimisation helped. Mistakes to avoid: timing one call; including setup in the timed code; comparing results from different inputs or machines; using the average of noisy rounds instead of the minimum; and celebrating a 3% difference. For a whole program, measure *where* the time goes first (cProfile, next lesson).

Review

timeit runs code number times; repeat gives several rounds; best round ÷ number = per-call time.
Same input, setup outside the timing, results checked equal, compare by ratio.
Inject the clock (timer=) so timing code can be tested exactly.
Next: cProfile, which shows where a whole program spends its time.

Practise this