lesson 248 Production Readiness Capstone · advanced

Find and fix the hot path

A full night is about forty thousand trips, and the report has started to take minutes. A performance review measures before changing anything, finds the one step that dominates, fixes its algorithm, and proves the output is unchanged.

You'll be able to

Profile the pipeline with cProfile and read call counts to find the hot path
Replace a repeated scan with an index built once, and show the work saved with a counted measure
Check that the optimised pipeline produces exactly the same result, and keep memory flat by streaming

The idea

The Algorithms & Complexity module–the Profiling & Optimization module gave you the tools: counting operations, timeit, cProfile, indexes, caches and streaming. A performance review on a real application puts them in order:

Measure the whole run on realistic input. Guessing is how people spend a day speeding up code that took 2% of the time.
Find the hot path: the function with the most calls or the most work. cProfile's call counts are exact and repeatable; times vary by machine.
Fix the algorithm, not the syntax: an index built once instead of a search per row; one pass instead of two; streaming instead of loading everything.
Prove nothing changed: the new result must equal the old one on the same input (your pipeline tests from L02 do this), and measure again.

The capstone's zone lookup is the classic case: the zone table is a list of 265 rows, searched from the top for every trip.

Run each example to see what it prints. Running examples doesn't count toward progress.

example-1.py · Profile the report

import cProfile
import pstats

BOROUGHS = ["Manhattan", "Brooklyn", "Queens", "Bronx", "Staten Island"]
ZONE_ROWS = [{"zone_id": z, "borough": BOROUGHS[z % 5]} for z in range(1, 266)]
TRIPS = [{"pickup_zone_id": (i * 37) % 265 + 1, "total_amount": 10 + i % 7} for i in range(800)]
examined = [0]

def borough_of(zone_id, zone_rows):
    for row in zone_rows:
        examined[0] += 1
        if row["zone_id"] == zone_id:
            return row["borough"]
    return "Unknown"

def summarise(trips, zone_rows):
    revenue = {}
    for trip in trips:
        borough = borough_of(trip["pickup_zone_id"], zone_rows)
        revenue[borough] = revenue.get(borough, 0) + trip["total_amount"]
    return revenue

profiler = cProfile.Profile()
before = profiler.runcall(summarise, TRIPS, ZONE_ROWS)
stats = pstats.Stats(profiler)
calls = sorted(((total, name) for (filename, line, name), (prim, total, own, cum, callers) in stats.stats.items() if filename != "~"), reverse=True)
for total, name in calls:
    print(name, total)
print("zone rows examined:", examined[0])

An "optimisation" that changes the answer isn't one. A tempting shortcut keys the index by borough name instead of zone ID, then can't look trips up at all:

example-2.py · A faster version with a different answer

ZONE_ROWS = [{"zone_id": 132, "borough": "Queens"}, {"zone_id": 186, "borough": "Manhattan"}]
TRIPS = [{"pickup_zone_id": 186, "total_amount": 64.19}, {"pickup_zone_id": 132, "total_amount": 70.0}]

def slow(trips, zone_rows):
    revenue = {}
    for trip in trips:
        borough = next((r["borough"] for r in zone_rows if r["zone_id"] == trip["pickup_zone_id"]), "Unknown")
        revenue[borough] = revenue.get(borough, 0) + trip["total_amount"]
    return revenue

def fast(trips, zone_rows):
    index = {row["borough"]: row["zone_id"] for row in zone_rows}
    revenue = {}
    for trip in trips:
        borough = index.get(trip["pickup_zone_id"], "Unknown")
        revenue[borough] = revenue.get(borough, 0) + trip["total_amount"]
    return revenue

assert fast(TRIPS, ZONE_ROWS) == slow(TRIPS, ZONE_ROWS), "the optimised report differs"
print("same result")

example-3.py · Build the index once, prove the result is the same

BOROUGHS = ["Manhattan", "Brooklyn", "Queens", "Bronx", "Staten Island"]
ZONE_ROWS = [{"zone_id": z, "borough": BOROUGHS[z % 5]} for z in range(1, 266)]
TRIPS = [{"pickup_zone_id": (i * 37) % 265 + 1, "total_amount": 10 + i % 7} for i in range(800)]

def summarise_scan(trips, zone_rows):
    revenue = {}
    for trip in trips:
        borough = "Unknown"
        for row in zone_rows:
            if row["zone_id"] == trip["pickup_zone_id"]:
                borough = row["borough"]
                break
        revenue[borough] = revenue.get(borough, 0) + trip["total_amount"]
    return revenue

def summarise_indexed(trips, zone_rows):
    borough_by_zone = {row["zone_id"]: row["borough"] for row in zone_rows}
    revenue = {}
    for trip in trips:
        borough = borough_by_zone.get(trip["pickup_zone_id"], "Unknown")
        revenue[borough] = revenue.get(borough, 0) + trip["total_amount"]
    return revenue

print(summarise_indexed(TRIPS, ZONE_ROWS) == summarise_scan(TRIPS, ZONE_ROWS))
print("lookups:", len(TRIPS), "dictionary lookups instead of", len(TRIPS), "scans of", len(ZONE_ROWS), "rows")
print(summarise_indexed(TRIPS, ZONE_ROWS))

The other resource is memory. Reading a night's export with f.readlines() or list(csv.DictReader(f)) holds every row at once; iterating the reader keeps one row at a time (the Iterators & Generators module, the Profiling & Optimization module), so memory stays flat however big the file gets. The second challenge measures that with tracemalloc.

Mistakes to avoid: optimising without a profile; micro-tweaks (local variable caching, string tricks) before fixing the algorithm; rebuilding the index inside the loop; comparing timings from one noisy run; and skipping the old-versus-new comparison.

Review

Measure the whole run first; cProfile's call counts show the hot path exactly.
Fix the algorithm: an index built once instead of a scan per row; stream rows instead of loading them all.
Prove the optimised version gives the same result on the same input, then measure again.
Next: make the fast, correct pipeline survive bad rows, flaky storage and crashes, all at once.

Practise this