lesson 195 Async Python · advanced

Async HTTP workflows: limits, timeouts and retries

A real client combines everything: many requests at once under a limit, a timeout on each, retries for temporary failures, and a report that keeps every result. The asyncio parts are the same whatever HTTP library you use.

Simulated: this runs against results recorded on real Python.

You'll be able to

Await an async HTTP client's response, check its status and parse JSON
Retry only temporary failures (timeouts, 5xx), a limited number of times, waiting longer each time
Fetch many URLs under a semaphore and report failures without losing results

The idea

On a computer, async HTTP comes from a library such as httpx or aiohttp: response = await client.get(url), then the status and response.json(). There is no network in this page, so the lesson uses a small stand-in, SavedTransport (in saved_http.py for the exercises): await transport.get(url) waits for the latency given with a saved answer, on the event loop's clock, and returns a response with status and json(). The answers are trimmed copies of real USGS earthquake API data. Everything above the transport is the asyncio you have learned.

Run each example to see what it prints. Running examples doesn't count toward progress.

example-1.py · Three days of earthquakes, fetched together

import asyncio

class BrowserLoop(asyncio.BaseEventLoop):  # a real asyncio loop whose clock jumps instead of waiting
    def __init__(self):
        super().__init__()
        self._clock, self._selector = 0.0, self
    def time(self):
        return self._clock
    def select(self, timeout):
        self._clock += timeout
        return []
    def _process_events(self, events):
        pass
    def _write_to_self(self):
        pass

def run(main):  # on a computer: asyncio.run(main)
    outer = asyncio.events._get_running_loop()
    asyncio.events._set_running_loop(None)
    try:
        return asyncio.run(main, loop_factory=BrowserLoop)
    finally:
        asyncio.events._set_running_loop(outer)

import json

class Response:
    def __init__(self, status, body):
        self.status = status
        self.text = json.dumps(body)

    def json(self):
        return json.loads(self.text)

class SavedTransport:
    def __init__(self, saved):
        self.saved = saved

    async def get(self, url):
        status, body, seconds = self.saved[url]
        await asyncio.sleep(seconds)
        return Response(status, body)

API = "https://earthquake.usgs.gov/fdsnws/event/1/query?format=geojson&minmagnitude=4&date="
saved = {
    API + "2025-03-01": (200, {"features": [{"properties": {"mag": 4.7, "place": "Arctic Ocean"}}, {"properties": {"mag": 4.0, "place": "31 km ENE of Firá, Greece"}}]}, 0.4),
    API + "2025-03-02": (503, {"error": "Service Unavailable"}, 0.1),
    API + "2025-03-03": (200, {"features": [{"properties": {"mag": 4.3, "place": "126 km NE of Tobelo, Indonesia"}}]}, 0.3),
}

async def quakes_on(transport, day):
    response = await transport.get(API + day)
    if response.status != 200:
        raise RuntimeError(f"HTTP {response.status} for {day}")
    return [f["properties"]["place"] for f in response.json()["features"]]

async def main():
    loop = asyncio.get_running_loop()
    transport = SavedTransport(saved)
    days = ["2025-03-01", "2025-03-02", "2025-03-03"]
    results = await asyncio.gather(*(quakes_on(transport, d) for d in days), return_exceptions=True)
    for day, result in zip(days, results):
        print(day, result)
    print(loop.time())

run(main())

Retry only what can succeed later. Timeouts, 429 *Too Many Requests* and 5xx server errors are often temporary; 404 or 400 will fail the same way every time. Retry a limited number of times and wait longer between attempts (exponential backoff: 0.5 s, 1 s, 2 s…), so a struggling server gets room to recover. Each attempt also gets its own timeout.

example-2.py · Retry with backoff and a timeout per attempt

import asyncio

class BrowserLoop(asyncio.BaseEventLoop):  # a real asyncio loop whose clock jumps instead of waiting
    def __init__(self):
        super().__init__()
        self._clock, self._selector = 0.0, self
    def time(self):
        return self._clock
    def select(self, timeout):
        self._clock += timeout
        return []
    def _process_events(self, events):
        pass
    def _write_to_self(self):
        pass

def run(main):  # on a computer: asyncio.run(main)
    outer = asyncio.events._get_running_loop()
    asyncio.events._set_running_loop(None)
    try:
        return asyncio.run(main, loop_factory=BrowserLoop)
    finally:
        asyncio.events._set_running_loop(outer)

async def get_with_retries(get, url, attempts, timeout, backoff):
    for attempt in range(attempts):
        try:
            async with asyncio.timeout(timeout):
                status, body = await get(url)
            if status < 500:
                return status, body
        except TimeoutError:
            pass
        if attempt < attempts - 1:
            await asyncio.sleep(backoff * 2 ** attempt)
    raise RuntimeError(f"gave up on {url} after {attempts} attempts")

answers = [("hang", None), (503, None), (200, {"count": 466})]

async def flaky_get(url):
    kind, body = answers.pop(0)
    if kind == "hang":
        await asyncio.sleep(60)
    await asyncio.sleep(0.1)
    return kind, body

async def main():
    loop = asyncio.get_running_loop()
    print(await get_with_retries(flaky_get, "/count", attempts=3, timeout=1, backoff=0.5), loop.time())

run(main())

Putting it together: a semaphore bounds how many requests are in flight, each request has a timeout and a small retry budget, and the caller gathers outcomes without letting one failure discard the rest. Mistakes to avoid: retrying 4xx errors (they won't change); retrying forever or without backoff (a retry storm); a single timeout around the whole batch when you meant one per request; and creating a new client per request on a real HTTP library (reuse one, it keeps connections open).

Review

Check the status before parsing; turn errors into clear exceptions.
Retry only temporary failures, a limited number of times, with growing waits and a timeout per attempt.
Semaphore + gather + a report of failures = a robust async client.
Next module: organising a growing program into well-separated parts.

Practise this