Why Async Exists

Async reclaims the time a program spends waiting on IO; it adds nothing to work that is busy computing.

Syntaxasyncio.run(main()) # one thread, one event loop, many overlapping waits

Most slow programs are not computing anything. They are waiting: for an HTTP response, a database round trip, a disk read. A script that makes twenty calls at 100ms each spends two seconds holding sockets open and almost no CPU. Async exists to reclaim that waiting.

What the event loop actually does

An await on something that is not ready yet is a suspension point. The loop parks that coroutine, runs another one, and comes back when the socket is readable. One thread, one interpreter, no parallelism — just the gaps filled in. Twenty overlapping requests take about as long as the slowest one instead of the sum of all twenty.

What it does not do

Nothing inside async def is faster. A function that hashes a password or parses a 50MB document never reaches a suspension point, so the loop cannot run anything else while it works: you have not added concurrency, you have added one task that stalls every other task in the process. CPU-bound work belongs in a process pool; a blocking library with no async API belongs in a thread.

The cost

Async is contagious. await is only legal inside async def, so one async call at the bottom pushes async up through every caller, and a synchronous HTTP library halfway up that chain defeats the whole arrangement. Converting a script that makes three sequential calls is rarely worth it; converting one that fans out to hundreds always is.

Example

Try it yourself
Loading editor…
Press Run to execute the code.

When to use it

  • Pulling 200 product pages from a vendor API in a nightly sync that currently takes 20 minutes.
  • A request handler that calls three internal services before it can answer, serially today.
  • Deciding whether rewriting a slow report generator as async will help at all, before doing the work.
  • Draining a queue where each message costs one network round trip to acknowledge.

More examples

The same fan-out with a real HTTP client

The shape every async IO program takes: one client held open, a coroutine per unit of work, one gather. Twenty 100ms calls finish in roughly 100ms rather than two seconds. Reach for this when the work is independent and the time is spent on the wire.

Example · python
# sync_items.py
import asyncio
import time

import httpx

URLS = [f"https://api.example.com/items/{n}" for n in range(20)]

async def fetch(client, url):
    r = await client.get(url, timeout=5.0)
    r.raise_for_status()
    return r.json()              # httpx decodes synchronously

async def main():
    # one client, so all 20 requests share one connection pool
    async with httpx.AsyncClient() as client:
        start = time.perf_counter()
        bodies = await asyncio.gather(*(fetch(client, u) for u in URLS))
        print(len(bodies), "bodies in", round(time.perf_counter() - start, 2), "s")

asyncio.run(main())

Async buys nothing for CPU-bound work

Both functions are coroutines and only one of them is concurrent. The test for whether async will help is not how slow the function is, it is whether the function waits. If profiling shows CPU time rather than wall-clock wait, async is the wrong tool and a process pool is the right one.

Example · python
import asyncio
import hashlib
from concurrent.futures import ProcessPoolExecutor

def derive(password):
    # about 100ms of pure CPU - there is no await point to be had in here
    return hashlib.scrypt(password.encode(), salt=b"demo", n=16384, r=8, p=1).hex()

async def wrong(passwords):
    # async def changes nothing: each call holds the loop for its full 100ms
    return [derive(p) for p in passwords]

async def right(passwords):
    loop = asyncio.get_running_loop()
    with ProcessPoolExecutor() as pool:
        # separate processes, separate interpreters, real cores
        return await asyncio.gather(
            *(loop.run_in_executor(pool, derive, p) for p in passwords)
        )

The colour problem, and the escape hatch

One synchronous dependency in the middle of an async call stack cancels the benefit for the whole process. Until the dependency is replaced with an async client, asyncio.to_thread (3.9+) moves the blocking call off the loop thread. Useful when you are adding async to an existing codebase one layer at a time.

Example · python
import asyncio

import requests          # synchronous: no async API at all

def legacy_lookup(sku):
    return requests.get(f"https://api.example.com/sku/{sku}", timeout=5).json()

async def handler(skus):
    # calling legacy_lookup directly here would block every other request
    # being served by this process for the duration of each HTTP call
    return await asyncio.gather(*(asyncio.to_thread(legacy_lookup, s) for s in skus))

Related

py async basicspy async gatherpy async httppy async threadspy async pitfalls

Discussion

  • Be the first to comment on this lesson.