Generators & yield
Produce values lazily, one at a time, so you can process streams bigger than memory.
A generator is a function that yields values instead of returning once. Each yield hands a value to the caller and pauses the function, keeping all its local state. On the next request it resumes right where it left off.
def count_up(n):
i = 0
while i < n:
yield i # pause here, hand out i
i += 1
for x in count_up(3):
print(x) # 0, 1, 2Why this is powerful
Generators are lazy: they compute each value only when asked and never hold the whole sequence in memory. That means you can iterate over a ten-gigabyte file, an infinite sequence, or a live stream while using almost no RAM.
Generator expressions
The comprehension's leaner cousin uses parentheses: sum(x*x for x in range(1000)) streams the squares straight into sum() without building a list first.
yield from
yield from another_generator() delegates to a sub-generator, flattening nested producers cleanly.
Example
When to use it
- A large CSV reader uses a generator to yield one parsed row at a time, keeping memory flat.
- An infinite ID generator yields successive unique integers for new database rows on demand.
- A chained data pipeline uses generator expressions to filter and transform records without materialising intermediate lists.
More examples
Generator with send() for coroutine
Uses send() to push values into a generator, enabling a stateful running total without a class.
def accumulator():
total = 0
while True:
value = yield total
if value is None:
break
total += value
acc = accumulator()
next(acc) # prime the generator
print(acc.send(10)) # 10
print(acc.send(20)) # 30
print(acc.send(5)) # 35yield from to delegate
Uses 'yield from' to delegate to a recursive generator call, flattening an arbitrarily nested list.
def flatten(nested):
for item in nested:
if isinstance(item, list):
yield from flatten(item)
else:
yield item
data = [1, [2, [3, 4]], 5, [6]]
print(list(flatten(data))) # [1, 2, 3, 4, 5, 6]Chained generator pipeline
Composes three generators into a lazy pipeline; no intermediate list is ever created.
def read_ints(n):
yield from range(n)
def only_even(seq):
return (x for x in seq if x % 2 == 0)
def squared(seq):
return (x ** 2 for x in seq)
pipeline = squared(only_even(read_ints(10)))
print(list(pipeline)) # [0, 4, 16, 36, 64]
Discussion