functools: lru_cache, partial & more

Reach for cached_property, lru_cache, partial and reduce to write faster, cleaner functional code.

The functools module is a toolbox of higher-order helpers. Three of them will change how you write code.

@lru_cache — memoise expensive calls

Decorate a pure function and its results are cached by arguments. A naively recursive Fibonacci goes from exponential to linear with one line:

from functools import lru_cache

@lru_cache(maxsize=None)
def fib(n):
    return n if n < 2 else fib(n - 1) + fib(n - 2)

print(fib(50))        # instant, thanks to caching

partial — pre-fill arguments

partial(func, arg) returns a new callable with some arguments already supplied. It is the clean way to adapt a general function into a specific one — perfect for callbacks and map().

reduce — fold a sequence to one value

reduce(op, items, start) repeatedly combines items into a single result. Most of the time a built-in like sum() or max() is clearer, but reduce handles the custom cases.

@cached_property

Like @property, but the result is computed once and stored — later accesses are free. Ideal for a derived value that is costly and never changes.

Example

from functools import lru_cache, partial, reduce, cached_property

# 1) Memoised recursion — exponential becomes linear
@lru_cache(maxsize=None)
def fib(n):
    return n if n < 2 else fib(n - 1) + fib(n - 2)

print('fib(50)     :', fib(50))
print('cache stats :', fib.cache_info().hits, 'hits')

# 2) partial: turn a general function into specific ones
def power(base, exp):
    return base ** exp

square = partial(power, exp=2)
cube   = partial(power, exp=3)
print('squares     :', list(map(square, [1, 2, 3, 4])))
print('cubes       :', list(map(cube, [1, 2, 3])))

# 3) reduce: fold a list into a running product
product = reduce(lambda acc, x: acc * x, [1, 2, 3, 4, 5], 1)
print('5! via reduce:', product)

# 4) cached_property: compute once, reuse forever
class Dataset:
    def __init__(self, values): self.values = values
    @cached_property
    def stats(self):
        print('  (computing stats once...)')
        return {'sum': sum(self.values), 'max': max(self.values)}

d = Dataset([3, 1, 4, 1, 5])
print('stats       :', d.stats)   # computes
print('stats again :', d.stats)   # cached, no recompute

# Output:
# fib(50)     : 12586269025
# cache stats : 48 hits
# squares     : [1, 4, 9, 16]
# cubes       : [1, 8, 27]
# 5! via reduce: 120
#   (computing stats once...)
# stats       : {'sum': 14, 'max': 5}
# stats again : {'sum': 14, 'max': 5}

When to use it

A Fibonacci function uses @lru_cache to memoize recursive calls, cutting exponential time to linear.
A URL builder uses functools.partial to pre-fill a base URL and only vary the path on each call.
A reduce() call computes a running product of a list of factors without an explicit loop.

More examples

lru_cache for memoisation

Caches the result of each unique argument so recursive fib() runs in O(n) instead of O(2^n).

from functools import lru_cache

@lru_cache(maxsize=None)
def fib(n):
    if n < 2: return n
    return fib(n - 1) + fib(n - 2)

print(fib(50))             # 12586269025
print(fib.cache_info())    # hits, misses, maxsize, currsize

partial to fix arguments

Creates square() and cube() by pre-filling the 'exponent' argument with partial().

from functools import partial

def power(base, exponent):
    return base ** exponent

square = partial(power, exponent=2)
cube   = partial(power, exponent=3)

print(square(5))   # 25
print(cube(3))     # 27

reduce for cumulative operations

Uses reduce() to accumulate a product and to join path segments without an explicit loop.

from functools import reduce
import operator

factors = [1, 2, 3, 4, 5]
product = reduce(operator.mul, factors)
print(product)   # 120

paths = ['home', 'alice', 'documents']
path = reduce(lambda a, b: a + '/' + b, paths)
print(path)   # home/alice/documents

Discussion

Be the first to comment on this lesson.