Bulkheads and Concurrency Limits
Cap how many goroutines may be inside one dependency at once, so a slow dependency cannot consume the whole process.
slots := make(chan struct{}, 8) // take: slots <- struct{}{} ; give back: <-slotsA ship is divided into sealed compartments called bulkheads. A hole in one floods that compartment only. A bulkhead in a service is the same idea: a limit on how much of your process any single dependency can occupy.
Without one, a dependency that slows from 10ms to 5s does not fail, it accumulates. Goroutines pile up inside it, and because every goroutine holds memory and often a connection, the slow dependency starves the handlers that have nothing to do with it.
A counting semaphore
The tool is a semaphore: a counter of permits, where taking a permit blocks once they are all out. In Go the direct way is a buffered channel used for its capacity and nothing else. make(chan struct{}, 8) creates a channel that holds eight values. Sending takes a permit, receiving gives it back. struct{}{} is an empty value that occupies no space, so only the count matters.
Two shorthands do the same job. errgroup.Group has SetLimit(n), which caps how many of its goroutines run at once. golang.org/x/sync/semaphore offers NewWeighted, where one job can take several permits because it is heavier than the others.
What trips people up
Size each bulkhead separately, per dependency. Decide what happens when it is full: wait with a deadline, or refuse immediately. Waiting with no limit just moves the queue one layer up.
Example
package main
import (
"errors"
"fmt"
)
var errBusy = errors.New("bulkhead full")
// bulkhead caps how many callers may be inside one dependency at once.
type bulkhead struct{ slots chan struct{} }
func newBulkhead(n int) *bulkhead {
return &bulkhead{slots: make(chan struct{}, n)}
}
// try runs work only if a permit is free. It never queues.
func (b *bulkhead) try(work func() error) error {
select {
case b.slots <- struct{}{}: // took a permit
default:
return errBusy // refuse rather than add another waiting goroutine
}
defer func() { <-b.slots }() // give the permit back on every path out
return work()
}
func main() {
b := newBulkhead(2)
// Stand in for two slow calls that are already in flight.
b.slots <- struct{}{}
b.slots <- struct{}{}
fmt.Println("third call:", b.try(func() error { return nil }))
<-b.slots // one of the two finishes
fmt.Println("after one finishes:", b.try(func() error { return nil }))
}
// Output:
// third call: bulkhead full
// after one finishes: <nil>When to use it
- Stop a slow search backend from occupying every goroutine in a service that also serves checkout.
- Cap a batch importer at eight concurrent API calls so it does not trip the provider's rate limit.
- Reserve a fixed share of in-flight work for an essential dependency, separate from optional ones.
- Bound a fan-out over a user-supplied list so a request with 10,000 ids cannot start 10,000 goroutines.
More examples
Waiting for a permit, but not forever
Refusing instantly is right at the edge of a service. Inside, it is often better to wait a little. The select waits on two things at once: a free permit, or the request's context ending. Whichever happens first wins, so a caller that has run out of budget stops queueing instead of joining the back of the line.
// run waits for a permit, or gives up when ctx is done.
func (b *bulkhead) run(ctx context.Context, work func() error) error {
select {
case b.slots <- struct{}{}:
case <-ctx.Done():
return ctx.Err() // out of budget; do not join the queue
}
defer func() { <-b.slots }()
return work()
}Limiting a fan-out with errgroup
A loop that starts one goroutine per item is a bulkhead with no wall: a thousand ids means a thousand concurrent requests. SetLimit caps the group, and g.Go blocks until a slot frees up, so the loop paces itself. Since Go 1.22 the loop variable is a fresh variable each iteration, so the old id := id copy is no longer needed.
func fetchAll(ctx context.Context, ids []string) error {
g, ctx := errgroup.WithContext(ctx)
g.SetLimit(8) // at most 8 requests in flight across the whole fan-out
for _, id := range ids {
g.Go(func() error {
return fetch(ctx, id)
})
}
return g.Wait() // nil only if every fetch succeeded
}The database pool is already a bulkhead
sql.DB is a pool of connections, and its maximum size is the bulkhead for your database: with 20 open connections at most, only 20 goroutines can be inside Postgres, and the rest wait in Go. Set it deliberately, because the default is unlimited, which lets one slow query open hundreds of connections and exhaust the server's own limit.
// At most 20 goroutines can be inside Postgres; the rest queue in Go.
db.SetMaxOpenConns(20)
// Keep the same number idle so a burst does not re-dial every time.
db.SetMaxIdleConns(20)
// Retire connections periodically so load balancers can rotate backends.
db.SetConnMaxLifetime(30 * time.Minute)
Discussion