Rate limiting
Use golang.org/x/time/rate to cap requests per client, and answer the ones you refuse with 429 and Retry-After.
limiter := rate.NewLimiter(rate.Limit(10), 20) // 10 per second, burst of 20A rate limiter decides how often one client may be served. It protects a login endpoint from password guessing, a database from a runaway script, and your bill from a loop someone forgot to stop.
The token bucket
The golang.org/x/time/rate package is maintained by the Go team but lives outside the standard library, so add it with go get golang.org/x/time/rate. It keeps a bucket of tokens. rate.NewLimiter(r, b) refills at r tokens per second, and the bucket holds at most b, so b is how large a sudden burst may be. Allow() takes one token and reports whether there was one to take. rate.Every(200*time.Millisecond) and rate.Limit(5) describe the same rate: one form says how often, the other says how many per second.
Per-client keys
A single limiter for the whole service punishes every user for one noisy client. Keep a map from a key to a *rate.Limiter, guarded by a sync.Mutex, and create the limiter the first time a key appears. Pick the most specific key you can trust: an API key or an account id, falling back to the client address. That map grows forever unless you drop entries you have not seen for a while.
Answering a refusal
Reply 429 Too Many Requests with a Retry-After header in whole seconds; Reserve().Delay() gives you that number. Do not call Wait in a handler, because it holds the connection open. Wait belongs in outbound code, where you are the one obeying someone else's limit.
Example
package main
import (
"fmt"
"net/http"
"net/http/httptest"
"sync"
"golang.org/x/time/rate"
)
var (
mu sync.Mutex
limiters = map[string]*rate.Limiter{}
)
// limiterFor hands out one bucket per client key, creating it on first sight.
func limiterFor(key string) *rate.Limiter {
mu.Lock()
defer mu.Unlock()
if limiters[key] == nil {
limiters[key] = rate.NewLimiter(1, 3) // 1 per second, burst of 3
}
return limiters[key]
}
func handle(w http.ResponseWriter, r *http.Request) {
if !limiterFor(r.Header.Get("X-Api-Key")).Allow() {
w.Header().Set("Retry-After", "1") // seconds to wait
http.Error(w, "too many requests", http.StatusTooManyRequests)
return
}
fmt.Fprint(w, "ok")
}
func main() {
for i := 0; i < 4; i++ {
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Header.Set("X-Api-Key", "k1")
rec := httptest.NewRecorder()
handle(rec, req)
fmt.Println(rec.Code)
}
}
// Output:
// 200
// 200
// 200
// 429
When to use it
- Cap an unauthenticated login endpoint at a few attempts per minute per address.
- Give each API key its own quota instead of one shared global ceiling.
- Stay under a third-party API's documented requests-per-second limit.
- Tell a refused client exactly how long to wait with Retry-After.
More examples
A Retry-After the client can actually use
Reserve tells you how long until a token is free, which turns a guessed Retry-After into a true one. The Cancel call matters: a reservation you abandon without cancelling still consumes its token, so the limiter drifts stricter than you configured.
package main
import (
"math"
"net/http"
"strconv"
"golang.org/x/time/rate"
)
func limit(lim *rate.Limiter, next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
res := lim.Reserve()
if !res.OK() {
// Burst is 0, so this event can never be allowed at all.
http.Error(w, "unavailable", http.StatusServiceUnavailable)
return
}
if d := res.Delay(); d > 0 {
// We are not going to wait, so return the token to the bucket.
res.Cancel()
secs := int(math.Ceil(d.Seconds()))
w.Header().Set("Retry-After", strconv.Itoa(secs))
http.Error(w, "too many requests", http.StatusTooManyRequests)
return
}
next.ServeHTTP(w, r)
})
}Obeying someone else's published limit
The outbound half of the subject. Wait is the right call here, because blocking a background worker costs nothing, while the alternative is getting 429s from the upstream service. Share one limiter across every goroutine that calls that API: a limiter is safe for concurrent use.
package main
import (
"context"
"fmt"
"net/http"
"golang.org/x/time/rate"
)
// fetchAll walks ids while staying under 5 requests per second, the rate the
// upstream API documents. Wait blocks until a token is free, or returns an
// error as soon as ctx is cancelled or its deadline passes.
func fetchAll(ctx context.Context, c *http.Client, ids []string) error {
lim := rate.NewLimiter(5, 1)
for _, id := range ids {
if err := lim.Wait(ctx); err != nil {
return err
}
url := "https://api.example.com/v1/items/" + id
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return err
}
resp, err := c.Do(req)
if err != nil {
return fmt.Errorf("fetch %s: %w", id, err)
}
resp.Body.Close()
}
return nil
}Dropping limiters for clients that went away
Without this, a public endpoint keyed by IP address is a slow memory leak: every address that ever called keeps a limiter forever. Deleting an idle entry is safe, because the next request simply creates a full bucket, which is what an absent client deserves.
package main
import (
"sync"
"time"
"golang.org/x/time/rate"
)
type client struct {
lim *rate.Limiter
seen time.Time
}
type store struct {
mu sync.Mutex
clients map[string]*client
}
func (s *store) get(key string) *rate.Limiter {
s.mu.Lock()
defer s.mu.Unlock()
c, ok := s.clients[key]
if !ok {
c = &client{lim: rate.NewLimiter(10, 20)}
s.clients[key] = c
}
c.seen = time.Now()
return c.lim
}
// sweep deletes idle entries. Run it in its own goroutine at start-up.
func (s *store) sweep(every, idle time.Duration) {
for range time.Tick(every) {
s.mu.Lock()
for key, c := range s.clients {
if time.Since(c.seen) > idle {
delete(s.clients, key)
}
}
s.mu.Unlock()
}
}
Discussion