In-Process Caching
Keep a map behind a mutex with an expiry time, serve reads in nanoseconds, and bound it so it cannot grow forever.
type cache struct { mu sync.Mutex; m map[string]entry } // entry holds a value plus its expiry timeA cache is a saved copy of an answer, kept somewhere fast, so the program does not have to work the answer out twice. In-process means the copy lives inside your own running program, in ordinary memory. Nothing travels over a network. A read is a map lookup: tens of nanoseconds, against a few milliseconds for a database query.
How it works
Three parts. A map, which is Go's table of keys to values. A lock, because several goroutines serve requests at once and a map cannot be written by two goroutines at the same moment; doing that crashes the program with concurrent map writes. And an expiry time on each entry, so an old answer stops being served. That expiry is called a TTL, short for time to live.
Keep the map unexported and private to the type. Every read and write goes through a method that takes the lock. Store the moment the entry dies rather than the duration it was given, so a read is one comparison against time.Now(). Treat an expired entry exactly like a missing one.
What trips people up
Nothing removes entries by itself. An expired entry still holds its memory until some code deletes it, and a key that is never read again is never noticed. A cache keyed by user id therefore grows for as long as the process runs. Bound it: delete expired keys on a timer, or cap the number of entries.
Each replica of your service holds its own copy, so two replicas can serve different values. In-process caching suits small, slow-changing data. Anything a user must see change immediately belongs in a shared cache.
Example
package main
import (
"fmt"
"sync"
"time"
)
// entry is one cached value and the moment it stops being valid.
type entry struct {
value string
expires time.Time
}
type cache struct {
mu sync.Mutex
m map[string]entry // guarded by mu
}
func (c *cache) set(key, value string, ttl time.Duration) {
c.mu.Lock()
defer c.mu.Unlock()
c.m[key] = entry{value: value, expires: time.Now().Add(ttl)}
}
// get returns the value and true only when the key is present and fresh.
func (c *cache) get(key string) (string, bool) {
c.mu.Lock()
defer c.mu.Unlock()
e, ok := c.m[key]
if !ok || time.Now().After(e.expires) {
return "", false
}
return e.value, true
}
func main() {
// map[string]entry{} is the same thing as make(map[string]entry).
c := &cache{m: map[string]entry{}}
c.set("user:7", "ada", time.Minute)
c.set("user:8", "grace", -time.Minute) // expired a minute ago
for _, k := range []string{"user:7", "user:8"} {
v, ok := c.get(k)
fmt.Printf("%s=%q fresh=%t\n", k, v, ok)
}
}
// Output:
// user:7="ada" fresh=true
// user:8="" fresh=falseWhen to use it
- Cache a feature-flag table that changes a few times a day but is read on every request.
- Hold parsed templates or compiled regular expressions so the parsing happens once per process.
- Remember the result of an expensive permission check for thirty seconds.
- Keep a small currency or country lookup in memory and refresh it on a timer.
More examples
A janitor that deletes expired entries
Deleting an expired entry when someone reads it is not enough, because a key nobody reads again is never visited. This goroutine walks the whole map on a timer and frees what has died. Deleting from a map while ranging over it is allowed in Go. Reach for this as soon as the key space is unbounded, such as one entry per user id, and tie it to the context you already cancel at shutdown so the goroutine ends with the process.
// janitor removes expired entries until ctx is cancelled. Start it once:
//
// go c.janitor(ctx, time.Minute)
func (c *cache) janitor(ctx context.Context, every time.Duration) {
t := time.NewTicker(every)
defer t.Stop() // releases the ticker as soon as the janitor returns
for {
select {
case <-ctx.Done():
return
case now := <-t.C:
c.mu.Lock()
for k, e := range c.m {
if now.After(e.expires) {
delete(c.m, k)
}
}
c.mu.Unlock()
}
}
}The same cache with type parameters
One cache type for the whole program instead of a copy per value type. The key type is constrained to comparable because map keys must be comparable. Use this once you want more than one cache in a service; a single string-to-string cache is not worth the extra notation.
// cache[K, V] works for any comparable key and any value type. The caller
// fills the types in, so nothing has to be converted back on the way out.
type cache[K comparable, V any] struct {
mu sync.Mutex
m map[K]entry[V]
}
type entry[V any] struct {
value V
expires time.Time
}
func newCache[K comparable, V any]() *cache[K, V] {
return &cache[K, V]{m: make(map[K]entry[V])}
}
func (c *cache[K, V]) set(k K, v V, ttl time.Duration) {
c.mu.Lock()
defer c.mu.Unlock()
c.m[k] = entry[V]{value: v, expires: time.Now().Add(ttl)}
}
func (c *cache[K, V]) get(k K) (V, bool) {
c.mu.Lock()
defer c.mu.Unlock()
e, ok := c.m[k]
if !ok || time.Now().After(e.expires) {
var zero V // the zero value of whatever V is
return zero, false
}
return e.value, true
}
// c := newCache[int, user]()The cache as a field on the server
The cache is a dependency like the database handle, not a package-level variable. That keeps it swappable in tests: give the test server a cache with a one-nanosecond TTL and every call goes to the database. Note the order, which is the whole pattern: look, load, store, return.
type server struct {
db *sql.DB
users *cache[int, string]
}
// userName reads the cache first and falls back to the database on a miss.
func (s *server) userName(ctx context.Context, id int) (string, error) {
if name, ok := s.users.get(id); ok {
return name, nil
}
var name string
err := s.db.QueryRowContext(ctx,
"SELECT name FROM users WHERE id = $1", id).Scan(&name)
if err != nil {
return "", err
}
s.users.set(id, name, 30*time.Second)
return name, nil
}
Discussion