Trusting nothing from the client

Limit body size, check types, confine file paths and vet outbound URLs, because every byte from a client is a claim.

Syntaxr.Body = http.MaxBytesReader(w, r.Body, 1<<20)

A request is a set of claims. Its length, the types of its fields, the file name it mentions, the URL it asks you to fetch: all of it was chosen by someone you do not know. Checking it is the defence, not a formality.

Size

An unbounded body is a memory bill. r.Body = http.MaxBytesReader(w, r.Body, 1<<20) replaces the body with one that fails past a million bytes. 1<<20 is shorthand for 1048576; both write the same number. Reads past the limit return a *http.MaxBytesError, which errors.As can detect so you can reply 413.

Shape and type

Decode into a struct, so a number cannot land where a string belongs. Call DisallowUnknownFields on the decoder, so a field you never declared is an error instead of silence. Then check the values themselves: lengths, ranges, allowed options.

Paths

filepath.Join cleans a path but does not confine it. Open user-named files through os.OpenRoot, added in Go 1.24: its Open refuses any name that would leave the directory, including by following a symbolic link.

Outbound URLs

If a client hands you a URL to fetch, your server becomes its proxy, and your server sits inside the private network. That attack is called SSRF. Require the https scheme, resolve the host, and reject loopback, private, unspecified and link-local addresses. A redirect can still point somewhere private, so check again in the client's CheckRedirect.

Example

Example · go
package main

import (
	"encoding/json"
	"errors"
	"fmt"
	"net/http"
	"net/http/httptest"
	"strings"
)

type note struct {
	Text string `json:"text"`
}

func create(w http.ResponseWriter, r *http.Request) {
	r.Body = http.MaxBytesReader(w, r.Body, 32) // tiny, to keep the demo short
	dec := json.NewDecoder(r.Body)
	dec.DisallowUnknownFields() // an undeclared field is an error

	var n note
	if err := dec.Decode(&n); err != nil {
		var tooBig *http.MaxBytesError
		if errors.As(err, &tooBig) {
			http.Error(w, "body too large", http.StatusRequestEntityTooLarge)
			return
		}
		http.Error(w, "bad request", http.StatusBadRequest)
		return
	}
	fmt.Fprintf(w, "stored %q\n", n.Text)
}

func main() {
	body := `{"text":"` + strings.Repeat("a", 64) + `"}`
	req := httptest.NewRequest(http.MethodPost, "/notes", strings.NewReader(body))
	rec := httptest.NewRecorder()
	create(rec, req)
	fmt.Print(rec.Code, " ", rec.Body.String())

}

// Output:
// 413 body too large

When to use it

  • Stop a two gigabyte JSON body from exhausting the server's memory.
  • Serve files by name without letting ../ reach your configuration directory.
  • Accept a webhook URL from a customer that cannot point at the cloud metadata service.
  • Reject an avatar upload that is really HTML with a .png name.

More examples

Serving user files from a directory they cannot leave

The modern answer to path traversal: stop sanitising strings and let the operating system refuse the escape. Try it with curl 'http://localhost:8080/files/..%2f..%2fetc%2fpasswd' and you get a 404 rather than the file. Before Go 1.24 this needed filepath.Clean plus a prefix check, which was easy to get subtly wrong.

Example · go
package main

import (
	"io"
	"log"
	"net/http"
	"os"
)

// os.Root (Go 1.24) refuses any name that would leave the directory, and it
// will not follow a symbolic link out of it either.
func download(root *os.Root) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		f, err := root.Open(r.PathValue("name")) // PathValue needs Go 1.22
		if err != nil {
			http.Error(w, "not found", http.StatusNotFound)
			return
		}
		defer f.Close()
		w.Header().Set("X-Content-Type-Options", "nosniff")
		io.Copy(w, f)
	}
}

func main() {
	root, err := os.OpenRoot("uploads")
	if err != nil {
		log.Fatal(err)
	}
	defer root.Close()

	mux := http.NewServeMux()
	mux.Handle("GET /files/{name}", download(root))
	log.Fatal(http.ListenAndServe(":8080", mux))
}

Checking a URL the client asked you to fetch

Run this before fetching any webhook URL, avatar URL or callback a customer typed in. IsLinkLocalUnicast is the one people forget: it covers 169.254.169.254, the address cloud instances use to hand out credentials. LookupNetIP returns netip.Addr values, which carry these checks as methods.

Example · go
package main

import (
	"context"
	"errors"
	"net"
	"net/url"
)

// checkOutbound rejects a client supplied URL that does not resolve to a
// public address. It is one layer, not the whole answer: the name could
// resolve differently when the request is actually made, so a hardened
// client also checks the address inside the dialer's Control function.
func checkOutbound(ctx context.Context, raw string) error {
	u, err := url.Parse(raw)
	if err != nil {
		return err
	}
	if u.Scheme != "https" {
		return errors.New("only https is allowed")
	}

	addrs, err := net.DefaultResolver.LookupNetIP(ctx, "ip", u.Hostname())
	if err != nil {
		return err
	}
	for _, a := range addrs {
		if a.IsLoopback() || a.IsPrivate() || a.IsUnspecified() || a.IsLinkLocalUnicast() {
			return errors.New("address is not public: " + a.String())
		}
	}
	return nil
}

An upload limited in size and checked by its bytes

Two limits do two jobs: MaxBytesReader caps the whole request, while the argument to ParseMultipartForm caps how much is held in memory before the rest spills to temporary files. DetectContentType reads the first 512 bytes, which is why head is that size. A short file is fine: io.EOF and io.ErrUnexpectedEOF simply mean there were fewer bytes to read.

Example · go
package main

import (
	"errors"
	"fmt"
	"io"
	"net/http"
)

func upload(w http.ResponseWriter, r *http.Request) {
	r.Body = http.MaxBytesReader(w, r.Body, 5<<20)       // 5 MiB in total
	if err := r.ParseMultipartForm(1 << 20); err != nil { // 1 MiB kept in memory
		http.Error(w, "bad upload", http.StatusBadRequest)
		return
	}
	f, hdr, err := r.FormFile("avatar")
	if err != nil {
		http.Error(w, "avatar is required", http.StatusBadRequest)
		return
	}
	defer f.Close()

	// The file name and the declared Content-Type are claims. Sniff instead.
	head := make([]byte, 512)
	n, err := io.ReadFull(f, head)
	if err != nil && !errors.Is(err, io.EOF) && !errors.Is(err, io.ErrUnexpectedEOF) {
		http.Error(w, "unreadable upload", http.StatusBadRequest)
		return
	}
	switch http.DetectContentType(head[:n]) {
	case "image/png", "image/jpeg":
	default:
		http.Error(w, "only png or jpeg", http.StatusUnsupportedMediaType)
		return
	}
	fmt.Fprintf(w, "accepted %d bytes\n", hdr.Size)
}

Related

go validationgo reading jsongo forms uploadsgo file servergo status codes

Discussion

  • Be the first to comment on this lesson.