internal/ and Project Layout

The internal directory is a private boundary the compiler enforces by import path, and a reason to delay folder structure.

Syntaximport "example.com/app/internal/store" // allowed only from inside example.com/app

A package in a directory named internal, or anywhere below one, can only be imported by code inside the tree rooted at that internal directory's parent. This is not a convention. The go command refuses the import and the build fails. The rule has applied to all code since Go 1.5.

So example.com/app/internal/store can be imported by example.com/app/web, or by anything else under example.com/app. Code in a different module cannot import it at all, even though the type Store inside it starts with a capital letter and is exported.

Why it exists

An unexported name hides something inside one package. internal hides a whole package inside one project. That lets your own packages talk to each other through wide, exported APIs while no stranger can build on them. Anything you place outside internal is a promise: other people may import it, and renaming it breaks their code.

Layout comes later

A new program needs no folders. One directory, package main, as many files as you like. Split a package out when something concrete forces it: a second program that needs the same code, a package too large to hold in your head, or a boundary you want checked.

Two directory names carry weight. cmd/api/main.go gives each program its own directory when a module builds more than one. internal/ holds everything not meant for outsiders. pkg/ means nothing at all to the compiler.

Example

Example · go
// canImport applies the rule the go command applies, so you can see it
// as plain string handling rather than as a mystery from the toolchain.
package main

import (
	"fmt"
	"strings"
)

// canImport reports whether a package at importer may import path.
// Only paths whose "internal" element is below the top level are judged;
// for anything else this returns true.
func canImport(importer, path string) bool {
	parts := strings.Split(path, "/")
	root := ""
	for i, p := range parts {
		if p == "internal" {
			root = strings.Join(parts[:i], "/") // internal's parent
			break
		}
	}
	if root == "" {
		return true
	}
	// Allowed only inside the tree rooted at that parent.
	return importer == root || strings.HasPrefix(importer, root+"/")
}

func main() {
	const p = "example.com/app/internal/token"
	fmt.Println(canImport("example.com/app/web", p))
	fmt.Println(canImport("example.com/other", p))
}

// Output:
// true
// false

When to use it

  • Sharing code between two binaries in one repository without publishing an API.
  • Stopping another team from importing a package you intend to redesign next month.
  • Deciding whether a new program needs any directories yet.
  • Reading the "use of internal package not allowed" error and knowing which tree the package belongs to.

More examples

The layout, and the error from outside it

A layout you can defend line by line: two programs under cmd/, their shared code under internal/, and one package outside internal because you intend other people to use it. The error message is the one to recognise when a colleague moves a package and your build stops.

Example · text
example.com/app/
  go.mod                      module example.com/app
  cmd/
    api/main.go               package main  -> the server binary
    migrate/main.go           package main  -> the migration tool
  internal/
    store/store.go            package store  (both binaries import it)
    token/token.go            package token
  httpapi/client.go           package httpapi  (public on purpose)

From cmd/api/main.go, this compiles:

  import "example.com/app/internal/store"

From another module, the same line fails to build:

  package example.com/other/cli imports example.com/app/internal/store:
  use of internal package example.com/app/internal/store not allowed

Growing into the structure, not starting with it

Shows that the cost of deferring layout is small and known: a move and some qualified names. Starting with empty cmd/, pkg/ and internal/ trees on day one costs more, because every early decision about which package owns what is made before you know.

Example · bash
# Day one. Nothing is wrong with this.
myapp/
  go.mod
  main.go
  handlers.go
  store.go

# Week three: a second program needs the storage code.
# Now the split has a reason, and it is a file move plus one import.
mkdir -p cmd/api cmd/migrate internal/store
git mv main.go handlers.go cmd/api/
git mv store.go internal/store/

# Edit store.go: package main  ->  package store
# Edit cmd/api: add import "example.com/myapp/internal/store"
#               and write store.New instead of New
go build ./...

internal is wider than unexported

The combination to reach for in an application: export freely within the module, and put the whole package under internal/ so the exported API is still private to the project. Use it for anything you expect to redesign, such as storage, auth or configuration types.

Example · go
// internal/token/token.go
package token

// Token and Sign are exported, so every package under example.com/app
// can use them. Nothing outside the module can, because of internal/.
type Token struct {
	UserID string
	Scopes []string
}

func Sign(t Token, key []byte) string {
	// ... signing happens here
	return ""
}

// This frees you to change Token, and to add or remove fields,
// knowing the only callers are in this repository.

Related

go packages basicsgo exported namesgo modules depthgo api structurego capstone design

Discussion

  • Be the first to comment on this lesson.