The AWS SDK for Go v2
Load one aws.Config, build service clients from it, and pass a context to every call.
cfg, err := config.LoadDefaultConfig(ctx, config.WithRegion("eu-west-2"))AWS is a collection of services you call over the network: file storage, queues, databases. The AWS SDK for Go v2 is a set of Go packages that make those calls for you. You do not build the HTTP requests or sign them by hand.
The SDK is split into many modules. Config loading lives in github.com/aws/aws-sdk-go-v2/config. Each service is a module of its own, such as .../service/s3. A program that only talks to S3 therefore downloads only the S3 code.
Config first, clients second
config.LoadDefaultConfig returns an aws.Config: which region to talk to, where credentials come from, and how failures are retried. Load it once at start-up and give the same value to every client. A client is the value holding the methods for one service, built by that service's NewFromConfig. s3.NewFromConfig(cfg, fns...) is the shorthand: the longhand is filling in an s3.Options yourself and calling s3.New(opts). Same result.
Where credentials come from
Credentials prove who you are. The SDK looks for them in a fixed order: environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN, or AWS_WEB_IDENTITY_TOKEN_FILE), then the shared files under ~/.aws, then the role attached to an ECS task, then the role attached to an EC2 instance. That order is why one binary runs on your laptop with a profile and in a cluster with a role, unchanged. Keys never belong in source code.
Context on every call
Every operation takes a context.Context first. Inside an HTTP handler, pass the request's context. When the caller gives up, the AWS call is cancelled with it.
Example
package main
import (
"context"
"fmt"
"log"
"time"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/credentials"
"github.com/aws/aws-sdk-go-v2/service/s3"
)
func main() {
// Every SDK call takes a context. This one gives up after five seconds.
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
// Fixed credentials are used here so the printed output cannot change.
// Real programs leave this option out and let the chain find them.
creds := credentials.NewStaticCredentialsProvider("AKIAEXAMPLE", "secret", "")
cfg, err := config.LoadDefaultConfig(ctx,
config.WithRegion("eu-west-2"),
config.WithCredentialsProvider(creds),
)
if err != nil {
log.Fatal(err)
}
// One config builds many clients. Each one gets this region and these keys.
client := s3.NewFromConfig(cfg)
// Credentials are fetched on demand, not during LoadDefaultConfig.
got, err := cfg.Credentials.Retrieve(ctx)
if err != nil {
log.Fatal(err)
}
fmt.Println("region:", cfg.Region)
fmt.Println("key:", got.AccessKeyID)
fmt.Printf("client: %T\n", client)
}
// Output:
// region: eu-west-2
// key: AKIAEXAMPLE
// client: *s3.ClientWhen to use it
- Read a settings file from S3 at start-up using the EC2 or EKS role instead of a stored key.
- Set region, credentials and retry policy in one place and share it with every client in the service.
- Run integration tests against a local S3 server by overriding BaseEndpoint on a single client.
- Cancel an in-flight AWS call as soon as the HTTP request that triggered it is abandoned.
More examples
One config, two clients, and the request's context
This is the shape a real service uses. Loading the config costs a file read and sometimes a network lookup, so it happens once, in newApp, and the two clients are kept on a struct. Note the first argument to PutObject: r.Context(), not context.Background(). Pass Background() in a handler and an abandoned request leaves an AWS call running to completion, holding a connection nobody is waiting for.
type app struct {
s3 *s3.Client
sqs *sqs.Client
}
// newApp loads the config once at start-up and builds both clients from it.
func newApp(ctx context.Context) (*app, error) {
cfg, err := config.LoadDefaultConfig(ctx, config.WithRegion("eu-west-2"))
if err != nil {
return nil, fmt.Errorf("load aws config: %w", err)
}
return &app{s3: s3.NewFromConfig(cfg), sqs: sqs.NewFromConfig(cfg)}, nil
}
func (a *app) handleUpload(w http.ResponseWriter, r *http.Request) {
// r.Context() ends when the client disconnects, so the S3 call ends too.
_, err := a.s3.PutObject(r.Context(), &s3.PutObjectInput{
Bucket: aws.String("uploads"),
Key: aws.String(r.URL.Path),
Body: r.Body,
})
if err != nil {
http.Error(w, "upload failed", http.StatusBadGateway)
return
}
w.WriteHeader(http.StatusCreated)
}Adding the SDK to a module
Reach for this whenever you start talking to a new AWS service: one go get per service. Each service module carries its own version number and can be upgraded on its own, which is why go.mod ends up listing several aws-sdk-go-v2 lines rather than one. Keep them close together in version; a very old service module against a very new core module is the usual cause of a confusing compile error.
go get github.com/aws/aws-sdk-go-v2/config
go get github.com/aws/aws-sdk-go-v2/service/s3
go get github.com/aws/aws-sdk-go-v2/service/sqs
# The core module, github.com/aws/aws-sdk-go-v2, arrives as a dependency
# of those three. You rarely ask for it by name.
go mod tidyPointing one client at a local S3 server
Use this for integration tests against a local S3 implementation such as MinIO or LocalStack. The second argument to NewFromConfig is a function that receives the client's options and may change them, so the override is scoped to this one client. BaseEndpoint replaces the address the SDK would have worked out from the region, and UsePathStyle switches the URL from bucket.host/key to host/bucket/key, which local servers understand.
cfg, err := config.LoadDefaultConfig(ctx, config.WithRegion("eu-west-2"))
if err != nil {
log.Fatal(err)
}
// Per-client overrides. The config is untouched, so other clients keep
// talking to real AWS.
local := s3.NewFromConfig(cfg, func(o *s3.Options) {
o.BaseEndpoint = aws.String("http://localhost:4566")
o.UsePathStyle = true // local servers rarely accept bucket-name-in-host URLs
})
Discussion