Queues with SQS
Send and receive SQS messages, understand why a message can arrive twice, and write a consumer that handles duplicates safely.
const { Messages } = await sqs.send(new ReceiveMessageCommand({ QueueUrl, MaxNumberOfMessages: 10, WaitTimeSeconds: 20 }));Amazon SQS is a managed queue. A queue holds messages until something reads them. A producer sends; a consumer receives, does the work, then deletes. The point is that the producer does not wait: an HTTP handler can accept an order in milliseconds and leave the slow part to a worker.
Sending
SendMessageCommand takes QueueUrl and MessageBody. The body is a string, so serialise objects yourself with JSON.stringify. The aggregated client offers the same call in shorthand:
// longhand
await sqs.send(new SendMessageCommand({ QueueUrl, MessageBody: json }));
// shorthand, identical operation
await sqs.sendMessage({ QueueUrl, MessageBody: json });Receiving
ReceiveMessageCommand takes MaxNumberOfMessages (1 to 10) and WaitTimeSeconds (up to 20). A wait time above zero is long polling: the call holds open until a message arrives or the wait runs out, which costs far fewer requests than asking repeatedly. Each message has MessageId, Body and ReceiptHandle.
Visibility timeout and duplicates
Receiving does not remove a message. It hides it from other consumers for the visibility timeout, 30 seconds by default. Delete it with DeleteMessageCommand({ QueueUrl, ReceiptHandle }) when the work is done. If your worker crashes first, the message reappears and another worker takes it. That safety net is why SQS delivers at least once: the same message can be processed twice. So make the handler idempotent — running it twice has the same effect as running it once. Record a business key, such as an order ID, with a unique constraint, and skip work you have already recorded.
Example
When to use it
- Accept an order in the request and send the confirmation email from a worker.
- Spread a batch of thousands of image conversions across several worker containers.
- Keep a slow third-party API call out of the request path so a timeout there does not fail the user's request.
- Make a payment worker safe against the same message being delivered twice.
More examples
A long-polling consumer loop
The standard worker shape. Long polling with WaitTimeSeconds 20 means an idle queue costs three requests a minute instead of thousands. Deleting after success and not before is what makes a crash recoverable. Flipping the flag on SIGTERM lets the container finish its current batch instead of abandoning ten half-processed messages.
import {
SQSClient, ReceiveMessageCommand, DeleteMessageCommand,
} from "@aws-sdk/client-sqs";
const sqs = new SQSClient({});
const QueueUrl = process.env.ORDERS_QUEUE_URL;
let running = true;
process.on("SIGTERM", () => { running = false; }); // stop after this batch
while (running) {
const out = await sqs.send(new ReceiveMessageCommand({
QueueUrl,
MaxNumberOfMessages: 10, // 10 is the maximum per call
WaitTimeSeconds: 20, // 20 is the maximum; this is long polling
}));
// No Messages property at all when nothing arrived in those 20 seconds.
for (const msg of out.Messages ?? []) {
try {
await handle(JSON.parse(msg.Body));
// Delete only after the work succeeded.
await sqs.send(new DeleteMessageCommand({
QueueUrl, ReceiptHandle: msg.ReceiptHandle,
}));
} catch (err) {
// Leave it undeleted: it becomes visible again and is retried.
console.error("handler failed", msg.MessageId, err);
}
}
}Holding a long job with ChangeMessageVisibility
Reach for this when job duration varies and a long tail would otherwise be processed twice. VisibilityTimeout sets the hidden period measured from the moment of the call, so 60 means sixty more seconds rather than sixty on top of the remainder. Keep the renewal interval well under the timeout, and clear it in a finally block or the process will not exit.
import { ChangeMessageVisibilityCommand } from "@aws-sdk/client-sqs";
// A job that takes longer than the queue's visibility timeout would be
// redelivered to another worker while this one is still working on it.
// Push the deadline out while you make progress.
async function withHeartbeat(sqs, QueueUrl, ReceiptHandle, work) {
const timer = setInterval(() => {
sqs.send(new ChangeMessageVisibilityCommand({
QueueUrl,
ReceiptHandle,
VisibilityTimeout: 60, // seconds from now, not added to what is left
})).catch((err) => console.error("heartbeat failed", err));
}, 30_000);
try {
return await work();
} finally {
clearInterval(timer); // always stop the timer, success or failure
}
}Sending a batch
Ten messages per request instead of ten requests, which matters when a job fans out into thousands. The trap is the response: a batch send can succeed for some entries and fail for others, and the call itself still resolves. Code that ignores Failed silently drops messages, so always read both lists and requeue the failures.
import { SendMessageBatchCommand } from "@aws-sdk/client-sqs";
// Up to 10 messages in one request. Each entry needs an Id that is unique
// within the batch; it is only used to match up the results.
const out = await sqs.send(new SendMessageBatchCommand({
QueueUrl,
Entries: orders.slice(0, 10).map((order, i) => ({
Id: String(i),
MessageBody: JSON.stringify(order),
})),
}));
// A batch can partly fail, so both lists must be read.
for (const ok of out.Successful ?? []) console.log("sent", ok.MessageId);
for (const bad of out.Failed ?? []) {
console.error("not sent", bad.Id, bad.Code, bad.SenderFault);
}
Discussion