Message Brokers
Publish and consume messages with the async RabbitMQ.Client 7 API, use acks, prefetch and dead letters correctly, and map them to Amazon SQS.
await channel.BasicPublishAsync("orders", "order.placed", mandatory: true,
new BasicProperties { Persistent = true, MessageId = id }, body);
await channel.BasicAckAsync(ea.DeliveryTag, multiple: false);A broker decouples the service that records a fact (an order was placed) from the services that react to it (inventory, email, analytics). It stores messages durably, delivers them to consumers and redelivers when a consumer fails. Every mainstream broker delivers at least once; design consumers for duplicates.
RabbitMQ model
Publishers send to an exchange with a routing key; bindings route copies into queues; consumers read queues. A topic exchange orders with routing key order.placed can feed an inventory.order-placed queue and an email.order-placed queue, each with its own consumers. Use quorum queues (x-queue-type: quorum) for anything that matters: they are replicated and durable.
RabbitMQ.Client 7
Version 7 is async throughout. IModel became IChannel; you call CreateConnectionAsync, CreateChannelAsync, BasicPublishAsync, and consume with AsyncEventingBasicConsumer and its ReceivedAsync event. Rules that hold in production:
- One long-lived connection per process (a singleton), channels per consumer and a long-lived publishing channel; never a connection or a channel per message.
- Publish with confirms:
new CreateChannelOptions(publisherConfirmationsEnabled: true, publisherConfirmationTrackingEnabled: true)makesBasicPublishAsynccomplete only when the broker has accepted the message, and fail if it was nacked or, withmandatory: true, unroutable. Without confirms a publish "succeeds" into a TCP buffer. - Mark messages
Persistent = trueand setMessageId, so consumers can de-duplicate. - Consume with
autoAck: falseand ack after the work is committed. Auto-ack removes the message the moment it is delivered, so a crash loses it. - Set a prefetch with
BasicQosAsync(0, 20, false). Without one, the broker pushes the whole queue to the first consumer. ea.Bodyis only valid inside the handler; copy it if work continues afterwards.
Failures and dead letters
| Consumer outcome | Call | Result |
|---|---|---|
| Processed | BasicAckAsync | Removed from the queue |
| Transient failure (database down) | Wait, then BasicNackAsync(tag, false, requeue: true); or requeue: false into a retry queue with a message TTL that dead-letters back | Redelivered with Redelivered = true, after the delay |
| Permanent failure (invalid payload) | BasicNackAsync(tag, false, requeue: false) | Sent to the queue's dead-letter exchange |
| Consumer crashes, channel closes | none | Unacked messages are redelivered to another consumer |
An immediate requeue has no backoff: a blip burns through the delivery limit in milliseconds and dead-letters good messages. Delay before the nack, use a TTL retry queue, or cancel the consumer while a dependency is down. On RabbitMQ 4, quorum queues default to a delivery limit of 20, then drop the message unless the queue has a dead-letter exchange. Always set x-dead-letter-exchange and x-delivery-limit, and monitor and replay the dead-letter queue.
Managed queues: Amazon SQS
SQS has no exchanges and no connections to manage. ReceiveMessage returns up to 10 messages; use long polling (WaitTimeSeconds = 20). A received message is hidden for the visibility timeout (30 seconds by default); deleting it is the ack, and if you do not delete it in time, it reappears for another consumer. Set the timeout above your worst processing time or extend it while working. A redrive policy with maxReceiveCount moves repeated failures to a dead-letter queue. Standard queues may reorder and duplicate; FIFO queues order per MessageGroupId and de-duplicate within a 5-minute window. Pick SQS when you run on AWS and want no broker operations; pick RabbitMQ for rich routing or when you run your own infrastructure.
Example
using System.Text;
using System.Text.Json;
using RabbitMQ.Client;
using RabbitMQ.Client.Events;
var builder = Host.CreateApplicationBuilder(args);
var rabbit = builder.Configuration.GetSection("RabbitMq");
var factory = new ConnectionFactory
{
// Required settings, no guest/guest fallback; the password comes from a secret store.
HostName = rabbit["Host"] ?? throw new InvalidOperationException("RabbitMq:Host is not configured"),
UserName = rabbit["User"] ?? throw new InvalidOperationException("RabbitMq:User is not configured"),
Password = rabbit["Password"] ?? throw new InvalidOperationException("RabbitMq:Password is not configured"),
ClientProvidedName = "inventory-worker",
};
// One connection for the process, opened asynchronously before the host starts. A broker that is
// down at boot fails start-up so the orchestrator restarts the pod; automatic recovery handles later drops.
await using var connection = await factory.CreateConnectionAsync();
builder.Services.AddSingleton<IConnection>(connection);
builder.Services.AddSingleton<OrderEventPublisher>();
builder.Services.AddHostedService<OrderPlacedConsumer>();
await builder.Build().RunAsync();
public sealed record OrderPlaced(Guid OrderId, int CustomerId, decimal Total, string[] Skus);
public sealed class OrderPlacedConsumer(IConnection connection, ILogger<OrderPlacedConsumer> logger) : BackgroundService
{
private const string Queue = "inventory.order-placed";
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
await using var channel = await connection.CreateChannelAsync(cancellationToken: stoppingToken);
await channel.BasicQosAsync(prefetchSize: 0, prefetchCount: 20, global: false, stoppingToken);
var consumer = new AsyncEventingBasicConsumer(channel);
consumer.ReceivedAsync += async (_, ea) =>
{
try
{
var message = JsonSerializer.Deserialize<OrderPlaced>(ea.Body.Span, JsonSerializerOptions.Web)
?? throw new JsonException("Empty message");
await ReserveStockAsync(message, stoppingToken);
await channel.BasicAckAsync(ea.DeliveryTag, multiple: false);
}
catch (JsonException ex)
{
logger.LogError(ex, "Unreadable message {MessageId}; dead-lettering", ea.BasicProperties.MessageId);
await channel.BasicNackAsync(ea.DeliveryTag, multiple: false, requeue: false);
}
catch (Exception ex) when (!stoppingToken.IsCancellationRequested)
{
logger.LogWarning(ex, "Message {MessageId} failed (redelivered: {Redelivered}); requeueing in 5 s",
ea.BasicProperties.MessageId, ea.Redelivered);
// An immediate requeue comes straight back and burns the delivery limit in milliseconds.
// Waiting also pauses this consumer, since handlers run one at a time by default.
await Task.Delay(TimeSpan.FromSeconds(5), stoppingToken);
await channel.BasicNackAsync(ea.DeliveryTag, multiple: false, requeue: true);
}
};
await channel.BasicConsumeAsync(Queue, autoAck: false, consumer, stoppingToken);
await Task.Delay(Timeout.Infinite, stoppingToken).ContinueWith(_ => { }, TaskScheduler.Default);
}
private Task ReserveStockAsync(OrderPlaced message, CancellationToken ct)
{
logger.LogInformation("Reserving {Count} SKUs for order {OrderId}", message.Skus.Length, message.OrderId);
return Task.CompletedTask;
}
}
// A singleton: one long-lived confirm channel, reused for every publish.
public sealed class OrderEventPublisher(IConnection connection) : IAsyncDisposable
{
private readonly SemaphoreSlim _gate = new(1, 1); // IChannel is not for concurrent publishes
private IChannel? _channel;
public async Task PublishAsync(Guid eventId, OrderPlaced message, CancellationToken ct)
{
var props = new BasicProperties
{
Persistent = true,
MessageId = eventId.ToString(), // the event's own id (the outbox row id), never the order id
ContentType = "application/json",
Type = nameof(OrderPlaced),
};
var body = Encoding.UTF8.GetBytes(JsonSerializer.Serialize(message, JsonSerializerOptions.Web));
await _gate.WaitAsync(ct);
try
{
if (_channel is not { IsOpen: true })
{
if (_channel is not null) await _channel.DisposeAsync();
_channel = await connection.CreateChannelAsync(
new CreateChannelOptions(publisherConfirmationsEnabled: true, publisherConfirmationTrackingEnabled: true), ct);
}
// Completes when the broker confirms; throws if it nacks or the message is unroutable.
await _channel.BasicPublishAsync("orders", "order.placed", mandatory: true, props, body, ct);
}
finally
{
_gate.Release();
}
}
public async ValueTask DisposeAsync()
{
if (_channel is not null) await _channel.DisposeAsync();
_gate.Dispose();
}
}When to use it
- Fan an OrderPlaced event out to inventory and email services through a topic exchange, each with its own durable quorum queue.
- Move a malformed PaymentCaptured message to a dead-letter queue after five failed deliveries instead of losing it silently at the default limit.
- Consume refund requests from SQS with 20-second long polling and a 120-second visibility timeout, deleting each only after the refund is stored.
More examples
Declare the topology: quorum queue, delivery limit, dead letters
Declarations are idempotent, so each service declares what it consumes at start-up. A queue's arguments are fixed at creation: redeclaring with different arguments fails with <code>PRECONDITION_FAILED</code>, so changing <code>x-delivery-limit</code> on a live queue means a policy or a new queue. After 5 failed deliveries the message moves to the dead-letter queue. Without the dead-letter exchange, RabbitMQ 4 would silently drop it at the limit (20 by default).
using RabbitMQ.Client;
public static class OrdersTopology
{
public static async Task DeclareAsync(IConnection connection, CancellationToken ct)
{
await using var channel = await connection.CreateChannelAsync(cancellationToken: ct);
await channel.ExchangeDeclareAsync("orders", ExchangeType.Topic, durable: true, autoDelete: false, cancellationToken: ct);
await channel.ExchangeDeclareAsync("orders.dlx", ExchangeType.Fanout, durable: true, autoDelete: false, cancellationToken: ct);
await channel.QueueDeclareAsync("inventory.order-placed.dead", durable: true, exclusive: false, autoDelete: false,
arguments: new Dictionary<string, object?> { ["x-queue-type"] = "quorum" }, cancellationToken: ct);
await channel.QueueBindAsync("inventory.order-placed.dead", "orders.dlx", routingKey: "", cancellationToken: ct);
await channel.QueueDeclareAsync("inventory.order-placed", durable: true, exclusive: false, autoDelete: false,
arguments: new Dictionary<string, object?>
{
["x-queue-type"] = "quorum",
["x-delivery-limit"] = 5,
["x-dead-letter-exchange"] = "orders.dlx",
}, cancellationToken: ct);
await channel.QueueBindAsync("inventory.order-placed", "orders", routingKey: "order.placed", cancellationToken: ct);
}
}An SQS consumer with long polling and explicit deletes
SQS has no ack call: <code>DeleteMessageAsync</code> with the receipt handle is the acknowledgement, and doing nothing is a delayed retry. A handler that outlives the visibility timeout causes a second consumer to process the same message concurrently, so size the timeout or extend it with <code>ChangeMessageVisibilityAsync</code>.
using Amazon.SQS;
using Amazon.SQS.Model;
public sealed class PaymentEventsConsumer(IAmazonSQS sqs, IConfiguration config, ILogger<PaymentEventsConsumer> logger) : BackgroundService
{
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
var queueUrl = config["Sqs:PaymentEventsUrl"]!;
while (!stoppingToken.IsCancellationRequested)
{
var response = await sqs.ReceiveMessageAsync(new ReceiveMessageRequest
{
QueueUrl = queueUrl,
MaxNumberOfMessages = 10,
WaitTimeSeconds = 20, // long polling: fewer empty, billed receives
VisibilityTimeout = 120, // longer than the slowest handler
MessageSystemAttributeNames = ["ApproximateReceiveCount"],
}, stoppingToken);
// SDK v4 leaves collections null when empty.
foreach (var message in response.Messages ?? [])
{
try
{
await HandleAsync(message.Body, stoppingToken);
await sqs.DeleteMessageAsync(queueUrl, message.ReceiptHandle, stoppingToken); // the ack
}
catch (Exception ex) when (!stoppingToken.IsCancellationRequested)
{
// Not deleting is the nack: the message reappears after the visibility timeout,
// and the redrive policy moves it to the DLQ after maxReceiveCount receives.
logger.LogWarning(ex, "Message {MessageId} failed", message.MessageId);
}
}
}
}
private static Task HandleAsync(string body, CancellationToken ct) => Task.CompletedTask;
}Local RabbitMQ for development
The management image adds the web UI on port 15672, where you can watch queue depth, unacked counts and the dead-letter queue while testing consumers. The named volume keeps queues and messages across container restarts. The health check lets dependent services wait for a broker that accepts connections.
services:
rabbitmq:
image: rabbitmq:4-management
ports:
- "5672:5672" # AMQP
- "15672:15672" # management UI
environment:
RABBITMQ_DEFAULT_USER: orders
RABBITMQ_DEFAULT_PASS: local-dev-only
healthcheck:
test: ["CMD", "rabbitmq-diagnostics", "-q", "ping"]
interval: 10s
timeout: 5s
retries: 5
volumes:
- rabbitmq-data:/var/lib/rabbitmq
volumes:
rabbitmq-data:
Discussion