Messaging for Web Developers
Know when a message broker beats a direct HTTP call, what queues and logs give you, and the delivery guarantees your code has to cope with.
producer --(message)--> broker --(delivery, at least once)--> consumer
consumer: check eventId -> do work -> record eventId -> acknowledgeMost of an API is request and response: the client asks, your controller answers, and the client waits. That model breaks down when the work is slow, when another service may be down, or when several parts of the system care about the same thing happening. A message broker sits between producer and consumer so that neither has to be running, or fast, at the same moment as the other.
Commands and events
A command asks one consumer to do something: send this invoice. An event states a fact that already happened: order 42 was placed. Name events in the past tense and let any number of consumers react. A producer that publishes events does not know who listens, which is what lets you add a new consumer without touching it.
Queues and logs
- RabbitMQ routes each message through an exchange into queues. A consumer acknowledges a message and the broker deletes it. It suits task distribution and per-message routing.
- Kafka appends records to a partitioned log and keeps them for a retention period. Each consumer group tracks its own offset, so a new service can replay history. It suits high-volume event streams.
The guarantees you actually get
Both brokers are normally run at least once: a consumer that crashes after doing the work but before acknowledging will see the same message again. Duplicates are therefore normal, not a bug, and every consumer must be idempotent. The usual technique is to give each message a unique id and record processed ids in the same transaction as the work.
@Transactional
public void on(OrderPlaced event) {
if (!processed.markIfNew(event.eventId())) {
return; // a redelivery: already handled
}
invoices.createFor(event.orderId());
}The check alone does not stop two concurrent deliveries of the same event: both can look before either has committed and both see nothing. What separates them is the primary key on the processed id. The second insert fails with a primary-key violation, its transaction rolls back together with the duplicate work, and the redelivery is treated as already handled. Keep that unique constraint in the database; without it, the check is a race.
Ordering is limited too. Kafka orders records only within one partition, so give related events the same key. RabbitMQ keeps queue order, but several competing consumers or a requeue will reorder work.
When not to add a broker
A broker is another piece of infrastructure to run, secure and monitor. If producer and consumer live in the same application and losing work on a crash is acceptable, Spring's application events are enough. Reach for a broker when work must survive restarts, cross service boundaries or absorb traffic spikes. Never publish to a broker inside a database transaction and assume both happened together: that gap is what the transactional outbox pattern closes.
Example
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import java.time.Instant;
import java.util.UUID;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
record OrderPlaced(UUID eventId, long orderId, Instant occurredAt) {
}
class ProcessedMessage {
private UUID id;
private Instant processedAt;
protected ProcessedMessage() {
}
ProcessedMessage(UUID id) {
this.id = id;
this.processedAt = Instant.now();
}
}
interface ProcessedMessageRepository extends JpaRepository<ProcessedMessage, UUID> {
}
public class InvoiceOnOrderPlaced {
private final ProcessedMessageRepository processed;
InvoiceOnOrderPlaced(ProcessedMessageRepository processed) {
this.processed = processed;
}
public void handle(OrderPlaced event) {
if (processed.existsById(event.eventId())) {
return;
}
createInvoice(event.orderId());
processed.save(new ProcessedMessage(event.eventId()));
}
private void createInvoice(long orderId) {
// insert the invoice rows here, in the same transaction
}
}
When to use it
- Accept an order in under 100 ms and let invoicing, stock reservation and email run behind a broker.
- Absorb a burst of 50,000 image uploads by queueing thumbnail jobs instead of resizing inside the request.
- Let a new analytics service replay a month of order events from Kafka without asking the order team for an export.
More examples
A synchronous call versus an event
The first flow couples the order endpoint to the availability and speed of two other services. The second returns as soon as the order is stored; the invoice and email services catch up later, and a third consumer can be added without changing the order code.
Synchronous (the client waits for every step):
POST /orders -> OrderController -> PaymentClient.charge() (300 ms)
-> InvoiceClient.create() (down: request fails)
With an event (the client waits only for the order to be stored):
POST /orders -> OrderController -> save order + publish OrderPlaced -> 201 Created
broker -> invoice-service consumes OrderPlaced when it is back up
broker -> email-service consumes OrderPlaced on its own schedule
An event record with the fields consumers need
A unique <code>eventId</code> lets consumers drop duplicates, <code>type</code> and <code>version</code> let a consumer handle old and new shapes during a rollout, and <code>occurredAt</code> records when the fact happened rather than when it was delivered. Treat the record as a public contract: add fields, never rename or remove them.
import com.fasterxml.jackson.annotation.JsonInclude;
import java.math.BigDecimal;
import java.time.Instant;
import java.util.UUID;
@JsonInclude(JsonInclude.Include.NON_NULL)
public record OrderPlacedEvent(
UUID eventId,
String type,
int version,
Instant occurredAt,
long orderId,
long customerId,
BigDecimal total) {
public static OrderPlacedEvent of(long orderId, long customerId, BigDecimal total) {
return new OrderPlacedEvent(UUID.randomUUID(), "order.placed", 1, Instant.now(), orderId, customerId, total);
}
}
Brokers for local development with Docker Compose
With <code>spring-boot-docker-compose</code> on the classpath, Spring Boot starts the services in <code>compose.yaml</code> when the application starts. It recognises the RabbitMQ image and points the AMQP client at it without any properties. It has no Docker Compose connection details for Kafka, so the Kafka client relies on <code>spring.kafka.bootstrap-servers</code>, whose default of <code>localhost:9092</code> matches the published port. Port 15672 serves the RabbitMQ management UI, the quickest way to watch queues fill and drain. This file runs a single Kafka broker, so any topic you create locally must use a replication factor of 1; the Kafka lesson reads the replica count from a property for that reason.
services:
rabbitmq:
image: rabbitmq:4-management
ports:
- "5672:5672"
- "15672:15672"
kafka:
image: apache/kafka:4.0.0
ports:
- "9092:9092"
Discussion