Limiting Concurrency with @ConcurrencyLimit

Cap how many calls to a fragile dependency run at the same time with Spring Framework 7's @ConcurrencyLimit, and choose between waiting and rejecting.

Syntax

@EnableResilientMethods   // on a @Configuration class

@ConcurrencyLimit(10)
@ConcurrencyLimit(limit = 10, policy = ConcurrencyLimit.ThrottlePolicy.REJECT)
@ConcurrencyLimit(limitString = "${app.limit:10}")

A thread pool used to be an accidental safety limit: with 200 request threads, at most 200 calls could hit a downstream service at once. Virtual threads remove that ceiling. A traffic spike can now produce thousands of simultaneous calls to a service that copes with ten, or thousands of threads queuing for a connection pool of twenty. The limit has to be explicit.

@ConcurrencyLimit from org.springframework.resilience.annotation is part of Spring Framework 7. It caps how many invocations of a method can run at the same time.

@ConcurrencyLimit(4)
public byte[] render(String reportId) { ... }

Switching it on

It is activated by the same @EnableResilientMethods as @Retryable. Spring Boot does not add that for you; without it the annotation has no effect. Like every proxy-based feature, it only applies to calls that come from another bean through the proxy, on methods that are not final: calls from inside the same class bypass it, and a final method is left unguarded with nothing but a startup log warning.

Wait or reject

The policy attribute decides what happens to the call that exceeds the limit:

ThrottlePolicy.BLOCK (the default): the caller waits until a slot frees up. Good for background work, and cheap on virtual threads. But waiting is unbounded, so under sustained overload requests pile up until your own clients time out.
ThrottlePolicy.REJECT: the call fails immediately with InvocationRejectedException (a RejectedExecutionException). Map it to 503 with a Retry-After header. This is usually right for user-facing requests: a fast "busy, try again" beats a slow timeout.

Scope of the limit

The limit is per bean instance. On a method, each annotated method has its own counter. On the class, all its methods share one counter, which is what you want when several methods hit the same downstream service. The limit is also per application instance: four replicas each allowing 10 calls means up to 40 at the dependency. Divide the downstream service's real capacity by your replica count.

limitString accepts a ${...} placeholder so the number can live in configuration.

What it is not

It limits calls in flight, not calls per second; for a quota such as 100 requests per minute you need a rate limiter. It also does not replace timeouts: a slot is held until the call returns, so without a read timeout a hung dependency keeps every slot forever.

Example

import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Configuration;
import org.springframework.http.MediaType;
import org.springframework.resilience.annotation.ConcurrencyLimit;
import org.springframework.resilience.annotation.EnableResilientMethods;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestClient;

record RenderRequest(String reportId, String format) {
}

@Configuration(proxyBeanMethods = false)
@EnableResilientMethods
class ResilienceConfig {
}

@Service
public class ReportRenderClient {

    private final RestClient rest;

    ReportRenderClient(RestClient.Builder builder, @Value("${app.renderer.base-url}") String baseUrl) {
        this.rest = builder.baseUrl(baseUrl).build();
    }

    // The rendering service handles 4 jobs at once; extra callers wait for a free slot.
    @ConcurrencyLimit(4)
    public byte[] render(String reportId) {
        return rest.post()
                .uri("/render")
                .contentType(MediaType.APPLICATION_JSON)
                .body(new RenderRequest(reportId, "pdf"))
                .retrieve()
                .requiredBody(byte[].class);
    }
}

When to use it

Keep a PDF rendering service that can process four jobs at a time from being flooded when a hundred users export reports at once.
Reject quote requests beyond ten in flight with a quick 503 and Retry-After instead of letting response times climb to 30 seconds.
Put a deliberate cap on outbound calls after switching the application to virtual threads, which removed the thread pool limit.

More examples

Reject excess calls and answer 503

The eleventh concurrent quote request fails at once instead of joining a queue that may never drain. The advice turns the rejection into a 503 with <code>Retry-After</code>, which browsers, mobile apps and gateways understand. Note that the annotated method is package-private here: the proxy still intercepts it because it is not <code>final</code>, but it must be called from another bean.

import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.ProblemDetail;
import org.springframework.http.ResponseEntity;
import org.springframework.resilience.InvocationRejectedException;
import org.springframework.resilience.annotation.ConcurrencyLimit;
import org.springframework.stereotype.Service;
import org.springframework.web.bind.annotation.ExceptionHandler;
import org.springframework.web.bind.annotation.RestControllerAdvice;
import org.springframework.web.client.RestClient;

record Quote(String postcode, long premiumPence) {
}

@Service
class QuoteEngineClient {

    private final RestClient rest;

    QuoteEngineClient(RestClient.Builder builder, @Value("${app.quotes.base-url}") String baseUrl) {
        this.rest = builder.baseUrl(baseUrl).build();
    }

    @ConcurrencyLimit(limit = 10, policy = ConcurrencyLimit.ThrottlePolicy.REJECT)
    Quote quote(String postcode) {
        return rest.get().uri("/quotes/{postcode}", postcode).retrieve().requiredBody(Quote.class);
    }
}

@RestControllerAdvice
public class BusyAdvice {

    @ExceptionHandler(InvocationRejectedException.class)
    ResponseEntity<ProblemDetail> busy(InvocationRejectedException ex) {
        ProblemDetail problem = ProblemDetail.forStatusAndDetail(
                HttpStatus.SERVICE_UNAVAILABLE, "Quotes are busy right now. Please try again in a few seconds.");
        return ResponseEntity.status(HttpStatus.SERVICE_UNAVAILABLE)
                .header(HttpHeaders.RETRY_AFTER, "5")
                .body(problem);
    }
}

One shared limit for a whole client

On the class, the annotation gives all methods of this bean one shared counter, so forward and reverse lookups together never exceed eight calls in flight. That matches reality: the geocoder's capacity is shared, whichever endpoint you call. The placeholder lets each environment set its own value, for example a lower one where a free-tier account is used.

import org.springframework.beans.factory.annotation.Value;
import org.springframework.resilience.annotation.ConcurrencyLimit;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestClient;

record Coordinates(double latitude, double longitude) {
}

record Place(String name, String country) {
}

@Service
@ConcurrencyLimit(limitString = "${app.geocoder.max-concurrent:8}")
public class GeocoderClient {

    private final RestClient rest;

    GeocoderClient(RestClient.Builder builder, @Value("${app.geocoder.base-url}") String baseUrl) {
        this.rest = builder.baseUrl(baseUrl).build();
    }

    public Coordinates forward(String address) {
        return rest.get()
                .uri(uri -> uri.path("/forward").queryParam("q", address).build())
                .retrieve()
                .requiredBody(Coordinates.class);
    }

    public Place reverse(double latitude, double longitude) {
        return rest.get()
                .uri(uri -> uri.path("/reverse").queryParam("lat", latitude).queryParam("lon", longitude).build())
                .retrieve()
                .requiredBody(Place.class);
    }
}

Configuration that goes with it

These settings belong together. Virtual threads make blocking calls cheap and remove the old implicit limit, the concurrency limit puts a deliberate one back, and the read timeout guarantees each slot is released even when the dependency hangs. If you run three replicas against a geocoder that handles 24 concurrent requests, 8 per replica is the right number.

# Virtual threads: no thread pool to cap outbound calls any more
spring.threads.virtual.enabled=true

# Explicit caps instead, sized to each dependency's capacity per replica
app.geocoder.max-concurrent=8

# A slot is held until the call returns, so timeouts are essential
spring.http.clients.connect-timeout=1s
spring.http.clients.read-timeout=4s

Discussion

Be the first to comment on this lesson.

Switching it on

Wait or reject

Scope of the limit

What it is not

Example

When to use it

More examples

Reject excess calls and answer 503

One shared limit for a whole client

Configuration that goes with it

Related

Discussion