Thread Pools and Connection Pools

Size Tomcat's worker threads, the Hikari connection pool and your own executors together, so load queues where you expect instead of failing.

Syntax

server.tomcat.threads.max=200
server.tomcat.accept-count=100
spring.datasource.hikari.maximum-pool-size=10
spring.datasource.hikari.connection-timeout=3000
spring.task.execution.pool.queue-capacity=500

Every request to a Spring MVC API passes through a chain of pools. Tomcat accepts the socket, a worker thread runs your controller, and the moment the code touches the database it borrows a connection from HikariCP. Background work started with @Async runs on yet another pool. Each pool has a limit, and the smallest one decides how much work the service can really do at once.

The defaults

Pool	Property	Default
Tomcat worker threads	`server.tomcat.threads.max`	200
Waiting connections when all threads are busy	`server.tomcat.accept-count`	100
Hikari connections	`spring.datasource.hikari.maximum-pool-size`	10
Wait for a free connection	`spring.datasource.hikari.connection-timeout`	30 seconds
`@Async` executor core threads	`spring.task.execution.pool.core-size`	8, with an unbounded queue

200 threads sharing 10 connections is not a mistake. Most of a request's time is spent outside the database, so threads take turns. It becomes a problem when transactions are long: the eleventh thread waits up to 30 seconds and then fails with SQLTransientConnectionException, while the load balancer has long since given up.

Sizing the connection pool

The instinct under load is to raise maximum-pool-size to 100. That usually makes things slower: the database has a fixed number of cores, and hundreds of concurrent queries fight over them and their locks. Work it out instead. By Little's law, connections in use equal requests per second times seconds each request holds a connection. 400 requests per second holding a connection for 15 ms need 6 connections. Remember that the pool is per instance: 8 replicas with 20 connections each is 160, which is more than PostgreSQL's default max_connections of 100.

Lower connection-timeout to a few seconds so an overloaded service fails fast and visibly, and keep remote HTTP calls out of transactions so a connection is never held while you wait on someone else.

Executors

Boot's applicationTaskExecutor has an unbounded queue, so max-size is never reached and a flood of tasks grows memory instead of being rejected. Set spring.task.execution.pool.queue-capacity, or give heavy jobs their own bounded executor. Defining your own Executor bean makes the auto-configured one back off unless you set spring.task.execution.mode=force.

With spring.threads.virtual.enabled=true, Tomcat and the task executor use virtual threads and the thread limits largely disappear. The connection pool then becomes the only brake, so a burst that used to queue in Tomcat now queues on Hikari. Watch hikaricp.connections.pending in Actuator metrics either way.

Example

import java.util.concurrent.ThreadPoolExecutor;

import org.springframework.boot.task.ThreadPoolTaskExecutorBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.Async;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
import org.springframework.stereotype.Service;

@Configuration
@EnableAsync
public class ExecutorConfig {

    // A small, bounded pool for heavy report jobs, kept apart from other async work.
    // Declaring any Executor bean makes Boot's applicationTaskExecutor back off, and
    // plain @Async methods would then run here too. Set spring.task.execution.mode=force
    // to keep Boot's executor for everything that does not name this one.
    @Bean
    ThreadPoolTaskExecutor reportExecutor(ThreadPoolTaskExecutorBuilder builder) {
        ThreadPoolTaskExecutor executor = builder
                .corePoolSize(4)
                .maxPoolSize(4)
                .queueCapacity(50)
                .threadNamePrefix("report-")
                .build();
        // When 4 are running and 50 are queued, the caller runs the job itself.
        // That slows the caller down instead of growing memory without limit.
        executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
        return executor;
    }
}

@Service
class ReportService {

    @Async("reportExecutor")
    public void buildMonthlyReport(long customerId) {
        // reads many rows; each run holds one pooled connection at most
    }
}

When to use it

An API starts returning 500s under a marketing spike with 'Connection is not available, request timed out after 30000ms' in the logs.
Scaling a service from 3 to 10 replicas suddenly exhausts PostgreSQL's max_connections during rolling deploys.
A nightly export submitted 50,000 @Async tasks and the instance ran out of memory because the default queue is unbounded.

More examples

Pools sized for one instance of a busy API

Setting <code>minimum-idle</code> equal to <code>maximum-pool-size</code> gives a fixed-size pool, which HikariCP recommends for steady traffic. The leak detection threshold logs a stack trace when a connection is held longer than 20 seconds, which points straight at the code that forgot to release it or ran a slow remote call inside a transaction. The Hikari timeouts are bound straight onto <code>HikariDataSource</code> as milliseconds, so write <code>3000</code>, not <code>3s</code>. With <code>spring.task.execution.mode=force</code>, Boot keeps <code>applicationTaskExecutor</code> and uses it for plain <code>@Async</code> methods even when other executors, such as a dedicated report pool, are defined.

# Tomcat: worker threads and the backlog of connections waiting for one.
server.tomcat.threads.max=200
server.tomcat.accept-count=100

# Hikari: sized from measured load, and quick to fail when exhausted.
# Hikari timeouts are plain milliseconds: a value like 3s fails to bind.
spring.datasource.hikari.maximum-pool-size=12
spring.datasource.hikari.minimum-idle=12
spring.datasource.hikari.connection-timeout=3000
spring.datasource.hikari.leak-detection-threshold=20000

# @Async: bounded queue so max-size actually applies.
# force keeps this executor even when you declare your own Executor beans.
spring.task.execution.mode=force
spring.task.execution.pool.core-size=8
spring.task.execution.pool.max-size=16
spring.task.execution.pool.queue-capacity=500

Keeping the remote call outside the transaction

If the whole method were <code>@Transactional</code>, every checkout would hold a database connection for the full duration of the pricing call. With a pool of 12 and a pricing service that slows to 2 seconds, throughput collapses to 6 checkouts a second and every other endpoint starts waiting for connections. Drawing the transaction around the write only keeps the hold time to milliseconds.

import java.math.BigDecimal;

import org.springframework.jdbc.core.simple.JdbcClient;
import org.springframework.stereotype.Service;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.support.TransactionTemplate;
import org.springframework.web.client.RestClient;

record Quote(BigDecimal amount, String currency) {}

@Service
public class CheckoutService {

    private final RestClient pricing;
    private final JdbcClient jdbc;
    private final TransactionTemplate tx;

    CheckoutService(RestClient.Builder builder, JdbcClient jdbc, PlatformTransactionManager txManager) {
        this.pricing = builder.baseUrl("https://pricing.internal.example").build();
        this.jdbc = jdbc;
        this.tx = new TransactionTemplate(txManager);
    }

    public void checkout(long orderId) {
        // 1. Slow network call: no connection is held while waiting.
        Quote quote = pricing.get().uri("/quotes/{id}", orderId).retrieve().body(Quote.class);

        // 2. Short transaction: the connection is borrowed for a few milliseconds.
        tx.executeWithoutResult(status -> jdbc
                .sql("UPDATE orders SET total = :total, currency = :currency, status = 'PRICED' WHERE id = :id")
                .param("total", quote.amount())
                .param("currency", quote.currency())
                .param("id", orderId)
                .update());
    }
}

Virtual threads with the pool as the real limit

With virtual threads, each request gets its own cheap thread, so the old 200-thread ceiling no longer protects the database. Hikari's pool becomes the queue: excess requests wait for a connection, and the short <code>connection-timeout</code> turns a sustained overload into quick errors rather than minutes of piled-up requests. Pair this with a rate limit or a concurrency limit in front of expensive endpoints.

spring:
  threads:
    virtual:
      enabled: true
  datasource:
    hikari:
      maximum-pool-size: 20
      connection-timeout: 2000  # milliseconds
server:
  tomcat:
    max-connections: 10000

Discussion

Be the first to comment on this lesson.

The defaults

Sizing the connection pool

Executors

Example

When to use it

More examples

Pools sized for one instance of a busy API

Keeping the remote call outside the transaction

Virtual threads with the pool as the real limit

Related

Discussion