Keyset Pagination
Page through large tables with a WHERE clause on the last row seen instead of OFFSET, so page 5,000 is as fast as page 1 and rows never repeat.
WHERE (sort_col, id) < (:lastSort, :lastId)
ORDER BY sort_col DESC, id DESC
LIMIT :size + 1
Window<T> findFirst20ByOrderByCreatedAtDescIdDesc(ScrollPosition position);Offset pagination asks the database for LIMIT 20 OFFSET 100000. The database cannot jump to row 100,000: it reads and throws away every row before it, so each page is slower than the last. Rows inserted or deleted while a client is paging shift everything, so items appear twice or are skipped. For admin tables with a page picker that is acceptable. For feeds, exports and infinite scroll it is not.
The idea
Keyset pagination, also called seek or cursor pagination, remembers the sort key of the last row the client saw and asks for rows after it:
SELECT id, title, published_at
FROM article
WHERE (published_at, id) < (:lastPublishedAt, :lastId)
ORDER BY published_at DESC, id DESC
LIMIT 21With an index on (published_at DESC, id DESC), the database seeks straight to the position and reads 21 rows, however deep the client is. Inserts above the cursor do not move it, so nothing repeats.
Rules that make it correct
- The sort must be unique. Sorting by
published_atalone breaks when two rows share a timestamp: one of them is skipped. Add the primary key as the last sort column. - The WHERE clause must match the ORDER BY. The row-value comparison
(a, b) < (x, y)works in PostgreSQL and MySQL; elsewhere write it out asa < x OR (a = x AND b < y). - Fetch one extra row. Asking for
size + 1tells you whether there is a next page without aCOUNT(*), which is itself slow on large tables. - Index the sort columns in the same order and direction.
What you give up
No total count and no jumping to page 37: clients can only go forward (or backward with a reversed query). That matches how feeds and sync jobs actually read data.
The API contract
Return the cursor as an opaque string, usually Base64 of the last row's sort values at full precision, plus a flag or link for the next page. Clients pass it back unchanged. Opaque cursors let you change the sort columns later without breaking clients, and stop clients building cursors by hand. Validate the decoded cursor like any other input: a tampered one must produce a 400, not a 500.
Spring Data supports the same technique with ScrollPosition and Window, shown in the examples.
Example
import java.nio.charset.StandardCharsets;
import java.time.Instant;
import java.time.ZoneOffset;
import java.util.Base64;
import java.util.List;
import org.springframework.http.HttpStatus;
import org.springframework.jdbc.core.simple.JdbcClient;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.server.ResponseStatusException;
record ArticleSummary(long id, String title, Instant publishedAt) {}
record ArticlePage(List<ArticleSummary> items, String nextCursor) {}
record Cursor(Instant publishedAt, long id) {
// Full ISO-8601 precision: truncating to milliseconds would skip rows whose
// microsecond timestamps fall between the truncated value and the real one.
String encode() {
String raw = id + ":" + publishedAt;
return Base64.getUrlEncoder().withoutPadding().encodeToString(raw.getBytes(StandardCharsets.UTF_8));
}
static Cursor decode(String value) {
try {
String raw = new String(Base64.getUrlDecoder().decode(value), StandardCharsets.UTF_8);
String[] parts = raw.split(":", 2);
return new Cursor(Instant.parse(parts[1]), Long.parseLong(parts[0]));
} catch (RuntimeException e) {
throw new ResponseStatusException(HttpStatus.BAD_REQUEST, "Invalid cursor");
}
}
}
public class ArticleFeedController {
private final JdbcClient jdbc;
ArticleFeedController(JdbcClient jdbc) {
this.jdbc = jdbc;
}
("/api/articles")
ArticlePage list((required = false) String cursor,
(defaultValue = "20") int size) {
int limit = Math.clamp(size, 1, 100);
Cursor after = cursor == null ? new Cursor(Instant.parse("9999-12-31T00:00:00Z"), Long.MAX_VALUE)
: Cursor.decode(cursor);
List<ArticleSummary> rows = jdbc.sql("""
SELECT id, title, published_at
FROM article
WHERE (published_at, id) < (:publishedAt, :id)
ORDER BY published_at DESC, id DESC
LIMIT :limit""")
// The PostgreSQL driver cannot bind Instant; OffsetDateTime maps to timestamptz.
.param("publishedAt", after.publishedAt().atOffset(ZoneOffset.UTC))
.param("id", after.id())
.param("limit", limit + 1)
.query(ArticleSummary.class)
.list();
if (rows.size() <= limit) {
return new ArticlePage(rows, null);
}
List<ArticleSummary> page = rows.subList(0, limit);
ArticleSummary last = page.getLast();
return new ArticlePage(page, new Cursor(last.publishedAt(), last.id()).encode());
}
}
When to use it
- An infinite-scroll feed whose deep pages took seconds with OFFSET and now return in a few milliseconds.
- A nightly job exports 30 million events to object storage in chunks of 500 without slowing down towards the end.
- A mobile sync endpoint returns changes since the last cursor so devices never receive the same record twice.
More examples
Spring Data scrolling with Window and ScrollPosition
<code>ScrollPosition.keyset()</code> starts at the beginning, and <code>positionAt</code> gives the keyset of a row in the current window. Spring Data then builds the seek <code>WHERE</code> clause from the sort for you. This loop is a good fit for exports and sync jobs: each query reads 500 rows directly from the index instead of skipping an ever-growing offset. For a REST endpoint, encode the keys from <code>getKeys()</code> into an opaque cursor rather than exposing column names. Keep the loop itself out of a transaction: one transaction around millions of rows keeps every loaded entity in the persistence context until it ends and runs the export out of memory.
import java.time.Instant;
import java.util.Map;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import org.springframework.data.domain.KeysetScrollPosition;
import org.springframework.data.domain.ScrollPosition;
import org.springframework.data.domain.Window;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Service;
@Entity
class Event {
@Id
private Long id;
private String type;
private Instant occurredAt;
protected Event() {
}
Long getId() { return id; }
Instant getOccurredAt() { return occurredAt; }
}
interface EventRepository extends JpaRepository<Event, Long> {
Window<Event> findFirst500ByOrderByOccurredAtAscIdAsc(ScrollPosition position);
}
@Service
public class EventExporter {
private final EventRepository events;
EventExporter(EventRepository events) {
this.events = events;
}
// No @Transactional here: each call runs in its own short read-only transaction,
// so loaded entities are released after every window instead of piling up
// in one persistence context for the whole export.
public long exportAll() {
long written = 0;
ScrollPosition position = ScrollPosition.keyset();
Window<Event> window;
do {
window = events.findFirst500ByOrderByOccurredAtAscIdAsc(position);
for (Event event : window) {
written++; // write the event to the export file
}
if (!window.isEmpty()) {
position = window.positionAt(window.size() - 1);
}
} while (window.hasNext());
return written;
}
// For an HTTP API, turn the position's keys into an opaque cursor.
static Map<String, Object> cursorKeys(ScrollPosition position) {
return ((KeysetScrollPosition) position).getKeys();
}
}
The index that makes it fast
Without a matching index, keyset pagination still has to sort the table and gains little. With one, PostgreSQL walks the index from the cursor and stops after 21 entries. When the feed is filtered, for example by author, put the equality column first so the seek happens inside that author's slice of the index. Add this in a Flyway migration rather than by hand.
-- Matches ORDER BY published_at DESC, id DESC exactly.
CREATE INDEX idx_article_feed
ON article (published_at DESC, id DESC);
-- Filtered feeds need the filter column first.
CREATE INDEX idx_article_author_feed
ON article (author_id, published_at DESC, id DESC);
-- Check: should show an index scan, not a sort over the whole table.
EXPLAIN ANALYZE
SELECT id, title, published_at
FROM article
WHERE (published_at, id) < ('2026-09-01T00:00:00Z', 81234)
ORDER BY published_at DESC, id DESC
LIMIT 21;
What the client sees
The client treats the cursor as an opaque token and sends it back for the next page. A <code>null</code> <code>nextCursor</code> means the end. Because the cursor encodes a position rather than a page number, articles published while the user scrolls appear at the top on refresh instead of pushing duplicates onto the next page.
GET /api/articles?size=2 HTTP/1.1
Host: api.example.com
HTTP/1.1 200 OK
Content-Type: application/json
{
"items": [
{ "id": 81236, "title": "Release notes", "publishedAt": "2026-09-02T10:15:00Z" },
{ "id": 81234, "title": "Keyset paging", "publishedAt": "2026-09-01T00:00:00Z" }
],
"nextCursor": "ODEyMzQ6MjAyNi0wOS0wMVQwMDowMDowMFo"
}
GET /api/articles?size=2&cursor=ODEyMzQ6MjAyNi0wOS0wMVQwMDowMDowMFo HTTP/1.1
Host: api.example.com
Discussion