Data and Migrations
Model orders in EF Core on PostgreSQL with xmin concurrency, an idempotency table, keyset pagination and migrations that ship safely.
modelBuilder.Entity<Order>().Property(o => o.Version).IsRowVersion(); // xmin
q.Where(o => o.CreatedAt < c.CreatedAt || (o.CreatedAt == c.CreatedAt && o.Id < c.Id))
.OrderByDescending(o => o.CreatedAt).ThenByDescending(o => o.Id).Take(limit + 1)The project stores everything in one PostgreSQL database through EF Core 10 and the Npgsql provider. The model is small: Customer, Product, Order with OrderLines, plus two tables that exist for reliability rather than for the business: IdempotencyRecord and OutboxMessage. Configuration lives in one IEntityTypeConfiguration<T> class per entity, picked up by ApplyConfigurationsFromAssembly.
Model decisions
- Money.
decimalwithHasPrecision(12, 2); without it Npgsql maps to unboundednumericand rounding happens wherever the code happens to round. Line prices are copied from the product at order time, never joined later. - Status as text.
HasConversion<string>()storesPaidrather than1, so reordering the enum cannot corrupt existing rows and SQL is readable during an incident. - Time.
DateTimeOffsetmaps totimestamptz, and Npgsql rejects a value whose offset is not zero; always writetimeProvider.GetUtcNow(). PostgreSQL keeps microseconds, .NET keeps 100 ns ticks, so a value read back can differ from the one written in memory. - Deletes. Lines cascade with their order; an order
Restricts its customer, so deleting a customer with orders fails loudly instead of erasing financial records.
Optimistic concurrency with xmin
Two staff members cancel and ship the same order at once. Without a concurrency token the last SaveChanges wins and the order ends up shipped and refunded. With Npgsql, a uint Version property configured with IsRowVersion() maps to PostgreSQL's xmin system column: no column is added, and the database changes the value on every update. EF Core adds WHERE "Id" = @id AND xmin = @version to the UPDATE; zero rows affected throws DbUpdateConcurrencyException. The API exposes the version as an ETag and takes it back in If-Match, so a stale client gets 412 instead of overwriting.
Keyset pagination
Offset (Skip/Take) | Keyset (cursor) | |
|---|---|---|
| Cost of page 500 | Reads and discards 10,000 rows | One index seek, same as page 1 |
| Rows inserted while paging | Duplicates or skipped rows | Stable |
| Jump to page N | Yes | No, next/previous only |
"My orders, newest first" pages on (CreatedAt, Id) descending, with Id as the tie-breaker because two orders can share a timestamp. The index (CustomerId, CreatedAt, Id) must have the equality column first and then the sort keys in the ORDER BY's order, all in the same direction or all reversed, since PostgreSQL scans a B-tree backwards; otherwise it sorts the customer's whole history for every page. Only mixed directions, such as CreatedAt DESC, Id ASC, need an index declared with matching DESC flags. Fetch limit + 1 rows: the extra row tells you whether a next page exists without a COUNT(*). The cursor is opaque base64url so clients cannot build or depend on it.
Migrations
dotnet ef migrations add generates C# that you review like any code: a renamed property becomes drop-and-add unless you edit it to RenameColumn. Production never runs Database.Migrate() at startup: three replicas race for the lock, and a failed migration crash-loops every pod. Build a migration bundle in CI and run it once, as a Kubernetes Job or pipeline step, before the new image rolls out. Every migration must work with the code that is already running (expand, deploy, contract): add a nullable column now, backfill, make it required in a later release.
Example
using System.Buffers.Text;
using System.Text;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Metadata.Builders;
public enum OrderStatus { Pending, Paid, Shipped, Cancelled }
public sealed class Customer
{
public long Id { get; set; }
public required string Subject { get; set; } // the identity provider's sub claim
public required string Email { get; set; }
public DateTimeOffset CreatedAt { get; set; }
}
public sealed class Product
{
public long Id { get; set; }
public required string Sku { get; set; }
public required string Name { get; set; }
public decimal UnitPrice { get; set; }
public bool IsActive { get; set; } = true;
public uint Version { get; set; }
}
public sealed class Order
{
public long Id { get; set; }
public long CustomerId { get; set; }
public OrderStatus Status { get; set; }
public decimal Total { get; set; }
public DateTimeOffset CreatedAt { get; set; }
public uint Version { get; set; }
public List<OrderLine> Lines { get; } = [];
}
public sealed class OrderLine
{
public long Id { get; set; }
public long OrderId { get; set; }
public required string Sku { get; set; }
public int Quantity { get; set; }
public decimal UnitPrice { get; set; } // copied from Product when the order is placed
}
public sealed class IdempotencyRecord
{
public long CustomerId { get; set; }
public required string Key { get; set; }
public required string RequestHash { get; set; }
public long OrderId { get; set; }
public DateTimeOffset CreatedAt { get; set; }
}
public sealed class OutboxMessage
{
public Guid Id { get; set; }
public required string Type { get; set; }
public required string Payload { get; set; }
public DateTimeOffset OccurredAt { get; set; }
public DateTimeOffset? PublishedAt { get; set; }
}
public sealed class OrdersDbContext(DbContextOptions<OrdersDbContext> options) : DbContext(options)
{
public DbSet<Customer> Customers => Set<Customer>();
public DbSet<Product> Products => Set<Product>();
public DbSet<Order> Orders => Set<Order>();
public DbSet<IdempotencyRecord> IdempotencyRecords => Set<IdempotencyRecord>();
public DbSet<OutboxMessage> Outbox => Set<OutboxMessage>();
protected override void OnModelCreating(ModelBuilder modelBuilder) =>
modelBuilder.ApplyConfigurationsFromAssembly(typeof(OrdersDbContext).Assembly);
}
internal sealed class CustomerConfiguration : IEntityTypeConfiguration<Customer>
{
public void Configure(EntityTypeBuilder<Customer> customer)
{
customer.Property(c => c.Subject).HasMaxLength(64);
customer.Property(c => c.Email).HasMaxLength(320);
customer.HasIndex(c => c.Subject).IsUnique();
}
}
internal sealed class ProductConfiguration : IEntityTypeConfiguration<Product>
{
public void Configure(EntityTypeBuilder<Product> product)
{
product.Property(p => p.Sku).HasMaxLength(32);
product.Property(p => p.Name).HasMaxLength(200);
product.Property(p => p.UnitPrice).HasPrecision(12, 2);
product.Property(p => p.Version).IsRowVersion();
product.HasIndex(p => p.Sku).IsUnique();
}
}
internal sealed class OrderConfiguration : IEntityTypeConfiguration<Order>
{
public void Configure(EntityTypeBuilder<Order> order)
{
order.Property(o => o.Status).HasConversion<string>().HasMaxLength(16);
order.Property(o => o.Total).HasPrecision(12, 2);
order.Property(o => o.Version).IsRowVersion(); // Npgsql maps this to the xmin system column
order.HasOne<Customer>().WithMany().HasForeignKey(o => o.CustomerId).OnDelete(DeleteBehavior.Restrict);
order.HasMany(o => o.Lines).WithOne().HasForeignKey(l => l.OrderId).OnDelete(DeleteBehavior.Cascade);
// Matches ORDER BY CreatedAt DESC, Id DESC for one customer, so each page is an index seek.
order.HasIndex(o => new { o.CustomerId, o.CreatedAt, o.Id }).IsDescending(false, true, true);
}
}
internal sealed class OrderLineConfiguration : IEntityTypeConfiguration<OrderLine>
{
public void Configure(EntityTypeBuilder<OrderLine> line)
{
line.Property(l => l.Sku).HasMaxLength(32);
line.Property(l => l.UnitPrice).HasPrecision(12, 2);
line.ToTable(t => t.HasCheckConstraint("CK_OrderLine_Quantity", "\"Quantity\" BETWEEN 1 AND 100"));
}
}
internal sealed class IdempotencyRecordConfiguration : IEntityTypeConfiguration<IdempotencyRecord>
{
public void Configure(EntityTypeBuilder<IdempotencyRecord> record)
{
// The composite key is the unique constraint that makes a concurrent retry fail instead of duplicating.
record.HasKey(r => new { r.CustomerId, r.Key });
record.Property(r => r.Key).HasMaxLength(64);
record.Property(r => r.RequestHash).HasMaxLength(64);
record.HasIndex(r => r.CreatedAt); // the nightly purge of keys older than 24 hours
}
}
internal sealed class OutboxMessageConfiguration : IEntityTypeConfiguration<OutboxMessage>
{
public void Configure(EntityTypeBuilder<OutboxMessage> message)
{
message.Property(m => m.Type).HasMaxLength(100);
message.Property(m => m.Payload).HasColumnType("jsonb");
message.HasIndex(m => m.OccurredAt).HasFilter("\"PublishedAt\" IS NULL");
}
}
public sealed record OrderSummary(long Id, OrderStatus Status, decimal Total, DateTimeOffset CreatedAt);
public sealed record OrderCursor(DateTimeOffset CreatedAt, long Id)
{
public string Encode() => Base64Url.EncodeToString(Encoding.UTF8.GetBytes($"{CreatedAt.UtcTicks}:{Id}"));
public static bool TryDecode(string? value, out OrderCursor cursor)
{
cursor = new OrderCursor(default, 0);
if (string.IsNullOrEmpty(value) || value.Length > 64) return false;
try
{
var parts = Encoding.UTF8.GetString(Base64Url.DecodeFromChars(value)).Split(':');
if (parts.Length != 2 || !long.TryParse(parts[0], out var ticks) || !long.TryParse(parts[1], out var id)) return false;
cursor = new OrderCursor(new DateTimeOffset(ticks, TimeSpan.Zero), id);
return true;
}
catch (FormatException)
{
return false;
}
}
}
public static class OrderQueries
{
public static async Task<(List<OrderSummary> Items, string? Next)> ListForCustomerAsync(
OrdersDbContext db, long customerId, string? after, int limit, CancellationToken ct)
{
var query = db.Orders.AsNoTracking().Where(o => o.CustomerId == customerId);
if (OrderCursor.TryDecode(after, out var c))
query = query.Where(o => o.CreatedAt < c.CreatedAt || (o.CreatedAt == c.CreatedAt && o.Id < c.Id));
var rows = await query
.OrderByDescending(o => o.CreatedAt).ThenByDescending(o => o.Id)
.Take(limit + 1)
.Select(o => new OrderSummary(o.Id, o.Status, o.Total, o.CreatedAt))
.ToListAsync(ct);
if (rows.Count <= limit) return (rows, null);
var last = rows[limit - 1];
return (rows[..limit], new OrderCursor(last.CreatedAt, last.Id).Encode());
}
}When to use it
- Page through a customer's 4,000 orders 20 at a time with a stable cursor, each page served by one index seek on (CustomerId, CreatedAt, Id).
- Return 412 when a staff member cancels order 1042 using an ETag that is older than a shipping update made a second earlier.
- Run the AddOrderCancellationReason migration bundle as a Kubernetes Job before the new API image starts rolling out.
More examples
Migrations from laptop to production
The idempotent script is for review: it shows the exact DDL, and a DBA can read it in a pull request. The bundle is what runs in production; it needs no SDK and no source, only a connection string, so it fits a Kubernetes Job in the release pipeline. The migrator connection belongs to a role that may run DDL, while the API's own role should only read and write rows, so a compromised API cannot drop tables.
# Create a migration and read what it will do before committing it
dotnet ef migrations add AddOrderCancellationReason \
--project src/Orders.Api --output-dir Data/Migrations
dotnet ef migrations script --idempotent \
--project src/Orders.Api --output artifacts/migrate.sql
# CI: build a self-contained bundle for the runtime image's platform
dotnet ef migrations bundle --project src/Orders.Api \
--configuration Release --runtime linux-x64 --self-contained \
--output artifacts/efbundle --force
# Release: run it once, before the new pods start
./efbundle --connection "$ORDERS_DB_MIGRATOR_CONNECTION"Cancel guarded by If-Match and xmin
Loading the order and saving it happen in one request, so the loaded <code>xmin</code> alone only protects the few milliseconds in between. Setting <code>OriginalValue</code> to the version the client saw makes the UPDATE fail when anyone changed the order since the client read it, which is the conflict users actually hit. A missing <code>If-Match</code> is 428 Precondition Required, and after the save EF Core reads the new <code>xmin</code> back for the next ETag.
using Microsoft.AspNetCore.Http.HttpResults;
using Microsoft.AspNetCore.Mvc;
using Microsoft.EntityFrameworkCore;
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddDbContext<OrdersDbContext>(o => o.UseNpgsql(builder.Configuration.GetConnectionString("Orders")));
builder.Services.AddProblemDetails();
var app = builder.Build();
app.MapPost("/api/orders/{id:long}/cancel", async Task<Results<NoContent, NotFound, ProblemHttpResult>> (
long id, [FromHeader(Name = "If-Match")] string? ifMatch, OrdersDbContext db, HttpResponse response, CancellationToken ct) =>
{
if (!uint.TryParse(ifMatch?.Trim('"'), out var expected))
return TypedResults.Problem(statusCode: 428, title: "If-Match header with the order's ETag is required");
var order = await db.Orders.FirstOrDefaultAsync(o => o.Id == id, ct);
if (order is null) return TypedResults.NotFound();
if (order.Status is "Shipped")
return TypedResults.Problem(statusCode: 409, title: "Order already shipped");
// Compare against the client's version, not the one just loaded.
db.Entry(order).Property(o => o.Version).OriginalValue = expected;
order.Status = "Cancelled";
try
{
await db.SaveChangesAsync(ct);
}
catch (DbUpdateConcurrencyException)
{
return TypedResults.Problem(statusCode: 412, title: "The order changed since you read it");
}
response.Headers.ETag = $"\"{order.Version}\"";
return TypedResults.NoContent();
});
app.Run();
public sealed class Order
{
public long Id { get; set; }
public string Status { get; set; } = "Pending";
public uint Version { get; set; }
}
public sealed class OrdersDbContext(DbContextOptions<OrdersDbContext> options) : DbContext(options)
{
public DbSet<Order> Orders => Set<Order>();
protected override void OnModelCreating(ModelBuilder modelBuilder) =>
modelBuilder.Entity<Order>().Property(o => o.Version).IsRowVersion();
}Order, idempotency record and outbox in one transaction
The order, its idempotency record and the <code>OrderPlaced</code> outbox row commit together or not at all, so there is never an order without its key or an event for an order that rolled back. An explicit transaction is needed because the order id only exists after the first <code>SaveChangesAsync</code>. The first lookup handles the common retry; the unique violation (SQLSTATE 23505) handles two retries racing, where the loser rolls back and returns the winner's order. With <code>EnableRetryOnFailure</code> on, wrap the whole method in <code>db.Database.CreateExecutionStrategy().ExecuteAsync</code>, because a user-started transaction cannot be retried piecemeal.
using System.Text.Json;
using Microsoft.EntityFrameworkCore;
using Npgsql;
public sealed class OrderPlacement(OrdersDbContext db, TimeProvider clock)
{
public async Task<long> PlaceAsync(long customerId, string idempotencyKey, string requestHash,
IReadOnlyList<(string Sku, int Quantity, decimal UnitPrice)> lines, CancellationToken ct)
{
var existing = await db.IdempotencyRecords.AsNoTracking()
.FirstOrDefaultAsync(r => r.CustomerId == customerId && r.Key == idempotencyKey, ct);
if (existing is not null)
return existing.RequestHash == requestHash ? existing.OrderId : throw new IdempotencyKeyReusedException();
var now = clock.GetUtcNow();
await using var tx = await db.Database.BeginTransactionAsync(ct);
try
{
var order = new Order { CustomerId = customerId, CreatedAt = now, Total = lines.Sum(l => l.Quantity * l.UnitPrice) };
db.Orders.Add(order);
await db.SaveChangesAsync(ct); // assigns order.Id; nothing is visible to others until Commit
db.IdempotencyRecords.Add(new IdempotencyRecord
{
CustomerId = customerId, Key = idempotencyKey, RequestHash = requestHash, OrderId = order.Id, CreatedAt = now,
});
db.Outbox.Add(new OutboxMessage
{
Id = Guid.CreateVersion7(), Type = "OrderPlaced", OccurredAt = now,
Payload = JsonSerializer.Serialize(new { OrderId = order.Id, order.CustomerId, order.Total }),
});
await db.SaveChangesAsync(ct);
await tx.CommitAsync(ct);
return order.Id;
}
catch (DbUpdateException ex) when (ex.InnerException is PostgresException { SqlState: PostgresErrorCodes.UniqueViolation })
{
// A concurrent retry with the same key committed first: undo ours and answer with its order.
await tx.RollbackAsync(ct);
db.ChangeTracker.Clear();
var winner = await db.IdempotencyRecords.AsNoTracking()
.FirstAsync(r => r.CustomerId == customerId && r.Key == idempotencyKey, ct);
return winner.RequestHash == requestHash ? winner.OrderId : throw new IdempotencyKeyReusedException();
}
}
}
public sealed class IdempotencyKeyReusedException() : Exception("Idempotency-Key reused with a different request body.");
public sealed class Order
{
public long Id { get; set; }
public long CustomerId { get; set; }
public decimal Total { get; set; }
public DateTimeOffset CreatedAt { get; set; }
}
public sealed class IdempotencyRecord
{
public long CustomerId { get; set; }
public required string Key { get; set; }
public required string RequestHash { get; set; }
public long OrderId { get; set; }
public DateTimeOffset CreatedAt { get; set; }
}
public sealed class OutboxMessage
{
public Guid Id { get; set; }
public required string Type { get; set; }
public required string Payload { get; set; }
public DateTimeOffset OccurredAt { get; set; }
}
public sealed class OrdersDbContext(DbContextOptions<OrdersDbContext> options) : DbContext(options)
{
public DbSet<Order> Orders => Set<Order>();
public DbSet<IdempotencyRecord> IdempotencyRecords => Set<IdempotencyRecord>();
public DbSet<OutboxMessage> Outbox => Set<OutboxMessage>();
protected override void OnModelCreating(ModelBuilder modelBuilder) =>
modelBuilder.Entity<IdempotencyRecord>().HasKey(r => new { r.CustomerId, r.Key });
}
Discussion