ADO.NET with Npgsql

Use NpgsqlDataSource, commands, typed parameters, data readers, batches and binary COPY to talk to PostgreSQL with no ORM in between.

Syntaxawait using var cmd = dataSource.CreateCommand("SELECT name, unit_price FROM products WHERE sku = $1"); cmd.Parameters.Add(new NpgsqlParameter<string> { TypedValue = sku }); await using var reader = await cmd.ExecuteReaderAsync(ct);

Under EF Core and Dapper sits ADO.NET: DbConnection, DbCommand, DbDataReader. Npgsql is the PostgreSQL provider, and using it directly exposes what no abstraction does: binary COPY, batching, streaming readers and LISTEN/NOTIFY.

NpgsqlDataSource is the entry point

An NpgsqlDataSource owns a connection string, its pool and its type mappings. Build it once with NpgsqlDataSourceBuilder (the place for MapEnum and UseLoggerFactory), register it as a singleton and let the container dispose it at shutdown; the Npgsql.DependencyInjection package wraps that as AddNpgsqlDataSource. dataSource.CreateCommand(sql) rents a connection for one execution; OpenConnectionAsync() hands you one for several commands or a transaction. new NpgsqlConnection(connectionString) still works, with a hidden data source and pool per distinct string.

Parameters

The PostgreSQL protocol only knows positional placeholders. Write $1, $2 with unnamed parameters added in order and Npgsql sends your SQL untouched; write @sku with named parameters and Npgsql parses and rewrites it first. NpgsqlParameter<T> with TypedValue avoids boxing. The PostgreSQL type is inferred from the CLR type; set NpgsqlDbType when that is ambiguous, such as Jsonb for a string of JSON.

Readers

Read by ordinal with GetFieldValue<T> or the typed getters, and check IsDBNull first for nullable columns: GetString on NULL throws. Rows stream from the socket, so an open reader holds its connection until disposed. timestamptz reads as a UTC DateTime. A parameter typed as timestamptz (an explicit NpgsqlDbType.TimestampTz, or an EF Core column mapping) throws for a DateTime whose Kind is not Utc and for a DateTimeOffset with a non-zero offset. An untyped Local DateTime is sent as timestamp instead and PostgreSQL converts it silently with the session TimeZone, so always send UTC.

Batches and COPY

NeedUse
One statementdataSource.CreateCommand
Several statements, one round tripNpgsqlBatch
A transactionOpenConnectionAsync + BeginTransactionAsync
Thousands of rows inBinary COPY

Inserting 100,000 rows one INSERT at a time costs 100,000 round trips. BeginBinaryImportAsync("COPY ... FROM STDIN (FORMAT BINARY)") streams the rows in one statement, often an order of magnitude faster or more. Per row call StartRowAsync, then WriteAsync for each column in the listed order, and finish with CompleteAsync: disposing the importer without completing cancels the COPY and nothing is stored. COPY is all or nothing and has no ON CONFLICT, so for upserts COPY into a temporary table and finish with INSERT ... SELECT ... ON CONFLICT.

Production failure modes

A missing await using on a connection, command or reader leaks a pooled connection, which surfaces later as pool exhaustion. Type mistakes, such as comparing a text parameter to an integer column (operator does not exist), appear only at run time, so every hand-written statement needs an integration test against real PostgreSQL. Behind an older PgBouncer in transaction mode, prepared statements (including Max Auto Prepare) break. LISTEN needs a dedicated long-lived connection that calls WaitAsync in a loop; it sits outside the request path but still counts against the pool and max_connections.

Example

Example · csharp
using System.Runtime.CompilerServices;
using Microsoft.AspNetCore.Http.HttpResults;
using Npgsql;
using NpgsqlTypes;

var builder = WebApplication.CreateBuilder(args);

var dataSourceBuilder = new NpgsqlDataSourceBuilder(builder.Configuration.GetConnectionString("Shop")
    ?? throw new InvalidOperationException("ConnectionStrings:Shop is not configured."));
builder.Services.AddSingleton(dataSourceBuilder.Build());

var app = builder.Build();

app.MapGet("/products/{sku}", async Task<Results<Ok<Product>, NotFound>> (string sku, NpgsqlDataSource db, CancellationToken ct) =>
{
    await using var cmd = db.CreateCommand("SELECT sku, name, unit_price, description FROM products WHERE sku = $1");
    cmd.Parameters.Add(new NpgsqlParameter<string> { TypedValue = sku });
    await using var reader = await cmd.ExecuteReaderAsync(ct);
    if (!await reader.ReadAsync(ct)) return TypedResults.NotFound();
    return TypedResults.Ok(new Product(
        reader.GetString(0),
        reader.GetString(1),
        reader.GetDecimal(2),
        reader.IsDBNull(3) ? null : reader.GetString(3)));
});

// Streams rows to the response as they arrive; the connection is held until the last row is written.
app.MapGet("/orders/export", (DateOnly from, NpgsqlDataSource db, CancellationToken ct) =>
    TypedResults.Ok(StreamOrders(db, from, ct)));

app.MapPost("/inventory/counts", async (StockCount[] counts, NpgsqlDataSource db, CancellationToken ct) =>
{
    await using var connection = await db.OpenConnectionAsync(ct);
    await using var importer = await connection.BeginBinaryImportAsync(
        "COPY stock_counts (sku, warehouse_id, quantity, counted_at) FROM STDIN (FORMAT BINARY)", ct);
    foreach (var count in counts)
    {
        await importer.StartRowAsync(ct);
        await importer.WriteAsync(count.Sku, NpgsqlDbType.Text, ct);
        await importer.WriteAsync(count.WarehouseId, NpgsqlDbType.Integer, ct);
        await importer.WriteAsync(count.Quantity, NpgsqlDbType.Integer, ct);
        // timestamptz accepts only UTC; UtcDateTime has Kind=Utc whatever offset the client sent.
        await importer.WriteAsync(count.CountedAt.UtcDateTime, NpgsqlDbType.TimestampTz, ct);
    }
    ulong imported = await importer.CompleteAsync(ct);
    return TypedResults.Ok(new { imported });
});

app.Run();

static async IAsyncEnumerable<OrderRow> StreamOrders(
    NpgsqlDataSource db, DateOnly from, [EnumeratorCancellation] CancellationToken ct)
{
    await using var cmd = db.CreateCommand(
        "SELECT id, customer_id, total_amount, placed_at FROM orders WHERE placed_at >= $1 ORDER BY id");
    cmd.Parameters.Add(new NpgsqlParameter<DateTime>
    {
        TypedValue = from.ToDateTime(TimeOnly.MinValue, DateTimeKind.Utc)
    });
    await using var reader = await cmd.ExecuteReaderAsync(ct);
    while (await reader.ReadAsync(ct))
    {
        yield return new OrderRow(
            reader.GetInt64(0),
            reader.GetInt64(1),
            reader.GetDecimal(2),
            reader.GetFieldValue<DateTime>(3));
    }
}

public sealed record Product(string Sku, string Name, decimal UnitPrice, string? Description);
public sealed record OrderRow(long Id, long CustomerId, decimal TotalAmount, DateTime PlacedAt);
public sealed record StockCount(string Sku, int WarehouseId, int Quantity, DateTimeOffset CountedAt);

When to use it

  • Load a 200,000-row nightly stock count from a warehouse system with binary COPY in seconds instead of minutes of row-by-row INSERTs
  • Stream a CSV export of a year of orders to the client row by row without buffering the result set in memory
  • Reserve stock for every line of an order in one round trip with NpgsqlBatch inside a transaction

More examples

Reserve stock for every order line in one round trip with NpgsqlBatch

All the <code>UPDATE</code> statements travel in one network round trip, and <code>RecordsAffected</code> on each <code>NpgsqlBatchCommand</code> reports its own row count, so the code can tell which line ran out of stock. The explicit transaction makes the all-or-nothing rule visible; the conditional <code>WHERE</code> keeps concurrent reservations from overselling without a separate read. Sorting the lines by SKU gives every transaction the same lock order; without it, two orders holding the same SKUs in opposite order can deadlock (<code>40P01</code>).

Example · csharp
using Npgsql;

public sealed record LineToReserve(string Sku, int Quantity);

public sealed class StockReservations(NpgsqlDataSource dataSource)
{
    public async Task<bool> ReserveAsync(IReadOnlyList<LineToReserve> lines, CancellationToken ct)
    {
        await using var connection = await dataSource.OpenConnectionAsync(ct);
        await using var tx = await connection.BeginTransactionAsync(ct);
        await using var batch = new NpgsqlBatch(connection, tx);
        // Lock rows in one global order (by SKU) so two concurrent orders cannot deadlock.
        foreach (var line in lines.OrderBy(l => l.Sku, StringComparer.Ordinal))
        {
            batch.BatchCommands.Add(new NpgsqlBatchCommand(
                "UPDATE inventory SET reserved = reserved + $1 WHERE sku = $2 AND on_hand - reserved >= $1")
            {
                Parameters =
                {
                    new NpgsqlParameter<int> { TypedValue = line.Quantity },
                    new NpgsqlParameter<string> { TypedValue = line.Sku },
                },
            });
        }

        await batch.ExecuteNonQueryAsync(ct);
        // A line that matched no row means not enough stock: undo the whole reservation.
        if (batch.BatchCommands.Any<NpgsqlBatchCommand>(c => c.RecordsAffected != 1))
        {
            await tx.RollbackAsync(ct);
            return false;
        }
        await tx.CommitAsync(ct);
        return true;
    }
}

COPY into a staging table, then upsert

COPY cannot resolve conflicts, so it fills a temporary table that disappears at commit and a single <code>INSERT ... SELECT ... ON CONFLICT</code> merges it. Without <code>DISTINCT ON</code> a file that lists one SKU twice fails with <code>ON CONFLICT DO UPDATE command cannot affect row a second time</code>. The <code>WHERE</code> on the update ignores prices older than the stored one, so replaying an old file changes nothing.

Example · csharp
using Npgsql;
using NpgsqlTypes;

public sealed record PriceUpdate(string Sku, decimal UnitPrice, DateTime EffectiveAtUtc);

public sealed class PriceImporter(NpgsqlDataSource dataSource)
{
    public async Task<int> ImportAsync(IAsyncEnumerable<PriceUpdate> rows, CancellationToken ct)
    {
        await using var connection = await dataSource.OpenConnectionAsync(ct);
        await using var tx = await connection.BeginTransactionAsync(ct);

        await using (var create = new NpgsqlCommand(
            "CREATE TEMP TABLE prices_in (sku text, unit_price numeric(12,2), effective_at timestamptz) ON COMMIT DROP",
            connection, tx))
        {
            await create.ExecuteNonQueryAsync(ct);
        }

        // The importer must be completed and disposed before the connection can run another command.
        await using (var importer = await connection.BeginBinaryImportAsync(
            "COPY prices_in (sku, unit_price, effective_at) FROM STDIN (FORMAT BINARY)", ct))
        {
            await foreach (var row in rows.WithCancellation(ct))
            {
                await importer.StartRowAsync(ct);
                await importer.WriteAsync(row.Sku, NpgsqlDbType.Text, ct);
                await importer.WriteAsync(row.UnitPrice, NpgsqlDbType.Numeric, ct);
                await importer.WriteAsync(row.EffectiveAtUtc, NpgsqlDbType.TimestampTz, ct);
            }
            await importer.CompleteAsync(ct);
        }

        // DISTINCT ON keeps the newest row per SKU: ON CONFLICT cannot touch the same row twice.
        await using var upsert = new NpgsqlCommand("""
            INSERT INTO product_prices (sku, unit_price, effective_at)
            SELECT DISTINCT ON (sku) sku, unit_price, effective_at
            FROM prices_in
            ORDER BY sku, effective_at DESC
            ON CONFLICT (sku) DO UPDATE
               SET unit_price = EXCLUDED.unit_price, effective_at = EXCLUDED.effective_at
             WHERE product_prices.effective_at < EXCLUDED.effective_at
            """, connection, tx);
        var changed = await upsert.ExecuteNonQueryAsync(ct);
        await tx.CommitAsync(ct);
        return changed;
    }
}

Pitfall: local times into timestamptz

Npgsql infers <code>timestamptz</code> for a <code>DateTime</code> with <code>Kind=Utc</code> and <code>timestamp</code> (without time zone) for any other kind. The broken version therefore does not throw: PostgreSQL's implicit <code>timestamp</code> to <code>timestamptz</code> cast reads the local wall-clock time in the server session's <code>TimeZone</code> and stores a shifted instant. Npgsql only refuses a non-UTC value when the parameter is typed <code>TimestampTz</code>, as in the fix, or when EF Core knows the column type. Store instants as <code>timestamptz</code>, produce them from <code>TimeProvider.GetUtcNow()</code>, and convert to a user's zone only at the edge.

Example · csharp
using Npgsql;
using NpgsqlTypes;

public sealed class PaymentLog(NpgsqlDataSource dataSource, TimeProvider clock)
{
    // Silently wrong: an untyped DateTime with Kind=Local is sent as timestamp (no zone),
    // and PostgreSQL casts it to timestamptz using the session TimeZone, not the app's.
    public async Task RecordBrokenAsync(long paymentId, CancellationToken ct)
    {
        await using var cmd = dataSource.CreateCommand("UPDATE payments SET captured_at = $1 WHERE id = $2");
        cmd.Parameters.Add(new NpgsqlParameter<DateTime> { TypedValue = DateTime.Now });
        cmd.Parameters.Add(new NpgsqlParameter<long> { TypedValue = paymentId });
        await cmd.ExecuteNonQueryAsync(ct);
    }

    // Works: a UTC instant from the injected clock, which tests can also control. The explicit
    // TimestampTz type makes a Local value throw ("Cannot write DateTime with Kind=Local ...").
    public async Task RecordAsync(long paymentId, CancellationToken ct)
    {
        await using var cmd = dataSource.CreateCommand("UPDATE payments SET captured_at = $1 WHERE id = $2");
        cmd.Parameters.Add(new NpgsqlParameter<DateTime>
        {
            TypedValue = clock.GetUtcNow().UtcDateTime,
            NpgsqlDbType = NpgsqlDbType.TimestampTz,
        });
        cmd.Parameters.Add(new NpgsqlParameter<long> { TypedValue = paymentId });
        await cmd.ExecuteNonQueryAsync(ct);
    }
}

Discussion

  • Be the first to comment on this lesson.