Measuring Before Optimising
Use BenchmarkDotNet for a microbenchmark and a load test for an endpoint, because a guess about what is slow is usually wrong.
BenchmarkRunner.Run<JoinBenchmarks>();Guesses about performance are unreliable for a simple reason: the slow part of a program is rarely the complicated-looking part. It is usually a call made more times than you thought, or a wait on something outside the process. Measurement is cheap, and a rewrite based on a guess is not.
Two different tools
A microbenchmark times one small piece of code, millions of times, in isolation. Use it to choose between two implementations of the same function. A load test drives your running service with concurrent requests. Use it to find where latency starts climbing. A microbenchmark cannot tell you a connection pool is too small, and a load test cannot tell you which of two parsers is faster.
What BenchmarkDotNet does for you
Timing code by hand with a stopwatch gives wrong answers, because the first run pays for just-in-time compilation, the garbage collector may run mid-measurement, and a single sample has no error bar. BenchmarkDotNet builds your benchmark into its own project in Release configuration, warms it up, runs enough iterations to get a stable mean, and reports mean, error and standard deviation. [MemoryDiagnoser] adds an Allocated column, which is often the number that actually matters.
Reading the result
Compare against the Baseline = true row; BenchmarkDotNet prints a Ratio column against it. Treat anything under a few percent as noise. For a load test, read percentiles rather than the mean: an average hides the slow tail that users notice.
Shorthand
private string[] _parts = []; is a collection expression, added in C# 12, and compiles to the same thing as Array.Empty<string>().
Example
// Package: BenchmarkDotNet. Run with: dotnet run -c Release
// The result table is specific to the machine it ran on, so it is not
// reproduced here. Columns: Mean, Error, StdDev, Ratio, Allocated.
using System.Text;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
BenchmarkRunner.Run<JoinBenchmarks>();
// MemoryDiagnoser adds the Allocated column to the table.
[MemoryDiagnoser]
public class JoinBenchmarks
{
private readonly string[] _parts =
Enumerable.Range(0, 1000).Select(i => "part" + i).ToArray();
// Every other row is reported as a ratio against this one.
[Benchmark(Baseline = true)]
public string Concat()
{
string result = "";
foreach (string part in _parts)
{
// Each += builds a whole new string.
result += part;
}
return result;
}
[Benchmark]
public string Builder()
{
var builder = new StringBuilder();
foreach (string part in _parts)
{
builder.Append(part);
}
return builder.ToString();
}
}When to use it
- Decide whether a rewrite is worth it by timing the old and new version side by side.
- Prove a change removed allocations rather than only lines of code.
- Find the request rate at which an endpoint's latency starts to climb.
- Settle an argument about which of two parsing approaches is faster, with numbers.
More examples
Varying the input size, because the winner can change
Results at one input size mislead often: string concatenation beats StringBuilder for four items and loses badly for ten thousand, because the builder's own allocation has to be paid back. Params makes the crossover visible instead of leaving you to pick a constant and hope. GlobalSetup is where you build the input, since time spent there is not counted.
using BenchmarkDotNet.Attributes;
[MemoryDiagnoser]
public class JoinBenchmarks
{
private string[] _parts = [];
// One row per value, for each benchmark method.
[Params(4, 100, 10_000)]
public int Count { get; set; }
// Runs once per Count value, outside the measurement.
[GlobalSetup]
public void Setup() =>
_parts = Enumerable.Range(0, Count).Select(i => "part" + i).ToArray();
[Benchmark]
public string Join() => string.Join("", _parts);
}Load testing the endpoint, and watching it from inside
Run the two together and you can tell which side is slow. If the client reports 400 ms while http.server.request.duration reports 20 ms, the time is in queuing, the network or the client, not in your handler. Start with a load you know the service survives and raise it until latency climbs; the rate just below that point is your real capacity number.
# Terminal 1: the service, built the way production builds it.
dotnet run -c Release
# Terminal 2: watch the server's own view of latency live.
dotnet-counters monitor -n MyApi --counters Microsoft.AspNetCore.Hosting
# Terminal 3: 50 virtual users for one minute.
k6 run --vus 50 --duration 1m load.jsChecking the two versions agree before timing them
The fastest wrong answer wins every benchmark. A faster version that skips a case, or returns an empty result, will show a glorious ratio and then fail in production. One assertion in GlobalSetup makes the comparison honest, and it costs nothing because setup time is not measured.
using System.Text;
using BenchmarkDotNet.Attributes;
[MemoryDiagnoser]
public class JoinBenchmarks
{
private readonly string[] _parts = ["a", "b", "c"];
// Throws before any timing happens if the implementations diverge.
[GlobalSetup]
public void Verify()
{
if (Concat() != Builder())
{
throw new InvalidOperationException("implementations disagree");
}
}
[Benchmark(Baseline = true)]
public string Concat()
{
string result = "";
foreach (string part in _parts)
{
result += part;
}
return result;
}
[Benchmark]
public string Builder()
{
var builder = new StringBuilder();
foreach (string part in _parts)
{
builder.Append(part);
}
return builder.ToString();
}
}
Discussion