Tests That Stay Useful
Name a test after the behaviour, give it one reason to fail, and know what a coverage number does not tell you.
dotnet test --collect:"XPlat Code Coverage"Tests are code you will read far more often than you write. A suite that is hard to read gets deleted during the next refactor, which means the effort of writing it is lost. Four habits keep a suite worth keeping.
Name the behaviour
A failing test's name is often all you get in a build log, so make it a sentence. Method_Situation_ExpectedResult works well: Withdraw_AmountOverBalance_Throws. Names like Test1 or CartWorks force whoever is on call to open the file.
One reason to fail
Each test should break for exactly one reason. When a test asserts four unrelated things, the first failure hides the other three, and you learn one fact per run instead of four. Split it into separate tests with separate names. Several assertions about one outcome are fine; several assertions about different rules are not.
Assert on what you mean
Assert the behaviour, not the detail that happens to come with it. Exact exception messages, log text, property ordering, whitespace: these change for cosmetic reasons and then dozens of tests go red while the code is still correct. Assert on the status code, the type, the value you care about.
What coverage tells you
Coverage is the percentage of your lines that ran during the test run. A low number reliably names code no test touches. A high number proves only that the lines ran, not that anything checked the result, and not that the inputs included the empty string or the maximum value. Use it to find the gaps, never as the target.
Example
using System;
using Xunit;
namespace Billing.Tests;
public record Invoice(string Customer, int AmountCents, bool Paid);
public static class Invoices
{
public static Invoice MarkPaid(Invoice invoice)
{
if (invoice.Paid)
{
throw new InvalidOperationException("Invoice is already paid.");
}
return invoice with { Paid = true };
}
}
public class InvoiceTests
{
// Name reads as: method, situation, expected result.
[Fact]
public void MarkPaid_UnpaidInvoice_SetsPaid()
{
Invoice invoice = new Invoice("acme", 1000, Paid: false);
Invoice result = Invoices.MarkPaid(invoice);
Assert.True(result.Paid); // the one reason this test can fail
}
[Fact]
public void MarkPaid_AlreadyPaidInvoice_Throws()
{
Invoice invoice = new Invoice("acme", 1000, Paid: true);
// The type is the behaviour. The message text is incidental,
// so it is deliberately not asserted here.
Assert.Throws<InvalidOperationException>(() => Invoices.MarkPaid(invoice));
}
}
// dotnet test reports: Failed: 0, Passed: 2, Skipped: 0, Total: 2When to use it
- Rename a suite of vague test names so a CI failure is diagnosable from the log alone.
- Split a long test that asserts several unrelated rules into separate named tests.
- Use a coverage report to find the files no test touches, before adding tests at random.
More examples
One vague test, then three named ones
ShoppingCart here has Add, a Lines list and an IsEmpty property. The before version does cover more behaviour in fewer lines, which is why it is tempting; it just reports badly. Break the quantity-summing rule and the split version tells you so by name, while CartWorks only says something about carts is wrong. Do the split when a test's assertions start describing different rules, not merely because there are several of them.
using Xunit;
namespace Cart.Tests;
public class CartTests
{
// Before: one name that says nothing, four unrelated reasons to fail.
[Fact]
public void CartWorks()
{
ShoppingCart cart = new ShoppingCart();
cart.Add("bolt", 2);
Assert.Single(cart.Lines);
Assert.Equal("bolt", cart.Lines[0].Sku);
Assert.Equal(2, cart.Lines[0].Quantity);
Assert.False(cart.IsEmpty);
}
// After: each test states one rule the cart must obey.
[Fact]
public void Add_NewSku_AddsOneLine()
{
ShoppingCart cart = new ShoppingCart();
cart.Add("bolt", 2);
Assert.Single(cart.Lines);
}
[Fact]
public void Add_SameSkuTwice_SumsQuantity()
{
ShoppingCart cart = new ShoppingCart();
cart.Add("bolt", 2);
cart.Add("bolt", 3);
Assert.Equal(5, cart.Lines[0].Quantity);
}
[Fact]
public void Add_AnyItem_MakesCartNotEmpty()
{
ShoppingCart cart = new ShoppingCart();
cart.Add("bolt", 2);
Assert.False(cart.IsEmpty);
}
}What a 100% coverage number can hide
One test, one line of production code, 100% coverage, and a double-hyphen defect sitting in plain sight. Coverage can only tell you a line ran with the inputs you supplied. Read it the useful way round: a file at 0% is a real gap worth a look, and a file at 100% is a prompt to ask which inputs were used.
using Xunit;
namespace Slugs.Tests;
public static class Slugs
{
public static string ToSlug(string title)
{
return title.Trim().ToLowerInvariant().Replace(' ', '-');
}
}
public class SlugTests
{
// This single test executes every line of ToSlug: 100% line coverage.
[Fact]
public void ToSlug_TwoWords_JoinsWithHyphen()
{
Assert.Equal("hello-world", Slugs.ToSlug(" Hello World "));
}
// Still untested, and the second one is arguably a bug:
// ToSlug("") => ""
// ToSlug("a b") => "a--b"
}Running coverage and filtering the suite
The coverage file is XML meant for a report tool such as ReportGenerator, or for your CI system to chart over time; it is not meant to be read directly. The --filter switch is what you use while fixing one test, so you wait seconds instead of minutes. Run the full suite in Release configuration before you push, since that is what the build server will do.
# The xunit template already references coverlet.collector, so this works
# with no extra packages. It writes TestResults/<id>/coverage.cobertura.xml
dotnet test --collect:"XPlat Code Coverage"
# Run one test, or everything whose full name contains "Slug"
dotnet test --filter "FullyQualifiedName~Slug"
# Run the suite the way the build server will, in Release configuration
dotnet test --configuration Release
Discussion