Measuring Properly
Hold everything constant except your change: a query-count test, a load test with percentiles, and a profiler you can switch on in production.
framework: { profiler: { collect: false, collect_parameter: 'debug_profile' } }A performance change you cannot measure is a guess. Measuring properly means holding everything constant except the change: the same dataset, the same host, the same concurrency, caches in the same state, and the result written down somewhere you can read it again after the next release.
The free number: query count
Query count is deterministic, so it belongs in a test rather than a benchmark. A functional test can switch profiling on for one request and assert the number of queries, which turns "the list page got slow again" into a failing build. Pick the ten endpoints that matter and give each a budget.
The number that needs a tool: latency under load
One request through a browser tells you little. Use a load generator - k6 is one - against a dataset of realistic size. Discard the first run while caches fill, then record a percentile rather than an average: the 95th percentile is what your slowest users actually experience, and an average hides it. Keep the script in the repository so the release after next runs the same test.
Profiling in production
Set framework.profiler.collect to false in production and give collect_parameter a name. Profiling is then off for every request except those carrying that query parameter, which lets you capture one real slow request with real data. The toolbar and the /_profiler pages come from WebProfilerBundle, which the recipe wires up in dev only, so in production you are collecting profiles rather than browsing them unless you enable and protect those routes deliberately.
Example
<?php
// tests/Controller/ArticleControllerTest.php
declare(strict_types=1);
namespace App\Tests\Controller;
use Symfony\Bundle\FrameworkBundle\Test\WebTestCase;
use Symfony\Component\HttpKernel\Profiler\Profile;
final class ArticleControllerTest extends WebTestCase
{
public function testArticleListStaysWithinItsQueryBudget(): void
{
$client = static::createClient();
// Collect profiler data for the next request only. The profiler must
// be enabled in config for the test environment, with collect: false.
$client->enableProfiler();
$client->request('GET', '/articles');
self::assertResponseIsSuccessful();
$profile = $client->getProfile();
self::assertInstanceOf(Profile::class, $profile);
// The page needs: articles, their authors, the tag counts. Three.
// A change that reintroduces an N+1 fails here instead of shipping.
self::assertSame(3, $profile->getCollector('db')->getQueryCount());
// Milliseconds spent inside the framework. Keep the budget loose:
// this number moves with the machine running the suite.
self::assertLessThan(500, $profile->getCollector('time')->getDuration());
}
}When to use it
- Prove a caching change helped before merging it, not after deploying it.
- Keep a query-count budget on the handful of endpoints that carry your traffic.
- Capture a profile for one slow production request without profiling all of them.
- Compare 95th-percentile latency between two releases on the same dataset.
More examples
A load test you can re-run next release
Thresholds are what make this a test rather than a demonstration: the run exits non-zero when the 95th percentile crosses the line, so it can sit in a pipeline. Committing the script matters more than the exact numbers, because a benchmark is only comparable against another run of the same script.
// tests/load/articles.js
import http from 'k6/http';
export const options = {
vus: 20,
duration: '60s',
thresholds: {
// 95% of requests under 300ms, fewer than 1% failures.
http_req_duration: ['p(95)<300'],
http_req_failed: ['rate<0.01'],
},
};
export default function () {
http.get('https://staging.example.com/articles');
}
// k6 run tests/load/articles.js
//
// The same two options given on the command line, long and short form -
// they are the same flags:
// k6 run --vus 20 --duration 60s tests/load/articles.js
// k6 run -u 20 -d 60s tests/load/articles.jsThe profiler in production, behind a parameter
This is how you profile the request that is actually slow, with production data and a production cache, instead of trying to reproduce it locally. Treat the parameter name as sensitive: anyone who knows it can make your application do collector work and write a profile to disk, so choose something unguessable and consider restricting it to authenticated staff.
# config/packages/profiler.yaml
when@prod:
framework:
profiler:
# The service exists, but collects nothing by default.
enabled: true
collect: false
# Add ?debug_profile=1 to a URL to collect that one request.
collect_parameter: 'debug_profile'
# Skip sub-requests such as rendered fragments.
only_main_requests: true
dsn: 'file:%kernel.cache_dir%/profiler'Holding everything else constant
Most performance arguments are really arguments about method: one side measured a warm cache and the other a cold one. Writing the steps down as a script removes that. Saving the output under the commit hash gives you the comparison between releases that a number in a chat message never does.
# bin/measure.sh - run once before the change and once after, same host
set -eu
# 1. Same build both times.
composer install --no-dev --no-scripts
composer dump-autoload -a --no-dev
APP_ENV=prod php bin/console cache:clear
APP_ENV=prod php bin/console cache:warmup
# 2. Same data both times: restore the identical dump, never a live copy.
mysql app < var/perf/dataset.sql
# 3. Warm run, discarded. Caches, OPcache and the pool are all cold here.
k6 run -u 20 -d 20s tests/load/articles.js > /dev/null
# 4. The run you keep, filed under the commit it measured.
mkdir -p var/perf/results
k6 run -u 20 -d 60s tests/load/articles.js \
| tee "var/perf/results/$(git rev-parse --short HEAD).txt"
Discussion