Running Tests in CI
A Magento CI build needs a database and a search engine, a warm Composer cache, and a static check that fails the job.
vendor/bin/phpcs --standard=Magento2 app/code/Vendor --extensions=php,phtmlMagento's suites differ in what they need from the machine, so a pipeline is really several jobs with different service requirements. Running them as one job means the slowest dependency gates the fastest feedback.
What each suite needs
- Static and unit: PHP and the installed dependencies. No services at all, so these run first and fail in under a minute.
- Integration: a database the job may destroy, and a reachable search engine, because the framework runs
setup:installbefore your first test. RabbitMQ only if you have message queue tests. - API functional and MFTF: an already installed store served over HTTP at a known base URL, and for MFTF also a Selenium server and Chrome.
Composer
Cache Composer's own cache directory, which composer config --global cache-dir reports, rather than vendor; a cached vendor survives a lockfile change and produces builds that pass on stale code. Install with composer install --no-interaction --no-progress --prefer-dist. Adobe Commerce packages come from repo.magento.com and need an auth.json built from secrets; Open Source does not.
Failing on static violations
bin/magento dev:tests:run static is shorthand for running PHPUnit against dev/tests/static/phpunit.xml; both are the same run. Either way the suite exits non-zero when PHP_CodeSniffer, PHPMD or PHPStan reports anything, and that exit code is what fails the build. Nothing else is needed, and no step should swallow it with || true.
Example
<?php
// dev/tests/integration/etc/install-config-mysql.php
declare(strict_types=1);
use Magento\TestFramework\Bootstrap;
return [
'db-host' => getenv('MYSQL_HOST') ?: '127.0.0.1',
'db-user' => getenv('MYSQL_USER') ?: 'root',
'db-password' => getenv('MYSQL_PASSWORD') ?: '',
'db-name' => getenv('MYSQL_DATABASE') ?: 'magento_integration_tests',
'db-prefix' => '',
'backend-frontname' => 'backend',
'search-engine' => 'opensearch',
'opensearch-host' => getenv('OPENSEARCH_HOST') ?: '127.0.0.1',
'opensearch-port' => (int)(getenv('OPENSEARCH_PORT') ?: 9200),
'admin-user' => Bootstrap::ADMIN_NAME,
'admin-password' => Bootstrap::ADMIN_PASSWORD,
'admin-email' => Bootstrap::ADMIN_EMAIL,
'admin-firstname' => Bootstrap::ADMIN_FIRSTNAME,
'admin-lastname' => Bootstrap::ADMIN_LASTNAME,
];When to use it
- Splitting a single slow job into fast static and unit checks plus a slower integration job.
- Fixing a pipeline that passes on a cached vendor directory while the lockfile has moved on.
- Making a coding standard violation fail the build instead of being noted in a log nobody reads.
- Standing up the database and search engine services an integration job needs from scratch.
More examples
A project ruleset so one command covers the whole module
With this file at the project root, vendor/bin/phpcs needs no arguments: it reads the paths, the extensions and the standard from here, and the CI step is the same command a developer runs locally. The Magento2 standard comes from the magento/magento-coding-standard dev dependency, registered by the phpcodesniffer-composer-installer plugin, so composer install must be allowed to run plugins. Include .phtml in the extensions: that is where unescaped output lives.
<!-- phpcs.xml -->
<ruleset name="VendorStandard">
<description>Magento2 coding standard applied to our own modules only.</description>
<file>app/code/Vendor</file>
<arg name="extensions" value="php,phtml"/>
<arg name="colors"/>
<arg value="sp"/>
<exclude-pattern>*/Test/Mftf/*</exclude-pattern>
<exclude-pattern>*/vendor/*</exclude-pattern>
<rule ref="Magento2"/>
</ruleset>Running only the PHP coding standard suite
Copy phpunit.xml.dist to phpunit.xml and keep the suites you want; the .dist file also declares Less, HTML, GraphQL and XML suites. Run it as vendor/bin/phpunit -c dev/tests/static/phpunit.xml. The integrity suite is worth keeping in CI: it is what catches a class referenced in di.xml that does not exist, and a circular dependency between modules, before compilation fails in a deploy.
<!-- dev/tests/static/phpunit.xml -->
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/12.5/phpunit.xsd"
colors="true" columns="max"
beStrictAboutTestsThatDoNotTestAnything="false"
bootstrap="./framework/bootstrap.php">
<testsuites>
<testsuite name="PHP Coding Standard Verification">
<file>testsuite/Magento/Test/Php/LiveCodeTest.php</file>
</testsuite>
<testsuite name="Code Integrity Tests">
<directory>testsuite/Magento/Test/Integrity</directory>
</testsuite>
</testsuites>
<php>
<ini name="date.timezone" value="UTC"/>
<const name="TESTCODESTYLE_IS_FULL_SCAN" value="0"/>
</php>
</phpunit>Trimming integration-test runtime in a container
Keep TESTS_CLEANUP enabled in CI so every build installs a clean store and a leftover row cannot make a build pass; set it to disabled only in your local copy, where reinstalling on every run wastes minutes. memory_limit -1 is the documented fix for integration runs that die partway through on a container's default limit. These constants are read at the start of the run, so there is no command to apply the file.
<!-- dev/tests/integration/phpunit.xml (extract) -->
<php>
<includePath>.</includePath>
<includePath>testsuite</includePath>
<ini name="date.timezone" value="UTC"/>
<ini name="memory_limit" value="-1"/>
<const name="TESTS_INSTALL_CONFIG_FILE" value="etc/install-config-mysql.php"/>
<const name="TESTS_GLOBAL_CONFIG_DIR" value="../../../app/etc"/>
<const name="TESTS_CLEANUP" value="enabled"/>
</php>
Discussion