diff --git a/developer_manual/architecture.rst b/developer_manual/architecture.rst index 9be3de2..ad81ee7 100644 --- a/developer_manual/architecture.rst +++ b/developer_manual/architecture.rst @@ -25,11 +25,14 @@ The main application areas are: ``src/`` Vue and TypeScript frontend code. -``tests/php/`` - PHP unit tests. +``tests/php/Unit/`` + Isolated PHP unit tests. + +``tests/php/Api/`` and ``tests/php/Integration/`` + PHPUnit tests that exercise a bootstrapped Nextcloud runtime. ``tests/integration/`` - Integration scenarios and their support code. + Behat behavior/integration scenarios and their support code. ``src/tests/`` Frontend unit tests. diff --git a/developer_manual/getting-started/tests.rst b/developer_manual/getting-started/tests.rst index a6324f4..e04e37e 100644 --- a/developer_manual/getting-started/tests.rst +++ b/developer_manual/getting-started/tests.rst @@ -1,78 +1,183 @@ Testing ======= -LibreSign includes multiple test suites (see ``.github/workflows`` for CI jobs). -Below are the most relevant for day-to-day development and how to run them locally. +LibreSign has several test layers. Choose the narrowest layer that proves the +behavior you changed, then broaden validation before opening or updating a pull +request. -.. admonition:: Tips +The application repository is the source of truth for current scripts and CI +jobs. See ``composer.json``, ``package.json``, ``tests/``, ``playwright/`` and +``.github/workflows``. - - Keep your local branches rebased and dependencies up to date to reduce noise in lints and type checks. - - Tests and CI are also part of the documentation, check ``.github/workflows`` and the ``tests`` folder for usage examples. +PHP unit tests +-------------- -Unit tests ----------- +Unit tests live under ``tests/php/Unit`` and should not depend on a bootstrapped +Nextcloud runtime. -Run a specific unit test (filter by class, method, or pattern): +Run the unit suite: .. code-block:: bash - composer test:unit -- --filter MyClassTest + composer test:unit -.. note:: - The double dash ``--`` is required to pass arguments to the script - ``test:unit`` (and not to Composer itself). +Run a specific class, method or filter: -Running the entire unit suite locally may take a while. Prefer filtering by the -tests you added or modified. +.. code-block:: bash + + composer test:unit -- --filter CrlServiceTest + composer test:unit -- --filter testMethodName -Integration tests (Behat) -------------------------- +The double dash ``--`` separates Composer arguments from PHPUnit arguments. -Integration tests live under ``tests/integration``. To run them: +PHP runtime integration tests +----------------------------- -1. Install dependencies inside that folder: +PHPUnit tests that require a bootstrapped Nextcloud runtime live under +``tests/php/Api`` and ``tests/php/Integration``. - .. code-block:: bash +Run them with: + +.. code-block:: bash - cd tests/integration - composer install + composer test:integration -2. Run a specific scenario (example): +A PHPUnit filter can be forwarded in the same way: - .. code-block:: bash +.. code-block:: bash - runuser -u www-data -- vendor/bin/behat --xdebug features/account/me.feature:5 + composer test:integration -- --filter ClassName - The example above executes the scenario that **starts at line 5** of - ``features/account/me.feature``. +Do not move tests with hidden runtime requirements such as database services, +``AppData`` or configured Nextcloud services into the unit suite merely to make +them faster. -Static analysis +Behat scenarios --------------- -**PHPCS** (coding style): +Behavior/integration scenarios live under ``tests/integration`` and have their +own Composer dependencies. + +Install them with: + +.. code-block:: bash + + composer --working-dir=tests/integration install + +Before creating a new step, inspect the existing vocabulary: + +.. code-block:: bash + + cd tests/integration + vendor/bin/behat -dl + +Run a feature or a scenario starting at a specific line: + +.. code-block:: bash + + vendor/bin/behat features/account/me.feature -v + vendor/bin/behat features/account/me.feature:5 -v + +Behat exercises a running Nextcloud environment and can modify application +state. Prefer the relevant feature or scenario while diagnosing a change. + +Frontend unit tests +------------------- + +Frontend unit tests live under ``src/tests`` and run with Vitest. + +Run the complete frontend unit suite: + +.. code-block:: bash + + npm test + +Run one test file: + +.. code-block:: bash + + npx vitest run src/tests/path/to/spec.ts + +Use ``npm run test:coverage`` when coverage output is needed. + +Browser/end-to-end tests +------------------------ + +Browser tests live under ``playwright`` and use Playwright. + +Run the configured E2E suite: + +.. code-block:: bash + + npm run test:e2e + +Run one Playwright test file: + +.. code-block:: bash + + npx playwright test playwright/e2e/path/to/spec.ts + +These tests require the runtime described by the repository's current +Playwright configuration and CI workflow. + +PHP linting, coding style and static analysis +--------------------------------------------- + +Check PHP syntax: + +.. code-block:: bash + + composer lint + +Check PHP coding style with PHP-CS-Fixer without changing files: + +.. code-block:: bash + + composer cs:check + +Apply PHP-CS-Fixer changes: .. code-block:: bash composer cs:fix -**Psalm** (type analysis): +Run Psalm: .. code-block:: bash composer psalm -Update Psalm baseline (only when appropriate): +Update the Psalm baseline only when the remaining findings are intentionally +accepted and reviewed: .. code-block:: bash composer psalm:update-baseline -JavaScript linters ------------------- +Frontend linting and type checking +---------------------------------- -**ESLint** and **Stylelint**: +Check ESLint, Stylelint and TypeScript: + +.. code-block:: bash + + npm run lint + npm run stylelint + npm run ts:check + +Apply the available frontend lint fixes: .. code-block:: bash npm run lint:fix npm run stylelint:fix + +Validation strategy +------------------- + +During implementation, run the smallest test that can fail for the behavior +being changed. Before declaring work complete, broaden validation according to +the changed surface and inspect the relevant GitHub Actions results. + +A green rerun alone is not evidence that an earlier failure was flaky. Diagnose +the first causal failure before adding retries, sleeps, suppressions or pins.