diff --git a/docs/articles/nunit/running-tests/NUnitLite-Options.md b/docs/articles/nunit/running-tests/NUnitLite-Options.md index 5cf2017fa..bc521e364 100644 --- a/docs/articles/nunit/running-tests/NUnitLite-Options.md +++ b/docs/articles/nunit/running-tests/NUnitLite-Options.md @@ -7,7 +7,7 @@ The NUnitLite runner is invoked by executing the test program. If desired, any o | `--test=NAMES` | Comma-separated list of NAMES of tests to run or explore. This option may be repeated. | | `--testlist=PATH` | File PATH containing a list of tests to run, one per line. This option may be repeated. | | `--prefilter=NAMES` | Comma-separated list of NAMES of test classes or namespaces to be loaded. This option may be repeated. | -| `--where=EXPRESSION` | Test selection EXPRESSION indicating what tests will be run. | +| `--where=EXPRESSION` | Test selection EXPRESSION indicating what tests will be run. See [Test Selection Language](Test-Selection-Language.md) for details. | | `--params, -p=VALUE` | Define a test parameter. | | `--timeout=MILLISECONDS` | Set timeout for each test case in MILLISECONDS. | | `--seed=SEED` | Set the random SEED used to generate test cases. | diff --git a/docs/articles/nunit/running-tests/Partitioning-Tests.md b/docs/articles/nunit/running-tests/Partitioning-Tests.md new file mode 100644 index 000000000..61bcb1ab0 --- /dev/null +++ b/docs/articles/nunit/running-tests/Partitioning-Tests.md @@ -0,0 +1,53 @@ +--- +uid: partitioningtests +--- + +# Partitioning Tests + +When a test suite is too large to run as one job, split it into numbered partitions and run each partition separately. +NUnit assigns each test to a partition, so the jobs together cover the suite without selecting tests by hand. + +## Selecting a partition + +Use the `partition` property in a Test Selection Language expression. The value has the form `PARTITION/TOTAL`: + +```shell +# Run partition 2 of 5 +nunit3-console MyTests.dll --where "partition == 2/5" + +# Or, with dotnet test and an NUnit adapter that supports partition filters +dotnet test -- NUnit.Where="partition == 2/5" +``` + +Partition numbers start at **1**, not 0. To run the whole suite, start one job for each partition from `1` to the same +total. For five jobs, use `1/5`, `2/5`, `3/5`, `4/5` and `5/5`. Keep the total the same for every job. + +NUnit uses a hash of each test's full name to assign it to a partition. This makes assignment repeatable for a given +suite, but does not balance partitions by test duration. Renaming tests can also change their assignments. + +## Keeping fixtures together + +By default, NUnit partitions individual test cases. Tests from one fixture can therefore run in different jobs, with +fixture setup and teardown occurring in more than one job. + +To keep each fixture together, add `:fixture` to the partition value: + +```shell +nunit3-console MyTests.dll --where "partition == 2/5:fixture" +dotnet test -- NUnit.Where="partition == 2/5:fixture" +``` + +Fixture partitioning assigns a fixture as a unit. It may result in less even runtimes when fixtures differ in size. +Explicitly specifying `:test` selects the default test-case partitioning mode. + +## Things to keep in mind + +- Partition filters require NUnit 3.14 or later; fixture partitioning requires NUnit 5.1 or later. The Console Runner + supports partitions from 3.17, and NUnit3TestAdapter supports them from 4.6. +- Run every partition with the same test assembly and partition count. Changing the total can assign tests to different + partitions. +- Use a runner that supports partition filters. For `dotnet test`, this requires a compatible NUnit adapter. +- Test dependencies may cause a dependent test or fixture to be included in more than one partition. See + [Tests That Depend on Other Tests](xref:dependenttests) for details. + +See the [Test Selection Language](xref:testselectionlanguage) for more about filter expressions. diff --git a/docs/articles/nunit/running-tests/Test-Selection-Language.md b/docs/articles/nunit/running-tests/Test-Selection-Language.md index f50dffdf1..500a18fe4 100644 --- a/docs/articles/nunit/running-tests/Test-Selection-Language.md +++ b/docs/articles/nunit/running-tests/Test-Selection-Language.md @@ -44,6 +44,7 @@ The following key words are recognized on the left-hand side of the comparison: * `namespace` - The fully qualified name of the namespace containing the test(s), e.g. My.Name.Space * `method` - The name of the method, e.g. TestMethod * `cat` - A category assigned to the test, e.g. SmokeTests +* `partition` - A numbered partition of the test suite; see [Partitioning Tests](xref:partitioningtests) If the left-hand side of the comparison does not consist of a key word, it is treated as the name of a property on the test whose value is to be checked. See below for restrictions on use of properties. @@ -117,6 +118,15 @@ cached the ids, not for general use by users. The reason for this restriction is the id that will be assigned to a test. The id is not persistent across test runs and its format can differ between different framework drivers. +## Partitioning Tests + +Use the `partition` keyword with a value of `PARTITION/TOTAL` to select one numbered partition. Partition numbers start +at 1. For example, `partition == 2/5` selects partition 2 of 5. Add `:fixture` to the value to keep each fixture's tests +together, as in `partition == 2/5:fixture`. Without a suffix, the filter partitions individual test cases. + +Partition filters were added in NUnit 3.14. Fixture partitioning was added in NUnit 5.1. Support for partition filters +was added to the NUnit Console Runner in 3.17 and to NUnit3TestAdapter (including `dotnet test`) in 4.6. + ## Compound Expressions Simple expressions may be combined using logical and, logical or, parentheses or negation operators. diff --git a/docs/articles/nunit/running-tests/toc.yml b/docs/articles/nunit/running-tests/toc.yml index c72f910d4..6de59452f 100644 --- a/docs/articles/nunit/running-tests/toc.yml +++ b/docs/articles/nunit/running-tests/toc.yml @@ -11,4 +11,6 @@ - name: Template Based Test Naming href: Template-Based-Test-Naming.md - name: Test Selection Language - href: Test-Selection-Language.md \ No newline at end of file + href: Test-Selection-Language.md +- name: Partitioning Tests + href: Partitioning-Tests.md \ No newline at end of file