Repository navigation
Select configs among the configurations of the current tasks, and give the taskgraph their labels - #7598
Draft
marco-c wants to merge 9 commits into
Draft
Select configs among the configurations of the current tasks, and give the taskgraph their labels#7598marco-c wants to merge 9 commits into
marco-c wants to merge 9 commits into
Conversation
marco-c
force-pushed
the
config-selection-task-configs
branch
2 times, most recently
from
November 19, 2026 23:03
7643f34 to
cffcefd
Compare
… as the models name them The config selection models are trained with the configurations of the push data, as mozci names them from the task definitions (renamed, e.g. shippable configurations are the ones of their non-shippable equivalents). The candidates were instead obtained by matching the labels of the current tasks with the configs seen in training, which isn't the same: - configs of tasks whose label doesn't have their Treeherder platform (e.g. the Windows ASan ones) and shippable tasks were unknown to the models; - suites and variants named alike were confused (e.g. mochitest-webgpu and the webgpu variant); - tasks which test selection doesn't optimize (e.g. the standalone ones, which only run on backstop pushes) were candidates. The HTTP service now retrieves the test tasks that test selection optimizes from the task graph of the latest backstop push (which runs all of them, with all their manifests), with their configs computed with mozci's get_configuration, like the ones of the push data. Configs are selected among them. Shippable tasks are left out, as they are too costly: the models are trained with the configs of their non-shippable equivalents, which are selected instead.
The taskgraph matched the selected configs with the labels of its tasks, so it couldn't run the configs of tasks whose label doesn't have their Treeherder platform (e.g. the Windows ASan ones), and it also ran the variants of a selected config (e.g. test-linux2404-64/opt-* matching the swr tasks). The replies now give it the labels (without chunk number) of the tasks of the selected configs, and the selected configs themselves in "config_group_configs".
…ection models They were only used to name the tasks of the current task graph, which are now named after their configurations.
… parsing all of it A decision task's full-task-graph.json is hundreds of MB (and much more once parsed), so only the definitions of the given tasks are parsed, where they are in the text. If one isn't found this way (e.g. if the format of the file changed), the whole graph is parsed.
… their configs Tasks can be renamed after the task graph the config selection tasks come from. The tasks whose manifests aren't known, with their configs, are given for all the groups selected on their configs, as the taskgraph only runs a task for a group if the task runs it.
As the taskgraph matches the labels, they must be the ones of the push's tasks: the test tasks of the pushes after the backstop push which weren't in its task graph (e.g. renamed since then) are stored too. Those are found among the target tasks of the pushes (before optimization, which could remove them as they match none of the labels in the replies), with their definitions from the full task graph, which is only retrieved when there are unknown test tasks: the ones which aren't eligible are remembered.
The tasks of the pushes after the backstop push (e.g. renamed since then) are only useful if they are found soon after they appear, so the test tasks among which configs are selected are updated every 10 minutes in the refresh process, rather than once per day: from the latest backstop push as soon as there is a new one, and with the pushes after it which weren't processed yet. The pushes known not to be backstop pushes aren't checked again, and the file is only written when it changes.
…t variants of their push mozci memoizes the test variants of mozilla-central for the lifetime of the process (the refresh runs for weeks), and those don't have the variants added on autoland until they are merged, so the configurations of their tasks couldn't be computed. The HTTP service gives mozci the test variants of each push it reads tasks from (from the push's taskcluster/test_configs/variants.yml), or makes it retrieve the current ones of mozilla-central again if they can't be retrieved.
… which failed Tasks whose configuration can't be computed (e.g. if retrieving the push's test variants failed) were ignored, or skipped and left behind once their push was processed. They are now kept pending, with the parts of their definitions needed to try again, and resolved by a later update: the ones of the backstop push become candidates (with their manifests), the ones of the later pushes new tasks. Other pushes targeting them don't need their full task graph meanwhile.
marco-c
force-pushed
the
config-selection-task-configs
branch
from
November 19, 2026 23:16
cffcefd to
3d6a2d2
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The config selection models are trained with the configurations of the push data, as mozci names them from the task definitions (renamed by
rename_task, e.g. shippable configurations are the ones of their non-shippable equivalents). But the candidates were obtained by matching the labels of the tasks in a decision task'smanifests-by-task.jsonwith the configs seen in training, and the taskgraph matches the selected configs with the labels of its tasks. Those aren't the same:test-windows11-64-25h2-asan/opt-…vstest-windows11-64-25h2/asan-*) and of shippable tasks were unknown to the models, and the taskgraph couldn't run the Windows ASan ones when selected;mochitest-webgpuand thewebgpuvariant), and the taskgraph also ran all the variants of a selected config (test-linux2404-64/opt-*matches the-swr,-standalone… tasks);standaloneandrepeat, which only run on backstop pushes) were candidates.On a backstop push, about 8–12% of the target test tasks got the wrong config this way, and 36–64 candidate configs were unknown to the model. With this change, the configs of all the current tasks are known to the model, except those of brand-new variants.
What changes
get_configuration, like the push data. Configs are selected among them.config_groupsin the replies now contains the exact labels (without chunk number) of the tasks of the selected configs, andconfig_group_configscontains the selected configs. The taskgraph matches labels withfnmatch, so exact labels work without any Gecko change. Selectingtest-linux2404-64/opt-*no longer runs its variants, so the config selection strategies schedule fewer tasks.variants.yml, since mozci memoizes mozilla-central's, which lacks variants added on autoland until they're merged. Tasks whose configuration can't be computed are retried later rather than ignored.get_configurationreads the test variants fromtest-setting. Addspyyamlto the HTTP service's dependencies.Checks
Known limitation
A push that renames its own tasks (and pushes racing with it) can still lose the groups that only ran on the old labels, until its decision task finishes and the next update picks up the new labels. Fixing that needs the taskgraph to leave a group unrestricted when none of its labels match, which needs the global view discussed in bug 1927227.
The commits are meant to be reviewed one by one.