When a path is passed to |mach try fuzzy|, the requested tests were all run by
chunk 1 of each selected configuration: the taskgraph left the task's manifest
list set to the whole suite, so the chunk count and the manifests of each chunk
were meaningless, the target task selection kept only chunk 1, and the try
config env overwrote every chunk's MOZHARNESS_TEST_PATHS with the requested
paths. A path holding more tests than one job can run within its maximum
duration therefore lost that configuration entirely.
Restrict the task's manifests to those holding tests under the requested paths
instead, keeping a path that is narrower than a manifest (a single test file or
a subdirectory) as is, and scale the configured chunk counts down to that share
of the suite. Chunking then spreads the requested paths over as many chunks as
they need, each chunk running only its own share. The skipped manifests and the
directories of the other manifests are dropped along the way, as neither is
meaningful for a task that only runs part of the suite.
Only tasks whose manifests the taskgraph could restrict this way are chunked
and scheduled per chunk, which the new 'test-manifests-restricted' attribute
marks. Everything else keeps running the request in its first chunk as before:
suites whose tests the decision task doesn't resolve, such as jsreftest,
reftest-qr, crashtest-qr, marionette-unittest, jittest, cppunittest and gtest,
web-platform-tests, whose manifest names hold namespaces rather than source
paths, and pushes that also carry a test tag, where only part of each manifest
runs and the runtime data, which is per manifest, can't say how long that is.
Two parts of the runtime estimate are sized for a whole suite and give a single
over-long chunk once only part of one runs, so they are adjusted for a
restricted task:
- a manifest for which the runtime data holds no time is filled in with the
average of the manifests that do have data, rather than counted as instant.
The few manifests a restricted task runs can all be missing from the data,
which is what collapsed the slowest configurations to a single chunk; over a
whole suite the manifests the data does cover dominate.
- the number of chunks is rounded up rather than to the nearest, as rounding
to the nearest lets a chunk run up to half the target duration longer than
it. That is amortized over the many chunks a whole suite needs, but with a
handful of chunks it means one of them running half again as long.
The estimate stays a guess where the data is thin: a configuration for which
only a couple of the requested manifests have a runtime is chunked from the
average of those two, and one the data doesn't cover at all falls back to a
share of the suite's configured chunk count, which for a small request is one
chunk.
Because chunk counts now depend on the request, a test task label naming a
chunk that no longer exists, which `mach try coverage` and a `mach try again`
of an older push can both produce, is replaced by the chunks the task ended up
with rather than dropped with a warning.
Also fixes |mach try fuzzy <test file> --allow-testfile-path|, which scheduled
no test task at all, as no manifest path starts with a test file path.
Differential Revision: https://phabricator.services.mozilla.com/D317430
Florian Quèze
77a7a17ba1
