Skip to content

A queued intent is skipped indefinitely while its class serves fourteen intents admitted after it, including six admitted 66 minutes later #198

Description

@rldyourmnd

One job intent on nddev-linux-fast has been assigned for 101 minutes and never served, while the same class served at least fourteen intents assigned after it - six of them assigned 66 minutes later and served within five minutes. The intent is still alive and still reconciling on its cycle; it is not lost, not terminal, and not backed off. It is skipped.

This is not #197. That one was a stalled create path with a leaked retry record and it self-cleared; this is a single intent passed over while its class works.

No private repository, host, tenant or credential is required to describe it. Two accounts share the queue below and are called A and B.

The ordering

Assign time against start time, same class, taken from the manager journal:

intent account assigned started waited
1f0dbfbe B 11:02:23Z 11:04:54Z 2m31s
f7b22c5d A 11:12:09Z 11:16:22Z 4m13s
161c5248 B 11:12:24Z 12:14:27Z 62m03s
782eab20 A 11:17:10Z 11:19:46Z 2m36s
342b3571 B 11:20:58Z never 101m and counting
84159fa5 A 11:28:13Z 12:20:10Z 51m57s
81a3be77 A 11:28:14Z 12:21:02Z 52m48s
86718ff7 A 11:28:14Z 12:21:10Z 52m56s
65a3dea0 A 11:28:14Z 12:22:15Z 54m01s
ca50745a A 11:33:53Z 12:22:32Z 48m39s
82a9a46a A 11:33:54Z 12:22:48Z 48m54s
1ff96504 A 11:33:54Z 12:27:48Z 53m54s
e6ddbd23 A 12:25:11Z 12:30:05Z 4m54s
232151a4 A 12:27:24Z 12:31:47Z 4m23s
bb85563a A 12:27:24Z 12:31:47Z 4m23s
73471dad A 12:27:34Z 12:32:15Z 4m41s
54b17bf4 A 12:27:34Z 12:33:17Z 5m43s
7b2e2b9a A 12:27:34Z 12:35:39Z 8m05s
7cbd91f1 A 12:27:36Z 12:35:39Z 8m03s

The skipped intent is not the oldest and not the newest. Its own sibling from the same account, assigned 8 minutes earlier, was served at 12:14:27Z as the first job of that burst - so the account is not blocked either.

It is alive the whole time

It has been re-emitting reconciled authoritative queued job intent on its cycle throughout, most recently 12:55:58Z, 95 minutes after assignment. There is no retry record for the class at all right now, and the pool holds zero leases, so nothing is holding a slot on its behalf and nothing has classified it as failing. Every observable says it is a healthy queued intent that the dispatcher does not pick.

The class serves in bursts, which is what hides it

Creates and starts on this class per 10-minute bucket over the preceding six hours - every other bucket in that window is empty:

10:5x   9      12:1x   1
11:0x   2      12:2x  12
11:1x   3      12:3x  13

26 starts in six hours against 142 on another class in the same window, in two bursts of about 25 minutes each, with nothing since 12:37:08Z. Both matchers - the scale-set worker name and the class name on the line - agree exactly on those counts, so this is not a measurement artefact.

A skipped intent is invisible against that: each burst looks like a queue draining, and the one intent that never drains reads as "still waiting its turn" until you sort by assign time.

One correction to #197

The job named in that issue's parked retry record, 478cce5a, was assigned 11:12:09Z and started 11:16:43Z. It ran. The record outlived a job that had already succeeded, which settles the "leak rather than backoff" reading there: a terminal outcome did not clear its retry entry.

What would catch it

The queue-age predicate in #146 fires here, but only after the intent is old - and on a bursty class that is hard to tell from an ordinary busy period. The narrower one is an ordering invariant, checkable at every dispatch:

no intent may be served while an intent for the same class, admitted earlier and still queued, is skipped more than N times

Here that is violated fourteen times and would have fired at 12:20:10Z, the first out-of-order start, forty minutes before this report.

Not requested

Nothing was changed, no unit restarted, no recovery command run. The intent is still queued as of 13:01:59Z, so the state is inspectable right now if that is useful.

What it costs us: the skipped intent is a required merge context. Two pull requests are held behind it with auto-merge armed, and we are not lifting branch protection.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions