Allow configuring human-readable names for workers and dispatchers#754
Allow configuring human-readable names for workers and dispatchers#754cgmoore120 wants to merge 1 commit into
Conversation
Processes are currently named with their kind plus a random hex string,
e.g. `worker-5a3b1c0d9e8f7a6b5c4d`. That name is what shows up in the
processes table and in the process title, which makes it hard to tell
which process is which when you run several workers with different
configurations, either in `ps` output or in the dashboard.
This adds an optional `name` setting for workers and dispatchers:
production:
dispatchers:
- polling_interval: 1
name: main-dispatcher
workers:
- queues: background
threads: 3
processes: 3
name: pipeline
- queues: default
threads: 1
name: housekeeping
When a worker sets `processes` to more than one, each forked process
gets a numbered suffix (`pipeline-1`, `pipeline-2`, `pipeline-3`) so the
names stay unique. With a single process, the name is used as-is.
The name is also reflected in the process title, for example:
solid-queue-worker[pipeline-1](1.3.1): waiting for jobs in background
Processes without a `name` keep generating a random one, so this is
backwards compatible.
41482ab to
c6f2dd0
Compare
|
The two red jobs here — One failure out of 236 runs in each job, while the other 62 jobs in the matrix are green — including the same Ruby and database on other Rails versions. This PR only touches process naming, so it shouldn't be able to affect concurrency controls. It looks like the known flakiness in Happy to rebase or push again if you'd like a fresh run — I can't re-trigger CI on the PR myself. |
The problem
Processes are named with their kind plus a random hex string, e.g.
worker-5a3b1c0d9e8f7a6b5c4d. That name is what's recorded insolid_queue_processesand rendered in the process title, so when you run several workers with different configurations, there's no way to tell which is which — not inpsoutput, and not in the processes table of the dashboard. You end up cross-referencing PIDs against metadata to answer "which of these five workers is the one drainingbackground?".The change
An optional
namefor workers and dispatchers, set inconfig/queue.yml:When a worker sets
processesto more than one, each forked process gets a numbered suffix —pipeline-1,pipeline-2,pipeline-3— so names stay unique. With a single process, the name is used as-is.The name is also reflected in the process title:
Processes without a
namekeep generating a random one, so this is fully backwards compatible.Processes::Base#custom_name?distinguishes the two cases, and only named processes get the[...]segment in the procline.Notes
Supervisor#initializetakes its configuration positionally and called a baresuper, which forwarded that argument up. SinceProcesses::Base#initializenow takes keywords, that had to become an explicitsuper().processes.