Skip to content

Allow configuring human-readable names for workers and dispatchers#754

Open
cgmoore120 wants to merge 1 commit into
rails:mainfrom
cgmoore120:configurable-process-names
Open

Allow configuring human-readable names for workers and dispatchers#754
cgmoore120 wants to merge 1 commit into
rails:mainfrom
cgmoore120:configurable-process-names

Conversation

@cgmoore120

@cgmoore120 cgmoore120 commented Jul 14, 2026

Copy link
Copy Markdown

The problem

Processes are named with their kind plus a random hex string, e.g. worker-5a3b1c0d9e8f7a6b5c4d. That name is what's recorded in solid_queue_processes and rendered in the process title, so when you run several workers with different configurations, there's no way to tell which is which — not in ps output, and not in the processes table of the dashboard. You end up cross-referencing PIDs against metadata to answer "which of these five workers is the one draining background?".

The change

An optional name for workers and dispatchers, set in config/queue.yml:

production:
  dispatchers:
    - polling_interval: 1
      batch_size: 500
      name: main-dispatcher
  workers:
    - queues: background
      threads: 3
      processes: 3
      name: pipeline
    - queues: default
      threads: 1
      name: housekeeping

When a worker sets processes to more than one, each forked process gets a numbered suffix — pipeline-1, pipeline-2, pipeline-3 — so names stay unique. With a single process, the name is used as-is.

The name is also reflected in the process title:

solid-queue-worker[pipeline-1](1.3.1): waiting for jobs in background

Processes without a name keep generating a random one, so this is fully backwards compatible. Processes::Base#custom_name? distinguishes the two cases, and only named processes get the [...] segment in the procline.

Notes

  • Supervisor#initialize takes its configuration positionally and called a bare super, which forwarded that argument up. Since Processes::Base#initialize now takes keywords, that had to become an explicit super().
  • Added unit tests covering name expansion across multiple processes, the single-process case, dispatcher names, loading names from a config file, and that unnamed processes still get a random name.
  • README documents the new option alongside processes.

Processes are currently named with their kind plus a random hex string,
e.g. `worker-5a3b1c0d9e8f7a6b5c4d`. That name is what shows up in the
processes table and in the process title, which makes it hard to tell
which process is which when you run several workers with different
configurations, either in `ps` output or in the dashboard.

This adds an optional `name` setting for workers and dispatchers:

  production:
    dispatchers:
      - polling_interval: 1
        name: main-dispatcher
    workers:
      - queues: background
        threads: 3
        processes: 3
        name: pipeline
      - queues: default
        threads: 1
        name: housekeeping

When a worker sets `processes` to more than one, each forked process
gets a numbered suffix (`pipeline-1`, `pipeline-2`, `pipeline-3`) so the
names stay unique. With a single process, the name is used as-is.

The name is also reflected in the process title, for example:

  solid-queue-worker[pipeline-1](1.3.1): waiting for jobs in background

Processes without a `name` keep generating a random one, so this is
backwards compatible.
@cgmoore120 cgmoore120 force-pushed the configurable-process-names branch from 41482ab to c6f2dd0 Compare July 14, 2026 01:18
@cgmoore120

Copy link
Copy Markdown
Author

The two red jobs here — Tests (3.2, mysql, rails_7_1) and Tests (4, postgres, rails_7_2) — both fail on the same test, and I don't think it's related to this PR:

ConcurrencyControlsTest#test_run_several_jobs_over_the_same_record_limiting_concurrency
[test/integration/concurrency_controls_test.rb:84]
Expected: "sCcC"

One failure out of 236 runs in each job, while the other 62 jobs in the matrix are green — including the same Ruby and database on other Rails versions. This PR only touches process naming, so it shouldn't be able to affect concurrency controls.

It looks like the known flakiness in ConcurrencyControlsTest that #734 is addressing. For what it's worth, the same test class fails on unmodified main at b11ce85, which is the commit this branch is based on (run, Tests (4, sqlite, rails_main), failing on test_run_several_jobs_over_the_same_record_sequentially,_with_some_of_them_failing).

Happy to rebase or push again if you'd like a fresh run — I can't re-trigger CI on the PR myself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant