Skip to content

Run Emscripten RestClient fetches on a worker thread - #409

Merged
RalphSteinhagen merged 1 commit into
mainfrom
fetch_work_thread
Aug 24, 2026
Merged

Run Emscripten RestClient fetches on a worker thread#409
RalphSteinhagen merged 1 commit into
mainfrom
fetch_work_thread

Conversation

@drslebedev

@drslebedev drslebedev commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Emscripten delivers fetch callbacks on the thread that issued the request,
after that thread returns to the JS event loop. ClientContext worked around
this by starting a std::thread for every command and immediately joining it,
blocking its poller. Direct RestClient users such as OpenDigitizer had no
workaround, so fetch callbacks could run on the browser main thread.

Give each Emscripten RestClient a persistent worker and route requests, long
polling, unsubscribe and cleanup through it. Resume a 504 long-poll timeout
at the same index instead of ending the subscription.

Fetch completion and subscription callbacks now run on the REST worker, so
consumers must synchronise shared state. Each active client uses one worker,
which is reclaimed asynchronously during shutdown.

Remove the per-request thread workaround from ClientContext. The native
RestClient implementation is unchanged.

Add an Emscripten integration test covering subscriptions, unsubscribe,
shutdown, callback threading and worker reclamation. The Node test requires
the xhr2 npm package. Browser tests are registered when emrun and a matching
browser are available; OPENCMW_REQUIRE_BROWSER_TESTS requires at least one
browser test.

@drslebedev
drslebedev marked this pull request as draft August 21, 2026 10:24
@drslebedev
drslebedev force-pushed the fetch_work_thread branch 2 times, most recently from 15c7ed1 to b49f7bf Compare August 21, 2026 10:42
@drslebedev
drslebedev temporarily deployed to configure coverage August 21, 2026 13:29 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 21, 2026 13:29 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 21, 2026 13:29 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 21, 2026 13:29 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 21, 2026 13:29 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 21, 2026 13:29 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 24, 2026 09:23 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 24, 2026 09:23 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 24, 2026 09:23 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 24, 2026 09:23 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 24, 2026 09:23 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 24, 2026 09:23 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 24, 2026 09:31 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 24, 2026 09:31 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 24, 2026 09:31 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 24, 2026 09:31 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 24, 2026 09:31 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 24, 2026 09:31 — with GitHub Actions Inactive
  Emscripten delivers fetch callbacks on the thread that issued the request,
  after that thread returns to the JS event loop. ClientContext worked around
  this by starting a std::thread for every command and immediately joining it,
  blocking its poller. Direct RestClient users such as OpenDigitizer had no
  workaround, so fetch callbacks could run on the browser main thread.

  Give each Emscripten RestClient a persistent worker and route requests, long
  polling, unsubscribe and cleanup through it. Resume a 504 long-poll timeout
  at the same index instead of ending the subscription.

  Fetch completion and subscription callbacks now run on the REST worker, so
  consumers must synchronise shared state. Each active client uses one worker,
  which is reclaimed asynchronously during shutdown.

  Remove the per-request thread workaround from ClientContext. The native
  RestClient implementation is unchanged.

  Add an Emscripten integration test covering subscriptions, unsubscribe,
  shutdown, callback threading and worker reclamation. The Node test requires
  the xhr2 npm package. Browser tests are registered when emrun and a matching
  browser are available; OPENCMW_REQUIRE_BROWSER_TESTS requires at least one
  browser test.

Signed-off-by: drslebedev <dr.s.lebedev@gmail.com>
@drslebedev
drslebedev temporarily deployed to configure coverage August 24, 2026 10:38 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 24, 2026 10:39 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 24, 2026 10:39 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 24, 2026 10:39 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 24, 2026 10:39 — with GitHub Actions Inactive
@drslebedev
drslebedev temporarily deployed to configure coverage August 24, 2026 10:39 — with GitHub Actions Inactive
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

@drslebedev drslebedev changed the title WIP: Run Emscripten RestClient fetches on a worker thread Run Emscripten RestClient fetches on a worker thread Aug 24, 2026
@drslebedev
drslebedev marked this pull request as ready for review August 24, 2026 12:53

@RalphSteinhagen RalphSteinhagen left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great addition to the unit-test suites and thanks a lot for the WASM-worker changes of the REST Clients. That should resolve a lot of the CPU and network contention, eliminate dropping of data in high network load conditions, and alleviate the CPU performance issues.

The data point reduction algorithm you indicated would be a further improvement but I'd rather de-prioritize this in favour of tackling the issue at the source (i.e. ImPlot/ImGUI itself).

@RalphSteinhagen
RalphSteinhagen merged commit 3ff532d into main Aug 24, 2026
9 of 11 checks passed
@RalphSteinhagen
RalphSteinhagen deleted the fetch_work_thread branch August 24, 2026 13:02
@github-project-automation github-project-automation Bot moved this to 🆕 Ideas (∞) in Digitizer Reimplementation Aug 25, 2026
@github-project-automation github-project-automation Bot moved this from 🆕 Ideas (∞) to ✅ QA-Accepted/Merged (∞) in Digitizer Reimplementation Aug 25, 2026
RalphSteinhagen pushed a commit to fair-acc/opendigitizer that referenced this pull request Aug 26, 2026
* Bump OpenCMW for fair-acc/opencmw-cpp#409
* Bump gr-digitizers for fair-acc/gr-digitizers#207

Update OpenDigitizer to:
  - use canonical gr:: tag keys throughout the service and UI
  - propagate canonical trigger metadata from remote sources
  - interpret trigger offsets in seconds
  - handle canonical dropped-sample tags

Signed-off-by: drslebedev <dr.s.lebedev@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅ QA-Accepted/Merged (∞)

Development

Successfully merging this pull request may close these issues.

[5SP,5SP] REST long polling: report backlog so clients can catch up

2 participants