feat(bson-parsing)!: move parsing into web-worker COMPASS-11127 by mabaasit · Pull Request #891 · mongodb-js/devtools-shared

mabaasit · 2026-09-21T08:50:57Z

~~In draft because~~

~~I need to spend a bit more on (de)serialization (Neal is working on something very similar) and bundling.~~
I need to figure out the sync -> async api change for mongodb-query-parser package. This changes happened in this PR and for monorepo, it resolves to the local export and not the one pinned in package.json file.

Description

COMPASS-11127

Open Questions

Checklist

I have signed the Contributor License Agreement (https://www.mongodb.com/legal/contributor-agreement)
New tests and/or benchmarks are included
Documentation is changed or added

_{Stack created with GitHub Stacks CLI • Give Feedback 💬}

Copilot

Copilot review overview

🟡 Changes recommended

Critical API, compatibility, browser-runtime, and BSON-cloning issues remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 4 · 1 · 1

Open (6)

Removing CommonJS entry breaks CommonJS consumers · New
Synchronous query-parser consumer breaks with async default API · New
Duck-typed BSON detection corrupts valid documents · New
Unguarded process access crashes browser parsing · New
Concurrent worker initialization leaks duplicate workers · New
README documents outdated synchronous parse API · New

What changed in this PR

Moves BSON parsing into a web worker while adding structured-clone support, an explicit synchronous API, and ESM/Webpack tooling.

Changes:

Adds worker communication, lifecycle management, and BSON serialization.
Makes default parsing asynchronous while retaining synchronous parsing.
Updates tests, packaging, build configuration, and the MQL docs crawler.

File	Review summary
`packages/shell-bson-parser/webpack.worker.config.cjs`	Bundles the worker target.
`packages/shell-bson-parser/tsconfig.json`	Enables NodeNext resolution.
`packages/shell-bson-parser/test/parse-test-cases.ts`	Updates ESM imports and test typing.
`packages/shell-bson-parser/src/worker.ts`	Handles worker parse requests.
`packages/shell-bson-parser/src/worker-types.ts`	Defines worker messages.
`packages/shell-bson-parser/src/worker-client.ts`	Critical: browser `process` access can throw. Moderate: concurrent initialization can leak workers, and termination can leave requests unresolved.
`packages/shell-bson-parser/src/structured-clone-bson.ts`	Critical: `_bsontype` duck typing can corrupt valid plain documents after cloning.
`packages/shell-bson-parser/src/structured-clone-bson.spec.ts`	Tests BSON round trips.
`packages/shell-bson-parser/src/stringify.spec.ts`	Updates ESM imports.
`packages/shell-bson-parser/src/parse.ts`	Updates internal ESM imports.
`packages/shell-bson-parser/src/parse.spec.ts`	Updates imports and Sinon usage.
`packages/shell-bson-parser/src/index.ts`	Critical: the async default API breaks synchronous `query-parser` usage. Nit: README examples remain synchronous.
`packages/shell-bson-parser/src/index.spec.ts`	Moderate: tests bypass the bundled worker path and do not cover the real worker artifact.
`packages/shell-bson-parser/src/eval.ts`	Updates an internal import.
`packages/shell-bson-parser/src/check.ts`	Updates internal imports.
`packages/shell-bson-parser/package.json`	Critical: removing the CommonJS condition breaks CommonJS consumers. Moderate: `type: module` breaks the existing CommonJS benchmark.
`packages/shell-bson-parser/.mocharc.cjs`	Adds shared Mocha configuration.
`packages/shell-bson-parser/.eslintrc.cjs`	Ignores the Webpack configuration.
`packages/mql-typescript/src/driverSchema/docs-crawler.ts`	Uses the synchronous parser API.
`package-lock.json`	Locks updated Webpack dependencies.
`configs/mocha-config-devtools/register/tsnode-register.js`	Adds NodeNext test-loader support.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

mabaasit · 2026-09-28T12:22:04Z

+  "type": "module",
  "main": "dist/index.js",
  "types": "./dist/index.d.ts",
  "exports": {
-    "require": {
-      "types": "./dist/index.d.ts",
-      "default": "./dist/index.js"
-    },
-    "import": {
-      "types": "./dist/index.d.ts",
-      "default": "./dist/.esm-wrapper.mjs"
-    }
+    "types": "./dist/index.d.ts",


Yes, this change in intentional. We deprecated mongodb-query-parser package and marked it as private in this PR (and also excluded it from tests/checks). As for mql-typescript, changed to dynamic imports

mabaasit · 2026-09-28T12:22:30Z

+
+export const parse = (
+  ...args: Parameters<typeof parseSync>
+): Promise<ReturnType<typeof parseSync>> => callWorker(args);


Yes this is a breaking change and mongodb-query-parser is deprecated.

mabaasit · 2026-09-28T12:22:50Z

+function isBsonValue(value: object): value is { _bsontype: string } {
+  return typeof (value as { _bsontype?: unknown })._bsontype === 'string';


Addressed in 334507d

+}
+
+async function getWorkerBlobUrl(): Promise<string> {
+  if (process.env.TEST_SKIP_WORKER_SCRIPT_FETCH) {


+  if (worker) {
+    return worker;
+  }
+
+  blobUrl = await getWorkerBlobUrl();


+export const parse = (
+  ...args: Parameters<typeof parseSync>
+): Promise<ReturnType<typeof parseSync>> => callWorker(args);


mabaasit · 2026-09-28T12:23:19Z

  "name": "mongodb-query-parser",
  "description": "Parse MongoDB queries",
  "deprecated": "This package is deprecated. Use @mongodb-js/shell-bson-parser instead.",
+  "private": true,


Ignores from being published.

esvm

just a small question but LGTM

mabaasit · 2026-10-05T13:06:46Z

3cad193 (compass-components bump) brought many changes to lock file.

Anemy

Changes are looking good, I left a few questions, not blockers.

nbbeeken · 2026-10-09T14:20:57Z

      const tag =
-        item._bsontype === 'Binary' &&
-        (item as { sub_type?: unknown }).sub_type === 4
+        Object.getPrototypeOf(item) === BSON.UUID.prototype


is this not just the same as doing instanceof?

nbbeeken · 2026-10-09T14:24:21Z

  }
-  const proto = Reflect.getPrototypeOf(value);
-  return proto !== Object.prototype && proto !== null;
+  return KNOWN_BSON_PROTOTYPES.has(Object.getPrototypeOf(value));


Couldn't we just rely on the value of the string here? and could we use the [bsonType] symbol that is exported from the BSON library? then its indisputable. The ambiguous case we have to watch out for is plain object with a _bsontype property, but that goes away with the symbol.

Due to wide range of bson version we support, i'll keep using _bsonType as symbol is available from bson 7.0

mabaasit added this pull request to stack #894 September 21, 2026 08:51

mabaasit changed the title ~~feat(bson-parsin)!: move parsing into web-worker~~ feat(bson-parsing)!: move parsing into web-worker Sep 21, 2026

mabaasit changed the title ~~feat(bson-parsing)!: move parsing into web-worker~~ feat(bson-parsing)!: move parsing into web-worker COMPASS-11127 Sep 22, 2026

Copilot AI lite review requested due to automatic review settings September 23, 2026 11:19

mabaasit force-pushed the web-worker-parsing branch from e5d8ab7 to a596dfa Compare September 23, 2026 11:19

Copilot started reviewing on behalf of mabaasit September 23, 2026 11:20 View session

Copilot AI reviewed Sep 23, 2026

View reviewed changes

mabaasit force-pushed the web-worker-parsing branch from d0e8103 to 15408ac Compare September 28, 2026 12:20

mabaasit commented Sep 28, 2026

View reviewed changes

mabaasit marked this pull request as ready for review September 28, 2026 12:24

mabaasit requested review from a team as code owners September 28, 2026 12:24

mabaasit requested a review from esvm September 28, 2026 12:24

Base automatically changed from consolidate-query-parser-in-bson-parser to main September 28, 2026 15:28

mabaasit force-pushed the web-worker-parsing branch 2 times, most recently from 89987eb to b87d456 Compare September 29, 2026 08:01

mabaasit mentioned this pull request Oct 2, 2026

chore(compass-components): drop query-parser dependency from compass-components COMPASS-11202 mongodb-js/compass#8520

Merged

13 tasks

esvm approved these changes Oct 2, 2026

View reviewed changes

Comment thread packages/shell-bson-parser/src/worker-client.ts

Comment thread packages/mql-typescript/src/driverSchema/docs-crawler.ts Outdated

mabaasit force-pushed the web-worker-parsing branch from fd1ff4b to 2d47a92 Compare October 5, 2026 13:00

Anemy reviewed Oct 6, 2026

View reviewed changes

Comment thread packages/oidc-http-server-pages/.depcheckrc

Comment thread packages/shell-bson-parser/src/check.ts

Comment thread packages/shell-bson-parser/src/worker-client.ts Outdated

nbbeeken requested changes Oct 7, 2026

View reviewed changes

nbbeeken mentioned this pull request Oct 8, 2026

chore(crud): move shell bson parser into a webworker mongodb-js/compass#8260

Closed

mabaasit force-pushed the web-worker-parsing branch from 0e2a4b5 to 1a74167 Compare October 8, 2026 21:42

nbbeeken requested changes Oct 9, 2026

View reviewed changes

mabaasit force-pushed the web-worker-parsing branch from 1a74167 to ebf38c5 Compare October 9, 2026 23:03

mabaasit added 2 commits October 12, 2026 13:31

web worker

4397117

clean up stringify

d43bb87

mabaasit added 25 commits October 12, 2026 13:31

expose unsafe and update in mql-typescript

75e097a

esbuild to webpack

89ae4cc

align with Neal's implementation

b18879f

rename

af49e86

check

d30e817

support for node

1990136

fix

50d17af

drop idle timeout

c77d84e

handle bson type check

54963d9

handle mql-ts

a546306

ignore query-parser package

f7b54cd

ignore in bootstrap as well

8b925b1

address CI

065a275

bring query-parser back

7733857

delete

ed5f057

drop dependency

4371366

bump compass-components

35e98bd

lint

7dd14ef

add cjs support

bb2a1fb

fix windows write

d45686a

add dependency

3b8b7b9

type: module

cf824b2

address PR comments

9662146

no data loss

3a5327c

better checks

cf32098

nbbeeken approved these changes Oct 12, 2026

View reviewed changes

Anemy approved these changes Oct 12, 2026

View reviewed changes

mabaasit force-pushed the web-worker-parsing branch from ebf38c5 to cf32098 Compare October 12, 2026 21:18

		function isBsonValue(value: object): value is { _bsontype: string } {
		return typeof (value as { _bsontype?: unknown })._bsontype === 'string';

Conversation

mabaasit commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Open Questions

Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

mabaasit Sep 28, 2026

Choose a reason for hiding this comment

Uh oh!

mabaasit Sep 28, 2026

Choose a reason for hiding this comment

Uh oh!

mabaasit Sep 28, 2026

Choose a reason for hiding this comment

Uh oh!

mabaasit Sep 28, 2026

Choose a reason for hiding this comment

Uh oh!

esvm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mabaasit commented Oct 5, 2026

Uh oh!

Anemy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nbbeeken Oct 9, 2026

Choose a reason for hiding this comment

Uh oh!

nbbeeken Oct 9, 2026

Choose a reason for hiding this comment

Uh oh!

mabaasit Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

mabaasit commented Sep 21, 2026 •

edited

Loading