Phase 6b: Server-Sent Events (#16) - #47
Merged
Merged
Conversation
Implement the full Server-Sent Events subsystem in `@dexpace/core` per `docs/product-spec/13-server-sent-events-and-streaming.md` (SSE-1 through SSE-41). The implementation is strictly pull-based, single-pass, and zero-dependency: events are parsed 1:1 on demand as the consumer polls the stream, with no unbounded buffering and no auto-reconnection logic. @dexpace/core additions: - `SseEvent`, `makeSseEvent`: Immutable event representation with defensively copied and frozen `data` lines (`SSE-20`). Structural equality (`sseEventsEqual`), string representation (`sseEventToString`), and content predicate (`isSseEventEmpty`, where comments count as content per `SSE-22`). - `SseLineReader` (@internal): Hand-rolled line framing supporting `\n`, `\r`, and `\r\n` line terminators across chunk boundaries (`SSE-2`), start-only UTF-8 BOM stripping via non-consuming `peek()` lookahead (`SSE-12`), and an optional configurable line length cap (`maxLineBytes` / `SseLineTooLongError`, `SSE-19`). - `SseParser` (@internal): Single-pass state machine implementing WHATWG SSE grammar with the three reference spec deviations: comments captured and dispatched (`SSE-6`), permissive dispatch when any of the 5 fields are set (`SSE-13`), and EOF dispatch of pending fields (`SSE-14`). Ignores NUL in IDs (`SSE-9`), unknown fields (`SSE-7`), and non-digit / overflow retries (`SSE-11`). - `SseStream` / `sseStreamFrom`: Resource-owning single-pass AsyncGenerator facade (`SSE-18`, `SSE-23`–`SSE-32`). Guarantees exactly-once release of the underlying Response body and BufferedSource across all termination routes (clean EOF, explicit `close()`, early `break`, consumer error, mid-stream read error, or abort signal). Teardown promise memoization ensures concurrent `close()` awaits in-flight releases and propagates failures (`SSE-30`). In-flight reader teardown on close is mapped to `IoError` (`SSE-31`). Abort listeners are cleaned up on normal completion. - `typedSseStream`: Lazy per-element stream adapter passing raw event name and newline-joined data (`SSE-33`, `SSE-35`). Dispatches `MapperOutcome<T>` union (`mapperValue`, `MAPPER_SKIP`, `MAPPER_DONE`, `SSE-34`). Releases stream resource before propagating any mapper error, attaching close failure as suppressed (`SSE-36`). Tooling, gates, and conformance: - `scripts/verify-sse-37.mjs` & `test:scripts`: Recursive AST/regex gate enforcing zero serde dependencies in core SSE (`SSE-37`) and no reconnect / `Last-Event-ID` paths (`SSE-38`). - `test/node-conformance/sse.test.mjs`: 8 new conformance test cases running over real Node Web Streams and TextDecoder under `node --test` across Node 20.3.0 and LTS (`test:node`). - `docs/superpowers/plans/2026-07-28-phase6b-sse-checklist.md`: Full requirement traceability checklist mapping all 41 SSE requirements to code and tests. - `docs/open-items.md`: Recorded Section I entries for `SSE-41` reactive adapter deferral (Phase 8b `@dexpace/rx`), line reader separation rationale (`IO-14` vs `SSE-2`), `Symbol.asyncDispose` runtime floor guard, and JavaScript hash equality. Gates verified: typecheck, lint, build, bun test (1,514 passing, 100% coverage on src/sse/*), api, lint:publish (publint + attw), verify:dual-consumption, verify:consumer-types, verify:seam-1, verify:sse-37, test:scripts (40 passing), verify:runtime-floor, test:node (87 passing), and audit.
Wahbeh-Mohammad
added a commit
that referenced
this pull request
Aug 27, 2026
* Phase 6a: Serde seam and @dexpace/codec-json (#45) * feat: phase 6a — the serde seam and @dexpace/codec-json Reshape the serialization seam around an explicit runtime type witness and ship the workspace's second package. Every decode now takes a caller-supplied `Schema<T>` value — the structural `{parse(input: unknown): T}` shape Zod, Valibot, ArkType and effect/schema all satisfy without an adapter. That closes SEAM-21: TypeScript erases types completely, so the schema value *is* the reification, and because the compiler infers `T` from it, the runtime witness and the static type are one artifact rather than two kept in sync by convention. `Serde` consequently drops its type parameter — a bundle is per wire format, not per DTO, which is what SERDE-1 actually says — so one `jsonSerde()` serves every DTO in an application. Phase 2 kept `Serde<T>` out of the public barrel precisely so this reshape would not break a published API. It is promoted here, forced rather than chosen: `@dexpace/codec-json` is a separate package and can reach core only through its public entry point. @dexpace/core: - `Schema`, `Serializer`, `Deserializer`, `Serde` — all four SEAM-20 allocation profiles, including the fresh-string one an earlier draft dropped - `Tristate<T>` with PATCH three-state semantics; `present()` takes `NonNullable<T>`, so SERDE-14's illegal fourth state is unrepresentable at the type level rather than rejected at construction - `SerializationError` / `DeserializationError` as two flat leaves under `DexpaceError` plus an `isSerdeError` guard — the tree stays two levels - `serdeBody()`: the serde's declared media type is the default `Content-Type`, with no format-agnostic fallback anywhere on the path (SERDE-2) - `decodeResponse()` / `decodeSuccessResponse()`, closing the response on every path via Phase 4b's `releaseQuietly`/`withReleaseFailure` so a teardown failure rides along as suppressed instead of displacing the decode failure @dexpace/codec-json (new, zero external dependencies — NFR-2): - `jsonSerde()`, the Tristate replacer installed by default with a named opt-out, and the `tristate()` / `tristateObject()` decode combinators Workspace, the three Phase-0 deferrals a second package makes live: - Bun `workspaces.catalog` as the single source of tool versions (NFR-14) - `@dexpace/core` peer + `peerDependenciesMeta`, with a cross-package test that proves the consequence rather than the declaration: `TRISTATE_BRAND` is a registry-global `Symbol.for`, so the codec keeps recognizing a caller's Tristate values even across two non-identical copies of core - `verify:seam-1`, `verify:consumer-types` and `verify:dual-consumption` generalized from core-only to every package Three adversarial-pass bugs worth naming, all reproduced against the built artifacts before fixing: - a stream failure was re-typed as a payload failure — the guard tested `instanceof IoError` while core's I/O tree is flat, so four of five classes were re-stamped as `DeserializationError`, inverting SERDE-12 - a JSON key named `""` collided with the replacer's top-level detection and emitted `null` where the key should have been omitted, silently turning a PATCH "leave unchanged" into "clear" - `key in source` walked the prototype chain, so all eleven `Object.prototype` member names decoded as Present when the wire omitted them Deviations, deferrals and the two open questions this phase could not settle (`IoError`'s reachability, `toHttpError`'s teardown masking) are recorded in docs/open-items.md §H1–H17. * fix: ci checks. * fix: ci checks. * feat(core): phase 6b — Server-Sent Events (SSE-1..41). (#47) Implement the full Server-Sent Events subsystem in `@dexpace/core` per `docs/product-spec/13-server-sent-events-and-streaming.md` (SSE-1 through SSE-41). The implementation is strictly pull-based, single-pass, and zero-dependency: events are parsed 1:1 on demand as the consumer polls the stream, with no unbounded buffering and no auto-reconnection logic. @dexpace/core additions: - `SseEvent`, `makeSseEvent`: Immutable event representation with defensively copied and frozen `data` lines (`SSE-20`). Structural equality (`sseEventsEqual`), string representation (`sseEventToString`), and content predicate (`isSseEventEmpty`, where comments count as content per `SSE-22`). - `SseLineReader` (@internal): Hand-rolled line framing supporting `\n`, `\r`, and `\r\n` line terminators across chunk boundaries (`SSE-2`), start-only UTF-8 BOM stripping via non-consuming `peek()` lookahead (`SSE-12`), and an optional configurable line length cap (`maxLineBytes` / `SseLineTooLongError`, `SSE-19`). - `SseParser` (@internal): Single-pass state machine implementing WHATWG SSE grammar with the three reference spec deviations: comments captured and dispatched (`SSE-6`), permissive dispatch when any of the 5 fields are set (`SSE-13`), and EOF dispatch of pending fields (`SSE-14`). Ignores NUL in IDs (`SSE-9`), unknown fields (`SSE-7`), and non-digit / overflow retries (`SSE-11`). - `SseStream` / `sseStreamFrom`: Resource-owning single-pass AsyncGenerator facade (`SSE-18`, `SSE-23`–`SSE-32`). Guarantees exactly-once release of the underlying Response body and BufferedSource across all termination routes (clean EOF, explicit `close()`, early `break`, consumer error, mid-stream read error, or abort signal). Teardown promise memoization ensures concurrent `close()` awaits in-flight releases and propagates failures (`SSE-30`). In-flight reader teardown on close is mapped to `IoError` (`SSE-31`). Abort listeners are cleaned up on normal completion. - `typedSseStream`: Lazy per-element stream adapter passing raw event name and newline-joined data (`SSE-33`, `SSE-35`). Dispatches `MapperOutcome<T>` union (`mapperValue`, `MAPPER_SKIP`, `MAPPER_DONE`, `SSE-34`). Releases stream resource before propagating any mapper error, attaching close failure as suppressed (`SSE-36`). Tooling, gates, and conformance: - `scripts/verify-sse-37.mjs` & `test:scripts`: Recursive AST/regex gate enforcing zero serde dependencies in core SSE (`SSE-37`) and no reconnect / `Last-Event-ID` paths (`SSE-38`). - `test/node-conformance/sse.test.mjs`: 8 new conformance test cases running over real Node Web Streams and TextDecoder under `node --test` across Node 20.3.0 and LTS (`test:node`). - `docs/superpowers/plans/2026-07-28-phase6b-sse-checklist.md`: Full requirement traceability checklist mapping all 41 SSE requirements to code and tests. - `docs/open-items.md`: Recorded Section I entries for `SSE-41` reactive adapter deferral (Phase 8b `@dexpace/rx`), line reader separation rationale (`IO-14` vs `SSE-2`), `Symbol.asyncDispose` runtime floor guard, and JavaScript hash equality. Gates verified: typecheck, lint, build, bun test (1,514 passing, 100% coverage on src/sse/*), api, lint:publish (publint + attw), verify:dual-consumption, verify:consumer-types, verify:seam-1, verify:sse-37, test:scripts (40 passing), verify:runtime-floor, test:node (87 passing), and audit. * feat(core): phase 6c — pagination engine and built-in strategies (#17). (#46) Implement the Phase 6c pagination subsystem, delivering transport- and serde-agnostic lazy pagination walks with dual consumption views, built-in strategies, RFC 8288 Link parsing, verbatim query parameter splicing, and deterministic response lifecycle management. Closes PAGE-1 through PAGE-36: @dexpace/core: - `Page<T>`: wraps a live transport response and frozen materialized items (`PAGE-1`, `PAGE-2`, `PAGE-30`); implements `AsyncDisposable` (`[Symbol.asyncDispose]`) for `await using` explicit resource management; metadata and items survive `close()` (`PAGE-2`). - `PageInfo<T>` & `PaginationStrategy<T>`: stateless async parser interface returning `Promise<PageInfo<T>>` with `{items, nextRequest}`; `undefined` signals end of stream (`PAGE-4`, `PAGE-5`, `PAGE-29`). - `Paginator<T>`: lazy generator driver (`PAGE-6`, `PAGE-7`) exposing: - `items()`: reusable sequence view yielding server-order items, closing each page's underlying response before yielding any items to eliminate stranded connections (`PAGE-8`, `PAGE-11`). - `pages()`: single-use sequence view yielding whole `Page` objects with raw response access (`PAGE-14`). - `maxPages` cap enforcement evaluated before wire dispatch (`PAGE-9`, `PAGE-10`). - Strict cancellation safety with pre-dispatch abort checks and drop-and-close handling for in-flight responses arriving after cancellation (`PAGE-25`, `PAGE-26`, `PAGE-33`). - Iterative generator loop guaranteeing stack safety across thousands of pages (`PAGE-31`). - Built-in pagination strategies (`PAGE-16`–`PAGE-20`): - `cursorStrategy()`: single body read with configurable cursor query parameter. - `pageNumberStrategy()`: 1-based (or configured) start-page fallback with next-page advance. - `linkHeaderStrategy()`: RFC 8288 Link header parser supporting multi-header concatenation, unquoted/quoted/case-insensitive `rel="next"`, and RFC 3986 reference resolution. - `query-splice` (internal): - Hand-rolled query substring tokenizer splicing targeted parameters without `URLSearchParams` canonicalization, preserving untargeted query bytes and non-query components byte-for-byte (`PAGE-21`, `PAGE-22`, `PAGE-23`, `PAGE-24`). - `paginateWithFetchers()`: - Higher-level functional front-end threading a single shared mutable `PagingOptions` bag across `first()` and `next()` invocations (`PAGE-34`, `PAGE-35`). - `PaginationError`: - Flat leaf under `DexpaceError` for precondition and engine misuse; underlying network, I/O, and parse causes propagate unwrapped (`PAGE-28`). Node conformance: - `test/node-conformance/pagination.test.mjs`: validates `Page` explicit resource management, `Paginator` items and pages walks, `AbortSignal` thread-through, and response stream cancellation against real Node Web Streams. Traceability & ledger: - `docs/superpowers/plans/2026-07-28-phase6c-pagination-checklist.md` tracking `PAGE-1`..`PAGE-36`. - `docs/open-items.md` Section I updated with ledger entries I1–I8 (close-before-yield precedence, async parse boundary, `AsyncDisposable` lib requirement, WHATWG query encode-set boundary, transport-direct execution, single-use view asymmetry, iterative loop drive, error unwrapping). Full gate sequence green (1527 unit tests with 100% pagination line/branch coverage, 82 Node conformance tests, publint, attw, dual-consumption, consumer-types, seam-1, runtime-floor). * docs: update changesets for phase6.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
1. What Changed
@dexpace/core(SSE-1–SSE-41): AddedSseEvent,makeSseEvent,sseEventsEqual,isSseEventEmpty,sseEventToString,SseStream,sseStreamFrom,typedSseStream, andSseStreamError.SseLineReaderhandling\n,\r, and\r\nline terminators across chunk boundaries (SSE-2), start-only UTF-8 BOM lookahead (SSE-12), and configurable line caps (maxLineBytes/SseLineTooLongError,SSE-19). Implemented internalSseParserstate machine with WHATWG SSE grammar and reference spec extensions (SSE-6,SSE-13,SSE-14).SseStreamprovides a single-passAsyncGeneratorfacade guaranteeing exactly-once release of the underlying response body across all termination paths (clean EOF, explicitclose(), earlybreak, error, or abort signal). Teardown promise memoization ensures concurrentclose()awaits in-flight release and propagates failures. In-flight read teardown is mapped toIoError(SSE-31). Abort listeners are cleaned up upon stream completion.typedSseStreamenables lazy per-element mapping viaMapperOutcome<T>(mapperValue,MAPPER_SKIP,MAPPER_DONE,SSE-33–SSE-36), releasing stream resources before propagating any mapper error.scripts/verify-sse-37.mjsAST/regex gate enforcing zero serde dependencies (SSE-37) and no reconnect /Last-Event-IDpaths (SSE-38). Added 8 Node conformance test cases intest/node-conformance/sse.test.mjsexercising SSE over real Node Web Streams and TextDecoder. Added full requirement checklist indocs/superpowers/plans/2026-07-28-phase6b-sse-checklist.md.2. Reviews Done
SSE-1throughSSE-41.SSE-31), race conditions, abort listener lifecycle, and dual-failure error suppression inclosingBoth.@public/@internalrelease tags, license headers (SPDX), and styleguide rules.ignoreBOM: truefor line decoding, removed[Symbol.asyncDispose]from.d.tsto preserveES2023lib compatibility, added abort listener unbinding on normal close, memoized teardown promises inSseStream.close(), hardenedverify:sse-37with recursive walking and dynamic import detection, and created Node conformance tests.bun test), 87 Node conformance tests passing (test:node), 40 script tests passing (test:scripts), 100% line coverage onsrc/sse/*, clean typecheck, clean lint, clean publint/attw, clean dual consumption, and zero audit vulnerabilities.3. Open Items / Deferred Items
I1—SSE-41Reactive Adapter: Backpressure-honoringObservableview deferred to Phase 8b bridge package (@dexpace/rx).I2—SseLineReaderSeparation: Recorded whyBufferedSource.readUtf8Line()(IO-14, keeps lone\ras content) was not reused for SSE (SSE-2, requires lone\rto terminate lines).I3—[Symbol.asyncDispose]Runtime Guard: Omitted from static.d.tsand installed conditionally at runtime because the pinned Node 20.3 floor predatesSymbol.asyncDispose(Node 20.4). Becomes unconditional whenengines.nodeadvances past Node 20.4.I4—SSE-21Hash Equality: JVM-stylehashCode()is N/A in JavaScript; structural value equality is provided viasseEventsEqual().