morph's unit suite (tests/CMakeLists.txt, Catch2) is deterministic, single-shot, and in-process: every test picks a
fixed input and asserts one outcome. Four additional, opt-in test
categories complement it by exercising the framework over distributions of
input and time, rather than hand-picked examples: a fuzz harness, a soak
test, a load/latency benchmark, and an adversarial cross-socket run. None of
these run in the default ctest sweep or the fast unit build — each lives
behind its own CMake option, default OFF, so an ordinary cmake --build
ctestis unaffected by their existence.
- Fuzz harness
- Soak tests
- Load / latency benchmark
- Concurrent-user benchmark
- Adversarial cross-socket run
- Install / export consumability
- Rung bundles load without warnings
- Declined techniques, and why
- Cross-references
Built only under -DMORPH_BUILD_FUZZERS=ON; requires Clang, since it uses
libFuzzer (-fsanitize=fuzzer), the compiler-builtin coverage-guided fuzzing
engine — no third-party fuzzing dependency or service. Configuring this option
under a non-Clang compiler fails the configure step with a clear message
rather than silently building nothing.
Two targets:
| Target | Exercises | Invariant |
|---|---|---|
fuzz_wire_decode |
morph::wire::decode (the outer envelope parse) and, for a decoded "execute" envelope with a non-empty body, ActionTraits<FuzzInnerAction>::fromJson(body) (the inner re-parse — see wire.md's "the body double-parse hazard") |
Every input either parses (at each stage) or throws std::runtime_error. Never crashes, trips a sanitizer, or hangs. |
fuzz_dispatch_execute |
morph::backend::RemoteServer::handle end to end, against a live server with one pre-registered Fuzz_DispatchModel instance |
Every input yields a reply that itself decodes as a wire Envelope with kind "ok" or "err". Never crashes; never hangs (bounded by libFuzzer's own -timeout, not by anything in the harness). |
Both binaries link -fsanitize=fuzzer,address (via the apply_fuzzer() CMake
function in cmake/compiler_options.cmake), so a crash found while fuzzing is
also an AddressSanitizer memory-safety finding, not just a caught exception.
Corpus. tests/fuzz/corpus/wire_decode/*.txt and
tests/fuzz/corpus/dispatch_execute/*.txt hold hand-picked seeds drawn from
the existing hardening tests (test_wire_hardening.cpp, test_server_limits.cpp)
— a valid register/execute/deregister envelope, a duplicate-key envelope, a
moderately-nested body, a malformed-UTF-8 body, and plain garbage.
tests/fuzz/findings/wire_decode/ and tests/fuzz/findings/dispatch_execute/
hold the committed reproducers and are where any input that triggers a
crash/hang/sanitizer report gets committed as a permanent regression case, once
the bug it found is fixed — see "Known findings" below.
CI-integrated regression check. ctest -R fuzz_.*_replay runs each target
once per file in its corpus + findings directories (libFuzzer's single-run
replay mode — passing individual files, not a directory, so it replays and
exits rather than starting a mutating fuzzing session) and fails if any input
crashes. This is fast and deterministic, suitable for CI; it is not a
fuzzing campaign.
The input globs are deliberately extension-agnostic (*, not *.txt): a
fuzzer input is an arbitrary byte string, and a libFuzzer-minimized reproducer
is conventionally saved as .bin. An extension-scoped glob silently skips
whatever it does not match — the committed reproducers are .bin, so a *.txt
glob would replay only the seeds and leave the regression cases unrun. An empty
glob is a configure-time FATAL_ERROR: a replay invocation with no FILE
arguments is an unbounded fuzzing run, not a regression check. The nightly
linux-all-features job additionally asserts, after the run, that every file
under tests/fuzz/findings/ was actually referenced — a guard that never fires
is indistinguishable from one that works.
Running an actual campaign (manual / scheduled, not CI-per-commit):
cmake --preset clang-release -DMORPH_BUILD_FUZZERS=ON
cmake --build build/clang-release --target fuzz_wire_decode fuzz_dispatch_execute
./build/clang-release/tests/fuzz/fuzz_wire_decode -max_total_time=3600 tests/fuzz/corpus/wire_decode
Passing the corpus directory as the sole argument makes libFuzzer both seed
from and write newly-discovered coverage-increasing inputs back into that same
directory — review and keep genuinely new, interesting cases; move anything
that crashes into findings/ instead, once the underlying bug is fixed (see
below).
Known findings. The committed reproducers under tests/fuzz/findings/
each guard a property of morph::wire's glaze-based parsing that fuzzing
showed can fail:
- Decode never reads past the buffer
(
tests/fuzz/findings/wire_decode/skip_ws_heap_overflow.bin). Everyglz::read<>call site that accepts an arbitrarystring_view—morph::wire::decode,journal::fromJson,BRIDGE_REGISTER_ACTION's generatedfromJson/resultFromJson,FileOfflineQueue'sfromJson— sets.null_terminated = false. With glaze's default (true),skip_wsscans past the buffer's real end for a terminator astring_viewnever guarantees, and a 5-byte input ({"{k) is an AddressSanitizer heap-buffer-overflow any peer can send aRemoteServer. The option costs nothing beyond an optimization that never legitimately applied.session_auth.hpp's token-claims decode parses aconst std::string&, which has the standard's null terminator, and needs no option. - Every reply and every body is valid JSON whatever bytes it carries
(
tests/fuzz/findings/dispatch_execute/err_reply_control_char_roundtrip.bin). Glaze's default writer escapes"and\but not ASCII control bytes (0x00-0x1F, 0x7F), so a string carrying one produces invalid JSON — and, when the same string also holds a\or", glaze's chunked write path corrupts the byte into two0x00bytes while the output still decodes. Three layers close it:errmessages go throughwire::makeErr, whosedetail::sanitizeControlCharsreplaces each control byte with a printable\xHHplaceholder. An err message echoes untrusted text (an unrecognizedEnvelope::kind, a caught exception'swhat()) and is log-bound, so substitution is right there: a raw0x1Bwould carry an ANSI escape into the reader's terminal.- Every other envelope string —
body,modelType,actionType,contextKey,typeId, the session'sprincipal/token— carries caller data that must round-trip byte for byte, sowire::encodewrites withwire::detail::EscapingWriteOpts(glaze'sescape_control_characters), which is lossless in both directions. - Action and result bodies are written not by
wire::encodebut byActionTraits<A>::toJson/resultToJson(registry.hpp'sBRIDGE_REGISTER_ACTION), with the same instrument one layer down,model::detail::EscapingWriteOpts(see docs/spec/core/registry.md, "Control bytes in action and result bodies"). Action bodies are pure caller data (a paste's content, a chat message, a filename), so they are at least as exposed as the envelope; an application-ladder rung replaystests/fuzz/findings/as stored user content to cover that round trip.
Each is also covered by a dedicated regression test in
tests/test_wire_hardening.cpp ("Bug C" through "Bug G") in addition to the
fuzz_*_replay findings above.
Built only under -DMORPH_BUILD_LOAD_TESTS=ON (requires MORPH_BUILD_TESTS=ON,
since these link Catch2). Two Catch2 test cases in the morph_soak binary:
test_soak_switch_backend.cpp— cyclesBridge::switchBackend()between aLocalBackendand a freshSimulatedRemoteBackend/RemoteServerpair every cycle, firing a burst ofexecute()calls each cycle. Asserts every issued completion eventually resolves (value orBackendChangedError— see backend.md), that the live model-instance count settles back down after the run (a leakedIModelHolderwould show as monotonic growth), and — on Linux, where/proc/self/statusis readable — that RSS growth across the run stays under a configurable bound. The final iteration always lands on a freshLocalBackendbefore the lastRemoteServeris dropped, so the bridge's active backend never outlives the object it references (seebackend.md's "Lifetime & ownership").test_soak_reconnect_churn.cpp— wires a realNetworkMonitor→ReconnectCoordinator→SyncWorkerpipeline exactly as offline.md's "End-to-end integration" shows, and flips online/offline hundreds of times. Asserts the offline queue is fully drained after every flap and everyonOnline()reconnects (this test'stryReconnectnever fails).
Both are CI-sized by default (a few hundred cycles, seconds to run) and scale to a real multi-hour soak via environment variables, without touching code:
| Variable | Default | Test |
|---|---|---|
MORPH_SOAK_CYCLES |
200 |
switch-backend churn |
MORPH_SOAK_EXECUTES_PER_CYCLE |
20 |
switch-backend churn |
MORPH_SOAK_RSS_SAMPLE_EVERY |
20 |
switch-backend churn |
MORPH_SOAK_RSS_GROWTH_KB_MAX |
102400 (100 MiB) |
switch-backend churn |
MORPH_SOAK_FLAP_CYCLES |
150 |
reconnect churn |
A production morph::observe metrics seam already exists
(observability.md) and instruments RemoteServer/
LocalBackend dispatch, SyncWorker::run(), and
ReconnectCoordinator::onOnline() directly. These soak tests deliberately do
not depend on a particular MetricSink being installed in the test process,
though: they instrument themselves directly instead (a model-local instance
counter, /proc/self/status, and locally-owned atomic call counters), so
their pass/fail signal never depends on how (or whether) a host application
has wired up observability.
Both morph_soak and morph_bench are sanitizer-instrumented when
AF_SANITIZER is set. They are opt-in, so no default sanitizer leg
pays for them; the reason to instrument them rather than exempt them is that
churn over thousands of cycles is exactly the shape of test whose finding is a
leak or a race and not a failed assertion. Under a sanitizer preset the
benchmark's published numbers are not comparable with an ordinary run's and are
not meant to be — there it is a correctness run over the dispatch path.
Built only under -DMORPH_BUILD_LOAD_TESTS=ON alongside the soak tests (same
option; different target, morph_bench). bench_dispatch_latency.cpp drives
a RemoteServer directly against a trivial echo model (isolating framework
overhead from business logic):
- Latency — 2000 serial (concurrency-1) round trips; reports p50/p95/p99 wall time in milliseconds (nearest-rank percentile over the sorted sample).
- Throughput — a window at each of concurrency 1/2/4/8/16, reporting
executes/second (
MORPH_BENCH_WINDOW_MS, default 200 ms). - Both phases run
MORPH_BENCH_TRIALStimes (default 5), and the run reports the best, median and worst trial of every figure rather than one number — see below. - Writes
bench_dispatch_latency.jsoninto the build directory. Thep50_ms/p95_ms/p99_ms/throughputkeys carry the best trial;trials,latency_samples_per_trial,throughput_window_ms,p99_ms_median_trial,p99_ms_worst_trial, each throughput point'sexecutes_per_sec_worst_trial, and atrial_detailarray with every trial's own figures are additive. Successive runs are archived and diffed for a regression, andtrial_detailis what a diff finer than the gates below has to read. - Enforces two coarse regression gates via
CHECK:p99 <= MORPH_BENCH_P99_MS_MAX(default 50.0 ms) and concurrency-1 throughput>= MORPH_BENCH_MIN_THROUGHPUT(default 500.0 executes/sec). Both are environment-variable-overridable so CI hardware differences don't need a code change, and both read the best trial — see below.
The serial phase must not wait on a polling step, or it reports the poll.
Waiting on each reply with morph::testing::WaitReply does exactly that: its
await() calls waitUntil, which sleeps 5 ms between predicate checks.
Measured that way, same binary, 20 processes per configuration, Release:
| idle | 16-way oversubscribed | |
|---|---|---|
| p50 | 5.0562 / 5.0714 / 5.0740 ms | 0.0166 / 0.0167 / 0.0216 ms |
| c=1 executes/sec | 171467 / 175928 / 178855 | 311.9 / 1749.9 / 18627.1 |
A 302x swing in the headline figure, selected by machine load, and neither mode is the dispatch latency: the same idle processes report ~176k executes/sec at concurrency 1, i.e. a round trip of about 5.7 µs. An idle machine reports one whole sleep step per call; a busy one reports the case where the reply beats the caller's first predicate check. 311.9 executes/sec is also below the 500/sec floor this file enforces, so the gate would fire on machine load rather than on morph. The benchmark therefore uses a local condition-variable reply sink, and a blocking drain at the end of each throughput window for the same reason — a polling drain sits inside the window's own elapsed time.
It reports a distribution because a wall-clock figure has no regime to pin.
morph_bench_alloc can pin its race, and an allocation count then comes out
exact. Contention is not a mode, it is a tax, so this benchmark reports the
spread over trials instead.
The gates read the best trial: contention can only make latency worse and
throughput lower, so the best of N is the least contaminated estimate of what
the code costs, while a real regression moves every trial including the best.
Measured, 20 processes per configuration, best-of-five figures:
| p99 (ms) | c=1 executes/sec | |
|---|---|---|
| Release idle | 0.0090 / 0.0096 / 0.0104 | 173169 / 176282 / 181641 |
| Release loaded | 0.0181 / 0.0210 / 0.0259 | 56825 / 61211 / 63479 |
| Debug idle | 0.0325 / 0.0343 / 0.0360 | 42560 / 43150 / 43996 |
| Debug loaded | 0.0598 / 0.0608 / 1.8717 | 1549 / 22684 / 22990 |
The Debug rows are the ones the defaults must hold for: the only CI leg setting
MORPH_BUILD_LOAD_TESTS=ON is the nightly linux-all-features job
(nightly-slow-checks.yml), on the gcc-debug / clang-debug presets, and its ctest --preset passes no label filter. The
two ceilings are therefore unchanged, and that is a measurement rather than
timidity: Debug-under-load still spans 28x on throughput and 57x on p99 for
the same binary, so no pair of constants separates a regression from a
contended runner. 50 ms is ~27x the worst best-trial p99 measured and 500/sec
~3.1x below the worst best-trial throughput — a dispatch path made three times
slower passes both. Setting the floor from a 12-core box instead turns 3 of 20
Debug-under-load processes red, so that is not the way out of it either.
The property those loose ceilings look like they leave ungated is gated
elsewhere, tightly. An allocation gate is what to weigh before tightening any
wall-clock ceiling, and it exists: bench.alloc_budget
(below) is set by rule at one allocation above the figure the benchmark
measures — not a tolerance band — and that figure came out identical on every
one of 20 processes across three toolchains, idle and loaded alike: a
one-allocation margin, on a quantity machine load cannot move.
The two numbers are deliberately not restated here. Both live in
tests/bench/CMakeLists.txt — the ceiling as MORPH_ALLOC_BUDGET_PER_CALL,
the measured figure in the comment that derives it — and they move together
every time the dispatch path gets cheaper, while the argument they support does
not; so this paragraph keeps the argument, and that file the current values.
So "dispatch does not get more expensive" is already gated to a resolution no
wall-clock constant on any host can approach. What the two ceilings gate is the
residue: a regression that costs time without costing allocations — a spin, a
syscall, a lock held longer. That is a real class, and it is the only class
they are for.
Two things a future tightening needs, which the benchmark produces.
- A cross-process distribution (
bench_dispatch_latency.jsonl, beside the per-process.json, overridable withMORPH_BENCH_LEDGER,offto disable).MORPH_BENCH_TRIALSgives a distribution over trials within a process, which mitigates the spread but does not record it — one process still prints one triple and cannot say where it sits among others. Each run appends one line (pid, the headline percentiles, concurrency-1 throughput,load_1m,inject_delay_us) and prints its own rank among every line already there, so N unorchestrated runs produce the distribution a candidate ceiling would be read off. The load average is on the row and not only on stdout, because load sweeps put the same binary 28x apart on throughput between an idle box and a loaded one: a figure without its load is not comparable with another figure. Rows are appendedO_APPENDin one write and a row that cannot be parsed is skipped, because a diagnostic ledger must never redden a benchmark. - An injectable regression (
MORPH_BENCH_INJECT_DELAY_US), which spins for that many microseconds inside the model'sexecute— inside the round trip both phases measure. A spin and not a sleep: a sleep yields the core and would flatter the throughput phase. This is what makes "what size of regression does this gate catch?" a command rather than an argument, and it is what any candidate ceiling should be shown firing against before it lands. A run carrying an injection says so on stdout and on its ledger row.
Both ceilings fire, and the size they fire at is measured. Release, clang
22.1.8, 12-core x86-64 Linux, MORPH_BENCH_TRIALS=1, load average 11.9–12.8:
| injected | p99 (ms) | c=1 executes/sec | p99 gate | throughput gate |
|---|---|---|---|---|
| 0 | 0.0142 | 168862 | pass | pass |
| 1000 µs | 1.02685 | 989.2 | pass | pass |
| 2000 µs | 2.03428 | 495.5 | pass | FAIL |
| 3000 µs | 3.18618 | 331.2 | pass | FAIL |
| 60000 µs | 61.0339 | 16.7 | FAIL | FAIL |
The baseline round trip is ~5.9 µs, so the throughput floor — the tighter of the
two — first speaks at roughly a 340× regression and the p99 ceiling at
roughly 8500×. The ratio, not the time, is the portable part: reproduce the
table on any host with MORPH_BENCH_INJECT_DELAY_US.
What is not established, and is why the ceilings stay loose: nobody has characterised the CI runner. Both sets of figures above come from a workstation, which is exactly the misleading configuration to set a CI ceiling from. Guessing a second time from the same box would be the first mistake with a different number.
The echo model/action (BenchEchoModel/BenchEchoAction) are declared at
file scope, not inside the file's anonymous namespace with its other local
helpers — Glaze's reflection needs external linkage to mangle the type name,
the same requirement tests/fuzz/'s harness fixtures document.
A second binary under the same option, and deliberately not a second case
in morph_bench: it replaces the global operator new/delete, which is
process-wide and would perturb any other measurement sharing the binary. It
counts the heap allocations one Ping -> Pong round trip costs through
LocalBackend — 50 warm-up calls excluded, every call waited out, no JSON and
no socket — and prints the total, the bytes, and (with --attribute) the size
of every allocation in one steady-state call.
It also runs a group of narrower censuses over morph's four registry lookups,
each one taken over ids past libstdc++'s 15-character SSO buffer and again over
ids inside it — the cost is entirely id-length-dependent, so a census over one
side alone either measures zero or overstates the saving, and morph's real ids
straddle the line ("CreateSwimlane" is 14 characters, one under):
| census | opaque key | transparent key | pure lookup? |
|---|---|---|---|
ActionDispatcher::coalesce + requiredFieldsFor |
2.00 / 0.00 | 0.00 / 0.00 | yes |
PayloadMigrationRegistry::find |
2.00 / 0.00 | 0.00 / 0.00 | yes |
BridgeHandler::executeJson |
24.07 / 21.06 | 21.07 / 21.06 | no — a whole round trip |
ModelRegistryFactory::create |
2.00 / 1.00 | 2.00 / 1.00 | no — constructs a holder; parked |
The first two decode nothing and execute nothing, so what they allocate is exactly what looking a registry key up costs. The last two do more than look up, so their figure is a floor plus the key and what is comparable between runs is the difference between the long-id and short-id columns.
It exists because a scoping number goes stale the moment anything on the path changes, and re-deriving one from a prose description of how it was once taken is how a fix ends up built against a figure nobody re-checked. The census re-takes it on demand instead.
It is both an instrument and a control, and which one it is at any moment
depends on the flags. Run bare, it asserts nothing and the number it prints
is the whole output; a green run of it in that mode proves nothing. Given
--budget=<n> and/or --lookup-budget=<n> it fails when the figure exceeds
the ceiling, and it is registered with ctest in exactly that form, as
bench.alloc_budget.
The reason it took a per-toolchain budget to get there is that an allocation
count is standard-library specific, not only morph-specific: std::function's
inline buffer and std::string's SSO threshold differ between libstdc++,
libc++ and MSVC's STL. So the ceiling is the cache variable
MORPH_ALLOC_BUDGET_PER_CALL, defaulted to a figure measured on the
toolchains that actually build this target in CI (Linux clang-debug and
gcc-debug), left unset on MSVC where nothing has measured it, and settable
to 0 to disable the gate. tests/bench/CMakeLists.txt carries the
measurements the default was chosen from and the headroom argument.
The --lookup-budget half takes no headroom at all: with transparent keys a
pure registry lookup allocates nothing, for ids of any length, and that is
a property which either holds or has regressed. The ctest
case passes --lookup-budget=0, and it covers ActionDispatcher and
PayloadMigrationRegistry.
--id-length-budget covers the third registry, ActionExecuteRegistry, which
cannot be probed on its own — it dispatches whatever it finds — so what is
gated is the gap between an executeJson over long ids and one over short
ones. 0.5 rather than 0, and the 0.5 is measured: the figure reads 0.01
because of a deterministic two-allocation one-off across a census's 200 calls,
shown to follow census order rather than id length by swapping the two
censuses, and identical in all of 10 processes. Making either half of that key
opaque again takes the gap to 3.00 (the Key's two std::strings, plus the one
executeJson builds from ModelTraits<Model>::typeId()), so 0.5 separates the
residue from the thing guarded with wide margin on both sides. Like the lookup half it needs no
per-toolchain default: it is a difference between two runs of one build.
All three ceilings were checked against a reverted fix. Removing
PayloadMigrationRegistry's PairKeyEqual takes --lookup-budget to 2.00 and
turns the case red; restoring ActionExecuteRegistry's Key{...} temporary
takes --id-length-budget to 2.01; restoring executeJson's
std::string{typeId()} alone takes it to 1.01.
Both halves are checked against a reverted fix rather than only against
themselves: each was run with the transparent keys taken back out, and each
went red. A budget that has never been seen to fail is the
control that measures nothing, which AGENTS.md ("Verify rather than assert")
warns about.
The dispatch census pins a race, and without that pin it is not comparable
between runs. A dispatch and the handlers attached to it race: win, and each
handler joins a vector the settle drains; lose, and each takes
CompletionState's attach-after-ready path, which costs differently. Ungated,
that makes the headline figure bimodal — measured, interleaved, 20 processes
per configuration: ~16.95 on an idle machine, ~13.06 with the machine 16-way
oversubscribed, same binary. The benchmark therefore holds the worker across
execute() and both attaches, so it always measures the attach-before-settle
regime: the more expensive of the two, and the one a real
GUI client is in.
With the race pinned, x86-64 Linux, clang 22.1.8 / libstdc++ 16.2.1, the opaque-key build costs 17.06 allocations per local round trip and 2.00 allocations per registry lookup for ids past the SSO buffer; the transparent-key build costs 14.06 and 0.00 per lookup — exactly 3.00 removed, in every one of 40 processes, idle and loaded, on clang Release, clang Debug and gcc Debug alike. A figure from an ungated benchmark is not comparable with either.
morph_bench measures one dispatch inside one process. This benchmark measures
a whole server from outside it: a ladder_bookmarks_server process, its
transport, its authorizer and its model lifecycle, under thousands of
simulated users who each connect, sign in, do 5-10 actions with think time,
and disconnect. Its question is how many concurrent users one server carries,
and which thread runs out first. scripts/scenario/README.md has its flags.
It measures the framework rather than the rung when the server runs with the
in-memory store (--store memory): each model instance then keeps its own
bookmarks in a container, journals nothing and touches no database, so an
action costs the framework plus a container operation. --workload framework
goes further and replaces every action with a Login round trip.
What it reports per run, and why each figure is there:
- Server CPU per thread, from
/proc/<pid>/task. A server whose total CPU is low can still be saturated, on one thread; the per-thread split is what shows which one. - Client CPU, so a run limited by its own load generator says so.
- Other processes' CPU on the host, from
/proc/statless the server and the clients. The machine this runs on is shared, and a run that competed for it warns rather than reporting a figure that is not comparable with a quiet one.
It is a manual tool, not a CI gate: its figures depend on the host, and its value is in comparing two runs on the same quiet machine.
Built under MORPH_BUILD_QT=ON (one more file in the morph_qt_tests
binary). A hostile QWebSocket
client, talking to a real QtWebSocketServer over loopback TCP (same-process,
like most of test_qt_websocket.cpp — see that file's dedicated
"Process separation: ..." cases for genuine OS-process separation), drives
four scenarios:
- An oversized frame (body past
wire::kMaxEnvelopeBytes). - A rapid flood of 5000 execute messages with no per-message wait.
- A duplicate-top-level-key envelope (the wire.md smuggling caveat, over a real socket).
- A connection that opens and then never sends a single frame.
After each, an honest QtWebSocketBackend client registers a model and
executes an action against the same server and must succeed — the
assertion in every scenario is that the server keeps serving honest
clients. RemoteServer::LimitPolicy and QtWebSocketServerConfig (see
backend.md) are
implemented, each with its own dedicated, precise unit coverage
(tests/test_limit_policy.cpp, tests/qt/test_qt_websocket.cpp) — this file
does not re-derive that coverage. Every server here uses the default,
unconfigured QtWebSocketServerConfig, under which maxConnections/
messagesPerSecond/handshakeTimeout/idleTimeout are all still
unbounded/disabled (0), but maxMessageBytes defaults to
wire::kMaxEnvelopeBytes (not 0) — so, concretely: scenario 1 (oversized
frame) is actively rejected by the default config and this file asserts
that directly (an err reply mentioning maxMessageBytes), while scenarios
2 and 4 (flood, stall) are not capped by default and this file asserts only
that the server keeps functioning under them, not that they are rejected.
tests/**/*.cpp files are written independently, and each is free to declare
its own local model/action types (OrderModel, Widget, Report, …) for
its scenario. A type declared at file scope — not inside an anonymous or
named namespace — has external linkage. If two different test files declare
a same-named, differently-defined type at file scope, that is a One
Definition Rule (ODR) violation: undefined behavior that neither the
compiler nor the linker diagnoses, since each translation unit only ever
sees its own definition. Which definition the linker keeps for a given call
site is link-order dependent, so the bug can pass locally and fail in CI (or
the reverse), with no diagnostic pointing at the cause: a bare OrderModel
stub in one file can silently win over another file's real OrderModel, so
the real one's onBackendChanged() is never invoked and its offline queue
never drains.
Convention: wrap local model/action types in an anonymous namespace by
default. Only lift a type to file scope when a macro that requires external
linkage (BRIDGE_REGISTER_MODEL, BRIDGE_REGISTER_ACTION) needs it — and in
that case, prefer a short, file/feature-specific prefix over a generic name
(StepILOrderModel, not OrderModel; see tests/test_remote_step_interleaving.cpp).
A type declared inside a named namespace is also safe, since its linker
symbol is namespace-qualified (e.g. namespace registration::models { struct Report { ... }; }, which is how a test that needs
BRIDGE_REGISTER_MODEL/BRIDGE_REGISTER_ACTION at namespace scope keeps its
types distinct).
Why UndefinedBehaviorSanitizer (clang-ubsan, cmake/compiler_options.cmake's
-fsanitize=undefined) does not catch this. UBSan instruments individual
operations within a translation unit at compile time — signed overflow, an
invalid enum load, a misaligned access, a bad vptr cast — and its runtime
checks fire while that instrumented code executes. An ODR violation across
two TUs has no such moment: each TU's own code is entirely well-formed and
runs exactly as instrumented: a real OrderModel calls its own
onBackendChanged() correctly when compiled and run alone, and a bare stub of
the same name in another file is equally well-formed on its own. The defect exists only in what the linker does
across the two .o files — silently keeping one TU's definition of
BackendChangedNotifiable<OrderModel> for both — which happens before any
sanitizer runtime is even loaded. No sanitizer (ASan, UBSan, TSan, MSan) is
designed to catch cross-TU ODR violations; this is a structural blind spot
of the whole compile-time-instrumentation family, not a configuration gap.
The linker's own ordinary duplicate-symbol diagnostics don't fire either,
for a related reason: the affected symbol
(BackendChangedNotifiable<OrderModel>, a template instantiation) is
implicitly inline, and the linker treats multiple same-mangled-name
inline/template definitions across TUs as expected COMDAT folding — the
normal, correct case when two TUs instantiate the same template
identically. It has no way to distinguish that from "two different
definitions that happen to share a mangled name," which is exactly this bug,
so it silently keeps one arbitrarily in both cases. (GNU gold's
--detect-odr-violations is the closest built-in linker feature, but it is
gold-specific, heuristic on this exact template-instantiation shape, and not
wired into any preset here.)
So only a source-level check could catch it — the one layer that sees both TUs' declarations before they are compiled down to symbols a linker or sanitizer could reason about. No such check runs in CI: the convention above is kept by review.
Every other suite in this document builds morph from inside the tree, where
include/ is on the include path because the build put it there. That hides a
whole class of defect: cmake --install can exit 0 having installed Glaze's
headers and a working glazeConfig.cmake — Glaze carries its own
install/export rules and gets them for free through FetchContent — while
installing zero morph headers and no morphConfig.cmake. Unless a CI leg
installs morph, nothing notices. A consumer following the standard CMake
workflow then gets a prefix holding
someone else's dependency and none of the library they meant to install.
CI-enforced by a dedicated job (install-export in ci.yml), which
self-tests the checker first, then runs it. The checker installs morph to a
scratch prefix and compiles and runs a consumer project outside the tree —
find_package(morph CONFIG REQUIRED COMPONENTS net) plus
target_link_libraries(consumer PRIVATE morph::morph morph::net) over one
translation unit including morph/core/bridge.hpp, morph/forms/forms.hpp,
morph/util/quantity.hpp, morph/util/rational.hpp and morph/version.hpp.
Compiling is the point, not find_package succeeding. Three defects in the
install rules produced a prefix that find_package accepted and that no
filename inspection would have questioned:
- installing only the declared
FILE_SET HEADERSomitsmorph/detail/fixed_string.hppandmorph/detail/quantity_equation.hpp, which public headers include, so the installed tree resolves in the build directory and nowhere else; install(EXPORT … NAMESPACE morph::)prefixes the target name, somorph_netarrives asmorph::morph_netwhile every in-tree alias saysmorph::net;- without
find_dependency(glaze)in the package config,morphTargets.cmakenames an imported target nobody created.
The checker also builds morph_verify_interface_header_sets, which is not part
of all and which nothing else in CI builds. That phase is what keeps
INTERFACE_HEADER_SETS_TO_VERIFY honest: the detail/ header set is
installed but deliberately outside the verified set, because
morph/detail/quantity_equation.hpp is included partway down
morph/util/quantity.hpp and is not self-contained. It is the one slow phase,
and --skip-header-set-verification drops it for the self-test's fast cases.
Two vacuity guards, because a consumability check that measures nothing looks
exactly like one that passes: the run fails if no optional component was
installed (leaving the exported-name check with nothing to read), and a
consumer asking for a component that was not installed must fail at
find_package rather than succeeding and breaking at link time.
Self-tested by scripts/test_check_install_export.sh, which reintroduces each
defect into a scratch copy of the tree one at a time and requires the checker
to catch each one for the stated reason — the checker is tested before it is
trusted.
Each rung's screen tests hold one case, <rung>_bundle_loads_without_warnings (bookmarks, bank, lims), in
examples/<rung>/screens/tests/. It builds the bundle the rung's server serves and loads every screen and app
document in it against the catalog facts the bundle carries, which include each action's effects and records
(examples/common/testkit/bundle_warnings.hpp). The case fails on any refusal and on any load-time warning
(Document::warnings): a mutation whose action declares no effects and after which nothing refreshes, a refresh
declaration the effects already cover, or a watch of a query that runs a mutation with effects. It is what keeps the
rungs' actions declaring their effects and their screens free of refresh declarations. It runs in the ordinary
ctest -L ladder sweep.
These are scope decisions about what this library is, not gaps.
- Crash / power-loss simulation. SQLite ships a storage engine of its own
and must survive an OS crash mid-write; morph does not own storage — its
offline queue (
include/morph/offline) delegates durability to SQLite itself (sqlite_offline_queue.hpp) or to plain file I/O (file_offline_queue.hpp), and a crash-durability claim about those belongs to SQLite's own test suite and the filesystem's fsync contract, not to a simulation morph would have to build and maintain. Declined as out-of-scope for what this library is, not as a gap. - A proprietary, dedicated coverage harness (SQLite's TH3). SQLite's
headline coverage figure is produced by TH3, which is not shipped with
SQLite and is not free to obtain. Citing that number as a model without
the harness that produces it would make the claim aspirational rather
than checkable. morph's coverage figures are produced entirely by
tools already in this repository (
scripts/coverage.sh,llvm-cov/llvm-profdata), so anyone can reproduce them. - 100% MC/DC as a stated target. This repository measures line and branch coverage, not MC/DC, and does not claim to. Adopting the number without the instrument that verifies it would be exactly the mistake the TH3 point above describes.
| Spec | Relationship |
|---|---|
| wire.md | decode, kMaxEnvelopeBytes, and the body double-parse / duplicate-key caveats fuzz_wire_decode and the adversarial run target. |
| backend.md | RemoteServer::handle (fuzz_dispatch_execute's target), switchBackend/cancelPending (the switch-backend soak test), the transports the load benchmark drives, and LimitPolicy/QtWebSocketServerConfig (the resource limits the adversarial run's default-config scenarios exercise). |
| offline/offline.md | NetworkMonitor/ReconnectCoordinator/SyncWorker, the pipeline test_soak_reconnect_churn.cpp drives through thousands of flaps. |
| security.md | The hardening tests (test_wire_hardening.cpp, test_server_limits.cpp, test_qt_websocket.cpp) these suites generalise from single-shot examples to distributions/time/a real adversary. |
| core/observability.md | The morph::observe metrics/trace seam already instruments the same dispatch paths these suites exercise; the soak tests deliberately instrument themselves directly rather than depend on a MetricSink being installed in the test process — see "Soak tests" above. |