json-tls coverage for flagship/emerging entries on the H1 table - #1273
Open
MDA2AV wants to merge 15 commits into
Open
json-tls coverage for flagship/emerging entries on the H1 table#1273MDA2AV wants to merge 15 commits into
MDA2AV wants to merge 15 commits into
Conversation
Each entry serves the JSON workload but had no TLS listener, so json-tls
scored 0. All four now bind 8081 with the same app that serves 8080:
koa app.callback() behind node:https
hono-node the same app.fetch, via the adapter's createServer option
node-h3 srvx takes the PEM paths and builds the https server itself
fastify Fastify binds one instance per server, so routes moved into
build(); both ports register from that one definition
The certs are only mounted for the TLS profiles, so every listener is
guarded on their presence and stays closed otherwise.
validate: koa 37/0, fastify 47/0, hono-node 37/0, node-h3 37/0
Both serve the JSON workload with no TLS listener, so json-tls scored 0.
rocket enables its own tls feature and configures a second instance on
8081. Routes moved into build() so the two ports mount the same
set; both configs start from Config::figment(), which keeps the
ROCKET_* env from the Dockerfile applying to each.
axum serves the same Router over axum-server's rustls acceptor, the
one axum's own tls-rustls example uses. ring rather than
aws-lc-rs, so the build image needs no C toolchain.
Guarded on the PEMs, which the harness only mounts for the TLS profiles.
validate: rocket 34/0, axum 37/0
plug-cowboy a second Plug.Cowboy listener in front of the same router.
Plug.Cowboy derives its ref from plug + scheme, so the two
children do not collide.
phoenix-bandit the endpoint's https: config already holds 8443 for the h2
profiles and Phoenix binds one https listener per endpoint,
so json-tls gets a second Bandit listener in front of the
same endpoint plug -- the identical pipeline, not a copy.
Both guarded on the PEMs, which the harness only mounts for TLS profiles.
validate: plug-cowboy 37/0, phoenix-bandit 70/0 (the latter needs #1272,
which fixes a probe bug that fails any container running as non-root)
MDA2AV
force-pushed
the
feat/json-tls-coverage
branch
from
August 22, 2026 17:28
aaad428 to
5fd521b
Compare
Both Jetty adapters take a KeyStore instance for :keystore, so the mounted PEMs are converted in-process and merged into the existing run-jetty opts as :ssl? / :ssl-port 8081 -- one server, same handler on both ports. The conversion uses CertificateFactory + KeyFactory rather than java.security.PEMDecoder: PEMDecoder is still a preview API on the JDK 25 in the clojure base image, so it would need --enable-preview to load. Verified against each adapter's source rather than assumed: ring sets .setKeyStore for a non-string :keystore and builds its SslConnectionFactory with "http/1.1", and sunng's does the same with h2? defaulting off. http-kit is deliberately absent -- 2.9.0-beta4's run-server has no TLS support at all (no ssl/tls anywhere in server.clj, none in its documented options), so it cannot serve json-tls without a hand-rolled TLS layer. validate: ring 52/0, ring-jetty9-adapter 46/0
http4k an https listener on the same Undertow builder, so 8081 runs the
identical handler chain as 8080
http4s Ember binds one port per builder, so server() is the single
definition and the two resources run parTupled
micronaut dual-protocol with ssl.port 8081; the listener stays Micronaut's,
only the key material is prepared before it starts
All three want a KeyStore/SSLContext and the harness mounts PEMs, so each
converts the pair in-process with CertificateFactory + KeyFactory rather
than java.security.PEMDecoder, which is still a preview API on these JDKs.
validate: http4k 37/0, http4s 37/0, micronaut 37/0
tools-deps writes .cpcache into the framework directory on any classpath resolution, including from inside a container where the files land root owned. Two of them were picked up by a git add -A.
nestjs the adapter's instance is the very Express app Nest wired the
controllers and compression onto, so putting it behind node:https
serves the same pipeline rather than a second copy.
sisk Cadente applies the certificate to the whole listening host rather
than per port -- setting SslOptions, or calling UseSsl, on the
plaintext host turns 8080 into HTTPS and it starts 301-ing plaintext
to https. Verified by running it: the log lists both ports as
https:// and curl gets a redirect. Two ListeningHosts do not work
either, since a Router binds to one host and sharing an instance
leaves both serving nothing. So TLS runs as its own server with its
own Router, and BuildRouter() is now the single definition both are
built from.
validate: sisk 47/0, nestjs 32 passed / 5 failed.
The nestjs failures are the POST TCP-fragmentation checks and are
pre-existing: main without this change fails the same five (29/5, the
three extra passes here being the json-tls checks). Not addressed here.
slim runs on FrankenPHP, so json-tls is an 8081 site block with the same php_server as 8080 -- TLS terminated by the server the entry already uses. The site cannot be declared unconditionally: Caddy refuses to start when a tls directive points at files that are not there, and the harness mounts /certs only for the TLS profiles. Caught by running it -- the container exited outright with no certs mounted, which would have taken out every non-TLS profile. So the block ships as a separate file the entrypoint drops into an imported directory only when the certificate exists. Verified both ways: 8080 serves with no certs mounted, both ports serve with them. sanic keeps no json-tls, but the comment now records what was actually found: 25.3.0 binds and accepts on a TLS listener and then never sends a ServerHello, at any worker count, whether it is the only listener or a second prepare(). A second plaintext prepare() answers fine, so it is TLS specifically, and single_process=True serves it -- which would pin the entry to one core. A prebuilt SSLContext is not a way around it either, since the worker manager spawns and an SSLContext cannot be pickled. validate: slim 37/0
A second Hyperf HTTP server on 8081 behind Swoole's own TLS. Three things the obvious version gets wrong, each found by running it: - Swoole port options go under 'settings'; 'options' is Hyperf's own and the ssl_cert_file/ssl_key_file put there never reach the port, which aborts startup with "require ssl_cert_file and ssl_key_file options". - Hyperf scopes routes to a server by name, so the TLS server needs the same set registered against it. routes.php now registers one closure twice. - Hyperf keys the ON_REQUEST callback by class, so pointing both servers at Hyperf\HttpServer\Server makes the second replace the first -- it logs "http will be replaced by http-tls" and then neither port answers. The TLS server gets its own Server subclass so it has its own router. The listener is only added when the certificate is mounted, which the harness does for the TLS profiles only; Swoole aborts at startup if a listener names certificate files that are not there. validate: hyperf 53/0
Contributor
|
👋 Heads up! This PR modifies the following frameworks:
|
Octane's Caddyfile carries a {$CADDY_EXTRA_CONFIG} placeholder that Octane
itself never sets, so Caddy resolves it from the process environment. The
json-tls site goes there -- same worker, same app as 8080, only the listener
differs -- rather than forking Octane's Caddyfile through --caddyfile.
Two things worth recording, both found by running it:
- The site cannot be declared unconditionally. Caddy refuses to start when a
tls directive points at files that are not there, and the harness mounts
/certs only for the TLS profiles, so the entrypoint sets the variable only
when the certificate exists. Verified 8080 still serves with none mounted.
- root * "{$APP_PUBLIC_PATH}" does not work inside the extra config. Octane
substitutes that placeholder into its own template rather than exporting
it, so in the extra config it resolves empty: TLS handshakes fine and then
the request hangs with no response at all. The path is written out.
validate: laravel 37/0
drogon addListener on 8081 with TLS, same handlers as 8080 mojolicious a second listen URL on the same Mojo::Server::Prefork lapis an ssl server block running the same lapis app All three abort at startup on a tls/ssl directive naming certificate files that are not there, and the harness mounts /certs only for the TLS profiles, so each is conditional -- drogon and mojolicious check in code, and nginx cannot, so lapis's block ships as a file the entrypoint drops into an included directory. Each verified running both ways: 8080 serves with no certs mounted, both ports serve with them. validate: drogon 37/0, lapis 37/0, mojolicious 37/0
An 8081 listener inside the existing cert-conditional block. 8443 advertises h2 through ALPN, so it cannot double as the HTTP/1.1 listener. Also adds libbrotli1 to the runtime image. The build stage installs libbrotli-dev, so h2o links against brotli, but the runtime stage only carried libssl3 and jq -- the container died on startup with "error while loading shared libraries: libbrotlidec.so.1". Pre-existing and unrelated to json-tls: the entry builds h2o from a --depth 1 clone of upstream master, so it picks up whatever that links against today. validate: h2o-mruby 53/0 (the clang build of mruby SIGSEGVs at random on this machine and needed several attempts -- see the toolchain note; the tree itself builds clean)
warp the same handler behind warp-tls on a forked thread; Debian
ships libghc-warp-tls-dev, so it is one more build dep
humming-bird a second Application over the same router, using
Hummingbird's own TLS server. One Application binds one
address, so the TLS listener cannot share the first. ALPN is
pinned to http/1.1 so json-tls does not fall into h2.
Both guarded on the PEMs, which the harness only mounts for TLS profiles.
validate: warp 37/0; humming-bird 46 passed / 1 failed.
The humming-bird failure is the static staleness probe and is pre-existing:
the entry loads every static file into an in-memory [String: StaticFile] map
at startup, .br and .gz buffers included, so it never revalidates against
disk. That is a cache assembled in the entry, which both the standard and
tuned rules exclude, and it fails the same way on main. Untouched here --
this change only adds a listener -- but it needs its own fix.
slimeweb's serve() makes its own listener TLS rather than adding one, so the second port needs a second serve(). It cannot share the process: asyncpg pools are bound to the loop that created them, and @app.start() runs per serve, so the second pool lands on the other loop and /async-db dies with "got Future attached to a different loop". Measured, not guessed -- the threaded version validated 51 passed / 6 failed against a 51/0 baseline. So the TLS listener runs as its own process, started without DATABASE_URL so it creates no pool at all. 8081 only carries json-tls, which never touches the database, and the connection budget is untouched -- the harness runs Postgres with max_connections=256 and this pool already asks for that many. robyn keeps no json-tls: 0.83.0 has no TLS support at all. start() takes host, port, _check_port, client_timeout and keep_alive_timeout, and there is no ssl/certificate/keyfile reference anywhere in the package. validate: slimeweb 54/0 (51 baseline + the three json-tls checks)
An :8081 site alongside the existing :8443, same worker as :8080, with the ALPN pinned to http/1.1 so json-tls does not fall into h2. /certs is mounted for every profile (scripts/lib/framework.sh), which is why :8443 is already unconditional and this can be too. Also adds libwebpdemux2 and libwebpmux3. The frankenphp binary in trueasync/php-true-async:latest-frankenphp links libwebp but the image does not carry it, so the container exited at startup with "error while loading shared libraries: libwebpdemux.so.2" -- the entry could not run at all. Found the full set with ldd on the binary rather than one failure at a time. The tag is :latest, so this arrives with whatever upstream last pushed. validate: 55 passed / 1 failed. The failure is the static staleness probe, and it is pre-existing: worker.php scandirs /data/static at startup and file_get_contents each file, .br and .gz included, into $staticFiles. That is a cache assembled in the entry, which the rules exclude, and it never revalidates against disk. Untouched here. symfony-spawn-franken is deliberately absent: it does not build on main either -- composer exits 127 in the same base image -- so its json-tls could not be validated.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
32 flagship/emerging entries on the H1 leaderboard have no
json-tlsnumber. This subscribes 27 of them; the other five cannot, each for a reason established by running it.Every entry serves the same app on 8081 that it serves on 8080 — no second copy of the routes — and every listener is guarded on the PEMs where the harness mounts them conditionally.
Implemented
koaapp.callback()behindnode:httpshono-nodeapp.fetch, via the adapter'screateServernode-h3fastifybuild(); both ports register from itnestjsrockettlsfeature, second instance offConfig::figment()axumRouterover axum-server's rustls acceptor; ring, so no C toolchainwarpdrogonaddListeneron 8081 with TLSplug-cowboyPlug.Cowboylistener on the same routerphoenix-banditring,ring-jetty9-adapterKeyStoreinstance, merged into the existingrun-jettyoptshttp4khttp4sserver()is the single definition and the two runparTupledmicronautssl.port8081humming-birdApplicationover the same router, ALPN pinned to http/1.1siskBuildRouter()is the single definitionslim,laravel,frankenphp-trueasynchyperflapismojoliciousMojo::Server::Preforkh2o-mrubyslimewebserve()as its own process — pools are loop-affinephpCannot have json-tls
http-kit— 2.9.0-beta4 has no TLS at all. Nossl/tlsanywhere inserver.clj, none in its documented options.robyn— 0.83.0 likewise.start()takes host, port,_check_port, and two timeouts; no ssl/certificate/keyfile reference in the package.sanic— 25.3.0 binds and accepts a TLS listener then never sends a ServerHello, at any worker count. A second plaintext listener answers fine, so it is TLS specifically; onlysingle_process=Trueserves it, which would pin the entry to one core. A prebuiltSSLContextis no way round it either — the manager spawns andSSLContextcannot be pickled.symfony-spawn-franken— does not build onmain:composerexits 127 in its base image. Its json-tls could not be validated, so it is left out rather than shipped unverified.rage— skipped at the maintainer's request.Each reason is recorded in the entry itself, so nobody re-derives it.
Two runtime fixes picked up on the way
Both are pre-existing breakage that stopped the entry from starting at all, found while getting it to a validating state:
h2o-mruby— the build stage installslibbrotli-devso h2o links brotli, but the runtime image carried onlylibssl3andjq; it died onlibbrotlidec.so.1.frankenphp-trueasync— the frankenphp binary links libwebp, which its base image does not carry; it died onlibwebpdemux.so.2. Found the full set withlddrather than one failure at a time.Both entries clone/pull
:latestupstream, so this arrives with whatever was last pushed.validate
All 27 pass, with two exceptions that are pre-existing and untouched here — both are entries that assemble a static cache in the entry and never revalidate against disk, which the staleness probe now catches:
humming-bird46/1 — loads every static file into a[String: StaticFile]map at startupfrankenphp-trueasync55/1 —worker.phpscandirs/data/staticandfile_get_contentseach fileBoth fail the same way on
main; each needs its own fix.nestjsalso carries 5 pre-existing POST fragmentation failures (mainfails the identical five).Benchmarking is a separate PR once this lands.
🤖 Generated with Claude Code