feat(server): migrate to Falcon fiber stack with non-blocking caching and TLS hardening - #1139
Merged
Conversation
…2rss to master Task: task_01 Intent: stack on experiment/httpx-default, track html2rss master, add falcon, remove puma and rack-timeout
Task: task_02 Intent: remove Rack::Timeout configuration from Boot::Setup and adjust unit specs
…ru, delete config/puma.rb Task: task_03 Intent: add Falcon service configuration with dual-mode endpoint binding, clean up config.ru, remove Puma config
…-aware Task: task_04 Intent: update Dockerfile CMD, dev scripts, and TLS-aware docker-healthcheck for Falcon
…options Task: task_05 Intent: update README and docs architecture/guide for Falcon server and HTTPX stack
… dead PUMA_LOG_CONFIG
gildesmarais
force-pushed
the
feat/falcon-stack
branch
from
September 5, 2026 23:27
0344b46 to
8218039
Compare
…r-request fiber allocations In Falcon, each request executes in an ephemeral fiber. Using Thread.current caused a new Logger and LogDevice to be allocated on every request. Hoisting to a module singleton eliminates this allocation churn.
… async fiber reactor Rack::Cache used a file-based metastore and entitystore with synchronous flocking and disk I/O, which blocks the Async fiber reactor thread. Feed responses are already cached in memory by Feeds::Cache and downstream HTTP caching is governed by HttpCache headers.
…alescing in cache - Enforce a 64 KiB maximum request body size on POST /api/v1/feeds to prevent memory exhaustion attacks. - Add in-flight request coalescing (single-flight) to Feeds::Cache using Concurrent::IVar to prevent cache stampedes and duplicate scrapes against Botasaurus and upstream sources.
gildesmarais
changed the title
…ross all content types
…a description churn
- Release reload_mutex before executing app.call in DevelopmentReloader so requests process concurrently in development mode. - Add default BOTASAURUS_SCRAPER_URL to devcontainer docker compose.
Merged
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
0.57.0) and addedconfig/falcon.rbwith app preloading (preload { [FalconConfig.rackup_file] }), 55s default request timeout, and dual HTTP/1.1 and HTTP/2 endpoint support.OP_NO_COMPRESSIONon Falcon's SSLContext; hardenedbin/docker-healthcheckwith on-disk certificate validity inspection.Rack::Cacheand its synchronous disk I/O / file locks fromapp.rb, dependencies, and Docker setup, relying on in-memoryFeeds::Cacheand HTTP cache response headers.POST /api/v1/feedsto reject oversized payloads with 400 Bad Request, and added single-flight request coalescing (Concurrent::Map+Concurrent::IVar) inFeeds::Cacheto prevent cache stampedes.AppLoggerto a process-level singleton instance variable, eliminating per-requestLogger,LogDevice, andMonitorallocations under Fiber scheduling.DockerfileCMD tobundle exec falcon host ./config/falcon.rb, dev scripts (bin/dev,bin/dev-ruby,bin/dev-with-frontend), and regenerated OpenAPI schema / frontend types.Why
Puma's thread-per-request architecture and Rack::Cache's synchronous disk locks stall the event loop under heavy or concurrent scraping. Falcon provides an async fiber reactor compatible with HTTPX's non-blocking I/O. Removing Rack::Cache, capping request bodies, coalescing concurrent feed builds, and enforcing RFC 7540 AEAD ciphers ensure resilient, non-blocking operation at the edge and eliminate memory/CPU exhaustion vectors.
Risk
WEB_CONCURRENCY, default 2) in production; graceful fallback to plain HTTP when TLS certificates are omitted.Review map
config/falcon.rb— Falcon configuration, timeout alignment (55s), modern AEAD TLS cipher suite, and preload definition.app.rb&Gemfile— Removal ofRack::Cacheandrack-timeout.app/web/feeds/cache.rb— Single-flight coalescing withConcurrent::IVarto prevent thundering herds.app/web/api/v1/create_feed.rb— 64 KiB request body limit and 400 Bad Request envelope.app/web/telemetry/app_logger.rb— Process singleton logger replacing per-fiber allocations.bin/docker-healthcheck— TLS-aware healthcheck with certificate expiration validation.spec/config/falcon_config_spec.rb&spec/html2rss/web/feeds/cache_spec.rb— RSpec test coverage for Falcon configuration, timeouts, and cache coalescing.Validation
All checks executed inside the Dev Container (
devcontainer-app-1):make ready(RuboCop, Zeitwerk verification, YARD public docs check, ESLint/Prettier, full RSpec test suite — 354 examples, 0 failures, exit 0)make openapi-verify(public/openapi.yaml schema generation and generated TypeScript client verification — exit 0)