feat: Add the file-based override source by kinyoklion · Pull Request #453 · launchdarkly/ruby-server-sdk

kinyoklion · 2026-09-25T22:39:12Z

Summary

This PR is stacked on #452 because it supplies the override store that the earlier PRs add, and its client-level spec relies on the marked reasons and events they produce.

This change adds LaunchDarkly::Integrations::FileData.override_source, the file-based override source that the OVERRIDE specification defines. It is configured with LaunchDarkly::DataSystem::ConfigBuilder#overrides:

overrides = LaunchDarkly::Integrations::FileData.override_source(paths: ["/etc/launchdarkly/overrides.json"])
config = LaunchDarkly::Config.new(data_system: LaunchDarkly::DataSystem.default.overrides(overrides))

The source reads one or more JSON or YAML files in the file data source document format (flags, flagValues, segments) and supplies complete snapshots to the SDK's override store. Its options are paths (required, non-empty strings in precedence order; a path listed more than once is logged and the repeats are ignored; on Windows, where file names ignore case, paths that differ only in case count as the same path), duplicate_keys_handling (:fail, the default, or :ignore, which keeps the first file's entry), change_detection (:polling, the default, or :watching through the optional listen gem), and poll_interval (default 1 second, raised to the 1 second minimum with a warning). Invalid options raise ArgumentError when the client is created, the same way other invalid component configuration is reported. Unlike the Go reference, an unrecognized duplicate_keys_handling value and an unknown option key are rejected rather than defaulted, because the specification treats invalid configuration as a construction error and an emergency mechanism should fail loudly.

Behavior comes from the shared file data code added in #449:

The format comes from each file's name, as sdk-specs fix: Add observer as explicit dependency #272 recommends: a name that ends in .yaml or .yml, in any case, is YAML, and any other file, including one with no extension, is JSON. The existing file data sources read every file with the YAML parser, which also reads most JSON; the override source does not, so a YAML document in a .json file, or JSON with a # comment or a trailing comma, does not load. Files must be UTF-8; a UTF-16 file, as Windows PowerShell 5.1 writes with >, fails to load with a clear message and keeps the overrides in effect.
An empty JSON file is not a valid document: it fails to load and the overrides in effect stay. To remove a file's overrides, delete the file or write {}.
Override files can be rewritten in place. When the source finds a file whose overrides are in effect empty (or, for YAML, with only a comment or ---), it holds the file back and keeps the overrides in effect until the new content is written: until the file has stayed so, unwritten, for one retry delay (1 second), and for at most five retry delays while it keeps being rewritten. Changes to the other files wait with it, since a reload applies all files or none. The mechanism is described in feat: Add file loading code for flag overrides #449; the public docs say only what an application needs.
A configured file that does not exist contributes no overrides, so a file can be created later and deleting a file removes its overrides; a file that exists but cannot be read or parsed fails that reload and keeps the last good overrides, with a logged failure and a retry; a rewrite with identical content is not reapplied; the initial load completes before the client constructor returns.

A flagValues entry expands into a flag that is on and serves the value by fallthrough, so the evaluation reports the FALLTHROUGH reason kind. Every applied change is logged at Info with the overrides in effect and what each file supplied, and the reloader's log lines for this source, including the hold's debug lines, carry a FileOverrideSource prefix.

The override_source docs say how the format is chosen, that files must be UTF-8, that comments and repeated keys in JSON depend on the application's json gem, that files can be rewritten in place, how to remove a file's overrides, that an empty JSON file is not a valid document, and that replacing a YAML file in one operation avoids briefly applying a partly written file.

Testing

Specs for the builder (defaults, absolute paths, every invalid option, listen requirement, poll interval clamping), the initial load (merge, fallthrough-form expansion, YAML, missing file, all files missing, malformed file, duplicate keys under both handlings), the Info log format, polling change detection (change, appearance, deletion, malformed edit with recovery, identical rewrite, stop, stop before start), watching change detection (change, appearance, deletion), and a client-level check of serving and reloading overrides through LDClient.
The specs rewrite override files in place with real, deliberately slow writes. End-to-end specs check that the source never supplies a snapshot without the override, and that a client never serves the LaunchDarkly value, while a JSON or YAML file is rewritten in place with a 300 ms pause, with polling and watching change detection; also for a YAML file whose leading comment is written before the pause, for five rewrites about a second apart with the default timing, for a JSON file and a YAML file that stay empty, and for files that are UTF-16 or not valid UTF-8, at startup and at runtime.
The in-place rewrite handling was reviewed in eight rounds as fix: Keep overrides through an in-place rewrite of their files #479 (multi-agent and Bugbot reviews); see feat: Add file loading code for flag overrides #449 for the mutation check (67 mutants, 60 killed, 7 equivalent) and the loops (10 runs in a row on Ruby 3.2 and on JRuby 9.4, no failures).
The flag overrides suite of sdk-test-harness (3.2.0-alpha.13, built from #515, whose override files are rewritten in place and have no extension) passes against the test service of feat: Add flag overrides to the contract test service #454: 79 tests ran, all passed.
Full suite and RuboCop are clean, apart from can be automatically loaded by Bundler.require, which needs write access to the system gem directory on the test machine. Each new spec was checked against a deliberate defect in the code it covers.

The existing FDv1 and FDv2 file data sources keep their current behavior. This series does not change them; the override feature is additive.

SDK-3249

_{Opened from termag}

Note

Overview
Adds LaunchDarkly::Integrations::FileData.override_source, an experimental file-based override source wired through DataSystem::ConfigBuilder#overrides. Operators can force per-key flag and segment overrides from JSON/YAML files while the client still uses LaunchDarkly for everything else.

The new FileOverrideSource reuses the shared FileData::Reloader (polling or listen watching), treats missing paths as empty, skips unchanged byte-identical reloads, and pushes full snapshots to the override sink with Info logs listing what each file contributed. FileOverrideSourceBuilder validates paths and options at client construction (ArgumentError on bad config).

FileData::Reloader gains a configurable log_prefix (default [LDClient]) on all its log lines and logs unexpected errors directly instead of Util.log_exception, so override reload logs are prefixed [LDClient] FileOverrideSource:.

Docs on FileData describe the override source and how it differs from the all-local file data source. Large new integration specs cover builder validation, reload behavior, in-place rewrites, encoding edge cases, and LDClient evaluation with overrides.

^{Reviewed by Cursor Bugbot for commit 704e7bd. Bugbot is set up for automated code reviews on this repo. Configure here.}

kinyoklion · 2026-10-03T01:23:53Z

bugbot review

cursor