Skip to content

feat: recover data history and archive vintages - #2

Open
jonit-dev wants to merge 2 commits into
masterfrom
swarm/20260802T212729Z/data-history-vintage
Open

feat: recover data history and archive vintages#2
jonit-dev wants to merge 2 commits into
masterfrom
swarm/20260802T212729Z/data-history-vintage

Conversation

@jonit-dev

Copy link
Copy Markdown
Owner

Status

PARTIAL: the full data-history/vintage implementation and offline tests are present. Real ALFRED history, committed vintage archives, live OI growth, and live on-chain missingness proof are blocked because no FRED_API_KEY exists locally or in GitHub repository secrets.

Source PRD: docs/PRDs/2026-08-02/DATA_HISTORY_RECOVERY_AND_VINTAGE_ARCHIVE.md (authoritative local input for this swarm run).

Changes

  • Added deterministic merge-by-key retention/precedence, named environment errors, append-only NDJSON vintages, and vintage reconstruction.
  • Updated all seven data updaters to merge caches and append vintages; added ALFRED/FRED macro URL construction and source freshness caps.
  • Added bounded three-day on-chain forward-fill reasons and insufficient-data regime handling.
  • Wired FRED_API_KEY into the data-only forecast refresh workflow and added coverage/freshness tests.

Verification

  • yarn test: PASS, 27 files / 116 tests.
  • yarn lint: PASS.
  • yarn build: PASS; existing large-chunk warning only.
  • yarn build:features: PASS, 5,842 rows.
  • yarn backtest: PASS quality and robustness gates.
  • yarn validate:features: expected FAIL, futuresOpenInterestUSD 28/30.
  • yarn check:freshness: expected FAIL on stale baseline sources versus declared caps.
  • yarn validate:data: known baseline FAIL, 5,722 CoinMetrics MVRV mismatches.

No fabricated src/data/vintages records were committed. The workflow has no deployment step.

External blocker / merge condition

FRED_API_KEY is absent and gh secret list is empty. The exact unblock is:

export FRED_API_KEY='<real-key>'
yarn update:macro

Do not claim the real 2010+ macro/vintage acceptance or merge this PR as complete until the real updater run produces and verifies the archives, reconstruction, and growing OI history.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant