diff --git a/.dz/briefs/health.md b/.dz/briefs/health.md index 181dacb..5a7b07f 100644 --- a/.dz/briefs/health.md +++ b/.dz/briefs/health.md @@ -9,7 +9,7 @@ maintained so that nobody has to watch it. This covers the failures no issue or | the last release published | `gh run list --workflow release.yml --limit 1 --json conclusion,displayTitle` | `success` (or no run yet) | | RubyGems serves `Ruby2html::VERSION` | `gem list ruby2html --remote --exact` vs `lib/gem/ruby2html/version.rb` | equal, or a release run is in flight | | benchmarks still run | `gh run list --workflow benchmark.yml --branch main --limit 1 --json conclusion` | `success` | -| Dependabot is not stuck | `gh pr list --app dependabot --json number,createdAt` | none older than 7 days (updates wait a 14-day cooldown before they open) | +| Dependabot is not stuck | `gh pr list --app dependabot --json number,createdAt` | none older than 7 days (app dependencies wait out the 14-day cooldown in `.github/dependabot.yml`; the dev-tools group is exempt via `cooldown.exclude` and opens within days) | | canary ran | `gh run list --workflow canary.yml --limit 1 --json conclusion,createdAt` | a run in the last 35 days; its failures are the monthly `improve` checkup's job, not an issue | For each unhealthy signal, file one issue unless an open issue already names it. The title names the diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 4761cc9..5689611 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -8,6 +8,11 @@ version: 2 # # cooldown: a release must be 14 days old before it is proposed, so broken or # compromised releases are usually yanked first. Security updates skip it. +# The dev-tools group (linters, scanners, test tooling) is exempt from the +# bundler cooldown via cooldown.exclude: Dependabot has no per-group cooldown +# key, so exclude is the only lever. A stale linter or scanner is itself a CI +# risk, so those bumps are proposed within days instead of two weeks. The +# exclude list mirrors the dev-tools group patterns below - keep both in sync. updates: - package-ecosystem: bundler directory: / @@ -16,6 +21,15 @@ updates: day: monday cooldown: default-days: 14 + exclude: + - 'rubocop*' + - 'brakeman' + - 'lefthook' + - 'rake*' + - 'benchmark-ips' + - 'capybara' + - 'faker' + - 'pry*' open-pull-requests-limit: 10 groups: rails: diff --git a/AGENTS.md b/AGENTS.md index b7b6d7f..1e38cfe 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -121,7 +121,7 @@ Goal: the repo needs no hand maintenance except major-version dependency upgrade | What | How | |---|---| | Maintainer agent | Repo managed by [developerz.ai](https://developerz.ai) (policy: `.dz/maintainer/maintainer.yml`): triages issues, reviews and merges every PR (contributors, members, bots) once its author marks it ready (out of draft), it is reviewed and `CI OK` is green; daily `health` + weekday `release-due` checkups and a monthly `improve` pass (`.dz/briefs/`: canary fixes, dependency majors, speed with benchmark evidence, correctness, docs drift); opens release PRs; breaking changes escalate to the owner. Sibling projects such as [wurk](https://github.com/sebyx07/wurk) run the same model | -| Dependencies | Dependabot (`.github/dependabot.yml`) opens grouped weekly PRs after a 14-day cooldown (security updates skip it); the agent reviews and merges on green — the only merger | +| Dependencies | Dependabot (`.github/dependabot.yml`) opens grouped weekly PRs: app dependencies wait out a 14-day cooldown (security updates skip it), while the dev-tools group is exempt from it (`cooldown.exclude`) and is proposed within days; the agent reviews and merges on green — the only merger | | CI | `.github/workflows/ci.yml`; branch protection requires only the aggregate `CI OK` job: RuboCop, bundle-audit, full suite + Brakeman, gem specs across the Ruby matrix (generated from maintained CRuby versions), native builds on Linux/macOS arm64, `-Werror` with gcc and clang, ASAN/UBSAN (`use_sigaltstack=0`), pure-Ruby fallback, JRuby (Java escaper jar + `-java` gem) and TruffleRuby (pure Ruby) via `gemfiles/standalone.gemfile`, Rails integration per `gemfiles/rails.gemfile`, `bin/gem-smoke` packaging on plain RubyGems. Every job required; no `continue-on-error` | | Canary | `.github/workflows/canary.yml`, monthly on the 1st, not gating: Ruby head and Rails edge surface upstream breaks before a release ships them; the monthly `improve` checkup fixes what it finds | | Benchmarks | `.github/workflows/benchmark.yml`, non-gating: every push to `main` and PR renders a table into the job summary and uploads JSON; PRs get a sticky comment with the delta vs `main`. Set repo variable `BENCH_RUNNER` for a stable self-hosted runner; README holds only a dated release snapshot | diff --git a/bin/brakeman b/bin/brakeman index ace1c9b..3db33ff 100755 --- a/bin/brakeman +++ b/bin/brakeman @@ -1,7 +1,59 @@ #!/usr/bin/env ruby -require "rubygems" -require "bundler/setup" +# frozen_string_literal: true -ARGV.unshift("--ensure-latest") +require 'rubygems' +require 'bundler/setup' -load Gem.bin_path("brakeman", "brakeman") +# The newest brakeman released upstream, or nil when rubygems.org cannot be +# reached (offline, proxy, ...) - the scan runs either way. +def latest_brakeman_version + Gem.latest_spec_for('brakeman').version +rescue StandardError + nil +end + +# The upstream-version check is opt-in (RUBY2HTML_CHECK_LATEST=1): it is a +# live rubygems.org round trip, and the scan must start without one - a CI +# job or an offline machine must not block (or hang until the fetcher +# timeout) before the scan. +def latest_check_requested? + !ENV.fetch('RUBY2HTML_CHECK_LATEST', '').empty? +end + +# Scan with the brakeman locked in Gemfile.lock. +# +# Gemfile.lock lists brakeman as a bare dependency (no version requirement), so +# the lock file is the only constraint there is. Resolve the locked spec +# explicitly instead of Gem.bin_path, which picks up whichever brakeman happens +# to be installed on the machine. +brakeman_spec = Gem.loaded_specs['brakeman'] +abort('brakeman is not in the bundle - run `bundle install`') if brakeman_spec.nil? + +brakeman_executable = File.join(brakeman_spec.bin_dir, brakeman_spec.executables.first) +abort("brakeman executable missing: #{brakeman_executable}") unless File.exist?(brakeman_executable) + +# Never re-add brakeman's "Fail when Brakeman is outdated" option (option key +# ensure_latest) here - its CLI flag is deliberately not spelled out in this +# file, so a grep for it finds no injection point. +# +# Brakeman runs that check BEFORE the scan (Brakeman::Commandline.run calls +# check_latest and only then run_report), so a newer upstream release makes it +# exit with Not_Latest_Version_Exit_Code (5) and the scan never runs at all: +# CI reports "not the latest version" instead of the scan result. A stale but +# valid lock must never turn CI red, and version skew must never mask a red +# scan. +# +# A newer upstream brakeman is reported on stderr only, below - and only when +# the check is requested (RUBY2HTML_CHECK_LATEST=1), because the lookup is a +# live rubygems.org round trip the scan must not wait on (see +# latest_check_requested?). The exit status stays brakeman's own: no warning +# flag is passed, so brakeman's default applies and findings still fail the +# build (Warnings_Found_Exit_Code). +if latest_check_requested? + latest_version = latest_brakeman_version + if latest_version && latest_version > brakeman_spec.version + warn "brakeman #{brakeman_spec.version} is behind the latest release #{latest_version} - the scan ran against the locked version." + end +end + +load brakeman_executable diff --git a/spec/bin/brakeman_spec.rb b/spec/bin/brakeman_spec.rb new file mode 100644 index 0000000..64d4cfd --- /dev/null +++ b/spec/bin/brakeman_spec.rb @@ -0,0 +1,245 @@ +# frozen_string_literal: true + +require 'fileutils' +require 'open3' +require 'tmpdir' + +RSpec.describe 'bin/brakeman' do + let(:script_path) { File.expand_path('../../bin/brakeman', __dir__) } + # The version the bundle actually locks (Gemfile.lock), not a literal: this + # spec cannot silently drift from the brakeman CI scans with. + let(:locked_version) { Gem.loaded_specs['brakeman'].version.to_s } + let(:upstream_version) { '99.0.0' } + + # bin/brakeman runs as a subprocess in a hermetic gem environment: + # bundler/setup is stubbed out, the locked brakeman spec points at a fake + # executable and Gem.latest_spec_for is stubbed, so nothing hits the network + # (an unstubbed lookup raises the way rubygems.org-unreachable does, and is + # logged so the specs can tell whether the script consulted it at all). + # The fake executable records the arguments it was called with and exits + # with the requested status, which keeps the script's own decisions + # observable: which executable ran, what it was told and how the script + # exited. + def run_brakeman(args: [], latest: 'same', mode: 'normal', exit_code: 0, check_latest: false) + Dir.mktmpdir do |dir| + bin_dir = File.join(dir, 'locked', 'bin') + empty_bin_dir = File.join(dir, 'locked', 'empty-bin') + stub_dir = File.join(dir, 'stub') + log = File.join(dir, 'brakeman.log') + lookup_log = File.join(dir, 'lookup.log') + decoy_log = File.join(dir, 'decoy.log') + decoy = File.join(dir, 'decoy', 'brakeman') + prelude = File.join(dir, 'prelude.rb') + + FileUtils.mkdir_p([bin_dir, empty_bin_dir, File.join(stub_dir, 'bundler'), File.dirname(decoy)]) + File.write(File.join(bin_dir, 'brakeman'), fake_brakeman_source) + File.write(decoy, decoy_source) + File.write(File.join(stub_dir, 'bundler', 'setup.rb'), stub_bundler_setup) + File.write(prelude, prelude_source) + + env = { + 'RUBYOPT' => nil, + 'RUBYLIB' => stub_dir, + 'R2H_BIN_DIR' => bin_dir, + 'R2H_DECOY' => decoy, + 'R2H_DECOY_LOG' => decoy_log, + 'R2H_EMPTY_BIN_DIR' => empty_bin_dir, + 'R2H_EXIT' => exit_code.to_s, + 'R2H_LATEST' => latest, + 'R2H_LOG' => log, + 'R2H_LOOKUP_LOG' => lookup_log, + 'R2H_MODE' => mode, + 'R2H_UPSTREAM' => upstream_version, + 'R2H_VERSION' => locked_version, + 'RUBY2HTML_CHECK_LATEST' => check_latest ? '1' : nil + } + + stdout, stderr, status = Open3.capture3(env, RbConfig.ruby, '-r', prelude, script_path, *args) + + [ + stdout, + stderr, + status, + File.exist?(log) ? File.read(log) : nil, + File.exist?(decoy_log) ? File.read(decoy_log) : nil, + File.exist?(lookup_log) ? File.read(lookup_log) : nil + ] + end + end + + def stub_bundler_setup + <<~'RUBY' + # frozen_string_literal: true + # bundler/setup is stubbed: the fake gem environment is enough here. + RUBY + end + + # The fake gem environment: the locked brakeman spec, a stubbed + # Gem.latest_spec_for (newer, same or unreachable; unstubbed calls raise + # instead of reaching rubygems.org) and a stubbed + # Gem.bin_path that points at a decoy executable, so the script cannot + # silently fall back to whatever brakeman happens to be installed. + def prelude_source + <<~'RUBY' + # frozen_string_literal: true + + require 'rubygems' + require 'rubygems/remote_fetcher' + + # Stands in for the locked Gem::Specification of brakeman. + class FakeBrakemanSpec + attr_reader :version, :bin_dir, :executables, :name + + def initialize(version, bin_dir) + @version = Gem::Version.new(version) + @bin_dir = bin_dir + @executables = ['brakeman'] + @name = 'brakeman' + end + end + + # Replaces the two RubyGems lookups the script relies on. Every + # latest_spec_for call is logged, so a spec can tell whether the script + # consulted rubygems.org at all; an unstubbed call raises, keeping the + # subprocess off the network. + module FakeGemEnvironment + def latest_spec_for(name, *args, **kwargs, &block) + File.open(ENV.fetch('R2H_LOOKUP_LOG'), 'a') { |file| file.puts(name.to_s) } + mode = ENV.fetch('R2H_LATEST', '') + if mode.empty? || mode == 'offline' + raise Gem::RemoteFetcher::FetchError.new('rubygems.org is unreachable', 'https://rubygems.org') + end + + version = mode == 'same' ? ENV.fetch('R2H_VERSION') : ENV.fetch('R2H_UPSTREAM') + Struct.new(:version).new(Gem::Version.new(version)) + end + + def bin_path(name, *args, **kwargs) + decoy = ENV['R2H_DECOY'] + return super if decoy.nil? || decoy.empty? + + decoy + end + end + + Gem.singleton_class.prepend(FakeGemEnvironment) + + if ENV['R2H_MODE'] == 'missing_gem' + Gem.loaded_specs.delete('brakeman') + else + bin_dir = ENV.fetch('R2H_BIN_DIR') + bin_dir = ENV.fetch('R2H_EMPTY_BIN_DIR') if ENV['R2H_MODE'] == 'missing_executable' + Gem.loaded_specs['brakeman'] = FakeBrakemanSpec.new(ENV.fetch('R2H_VERSION'), bin_dir) + end + RUBY + end + + # Stands in for the brakeman executable of the locked gem spec: it records + # its arguments, refuses the outdated-version option the way brakeman does + # (exit code 5) and exits with the requested status otherwise. + def fake_brakeman_source + <<~'RUBY' + #!/usr/bin/env ruby + # frozen_string_literal: true + + File.open(ENV.fetch('R2H_LOG'), 'a') { |file| file.puts(ARGV.join(' ')) } + + if ARGV.any? { |arg| arg.include?('ensure-latest') } + warn 'You are not running the latest version of Brakeman' + exit 5 + end + + puts 'fake brakeman: scan complete' + exit Integer(ENV.fetch('R2H_EXIT', '0')) + RUBY + end + + # The executable a Gem.bin_path lookup would return: it must never run. + def decoy_source + <<~'RUBY' + #!/usr/bin/env ruby + # frozen_string_literal: true + + File.open(ENV.fetch('R2H_DECOY_LOG'), 'a') { |file| file.puts('decoy') } + exit 99 + RUBY + end + + it 'runs the executable of the locked brakeman gem spec' do + stdout, _stderr, status, log, decoy_log, lookup_log = run_brakeman(args: ['--no-pager']) + + expect(status).to be_success + expect(stdout).to include('fake brakeman: scan complete') + expect(log).to include('--no-pager') + expect(decoy_log).to be_nil + # Opt-in upstream check: the scan starts without a rubygems.org round trip. + expect(lookup_log).to be_nil + end + + it 'consults rubygems.org only when RUBY2HTML_CHECK_LATEST is set' do + stdout, stderr, status, log, _decoy_log, lookup_log = run_brakeman(args: ['--no-pager'], check_latest: true) + + expect(status).to be_success + expect(lookup_log).to include('brakeman') + # Same version upstream: nothing to warn about, exit status untouched. + expect(stderr).to be_empty + expect(stdout).to include('fake brakeman: scan complete') + expect(log).to include('--no-pager') + end + + it 'reports a newer upstream brakeman on stderr without changing the exit status' do + stdout, stderr, status, log = run_brakeman(args: ['--no-pager'], latest: 'newer', check_latest: true) + + expect(status).to be_success + expect(status.exitstatus).not_to eq(5) + expect(stderr).to include(locked_version) + expect(stderr).to include(upstream_version) + expect(stderr).not_to include('not running the latest version') + expect(stdout).to include('fake brakeman: scan complete') + expect(log).to include('--no-pager') + end + + it 'never passes brakeman the ensure-latest option' do + _stdout, stderr, status, log = run_brakeman(args: ['--no-pager'], latest: 'newer', check_latest: true) + + expect(status).to be_success + expect(log).not_to include('ensure-latest') + expect(stderr).not_to include('not running the latest version') + expect(File.read(script_path)).not_to include('ensure-latest') + end + + it 'keeps brakeman warnings fatal' do + _stdout, _stderr, status, log = run_brakeman(args: ['--no-pager'], exit_code: 3) + + expect(status.exitstatus).to eq(3) + expect(log).to include('--no-pager') + expect(log).not_to include('exit-on-warn') + end + + it 'runs the scan when rubygems.org cannot be reached' do + stdout, stderr, status, log = run_brakeman(args: ['--no-pager'], latest: 'offline', check_latest: true) + + expect(status).to be_success + expect(stdout).to include('fake brakeman: scan complete') + expect(log).to include('--no-pager') + expect(stderr).not_to include(upstream_version) + end + + it 'aborts when brakeman is missing from the bundle' do + stdout, stderr, status, log = run_brakeman(mode: 'missing_gem') + + expect(status.exitstatus).to eq(1) + expect(stderr).to match(/brakeman.*not in the bundle/) + expect(stdout).to be_empty + expect(log).to be_nil + end + + it 'aborts when the locked brakeman executable is missing' do + stdout, stderr, status, log = run_brakeman(mode: 'missing_executable') + + expect(status.exitstatus).to eq(1) + expect(stderr).to match(/brakeman executable missing/) + expect(stdout).to be_empty + expect(log).to be_nil + end +end