Follow-up to #418 (Determinism Tier 1), which this gap escaped.
Summary
format_results_markdown in
packages/darnit/src/darnit/tools/audit.py embeds a wall-clock stamp
in every report:
**Generated At:** 2026-09-08T15:47:47.180106Z
Consequence: a markdown audit report can never be byte-for-byte
reproducible, even for an identical commit with an identical verdict
set. Two runs one second apart produce different bytes.
A second, smaller instance: pass_history[].duration_ms is
load-dependent. It reads 0 for fast file-existence checks in practice,
but nothing guarantees that.
How it surfaced
Feature 036's SC-002 required capturing a pre-feature output baseline
and diffing post-feature output against it byte-for-byte. Two
consecutive captures of the same commit did not match. Both fields had
to be scrubbed for the baseline to be stable at all -- see the scrub()
helper in tests/darnit/fixtures/error_class_baseline/capture_baseline.py
and its docstring, which documents both as out of scope for 036 and
points here.
#418's survey covered filesystem iteration order, wall-clock injection
into remediation templates, list ordering, and non-atomic writes. It
did not look at the report formatter, which is why this was missed.
Why it matters
Same argument #418 makes: a compliance tool whose report output drifts
run-to-run undermines the "same commit -> same result" premise that
makes an audit trustworthy. Concretely it also:
- Blocks any golden-file regression test of markdown output. Any future
feature wanting to prove "my change is output-neutral" has to
reinvent the same scrubbing.
- Makes report diffs noisy in review when an operator commits reports
to a repo (some fleet operators do).
Scope
Decide between:
- Drop the field. Simplest. The information is recoverable from
the attestation timestamp or the file mtime for anyone who needs it.
- Make it opt-in via a formatter argument, default off.
- Make it injectable -- feature 036 already established this
pattern for exactly this problem: RemediationExecutor gained a
now_provider kwarg (defaulting to datetime.now) so tests can
pin a fixed clock. Applying the same shape to
format_results_markdown keeps production behavior while making
the output testable.
Recommend (3) for consistency with the precedent, or (1) if nobody is
actually consuming the field.
Also decide whether duration_ms should be omitted from serialized
pass_history (it is diagnostic, not evidence) or kept and documented
as non-reproducible.
Non-goals
- Re-auditing the other formatters. JSON and SARIF do not embed
timestamps at the per-result level; the attestation predicate has its
own timestamp, which is legitimate and should stay.
- Any change to what the audit concludes. This is purely about output
byte-stability.
Constitution alignment
- II. Conservative-by-Default: reproducible reports are a
precondition for the report being usable as evidence.
Related
Follow-up to #418 (Determinism Tier 1), which this gap escaped.
Summary
format_results_markdowninpackages/darnit/src/darnit/tools/audit.pyembeds a wall-clock stampin every report:
Consequence: a markdown audit report can never be byte-for-byte
reproducible, even for an identical commit with an identical verdict
set. Two runs one second apart produce different bytes.
A second, smaller instance:
pass_history[].duration_msisload-dependent. It reads 0 for fast file-existence checks in practice,
but nothing guarantees that.
How it surfaced
Feature 036's SC-002 required capturing a pre-feature output baseline
and diffing post-feature output against it byte-for-byte. Two
consecutive captures of the same commit did not match. Both fields had
to be scrubbed for the baseline to be stable at all -- see the
scrub()helper in
tests/darnit/fixtures/error_class_baseline/capture_baseline.pyand its docstring, which documents both as out of scope for 036 and
points here.
#418's survey covered filesystem iteration order, wall-clock injection
into remediation templates, list ordering, and non-atomic writes. It
did not look at the report formatter, which is why this was missed.
Why it matters
Same argument #418 makes: a compliance tool whose report output drifts
run-to-run undermines the "same commit -> same result" premise that
makes an audit trustworthy. Concretely it also:
feature wanting to prove "my change is output-neutral" has to
reinvent the same scrubbing.
to a repo (some fleet operators do).
Scope
Decide between:
the attestation timestamp or the file mtime for anyone who needs it.
pattern for exactly this problem:
RemediationExecutorgained anow_providerkwarg (defaulting todatetime.now) so tests canpin a fixed clock. Applying the same shape to
format_results_markdownkeeps production behavior while makingthe output testable.
Recommend (3) for consistency with the precedent, or (1) if nobody is
actually consuming the field.
Also decide whether
duration_msshould be omitted from serializedpass_history(it is diagnostic, not evidence) or kept and documentedas non-reproducible.
Non-goals
timestamps at the per-result level; the attestation predicate has its
own timestamp, which is legitimate and should stay.
byte-stability.
Constitution alignment
precondition for the report being usable as evidence.
Related
was found.
now_providerinjection pattern option (3) would reuse.