Skip to content

fix: prevent ambient AWS config from auto-selecting Bedrock - #3305

Open
hopstreax wants to merge 1 commit into
Graphify-Labs:v8from
hopstreax:investigate/3300-backend-detection
Open

fix: prevent ambient AWS config from auto-selecting Bedrock#3305
hopstreax wants to merge 1 commit into
Graphify-Labs:v8from
hopstreax:investigate/3300-backend-detection

Conversation

@hopstreax

Copy link
Copy Markdown
Contributor

Summary

Fixes #3300 by preventing ambient AWS environment variables from implicitly selecting the Bedrock LLM backend.

What changed

  • Remove automatic Bedrock detection from AWS_PROFILE, AWS_REGION, and AWS_DEFAULT_REGION.

  • Add GRAPHIFY_BACKEND as a global backend override.

  • Support none, off, and auto sentinel values.

  • Validate GRAPHIFY_BACKEND against registered backends.

  • Preserve explicit --backend bedrock / backend="bedrock" behavior.

  • Handle invalid GRAPHIFY_BACKEND values cleanly in the CLI.

  • Document GRAPHIFY_BACKEND in the README.

  • Add regression and integration tests for backend precedence and detection.

Why

Previously, a generic AWS environment variable such as AWS_REGION=us-east-1 could silently cause Graphify to select Bedrock, even when the user had not requested Bedrock.

This could result in:

  • boto3 import failures on default installations.

  • Unexpected AWS authentication/API errors.

  • Potential unintended Bedrock API usage.

  • Ambient AWS configuration overriding intentionally configured Ollama or custom providers.

Bedrock is now explicit-only for automatic backend resolution.

New behavior

Configuration | Result -- | -- AWS_REGION=us-east-1 | Does not select Bedrock AWS_PROFILE=default + Ollama | Ollama is detected AWS_PROFILE=default + custom provider | Custom provider is detected --backend bedrock | Bedrock is selected GRAPHIFY_BACKEND=bedrock | Bedrock is selected GRAPHIFY_BACKEND=none / off | Automatic detection disabled GRAPHIFY_BACKEND=auto | Normal automatic detection Invalid GRAPHIFY_BACKEND | Clear error + exit code 1

Explicit CLI/backend arguments continue to take precedence over GRAPHIFY_BACKEND.

Verification

  • 116 passedtests/test_llm_backends.py

  • 4 passed — focused CLI regression tests

  • 21 passed — Ollama/provider registry tests

  • 78 passed, 2 skipped — Claude CLI/vision tests

  • python -m graphify update . — successful

  • git diff --check — clean

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 2 advisory finding(s) below merit a look before merge.

Formal verification. 1 change(s) tested, no difference found (not proven).


Graphify review — findings

Adds a GRAPHIFY_BACKEND environment variable to detect_backend that overrides automatic detection: a valid backend name selects it, none/off disables detection, auto falls through to the normal probe, and an unknown value raises ValueError. Stops auto-selecting Bedrock from ambient AWS_PROFILE/AWS_REGION variables so accidental spend or missing-boto3 crashes can't be triggered — Bedrock now requires an explicit --backend bedrock or GRAPHIFY_BACKEND=bedrock. Makes dispatch_command catch the new ValueError and exit 1 with a clean error: message instead of a traceback.

Worth a look

  • Ambient AWS Bedrock auto-detection was removedgraphify/llm.py:3123 · Escalate · medium
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
  • detect_backend can now raise ValueErrorgraphify/llm.py:3141 · Escalate · medium
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 1048 functions depend on the 496 functions this change touches.

Health — this change adds coupling hotspots:

  • new: deduplicate_entities() — 63 callers, 21 callees
  • new: build_merge() — 62 callers, 13 callees
  • new: extract_files_direct() — 18 callers, 20 callees
  • new: extract_corpus_parallel() — 26 callers, 11 callees
  • new: _call_claude_cli() — 31 callers, 9 callees
  • new: dispatch_command() — 2 callers, 123 callees
  • new: _extract_with_adaptive_retry() — 22 callers, 10 callees
  • new: _call_llm() — 11 callers, 18 callees
  • …and 18 more — each is listed as a finding

Verification — 1048 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 932 function(s) in the blast radius were not formally verified this run

Formal verification

Could not verify: Could not verify dispatch\_command.

The verifier did not have enough to check dispatch\_command, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)

No difference found (not proven): No behavior difference found in detect\_backend (not a proof).

The verifier ran both versions of detect\_backend on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

· 1 grounded finding(s) anchored inline below; 25 more finding(s) on lines outside this diff (see the check run).

Comment thread graphify/llm.py
@@ -3111,21 +3111,44 @@ def _validate_ollama_base_url(url: str, *, warn: bool = True) -> None:
def detect_backend() -> str | None:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regressiondetect_backend()

28 callers depend on it (afferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

detect_backend() silently routes extraction to Bedrock on the mere presence of AWS_PROFILE/AWS_REGION, with no env var or config key to override it

1 participant