feat(agentcore-demo): continuous dual-leg observability and demo reliability - #63
Merged
Merged
Conversation
…ability - Add in-cluster continuous dual-leg traffic generator (traffic-generator-deployment.yaml) driving Bedrock model tokens (Leg A) and BNK VIP MCP tool calls (Leg B). - Upgrade mcp-bedrock-token-shipper to receive in-cluster model metrics via HTTP :9090 while continuing background CloudWatch log polling. - Fix iRule rate limiter in mcp-security-policy.yaml using minute-bucketed keys to eliminate TMM 'table incr' type errors. - Ensure AmazonSSMManagedInstanceCore policy attachment on jumphost IAM role in setup-agentcore-network.sh and rebuild.sh for out-of-the-box demo.sh execution. - Use AGENT_TOKEN for TLS validation in demo.sh to isolate certificate checks from Act 4 rate-limiting exhaustion.
…eg, render the VIP - the generator's Leg A is labelled source=synthetic (stream label and record field); CloudWatch records carry source=bedrock-logs; README says which is which and that synthetic numbers are not spend - the shipper waits for the background boto3 install instead of exiting on the first ImportError, logs poll errors, coerces token fields to int - rebuild.sh reads the VIP off the live Gateway into the generator and points BNKDemoBedrockTokenShipper's trust at the live cluster's OIDC provider (a rebuilt cluster gets a new one; the role trusted the old one and every poll failed with AccessDenied) - generator cadence 30 s, tokens from the same literals as mcp-tool - setup/teardown attach and detach the SSM policy explicitly and loudly, with the AWS-17 rationale noted in phase 17b - README step 7 goes through rebuild.sh (placeholder substitution), and names the namespaces Verified on bnk-agentcore-demo: generator waves 200/403/200/401 against the VIP, Loki label source=synthetic on the records, shipper CloudWatch poller running against /aws/bedrock/modelinvocations without errors. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Owner
Author
|
Pushed a follow-up commit addressing the review:
Verified on 🤖 Generated with Claude Code |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR brings full parity between
bnk-agentcore-demo(AWS) andbnk-agentfoundry-demo(Azure) in Forge LLM Observability, ensuring continuous traffic generation and repeatable execution every time.Key Changes
traffic-generator-deployment.yaml):mcp-bedrock-token-shipper.yaml)::9090with internal Kubernetes service (bedrock-token-shipper.llm-egress.svc.cluster.local) to calculate pricing and push model reasoning metrics to Loki.mcp-security-policy.yaml):table incrwith minute-bucketed keys ("${gov_key}:${cur_window}") usingtable lookup/table setto eliminate BIG-IP TMM type error faults.setup-agentcore-network.sh,teardown-agentcore-network.sh,demo.sh):AmazonSSMManagedInstanceCoreis attached to the jumphost IAM role sodemo.shexecutes via SSM without manual intervention.demo.shto useAGENT_TOKENto prevent false 429 failures caused by Act 4's intentional rate-limit exhaustion.rebuild.sh,README.md):traffic-generator-deployment.yamldirectly intorebuild.shstep 7 and updated README documentation.Verification
gofmt,go vet,go test -race ./internal/... ./cmd/...) passed 100%.http://localhost:8000/api/k8s/clusters/18/llm-observability/displays active Anthropic usage curves, token counts, cost calculations, and model rankings matchingbnk-agentfoundry-demo.demo.sh --quicksuccessfully executes all acts with zero failures.