diff --git a/skills/aws-routing/CHANGELOG.md b/skills/aws-routing/CHANGELOG.md index 1b449b0c..31c6fe4f 100644 --- a/skills/aws-routing/CHANGELOG.md +++ b/skills/aws-routing/CHANGELOG.md @@ -1,5 +1,13 @@ # Changelog +## [1.2.5] - 2026-10-27 +### Added +- `references/tgw-routing-patterns.md`: new "DXGW Advertises Only ONE Best Path Per Prefix to TGW" subsection. A single DXGW propagates one winner per prefix to the TGW, so redundant VIFs on one DXGW are active/passive at the TGW; multiple paths per prefix at the TGW require multiple DXGW attachments. Marked `[Field knowledge]`. +### Changed +- `references/tgw-routing-patterns.md`: removed the "multiple DX-GW attachments (within the same DX-GW)" ECMP line, which contradicted the one-path-per-prefix behavior. Clarified that TGW ECMP across separate DXGWs needs identical AS-paths (same ASNs, not just equal length) because TGW does not support AS-path relax, and contrasted this with DX-side ECMP, where AS_PATH ASNs do not need to match. +- `SKILL.md`: `description` rewritten to open with "Use this skill when..." and focus on user intent, keeping the same scope and symptom triggers (BP-01). +- `SKILL.md`: reference files are now markdown links, each with a condition for when to load it; the routing-analysis steps are a checklist with a final self-check step. No technical guidance changed. Driven by skill evaluation tool best-practices findings (BP-04, BP-05, BP-16, BP-17). +- `evals/evals.json`: converted to the skill evaluation tool's functional schema (`skill_name`, `task_type`, `should_trigger`), keeping the nine existing evals unchanged and adding two negative evals. ## [1.2.4] - 2026-10-21 - Agent types now use the API values in `metadata.aws-devops-agent-skills.agent-types` (`CHAT`, `INCIDENT_RCA`, `INCIDENT_TRIAGE`), the field that sets which agents load the skill. diff --git a/skills/aws-routing/SKILL.md b/skills/aws-routing/SKILL.md index f469e91c..6495c123 100644 --- a/skills/aws-routing/SKILL.md +++ b/skills/aws-routing/SKILL.md @@ -1,8 +1,8 @@ --- name: aws-routing -description: 'Analyze and troubleshoot AWS routing and BGP path selection - how AWS chooses a network path, why traffic takes an unexpected route, and how to steer it. Use for AWS Cloud WAN route evaluation and CNE path selection; Direct Connect Gateway (DXGW) path selection and local vs remote region preference; Transit Gateway route tables, ECMP, and peering; VPC route tables and longest-prefix-match; BGP traffic engineering with local-preference communities (7224:7100/7200/7300), AS-path prepending, and MED; DX + VPN redundancy and active/active vs active/passive design; asymmetric routing and failover. Activate on symptoms like "traffic is leaving the region", "not using my local Direct Connect", "routing to the wrong VIF/location", "path is asymmetric", or "how do I make AWS prefer this path". Read-only: analysis, recommendations, and describe/get/list validation commands grounded in public AWS docs.' +description: 'Use this skill when the user wants to know which path AWS will pick for their traffic, why traffic took an unexpected route, or how to make AWS prefer a path. Covers Cloud WAN CNE route evaluation, Direct Connect Gateway local vs remote region preference, Transit Gateway route tables, ECMP and peering, VPC longest-prefix-match, BGP traffic engineering (LP communities 7224:7100/7200/7300, AS-path prepending, MED), DX + VPN redundancy and failover, and asymmetric routing. Use it even when the user only describes a symptom, such as "traffic is leaving the region", "not using my local Direct Connect", "routing to the wrong VIF/location", or "path is asymmetric". Read-only: analysis, recommendations, and describe/get/list validation commands grounded in public AWS docs.' metadata: - version: "1.2.4" + version: "1.2.5" author: midakin aws-devops-agent-skills.agent-types: "CHAT, INCIDENT_RCA" --- @@ -55,16 +55,22 @@ appropriate review, approvals, and testing). ## How to analyze a routing problem -1. **Establish the path.** Identify source, destination, and every routing construct in between - (VPC route table → TGW/Cloud WAN CNE → DXGW → VIF → on-prem, or the relevant subset). -2. **Name the route table.** Be explicit about which table makes each decision (VPC RT, TGW RT, - CNE RT, DXGW internal selection). -3. **Apply the evaluation order for that construct** (see below). -4. **Identify the deciding attribute** (longest prefix, local preference, AS-path, MED, source type). -5. **Flag non-determinism** (e.g., ECMP or "deterministically random" tiebreakers) and recommend a - deterministic alternative. -6. **Cite the AWS documentation** that supports the behavior. -7. **Provide read-only validation commands** the user can run to confirm. +Work through this checklist in order; each step depends on the one before it. + +- [ ] **1. Establish the path.** Identify source, destination, and every routing construct in + between (VPC route table → TGW/Cloud WAN CNE → DXGW → VIF → on-prem, or the relevant subset). +- [ ] **2. Name the route table.** Be explicit about which table makes each decision (VPC RT, TGW RT, + CNE RT, DXGW internal selection). +- [ ] **3. Apply the evaluation order for that construct** (see below). +- [ ] **4. Identify the deciding attribute** (longest prefix, local preference, AS-path, MED, source + type). +- [ ] **5. Flag non-determinism** (e.g., ECMP or "deterministically random" tiebreakers) and + recommend a deterministic alternative. +- [ ] **6. Cite the AWS documentation** that supports the behavior. +- [ ] **7. Provide read-only validation commands** the user can run to confirm. +- [ ] **8. Self-check before answering.** Re-read the answer against steps 1-7: every route table is + named, the evaluation order matches the construct, each cited URL was actually retrieved, and every + command is read-only. Fix any gap before presenting. ## Key evaluation rules @@ -86,14 +92,22 @@ evaluated before AS-path. DXGW prefers the local associated-region DX by default ## Reference material -Detailed mechanics, patterns, and caveats are in `references/`: -- `cloudwan-dx-routing-patterns.md` - Cloud WAN route evaluation, DXGW path selection, community - traffic-engineering patterns, regional inspection, multi-region egress -- `dx-routing-patterns.md` - DX BGP mechanics, BFD, MED, ASN ranges, VIF considerations, CloudHub, - when VIFs can talk, active/active vs active/passive -- `tgw-routing-patterns.md` - TGW route evaluation, ECMP limits, DX/VPN caveats, failover, peering -- `vpn-dx-redundancy-patterns.md` - VPN + DX redundancy, route preference, ECMP limits, asymmetry -- `networking-strategic-questions.md` - discovery questions per service +Detailed mechanics, patterns, and caveats are in `references/`. Load only the file that matches the +construct deciding the path: +- [Cloud WAN + DX routing patterns](references/cloudwan-dx-routing-patterns.md) - load when the path + crosses a Cloud WAN CNE: CNE route evaluation, DXGW path selection into Cloud WAN, community + traffic-engineering patterns, regional inspection, multi-region egress. +- [Direct Connect routing patterns](references/dx-routing-patterns.md) - load when the question is + about a DX VIF or DXGW without Cloud WAN: BGP mechanics, BFD, MED, ASN ranges, VIF considerations, + CloudHub, when VIFs can talk, active/active vs active/passive. +- [Transit Gateway routing patterns](references/tgw-routing-patterns.md) - load when a TGW route + table makes the decision: TGW route evaluation, ECMP limits, DXGW-to-TGW behavior, DX/VPN caveats, + failover, peering. +- [VPN + DX redundancy patterns](references/vpn-dx-redundancy-patterns.md) - load when VPN backs up + or runs alongside DX: route preference between them, ECMP limits, asymmetric return paths. +- [Networking strategic questions](references/networking-strategic-questions.md) - load when key + facts are missing (regions, attachments, ASNs, communities) and you need discovery questions to ask + before analyzing. ## Output expectations diff --git a/skills/aws-routing/evals/best-practices/v4/benchmark.json b/skills/aws-routing/evals/best-practices/v4/benchmark.json new file mode 100644 index 00000000..639cb3bf --- /dev/null +++ b/skills/aws-routing/evals/best-practices/v4/benchmark.json @@ -0,0 +1,307 @@ +{ + "timestamp": "2026-10-27T22:03:07Z", + "version": "v4", + "model": "us.anthropic.claude-sonnet-5", + "iterations": 3, + "avg_confidence": "high", + "consistency": { + "consistency_score": 100, + "reports": [ + { + "iteration": 1, + "result": "passed", + "score": 100, + "total": 17, + "passed": 14, + "failed": 0, + "skipped": 3, + "warning": 0, + "errored": 0, + "low_confidence": 0, + "avg_confidence": "high" + }, + { + "iteration": 2, + "result": "passed", + "score": 100, + "total": 17, + "passed": 14, + "failed": 0, + "skipped": 3, + "warning": 0, + "errored": 0, + "low_confidence": 0, + "avg_confidence": "high" + }, + { + "iteration": 3, + "result": "passed", + "score": 100, + "total": 17, + "passed": 14, + "failed": 0, + "skipped": 3, + "warning": 0, + "errored": 0, + "low_confidence": 0, + "avg_confidence": "high" + } + ], + "per_test": [ + { + "id": "BP-01", + "name": "Effective description", + "results": [ + "passed", + "passed", + "passed" + ], + "confidences": [ + "high", + "high", + "high" + ], + "avg_confidence": "high", + "consistent": true + }, + { + "id": "BP-02", + "name": "Verbosity and conciseness", + "results": [ + "passed", + "passed", + "passed" + ], + "confidences": [ + "high", + "high", + "high" + ], + "avg_confidence": "high", + "consistent": true + }, + { + "id": "BP-03", + "name": "Detailed reference materials must not be in body", + "results": [ + "passed", + "passed", + "passed" + ], + "confidences": [ + "medium", + "high", + "medium" + ], + "avg_confidence": "medium", + "consistent": true + }, + { + "id": "BP-04", + "name": "Reference files must be linked", + "results": [ + "passed", + "passed", + "passed" + ], + "confidences": [ + "high", + "high", + "high" + ], + "avg_confidence": "high", + "consistent": true + }, + { + "id": "BP-05", + "name": "Reference links must specify when to load", + "results": [ + "passed", + "passed", + "passed" + ], + "confidences": [ + "high", + "high", + "high" + ], + "avg_confidence": "high", + "consistent": true + }, + { + "id": "BP-06", + "name": "Reference path format", + "results": [ + "passed", + "passed", + "passed" + ], + "confidences": [ + "high", + "high", + "high" + ], + "avg_confidence": "high", + "consistent": true + }, + { + "id": "BP-07", + "name": "Prescription vs description", + "results": [ + "passed", + "passed", + "passed" + ], + "confidences": [ + "medium", + "medium", + "medium" + ], + "avg_confidence": "medium", + "consistent": true + }, + { + "id": "BP-08", + "name": "Defaults, not menus", + "results": [ + "passed", + "passed", + "passed" + ], + "confidences": [ + "medium", + "medium", + "medium" + ], + "avg_confidence": "medium", + "consistent": true + }, + { + "id": "BP-09", + "name": "Favor procedures over declarations", + "results": [ + "passed", + "passed", + "passed" + ], + "confidences": [ + "high", + "high", + "high" + ], + "avg_confidence": "high", + "consistent": true + }, + { + "id": "BP-10", + "name": "Gotchas section", + "results": [ + "passed", + "passed", + "passed" + ], + "confidences": [ + "medium", + "medium", + "medium" + ], + "avg_confidence": "medium", + "consistent": true + }, + { + "id": "BP-11", + "name": "Templates and outputs missing", + "results": [ + "passed", + "passed", + "passed" + ], + "confidences": [ + "medium", + "medium", + "medium" + ], + "avg_confidence": "medium", + "consistent": true + }, + { + "id": "BP-12", + "name": "Templates and outputs must not be in body", + "results": [ + "passed", + "passed", + "passed" + ], + "confidences": [ + "high", + "high", + "high" + ], + "avg_confidence": "high", + "consistent": true + }, + { + "id": "BP-13", + "name": "Asset files must be linked", + "results": [ + "skipped", + "skipped", + "skipped" + ], + "consistent": true + }, + { + "id": "BP-14", + "name": "Asset links must specify when to load", + "results": [ + "skipped", + "skipped", + "skipped" + ], + "consistent": true + }, + { + "id": "BP-15", + "name": "Asset path format", + "results": [ + "skipped", + "skipped", + "skipped" + ], + "consistent": true + }, + { + "id": "BP-16", + "name": "Step-by-step guidance", + "results": [ + "passed", + "passed", + "passed" + ], + "confidences": [ + "high", + "high", + "high" + ], + "avg_confidence": "high", + "consistent": true + }, + { + "id": "BP-17", + "name": "Validation loops", + "results": [ + "passed", + "passed", + "passed" + ], + "confidences": [ + "high", + "high", + "high" + ], + "avg_confidence": "high", + "consistent": true + } + ] + } +} \ No newline at end of file diff --git a/skills/aws-routing/evals/best-practices/v4/iteration-1/best-practices-tests-results.json b/skills/aws-routing/evals/best-practices/v4/iteration-1/best-practices-tests-results.json new file mode 100644 index 00000000..840700e7 --- /dev/null +++ b/skills/aws-routing/evals/best-practices/v4/iteration-1/best-practices-tests-results.json @@ -0,0 +1,164 @@ +{ + "version": "v4", + "iteration": 1, + "timestamp": "2026-10-27T22:03:03Z", + "test_type": "best-practices", + "passing_score": 100, + "score": 100, + "result": "passed", + "summary": { + "total": 17, + "passed": 14, + "failed": 0, + "skipped": 3, + "warning": 0, + "errored": 0, + "low_confidence": 0, + "avg_confidence": "high" + }, + "tests": [ + { + "id": "BP-01", + "name": "Effective description", + "result": "passed", + "confidence": "high", + "reasoning": "The description opens with imperative phrasing ('Use this skill when...'), focuses on user intent (which path AWS will pick, why traffic took an unexpected route), explicitly lists trigger contexts including symptom-based phrasing a user might use without naming the domain ('traffic is leaving the region', 'not using my local Direct Connect', 'routing to the wrong VIF/location', 'path is asymmetric'), and stays within a reasonably concise paragraph despite covering a technically dense domain.", + "evidence": "'Use this skill when the user wants to know which path AWS will pick for their traffic, why traffic took an unexpected route, or how to make AWS prefer a path. ... Use it even when the user only describes a symptom, such as \"traffic is leaving the region\", \"not using my local Direct Connect\", \"routing to the wrong VIF/location\", or \"path is asymmetric\".'", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/optimizing-descriptions" + }, + { + "id": "BP-02", + "name": "Verbosity and conciseness", + "result": "passed", + "confidence": "high", + "reasoning": "The body avoids explaining general networking concepts the agent already knows (e.g., what BGP is) and jumps straight into AWS-specific route evaluation order, LP communities, and construct-specific rules that are non-obvious and domain-specific.", + "evidence": "Longest prefix match \u2192 local preference \u2192 AS_PATH length \u2192 MED \u2192 ECMP (equal AS_PATH and BGP attributes). LP communities `7224:7300/7200/7100`...", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#add-what-the-agent-lacks-omit-what-it-knows" + }, + { + "id": "BP-03", + "name": "Detailed reference materials must not be in body", + "result": "passed", + "confidence": "medium", + "reasoning": "The body contains only condensed evaluation-order summaries (a few lines per construct), not full tables of error codes or exhaustive reference data; detailed mechanics are explicitly pushed to references/ files.", + "evidence": "Detailed mechanics, patterns, and caveats are in `references/`. Load only the file that matches the construct deciding the path:", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#structure-large-skills-with-progressive-disclosure" + }, + { + "id": "BP-04", + "name": "Reference files must be linked", + "result": "passed", + "confidence": "high", + "reasoning": "Every file in references/ has a corresponding markdown link in the body under 'Reference material'.", + "evidence": "[Cloud WAN + DX routing patterns](references/cloudwan-dx-routing-patterns.md) ... [Direct Connect routing patterns](references/dx-routing-patterns.md) ... [Transit Gateway routing patterns](references/tgw-routing-patterns.md) ... [VPN + DX redundancy patterns](references/vpn-dx-redundancy-patterns.md) ... [Networking strategic questions](references/networking-strategic-questions.md)" + }, + { + "id": "BP-05", + "name": "Reference links must specify when to load", + "result": "passed", + "confidence": "high", + "reasoning": "Each reference link includes a specific 'load when' condition describing the scenario that triggers using that file.", + "evidence": "load when the path crosses a Cloud WAN CNE: CNE route evaluation, DXGW path selection into Cloud WAN...", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#structure-large-skills-with-progressive-disclosure" + }, + { + "id": "BP-06", + "name": "Reference path format", + "result": "passed", + "confidence": "high", + "reasoning": "All reference links use relative paths rooted at references/ with no absolute paths or traversal.", + "evidence": "[Transit Gateway routing patterns](references/tgw-routing-patterns.md)", + "agent_skills_spec_reference": "https://agentskills.io/specification#file-references" + }, + { + "id": "BP-07", + "name": "Prescription vs description", + "result": "passed", + "confidence": "medium", + "reasoning": "Fragile areas (route evaluation order, LP communities, read-only constraints) are given exact, prescriptive rules, while open-ended analysis steps (comparing tradeoffs, recommending testing) leave room for agent judgment appropriately.", + "evidence": "Longest prefix match \u2192 static \u2192 VPC-propagated (same region) \u2192 unequal AS-path/MED (shortest wins) \u2192 equal AS-path & MED source preference...", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#match-specificity-to-fragility" + }, + { + "id": "BP-08", + "name": "Defaults, not menus", + "result": "passed", + "confidence": "medium", + "reasoning": "The skill does not present competing tools as equal menu options; it directs loading only the one reference file matching the construct in question, which functions as a clear default selection mechanism.", + "evidence": "Load only the file that matches the construct deciding the path:", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#provide-defaults-not-menus" + }, + { + "id": "BP-09", + "name": "Favor procedures over declarations", + "result": "passed", + "confidence": "high", + "reasoning": "The 'How to analyze a routing problem' checklist teaches a generalizable procedure (establish path, name route table, apply evaluation order, etc.) applicable to any routing question, not a one-off declarative task.", + "evidence": "**1. Establish the path.** Identify source, destination, and every routing construct in between...", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#favor-procedures-over-declarations" + }, + { + "id": "BP-10", + "name": "Gotchas section", + "result": "passed", + "confidence": "medium", + "reasoning": "The skill includes non-obvious, concrete corrections such as AS-path prepending being unreliable cross-region and DXGW defaulting to local-region preference, functioning as gotchas even without a dedicated heading.", + "evidence": "**AS-path prepending:** within a region only - unreliable across regions because DXGW local-region LP overrides it. **LP communities:** work within-region and cross-region.", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#gotchas-sections" + }, + { + "id": "BP-11", + "name": "Templates and outputs missing", + "result": "passed", + "confidence": "medium", + "reasoning": "The 'Output expectations' section already specifies the structured elements expected in output (route table name, path, tradeoffs, CLI commands) which functions as a lightweight template appropriate to this advisory skill.", + "evidence": "Explain the route-evaluation logic step by step. Name the specific route table involved. Give the end-to-end traffic path.", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#templates-for-output-format" + }, + { + "id": "BP-12", + "name": "Templates and outputs must not be in body", + "result": "passed", + "confidence": "high", + "reasoning": "No large inline templates exist in the body; the output expectations are a short bulleted list, well under 20 lines.", + "evidence": "Output expectations section contains only 6 short bullet points, no inline template skeleton.", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#templates-for-output-format" + }, + { + "id": "BP-13", + "name": "Asset files must be linked", + "result": "skipped", + "message": "No files in assets/, so this test does not apply" + }, + { + "id": "BP-14", + "name": "Asset links must specify when to load", + "result": "skipped", + "message": "No files in assets/, so this test does not apply" + }, + { + "id": "BP-15", + "name": "Asset path format", + "result": "skipped", + "message": "No files in assets/, so this test does not apply" + }, + { + "id": "BP-16", + "name": "Step-by-step guidance", + "result": "passed", + "confidence": "high", + "reasoning": "The procedural workflow is presented as an ordered checkbox checklist with explicit step numbers, satisfying the required format.", + "evidence": "- [ ] **1. Establish the path.** ... - [ ] **8. Self-check before answering.**", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#checklists-for-multi-step-workflows" + }, + { + "id": "BP-17", + "name": "Validation loops", + "result": "passed", + "confidence": "high", + "reasoning": "Step 8 explicitly instructs the agent to re-check its own answer against the prior steps before presenting, which is a genuine self-validation instruction.", + "evidence": "**8. Self-check before answering.** Re-read the answer against steps 1-7: every route table is named, the evaluation order matches the construct, each cited URL was actually retrieved, and every command is read-only. Fix any gap before presenting.", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#validation-loops" + } + ] +} \ No newline at end of file diff --git a/skills/aws-routing/evals/best-practices/v4/iteration-2/best-practices-tests-results.json b/skills/aws-routing/evals/best-practices/v4/iteration-2/best-practices-tests-results.json new file mode 100644 index 00000000..352293c2 --- /dev/null +++ b/skills/aws-routing/evals/best-practices/v4/iteration-2/best-practices-tests-results.json @@ -0,0 +1,164 @@ +{ + "version": "v4", + "iteration": 2, + "timestamp": "2026-10-27T22:03:07Z", + "test_type": "best-practices", + "passing_score": 100, + "score": 100, + "result": "passed", + "summary": { + "total": 17, + "passed": 14, + "failed": 0, + "skipped": 3, + "warning": 0, + "errored": 0, + "low_confidence": 0, + "avg_confidence": "high" + }, + "tests": [ + { + "id": "BP-01", + "name": "Effective description", + "result": "passed", + "confidence": "high", + "reasoning": "The description uses imperative phrasing ('Use this skill when...'), focuses on user intent/symptoms (e.g., 'traffic is leaving the region', 'not using my local Direct Connect'), explicitly lists trigger contexts and technical scope, and does so in a concise paragraph without delving into internal implementation mechanics.", + "evidence": "Use this skill when the user wants to know which path AWS will pick for their traffic, why traffic took an unexpected route, or how to make AWS prefer a path. ... Use it even when the user only describes a symptom, such as \"traffic is leaving the region\", \"not using my local Direct Connect\", \"routing to the wrong VIF/location\", or \"path is asymmetric\".", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/optimizing-descriptions" + }, + { + "id": "BP-02", + "name": "Verbosity and conciseness", + "result": "passed", + "confidence": "high", + "reasoning": "The body skips general networking explanations and goes straight into AWS-specific route-evaluation order, LP community values, and construct-specific decision logic that an agent would not already know. No generic concept explanations (e.g., 'what is BGP') are present.", + "evidence": "Longest prefix match \u2192 local preference \u2192 AS_PATH length \u2192 MED \u2192 ECMP (equal AS_PATH and BGP attributes). LP communities `7224:7300/7200/7100` (High/Medium/Low) set local preference and are evaluated before AS-path.", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#add-what-the-agent-lacks-omit-what-it-knows" + }, + { + "id": "BP-03", + "name": "Detailed reference materials must not be in body", + "result": "passed", + "confidence": "high", + "reasoning": "The 'Key evaluation rules' section is terse (a few lines per construct) and acts as a high-level summary, with detailed mechanics explicitly deferred to references/ files loaded conditionally. No large blocks of tables, API docs, or exhaustive lookup data sit in the body.", + "evidence": "Detailed mechanics, patterns, and caveats are in `references/`. Load only the file that matches the construct deciding the path:", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#structure-large-skills-with-progressive-disclosure" + }, + { + "id": "BP-04", + "name": "Reference files must be linked", + "result": "passed", + "confidence": "high", + "reasoning": "All five files in references/ are linked with proper markdown link syntax in the 'Reference material' section.", + "evidence": "[Cloud WAN + DX routing patterns](references/cloudwan-dx-routing-patterns.md) - load when the path crosses a Cloud WAN CNE..." + }, + { + "id": "BP-05", + "name": "Reference links must specify when to load", + "result": "passed", + "confidence": "high", + "reasoning": "Each reference link is followed by a specific 'load when' clause describing the exact condition under which the agent should load that file, rather than a generic 'see for details' phrase.", + "evidence": "[Transit Gateway routing patterns](references/tgw-routing-patterns.md) - load when a TGW route table makes the decision: TGW route evaluation, ECMP limits, DXGW-to-TGW behavior, DX/VPN caveats, failover, peering.", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#structure-large-skills-with-progressive-disclosure" + }, + { + "id": "BP-06", + "name": "Reference path format", + "result": "passed", + "confidence": "high", + "reasoning": "All reference links use relative paths rooted at references/ with no absolute paths or directory traversal.", + "evidence": "[VPN + DX redundancy patterns](references/vpn-dx-redundancy-patterns.md)", + "agent_skills_spec_reference": "https://agentskills.io/specification#file-references" + }, + { + "id": "BP-07", + "name": "Prescription vs description", + "result": "passed", + "confidence": "medium", + "reasoning": "The skill is prescriptive about the fragile checklist (ordered, dependent steps for route analysis) and the hard boundaries (read-only, no mutating calls), while remaining flexible in 'Output expectations' which allows comparing tradeoffs rather than dictating one exact answer. This matches fragility to prescriptiveness well.", + "evidence": "Work through this checklist in order; each step depends on the one before it. ... Compare options with tradeoffs (determinism, complexity, failover) when multiple solutions exist.", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#match-specificity-to-fragility" + }, + { + "id": "BP-08", + "name": "Defaults, not menus", + "result": "passed", + "confidence": "medium", + "reasoning": "The skill doesn't present competing tools as an open menu; it gives per-construct default decision logic and specifies exact LP community values and direction-specific guidance rather than listing multiple interchangeable options.", + "evidence": "AWS \u2192 on-prem (egress): control with DX LP communities. On-prem \u2192 AWS (ingress): control with the customer router's local-pref/weight/MED.", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#provide-defaults-not-menus" + }, + { + "id": "BP-09", + "name": "Favor procedures over declarations", + "result": "passed", + "confidence": "high", + "reasoning": "The 8-step checklist teaches a generalizable approach (establish path, name route table, apply evaluation order, flag non-determinism, cite docs, validate) applicable to any routing question, not a one-off instance-specific task.", + "evidence": "1. Establish the path. ... 2. Name the route table. ... 3. Apply the evaluation order for that construct", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#favor-procedures-over-declarations" + }, + { + "id": "BP-10", + "name": "Gotchas section", + "result": "passed", + "confidence": "medium", + "reasoning": "While there's no section explicitly titled 'Gotchas', the 'Direction & tooling guidance' and evaluation-order notes capture exactly the kind of non-obvious, environment-specific corrections (e.g., AS-path prepending being unreliable cross-region due to DXGW override, deterministically-random ECMP tiebreakers) that gotchas sections typically contain.", + "evidence": "AS-path prepending: within a region only - unreliable across regions because DXGW local-region LP overrides it. LP communities: work within-region and cross-region.", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#gotchas-sections" + }, + { + "id": "BP-11", + "name": "Templates and outputs missing", + "result": "passed", + "confidence": "medium", + "reasoning": "The skill's output is analysis/recommendations (route evaluation, path diagnosis) rather than a formal structured document/report, and the 'Output expectations' section already lists the required elements (route table name, path, tradeoffs, CLI commands) functioning as a lightweight template appropriate to the task.", + "evidence": "Explain the route-evaluation logic step by step. Name the specific route table involved. Give the end-to-end traffic path. ... Include read-only AWS CLI validation commands.", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#templates-for-output-format" + }, + { + "id": "BP-12", + "name": "Templates and outputs must not be in body", + "result": "passed", + "confidence": "high", + "reasoning": "No large inline output template exists in the body; 'Output expectations' is a short bullet list (6 lines), well under the ~20 line threshold.", + "evidence": "Explain the route-evaluation logic step by step.\n- Name the specific route table involved.\n- Give the end-to-end traffic path.", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#templates-for-output-format" + }, + { + "id": "BP-13", + "name": "Asset files must be linked", + "result": "skipped", + "message": "No files in assets/, so this test does not apply" + }, + { + "id": "BP-14", + "name": "Asset links must specify when to load", + "result": "skipped", + "message": "No files in assets/, so this test does not apply" + }, + { + "id": "BP-15", + "name": "Asset path format", + "result": "skipped", + "message": "No files in assets/, so this test does not apply" + }, + { + "id": "BP-16", + "name": "Step-by-step guidance", + "result": "passed", + "confidence": "high", + "reasoning": "The 'How to analyze a routing problem' section is a sequential, dependent procedure and is correctly presented as a checkbox checklist with explicit step numbers ('1.', '2.', etc. inside '- [ ]'), satisfying the required format.", + "evidence": "- [ ] **1. Establish the path.** ... - [ ] **2. Name the route table.** ... - [ ] **8. Self-check before answering.**", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#checklists-for-multi-step-workflows" + }, + { + "id": "BP-17", + "name": "Validation loops", + "result": "passed", + "confidence": "high", + "reasoning": "Step 8 explicitly instructs the agent to re-check its own answer against the prior steps (route table named, evaluation order correct, citations verified, commands read-only) before presenting, which is a genuine self-validation instruction.", + "evidence": "8. Self-check before answering. Re-read the answer against steps 1-7: every route table is named, the evaluation order matches the construct, each cited URL was actually retrieved, and every command is read-only. Fix any gap before presenting.", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#validation-loops" + } + ] +} \ No newline at end of file diff --git a/skills/aws-routing/evals/best-practices/v4/iteration-3/best-practices-tests-results.json b/skills/aws-routing/evals/best-practices/v4/iteration-3/best-practices-tests-results.json new file mode 100644 index 00000000..e7ee2245 --- /dev/null +++ b/skills/aws-routing/evals/best-practices/v4/iteration-3/best-practices-tests-results.json @@ -0,0 +1,164 @@ +{ + "version": "v4", + "iteration": 3, + "timestamp": "2026-10-27T22:03:04Z", + "test_type": "best-practices", + "passing_score": 100, + "score": 100, + "result": "passed", + "summary": { + "total": 17, + "passed": 14, + "failed": 0, + "skipped": 3, + "warning": 0, + "errored": 0, + "low_confidence": 0, + "avg_confidence": "high" + }, + "tests": [ + { + "id": "BP-01", + "name": "Effective description", + "result": "passed", + "confidence": "high", + "reasoning": "The description uses imperative phrasing ('Use this skill when...'), focuses on user intent/symptoms (not knowing why traffic took an unexpected path, wanting to prefer a path), explicitly lists trigger contexts including vague symptom phrasing like 'traffic is leaving the region', and covers scope concisely in a short paragraph.", + "evidence": "\"Use this skill when the user wants to know which path AWS will pick for their traffic, why traffic took an unexpected route, or how to make AWS prefer a path. ... Use it even when the user only describes a symptom, such as \\\"traffic is leaving the region\\\", \\\"not using my local Direct Connect\\\", \\\"routing to the wrong VIF/location\\\", or \\\"path is asymmetric\\\".\"", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/optimizing-descriptions" + }, + { + "id": "BP-02", + "name": "Verbosity and conciseness", + "result": "passed", + "confidence": "high", + "reasoning": "The body is dense with domain-specific routing rules (evaluation orders, LP communities, AS-path caveats) and does not explain general networking concepts the agent already knows. It stays focused on AWS-specific, non-obvious behavior.", + "evidence": "Longest prefix match \u2192 local preference \u2192 AS_PATH length \u2192 MED \u2192 ECMP (equal AS_PATH and BGP attributes). LP communities `7224:7300/7200/7100`...", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#add-what-the-agent-lacks-omit-what-it-knows" + }, + { + "id": "BP-03", + "name": "Detailed reference materials must not be in body", + "result": "passed", + "confidence": "medium", + "reasoning": "The 'Key evaluation rules' section is compact (a handful of lines per construct) and acts as a quick-reference summary rather than exhaustive tables or API docs; detailed mechanics are properly deferred to references/ files.", + "evidence": "### Cloud WAN (per CNE)\nLongest prefix match \u2192 static \u2192 VPC-propagated (same region) \u2192 unequal AS-path/MED (shortest wins) \u2192 equal AS-path & MED source preference...", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#structure-large-skills-with-progressive-disclosure" + }, + { + "id": "BP-04", + "name": "Reference files must be linked", + "result": "passed", + "confidence": "high", + "reasoning": "All five reference files are linked with markdown link syntax in the Reference material section.", + "evidence": "[Cloud WAN + DX routing patterns](references/cloudwan-dx-routing-patterns.md) - load when the path crosses a Cloud WAN CNE..." + }, + { + "id": "BP-05", + "name": "Reference links must specify when to load", + "result": "passed", + "confidence": "high", + "reasoning": "Each reference link includes explicit 'load when...' guidance describing the triggering condition.", + "evidence": "[Transit Gateway routing patterns](references/tgw-routing-patterns.md) - load when a TGW route table makes the decision: TGW route evaluation, ECMP limits...", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#structure-large-skills-with-progressive-disclosure" + }, + { + "id": "BP-06", + "name": "Reference path format", + "result": "passed", + "confidence": "high", + "reasoning": "All reference links use relative paths rooted at references/ with no absolute paths or traversal.", + "evidence": "[VPN + DX redundancy patterns](references/vpn-dx-redundancy-patterns.md) - load when VPN backs up or runs alongside DX...", + "agent_skills_spec_reference": "https://agentskills.io/specification#file-references" + }, + { + "id": "BP-07", + "name": "Prescription vs description", + "result": "passed", + "confidence": "medium", + "reasoning": "The skill is prescriptive where sequence matters (the 8-step checklist, evaluation order of route tables) and descriptive/flexible where appropriate (output expectations, comparing tradeoffs), matching fragility to specificity well.", + "evidence": "- [ ] **3. Apply the evaluation order for that construct** (see below).\n...\n- Compare options with tradeoffs (determinism, complexity, failover) when multiple solutions exist.", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#match-specificity-to-fragility" + }, + { + "id": "BP-08", + "name": "Defaults, not menus", + "result": "passed", + "confidence": "medium", + "reasoning": "Where multiple reference files/options exist, guidance clearly states which one to load based on specific conditions rather than presenting them as an undifferentiated menu.", + "evidence": "Load only the file that matches the construct deciding the path: ... - load when the path crosses a Cloud WAN CNE...", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#provide-defaults-not-menus" + }, + { + "id": "BP-09", + "name": "Favor procedures over declarations", + "result": "passed", + "confidence": "high", + "reasoning": "The skill teaches a generalizable procedure (establish path, name route table, apply evaluation order, flag non-determinism, cite docs, validate) applicable to any routing question, not a one-off instance.", + "evidence": "Work through this checklist in order; each step depends on the one before it.\n- [ ] **1. Establish the path.**...", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#favor-procedures-over-declarations" + }, + { + "id": "BP-10", + "name": "Gotchas section", + "result": "passed", + "confidence": "medium", + "reasoning": "The skill documents concrete non-obvious pitfalls specific to AWS routing, such as AS-path prepending being unreliable cross-region and DXGW local-region preference overriding it.", + "evidence": "**AS-path prepending:** within a region only - unreliable across regions because DXGW local-region LP overrides it. **LP communities:** work within-region and cross-region.", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#gotchas-sections" + }, + { + "id": "BP-11", + "name": "Templates and outputs missing", + "result": "passed", + "confidence": "medium", + "reasoning": "Output expectations are given as a concise bullet list of required elements rather than a rigid document template, which fits this skill's analytical/advisory nature; no large formatted report is implied.", + "evidence": "## Output expectations\n- Explain the route-evaluation logic step by step.\n- Name the specific route table involved.\n- Give the end-to-end traffic path.", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#templates-for-output-format" + }, + { + "id": "BP-12", + "name": "Templates and outputs must not be in body", + "result": "passed", + "confidence": "high", + "reasoning": "No large output template exists in the body; output expectations are a short bullet list well under 20 lines.", + "evidence": "## Output expectations\n- Explain the route-evaluation logic step by step.\n- Name the specific route table involved.", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#templates-for-output-format" + }, + { + "id": "BP-13", + "name": "Asset files must be linked", + "result": "skipped", + "message": "No files in assets/, so this test does not apply" + }, + { + "id": "BP-14", + "name": "Asset links must specify when to load", + "result": "skipped", + "message": "No files in assets/, so this test does not apply" + }, + { + "id": "BP-15", + "name": "Asset path format", + "result": "skipped", + "message": "No files in assets/, so this test does not apply" + }, + { + "id": "BP-16", + "name": "Step-by-step guidance", + "result": "passed", + "confidence": "high", + "reasoning": "The procedural workflow is presented as an ordered checkbox checklist with 'Step N' numbering, satisfying the required format.", + "evidence": "- [ ] **1. Establish the path.** ... - [ ] **8. Self-check before answering.** Re-read the answer against steps 1-7...", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#checklists-for-multi-step-workflows" + }, + { + "id": "BP-17", + "name": "Validation loops", + "result": "passed", + "confidence": "high", + "reasoning": "Step 8 explicitly instructs the agent to re-check its own answer against the prior steps before presenting, which is a genuine self-validation instruction.", + "evidence": "- [ ] **8. Self-check before answering.** Re-read the answer against steps 1-7: every route table is named, the evaluation order matches the construct, each cited URL was actually retrieved, and every command is read-only. Fix any gap before presenting.", + "agent_skills_spec_reference": "https://agentskills.io/skill-creation/best-practices#validation-loops" + } + ] +} \ No newline at end of file diff --git a/skills/aws-routing/evals/evals.json b/skills/aws-routing/evals/evals.json index 13d5b812..8ea2865a 100644 --- a/skills/aws-routing/evals/evals.json +++ b/skills/aws-routing/evals/evals.json @@ -1,98 +1,131 @@ -[ - { - "id": "cloudwan-local-dx-preference", - "prompt": "In AWS Cloud WAN, traffic for an aggregate prefix is leaving the local region instead of using the local Direct Connect. How does route evaluation decide this and how can I prefer the local DX?", - "expected_output": "Explains Cloud WAN route evaluation (longest-prefix-match first, then AS-path at the CNE), why a remote path can win, and recommends LP communities (7224:7300/7200/7100) and/or AS-path normalization to prefer the local DX.", - "assertions": [ - "references longest prefix match in Cloud WAN route evaluation", - "explains that AS-path length is evaluated at the CNE and can cause a remote path to win", - "recommends LP communities (7224:7300/7200/7100) as a deterministic remedy", - "does not contain 'error'" - ] - }, - { - "id": "dx-path-selection-order", - "prompt": "What is the outbound path-selection order for a Direct Connect private VIF, and where does MED fit?", - "expected_output": "Order: longest prefix match, then local preference, then AS_PATH length, then MED, then ECMP. MED is a low-priority tiebreaker AWS does not recommend relying on. LP communities 7224:7100/7200/7300 set local preference.", - "assertions": [ - "states the order longest prefix match, local preference, AS_PATH, MED, ECMP", - "notes MED is a low-priority tiebreaker", - "mentions LP communities 7224:7100/7200/7300", - "does not contain 'error'" - ] - }, - { - "id": "tgw-vs-vgw-vif-transit", - "prompt": "Can two on-premises sites on separate Direct Connect VIFs talk to each other through AWS?", - "expected_output": "By default AWS does not act as a transit hub (advertises only VPC CIDRs). VPN CloudHub enables site-to-site for VPN connections; TGW with an Allowed Prefix List (and two DXGWs for DX-to-DX) can enable it intentionally.", - "assertions": [ - "explains AWS does not act as a transit hub by default", - "mentions VPN CloudHub for VPN connections", - "mentions TGW or two DXGWs as a way to enable DX-to-DX intentionally", - "does not contain 'error'" - ] - }, - { - "id": "dx-prefer-one-location", - "prompt": "I have two Direct Connect locations and want traffic to prefer location A over location B, with B as backup. How do I do this and what should I avoid?", - "expected_output": "Use LP communities (7224:7300 on primary, lower on backup). AS-path prepending is reliable only within the same associated region; cross-region prepending is unreliable due to DXGW local-region LP.", - "assertions": [ - "recommends LP communities with a higher preference on the primary", - "explains AS-path prepending is reliable only within the same region", - "warns cross-region AS-path prepending is unreliable", - "does not contain 'error'" - ] - }, - { - "id": "tgw-ecmp-behavior", - "prompt": "When does a Transit Gateway load-balance across two Direct Connect paths, and when does it pick just one?", - "expected_output": "ECMP requires the same attachment type and equal AS-path length / BGP attributes; otherwise TGW installs a single (oldest) route. LP communities are not honored across multiple DXGW attachments on TGW.", - "assertions": [ - "states ECMP requires same attachment type and equal AS-path/BGP attributes", - "notes TGW picks a single route when ECMP conditions are not met", - "does not contain 'error'" - ] - }, - { - "id": "dx-vpn-redundancy-preference", - "prompt": "I have a Direct Connect and a backup Site-to-Site VPN to the same VGW. Which path does AWS prefer and how do I avoid asymmetric routing?", - "expected_output": "DX is always preferred over VPN for the same prefix at a VGW; there is no ECMP between DX and VPN. Asymmetric routing is the main failure mode; match ingress/egress preference and validate both directions.", - "assertions": [ - "states DX is always preferred over VPN for the same prefix at a VGW", - "identifies asymmetric routing as the main risk", - "does not contain 'error'" - ] - }, - { - "id": "verification-unknown-source", - "prompt": "Is there an official AWS 'DX Path Selection Flow Chart' document I can cite?", - "expected_output": "Does not fabricate a document or URL; states it cannot verify such a named artifact; points to the Direct Connect routing policies documentation for the path-selection order.", - "assertions": [ - "does not fabricate a document or URL", - "states it cannot verify such a named artifact", - "points to the Direct Connect routing policies documentation" - ] - }, - { - "id": "tgw-vpn-summarization", - "prompt": "A customer is at the TGW-to-VPN advertised-route limit. How do they summarize without breaking the return path?", - "expected_output": "Keep VPC propagation enabled (specifics needed for longest-prefix-match return forwarding), add a static summary route in the VPN-associated TGW route table; under the cap, static beats propagated and least-specific is advertised first, so the summary survives. Anchor the summary to a persistent attachment.", - "assertions": [ - "keeps VPC route propagation enabled for the return path", - "adds a static summary route in the TGW route table associated with the VPN attachment", - "explains static-over-propagated and least-specific selection under the advertisement cap", - "does not contain 'error'" - ] - }, - { - "id": "cloudwan-overlapping-supernet", - "prompt": "A customer advertises a supernet (10.0.0.0/8) from Direct Connect into Cloud WAN that overlaps VPC CIDRs (10.1.0.0/16, 10.2.0.0/16). What issues would this cause and how do they fix it?", - "expected_output": "Longest-prefix-match means VPC /16s win and on-prem hosts inside a VPC CIDR are shadowed/unreachable. Best fix is non-overlapping address planning. Cloud WAN Routing Policies can drop the inbound supernet and suppress VPC prefixes outbound on the DX attachment; BGP communities cannot be matched on DX attachments.", - "assertions": [ - "explains longest-prefix-match shadows on-prem hosts inside a VPC CIDR", - "recommends non-overlapping address planning as the best fix", - "mentions Cloud WAN Routing Policies can filter (drop) prefixes on the Direct Connect attachment", - "does not contain 'error'" - ] - } -] +{ + "skill_name": "aws-routing", + "evals": [ + { + "id": "cloudwan-local-dx-preference", + "task_type": "chat", + "prompt": "In AWS Cloud WAN, traffic for an aggregate prefix is leaving the local region instead of using the local Direct Connect. How does route evaluation decide this and how can I prefer the local DX?", + "should_trigger": true, + "expected_output": "Explains Cloud WAN route evaluation (longest-prefix-match first, then AS-path at the CNE), why a remote path can win, and recommends LP communities (7224:7300/7200/7100) and/or AS-path normalization to prefer the local DX.", + "assertions": [ + "references longest prefix match in Cloud WAN route evaluation", + "explains that AS-path length is evaluated at the CNE and can cause a remote path to win", + "recommends LP communities (7224:7300/7200/7100) as a deterministic remedy", + "does not contain 'error'" + ] + }, + { + "id": "dx-path-selection-order", + "task_type": "chat", + "prompt": "What is the outbound path-selection order for a Direct Connect private VIF, and where does MED fit?", + "should_trigger": true, + "expected_output": "Order: longest prefix match, then local preference, then AS_PATH length, then MED, then ECMP. MED is a low-priority tiebreaker AWS does not recommend relying on. LP communities 7224:7100/7200/7300 set local preference.", + "assertions": [ + "states the order longest prefix match, local preference, AS_PATH, MED, ECMP", + "notes MED is a low-priority tiebreaker", + "mentions LP communities 7224:7100/7200/7300", + "does not contain 'error'" + ] + }, + { + "id": "tgw-vs-vgw-vif-transit", + "task_type": "chat", + "prompt": "Can two on-premises sites on separate Direct Connect VIFs talk to each other through AWS?", + "should_trigger": true, + "expected_output": "By default AWS does not act as a transit hub (advertises only VPC CIDRs). VPN CloudHub enables site-to-site for VPN connections; TGW with an Allowed Prefix List (and two DXGWs for DX-to-DX) can enable it intentionally.", + "assertions": [ + "explains AWS does not act as a transit hub by default", + "mentions VPN CloudHub for VPN connections", + "mentions TGW or two DXGWs as a way to enable DX-to-DX intentionally", + "does not contain 'error'" + ] + }, + { + "id": "dx-prefer-one-location", + "task_type": "chat", + "prompt": "I have two Direct Connect locations and want traffic to prefer location A over location B, with B as backup. How do I do this and what should I avoid?", + "should_trigger": true, + "expected_output": "Use LP communities (7224:7300 on primary, lower on backup). AS-path prepending is reliable only within the same associated region; cross-region prepending is unreliable due to DXGW local-region LP.", + "assertions": [ + "recommends LP communities with a higher preference on the primary", + "explains AS-path prepending is reliable only within the same region", + "warns cross-region AS-path prepending is unreliable", + "does not contain 'error'" + ] + }, + { + "id": "tgw-ecmp-behavior", + "task_type": "chat", + "prompt": "When does a Transit Gateway load-balance across two Direct Connect paths, and when does it pick just one?", + "should_trigger": true, + "expected_output": "ECMP requires the same attachment type and equal AS-path length / BGP attributes; otherwise TGW installs a single (oldest) route. LP communities are not honored across multiple DXGW attachments on TGW.", + "assertions": [ + "states ECMP requires same attachment type and equal AS-path/BGP attributes", + "notes TGW picks a single route when ECMP conditions are not met", + "does not contain 'error'" + ] + }, + { + "id": "dx-vpn-redundancy-preference", + "task_type": "chat", + "prompt": "I have a Direct Connect and a backup Site-to-Site VPN to the same VGW. Which path does AWS prefer and how do I avoid asymmetric routing?", + "should_trigger": true, + "expected_output": "DX is always preferred over VPN for the same prefix at a VGW; there is no ECMP between DX and VPN. Asymmetric routing is the main failure mode; match ingress/egress preference and validate both directions.", + "assertions": [ + "states DX is always preferred over VPN for the same prefix at a VGW", + "identifies asymmetric routing as the main risk", + "does not contain 'error'" + ] + }, + { + "id": "verification-unknown-source", + "task_type": "chat", + "prompt": "Is there an official AWS 'DX Path Selection Flow Chart' document I can cite?", + "should_trigger": true, + "expected_output": "Does not fabricate a document or URL; states it cannot verify such a named artifact; points to the Direct Connect routing policies documentation for the path-selection order.", + "assertions": [ + "does not fabricate a document or URL", + "states it cannot verify such a named artifact", + "points to the Direct Connect routing policies documentation" + ] + }, + { + "id": "tgw-vpn-summarization", + "task_type": "chat", + "prompt": "A customer is at the TGW-to-VPN advertised-route limit. How do they summarize without breaking the return path?", + "should_trigger": true, + "expected_output": "Keep VPC propagation enabled (specifics needed for longest-prefix-match return forwarding), add a static summary route in the VPN-associated TGW route table; under the cap, static beats propagated and least-specific is advertised first, so the summary survives. Anchor the summary to a persistent attachment.", + "assertions": [ + "keeps VPC route propagation enabled for the return path", + "adds a static summary route in the TGW route table associated with the VPN attachment", + "explains static-over-propagated and least-specific selection under the advertisement cap", + "does not contain 'error'" + ] + }, + { + "id": "cloudwan-overlapping-supernet", + "task_type": "chat", + "prompt": "A customer advertises a supernet (10.0.0.0/8) from Direct Connect into Cloud WAN that overlaps VPC CIDRs (10.1.0.0/16, 10.2.0.0/16). What issues would this cause and how do they fix it?", + "should_trigger": true, + "expected_output": "Longest-prefix-match means VPC /16s win and on-prem hosts inside a VPC CIDR are shadowed/unreachable. Best fix is non-overlapping address planning. Cloud WAN Routing Policies can drop the inbound supernet and suppress VPC prefixes outbound on the DX attachment; BGP communities cannot be matched on DX attachments.", + "assertions": [ + "explains longest-prefix-match shadows on-prem hosts inside a VPC CIDR", + "recommends non-overlapping address planning as the best fix", + "mentions Cloud WAN Routing Policies can filter (drop) prefixes on the Direct Connect attachment", + "does not contain 'error'" + ] + }, + { + "id": "negative-python-linked-list", + "task_type": "chat", + "prompt": "Write a Python function to reverse a linked list.", + "should_trigger": false + }, + { + "id": "negative-dynamodb-iam-policy", + "task_type": "chat", + "prompt": "What IAM policy do I need to allow read-only access to DynamoDB?", + "should_trigger": false + } + ] +} diff --git a/skills/aws-routing/evals/functional/v1/benchmark.json b/skills/aws-routing/evals/functional/v1/benchmark.json new file mode 100644 index 00000000..ea46f12a --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/benchmark.json @@ -0,0 +1,1631 @@ +{ + "summary": "This evaluation covers 11 eval tasks (20 iterations total, no failures, no cleanup skipped). Two of the evals (negative-python-linked-list, negative-dynamodb-iam-policy) were \"should_trigger: false\" negative tests and were correctly skipped for expected_output/assertions/metrics/comparison on both variants \u2014 both variants' trigger_fired was 1/1 (correctly did not fire the behavior), so these are clean passes with no variant differentiation to report.\n\nAcross the 9 remaining chat evals, trigger_fired was 1/1 (100%) for all except \"verification-unknown-source,\" where it was 0/1 \u2014 note this trigger_fired metric is only measured once (not per-variant), so it reflects the overall test setup rather than a variant-specific result.\n\n**Expected output (met expectation):**\n- Both variants matched in 5 of 9 evals (dx-path-selection-order, tgw-vs-vgw-vif-transit, tgw-ecmp-behavior, dx-vpn-redundancy-preference, verification-unknown-source) \u2014 both 100%.\n- with_skill failed expected_output while without_skill passed in 2 evals: dx-prefer-one-location (with_skill's assertions are null/inconclusive here, limiting what can be concluded) and cloudwan-overlapping-supernet.\n- without_skill failed while with_skill passed in 2 evals: cloudwan-local-dx-preference and tgw-vpn-summarization.\n- Net effect: expected-output mismatches split roughly evenly (2 favoring each variant), so neither variant shows a clear overall edge on meeting expected output across the full set.\n\n**Assertions (per-criterion):**\n- Many assertions passed for both variants (\"always_passes\" pattern) \u2014 e.g., longest-prefix-match references, non-error checks, ECMP behavior statements, DX-over-VPN preference \u2014 these don't differentiate the variants.\n- Some assertions flipped: e.g., in cloudwan-local-dx-preference, without_skill passed the AS-path/CNE assertion while with_skill failed it, but with_skill passed the LP-communities assertion while without_skill failed it \u2014 a direct split within the same eval.\n- with_skill passed more individual assertions than without_skill in several evals (dx-path-selection-order 4/4 vs 3/4; tgw-vs-vgw-vif-transit 4/4 vs 3/4; tgw-vpn-summarization 4/4 vs 2/4; cloudwan-overlapping-supernet 4/4 vs 3/4), largely driven by without_skill missing specific recommendations (LP communities, VPN CloudHub, route-propagation/static-route details, Cloud WAN routing-policy filtering).\n- In dx-prefer-one-location, with_skill's assertion results are null (inconclusive pattern) due to the expected_output failure, so no assertion-level comparison is possible there.\n\n**Quality comparisons (LLM pairwise, where evaluated):** In the 4 evals where quality pairs were evaluated (dx-path-selection-order, tgw-vs-vgw-vif-transit, tgw-ecmp-behavior, dx-vpn-redundancy-preference), with_skill was favored on all three criteria (accuracy, actionability, completeness) in all 4 cases, with confidence ranging medium to high. The other 5 evals had quality comparisons skipped (1 pair skipped each), so no quality signal is available for those.\n\n**Runtime/cost:** Mixed and generally small in absolute terms. without_skill was often faster/cheaper (e.g., cloudwan-local-dx-preference: 16s/$0.14 vs 23s/$0.20; tgw-vs-vgw-vif-transit: 6s/$0.05 vs 11s/$0.09; tgw-ecmp-behavior: 7s/$0.07 vs 13s/$0.12; cloudwan-overlapping-supernet: 23s/$0.20 vs with_skill's notably longer 1m17s/$0.64), but with_skill was faster/cheaper in a couple of cases (dx-vpn-redundancy-preference: 21s/$0.18 vs 28s/$0.24; tgw-vpn-summarization: 17s/$0.14 vs without_skill's 39s/$0.33). No runtime threshold breaches were recorded in the evals where this was tracked.\n\n**Output consistency:** Not measured (null/overall not computed) for either variant across all evals, so no run-to-run consistency signal is available in this dataset.\n\n**Overall state:** The two variants show a mixed, non-dominant pattern: expected-output mismatches split evenly (2 vs 2), assertion-level results tend to favor with_skill slightly more often (particularly around missing specific recommendations like LP communities, VPN CloudHub, and routing-policy mentions in without_skill's outputs), and quality-comparison pairs (where available, only 4 of 9 evals) consistently favored with_skill on accuracy/actionability/completeness. Meanwhile without_skill was frequently faster and cheaper, with one case (cloudwan-overlapping-supernet) showing a large runtime/cost gap favoring without_skill. No single dimension is dominant, and with quality comparisons available for under half the evals and output consistency entirely unmeasured, the overall picture should be read as a set of tradeoffs rather than a conclusive difference between the variants.", + "timestamp": "2026-10-27T22:07:33Z", + "version": "v1", + "model": "us.anthropic.claude-sonnet-5", + "iterations": 1, + "total_evals": 11, + "total_iterations_run": 20, + "total_failed": 0, + "cleanup_skipped": false, + "understanding_agent_space_skill": { + "enabled": false, + "found": 0, + "timed_out": 0, + "errored": 0, + "not_waited": 20 + }, + "evals": { + "cloudwan-local-dx-preference": { + "summary": "This evaluation covers a single chat task about Cloud WAN route evaluation/BGP behavior, so it reflects one run per variant rather than an aggregate across many trials \u2014 conclusions should be read as a single data point, not a trend.\n\nTrigger: the behavior under test fired successfully (1/1), so the scenario was exercised as intended.\n\nExpected output match: with_skill met the expected output (1/1, 100%, high confidence). without_skill did not meet the expected output (0/1, 0%, medium confidence). The medium confidence on without_skill's judgment means this miss is fairly but not completely certain.\n\nAssertions: both variants passed 3 of 4 assertions (75%), but the pattern behind that tie is important and not visible from the aggregate:\n- One assertion (\"references longest prefix match...\") passed for both variants \u2014 this check doesn't differentiate them.\n- One assertion (\"does not contain 'error'\") also passed for both \u2014 again not discriminating.\n- One assertion (\"explains that AS-path length is evaluated at the CNE...\") passed for without_skill but failed for with_skill (with_skill's failure was at medium confidence).\n- One assertion (\"recommends LP communities as a deterministic remedy\") passed for with_skill but failed for without_skill (both at high confidence).\nSo the equal 75% pass rate masks a split: each variant failed a different specific technical point, meaning they have complementary gaps rather than one being strictly stronger on content coverage.\n\nRuntime and cost: with_skill took ~23s and cost ~$0.20; without_skill took ~16s and cost ~$0.14. Variant_b was faster and cheaper by a modest margin (~7s, ~$0.06). Context window utilization was low and similar for both (~5%), with no compaction events for either.\n\nQuality comparison: no comparable pairs were evaluated (0 pairs evaluated, 1 skipped), so there is no direct head-to-head quality judgment available for this task.\n\nOutput consistency: not measured (null) for both variants, so repeat-run stability cannot be assessed here.\n\nOverall: the two variants are roughly tied on assertion pass rate but diverge in opposite directions on two specific technical points, and only with_skill satisfied the overall expected-output criterion for this task (with the without_skill miss judged at medium confidence). Variant_b was somewhat faster and cheaper. With quality-comparison data skipped and consistency unmeasured, this is a narrow, single-instance result that highlights specific content gaps on each side rather than a clear overall difference.", + "task_type": "chat", + "trigger": { + "with_skill": { + "pass_rate": "1/1", + "percentage": 100 + }, + "without_skill": null + }, + "expected_output": { + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "low_confidence": 0, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "0/1", + "percentage": 0, + "low_confidence": 0, + "avg_confidence": "medium" + } + }, + "assertions": { + "per_assertion": [ + { + "index": 0, + "text": "references longest prefix match in Cloud WAN route evaluation", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "always_passes" + }, + { + "index": 1, + "text": "explains that AS-path length is evaluated at the CNE and can cause a remote path to win", + "evaluator": "llm", + "with_skill": { + "pass_rate": "0/1", + "percentage": 0, + "avg_confidence": "medium" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "skill_regression" + }, + { + "index": 2, + "text": "recommends LP communities (7224:7300/7200/7100) as a deterministic remedy", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "0/1", + "percentage": 0, + "avg_confidence": "high" + }, + "classification": "skill_uplift" + }, + { + "index": 3, + "text": "does not contain 'error'", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "always_passes" + } + ], + "with_skill": { + "pass_rate": "3/4", + "percentage": 75 + }, + "without_skill": { + "pass_rate": "3/4", + "percentage": 75 + } + }, + "metrics": { + "with_skill": { + "runtime_avg": "23s", + "cost_avg": "$0.20", + "context_window_avg": { + "utilization": "5.1%", + "compaction_count": 0 + } + }, + "without_skill": { + "runtime_avg": "16s", + "cost_avg": "$0.14", + "context_window_avg": { + "utilization": "4.7%", + "compaction_count": 0 + } + } + }, + "comparison": { + "quality": { + "pairs_evaluated": 0, + "pairs_skipped": 1, + "skip_reasons": [ + "without_skill expected_output test failed" + ], + "overall": { + "with_skill_wins": 0, + "without_skill_wins": 0, + "equivalent": 0, + "winner": "equivalent", + "summary": "Equivalent overall (0 vs 0 criterion wins out of 0 judgments)." + }, + "per_criterion": {}, + "iterations": [ + { + "iteration": 1, + "skipped": true, + "skip_reason": "without_skill expected_output test failed" + } + ] + } + }, + "output_consistency": { + "comparison": { + "applicable": false, + "reason": "with_skill was not judged: Only 1 iteration(s) configured; consistency needs at least 3." + }, + "with_skill": { + "skipped": true, + "skip_reason": "Only 1 iteration(s) configured; consistency needs at least 3." + }, + "without_skill": { + "skipped": true, + "skip_reason": "Only 1 iteration(s) configured; consistency needs at least 3." + } + } + }, + "dx-path-selection-order": { + "summary": "This evaluation covers a single chat task (BGP route-selection explanation) with one run per variant, so all findings here are based on a very small sample and should be read as directional rather than statistically robust.\n\nTrigger and expected output: The behavior under test fired as expected, and both without_skill and with_skill fully met the defined expected-output criterion (1/1, high confidence for both) \u2014 this dimension shows no difference between them.\n\nAssertions: Both variants passed 3 of the same 4 checks: stating the correct tie-break order (longest prefix match, local preference, AS_PATH, MED, ECMP), noting MED as a low-priority tiebreaker, and avoiding the word \"error.\" These three are passed by both variants and therefore don't differentiate them. The one assertion that did differentiate \u2014 mentioning the specific LP communities (7224:7100/7200/7300) \u2014 was passed by with_skill (1/1) and failed by without_skill (0/1), with high confidence on both sides. This gives with_skill a 4/4 (100%) assertion pass rate versus without_skill's 3/4 (75%).\n\nQuality comparison: For the single pair evaluated, with_skill was favored over without_skill on all three judged criteria (accuracy, actionability, completeness), with no criterion favoring without_skill and none rated as comparable. The confidence in this quality judgment is medium, and it rests on only one pair, so it should be treated as a limited but consistent signal rather than a definitive result.\n\nRuntime and cost: without_skill was slightly faster (10s vs 12s) and slightly cheaper ($0.09 vs $0.10) than with_skill. Both are well under the 30s runtime threshold, and neither run exceeded it. Context window utilization was low for both (4.2% vs 4.9%), with no compaction needed for either \u2014 a non-issue on this dimension.\n\nOutput consistency: No consistency data is available for either variant (marked null/overall), so nothing can be said about run-to-run stability from this evaluation.\n\nOverall picture: On this single task, the two variants were equivalent on trigger firing, expected output, and three of four assertions, and close in runtime/cost (without_skill modestly faster/cheaper). The clearest differentiator is the missed specific detail (LP community values) in without_skill's answer, which also aligns with with_skill being favored across all three quality criteria in the one pair judged. Given the very small sample (one task, one pair), this should be viewed as a limited but internally consistent signal favoring more complete/accurate detail in with_skill's output, traded against a small runtime/cost edge for without_skill, rather than a broad or highly-confident verdict.", + "task_type": "chat", + "trigger": { + "with_skill": { + "pass_rate": "1/1", + "percentage": 100 + }, + "without_skill": null + }, + "expected_output": { + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "low_confidence": 0, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "low_confidence": 0, + "avg_confidence": "high" + } + }, + "assertions": { + "per_assertion": [ + { + "index": 0, + "text": "states the order longest prefix match, local preference, AS_PATH, MED, ECMP", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "always_passes" + }, + { + "index": 1, + "text": "notes MED is a low-priority tiebreaker", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "always_passes" + }, + { + "index": 2, + "text": "mentions LP communities 7224:7100/7200/7300", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "0/1", + "percentage": 0, + "avg_confidence": "high" + }, + "classification": "skill_uplift" + }, + { + "index": 3, + "text": "does not contain 'error'", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "always_passes" + } + ], + "with_skill": { + "pass_rate": "4/4", + "percentage": 100 + }, + "without_skill": { + "pass_rate": "3/4", + "percentage": 75 + } + }, + "metrics": { + "with_skill": { + "runtime_avg": "12s", + "cost_avg": "$0.10", + "context_window_avg": { + "utilization": "4.9%", + "compaction_count": 0 + } + }, + "without_skill": { + "runtime_avg": "10s", + "cost_avg": "$0.09", + "context_window_avg": { + "utilization": "4.2%", + "compaction_count": 0 + } + } + }, + "comparison": { + "quality": { + "pairs_evaluated": 1, + "pairs_skipped": 0, + "skip_reasons": [], + "overall": { + "with_skill_wins": 3, + "without_skill_wins": 0, + "equivalent": 0, + "winner": "with_skill", + "avg_confidence": "medium", + "summary": "with_skill wins overall (3 vs 0 criterion wins out of 3 judgments, avg confidence: medium)." + }, + "per_criterion": { + "accuracy": { + "with_skill_wins": 1, + "without_skill_wins": 0, + "equivalent": 0 + }, + "actionability": { + "with_skill_wins": 1, + "without_skill_wins": 0, + "equivalent": 0 + }, + "completeness": { + "with_skill_wins": 1, + "without_skill_wins": 0, + "equivalent": 0 + } + }, + "iterations": [ + { + "iteration": 1, + "overall_winner": "with_skill", + "position_assignment": "with_skill=output_b", + "criteria": [ + { + "criterion": "accuracy", + "winner": "with_skill", + "confidence": "medium", + "reasoning": "Both outputs correctly identify the path-selection order (longest prefix match > local preference > AS_PATH length > MED > ECMP) and correctly place MED as second-to-last. However, Output B provides more precise and accurate detail on the local preference BGP communities, listing all three tiers (High 7224:7300, Medium 7224:7200, Low 7224:7100), which matches AWS's documented DX LP community values. Output A only mentions 7224:7200 as 'medium local preference for same-Region locations, lower otherwise' and frames local preference as being automatically set based on region association, which is a less complete/accurate characterization since AWS's default behavior for local preference is based on region matching, but the explicit community value scheme is more fully captured by Output B. Output B also adds a nuanced and accurate detail about AS-path prepending only working reliably within the same associated region, which is a more sophisticated and correct point that enhances accuracy of the overall picture." + }, + { + "criterion": "actionability", + "winner": "with_skill", + "confidence": "high", + "reasoning": "Output B gives clearer actionable guidance: it explicitly names LP communities as the right lever for deterministic steering, explains why AS-path prepending has limitations (only within-region), and ends with an offer to help trace actual path-selection decisions or check LP communities/AS-path currently advertised\u2014providing a clear next step. Output A mentions longest prefix match and local preference as 'preferred levers' but doesn't go into the same level of actionable depth, and does not offer concrete follow-up actions as explicitly as Output B does." + }, + { + "criterion": "completeness", + "winner": "with_skill", + "confidence": "medium", + "reasoning": "Both cover the path-selection order, MED placement, and the public VIF distinction. Output B provides additional valuable details: the full LP community table (High/Medium/Low), the nuance about AS-path prepending working only within the same associated region vs. LP communities working both within and across regions, and an offer for further assistance. Output A mentions ECMP ASN matching nuance (not needing matching ASNs for ECMP), which Output B does not explicitly state, but this is a minor point compared to the additional depth Output B provides on local preference communities and prepending limitations impacting cross-region steering, which is a core aspect of understanding path selection fully. Overall Output B covers more ground with greater depth, though Output A is not wrong, just less complete in certain areas." + } + ] + } + ] + }, + "runtime": { + "pairs_evaluated": 1, + "pairs_skipped": 0, + "threshold_exceeded_count": 0, + "threshold_seconds": 30.0, + "iterations": [ + { + "iteration": 1, + "with_skill_seconds": 12.071, + "without_skill_seconds": 10.706, + "delta_seconds": 1.4, + "threshold_exceeded": false + } + ] + } + }, + "output_consistency": { + "comparison": { + "applicable": false, + "reason": "with_skill was not judged: Only 1 iteration(s) configured; consistency needs at least 3." + }, + "with_skill": { + "skipped": true, + "skip_reason": "Only 1 iteration(s) configured; consistency needs at least 3." + }, + "without_skill": { + "skipped": true, + "skip_reason": "Only 1 iteration(s) configured; consistency needs at least 3." + } + } + }, + "tgw-vs-vgw-vif-transit": { + "summary": "This evaluation covers a single chat task (one trigger firing, one pair evaluated), so all findings here rest on a very small sample and should be read as a single data point rather than a robust trend.\n\nBoth variants met the expected output (1/1), though the confidence behind that judgment was \"high\" for with_skill and only \"medium\" for without_skill, meaning the expected-output match for without_skill is a slightly weaker measurement.\n\nOn assertions, with_skill passed all 4/4, while without_skill passed 3/4 (75%). Breaking this down: three assertions (\"explains AWS does not act as a transit hub by default,\" \"mentions TGW or two DXGWs...,\" and \"does not contain 'error'\") passed for both variants \u2014 these are not differentiating and suggest the underlying behavior is solid for both. The one assertion that split the variants was \"mentions VPN CloudHub for VPN connections,\" which with_skill passed and without_skill failed (both with high evaluator confidence), indicating a concrete, specific content gap in without_skill's response on this task.\n\nOn the head-to-head quality comparison (one pair, high confidence), with_skill was favored across all three criteria assessed \u2014 accuracy, actionability, and completeness \u2014 with no criterion favoring without_skill or rated comparable. Given it's a single pair, this is a consistent but not independently repeated signal.\n\nOn efficiency, without_skill was notably faster (~6s vs ~11s) and cheaper (~$0.05 vs ~$0.09), with slightly lower context-window utilization (4.2% vs 4.9%); neither variant triggered any context compaction. No runs exceeded the runtime threshold for either variant.\n\nOutput consistency across repeated runs was not available (null) for either variant, so repeatability cannot be assessed from this data.\n\nOverall, this is a single-task snapshot where with_skill showed higher assertion pass rate, a specific content advantage (VPN CloudHub), and was favored on all quality criteria, while without_skill ran faster and cheaper. Whether this reflects a general pattern or a one-off task-specific difference cannot be determined from one evaluated pair.", + "task_type": "chat", + "trigger": { + "with_skill": { + "pass_rate": "1/1", + "percentage": 100 + }, + "without_skill": null + }, + "expected_output": { + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "low_confidence": 0, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "low_confidence": 0, + "avg_confidence": "medium" + } + }, + "assertions": { + "per_assertion": [ + { + "index": 0, + "text": "explains AWS does not act as a transit hub by default", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "always_passes" + }, + { + "index": 1, + "text": "mentions VPN CloudHub for VPN connections", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "0/1", + "percentage": 0, + "avg_confidence": "high" + }, + "classification": "skill_uplift" + }, + { + "index": 2, + "text": "mentions TGW or two DXGWs as a way to enable DX-to-DX intentionally", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "always_passes" + }, + { + "index": 3, + "text": "does not contain 'error'", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "always_passes" + } + ], + "with_skill": { + "pass_rate": "4/4", + "percentage": 100 + }, + "without_skill": { + "pass_rate": "3/4", + "percentage": 75 + } + }, + "metrics": { + "with_skill": { + "runtime_avg": "11s", + "cost_avg": "$0.09", + "context_window_avg": { + "utilization": "4.9%", + "compaction_count": 0 + } + }, + "without_skill": { + "runtime_avg": "6s", + "cost_avg": "$0.05", + "context_window_avg": { + "utilization": "4.2%", + "compaction_count": 0 + } + } + }, + "comparison": { + "quality": { + "pairs_evaluated": 1, + "pairs_skipped": 0, + "skip_reasons": [], + "overall": { + "with_skill_wins": 3, + "without_skill_wins": 0, + "equivalent": 0, + "winner": "with_skill", + "avg_confidence": "high", + "summary": "with_skill wins overall (3 vs 0 criterion wins out of 3 judgments, avg confidence: high)." + }, + "per_criterion": { + "accuracy": { + "with_skill_wins": 1, + "without_skill_wins": 0, + "equivalent": 0 + }, + "actionability": { + "with_skill_wins": 1, + "without_skill_wins": 0, + "equivalent": 0 + }, + "completeness": { + "with_skill_wins": 1, + "without_skill_wins": 0, + "equivalent": 0 + } + }, + "iterations": [ + { + "iteration": 1, + "overall_winner": "with_skill", + "position_assignment": "with_skill=output_b", + "criteria": [ + { + "criterion": "accuracy", + "winner": "with_skill", + "confidence": "high", + "reasoning": "Output A makes a technically inaccurate claim that 'Transit VIFs + Direct Connect Gateway + Transit Gateway does work' while implying private VIFs with a shared DXGW won't work, but it doesn't clearly explain why a shared DXGW (without TGW) fails, and it glosses over the fact that even with a DXGW, you need a TGW or VPN CloudHub to get site-to-site connectivity - a single DXGW on its own (without TGW) does not enable site-to-site routing. Output A's framing 'Transit VIFs tied into the same Direct Connect Gateway' as sufficient is slightly misleading since it's the Transit Gateway doing the hub routing, not the DXGW itself, and Output A doesn't clarify this nuance as clearly. Output B is more precise and accurate: it correctly states that a single VGW or single DXGW only advertises VPC CIDRs and does not enable site-to-site communication, and correctly specifies that you need two DXGWs attached to a shared TGW, plus Allowed Prefix Lists to advertise on-prem prefixes between sites. Output B also correctly notes VPN CloudHub behavior as an alternative DX+VPN scenario. Output B's technical details are more rigorous and align with AWS's documented hybrid networking behavior." + }, + { + "criterion": "actionability", + "winner": "with_skill", + "confidence": "high", + "reasoning": "Both outputs end with a similar offer to help with the specific setup. However, Output B provides a clearer, more specific action pattern (two DXGWs + shared TGW + Allowed Prefix List) and explicitly mentions the need for Allowed Prefix Lists to control advertisement, which is a critical implementation detail for achieving site-to-site connectivity. Output A's actionable guidance is vaguer, telling the user to use 'Transit VIFs + Direct Connect Gateway + Transit Gateway' without detailing the need for prefix lists or two separate DXGWs, which could lead to incomplete implementation." + }, + { + "criterion": "completeness", + "winner": "with_skill", + "confidence": "high", + "reasoning": "Output B covers more scenarios and nuances: it explicitly states what does NOT work (same VGW, same DXGW) and provides four distinct patterns that DO work (Two DXGWs into one TGW, TGW + Allowed Prefix List nuance, DX+VPN CloudHub, customer-side hairpin). Output A only covers the private VIF (doesn't work) vs Transit VIF+DXGW+TGW (works) scenario, missing alternative patterns like VPN CloudHub or customer-side hairpin solutions, and doesn't mention the critical role of Allowed Prefix Lists in advertising on-premises prefixes." + } + ] + } + ] + }, + "runtime": { + "pairs_evaluated": 1, + "pairs_skipped": 0, + "threshold_exceeded_count": 0, + "threshold_seconds": 30.0, + "iterations": [ + { + "iteration": 1, + "with_skill_seconds": 11.346, + "without_skill_seconds": 6.501, + "delta_seconds": 4.8, + "threshold_exceeded": false + } + ] + } + }, + "output_consistency": { + "comparison": { + "applicable": false, + "reason": "with_skill was not judged: Only 1 iteration(s) configured; consistency needs at least 3." + }, + "with_skill": { + "skipped": true, + "skip_reason": "Only 1 iteration(s) configured; consistency needs at least 3." + }, + "without_skill": { + "skipped": true, + "skip_reason": "Only 1 iteration(s) configured; consistency needs at least 3." + } + } + }, + "dx-prefer-one-location": { + "summary": "This evaluation covered a single chat task about multi-region BGP/AS-path prepending advice. The trigger for the behavior under test fired successfully (1/1).\n\nExpected output: with_skill did not meet the expected output (0/1), while without_skill did (1/1, with high confidence in that judgment). This is the clearest differentiator in this dataset, though it is based on a single task instance, so it should be treated as a limited data point rather than a broad pattern.\n\nAssertions: no per-assertion data exists for with_skill (null/not evaluated), while without_skill passed all 4 assertions it was checked against (recommending LP communities with higher preference on the primary, explaining AS-path prepending is reliable only within the same region, warning that cross-region prepending is unreliable, and not containing the word \"error\"). Confidence on these assertion judgments for without_skill ranged from medium to high. Because with_skill has no assertion results to compare, we can't say whether it failed these same checks or simply wasn't scored on them \u2014 this is an absence of data, not evidence of poor performance on those specific points.\n\nQuality comparison: 0 pairs were evaluated and 1 pair was skipped, so there is no head-to-head quality signal available for this task.\n\nRuntime and cost: the two variants were very close \u2014 with_skill ran about 16s at $0.14 average cost, without_skill ran about 15s at $0.13, with slightly lower context-window utilization for without_skill (4.2% vs 4.9%). These differences are small and not practically meaningful on their own.\n\nOutput consistency: no consistency data (overall score) was recorded for either variant, so repeat-run stability cannot be assessed here.\n\nOverall, the main signal in this result set is the expected-output outcome (without_skill met it, with_skill did not), paired with a full set of passing assertions for without_skill that have no counterpart data for with_skill. Runtime/cost differences are negligible, and quality-comparison and consistency data are both absent, so this summary rests primarily on a single expected-output judgment and should be weighed with that limited scope in mind.", + "task_type": "chat", + "trigger": { + "with_skill": { + "pass_rate": "1/1", + "percentage": 100 + }, + "without_skill": null + }, + "expected_output": { + "with_skill": { + "pass_rate": "0/1", + "percentage": 0, + "low_confidence": 0 + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "low_confidence": 0, + "avg_confidence": "high" + } + }, + "assertions": { + "per_assertion": [ + { + "index": 0, + "text": "recommends LP communities with a higher preference on the primary", + "evaluator": "llm", + "with_skill": null, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "inconclusive" + }, + { + "index": 1, + "text": "explains AS-path prepending is reliable only within the same region", + "evaluator": "llm", + "with_skill": null, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "medium" + }, + "classification": "inconclusive" + }, + { + "index": 2, + "text": "warns cross-region AS-path prepending is unreliable", + "evaluator": "llm", + "with_skill": null, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "inconclusive" + }, + { + "index": 3, + "text": "does not contain 'error'", + "evaluator": "llm", + "with_skill": null, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "inconclusive" + } + ], + "with_skill": null, + "without_skill": { + "pass_rate": "4/4", + "percentage": 100 + } + }, + "metrics": { + "with_skill": { + "runtime_avg": "16s", + "cost_avg": "$0.14", + "context_window_avg": { + "utilization": "4.9%", + "compaction_count": 0 + } + }, + "without_skill": { + "runtime_avg": "15s", + "cost_avg": "$0.13", + "context_window_avg": { + "utilization": "4.2%", + "compaction_count": 0 + } + } + }, + "comparison": { + "quality": { + "pairs_evaluated": 0, + "pairs_skipped": 1, + "skip_reasons": [ + "with_skill expected_output test failed" + ], + "overall": { + "with_skill_wins": 0, + "without_skill_wins": 0, + "equivalent": 0, + "winner": "equivalent", + "summary": "Equivalent overall (0 vs 0 criterion wins out of 0 judgments)." + }, + "per_criterion": {}, + "iterations": [ + { + "iteration": 1, + "skipped": true, + "skip_reason": "with_skill expected_output test failed" + } + ] + } + }, + "output_consistency": { + "comparison": { + "applicable": false, + "reason": "with_skill was not judged: Only 1 iteration(s) configured; consistency needs at least 3." + }, + "with_skill": { + "skipped": true, + "skip_reason": "Only 1 iteration(s) configured; consistency needs at least 3." + }, + "without_skill": { + "skipped": true, + "skip_reason": "Only 1 iteration(s) configured; consistency needs at least 3." + } + } + }, + "tgw-ecmp-behavior": { + "summary": "This evaluation covers a single chat-task scenario about AWS Transit Gateway ECMP behavior, so all findings rest on just one evaluated pair \u2014 a very small sample that limits how much can be generalized.\n\nTrigger and expected output: The behavior under test fired as expected (1/1), and both with_skill and without_skill fully met the expected output (1/1, 100%), with \"medium\" confidence in that judgment for both \u2014 meaning the measurement is reasonably but not highly certain.\n\nAssertions: Both variants passed all three assertions (3/3, 100% each), with high confidence on each. Notably, every individual assertion \u2014 correctly describing ECMP's attachment-type/AS-path requirements, noting TGW's single-route fallback, and avoiding the word \"error\" \u2014 passed for both variants identically. Since none of the assertions differentiate the two variants, this dimension shows no meaningful contrast between them; it mainly confirms both outputs are factually and structurally sound.\n\nQuality comparison: Here a difference appears. Across the one evaluated pair, with_skill was favored on all three quality criteria (accuracy, actionability, and completeness), with without_skill favored on none, at \"medium\" confidence. Since this rests on a single pair, it's a real but limited signal, not a robust pattern.\n\nRuntime and cost: with_skill took about 13s and cost $0.12 on average, while without_skill was faster and cheaper at about 7s and $0.07 \u2014 roughly half the time and cost of with_skill. Neither exceeded the 30s runtime threshold. Context window utilization was low and similar for both (4.7% vs 4.1%), with no compaction events for either.\n\nOutput consistency: No consistency data is available for either variant (marked null), so run-to-run stability cannot be assessed here.\n\nOverall, this is a single-scenario, low-volume result. The two variants were functionally equivalent on correctness (expected output and all assertions passed identically), but diverged on quality (with_skill favored across all criteria in the one comparison) and on efficiency (without_skill ran about twice as fast and cost about half as much). This presents as a tradeoff between quality and efficiency rather than a clear overall difference, and the small sample size (one pair) means confidence in the quality gap should be treated cautiously.", + "task_type": "chat", + "trigger": { + "with_skill": { + "pass_rate": "1/1", + "percentage": 100 + }, + "without_skill": null + }, + "expected_output": { + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "low_confidence": 0, + "avg_confidence": "medium" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "low_confidence": 0, + "avg_confidence": "medium" + } + }, + "assertions": { + "per_assertion": [ + { + "index": 0, + "text": "states ECMP requires same attachment type and equal AS-path/BGP attributes", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "always_passes" + }, + { + "index": 1, + "text": "notes TGW picks a single route when ECMP conditions are not met", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "always_passes" + }, + { + "index": 2, + "text": "does not contain 'error'", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "always_passes" + } + ], + "with_skill": { + "pass_rate": "3/3", + "percentage": 100 + }, + "without_skill": { + "pass_rate": "3/3", + "percentage": 100 + } + }, + "metrics": { + "with_skill": { + "runtime_avg": "13s", + "cost_avg": "$0.12", + "context_window_avg": { + "utilization": "4.7%", + "compaction_count": 0 + } + }, + "without_skill": { + "runtime_avg": "7s", + "cost_avg": "$0.07", + "context_window_avg": { + "utilization": "4.1%", + "compaction_count": 0 + } + } + }, + "comparison": { + "quality": { + "pairs_evaluated": 1, + "pairs_skipped": 0, + "skip_reasons": [], + "overall": { + "with_skill_wins": 3, + "without_skill_wins": 0, + "equivalent": 0, + "winner": "with_skill", + "avg_confidence": "medium", + "summary": "with_skill wins overall (3 vs 0 criterion wins out of 3 judgments, avg confidence: medium)." + }, + "per_criterion": { + "accuracy": { + "with_skill_wins": 1, + "without_skill_wins": 0, + "equivalent": 0 + }, + "actionability": { + "with_skill_wins": 1, + "without_skill_wins": 0, + "equivalent": 0 + }, + "completeness": { + "with_skill_wins": 1, + "without_skill_wins": 0, + "equivalent": 0 + } + }, + "iterations": [ + { + "iteration": 1, + "overall_winner": "with_skill", + "position_assignment": "with_skill=output_a", + "criteria": [ + { + "criterion": "accuracy", + "winner": "with_skill", + "confidence": "medium", + "reasoning": "This is a technically nuanced AWS networking question with conflicting claims between the two outputs. Output A claims ECMP occurs across TWO SEPARATE DX Gateway attachments (not two VIFs on one DXGW), stating that a single DXGW only advertises one best path per prefix regardless of VIF count. Output B claims the opposite: that ECMP occurs when two VIFs are attached to the SAME DX Gateway, and that separate DX Gateways do NOT support ECMP. According to AWS documentation on Transit Gateway and Direct Connect Gateway, Direct Connect Gateway performs its own best-path selection among its own VIFs before advertising a single route to the Transit Gateway via its DXGW attachment. ECMP across DX paths at the TGW level actually occurs when there are multiple separate DX Gateway attachments (each representing a distinct best path from its own VIF set) advertising equal-cost routes to the TGW with identical AS-paths/ASNs. This aligns with Output A's description. Output B's claim that ECMP happens among multiple VIFs on a single DXGW contradicts how DXGW internally resolves to a single best path per prefix before forwarding to TGW. Output A's explanation is more consistent with known AWS behavior (DXGW active/passive internal failover, TGW ECMP across multiple DXGW attachments with matching ASN/AS-path). Output B's narrative, while structured, appears to invert this mechanism and also makes an unsupported claim that AWS 'explicitly recommends using a single DX Gateway so ECMP kicks in' when multisite ECMP is actually the documented pattern for redundancy if other criteria are met.", + "evidence": "Output A: '...Two VIFs, one DXGW: A single DX Gateway only ever advertises one best path per prefix to the TGW, no matter how many VIFs are attached to it.' vs Output B: 'Both Direct Connect paths terminate on transit virtual interfaces attached to the same Direct Connect Gateway... In this setup, Transit Gateway automatically ECMPs traffic across the multiple transit VIFs' and 'separate Direct Connect Gateways rather than a single shared one \u2014 AWS explicitly recommends using a single Direct Connect Gateway so ECMP kicks in'." + }, + { + "criterion": "actionability", + "winner": "with_skill", + "confidence": "high", + "reasoning": "Output A provides concrete, specific AWS CLI/API-style commands for validation (search-transit-gateway-routes, list-virtual-interface-routes) and offers to proactively check the user's actual TGW route table or DXGW attachments against the logic described. This gives the user clear, practical next steps to verify the behavior in their own environment. Output B ends with a summarized rule of thumb but provides no specific commands, tools, or offer to validate against the user's actual environment, making it comparatively less actionable.", + "evidence": "Output A: '**To validate in your account** (read-only): `search-transit-gateway-routes` \u2014 see the per-prefix candidates... `list-virtual-interface-routes` \u2014 see what each DXGW's VIF is actually advertising internally. Want me to check a specific TGW's route table or DXGW attachments against this logic?' vs Output B: 'So the practical rule of thumb: one DX Gateway, two transit VIFs, identical advertised routes \u2192 ECMP load-balancing...' (no validation steps or offer to investigate the user's specific setup)." + }, + { + "criterion": "completeness", + "winner": "with_skill", + "confidence": "medium", + "reasoning": "Both outputs cover the core ECMP conditions (matching AS-path/ASN) and the 'picks one' scenarios, but Output A additionally addresses an important edge case not mentioned in Output B: the interaction between DX and other attachment types (e.g., VPN) competing for the same prefix, noting that TGS never ECMPs across different attachment types and that static VPN outranks DX by default. Output A also includes a valuable nuance distinguishing DX-side ECMP across VIFs (where ASN matching isn't required) from TGW-level ECMP across DXGW attachments (where ASN matching is required), which adds depth and guards against a common point of confusion. Output B does not mention the DX-vs-VPN attachment-type scenario or the DX-side vs TGW-side distinction, making it less comprehensive, despite being reasonably thorough on its own stated model.", + "evidence": "Output A: 'DX vs. a different attachment type for the same prefix (e.g., DX-GW vs. VPN): TGW never ECMPs across different attachment types... notably, static VPN outranks DX by default.' and 'One nuance worth flagging: the \"ASNs must match, not just AS-path length\" requirement is specific to TGW ECMP across DXGW attachments. It's different from DX-side ECMP across VIFs within a region...' \u2014 Output B has no equivalent content addressing cross-attachment-type behavior or the DX-side vs TGW-side ECMP distinction." + } + ] + } + ] + }, + "runtime": { + "pairs_evaluated": 1, + "pairs_skipped": 0, + "threshold_exceeded_count": 0, + "threshold_seconds": 30.0, + "iterations": [ + { + "iteration": 1, + "with_skill_seconds": 13.962, + "without_skill_seconds": 7.838, + "delta_seconds": 6.1, + "threshold_exceeded": false + } + ] + } + }, + "output_consistency": { + "comparison": { + "applicable": false, + "reason": "with_skill was not judged: Only 1 iteration(s) configured; consistency needs at least 3." + }, + "with_skill": { + "skipped": true, + "skip_reason": "Only 1 iteration(s) configured; consistency needs at least 3." + }, + "without_skill": { + "skipped": true, + "skip_reason": "Only 1 iteration(s) configured; consistency needs at least 3." + } + } + }, + "dx-vpn-redundancy-preference": { + "summary": "This evaluation covers a single chat-task scenario about DX vs. VPN routing preference at a VGW, so all findings rest on just one evaluated pair \u2014 confidence in generalizing these results should be limited.\n\nTrigger and expected output: The behavior under test fired as expected (1/1), and both with_skill and without_skill fully met the expected output (1/1, 100%) with high-confidence judgments on both sides \u2014 no difference here.\n\nAssertions: Both variants passed all three assertions (3/3, 100% each), and every individual assertion passed for both variants with high confidence: stating DX is preferred over VPN for the same prefix, identifying asymmetric routing as the main risk, and avoiding the word \"error.\" Because these assertions passed uniformly for both variants, they don't differentiate the two systems \u2014 they mainly confirm both handled the core task requirements correctly.\n\nQuality comparison: Here is where a difference appears. Across all three judged criteria (accuracy, actionability, completeness), with_skill was favored over without_skill in the single pair evaluated, with high average confidence. Variant_b was not favored on any criterion, and nothing was rated comparable. This is a single-pair result, so while the signal is consistent across criteria, it comes from only one comparison.\n\nRuntime and cost: with_skill ran faster (~21s vs ~28s, about 7s less) and was cheaper (~$0.18 vs ~$0.24). Neither run exceeded the 30s runtime threshold. Context window utilization was low and similar for both (~4.7% vs ~4.5%), with no compaction events for either.\n\nOutput consistency: No consistency data was available for either variant (marked null), so run-to-run stability cannot be assessed from this evaluation.\n\nOverall state: On this single task, with_skill and without_skill performed equivalently on trigger firing, expected-output matching, and assertion pass rates \u2014 all identical and at ceiling. The only differentiating signal is the quality comparison, where with_skill was favored on all three criteria with high confidence, while also running somewhat faster and cheaper than without_skill. Given this rests on just one evaluated pair with no consistency data, it should be read as a single-instance observation rather than a robust, generalizable pattern.", + "task_type": "chat", + "trigger": { + "with_skill": { + "pass_rate": "1/1", + "percentage": 100 + }, + "without_skill": null + }, + "expected_output": { + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "low_confidence": 0, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "low_confidence": 0, + "avg_confidence": "high" + } + }, + "assertions": { + "per_assertion": [ + { + "index": 0, + "text": "states DX is always preferred over VPN for the same prefix at a VGW", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "always_passes" + }, + { + "index": 1, + "text": "identifies asymmetric routing as the main risk", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "always_passes" + }, + { + "index": 2, + "text": "does not contain 'error'", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "always_passes" + } + ], + "with_skill": { + "pass_rate": "3/3", + "percentage": 100 + }, + "without_skill": { + "pass_rate": "3/3", + "percentage": 100 + } + }, + "metrics": { + "with_skill": { + "runtime_avg": "21s", + "cost_avg": "$0.18", + "context_window_avg": { + "utilization": "4.7%", + "compaction_count": 0 + } + }, + "without_skill": { + "runtime_avg": "28s", + "cost_avg": "$0.24", + "context_window_avg": { + "utilization": "4.5%", + "compaction_count": 0 + } + } + }, + "comparison": { + "quality": { + "pairs_evaluated": 1, + "pairs_skipped": 0, + "skip_reasons": [], + "overall": { + "with_skill_wins": 3, + "without_skill_wins": 0, + "equivalent": 0, + "winner": "with_skill", + "avg_confidence": "high", + "summary": "with_skill wins overall (3 vs 0 criterion wins out of 3 judgments, avg confidence: high)." + }, + "per_criterion": { + "accuracy": { + "with_skill_wins": 1, + "without_skill_wins": 0, + "equivalent": 0 + }, + "actionability": { + "with_skill_wins": 1, + "without_skill_wins": 0, + "equivalent": 0 + }, + "completeness": { + "with_skill_wins": 1, + "without_skill_wins": 0, + "equivalent": 0 + } + }, + "iterations": [ + { + "iteration": 1, + "overall_winner": "with_skill", + "position_assignment": "with_skill=output_b", + "criteria": [ + { + "criterion": "accuracy", + "winner": "with_skill", + "confidence": "high", + "reasoning": "Both outputs correctly state that AWS always prefers Direct Connect over VPN for the same prefix and that longest-prefix-match can override this preference. However, Output B provides a more technically precise and complete explanation of AWS's actual documented route preference order for the VGW (BGP-propagated DX routes > static VPN routes > BGP-propagated VPN routes > AS-path/MED tiebreakers among VPN routes), which aligns with AWS's official VGW route selection documentation. Output B also correctly notes that VGW does not support ECMP and that VPN traffic rides a single 'Preferred Tunnel', which is an accurate and important technical detail that Output A omits entirely. Output A's claim that 'AS_PATH is the next tiebreaker after local preference' on the AWS side is a bit misleading, since this tiebreaker order applies only among VPN routes, not between DX and VPN (where the preference is hard-coded regardless of AS-path). Output A's framing could cause confusion, suggesting AS-path manipulation might influence DX vs VPN preference, which is incorrect per AWS documentation. Output B is more precise about the boundaries of what BGP attributes can and cannot influence.", + "evidence": "Output B: '1. BGP propagated routes from Direct Connect ... 2. Manually added static VPN routes ... 3. BGP propagated routes from Site-to-Site VPN ... 4. Among VPN BGP routes with matching prefixes, shortest AS-path, then lowest MED' and 'VGW doesn't support ECMP, so even on the VPN alone, return traffic always rides a single tunnel.' Output A: 'Use consistent AS_PATH length for the same prefix on both circuits if you're not using local preference, since AS_PATH is the next tiebreaker after local preference' \u2014 this is misleading since AS-path tiebreaking doesn't apply between DX and VPN, only within VPN routes." + }, + { + "criterion": "actionability", + "winner": "with_skill", + "confidence": "medium", + "reasoning": "Both outputs provide concrete, actionable steps to prevent asymmded routing (aligning CGW preference to prefer DX, avoiding more-specific VPN prefixes unless intentional, and offering to inspect actual route tables/BGP state). Output B includes an additional actionable and AWS-specific recommendation about using a CGW device that explicitly supports asymmetric routing (which is a real AWS best-practice recommendation for DX+VPN backup setups, important during VPN tunnel maintenance events) and frames the VPN as purely passive backup with clear fallback conditions BGP session drop. Output A's actions are generally good but item 4 introduces a somewhat tangential Transit Gateway caveat that isn't part of the original VGW-only setup described by the user, which could be seen as introducing unprompted complexity rather than focused actionability for the exact topology described (DX + VPN to the same VGW).", + "evidence": "Output B: 'Use a CGW device that explicitly supports asymmetric routing, per AWS's recommendation \u2014 this matters especially during automatic VPN tunnel endpoint maintenance...' and 'Treat the VPN purely as passive backup: it should only carry traffic when DX is fully down...' Output A: 'If you're running this through a Transit Gateway rather than directly at the VGW, apply route filtering on the CGW...' \u2014 introduces TGW scenario not asked about by the user, slightly diluting focus for a straightforward VGW question." + }, + { + "criterion": "completeness", + "winner": "with_skill", + "confidence": "high", + "reasoning": "Output B covers everything Output A covers (DX preference, longest-prefix-match caveat, CGW-side alignment, stateful/asymmetric routing causes) and adds several additional relevant details: the full VGW route preference hierarchy, the lack of ECMP support on VGW explaining single-tunnel VPN behavior, and specific AWS guidance about CGW devices supporting asymmetric routing during VPN tunnel maintenance. Output A does cover stateful device concerns which Output B does not explicitly mention, but this is a relatively minor point compared to the VGW route-selection hierarchy and ECMP details that Output B uniquely provides, which are core to answering 'which path does AWS prefer.'", + "evidence": "Output B's full route-preference list and the ECMP/Preferred Tunnel detail are unique additions not found in Output A: 'VGW doesn't support ECMP, so even on the VPN alone, return traffic always rides a single tunnel (the \"Preferred Tunnel\")...' Output A's unique content is mainly: 'Stateful devices (firewalls, NAT) on-prem seeing only one leg of a flow and dropping it...' which is useful but narrower in scope than the hierarchy/ECMP details B adds." + } + ] + } + ] + }, + "runtime": { + "pairs_evaluated": 1, + "pairs_skipped": 0, + "threshold_exceeded_count": 0, + "threshold_seconds": 30.0, + "iterations": [ + { + "iteration": 1, + "with_skill_seconds": 21.741, + "without_skill_seconds": 28.815, + "delta_seconds": -7.1, + "threshold_exceeded": false + } + ] + } + }, + "output_consistency": { + "comparison": { + "applicable": false, + "reason": "with_skill was not judged: Only 1 iteration(s) configured; consistency needs at least 3." + }, + "with_skill": { + "skipped": true, + "skip_reason": "Only 1 iteration(s) configured; consistency needs at least 3." + }, + "without_skill": { + "skipped": true, + "skip_reason": "Only 1 iteration(s) configured; consistency needs at least 3." + } + } + }, + "verification-unknown-source": { + "summary": "Both variants performed identically on nearly every measured dimension in this single chat task.\n\n- Expected output: Both with_skill and without_skill fully met the expected output (1/1, 100%), with high-confidence judgments and no low-confidence flags \u2014 this is a solid, trustworthy result for both.\n- Assertions: Both variants passed all three assertions (3/3, 100%). Notably, every one of the three individual assertions (\"does not fabricate a document or URL,\" \"states it cannot verify such a named artifact,\" \"points to the Direct Connect routing policies documentation\") passed for both variants and is flagged with an \"always_passes\" pattern \u2014 meaning these assertions did not differentiate the two variants at all in this run; they may reflect an easy case or a non-discriminating check rather than a meaningful distinction.\n- Trigger: The `trigger_fired` metric is 0/1 (0%), but this is a single-variant/overall diagnostic rather than a per-variant comparison, so it doesn't attribute a failure to either variant specifically \u2014 it's worth flagging as a data point but shouldn't be read as either variant underperforming.\n- Runtime and cost: Nearly identical \u2014 with_skill averaged 7s vs. without_skill's 8s (a negligible ~1s difference), and both cost $0.07 on average. Context window utilization was identical as well (4.2%, 0 compactions for both).\n- Quality comparison: No comparable pairs were evaluated (0 pairs evaluated, 1 skipped), so there is no data on relative output quality between the two variants for this task.\n- Output consistency: Not measured for either variant (both null), so repeat-run stability cannot be assessed here.\n\nOverall, this single eval shows the two variants performing essentially the same on expected output, assertions, runtime, and cost, with no meaningful quality or consistency comparison available due to skipped/missing data. There is no indication from this result that one variant outperforms the other \u2014 the differences observed (1s runtime, assertions that always pass) are too small or non-discriminating to support any directional conclusion.", + "task_type": "chat", + "trigger": { + "with_skill": { + "pass_rate": "0/1", + "percentage": 0 + }, + "without_skill": null + }, + "expected_output": { + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "low_confidence": 0, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "low_confidence": 0, + "avg_confidence": "high" + } + }, + "assertions": { + "per_assertion": [ + { + "index": 0, + "text": "does not fabricate a document or URL", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "medium" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "medium" + }, + "classification": "always_passes" + }, + { + "index": 1, + "text": "states it cannot verify such a named artifact", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "always_passes" + }, + { + "index": 2, + "text": "points to the Direct Connect routing policies documentation", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "always_passes" + } + ], + "with_skill": { + "pass_rate": "3/3", + "percentage": 100 + }, + "without_skill": { + "pass_rate": "3/3", + "percentage": 100 + } + }, + "metrics": { + "with_skill": { + "runtime_avg": "7s", + "cost_avg": "$0.07", + "context_window_avg": { + "utilization": "4.2%", + "compaction_count": 0 + } + }, + "without_skill": { + "runtime_avg": "8s", + "cost_avg": "$0.07", + "context_window_avg": { + "utilization": "4.2%", + "compaction_count": 0 + } + } + }, + "comparison": { + "quality": { + "pairs_evaluated": 0, + "pairs_skipped": 1, + "skip_reasons": [ + "with_skill trigger test failed" + ], + "overall": { + "with_skill_wins": 0, + "without_skill_wins": 0, + "equivalent": 0, + "winner": "equivalent", + "summary": "Equivalent overall (0 vs 0 criterion wins out of 0 judgments)." + }, + "per_criterion": {}, + "iterations": [ + { + "iteration": 1, + "skipped": true, + "skip_reason": "with_skill trigger test failed" + } + ] + } + }, + "output_consistency": { + "comparison": { + "applicable": false, + "reason": "with_skill was not judged: Only 1 iteration(s) configured; consistency needs at least 3." + }, + "with_skill": { + "skipped": true, + "skip_reason": "Only 1 iteration(s) configured; consistency needs at least 3." + }, + "without_skill": { + "skipped": true, + "skip_reason": "Only 1 iteration(s) configured; consistency needs at least 3." + } + } + }, + "tgw-vpn-summarization": { + "summary": "This evaluation covers a single chat-task scenario about TGW/VPN routing (route propagation, static summary routes, and route selection under the advertisement cap).\n\nTrigger/setup: The behavior under test fired as expected (1/1).\n\nExpected output: with_skill met the expected output (1/1, high confidence), while without_skill did not (0/1, high confidence). Since both confidence readings are high, this difference is not an artifact of weak measurement.\n\nAssertions (4 total, run once each):\n- Two assertions passed for both variants (\"adds a static summary route in the TGW route table associated with the VPN attachment\" and \"does not contain 'error'\") \u2014 these are not differentiating; both systems handle them fine.\n- Two assertions passed for with_skill but failed for without_skill: \"keeps VPC route propagation enabled for the return path\" (both high confidence) and \"explains static-over-propagated and least-specific selection under the advertisement cap\" (with_skill high confidence, without_skill medium confidence, so this particular failure read is somewhat less certain). Overall with_skill passed 4/4 assertions and without_skill passed 2/4.\n\nThis is based on a single task instance, so these results reflect one run rather than a pattern established over repeated trials; no output-consistency data (run-to-run variation) was available for either variant (both null), so we can't say how stable these results would be on repeated attempts.\n\nMetrics: with_skill ran faster (~17s vs ~39s) and cheaper (~$0.14 vs ~$0.33) than without_skill, with similar (low) context-window utilization for both (~5% vs ~4%) and no compaction events for either. The runtime/cost gap favors with_skill, but on its own this would normally be weighed against output quality \u2014 here, with_skill also scored better on expected-output match and assertions, so this is not a case of a clear speed/quality tradeoff; both the cost/time and the correctness signals point the same direction for this one task.\n\nQuality comparison: No side-by-side quality pairs were actually evaluated (0 pairs evaluated, 1 skipped), so no comparative quality judgment is available beyond the assertion/expected-output results above.\n\nOverall: For this single task, with_skill satisfied the expected output and all assertions while running faster and cheaper; without_skill missed the expected output and two of four assertions while taking longer and costing more. However, this is drawn from only one task instance with no consistency data, so it should be read as a single data point rather than a robust, repeated-trial comparison.", + "task_type": "chat", + "trigger": { + "with_skill": { + "pass_rate": "1/1", + "percentage": 100 + }, + "without_skill": null + }, + "expected_output": { + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "low_confidence": 0, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "0/1", + "percentage": 0, + "low_confidence": 0, + "avg_confidence": "high" + } + }, + "assertions": { + "per_assertion": [ + { + "index": 0, + "text": "keeps VPC route propagation enabled for the return path", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "0/1", + "percentage": 0, + "avg_confidence": "high" + }, + "classification": "skill_uplift" + }, + { + "index": 1, + "text": "adds a static summary route in the TGW route table associated with the VPN attachment", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "medium" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "always_passes" + }, + { + "index": 2, + "text": "explains static-over-propagated and least-specific selection under the advertisement cap", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "0/1", + "percentage": 0, + "avg_confidence": "medium" + }, + "classification": "skill_uplift" + }, + { + "index": 3, + "text": "does not contain 'error'", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "always_passes" + } + ], + "with_skill": { + "pass_rate": "4/4", + "percentage": 100 + }, + "without_skill": { + "pass_rate": "2/4", + "percentage": 50 + } + }, + "metrics": { + "with_skill": { + "runtime_avg": "17s", + "cost_avg": "$0.14", + "context_window_avg": { + "utilization": "5.0%", + "compaction_count": 0 + } + }, + "without_skill": { + "runtime_avg": "39s", + "cost_avg": "$0.33", + "context_window_avg": { + "utilization": "4.0%", + "compaction_count": 0 + } + } + }, + "comparison": { + "quality": { + "pairs_evaluated": 0, + "pairs_skipped": 1, + "skip_reasons": [ + "without_skill expected_output test failed" + ], + "overall": { + "with_skill_wins": 0, + "without_skill_wins": 0, + "equivalent": 0, + "winner": "equivalent", + "summary": "Equivalent overall (0 vs 0 criterion wins out of 0 judgments)." + }, + "per_criterion": {}, + "iterations": [ + { + "iteration": 1, + "skipped": true, + "skip_reason": "without_skill expected_output test failed" + } + ] + } + }, + "output_consistency": { + "comparison": { + "applicable": false, + "reason": "with_skill was not judged: Only 1 iteration(s) configured; consistency needs at least 3." + }, + "with_skill": { + "skipped": true, + "skip_reason": "Only 1 iteration(s) configured; consistency needs at least 3." + }, + "without_skill": { + "skipped": true, + "skip_reason": "Only 1 iteration(s) configured; consistency needs at least 3." + } + } + }, + "cloudwan-overlapping-supernet": { + "summary": "This evaluation covered a single chat task about diagnosing a longest-prefix-match routing conflict shadowing on-prem hosts inside a VPC CIDR. The trigger for the test fired as expected (1/1).\n\nExpected output: without_skill met the expected output (1/1, 100%), while with_skill did not (0/1, 0%). Both judgments carry only \"medium\" average confidence (and are not flagged low-confidence), so this is a real but not highly certain signal \u2014 it's based on a single task/run, which limits how much weight it can bear.\n\nPer-assertion results (4 assertions total, each run once):\n- Two assertions passed for both variants: explaining that longest-prefix-match shadows on-prem hosts, and recommending non-overlapping address planning as the best fix. These don't differentiate the variants.\n- One assertion (\"does not contain 'error'\") also passed for both \u2014 again not differentiating.\n- One assertion \u2014 mentioning that Cloud WAN Routing Policies can filter/drop prefixes on the Direct Connect attachment \u2014 passed for with_skill but failed for without_skill. This is the one substantive point of divergence and it aligns with with_skill's higher aggregate assertion pass rate (4/4 = 100% vs without_skill's 3/4 = 75%). Notably, this same divergent point is also likely what drove without_skill's failure on the \"expected_output\" judgment, so the expected-output result and the assertion results are telling a consistent (not independent) story rather than two separate pieces of evidence.\n\nMetrics: without_skill ran much faster (23s vs 1m17s, roughly 3-4x quicker) and cost less ($0.20 vs $0.64, about a third of the cost). Context window utilization was low and similar for both (4.6% vs 5.1%), with no compaction events for either.\n\nQuality comparison and output consistency: no quality pairs could be evaluated (0 evaluated, 1 skipped), and output consistency data is null for both variants, so neither of these dimensions offers any evidence here.\n\nOverall, this is a single-task, single-run comparison with limited statistical weight. The clearest and most consistent signal is that with_skill covered the Cloud WAN Routing Policy filtering point that without_skill omitted, which drove both the assertion-rate difference and the expected-output miss for without_skill. Against that, without_skill was substantially faster and cheaper to run. There is no quality or consistency data to indicate whether these differences reflect a broader, reliable pattern or are specific to this single run.", + "task_type": "chat", + "trigger": { + "with_skill": { + "pass_rate": "1/1", + "percentage": 100 + }, + "without_skill": null + }, + "expected_output": { + "with_skill": { + "pass_rate": "0/1", + "percentage": 0, + "low_confidence": 0, + "avg_confidence": "medium" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "low_confidence": 0, + "avg_confidence": "medium" + } + }, + "assertions": { + "per_assertion": [ + { + "index": 0, + "text": "explains longest-prefix-match shadows on-prem hosts inside a VPC CIDR", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "always_passes" + }, + { + "index": 1, + "text": "recommends non-overlapping address planning as the best fix", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "always_passes" + }, + { + "index": 2, + "text": "mentions Cloud WAN Routing Policies can filter (drop) prefixes on the Direct Connect attachment", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "0/1", + "percentage": 0, + "avg_confidence": "medium" + }, + "classification": "skill_uplift" + }, + { + "index": 3, + "text": "does not contain 'error'", + "evaluator": "llm", + "with_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "without_skill": { + "pass_rate": "1/1", + "percentage": 100, + "avg_confidence": "high" + }, + "classification": "always_passes" + } + ], + "with_skill": { + "pass_rate": "4/4", + "percentage": 100 + }, + "without_skill": { + "pass_rate": "3/4", + "percentage": 75 + } + }, + "metrics": { + "with_skill": { + "runtime_avg": "1m17s", + "cost_avg": "$0.64", + "context_window_avg": { + "utilization": "5.1%", + "compaction_count": 0 + } + }, + "without_skill": { + "runtime_avg": "23s", + "cost_avg": "$0.20", + "context_window_avg": { + "utilization": "4.6%", + "compaction_count": 0 + } + } + }, + "comparison": { + "quality": { + "pairs_evaluated": 0, + "pairs_skipped": 1, + "skip_reasons": [ + "with_skill expected_output test failed" + ], + "overall": { + "with_skill_wins": 0, + "without_skill_wins": 0, + "equivalent": 0, + "winner": "equivalent", + "summary": "Equivalent overall (0 vs 0 criterion wins out of 0 judgments)." + }, + "per_criterion": {}, + "iterations": [ + { + "iteration": 1, + "skipped": true, + "skip_reason": "with_skill expected_output test failed" + } + ] + } + }, + "output_consistency": { + "comparison": { + "applicable": false, + "reason": "with_skill was not judged: Only 1 iteration(s) configured; consistency needs at least 3." + }, + "with_skill": { + "skipped": true, + "skip_reason": "Only 1 iteration(s) configured; consistency needs at least 3." + }, + "without_skill": { + "skipped": true, + "skip_reason": "Only 1 iteration(s) configured; consistency needs at least 3." + } + } + }, + "negative-python-linked-list": { + "task_type": "chat", + "trigger": { + "with_skill": { + "pass_rate": "1/1", + "percentage": 100 + }, + "without_skill": null + }, + "expected_output": { + "skipped": true, + "skip_reason": "Eval is should_trigger: false" + }, + "assertions": { + "skipped": true, + "skip_reason": "Eval is should_trigger: false" + }, + "metrics": { + "skipped": true, + "skip_reason": "Eval is should_trigger: false" + }, + "comparison": { + "skipped": true, + "skip_reason": "Eval is should_trigger: false" + }, + "output_consistency": { + "skipped": true, + "skip_reason": "Eval is should_trigger: false" + } + }, + "negative-dynamodb-iam-policy": { + "task_type": "chat", + "trigger": { + "with_skill": { + "pass_rate": "1/1", + "percentage": 100 + }, + "without_skill": null + }, + "expected_output": { + "skipped": true, + "skip_reason": "Eval is should_trigger: false" + }, + "assertions": { + "skipped": true, + "skip_reason": "Eval is should_trigger: false" + }, + "metrics": { + "skipped": true, + "skip_reason": "Eval is should_trigger: false" + }, + "comparison": { + "skipped": true, + "skip_reason": "Eval is should_trigger: false" + }, + "output_consistency": { + "skipped": true, + "skip_reason": "Eval is should_trigger: false" + } + } + } +} \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/evals.json b/skills/aws-routing/evals/functional/v1/evals.json new file mode 100644 index 00000000..8ea2865a --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/evals.json @@ -0,0 +1,131 @@ +{ + "skill_name": "aws-routing", + "evals": [ + { + "id": "cloudwan-local-dx-preference", + "task_type": "chat", + "prompt": "In AWS Cloud WAN, traffic for an aggregate prefix is leaving the local region instead of using the local Direct Connect. How does route evaluation decide this and how can I prefer the local DX?", + "should_trigger": true, + "expected_output": "Explains Cloud WAN route evaluation (longest-prefix-match first, then AS-path at the CNE), why a remote path can win, and recommends LP communities (7224:7300/7200/7100) and/or AS-path normalization to prefer the local DX.", + "assertions": [ + "references longest prefix match in Cloud WAN route evaluation", + "explains that AS-path length is evaluated at the CNE and can cause a remote path to win", + "recommends LP communities (7224:7300/7200/7100) as a deterministic remedy", + "does not contain 'error'" + ] + }, + { + "id": "dx-path-selection-order", + "task_type": "chat", + "prompt": "What is the outbound path-selection order for a Direct Connect private VIF, and where does MED fit?", + "should_trigger": true, + "expected_output": "Order: longest prefix match, then local preference, then AS_PATH length, then MED, then ECMP. MED is a low-priority tiebreaker AWS does not recommend relying on. LP communities 7224:7100/7200/7300 set local preference.", + "assertions": [ + "states the order longest prefix match, local preference, AS_PATH, MED, ECMP", + "notes MED is a low-priority tiebreaker", + "mentions LP communities 7224:7100/7200/7300", + "does not contain 'error'" + ] + }, + { + "id": "tgw-vs-vgw-vif-transit", + "task_type": "chat", + "prompt": "Can two on-premises sites on separate Direct Connect VIFs talk to each other through AWS?", + "should_trigger": true, + "expected_output": "By default AWS does not act as a transit hub (advertises only VPC CIDRs). VPN CloudHub enables site-to-site for VPN connections; TGW with an Allowed Prefix List (and two DXGWs for DX-to-DX) can enable it intentionally.", + "assertions": [ + "explains AWS does not act as a transit hub by default", + "mentions VPN CloudHub for VPN connections", + "mentions TGW or two DXGWs as a way to enable DX-to-DX intentionally", + "does not contain 'error'" + ] + }, + { + "id": "dx-prefer-one-location", + "task_type": "chat", + "prompt": "I have two Direct Connect locations and want traffic to prefer location A over location B, with B as backup. How do I do this and what should I avoid?", + "should_trigger": true, + "expected_output": "Use LP communities (7224:7300 on primary, lower on backup). AS-path prepending is reliable only within the same associated region; cross-region prepending is unreliable due to DXGW local-region LP.", + "assertions": [ + "recommends LP communities with a higher preference on the primary", + "explains AS-path prepending is reliable only within the same region", + "warns cross-region AS-path prepending is unreliable", + "does not contain 'error'" + ] + }, + { + "id": "tgw-ecmp-behavior", + "task_type": "chat", + "prompt": "When does a Transit Gateway load-balance across two Direct Connect paths, and when does it pick just one?", + "should_trigger": true, + "expected_output": "ECMP requires the same attachment type and equal AS-path length / BGP attributes; otherwise TGW installs a single (oldest) route. LP communities are not honored across multiple DXGW attachments on TGW.", + "assertions": [ + "states ECMP requires same attachment type and equal AS-path/BGP attributes", + "notes TGW picks a single route when ECMP conditions are not met", + "does not contain 'error'" + ] + }, + { + "id": "dx-vpn-redundancy-preference", + "task_type": "chat", + "prompt": "I have a Direct Connect and a backup Site-to-Site VPN to the same VGW. Which path does AWS prefer and how do I avoid asymmetric routing?", + "should_trigger": true, + "expected_output": "DX is always preferred over VPN for the same prefix at a VGW; there is no ECMP between DX and VPN. Asymmetric routing is the main failure mode; match ingress/egress preference and validate both directions.", + "assertions": [ + "states DX is always preferred over VPN for the same prefix at a VGW", + "identifies asymmetric routing as the main risk", + "does not contain 'error'" + ] + }, + { + "id": "verification-unknown-source", + "task_type": "chat", + "prompt": "Is there an official AWS 'DX Path Selection Flow Chart' document I can cite?", + "should_trigger": true, + "expected_output": "Does not fabricate a document or URL; states it cannot verify such a named artifact; points to the Direct Connect routing policies documentation for the path-selection order.", + "assertions": [ + "does not fabricate a document or URL", + "states it cannot verify such a named artifact", + "points to the Direct Connect routing policies documentation" + ] + }, + { + "id": "tgw-vpn-summarization", + "task_type": "chat", + "prompt": "A customer is at the TGW-to-VPN advertised-route limit. How do they summarize without breaking the return path?", + "should_trigger": true, + "expected_output": "Keep VPC propagation enabled (specifics needed for longest-prefix-match return forwarding), add a static summary route in the VPN-associated TGW route table; under the cap, static beats propagated and least-specific is advertised first, so the summary survives. Anchor the summary to a persistent attachment.", + "assertions": [ + "keeps VPC route propagation enabled for the return path", + "adds a static summary route in the TGW route table associated with the VPN attachment", + "explains static-over-propagated and least-specific selection under the advertisement cap", + "does not contain 'error'" + ] + }, + { + "id": "cloudwan-overlapping-supernet", + "task_type": "chat", + "prompt": "A customer advertises a supernet (10.0.0.0/8) from Direct Connect into Cloud WAN that overlaps VPC CIDRs (10.1.0.0/16, 10.2.0.0/16). What issues would this cause and how do they fix it?", + "should_trigger": true, + "expected_output": "Longest-prefix-match means VPC /16s win and on-prem hosts inside a VPC CIDR are shadowed/unreachable. Best fix is non-overlapping address planning. Cloud WAN Routing Policies can drop the inbound supernet and suppress VPC prefixes outbound on the DX attachment; BGP communities cannot be matched on DX attachments.", + "assertions": [ + "explains longest-prefix-match shadows on-prem hosts inside a VPC CIDR", + "recommends non-overlapping address planning as the best fix", + "mentions Cloud WAN Routing Policies can filter (drop) prefixes on the Direct Connect attachment", + "does not contain 'error'" + ] + }, + { + "id": "negative-python-linked-list", + "task_type": "chat", + "prompt": "Write a Python function to reverse a linked list.", + "should_trigger": false + }, + { + "id": "negative-dynamodb-iam-policy", + "task_type": "chat", + "prompt": "What IAM policy do I need to allow read-only access to DynamoDB?", + "should_trigger": false + } + ] +} diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-local-dx-preference/with_skill/functional-tests-results.json b/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-local-dx-preference/with_skill/functional-tests-results.json new file mode 100644 index 00000000..a89b4c46 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-local-dx-preference/with_skill/functional-tests-results.json @@ -0,0 +1,74 @@ +{ + "version": "v1", + "iteration": 1, + "eval_id": "cloudwan-local-dx-preference", + "run_type": "with_skill", + "success": true, + "error": null, + "understanding_agent_space_skill": null, + "tests": { + "trigger": { + "result": "passed", + "message": "Skill 'aws-routing' was triggered as expected", + "skill_loads_found": 1 + }, + "expected_output": { + "result": "passed", + "reasoning": "The agent's response explains the route evaluation process in detail: longest-prefix-match first, then AS-path length as a tiebreaker at the CNE/DXGW level, consistent with the expected output. It explains why a remote path can win (no local advertisement of the aggregate, causing DXGW to treat all candidates as remote and fall back to AS-path length, or an LP community override). It explicitly recommends using LP communities (7224:7300/7200/7100) to prefer the local DX, and discusses AS-path considerations (noting prepending's limitations but still addressing AS-path normalization as a lever within the explanation). This covers all the key elements: LPM, AS-path, remote path winning scenario, and LP community recommendation (7224:7300/7200/7100 specifically mentioned).\"", + "evidence": "\"Longest prefix match \u2192 static \u2192 VPC-propagated (same region) \u2192 unequal AS-path/MED (shortest wins)... Use LP communities, not AS-path prepending, for cross-region steering. Tag the local DX's advertisement 7224:7300 (High)... Advertise the aggregate locally too (or a covering more-specific). Longest-prefix-match at Step 1 is absolute...\"", + "confidence": "high" + }, + "assertions": { + "assertion_results": [ + { + "text": "references longest prefix match in Cloud WAN route evaluation", + "evaluator": "llm", + "passed": true, + "evidence": "\"Lookup 2 \u2014 Cloud WAN CNE route lookup ... each CNE evaluates its own table in this order \u2014 longest prefix match \u2192 static \u2192 VPC-propagated...\" and also \"Longest-prefix-match at Step 1 is absolute, so if local-region DX advertises a more-specific...\"", + "reasoning": "The output explicitly references longest prefix match as part of the Cloud WAN route evaluation order at the CNE.", + "confidence": "high" + }, + { + "text": "explains that AS-path length is evaluated at the CNE and can cause a remote path to win", + "evaluator": "llm", + "passed": false, + "evidence": "The agent attributes AS-path length evaluation to the DXGW lookup ('DXGW falls through to AS-path length \u2014 shortest wins'), not explicitly to the CNE. The CNE lookup (Lookup 2) is described as 'unequal AS-path/MED (shortest wins)' but this is framed as the return path (AWS \u2192 on-prem), and the cause of the symptom (remote path winning) is attributed to DXGW's lookup, not the CNE's AS-path evaluation.", + "reasoning": "The assertion specifically claims AS-path length is evaluated at the CNE and causes a remote path to win. The output does mention AS-path/MED evaluation at the CNE in Lookup 2, but the explanation for why the remote path wins in the user's reported symptom is attributed to DXGW's lookup (Lookup 1), not the CNE. The CNE's AS-path evaluation is mentioned but in the context of the return path, not the specific scenario causing the issue described in the prompt. This is a partial match - the CNE AS-path mention exists but the causal explanation for the remote path winning is via DXGW, not CNE directly.", + "confidence": "medium" + }, + { + "text": "recommends LP communities (7224:7300/7200/7100) as a deterministic remedy", + "evaluator": "llm", + "passed": true, + "evidence": "\"Use LP communities, not AS-path prepending, for cross-region steering. Tag the local DX's advertisement 7224:7300 (High) \u2014 LP communities are evaluated before AS-path at the DXGW and work both within- and cross-region.\"", + "reasoning": "The output explicitly recommends LP communities (specifically 7224:7300, part of the 7224:7300/7200/7100 set mentioned earlier) as a deterministic remedy, explaining why it works before AS-path evaluation.", + "confidence": "high" + }, + { + "text": "does not contain 'error'", + "evaluator": "llm", + "passed": true, + "evidence": "No instance of the word 'error' appears anywhere in the output text.", + "reasoning": "Scanning the full output text for the literal word 'error' yields no matches.", + "confidence": "high" + } + ], + "summary": { + "passed": 3, + "failed": 1, + "errored": 0, + "low_confidence": 0, + "total": 4, + "pass_rate": 0.75 + } + } + }, + "metrics": { + "runtime": "23s", + "cost": "$0.20", + "context_window": { + "utilization": "5.1%", + "compaction_count": 0 + } + } +} \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-local-dx-preference/with_skill/outputs/journal_records.json b/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-local-dx-preference/with_skill/outputs/journal_records.json new file mode 100644 index 00000000..b7dad6a5 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-local-dx-preference/with_skill/outputs/journal_records.json @@ -0,0 +1,154 @@ +[ + { + "agentSpaceId": "71f32cf0-d741-4b1c-bbef-25113fedd3ca", + "executionId": "e08b9db9-0cd3-4afc-a820-9c47e5428d53", + "recordId": "2fdf62e6-6dd2-4f65-b127-8da09c022c3b", + "content": "{\"id\": \"2fdf62e6-6dd2-4f65-b127-8da09c022c3b\", \"role\": \"user\", \"content\": [{\"text\": \"In AWS Cloud WAN, traffic for an aggregate prefix is leaving the local region instead of using the local Direct Connect. How does route evaluation decide this and how can I prefer the local DX?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:02.020000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "71f32cf0-d741-4b1c-bbef-25113fedd3ca", + "executionId": "e08b9db9-0cd3-4afc-a820-9c47e5428d53", + "recordId": "c766c399-3419-46cf-a746-7f631faa68d6", + "content": "{\"id\": \"e662562d-25cc-44ed-ba4d-da5de87da971\", \"role\": \"assistant\", \"content\": [{\"type\": \"chat_title\", \"title\": \"AWS Cloud WAN aggregate prefix routing preference\"}]}", + "createdAt": "2026-10-27T17:05:02.639000-05:00", + "recordType": "chat_title" + }, + { + "agentSpaceId": "71f32cf0-d741-4b1c-bbef-25113fedd3ca", + "executionId": "e08b9db9-0cd3-4afc-a820-9c47e5428d53", + "recordId": "f851fbef-0a2c-44da-ba2b-2ee4e2fa2715", + "content": "{\"id\": \"f851fbef-0a2c-44da-ba2b-2ee4e2fa2715\", \"role\": \"assistant\", \"content\": [{\"id\": \"tooluse_h1gYSr4WZlPfLOU8FsH6G2\", \"tool_name\": \"skill_read\", \"input\": {\"skill_id\": \"aws-routing\"}, \"type\": \"tool_use\", \"display_intent\": \"Reading AWS routing skill for Cloud WAN CNE route evaluation guidance\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:05.818000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "71f32cf0-d741-4b1c-bbef-25113fedd3ca", + "executionId": "e08b9db9-0cd3-4afc-a820-9c47e5428d53", + "recordId": "8438d14d-a83f-423c-9d70-d19c262c6a23", + "content": "{\"id\": \"4fb2e436-541c-4361-a767-532aada4e9de\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This is a question about AWS Cloud WAN routing behavior, which lines up with the aws-routing skill, so I'll check that resource.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:05.818000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "71f32cf0-d741-4b1c-bbef-25113fedd3ca", + "executionId": "e08b9db9-0cd3-4afc-a820-9c47e5428d53", + "recordId": "0117e21d-0f0e-410b-bc41-be9db4a38177", + "content": "{\"id\": \"4c4921d6-719d-4941-9c17-81ee275d8502\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"359\"}]}", + "createdAt": "2026-10-27T17:05:05.900000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "71f32cf0-d741-4b1c-bbef-25113fedd3ca", + "executionId": "e08b9db9-0cd3-4afc-a820-9c47e5428d53", + "recordId": "fe860c12-e947-44aa-9c6f-db6ae32375f6", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 3.8, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:05.904000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "71f32cf0-d741-4b1c-bbef-25113fedd3ca", + "executionId": "e08b9db9-0cd3-4afc-a820-9c47e5428d53", + "recordId": "b64407c9-df35-409d-9724-262da7c72dc3", + "content": "{\"id\": \"a8903f78-a74b-491b-882b-53b670b641fc\", \"role\": \"assistant\", \"block_type\": \"load_skill\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_h1gYSr4WZlPfLOU8FsH6G2\", \"status\": \"success\", \"content\": [{\"text\": \"Done\"}]}], \"label\": \"Reading AWS routing skill for Cloud WAN CNE route evaluation guidance\"}", + "createdAt": "2026-10-27T17:05:05.984000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "71f32cf0-d741-4b1c-bbef-25113fedd3ca", + "executionId": "e08b9db9-0cd3-4afc-a820-9c47e5428d53", + "recordId": "34fd4573-8884-4f2c-9dcd-c600ac952891", + "content": "{\"id\": \"931a5d01-03db-4a29-be73-b5755a45ee54\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This looks like it matches the Cloud WAN and Direct Connect routing patterns reference, so I'll pull that up.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:09.268000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "71f32cf0-d741-4b1c-bbef-25113fedd3ca", + "executionId": "e08b9db9-0cd3-4afc-a820-9c47e5428d53", + "recordId": "7e5e8481-3003-4b41-aa7f-fada2ff09ae5", + "content": "{\"id\": \"7e5e8481-3003-4b41-aa7f-fada2ff09ae5\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_h1gYSr4WZlPfLOU8FsH6G2\", \"content\": \"[{'text': '{\\\"content\\\": \\\"---\\\\\\\\r\\\\\\\\nname: aws-routing\\\\\\\\r\\\\\\\\ndescription: \\\\'Use this skill when the user wants to know which path AWS will pick for their traffic, why traffic took an unexpected route, or how to make AWS prefer a path. Covers Cloud WAN CNE route evaluation, Direct Connect Gateway local vs remote region preference, Transit Gateway route tables, ECMP and peering, VPC longest-prefix-match, BGP traffic engineering (LP communities 7224:7100/7200/7300, AS-path prepending, MED), DX + VPN redundancy and failover, and asymmetric routing. Use it even when the user only describes a symptom, such as \\\\\\\\\\\"traffic is leaving the region\\\\\\\\\\\", \\\\\\\\\\\"not using my local Direct Connect\\\\\\\\\\\", \\\\\\\\\\\"routing to the wrong VIF/location\\\\\\\\\\\", or \\\\\\\\\\\"path is asymmetric\\\\\\\\\\\". Read-only: analysis, recommendations, and describe/get/list validation commands grounded in public AWS docs.\\\\'\\\\\\\\r\\\\\\\\nmetadata:\\\\\\\\r\\\\\\\\n version: \\\\\\\\\\\"1.2.5\\\\\\\\\\\"\\\\\\\\r\\\\\\\\n author: midakin\\\\\\\\r\\\\\\\\n aws-devops-agent-skills.agent-types: \\\\\\\\\\\"CHAT, INCIDENT_RCA\\\\\\\\\\\"\\\\\\\\r\\\\\\\\n---\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Overview\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nThis skill provides routing-first analysis for AWS networking. It reasons about the full traffic\\\\\\\\r\\\\\\\\npath, applies the documented route-evaluation logic for each service, and grounds recommendations\\\\\\\\r\\\\\\\\nin public AWS documentation. It is **read-only**: it produces analysis, recommendations, and\\\\\\\\r\\\\\\\\nread-only validation commands (`describe-*`, `get-*`, `list-*`) - never mutating actions.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Verification & Anti-Hallucination Directives (Mandatory)\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- Do not generate any fact, citation, or source that is not fully verifiable. If you lack a verified\\\\\\\\r\\\\\\\\n source, state \\\\\\\\\\\"I cannot verify this\\\\\\\\\\\" rather than inventing details.\\\\\\\\r\\\\\\\\n- State assumptions explicitly before answering.\\\\\\\\r\\\\\\\\n- Cross-check each claim against public AWS documentation or the `references/` files. Omit claims you\\\\\\\\r\\\\\\\\n cannot confirm.\\\\\\\\r\\\\\\\\n- Only cite AWS documentation URLs you have actually retrieved or confirmed. Do not fabricate links.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## When to use this skill\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nActivate when the user asks about:\\\\\\\\r\\\\\\\\n- Cloud WAN route evaluation / Core Network Edge (CNE) path selection\\\\\\\\r\\\\\\\\n- Direct Connect Gateway path selection, local-region preference, or LP communities\\\\\\\\r\\\\\\\\n- Transit Gateway route evaluation, ECMP, or peering routing\\\\\\\\r\\\\\\\\n- VPC route tables and longest-prefix-match behavior\\\\\\\\r\\\\\\\\n- BGP traffic engineering (communities, AS-path prepending, MED)\\\\\\\\r\\\\\\\\n- DX + VPN redundancy, active/active vs active/passive, or failover design\\\\\\\\r\\\\\\\\n- Why traffic is taking an unexpected path\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## What this skill does NOT do\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nThis skill is **read-only and advisory**. It will not:\\\\\\\\r\\\\\\\\n- **Design or apply infrastructure changes.** It does not create, modify, or delete AWS resources,\\\\\\\\r\\\\\\\\n and it does not run mutating CLI/API calls (`create-*`, `modify-*`, `associate-*`, `delete-*`,\\\\\\\\r\\\\\\\\n `put-*`, `update-*`) or write Infrastructure-as-Code to be deployed. Any change it describes is a\\\\\\\\r\\\\\\\\n *recommendation* for you to review and implement yourself.\\\\\\\\r\\\\\\\\n- **Configure BGP or push routing policy.** It explains which communities, AS-path, or MED settings\\\\\\\\r\\\\\\\\n to use, but it does not apply them to VIFs, route tables, or Cloud WAN policy documents.\\\\\\\\r\\\\\\\\n- **Execute anything against a live account beyond read-only validation.** Commands it provides are\\\\\\\\r\\\\\\\\n limited to `describe-*`, `get-*`, and `list-*` for confirming state.\\\\\\\\r\\\\\\\\n- **Guarantee production outcomes.** It recommends non-production testing and blue/green rollout;\\\\\\\\r\\\\\\\\n it does not perform the cutover or validate the result for you.\\\\\\\\r\\\\\\\\n- **Provide compliance, security, or contractual sign-off**, or make claims it cannot ground in\\\\\\\\r\\\\\\\\n public AWS documentation or the `references/` files.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nFor designing and applying changes, hand the recommendation to a change-capable workflow (with\\\\\\\\r\\\\\\\\nappropriate review, approvals, and testing).\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## How to analyze a routing problem\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nWork through this checklist in order; each step depends on the one before it.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- [ ] **1. Establish the path.** Identify source, destination, and every routing construct in\\\\\\\\r\\\\\\\\n between (VPC route table \\\\\\\\u2192 TGW/Cloud WAN CNE \\\\\\\\u2192 DXGW \\\\\\\\u2192 VIF \\\\\\\\u2192 on-prem, or the relevant subset).\\\\\\\\r\\\\\\\\n- [ ] **2. Name the route table.** Be explicit about which table makes each decision (VPC RT, TGW RT,\\\\\\\\r\\\\\\\\n CNE RT, DXGW internal selection).\\\\\\\\r\\\\\\\\n- [ ] **3. Apply the evaluation order for that construct** (see below).\\\\\\\\r\\\\\\\\n- [ ] **4. Identify the deciding attribute** (longest prefix, local preference, AS-path, MED, source\\\\\\\\r\\\\\\\\n type).\\\\\\\\r\\\\\\\\n- [ ] **5. Flag non-determinism** (e.g., ECMP or \\\\\\\\\\\"deterministically random\\\\\\\\\\\" tiebreakers) and\\\\\\\\r\\\\\\\\n recommend a deterministic alternative.\\\\\\\\r\\\\\\\\n- [ ] **6. Cite the AWS documentation** that supports the behavior.\\\\\\\\r\\\\\\\\n- [ ] **7. Provide read-only validation commands** the user can run to confirm.\\\\\\\\r\\\\\\\\n- [ ] **8. Self-check before answering.** Re-read the answer against steps 1-7: every route table is\\\\\\\\r\\\\\\\\n named, the evaluation order matches the construct, each cited URL was actually retrieved, and every\\\\\\\\r\\\\\\\\n command is read-only. Fix any gap before presenting.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Key evaluation rules\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Cloud WAN (per CNE)\\\\\\\\r\\\\\\\\nLongest prefix match \\\\\\\\u2192 static \\\\\\\\u2192 VPC-propagated (same region) \\\\\\\\u2192 unequal AS-path/MED (shortest wins) \\\\\\\\u2192\\\\\\\\r\\\\\\\\nequal AS-path & MED source preference: DXGW-propagated \\\\\\\\u2192 Cloud WAN Connect \\\\\\\\u2192 S2S VPN \\\\\\\\u2192 other\\\\\\\\r\\\\\\\\n(TGW peering, remote CNEs; identical from 2+ sources = deterministically random).\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Direct Connect (private/transit VIF, outbound AWS\\\\\\\\u2192on-prem)\\\\\\\\r\\\\\\\\nLongest prefix match \\\\\\\\u2192 local preference \\\\\\\\u2192 AS_PATH length \\\\\\\\u2192 MED \\\\\\\\u2192 ECMP (equal AS_PATH and BGP\\\\\\\\r\\\\\\\\nattributes). LP communities `7224:7300/7200/7100` (High/Medium/Low) set local preference and are\\\\\\\\r\\\\\\\\nevaluated before AS-path. DXGW prefers the local associated-region DX by default.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Direction & tooling guidance\\\\\\\\r\\\\\\\\n- **AWS \\\\\\\\u2192 on-prem (egress):** control with DX LP communities.\\\\\\\\r\\\\\\\\n- **On-prem \\\\\\\\u2192 AWS (ingress):** control with the customer router\\\\'s local-pref/weight/MED.\\\\\\\\r\\\\\\\\n- **AS-path prepending:** within a region only - unreliable across regions because DXGW local-region\\\\\\\\r\\\\\\\\n LP overrides it. **LP communities:** work within-region and cross-region.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Reference material\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nDetailed mechanics, patterns, and caveats are in `references/`. Load only the file that matches the\\\\\\\\r\\\\\\\\nconstruct deciding the path:\\\\\\\\r\\\\\\\\n- [Cloud WAN + DX routing patterns](references/cloudwan-dx-routing-patterns.md) - load when the path\\\\\\\\r\\\\\\\\n crosses a Cloud WAN CNE: CNE route evaluation, DXGW path selection into Cloud WAN, community\\\\\\\\r\\\\\\\\n traffic-engineering patterns, regional inspection, multi-region egress.\\\\\\\\r\\\\\\\\n- [Direct Connect routing patterns](references/dx-routing-patterns.md) - load when the question is\\\\\\\\r\\\\\\\\n about a DX VIF or DXGW without Cloud WAN: BGP mechanics, BFD, MED, ASN ranges, VIF considerations,\\\\\\\\r\\\\\\\\n CloudHub, when VIFs can talk, active/active vs active/passive.\\\\\\\\r\\\\\\\\n- [Transit Gateway routing patterns](references/tgw-routing-patterns.md) - load when a TGW route\\\\\\\\r\\\\\\\\n table makes the decision: TGW route evaluation, ECMP limits, DXGW-to-TGW behavior, DX/VPN caveats,\\\\\\\\r\\\\\\\\n failover, peering.\\\\\\\\r\\\\\\\\n- [VPN + DX redundancy patterns](references/vpn-dx-redundancy-patterns.md) - load when VPN backs up\\\\\\\\r\\\\\\\\n or runs alongside DX: route preference between them, ECMP limits, asymmetric return paths.\\\\\\\\r\\\\\\\\n- [Networking strategic questions](references/networking-strategic-questions.md) - load when key\\\\\\\\r\\\\\\\\n facts are missing (regions, attachments, ASNs, communities) and you need discovery questions to ask\\\\\\\\r\\\\\\\\n before analyzing.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Output expectations\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- Explain the route-evaluation logic step by step.\\\\\\\\r\\\\\\\\n- Name the specific route table involved.\\\\\\\\r\\\\\\\\n- Give the end-to-end traffic path.\\\\\\\\r\\\\\\\\n- Compare options with tradeoffs (determinism, complexity, failover) when multiple solutions exist.\\\\\\\\r\\\\\\\\n- Include read-only AWS CLI validation commands.\\\\\\\\r\\\\\\\\n- Recommend testing in a non-production environment and blue/green for migrations.\\\\\\\\r\\\\\\\\n\\\", \\\"resource_manifest\\\": [\\\"SKILL.md\\\", \\\"references/cloudwan-dx-routing-patterns.md\\\", \\\"references/dx-routing-patterns.md\\\", \\\"references/networking-strategic-questions.md\\\", \\\"references/tgw-routing-patterns.md\\\", \\\"references/vpn-dx-redundancy-patterns.md\\\"]}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:09.268000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "71f32cf0-d741-4b1c-bbef-25113fedd3ca", + "executionId": "e08b9db9-0cd3-4afc-a820-9c47e5428d53", + "recordId": "27b0851f-17ce-47f5-bdcb-ba0a5fac0665", + "content": "{\"id\": \"a48b236f-89c7-4083-95a9-b21bb4d64b17\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"344\"}]}", + "createdAt": "2026-10-27T17:05:09.345000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "71f32cf0-d741-4b1c-bbef-25113fedd3ca", + "executionId": "e08b9db9-0cd3-4afc-a820-9c47e5428d53", + "recordId": "e0587a00-600d-44b2-9eda-e5fa679508cd", + "content": "{\"id\": \"e0587a00-600d-44b2-9eda-e5fa679508cd\", \"role\": \"assistant\", \"content\": [{\"id\": \"tooluse_sldxas9NtSTlTyR7jq2LU8\", \"tool_name\": \"read_skill_resource\", \"input\": {\"skill_id\": \"aws-routing\", \"path\": \"references/cloudwan-dx-routing-patterns.md\"}, \"type\": \"tool_use\", \"display_intent\": \"Reading Cloud WAN + Direct Connect routing patterns reference\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:09.363000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "71f32cf0-d741-4b1c-bbef-25113fedd3ca", + "executionId": "e08b9db9-0cd3-4afc-a820-9c47e5428d53", + "recordId": "25a164da-c904-4999-bdcc-a81e0ebf0b6d", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.2, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:09.440000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "71f32cf0-d741-4b1c-bbef-25113fedd3ca", + "executionId": "e08b9db9-0cd3-4afc-a820-9c47e5428d53", + "recordId": "ad92f8a6-e9e8-4262-874b-ed4a777ddeb9", + "content": "{\"id\": \"097e5c3f-246a-4316-b97c-91259005aa83\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_sldxas9NtSTlTyR7jq2LU8\", \"status\": \"success\", \"content\": [{\"text\": \"Done\"}]}], \"label\": \"Reading Cloud WAN + Direct Connect routing patterns reference\"}", + "createdAt": "2026-10-27T17:05:09.726000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "71f32cf0-d741-4b1c-bbef-25113fedd3ca", + "executionId": "e08b9db9-0cd3-4afc-a820-9c47e5428d53", + "recordId": "343e99eb-a7d8-4e0f-9e95-add633c34cf0", + "content": "{\"id\": \"5ee3d644-6e9e-4693-9dcc-bda5b2c2ddd4\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"I'm now considering why traffic for an aggregate prefix would exit the region instead of using the local Direct Connect\\u2014likely because the local region only advertises a more specific route rather than the aggregate itself, causing the broader prefix to route elsewhere.\\n\\nI want to structure the answer around the two-stage route evaluation (DXGW lookup, then CNE lookup), name the relevant route tables, trace the end-to-end path, and give concrete validation commands alongside fix patterns like local-preference communities and more-specific route advertisements.\\n\\nSince I don't have account-specific details, I'll give this as a general diagnostic explainer while offering to validate against their actual setup if they'd like, and since I already have solid documentation grounding, I can answer directly without pulling in more tools.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:20.523000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "71f32cf0-d741-4b1c-bbef-25113fedd3ca", + "executionId": "e08b9db9-0cd3-4afc-a820-9c47e5428d53", + "recordId": "40ae64d2-2acd-4749-823a-3a16a0208298", + "content": "{\"id\": \"a03afe78-2edf-4acf-9ee6-07f075f9fd43\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"6019\"}]}", + "createdAt": "2026-10-27T17:05:20.625000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "71f32cf0-d741-4b1c-bbef-25113fedd3ca", + "executionId": "e08b9db9-0cd3-4afc-a820-9c47e5428d53", + "recordId": "5606c3c4-73ff-41f3-a413-e4b0abb43a2f", + "content": "{\"id\": \"5606c3c4-73ff-41f3-a413-e4b0abb43a2f\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_sldxas9NtSTlTyR7jq2LU8\", \"content\": \"[{'text': '\\\\r\\\\n\\\\r\\\\n# Cloud WAN + Direct Connect Routing Patterns & Considerations\\\\r\\\\n\\\\r\\\\nThis document captures key routing mechanics and traffic engineering patterns for Cloud WAN architectures with Direct Connect egress. Use this knowledge when advising on Cloud WAN route evaluation, DX Gateway behavior, BGP community usage, and regional failover design.\\\\r\\\\n\\\\r\\\\n## Reference Documentation\\\\r\\\\n\\\\r\\\\n- [Cloud WAN Route Evaluation](https://docs.aws.amazon.com/network-manager/latest/cloudwan/cloudwan-route-evaluation.html)\\\\r\\\\n- [Cloud WAN Routing Policies](https://docs.aws.amazon.com/network-manager/latest/cloudwan/cloudwan-routing-policies.html)\\\\r\\\\n- [Direct Connect Routing Policies and BGP Communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html)\\\\r\\\\n\\\\r\\\\n## Cloud WAN Route Evaluation Order (Per CNE)\\\\r\\\\n\\\\r\\\\nAt each Core Network Edge, Cloud WAN evaluates routes in this order:\\\\r\\\\n\\\\r\\\\n1. **Most specific route** (longest prefix match) \\u2014 absolute, wins before everything\\\\r\\\\n2. For same-destination routes with different targets:\\\\r\\\\n 1. Static routes\\\\r\\\\n 2. VPC-propagated routes (same region)\\\\r\\\\n 3. **Unequal AS-path length and/or MED** \\u2014 shortest wins\\\\r\\\\n 4. **Equal AS-path and MED** \\u2014 preference order:\\\\r\\\\n 1. Direct Connect Gateway-propagated routes\\\\r\\\\n 2. Cloud WAN Connect (same region)\\\\r\\\\n 3. Site-to-Site VPN (same region)\\\\r\\\\n 4. Other sources (TGW peering, CNEs in other regions) \\u2014 if identical from 2+ sources: **deterministically random**\\\\r\\\\n\\\\r\\\\n## DX Gateway Path Selection Mechanics\\\\r\\\\n\\\\r\\\\n### Key Behaviors\\\\r\\\\n\\\\r\\\\n1. **DXGW propagates only ONE path per prefix to each CNE route table** \\u2014 it performs internal path selection and sends only the winner. CNEs do not see all available DX paths.\\\\r\\\\n\\\\r\\\\n2. **DXGW uses Local Preference to prefer local-region DX over remote regions** \\u2014 if a DX VIF exists in the same associated AWS region as the target CNE, DXGW prefers it regardless of AS-path length.\\\\r\\\\n\\\\r\\\\n **\\u26a0\\ufe0f Always verify the Associated Region for each DX location** \\u2014 some DX locations can associate to different regions than expected based on geography. For example, London-area locations associate to different regions:\\\\r\\\\n - Equinix LD5 (Slough) \\u2192 **eu-west-1** (Ireland)\\\\r\\\\n - Digital Realty LHR20 (London) \\u2192 **eu-west-1** (Ireland)\\\\r\\\\n - Telehouse (London Docklands) \\u2192 **eu-west-2** (London)\\\\r\\\\n - Equinix MA3 (Manchester) \\u2192 **eu-west-2** (London)\\\\r\\\\n \\\\r\\\\n The associated region determines which CNE/TGW considers the DX \\\"local\\\" for LP preference. Verify a location\\\\'s associated region with the `DescribeLocations` API/CLI (`aws directconnect describe-locations`) or the official AWS Direct Connect locations reference: https://aws.amazon.com/directconnect/locations/\\\\r\\\\n\\\\r\\\\n3. **Explicit LP communities override DXGW\\\\'s default local-region preference** \\u2014 applying `7224:7300/7200/7100` on advertised prefixes forces DXGW to honor the explicit LP before applying its local-vs-remote default.\\\\r\\\\n\\\\r\\\\n4. **LP communities are evaluated before AS-path** \\u2014 per DX documentation: \\\"Local preference BGP community tags are evaluated before any AS_PATH attribute.\\\" This applies at the DXGW path selection stage.\\\\r\\\\n\\\\r\\\\n5. **LP communities from DX VIFs are NOT visible as community tags in Cloud WAN** \\u2014 the community tags themselves don\\\\'t propagate into Cloud WAN routing policies. However, the LP value they set at the DXGW level influences which path gets propagated to the CNE route table.\\\\r\\\\n\\\\r\\\\n6. **DXGW ECMPs across equal remote paths** \\u2014 when the local-region DX fails and multiple remote DX locations have equal attributes (same LP, same AS-path), DXGW will **ECMP across them** rather than picking one non-deterministically. This provides load-balanced failover across all remaining DX locations.\\\\r\\\\n\\\\r\\\\n7. **Route filtering on the native Cloud WAN DXGW attachment (Routing Policies)** \\u2014 Cloud WAN Routing Policies support **prefix-based route filtering (drop) and summarization, inbound and outbound, on Direct Connect attachments**. This is a **separate, supported feature** from the (unsupported) DXGW allowed-prefixes list. Use it to drop an overlapping inbound supernet or to suppress specific VPC prefixes advertised outbound to on-prem.\\\\r\\\\n - Match prefixes via `prefix-in-cidr` or `prefix-in-prefix-list`; actions are `allow`/`drop`; `routing-policy-direction` is `inbound` or `outbound`.\\\\r\\\\n - **DX caveat:** BGP communities **cannot** be matched or set on Direct Connect attachments in Cloud WAN \\u2014 prefix/CIDR matching only (consistent with point 5). Community-based routing policies work on other attachment types but not DX.\\\\r\\\\n - Source: [Cloud WAN Routing Policies](https://docs.aws.amazon.com/network-manager/latest/cloudwan/cloudwan-routing-policies.html); the DXGW-attachment \\\"no allowed-prefixes list\\\" limitation ([DX gateway attachments in Cloud WAN](https://docs.aws.amazon.com/network-manager/latest/cloudwan/cloudwan-dxattach-about.html)) refers to the legacy allowed-prefixes construct, not routing-policy filtering.\\\\r\\\\n\\\\r\\\\n**Important distinction \\u2014 path selection behavior by service:**\\\\r\\\\n- **DXGW:** ECMPs across equal remote paths (load-balanced failover)\\\\r\\\\n- **TGW:** Deterministic \\u2014 selects oldest route for equal paths. Consistent but not customer-controllable.\\\\r\\\\n- **Cloud WAN CNE:** Deterministic \\u2014 \\\"deterministically random\\\" at Step 2.4.4 for equal remote CNE paths. Consistent but not customer-controllable.\\\\r\\\\n\\\\r\\\\n### Common Pitfall: AS-Path Prepending Does Not Override Local-Region LP at the DXGW\\\\r\\\\n\\\\r\\\\n**The mistaken assumption:** \\\"If I prepend a DX VIF\\\\'s prefix enough, I can steer that region\\\\'s traffic away from (or toward) it across regions.\\\"\\\\r\\\\n\\\\r\\\\n**Why it\\\\'s wrong:** DX egress selection happens at the **DXGW lookup** (see \\\"Two Route Lookups\\\" below), where **local-region Local Preference is evaluated before AS-path**. For a region that has a **local** DX advertisement, the DXGW picks that local path on LP \\u2014 AS-path (and therefore any prepend) is never consulted. So prepending a local VIF does **not** move that region\\\\'s traffic off its local DX.\\\\r\\\\n\\\\r\\\\n**Where prepending actually acts:** only as the **tiebreaker among remote advertisements** for regions that have **no local** advertisement for the prefix. There, all candidates are remote (LP equal), so the DXGW falls through to AS-path length and the shortest remote advertisement wins. Prepending shapes *which remote DX* those regions use \\u2014 nothing more.\\\\r\\\\n\\\\r\\\\n**Consequence:** Over-prepending a local VIF is not a cross-region steering lever; it only changes that location\\\\'s standing in the remote-tiebreak for *other* regions. To steer cross-region DX preference, use LP communities (which are evaluated before AS-path at the DXGW), not prepending. Confirm the installed per-region result with `get-network-routes` and the advertised paths with `list-virtual-interface-routes`.\\\\r\\\\n\\\\r\\\\n### Foundational Principle: Every Route Table Does Its Own Independent Lookup\\\\r\\\\n\\\\r\\\\nThe single most important mental model for reasoning about AWS routing: **every construct that has its own route table performs its own independent route lookup**, and a packet is evaluated **hop-by-hop** \\u2014 each route table it traverses makes a fresh, local forwarding decision using only the routes present in *that* table. There is no single end-to-end path calculation; the path is the concatenation of per-table decisions.\\\\r\\\\n\\\\r\\\\nRoute-table-owning constructs each do this independently, including:\\\\r\\\\n- **VPC route tables** (subnet \\u2192 destination: local, IGW, NAT, ENI, TGW/VGW/CWAN attachment, peering, endpoint)\\\\r\\\\n- **Transit Gateway route tables** (per-attachment association/propagation; oldest-route tiebreak on equal paths)\\\\r\\\\n- **DX Gateway** (its own BGP path selection; installs one winning path per prefix into each attached CNE/VGW)\\\\r\\\\n- **Cloud WAN Core Network Edge (CNE) route tables** (the CNE evaluation order over propagated routes)\\\\r\\\\n- **On-prem / customer routers** (their own BGP/IGP tables)\\\\r\\\\n\\\\r\\\\nThe routing \\\"outcome\\\" is what you get when you chain these together. Each table\\\\'s chosen next-hop simply hands the packet to the next construct, which then runs *its own* lookup. So to predict or debug a path, walk it table by table: at each hop, ask \\\"what does *this* route table select for this destination, given only the routes it has?\\\" The output (installed route) of one table\\\\'s lookup becomes an input (a propagated/attached route) to the next.\\\\r\\\\n\\\\r\\\\n**Common analysis error:** treating an end-to-end path as one decision, or assuming an attribute (like a DX BGP community or a local preference) that decided at one table still applies at the next. It does not automatically \\u2014 each table applies only its own selection rules over its own routes. (Example: DX LP communities decide at the DXGW\\\\'s table but are not visible inside the Cloud WAN CNE table.)\\\\r\\\\n\\\\r\\\\n### Applying the Principle: DXGW Lookup then CNE Lookup\\\\r\\\\n\\\\r\\\\nIn a DX \\u2192 Cloud WAN egress path, two of these independent lookups sit back-to-back, for the two directions. The DXGW\\\\'s lookup output is the CNE\\\\'s lookup input.\\\\r\\\\n\\\\r\\\\n**Lookup 1 \\u2014 DXGW path selection (on-prem \\u2192 AWS; decides what is INSTALLED into each CNE).**\\\\r\\\\nThe DXGW performs its own selection for a prefix and **installs one winning path into each CNE\\\\'s route table**. Its selection order is:\\\\r\\\\n1. **Local-region Local Preference first** \\u2014 if a DX VIF advertising the prefix is in the same associated region as the CNE, the DXGW prefers it **regardless of AS-path**. That local copy is installed into that region\\\\'s CNE.\\\\r\\\\n2. **If no local advertisement exists for that CNE**, every candidate is remote, so they share the **same (remote) LP** \\u2014 LP is equal. The DXGW then falls through to **AS-path length**, and the **shortest AS-path wins** and is installed into that CNE.\\\\r\\\\n\\\\r\\\\n**Lookup 2 \\u2014 Cloud WAN CNE route lookup (AWS \\u2192 on-prem return path).**\\\\r\\\\nFor return traffic, each CNE runs its own lookup over the routes **propagated into its table** (what Lookup 1 installed, plus routes propagated from other CNEs). This is where the CNE evaluation order (longest-prefix \\u2192 AS-path \\u2192 source preference) applies, operating on the already-propagated entries.\\\\r\\\\n\\\\r\\\\n### Worked Example (from the field diagram)\\\\r\\\\n\\\\r\\\\nOnly **FR5/Frankfurt (eu-central-1)** and **LD5/London (eu-west-1)** advertise `11.0.0.0/8`. Applying **Lookup 1 (DXGW)** per region:\\\\r\\\\n\\\\r\\\\n| CNE region | Has local `/8` advertisement? | DXGW decision | Installed `/8` path |\\\\r\\\\n|---|---|---|---|\\\\r\\\\n| eu-central-1 (FR5) | Yes (FR5 local) | Local-region LP wins \\u2014 AS-path not consulted | **FR5 (local DX)** |\\\\r\\\\n| eu-west-1 (LD5) | Yes (LD5 local) | Local-region LP wins \\u2014 AS-path not consulted | **LD5 (local DX)** |\\\\r\\\\n| ap-southeast-1 | No | FR5 and LD5 **both remote \\u2192 LP equal** \\u2192 fall through to AS-path \\u2192 **FR5 shortest wins** | **FR5** |\\\\r\\\\n| us-east-1 | No | both remote \\u2192 LP equal \\u2192 AS-path \\u2192 **FR5 shortest** | **FR5** |\\\\r\\\\n| us-east-2 | No | both remote \\u2192 LP equal \\u2192 AS-path \\u2192 **FR5 shortest** | **FR5** |\\\\r\\\\n\\\\r\\\\n**Why the non-LD5 regions use FR5\\\\'s advertisement:** they have no local `/8`, so at the DXGW both FR5 and LD5 are **remote** and carry the **same (remote) LP** \\u2014 LP is equal. The DXGW therefore falls to the next attribute, **AS-path length**, where **FR5 is shorter than the (heavily prepended) LD5 advertisement**, so FR5 wins and is installed. This is a straightforward DXGW LP-then-AS-path decision \\u2014 **not** a CNE-to-CNE backbone comparison.\\\\r\\\\n\\\\r\\\\n**Role of the AS-path prepends:** the prepends matter at **Lookup 1 (DXGW)**, specifically as the **tiebreaker when LP is equal** (i.e., only for regions with no local advertisement). They set which remote advertisement (FR5 vs LD5) wins for those regions. They do **not** override the local-region LP for regions that *do* have a local advertisement \\u2014 eu-central-1 and eu-west-1 pick their local DX on LP before AS-path is ever considered. So over-prepending LD5 does not stop eu-west-1 from choosing LD5 locally; it affects LD5\\\\'s standing in the **AS-path tiebreak for the other regions**, where FR5\\\\'s shorter path beats it.\\\\r\\\\n\\\\r\\\\n**Rule of thumb:** Because local-region LP is evaluated before AS-path at the DXGW, a region with a local `/8` advertisement egresses locally regardless of prepends. Use AS-path prepending only to shape which **remote** advertisement wins for regions that have **no local** advertisement (the LP-equal tiebreak). Verify the installed per-region result with `get-network-routes` (Lookup 2\\\\'s table) and the per-VIF advertised paths with `list-virtual-interface-routes` (Lookup 1\\\\'s inputs).\\\\r\\\\n\\\\r\\\\n### Common Pitfall: Confusing Prepend Count with Total AS-Path Length (Origination vs. Transit)\\\\r\\\\n\\\\r\\\\nWhen comparing AS-path lengths at a CNE (Step 2.3), the number that matters is the **total AS-path AWS receives on the VIF**, not the prepend count configured on a site. These are only the same when the advertising site is the **originator** of the prefix.\\\\r\\\\n\\\\r\\\\n**The distinction:**\\\\r\\\\n- **Originating site** \\u2014 the site owns/aggregates the prefix (e.g., `aggregate-address 11.0.0.0/8`). The AS-path AWS sees is just that site\\\\'s ASN, repeated by its prepend count. A `prepend 60208 2` on an originated route \\u2192 AS-path `60208 60208 60208` (length 3).\\\\r\\\\n- **Transiting site** \\u2014 the site does NOT originate the prefix; it **learns it from another site over the on-prem WAN** and re-advertises it to its DX. The AS-path already contains the origin\\\\'s ASN, so the transiting site\\\\'s ASN and prepends stack **on top of** the origin ASN. A `prepend 60208 2` applied to a `/8` learned from origin AS 56608 \\u2192 AS-path `60208 60208 60208 56608` (length **4**, not 3).\\\\r\\\\n\\\\r\\\\n**Config signature to tell them apart:** A site that **matches-and-prepends** a prefix in a route-policy but has **no `aggregate-address`/origination** for it is almost certainly **transiting** a route learned from elsewhere. Do not assume it originates the prefix just because the prefix appears in its outbound policy.\\\\r\\\\n\\\\r\\\\n**Why this matters for path selection:** Step 2.3 compares total path length. If you model a transiting site\\\\'s length as \\\"prepend count + 1\\\" (treating it as an originator) you will miscompute which copy wins \\u2014 often flipping the predicted winner and the failover order. Example: three sites advertise a `/8` that **originates at only one** of them; the two that transit-and-prepend it are longer than a naive prepend-count model suggests, and can tie with the origin.\\\\r\\\\n\\\\r\\\\n**How to get the ground truth:** Never infer AS-path length from prepend config alone when a prefix may be transited. Read the literal path AWS receives per VIF:\\\\r\\\\n```\\\\r\\\\naws directconnect list-virtual-interface-routes --virtual-interface-id \\\\r\\\\n```\\\\r\\\\nThis shows the exact AS-path (and communities) AWS accepted, settling origination-vs-transit and the true length used at Step 2.3.\\\\r\\\\n\\\\r\\\\n## DX Routing Selection \\u2014 General Best Practice\\\\r\\\\n\\\\r\\\\n- **Always use LP communities (`7224:7300/7200/7100`) for expected egress routing** \\u2014 High Preference BGP communities are the recommended mechanism to achieve predictable DX egress behavior.\\\\r\\\\n- **AS-path prepending: within a region ONLY** \\u2014 AS-path prepending can be used to influence egress traffic between DX connections belonging to the **same associated AWS region** (e.g., two DX locations both in the same region). It should NOT be used to influence traffic across regions \\u2014 DXGW\\\\'s local-region LP behavior makes cross-region AS-path prepending unreliable.\\\\r\\\\n- **BGP communities: within region OR across regions** \\u2014 LP communities work for both intra-region path selection (between VIFs in the same region) and cross-region preference (overriding DXGW\\\\'s default local-region LP to prefer a remote DX location globally).\\\\r\\\\n\\\\r\\\\n| Scope | Recommended Mechanism | Why |\\\\r\\\\n|---|---|---|\\\\r\\\\n| Cross-region DX preference | LP Communities (`7224:7300/7200/7100`) | Overrides DXGW local-region LP; evaluated before AS-path |\\\\r\\\\n| Within-region DX preference (same associated region) | AS-path prepending OR LP communities | Both work; AS-path provides granular control between same-region VIFs |\\\\r\\\\n| Never use for cross-region | AS-path prepending alone | DXGW local-region LP overrides AS-path at the DXGW level, making prepending unreliable across regions |\\\\r\\\\n\\\\r\\\\n## Traffic Engineering Patterns\\\\r\\\\n\\\\r\\\\n### Pattern 1: Global DX Preference (Single Primary)\\\\r\\\\n\\\\r\\\\n**Use case:** One DX location should be globally preferred for a prefix, with another as backup.\\\\r\\\\n\\\\r\\\\n**Solution:** Apply tiered LP communities from on-premises routers:\\\\r\\\\n\\\\r\\\\n| DX Location | Community | Role |\\\\r\\\\n|---|---|---|\\\\r\\\\n| Primary DX location | `7224:7300` | HIGH \\u2014 globally preferred |\\\\r\\\\n| Secondary DX location | `7224:7200` | MEDIUM \\u2014 failover |\\\\r\\\\n| All other DX locations | `7224:7100` | LOW \\u2014 last resort |\\\\r\\\\n\\\\r\\\\n**Result:** DXGW selects the primary for ALL CNEs (overriding local-region preference). If the primary fails, the medium-priority location (`7224:7200`) takes over automatically. (Note: \\\"medium\\\" here refers to the medium local-preference community, not the BGP MED attribute.)\\\\r\\\\n\\\\r\\\\n**Tradeoff:** Regions with a local DX that isn\\\\'t the primary will cross the AWS backbone to reach the preferred DX. Traffic still traverses the local-region firewall (service insertion happens at segment level, before the CNE egress decision).\\\\r\\\\n\\\\r\\\\n### Pattern 2: Per-Region DX Preference (Split DXGW)\\\\r\\\\n\\\\r\\\\n**Use case:** Different regions need different primary DX locations (e.g., one region group prefers DX-A, another region group prefers DX-B).\\\\r\\\\n\\\\r\\\\n**Problem:** Communities from a single DXGW are global \\u2014 cannot provide per-region preferences.\\\\r\\\\n\\\\r\\\\n**Solution:** Create separate DXGWs per region group, each with their own VIFs:\\\\r\\\\n- DXGW-A: Region-A DX locations (primary `7224:7300`, secondary `7224:7200`)\\\\r\\\\n- DXGW-B: Region-B DX locations (`7224:7300`)\\\\r\\\\n\\\\r\\\\nBoth DXGWs propagate their path to all CNEs. The CNE route table evaluates them at Step 2.3 (AS-path length). Control which DXGW wins per region by differentiating AS-path length on the VIFs between the two DXGWs.\\\\r\\\\n\\\\r\\\\nUse communities within each DXGW for internal failover. No Cloud WAN Routing Policies needed \\u2014 VIF-level BGP attributes provide per-region control at the CNE route table.\\\\r\\\\n\\\\r\\\\n### Pattern 3: Per-Region Control via More Specific Routes\\\\r\\\\n\\\\r\\\\n**Use case:** On-prem address space is regionally segmented.\\\\r\\\\n\\\\r\\\\n**Solution:** Advertise more-specific prefixes from the corresponding DX location:\\\\r\\\\n- DX location A advertises a more-specific covering Region-A on-prem hosts\\\\r\\\\n- DX location B advertises a more-specific covering Region-B on-prem hosts\\\\r\\\\n- All locations advertise the aggregate as a failover catch-all\\\\r\\\\n\\\\r\\\\n**Result:** Step 1 (longest prefix match) is absolute \\u2014 each CNE route table forwards to the correct DX based on destination address.\\\\r\\\\n\\\\r\\\\n**Requirement:** Only works if on-prem address space is subdivided into regional ranges. Does NOT work for flat supernets.\\\\r\\\\n\\\\r\\\\n## Multi-Region Centralized Internet Egress\\\\r\\\\n\\\\r\\\\n### Non-Local Exit Point Selection\\\\r\\\\n\\\\r\\\\nIn architectures where internet egress is centralized (e.g., a single region hosts the NAT/internet gateway or firewall for outbound traffic), Cloud WAN must route traffic from remote regions to the centralized exit point. Understanding how non-local exit points are selected is critical:\\\\r\\\\n\\\\r\\\\n- Each CNE evaluates routes per the standard route evaluation order\\\\r\\\\n- If the default route (0.0.0.0/0) or internet-bound prefix is only advertised from the centralized region\\\\'s attachment (e.g., via TGW Connect or VPN), all CNEs will install that path\\\\r\\\\n- Remote CNEs forward internet-bound traffic across the AWS backbone to the centralized region\\\\'s CNE\\\\r\\\\n\\\\r\\\\n### AS-Path Prepending for Deterministic Exit (via TGW/VPN)\\\\r\\\\n\\\\r\\\\nWhen routes are propagated from DX or VPN **via TGW** to Cloud WAN (TGW peering attachment), AS-path length information is preserved. This means:\\\\r\\\\n\\\\r\\\\n- **AS-path prepending can define a deterministic exit path** \\u2014 if multiple regions advertise the same prefix via TGW attachments to Cloud WAN, prepending on less-preferred paths ensures remote CNEs choose the shorter AS-path (preferred exit)\\\\r\\\\n- **More-specific routing also applies** \\u2014 advertising a more-specific prefix from the preferred exit region wins at Step 1 (longest prefix match) regardless of AS-path\\\\r\\\\n\\\\r\\\\n**Note:** This behavior differs from DX-attached prefixes where DXGW\\\\'s local-region LP can override AS-path. When routes come via TGW attachments to Cloud WAN, AS-path evaluation at the CNE operates without DXGW LP interference \\u2014 making prepending a valid and effective traffic engineering tool in this context.\\\\r\\\\n\\\\r\\\\n### Example: Centralized Internet Egress\\\\r\\\\n\\\\r\\\\n- Region-A TGW advertises `0.0.0.0/0` to Cloud WAN via TGW peering (short AS-path)\\\\r\\\\n- Region-B TGW also advertises `0.0.0.0/0` but with prepending (longer AS-path)\\\\r\\\\n- All CNEs prefer Region-A as the internet exit (Step 2.3: shorter AS-path wins)\\\\r\\\\n- If Region-A fails, Region-B\\\\'s longer path takes over as backup\\\\r\\\\n\\\\r\\\\n## Regional Firewall Inspection\\\\r\\\\n\\\\r\\\\n- Cloud WAN segment policy / service insertion routes traffic through the **same-region firewall** before it reaches the CNE\\\\r\\\\n- Traffic flow: VPC \\u2192 local firewall (service insertion) \\u2192 returns to local CNE \\u2192 CNE route table \\u2192 DXGW \\u2192 DX\\\\r\\\\n- With a local DX advertisement present, the DXGW installs the local path into the local CNE (local-region LP, Lookup 1) \\u2014 traffic stays at the local CNE for egress, preserving the local inspection chain\\\\r\\\\n- If the region has no local DX advertisement (or a community forces a remote DX to win), the installed path points out of region, and traffic leaves the local region for egress \\u2014 potentially bypassing local firewall inspection\\\\r\\\\n- **Requirement:** Firewall VPCs must exist in every region with workloads\\\\r\\\\n\\\\r\\\\n## Directional Control (Mapped to the Two Lookups)\\\\r\\\\n\\\\r\\\\nThe two route lookups map to the two directions. Keep these straight \\u2014 they are governed by different services and different mechanisms.\\\\r\\\\n\\\\r\\\\n| Direction | Governing lookup | Controlled by | Mechanism |\\\\r\\\\n|---|---|---|---|\\\\r\\\\n| **On-prem \\u2192 AWS** (which DX path is INSTALLED into each CNE) | **Lookup 1 \\u2014 DXGW path selection** | On-prem BGP attributes the DXGW sees: DX LP communities (`7224:7300/7200/7100`), then AS-path as the LP-equal tiebreak | DXGW evaluates local-region LP first, then AS-path; installs one winning path per prefix into each CNE |\\\\r\\\\n| **AWS \\u2192 On-prem** (return path from each CNE) | **Lookup 2 \\u2014 Cloud WAN CNE route lookup** | The routes propagated into the CNE table (from Lookup 1 + other CNEs), evaluated by the CNE order | CNE evaluation: longest-prefix \\u2192 AS-path \\u2192 source preference, over already-propagated routes |\\\\r\\\\n\\\\r\\\\nNotes:\\\\r\\\\n- DX LP communities set the LP the **DXGW** uses in Lookup 1 \\u2014 they are the recommended lever for steering which DX path is installed. They are **not visible inside Cloud WAN** and do not act at Lookup 2.\\\\r\\\\n- If on-prem AS-path prepending was being used to influence selection, prefer LP communities at the DXGW (Lookup 1); over-prepending a local VIF does not achieve cross-region steering (see the AS-Path Prepending pitfall above).\\\\r\\\\n\\\\r\\\\n## Cleanup Recommendations\\\\r\\\\n\\\\r\\\\n- **Prefer LP communities over prepending for DX egress steering.** Communities set the LP the DXGW uses in Lookup 1 and are evaluated before AS-path; they can steer cross-region preference, which prepending cannot (local-region LP overrides AS-path for any region with a local advertisement).\\\\r\\\\n- **Do not use prepending on a local VIF as a cross-region lever.** At the DXGW, a region with a local advertisement selects it on LP regardless of prepends. Prepending only affects the AS-path tiebreak among **remote** advertisements for regions that have no local advertisement \\u2014 so its only legitimate use here is choosing which remote DX those regions fall to.\\\\r\\\\n- **Once communities are in place, remove leftover prepends** as a cleanup step, keeping only any within-region or remote-tiebreak prepends that serve a confirmed purpose.\\\\r\\\\n- Always confirm the actual outcome with `get-network-routes` (installed per-CNE result) and `list-virtual-interface-routes` (the AS-path/communities the DXGW actually received) rather than reasoning from configured prepend counts alone.\\\\r\\\\n'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:29.255000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "71f32cf0-d741-4b1c-bbef-25113fedd3ca", + "executionId": "e08b9db9-0cd3-4afc-a820-9c47e5428d53", + "recordId": "a47f446b-fc00-40bf-a784-2767e9694652", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 5.1, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:29.365000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "71f32cf0-d741-4b1c-bbef-25113fedd3ca", + "executionId": "e08b9db9-0cd3-4afc-a820-9c47e5428d53", + "recordId": "6040f85a-f885-4c80-a168-cdf32d4e5bf6", + "content": "{\"id\": \"6040f85a-f885-4c80-a168-cdf32d4e5bf6\", \"role\": \"assistant\", \"content\": [{\"text\": \"Here's how AWS decides this, and the levers you have to fix it.\\n\\n## Why this happens: two independent route lookups\\n\\nTraffic in a DX \\u2192 Cloud WAN path crosses two separate route tables, each making its own lookup \\u2014 DXGW's lookup output becomes the CNE's lookup input.\\n\\n**Lookup 1 \\u2014 DX Gateway path selection** (decides what gets *installed* into each region's Core Network Edge route table):\\n1. **Local-region Local Preference first.** If a DX VIF advertising that prefix exists in the *same associated region* as a given CNE, DXGW picks it \\u2014 AS-path is never even consulted.\\n2. **If there's no local advertisement** for that region, all candidates are remote and share equal LP, so DXGW falls through to **AS-path length** \\u2014 shortest wins.\\n\\n**Lookup 2 \\u2014 Cloud WAN CNE route lookup** (the return path, AWS \\u2192 on-prem): each CNE evaluates its own table in this order \\u2014 longest prefix match \\u2192 static \\u2192 VPC-propagated (same region) \\u2192 unequal AS-path/MED (shortest wins) \\u2192 equal AS-path/MED source preference (DXGW-propagated \\u2192 Cloud WAN Connect \\u2192 S2S VPN \\u2192 other).\\n\\n**The likely cause of your symptom:** your local region has *no local advertisement for that aggregate* \\u2014 maybe only a more-specific is advertised locally, or the aggregate is only advertised from a remote DX location. With no local copy, DXGW treats every candidate as remote (LP equal) and picks whichever has the shortest AS-path \\u2014 which may well be a remote location, not yours. The result: DXGW installs a remote path into your local CNE, and your traffic leaves the region even though a local DX exists.\\n\\nA second possibility: an LP community (`7224:7300/7200/7100`) applied upstream is explicitly forcing a remote DX to win globally \\u2014 that overrides DXGW's default local-region preference by design.\\n\\n## How to make the local DX win\\n\\n1. **Advertise the aggregate locally too** (or a covering more-specific). Longest-prefix-match at Step 1 is absolute, so if local-region DX advertises a more-specific that covers your traffic, it wins outright regardless of anything else downstream.\\n2. **Use LP communities, not AS-path prepending, for cross-region steering.** Tag the local DX's advertisement `7224:7300` (High) \\u2014 LP communities are evaluated *before* AS-path at the DXGW and work both within- and cross-region. Prepending only changes the tiebreak among *remote* advertisements for regions with no local copy \\u2014 it cannot pull a region off its local DX, and it's unreliable as a cross-region lever precisely because local-region LP overrides it.\\n3. If you were relying on prepending a local VIF to try to influence this \\u2014 stop; per DX docs that configuration doesn't do what it looks like it does.\\n\\n## Validate with read-only commands\\n\\n```\\naws directconnect list-virtual-interface-routes --virtual-interface-id # confirms AS-path/communities DXGW actually received\\naws networkmanager get-network-routes --core-network-id --route-table-identifier # confirms what's actually installed per CNE\\naws directconnect describe-locations # confirms the associated region for the DX location (don't assume by geography)\\n```\\n\\nWant me to check the actual DXGW/Cloud WAN route tables and VIF advertisements in your account to confirm which of these is happening?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:29.439000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "71f32cf0-d741-4b1c-bbef-25113fedd3ca", + "executionId": "e08b9db9-0cd3-4afc-a820-9c47e5428d53", + "recordId": "618298f3-e361-4556-9610-da16ae1f2e7b", + "content": "{\"id\": \"0584ee66-acc3-4c4c-adce-31f701149878\", \"role\": \"assistant\", \"content\": [{\"type\": \"text\", \"text\": \"Here's how AWS decides this, and the levers you have to fix it.\\n\\n## Why this happens: two independent route lookups\\n\\nTraffic in a DX \\u2192 Cloud WAN path crosses two separate route tables, each making its own lookup \\u2014 DXGW's lookup output becomes the CNE's lookup input.\\n\\n**Lookup 1 \\u2014 DX Gateway path selection** (decides what gets *installed* into each region's Core Network Edge route table):\\n1. **Local-region Local Preference first.** If a DX VIF advertising that prefix exists in the *same associated region* as a given CNE, DXGW picks it \\u2014 AS-path is never even consulted.\\n2. **If there's no local advertisement** for that region, all candidates are remote and share equal LP, so DXGW falls through to **AS-path length** \\u2014 shortest wins.\\n\\n**Lookup 2 \\u2014 Cloud WAN CNE route lookup** (the return path, AWS \\u2192 on-prem): each CNE evaluates its own table in this order \\u2014 longest prefix match \\u2192 static \\u2192 VPC-propagated (same region) \\u2192 unequal AS-path/MED (shortest wins) \\u2192 equal AS-path/MED source preference (DXGW-propagated \\u2192 Cloud WAN Connect \\u2192 S2S VPN \\u2192 other).\\n\\n**The likely cause of your symptom:** your local region has *no local advertisement for that aggregate* \\u2014 maybe only a more-specific is advertised locally, or the aggregate is only advertised from a remote DX location. With no local copy, DXGW treats every candidate as remote (LP equal) and picks whichever has the shortest AS-path \\u2014 which may well be a remote location, not yours. The result: DXGW installs a remote path into your local CNE, and your traffic leaves the region even though a local DX exists.\\n\\nA second possibility: an LP community (`7224:7300/7200/7100`) applied upstream is explicitly forcing a remote DX to win globally \\u2014 that overrides DXGW's default local-region preference by design.\\n\\n## How to make the local DX win\\n\\n1. **Advertise the aggregate locally too** (or a covering more-specific). Longest-prefix-match at Step 1 is absolute, so if local-region DX advertises a more-specific that covers your traffic, it wins outright regardless of anything else downstream.\\n2. **Use LP communities, not AS-path prepending, for cross-region steering.** Tag the local DX's advertisement `7224:7300` (High) \\u2014 LP communities are evaluated *before* AS-path at the DXGW and work both within- and cross-region. Prepending only changes the tiebreak among *remote* advertisements for regions with no local copy \\u2014 it cannot pull a region off its local DX, and it's unreliable as a cross-region lever precisely because local-region LP overrides it.\\n3. If you were relying on prepending a local VIF to try to influence this \\u2014 stop; per DX docs that configuration doesn't do what it looks like it does.\\n\\n## Validate with read-only commands\\n\\n```\\naws directconnect list-virtual-interface-routes --virtual-interface-id # confirms AS-path/communities DXGW actually received\\naws networkmanager get-network-routes --core-network-id --route-table-identifier # confirms what's actually installed per CNE\\naws directconnect describe-locations # confirms the associated region for the DX location (don't assume by geography)\\n```\\n\\nWant me to check the actual DXGW/Cloud WAN route tables and VIF advertisements in your account to confirm which of these is happening?\"}]}", + "createdAt": "2026-10-27T17:05:29.523000-05:00", + "recordType": "final_response" + } +] \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-local-dx-preference/without_skill/functional-tests-results.json b/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-local-dx-preference/without_skill/functional-tests-results.json new file mode 100644 index 00000000..aa3a5b9e --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-local-dx-preference/without_skill/functional-tests-results.json @@ -0,0 +1,69 @@ +{ + "version": "v1", + "iteration": 1, + "eval_id": "cloudwan-local-dx-preference", + "run_type": "without_skill", + "success": true, + "error": null, + "understanding_agent_space_skill": null, + "tests": { + "trigger": null, + "expected_output": { + "result": "failed", + "reasoning": "The expected output specifically requires mentioning the well-known AWS Cloud WAN local-preference community values (7224:7300/7200/7100) as the recommended mechanism to prefer local DX, and/or AS-path normalization, alongside longest-prefix-match and AS-path evaluation order at the CNE. The agent's response does correctly explain longest-prefix-match as step 1 and touches on AS-path/MED tiebreaks, which partially aligns. However, it explicitly states 'local-preference itself isn't user-settable directly' and suggests using MED/AS-path manipulation instead, which contradicts the expected guidance to use the specific LP community tagging (7224:7300/7200/7100) that AWS Cloud WAN actually supports for this exact use case. This is a factual gap/error relative to the expected output's core recommendation. The agent never mentions the specific community values or confirms LP communities are a real supported mechanism, which was the key expected guidance.\n\"local-preference itself isn't user-settable directly \u2014 practically this means using the routing policy's MED/AS-path-length actions to make the local-region advertisement look 'better' to the evaluation algorithm.\"", + "confidence": "medium" + }, + "assertions": { + "assertion_results": [ + { + "text": "references longest prefix match in Cloud WAN route evaluation", + "evaluator": "llm", + "passed": true, + "evidence": "The output states: 'Most specific prefix wins, period. ... longest prefix match sends that traffic there instead, regardless of path type or BGP attributes.'", + "reasoning": "The output explicitly references longest prefix match as the first rule in Cloud WAN route evaluation.", + "confidence": "high" + }, + { + "text": "explains that AS-path length is evaluated at the CNE and can cause a remote path to win", + "evaluator": "llm", + "passed": true, + "evidence": "The output lists tiebreak order including 'Unequal AS-path length / MED (shorter AS-path, then lower MED wins)' and later says 'The AS-path or MED on the DX-advertised route is longer/higher than the competing route, so it loses at the BGP-attribute tiebreak before DX preference is even applied.'", + "reasoning": "The output explains AS-path length evaluation as part of the route selection process and explicitly states a remote path can win due to AS-path/MED comparison, implying this occurs at the CNE (core network edge) since the whole explanation is framed as happening 'at each core network edge'.", + "confidence": "high" + }, + { + "text": "recommends LP communities (7224:7300/7200/7100) as a deterministic remedy", + "evaluator": "llm", + "passed": false, + "evidence": "The output says 'local-preference itself isn't user-settable directly \u2014 practically this means using the routing policy's MED/AS-path-length actions' and never mentions the specific LP community values 7224:7300, 7224:7200, or 7224:7100.", + "reasoning": "The assertion requires recommending specific LP communities (7224:7300/7200/7100) as a deterministic remedy for preferring local DX. The agent explicitly states local-preference is not directly settable and instead recommends MED/AS-path manipulation, never mentioning the actual AWS-documented LP community tags used for this exact purpose in Cloud WAN/Direct Connect. This is a factual gap/inaccuracy relative to the assertion's expectation.", + "confidence": "high" + }, + { + "text": "does not contain 'error'", + "evaluator": "llm", + "passed": true, + "evidence": "No occurrence of the word 'error' appears anywhere in the output text.", + "reasoning": "A plain text search of the response shows no instance of the word 'error'.", + "confidence": "high" + } + ], + "summary": { + "passed": 3, + "failed": 1, + "errored": 0, + "low_confidence": 0, + "total": 4, + "pass_rate": 0.75 + } + } + }, + "metrics": { + "runtime": "16s", + "cost": "$0.14", + "context_window": { + "utilization": "4.7%", + "compaction_count": 0 + } + } +} \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-local-dx-preference/without_skill/outputs/journal_records.json b/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-local-dx-preference/without_skill/outputs/journal_records.json new file mode 100644 index 00000000..aac62ae7 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-local-dx-preference/without_skill/outputs/journal_records.json @@ -0,0 +1,98 @@ +[ + { + "agentSpaceId": "a110b576-edea-40ad-a583-332cfd2f2396", + "executionId": "85f0df89-08a7-4486-a233-eea2ec390f97", + "recordId": "1cfdc39e-a438-4322-a4e7-ae0abb241a00", + "content": "{\"id\": \"1cfdc39e-a438-4322-a4e7-ae0abb241a00\", \"role\": \"user\", \"content\": [{\"text\": \"In AWS Cloud WAN, traffic for an aggregate prefix is leaving the local region instead of using the local Direct Connect. How does route evaluation decide this and how can I prefer the local DX?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:04:59.903000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "a110b576-edea-40ad-a583-332cfd2f2396", + "executionId": "85f0df89-08a7-4486-a233-eea2ec390f97", + "recordId": "2695065a-6f94-4bc0-9f72-8ac851b202fa", + "content": "{\"id\": \"013674c2-689b-4982-8a45-d9fcbd64d0e4\", \"role\": \"assistant\", \"content\": [{\"type\": \"chat_title\", \"title\": \"AWS Cloud WAN traffic routing to local Direct Connect\"}]}", + "createdAt": "2026-10-27T17:05:00.495000-05:00", + "recordType": "chat_title" + }, + { + "agentSpaceId": "a110b576-edea-40ad-a583-332cfd2f2396", + "executionId": "85f0df89-08a7-4486-a233-eea2ec390f97", + "recordId": "cf7928a3-6084-4036-b0a6-d69af5fa6124", + "content": "{\"id\": \"78f6f2f2-8886-461d-a545-df9ee562ffc7\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This is a question about AWS Cloud WAN's routing behavior, particularly how it evaluates aggregate prefixes alongside Direct Connect preference. Since this involves a specific factual claim about AWS service behavior rather than the user's own setup, I want to verify it against AWS documentation instead of relying purely on training data.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:05.821000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "a110b576-edea-40ad-a583-332cfd2f2396", + "executionId": "85f0df89-08a7-4486-a233-eea2ec390f97", + "recordId": "26ccac1f-b712-4647-a9ee-246aab1c61a7", + "content": "{\"id\": \"26ccac1f-b712-4647-a9ee-246aab1c61a7\", \"role\": \"assistant\", \"content\": [{\"text\": \"Let me verify this against AWS Cloud WAN documentation before answering, since route evaluation behavior is exactly the kind of detail that can shift between releases.\", \"type\": \"text\"}, {\"id\": \"tooluse_Mkko0chj5r9O0wcHQvUZYg\", \"tool_name\": \"verify_aws_claim\", \"input\": {\"claim\": \"AWS Cloud WAN route evaluation and preference for Direct Connect attachments with aggregate prefixes across regions\"}, \"type\": \"tool_use\", \"display_intent\": \"Look up AWS docs on Cloud WAN route evaluation and Direct Connect preference\"}, {\"id\": \"tooluse_qYdyckFlnikQ01OlOgrKf2\", \"tool_name\": \"verify_aws_claim\", \"input\": {\"claim\": \"AWS Cloud WAN longest prefix match and route preference order between segments, attachments, and static routes\"}, \"type\": \"tool_use\", \"display_intent\": \"Look up AWS docs on Cloud WAN route preference and path selection order\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:05.858000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "a110b576-edea-40ad-a583-332cfd2f2396", + "executionId": "85f0df89-08a7-4486-a233-eea2ec390f97", + "recordId": "79cb187e-9054-4105-8dbf-ba40a6dfe63d", + "content": "{\"id\": \"718b6383-089b-42ef-9b07-bdefedf137bf\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"1881\"}]}", + "createdAt": "2026-10-27T17:05:05.896000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "a110b576-edea-40ad-a583-332cfd2f2396", + "executionId": "85f0df89-08a7-4486-a233-eea2ec390f97", + "recordId": "f5c62e31-3f89-4626-9555-d5a6a7c447d3", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 3.8, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:05.946000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "a110b576-edea-40ad-a583-332cfd2f2396", + "executionId": "85f0df89-08a7-4486-a233-eea2ec390f97", + "recordId": "29c85ecf-2945-4fba-98f7-4eb853443abc", + "content": "{\"id\": \"24a8f8ba-5a66-46c1-a55c-b39e70d7ae15\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_Mkko0chj5r9O0wcHQvUZYg\", \"status\": \"success\", \"content\": [{\"text\": \"## Documentation search results\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"Direct Connect gateway attachments in AWS Cloud WAN\\\",\\\"context\\\":\\\"## Route propagation\\\\n\\\\nA Direct Connect gateway attachments support BGP-based dynamic routing for both inbound and\\\\noutbound directions.\\\\n\\\\nFor inbound routes,\\\\n\\\\n* Cloud WAN learns BGP routes advertised from your on-premises location\\\\n via the Direct Connect gateway and the transit virtual interface. Routes are learnt\\\\n in the segment route-tables of the associated core network edges for the\\\\n attachment.\\\\n* Routes learned in segment route table can be routed across all\\\\n AWS Regions for that segment.\\\\n* Cloud WAN follows the route evaluation order for the same prefixes learned\\\\n over multiple attachments. See Route evaluation for\\\\n more information.\\\\n* Direct Connect gateway will advertise routes from all on-premises locations towards\\\\n all segment route-tables of the associated core network edges for the attachment.\\\\n\\\\nFor outbound routes,\\\\n\\\\n* Cloud WAN propagates routes from the segment route table to the Direct Connect\\\\n gateway, which in turn advertises these routes over transit virtual interfaces\\\\n to your on-premises locations via BGP.\\\\n* Each core network edge associated with the Direct Connect gateway attachment\\\\n advertises only its local routes towards the Direct Connect gateway.\\\\n* The AS_PATH BGP attribute is retained in these route advertisements to your\\\\n on-premises locations. For more information about AS_PATH and BGP, see Private virtual interface and transit virtual interface routing\\\\n policies in the *AWS Direct Connect User\\\\n Guide*\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/network-manager/latest/cloudwan/cloudwan-dxattach-about.html\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"Simplify global hybrid connectivity with AWS Cloud WAN and AWS Direct Connect integration | Networking & Content Delivery\\\",\\\"context\\\":\\\"## Things to know\\\\n\\\\n* You can associate a Direct Connect gateway only with a single AWS Cloud WAN segment. You can have multiple Direct Connect gateways associated to the same segment. You can also have different Direct Connect gateways associated to multiple Cloud WAN segments. However, you cannot have the same Direct Connect gateway associated to multiple segments.\\\\n* When you associate a Direct Connect gateway with an AWS Cloud WAN segment using a Direct Connect attachment, you can choose all or a subset of the Cloud WAN CNEs. We recommend associating the attachment with all CNEs where the respective segment is configured, unless you have bespoke routing requirements, as shown in Scenario 3. If you choose to associate the Direct Connect gateway attachment with only a subset of the CNEs, the Direct Connect gateway receives only the routes from local attachments of the associated CNEs.\\\\n* Direct Connect BGP community tags are only relevant to the Direct Connect gateway, and do not influence the Cloud WAN core network routing decisions.\\\\n* Cloud WAN route evaluation algorithm is detailed here. Remember that MED is a non-transitive BGP attribute, therefore the Direct Connect gateway resets MED on all routes advertised to Cloud WAN.\\\\n* The new Direct Connect attachment on AWS Cloud WAN allows for routes to be propagated dynamically in both directions, without the need of using allowed prefixes list. The use of allowed prefixes list is still required for the Transit Gateway association to a Direct Connect gateway. If you attach a Direct Connect gateway to an isolated segment, VPC routes are not propagated to your on-premises routers, and on-premises routes are not propagated to the core network segment.\\\\n* You can attach Direct Connect gateways from multiple accounts to your AWS Cloud WAN core network following the steps described in the documentation.\\\\n* The quotas for the number of routes you can advertise from your core network segment to a Direct Connect gateway attachment, and the number of routes you can advertise from your on-premises environments on a Direct Connect virtual interface are detailed in the AWS Cloud WAN documentation and AWS Direct Connect documentation\\\",\\\"url\\\":\\\"https://aws.amazon.com/blogs/networking-and-content-delivery/simplify-global-hybrid-connectivity-with-aws-cloud-wan-and-aws-direct-connect-integration/\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"AWS Cloud WAN Routing Policy: Real-World Global Network Scenarios \\u2013 Part 2 | Networking & Content Delivery\\\",\\\"context\\\":\\\"### How it works\\\\n\\\\nAll three DCs advertise all three prefixes for resiliency, but each prefix belongs to a specific DC. When AWS Cloud WAN learns 10.1.0.0/16 from all three DXGWs, the routing policies assign it local preference 300 from DXGW 65010 (DC1, the owner), 200 from DXGW 65011 (DC2), and 100 from DXGW 65012 (DC3). AWS Cloud WAN selects the path with the highest local preference, steering traffic for 10.1.0.0/16 through DC1. If DC1\\u2019s Direct Connect path fails, traffic automatically falls back to DC2 (local preference 200), then DC3 (local preference 100). The same logic applies for DC2-owned prefixes (primary through DXGW 65011, secondary through 65012, tertiary through 65010) and DC3-owned prefixes (primary through DXGW 65012, secondary through 65011, tertiary through 65010).\\\\n\\\\nAt the DXGW level, an additional layer of path control is available through BGP communities on the VIFs. Each DXGW has two VIFs, one primary and one secondary, and community tags determine which VIF is preferred for outbound traffic: 7224:7300 for high preference and 7224:7100 for low preference. Alternatively, 7224:7200 can be applied to both VIFs for active-active ECMP load balancing.\\\\n\\\\nWith this configuration, AWS Cloud WAN and the DXGWs steer outbound traffic toward on premises using inbound route preferences. To achieve symmetric routing, on-premises devices should be configured to prefer the same paths for traffic flowing from on premises to AWS.\\\\n\\\\n**Note:** Prefixes that do not match any of the three defined prefix lists pass through with no explicit local preference set, resulting in non-deterministic path selection across all three DXGWs. Make sure prefix lists are kept up to date as new prefixes are advertised from on premises\\\",\\\"url\\\":\\\"https://aws.amazon.com/blogs/networking-and-content-delivery/aws-cloud-wan-routing-policy-real-world-global-network-scenarios-part-2/\\\"}]}}\\n\\n## Recent AWS announcements (What's New)\\n\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"AWS Direct Connect introduces inbound prefix controls and higher prefix scale - AWS\\\",\\\"context\\\":\\\"AWS Direct Connect has introduced new capabilities in the form of inbound prefix controls and higher prefix scale. This update allows users to allocate and manage inbound route-prefix allocations for private and transit virtual interfaces (VIFs) based on their workload needs. The maximum number of prefixes that can be accepted on a VIF has increased from 100 to 1,000 for both IPv4 and IPv6. This change eliminates the need for architectural workarounds like summarizing routes or segmenting across multiple VIFs or connections.\\\\n\\\\nInbound prefix controls have introduced new prefix capacity pools at the dedicated connection level and at the Direct Connect gateway (DXGW) level. When creating or updating a VIF, users allocate a specific number of prefixes to it, which draws from the dedicated connection's pool and the DXGW's pool when attached. This allows users to right-size prefix capacity per workload. Connection pool sizes scale with connection speed, and link aggregation group (LAG) pools scale with the number of member connections.\\\\n\\\\nUsers can configure prefix allocations using the AWS Direct Connect console or CLI/API. Inbound prefix controls are available at no additional cost in all commercial AWS Regions where AWS Direct Connect is available, AWS GovCloud Regions (US-East and US-West), as well as the Amazon Web Services China (Beijing) Region, operated by Sinnet, and the Amazon Web Services China (Ningxia) Region, operated by NWCD\\\",\\\"url\\\":\\\"https://aws.amazon.com/about-aws/whats-new/2026/08/aws-direct-connect-new-prefix-controls/\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\n\\\\n\\\\nServices: directconnect, vpc, transitgateway\\\\nTasks: configure, secure, deploy\\\\nPersona: network-admin, devops, developer, sre\\\\nWorkload: networking, hybrid-cloud\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"transitgateway\\\",\\\"skill_description\\\":\\\"Configures AWS Transit Gateway: creating a hub and attaching VPCs, segmenting traffic with route tables, centralizing egress and forcing east-west traffic between VPCs through a central firewall via transit gateway route tables and appliance-mode attachments, connecting on-premises networks over the transit gateway side of a Site-to-Site VPN or Direct Connect attachment (including ECMP to aggregate bandwidth across multiple VPN tunnels), peering transit gateways across Regions, migrating from a VPC peering mesh, and routing IP multicast. Applicable when the transit gateway is the resource being configured: connecting many VPCs, isolating environments, routing egress or east-west traffic through the hub for inspection, reaching on-premises over the hub, linking Regions, or moving off a peering mesh. Not applicable for single-VPC routing, VPC peering between two VPCs (vpcpeering skill), Direct Connect gateway or virtual interface setup (directconnect skill), or Route 53 DNS work.\\\\n\\\\n\\\\nServices: transitgateway, vpc, directconnect, sitetositevpn, gatewayloadbalancer\\\\nTasks: configure, secure, deploy, migrate\\\\nPersona: developer, devops, network-admin, security-engineer, sre\\\\nWorkload: networking, security, hybrid, multi-region\\\\n*\\\",\\\"skill_name\\\":\\\"transitgateway\\\"},{\\\"rank_order\\\":4,\\\"title\\\":\\\"aws-networking\\\",\\\"skill_description\\\":\\\"Routes AWS networking requests to the correct service skill for implementation. Covers Route 53 (DNS, health checks, routing policies, Resolver, DNS Firewall), CloudFront (caching, edge, OAC, mTLS, signed URLs), Transit Gateway (multi-VPC hub, segmentation, centralized egress), Direct Connect (hybrid link, DX Gateway, MACsec), Site-to-Site VPN (IPsec tunnels, static or BGP), WAF (web ACLs, AWS Managed Rules, rate-based rules, Bot and Fraud Control), and Shield Advanced (L3/L4 DDoS). Applicable when creating, configuring, troubleshooting, or designing across these services, choosing between them, or diagnosing connectivity or traffic-filtering issues. Not for VPC subnets and route tables, load balancers, VPC endpoints, PrivateLink, API Gateway, IAM policy logic, container or serverless networking, or IaC authoring.\\\\n\\\\nServices: route53, route53resolver, cloudfront, directconnect, transitgateway, vpn, waf, shield\\\\nTasks: configure, troubleshoot, debug, deploy, secure, migrate\\\\nPersona: developer, network-engineer, devops, security-engineer\\\\nWorkload: networking, security\\\\n*\\\",\\\"skill_name\\\":\\\"aws-networking\\\"}]}}\"}]}], \"label\": \"Look up AWS docs on Cloud WAN route evaluation and Direct Connect preference\"}", + "createdAt": "2026-10-27T17:05:07.848000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "a110b576-edea-40ad-a583-332cfd2f2396", + "executionId": "85f0df89-08a7-4486-a233-eea2ec390f97", + "recordId": "e2bc4943-591a-4d4f-aa15-e8a6fcb08f8a", + "content": "{\"id\": \"9dc75e3a-89ce-42cd-9f2c-983bbf102f62\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_qYdyckFlnikQ01OlOgrKf2\", \"status\": \"success\", \"content\": [{\"text\": \"## Documentation search results\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"Route evaluation\\\",\\\"context\\\":\\\"# Route evaluation\\\\n\\\\nCloud WAN evaluates routes at each core network edge in the following order:\\\\n\\\\n1. The most specific route for the destination\\\\n2. For routes with the same destination IP address, but different targets, the\\\\n following route priority is used:\\\\n\\\\n 1. Static routes\\\\n 2. System routes. For more information, see Routing behavior for 1:1 NAT in AWS Cloud WAN.\\\\n 3. VPC-propagated routes in the same Region.\\\\n 4. For dynamic routes received at the core network with an\\\\n *unequal* AS path length and/or MED BGP\\\\n attributes, Cloud WAN evaluates them in the following order:\\\\n\\\\n 1. AS path length\\\\n 2. MED\\\\n 5. For dynamic routes received at the core network with\\\\n *equal* AS path length and MED BGP attributes,\\\\n Cloud WAN evaluates them in the following order:\\\\n\\\\n 1. Direct Connect gateway-propagated routes.\\\\n 2. Cloud WAN Connect-propagated routes in the same\\\\n Region.\\\\n 3. Site-to-Site VPN-propagated routes in the same\\\\n Region.\\\\n 4. Routes propagated from other sources, such as transit gateway peering\\\\n and core network edges in other remote Regions\\\\n over the AWS global infrastructure. If identical routes are\\\\n received from two or more sources, a single attachment will be\\\\n chosen in a deterministically random manner (this occurs on a per\\\\n segment/network function group basis so you could see different destinations\\\\n for the same route across different segment/network function groups)\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/network-manager/latest/cloudwan/cloudwan-route-evaluation.html\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"AWS Cloud WAN Routing policies\\\",\\\"context\\\":\\\"## Key considerations\\\\n\\\\nThe following is a list of considerations that should be taken into account before using Cloud WAN Routing Policies:\\\\n\\\\n### Route evaluation and ordering\\\\n\\\\n* Route summarization will remove all matched prefixes and replace them with a single summarized route. The summarized prefix will be advertised at the same time as matched prefixes are withdrawn.\\\\n* The allow and drop actions are terminal. When a route matches either one, no further rules are evaluated for that route. Other actions, such as set-med and summarize, are non-terminal, so processing continues. A route that reaches the end of all rules without matching a terminal action is allowed, with any modifications applied.\\\\n* Segment share policies are applied after attachment policies.\\\\n* Routing policies associated across segments and Regions are unidirectional.\\\\n* Routing policies are not supported for network function groups (service insertion).\\\\n\\\\n### Attachment type support\\\\n\\\\n* VPC attachments don't support BGP attribute modification.\\\\n* Summarization only works outbound and on BGP-capable attachments.\\\\n* No BGP community support on Direct Connect and TGW peering attachments.\\\\n* TGW route table attachments that use the same peering and are associated with the same segment share the same outbound routing policies across all similar attachments. For example, if attachment-1 has outbound routing policy 1 on segment prod and peering 1, and attachment-2 has outbound routing policy 2 on the same segment and peering, both policies apply to both attachments.\\\\n\\\\n### BGP and ASN constraints\\\\n\\\\n* ASNs specified in the routing policy (replace/remove ASN, community tags) cannot overlap with the ASN range specified in the core network configuration. This also means you cannot advertise communities into a core network that contain an ASN currently in use by the core network.\\\\n* Replace ASN is not supported cross-Region (CNE-to-CNE).\\\\n* External AWS devices cannot advertise routes with BGP communities containing internal ASNs.\\\\n* Route summarization does not preserve the BGP attributes of the matched prefixes. The summarized route is advertised as a new route with reset attribute values: MED of 100, local-preference of 0, an empty AS-path, and empty communities.\\\\n\\\\n### Prefix lists and visibility\\\\n\\\\n* Prefix list aliases must be unique per prefix list core network association.\\\\n* Prefix list modifications of entry values to the underlying core network routing state may not align with the prefix list state.\\\\n* The `list-core-network-routing-information` API shows the routing information before routing policies have been applied\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/network-manager/latest/cloudwan/cloudwan-routing-policies.html\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"AWS Cloud WAN Routing Policy: Real-World Global Network Scenarios \\u2013 Part 2 | Networking & Content Delivery\\\",\\\"context\\\":\\\"### How it works\\\\n\\\\nAll three DCs advertise all three prefixes for resiliency, but each prefix belongs to a specific DC. When AWS Cloud WAN learns 10.1.0.0/16 from all three DXGWs, the routing policies assign it local preference 300 from DXGW 65010 (DC1, the owner), 200 from DXGW 65011 (DC2), and 100 from DXGW 65012 (DC3). AWS Cloud WAN selects the path with the highest local preference, steering traffic for 10.1.0.0/16 through DC1. If DC1\\u2019s Direct Connect path fails, traffic automatically falls back to DC2 (local preference 200), then DC3 (local preference 100). The same logic applies for DC2-owned prefixes (primary through DXGW 65011, secondary through 65012, tertiary through 65010) and DC3-owned prefixes (primary through DXGW 65012, secondary through 65011, tertiary through 65010).\\\\n\\\\nAt the DXGW level, an additional layer of path control is available through BGP communities on the VIFs. Each DXGW has two VIFs, one primary and one secondary, and community tags determine which VIF is preferred for outbound traffic: 7224:7300 for high preference and 7224:7100 for low preference. Alternatively, 7224:7200 can be applied to both VIFs for active-active ECMP load balancing.\\\\n\\\\nWith this configuration, AWS Cloud WAN and the DXGWs steer outbound traffic toward on premises using inbound route preferences. To achieve symmetric routing, on-premises devices should be configured to prefer the same paths for traffic flowing from on premises to AWS.\\\\n\\\\n**Note:** Prefixes that do not match any of the three defined prefix lists pass through with no explicit local preference set, resulting in non-deterministic path selection across all three DXGWs. Make sure prefix lists are kept up to date as new prefixes are advertised from on premises\\\",\\\"url\\\":\\\"https://aws.amazon.com/blogs/networking-and-content-delivery/aws-cloud-wan-routing-policy-real-world-global-network-scenarios-part-2/\\\"}]}}\\n\\n## Recent AWS announcements (What's New)\\n\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"aws-networking\\\",\\\"skill_description\\\":\\\"Routes AWS networking requests to the correct service skill for implementation. Covers Route 53 (DNS, health checks, routing policies, Resolver, DNS Firewall), CloudFront (caching, edge, OAC, mTLS, signed URLs), Transit Gateway (multi-VPC hub, segmentation, centralized egress), Direct Connect (hybrid link, DX Gateway, MACsec), Site-to-Site VPN (IPsec tunnels, static or BGP), WAF (web ACLs, AWS Managed Rules, rate-based rules, Bot and Fraud Control), and Shield Advanced (L3/L4 DDoS). Applicable when creating, configuring, troubleshooting, or designing across these services, choosing between them, or diagnosing connectivity or traffic-filtering issues. Not for VPC subnets and route tables, load balancers, VPC endpoints, PrivateLink, API Gateway, IAM policy logic, container or serverless networking, or IaC authoring.\\\\n\\\\nServices: route53, route53resolver, cloudfront, directconnect, transitgateway, vpn, waf, shield\\\\nTasks: configure, troubleshoot, debug, deploy, secure, migrate\\\\nPersona: developer, network-engineer, devops, security-engineer\\\\nWorkload: networking, security\\\\n*\\\",\\\"skill_name\\\":\\\"aws-networking\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"route53\\\",\\\"skill_description\\\":\\\"Configures Amazon Route 53 DNS: public and private records, traffic-steering routing policies, health checks, DNS Firewall, Route 53 Profiles, VPC Resolver (also known as Route 53 Resolver) for hybrid and Outposts networks, and Global Resolver. Applicable when the customer wants to point a hostname at a target, split or fail over traffic across endpoints, monitor an endpoint, block malicious domains, centralize DNS across accounts, or resolve private DNS across a hybrid network. Routes to the right per-task procedure in references. Does not cover CloudFront-specific setup or non-DNS networking.\\\\n\\\\n\\\\nServices: route53, vpc, outposts, cloudwatch, sns\\\\nTasks: configure, secure, monitor, deploy\\\\nPersona: developer, devops, network-admin, security-engineer, sre\\\\nWorkload: networking, security, hybrid, disaster-recovery\\\\n*\\\",\\\"skill_name\\\":\\\"route53\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"transitgateway\\\",\\\"skill_description\\\":\\\"Configures AWS Transit Gateway: creating a hub and attaching VPCs, segmenting traffic with route tables, centralizing egress and forcing east-west traffic between VPCs through a central firewall via transit gateway route tables and appliance-mode attachments, connecting on-premises networks over the transit gateway side of a Site-to-Site VPN or Direct Connect attachment (including ECMP to aggregate bandwidth across multiple VPN tunnels), peering transit gateways across Regions, migrating from a VPC peering mesh, and routing IP multicast. Applicable when the transit gateway is the resource being configured: connecting many VPCs, isolating environments, routing egress or east-west traffic through the hub for inspection, reaching on-premises over the hub, linking Regions, or moving off a peering mesh. Not applicable for single-VPC routing, VPC peering between two VPCs (vpcpeering skill), Direct Connect gateway or virtual interface setup (directconnect skill), or Route 53 DNS work.\\\\n\\\\n\\\\nServices: transitgateway, vpc, directconnect, sitetositevpn, gatewayloadbalancer\\\\nTasks: configure, secure, deploy, migrate\\\\nPersona: developer, devops, network-admin, security-engineer, sre\\\\nWorkload: networking, security, hybrid, multi-region\\\\n*\\\",\\\"skill_name\\\":\\\"transitgateway\\\"}]}}\"}]}], \"label\": \"Look up AWS docs on Cloud WAN route preference and path selection order\"}", + "createdAt": "2026-10-27T17:05:07.925000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "a110b576-edea-40ad-a583-332cfd2f2396", + "executionId": "85f0df89-08a7-4486-a233-eea2ec390f97", + "recordId": "22b6d97a-aecd-4208-bfa7-fcc8fb7b0d54", + "content": "{\"id\": \"22b6d97a-aecd-4208-bfa7-fcc8fb7b0d54\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_Mkko0chj5r9O0wcHQvUZYg\", \"content\": \"[{'text': '## Documentation search results\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"Direct Connect gateway attachments in AWS Cloud WAN\\\",\\\"context\\\":\\\"## Route propagation\\\\\\\\n\\\\\\\\nA Direct Connect gateway attachments support BGP-based dynamic routing for both inbound and\\\\\\\\noutbound directions.\\\\\\\\n\\\\\\\\nFor inbound routes,\\\\\\\\n\\\\\\\\n* Cloud WAN learns BGP routes advertised from your on-premises location\\\\\\\\n via the Direct Connect gateway and the transit virtual interface. Routes are learnt\\\\\\\\n in the segment route-tables of the associated core network edges for the\\\\\\\\n attachment.\\\\\\\\n* Routes learned in segment route table can be routed across all\\\\\\\\n AWS Regions for that segment.\\\\\\\\n* Cloud WAN follows the route evaluation order for the same prefixes learned\\\\\\\\n over multiple attachments. See Route evaluation for\\\\\\\\n more information.\\\\\\\\n* Direct Connect gateway will advertise routes from all on-premises locations towards\\\\\\\\n all segment route-tables of the associated core network edges for the attachment.\\\\\\\\n\\\\\\\\nFor outbound routes,\\\\\\\\n\\\\\\\\n* Cloud WAN propagates routes from the segment route table to the Direct Connect\\\\\\\\n gateway, which in turn advertises these routes over transit virtual interfaces\\\\\\\\n to your on-premises locations via BGP.\\\\\\\\n* Each core network edge associated with the Direct Connect gateway attachment\\\\\\\\n advertises only its local routes towards the Direct Connect gateway.\\\\\\\\n* The AS_PATH BGP attribute is retained in these route advertisements to your\\\\\\\\n on-premises locations. For more information about AS_PATH and BGP, see Private virtual interface and transit virtual interface routing\\\\\\\\n policies in the *AWS Direct Connect User\\\\\\\\n Guide*\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/network-manager/latest/cloudwan/cloudwan-dxattach-about.html\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"Simplify global hybrid connectivity with AWS Cloud WAN and AWS Direct Connect integration | Networking & Content Delivery\\\",\\\"context\\\":\\\"## Things to know\\\\\\\\n\\\\\\\\n* You can associate a Direct Connect gateway only with a single AWS Cloud WAN segment. You can have multiple Direct Connect gateways associated to the same segment. You can also have different Direct Connect gateways associated to multiple Cloud WAN segments. However, you cannot have the same Direct Connect gateway associated to multiple segments.\\\\\\\\n* When you associate a Direct Connect gateway with an AWS Cloud WAN segment using a Direct Connect attachment, you can choose all or a subset of the Cloud WAN CNEs. We recommend associating the attachment with all CNEs where the respective segment is configured, unless you have bespoke routing requirements, as shown in Scenario 3. If you choose to associate the Direct Connect gateway attachment with only a subset of the CNEs, the Direct Connect gateway receives only the routes from local attachments of the associated CNEs.\\\\\\\\n* Direct Connect BGP community tags are only relevant to the Direct Connect gateway, and do not influence the Cloud WAN core network routing decisions.\\\\\\\\n* Cloud WAN route evaluation algorithm is detailed here. Remember that MED is a non-transitive BGP attribute, therefore the Direct Connect gateway resets MED on all routes advertised to Cloud WAN.\\\\\\\\n* The new Direct Connect attachment on AWS Cloud WAN allows for routes to be propagated dynamically in both directions, without the need of using allowed prefixes list. The use of allowed prefixes list is still required for the Transit Gateway association to a Direct Connect gateway. If you attach a Direct Connect gateway to an isolated segment, VPC routes are not propagated to your on-premises routers, and on-premises routes are not propagated to the core network segment.\\\\\\\\n* You can attach Direct Connect gateways from multiple accounts to your AWS Cloud WAN core network following the steps described in the documentation.\\\\\\\\n* The quotas for the number of routes you can advertise from your core network segment to a Direct Connect gateway attachment, and the number of routes you can advertise from your on-premises environments on a Direct Connect virtual interface are detailed in the AWS Cloud WAN documentation and AWS Direct Connect documentation\\\",\\\"url\\\":\\\"https://aws.amazon.com/blogs/networking-and-content-delivery/simplify-global-hybrid-connectivity-with-aws-cloud-wan-and-aws-direct-connect-integration/\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"AWS Cloud WAN Routing Policy: Real-World Global Network Scenarios \\u2013 Part 2 | Networking & Content Delivery\\\",\\\"context\\\":\\\"### How it works\\\\\\\\n\\\\\\\\nAll three DCs advertise all three prefixes for resiliency, but each prefix belongs to a specific DC. When AWS Cloud WAN learns 10.1.0.0/16 from all three DXGWs, the routing policies assign it local preference 300 from DXGW 65010 (DC1, the owner), 200 from DXGW 65011 (DC2), and 100 from DXGW 65012 (DC3). AWS Cloud WAN selects the path with the highest local preference, steering traffic for 10.1.0.0/16 through DC1. If DC1\\u2019s Direct Connect path fails, traffic automatically falls back to DC2 (local preference 200), then DC3 (local preference 100). The same logic applies for DC2-owned prefixes (primary through DXGW 65011, secondary through 65012, tertiary through 65010) and DC3-owned prefixes (primary through DXGW 65012, secondary through 65011, tertiary through 65010).\\\\\\\\n\\\\\\\\nAt the DXGW level, an additional layer of path control is available through BGP communities on the VIFs. Each DXGW has two VIFs, one primary and one secondary, and community tags determine which VIF is preferred for outbound traffic: 7224:7300 for high preference and 7224:7100 for low preference. Alternatively, 7224:7200 can be applied to both VIFs for active-active ECMP load balancing.\\\\\\\\n\\\\\\\\nWith this configuration, AWS Cloud WAN and the DXGWs steer outbound traffic toward on premises using inbound route preferences. To achieve symmetric routing, on-premises devices should be configured to prefer the same paths for traffic flowing from on premises to AWS.\\\\\\\\n\\\\\\\\n**Note:** Prefixes that do not match any of the three defined prefix lists pass through with no explicit local preference set, resulting in non-deterministic path selection across all three DXGWs. Make sure prefix lists are kept up to date as new prefixes are advertised from on premises\\\",\\\"url\\\":\\\"https://aws.amazon.com/blogs/networking-and-content-delivery/aws-cloud-wan-routing-policy-real-world-global-network-scenarios-part-2/\\\"}]}}\\\\n\\\\n## Recent AWS announcements (What\\\\'s New)\\\\n\\\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"AWS Direct Connect introduces inbound prefix controls and higher prefix scale - AWS\\\",\\\"context\\\":\\\"AWS Direct Connect has introduced new capabilities in the form of inbound prefix controls and higher prefix scale. This update allows users to allocate and manage inbound route-prefix allocations for private and transit virtual interfaces (VIFs) based on their workload needs. The maximum number of prefixes that can be accepted on a VIF has increased from 100 to 1,000 for both IPv4 and IPv6. This change eliminates the need for architectural workarounds like summarizing routes or segmenting across multiple VIFs or connections.\\\\\\\\n\\\\\\\\nInbound prefix controls have introduced new prefix capacity pools at the dedicated connection level and at the Direct Connect gateway (DXGW) level. When creating or updating a VIF, users allocate a specific number of prefixes to it, which draws from the dedicated connection\\\\'s pool and the DXGW\\\\'s pool when attached. This allows users to right-size prefix capacity per workload. Connection pool sizes scale with connection speed, and link aggregation group (LAG) pools scale with the number of member connections.\\\\\\\\n\\\\\\\\nUsers can configure prefix allocations using the AWS Direct Connect console or CLI/API. Inbound prefix controls are available at no additional cost in all commercial AWS Regions where AWS Direct Connect is available, AWS GovCloud Regions (US-East and US-West), as well as the Amazon Web Services China (Beijing) Region, operated by Sinnet, and the Amazon Web Services China (Ningxia) Region, operated by NWCD\\\",\\\"url\\\":\\\"https://aws.amazon.com/about-aws/whats-new/2026/08/aws-direct-connect-new-prefix-controls/\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: directconnect, vpc, transitgateway\\\\\\\\nTasks: configure, secure, deploy\\\\\\\\nPersona: network-admin, devops, developer, sre\\\\\\\\nWorkload: networking, hybrid-cloud\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"transitgateway\\\",\\\"skill_description\\\":\\\"Configures AWS Transit Gateway: creating a hub and attaching VPCs, segmenting traffic with route tables, centralizing egress and forcing east-west traffic between VPCs through a central firewall via transit gateway route tables and appliance-mode attachments, connecting on-premises networks over the transit gateway side of a Site-to-Site VPN or Direct Connect attachment (including ECMP to aggregate bandwidth across multiple VPN tunnels), peering transit gateways across Regions, migrating from a VPC peering mesh, and routing IP multicast. Applicable when the transit gateway is the resource being configured: connecting many VPCs, isolating environments, routing egress or east-west traffic through the hub for inspection, reaching on-premises over the hub, linking Regions, or moving off a peering mesh. Not applicable for single-VPC routing, VPC peering between two VPCs (vpcpeering skill), Direct Connect gateway or virtual interface setup (directconnect skill), or Route 53 DNS work.\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: transitgateway, vpc, directconnect, sitetositevpn, gatewayloadbalancer\\\\\\\\nTasks: configure, secure, deploy, migrate\\\\\\\\nPersona: developer, devops, network-admin, security-engineer, sre\\\\\\\\nWorkload: networking, security, hybrid, multi-region\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"transitgateway\\\"},{\\\"rank_order\\\":4,\\\"title\\\":\\\"aws-networking\\\",\\\"skill_description\\\":\\\"Routes AWS networking requests to the correct service skill for implementation. Covers Route 53 (DNS, health checks, routing policies, Resolver, DNS Firewall), CloudFront (caching, edge, OAC, mTLS, signed URLs), Transit Gateway (multi-VPC hub, segmentation, centralized egress), Direct Connect (hybrid link, DX Gateway, MACsec), Site-to-Site VPN (IPsec tunnels, static or BGP), WAF (web ACLs, AWS Managed Rules, rate-based rules, Bot and Fraud Control), and Shield Advanced (L3/L4 DDoS). Applicable when creating, configuring, troubleshooting, or designing across these services, choosing between them, or diagnosing connectivity or traffic-filtering issues. Not for VPC subnets and route tables, load balancers, VPC endpoints, PrivateLink, API Gateway, IAM policy logic, container or serverless networking, or IaC authoring.\\\\\\\\n\\\\\\\\nServices: route53, route53resolver, cloudfront, directconnect, transitgateway, vpn, waf, shield\\\\\\\\nTasks: configure, troubleshoot, debug, deploy, secure, migrate\\\\\\\\nPersona: developer, network-engineer, devops, security-engineer\\\\\\\\nWorkload: networking, security\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"aws-networking\\\"}]}}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}, {\"id\": \"tooluse_qYdyckFlnikQ01OlOgrKf2\", \"content\": \"[{'text': '## Documentation search results\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"Route evaluation\\\",\\\"context\\\":\\\"# Route evaluation\\\\\\\\n\\\\\\\\nCloud WAN evaluates routes at each core network edge in the following order:\\\\\\\\n\\\\\\\\n1. The most specific route for the destination\\\\\\\\n2. For routes with the same destination IP address, but different targets, the\\\\\\\\n following route priority is used:\\\\\\\\n\\\\\\\\n 1. Static routes\\\\\\\\n 2. System routes. For more information, see Routing behavior for 1:1 NAT in AWS Cloud WAN.\\\\\\\\n 3. VPC-propagated routes in the same Region.\\\\\\\\n 4. For dynamic routes received at the core network with an\\\\\\\\n *unequal* AS path length and/or MED BGP\\\\\\\\n attributes, Cloud WAN evaluates them in the following order:\\\\\\\\n\\\\\\\\n 1. AS path length\\\\\\\\n 2. MED\\\\\\\\n 5. For dynamic routes received at the core network with\\\\\\\\n *equal* AS path length and MED BGP attributes,\\\\\\\\n Cloud WAN evaluates them in the following order:\\\\\\\\n\\\\\\\\n 1. Direct Connect gateway-propagated routes.\\\\\\\\n 2. Cloud WAN Connect-propagated routes in the same\\\\\\\\n Region.\\\\\\\\n 3. Site-to-Site VPN-propagated routes in the same\\\\\\\\n Region.\\\\\\\\n 4. Routes propagated from other sources, such as transit gateway peering\\\\\\\\n and core network edges in other remote Regions\\\\\\\\n over the AWS global infrastructure. If identical routes are\\\\\\\\n received from two or more sources, a single attachment will be\\\\\\\\n chosen in a deterministically random manner (this occurs on a per\\\\\\\\n segment/network function group basis so you could see different destinations\\\\\\\\n for the same route across different segment/network function groups)\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/network-manager/latest/cloudwan/cloudwan-route-evaluation.html\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"AWS Cloud WAN Routing policies\\\",\\\"context\\\":\\\"## Key considerations\\\\\\\\n\\\\\\\\nThe following is a list of considerations that should be taken into account before using Cloud WAN Routing Policies:\\\\\\\\n\\\\\\\\n### Route evaluation and ordering\\\\\\\\n\\\\\\\\n* Route summarization will remove all matched prefixes and replace them with a single summarized route. The summarized prefix will be advertised at the same time as matched prefixes are withdrawn.\\\\\\\\n* The allow and drop actions are terminal. When a route matches either one, no further rules are evaluated for that route. Other actions, such as set-med and summarize, are non-terminal, so processing continues. A route that reaches the end of all rules without matching a terminal action is allowed, with any modifications applied.\\\\\\\\n* Segment share policies are applied after attachment policies.\\\\\\\\n* Routing policies associated across segments and Regions are unidirectional.\\\\\\\\n* Routing policies are not supported for network function groups (service insertion).\\\\\\\\n\\\\\\\\n### Attachment type support\\\\\\\\n\\\\\\\\n* VPC attachments don\\\\'t support BGP attribute modification.\\\\\\\\n* Summarization only works outbound and on BGP-capable attachments.\\\\\\\\n* No BGP community support on Direct Connect and TGW peering attachments.\\\\\\\\n* TGW route table attachments that use the same peering and are associated with the same segment share the same outbound routing policies across all similar attachments. For example, if attachment-1 has outbound routing policy 1 on segment prod and peering 1, and attachment-2 has outbound routing policy 2 on the same segment and peering, both policies apply to both attachments.\\\\\\\\n\\\\\\\\n### BGP and ASN constraints\\\\\\\\n\\\\\\\\n* ASNs specified in the routing policy (replace/remove ASN, community tags) cannot overlap with the ASN range specified in the core network configuration. This also means you cannot advertise communities into a core network that contain an ASN currently in use by the core network.\\\\\\\\n* Replace ASN is not supported cross-Region (CNE-to-CNE).\\\\\\\\n* External AWS devices cannot advertise routes with BGP communities containing internal ASNs.\\\\\\\\n* Route summarization does not preserve the BGP attributes of the matched prefixes. The summarized route is advertised as a new route with reset attribute values: MED of 100, local-preference of 0, an empty AS-path, and empty communities.\\\\\\\\n\\\\\\\\n### Prefix lists and visibility\\\\\\\\n\\\\\\\\n* Prefix list aliases must be unique per prefix list core network association.\\\\\\\\n* Prefix list modifications of entry values to the underlying core network routing state may not align with the prefix list state.\\\\\\\\n* The `list-core-network-routing-information` API shows the routing information before routing policies have been applied\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/network-manager/latest/cloudwan/cloudwan-routing-policies.html\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"AWS Cloud WAN Routing Policy: Real-World Global Network Scenarios \\u2013 Part 2 | Networking & Content Delivery\\\",\\\"context\\\":\\\"### How it works\\\\\\\\n\\\\\\\\nAll three DCs advertise all three prefixes for resiliency, but each prefix belongs to a specific DC. When AWS Cloud WAN learns 10.1.0.0/16 from all three DXGWs, the routing policies assign it local preference 300 from DXGW 65010 (DC1, the owner), 200 from DXGW 65011 (DC2), and 100 from DXGW 65012 (DC3). AWS Cloud WAN selects the path with the highest local preference, steering traffic for 10.1.0.0/16 through DC1. If DC1\\u2019s Direct Connect path fails, traffic automatically falls back to DC2 (local preference 200), then DC3 (local preference 100). The same logic applies for DC2-owned prefixes (primary through DXGW 65011, secondary through 65012, tertiary through 65010) and DC3-owned prefixes (primary through DXGW 65012, secondary through 65011, tertiary through 65010).\\\\\\\\n\\\\\\\\nAt the DXGW level, an additional layer of path control is available through BGP communities on the VIFs. Each DXGW has two VIFs, one primary and one secondary, and community tags determine which VIF is preferred for outbound traffic: 7224:7300 for high preference and 7224:7100 for low preference. Alternatively, 7224:7200 can be applied to both VIFs for active-active ECMP load balancing.\\\\\\\\n\\\\\\\\nWith this configuration, AWS Cloud WAN and the DXGWs steer outbound traffic toward on premises using inbound route preferences. To achieve symmetric routing, on-premises devices should be configured to prefer the same paths for traffic flowing from on premises to AWS.\\\\\\\\n\\\\\\\\n**Note:** Prefixes that do not match any of the three defined prefix lists pass through with no explicit local preference set, resulting in non-deterministic path selection across all three DXGWs. Make sure prefix lists are kept up to date as new prefixes are advertised from on premises\\\",\\\"url\\\":\\\"https://aws.amazon.com/blogs/networking-and-content-delivery/aws-cloud-wan-routing-policy-real-world-global-network-scenarios-part-2/\\\"}]}}\\\\n\\\\n## Recent AWS announcements (What\\\\'s New)\\\\n\\\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"aws-networking\\\",\\\"skill_description\\\":\\\"Routes AWS networking requests to the correct service skill for implementation. Covers Route 53 (DNS, health checks, routing policies, Resolver, DNS Firewall), CloudFront (caching, edge, OAC, mTLS, signed URLs), Transit Gateway (multi-VPC hub, segmentation, centralized egress), Direct Connect (hybrid link, DX Gateway, MACsec), Site-to-Site VPN (IPsec tunnels, static or BGP), WAF (web ACLs, AWS Managed Rules, rate-based rules, Bot and Fraud Control), and Shield Advanced (L3/L4 DDoS). Applicable when creating, configuring, troubleshooting, or designing across these services, choosing between them, or diagnosing connectivity or traffic-filtering issues. Not for VPC subnets and route tables, load balancers, VPC endpoints, PrivateLink, API Gateway, IAM policy logic, container or serverless networking, or IaC authoring.\\\\\\\\n\\\\\\\\nServices: route53, route53resolver, cloudfront, directconnect, transitgateway, vpn, waf, shield\\\\\\\\nTasks: configure, troubleshoot, debug, deploy, secure, migrate\\\\\\\\nPersona: developer, network-engineer, devops, security-engineer\\\\\\\\nWorkload: networking, security\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"aws-networking\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"route53\\\",\\\"skill_description\\\":\\\"Configures Amazon Route 53 DNS: public and private records, traffic-steering routing policies, health checks, DNS Firewall, Route 53 Profiles, VPC Resolver (also known as Route 53 Resolver) for hybrid and Outposts networks, and Global Resolver. Applicable when the customer wants to point a hostname at a target, split or fail over traffic across endpoints, monitor an endpoint, block malicious domains, centralize DNS across accounts, or resolve private DNS across a hybrid network. Routes to the right per-task procedure in references. Does not cover CloudFront-specific setup or non-DNS networking.\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: route53, vpc, outposts, cloudwatch, sns\\\\\\\\nTasks: configure, secure, monitor, deploy\\\\\\\\nPersona: developer, devops, network-admin, security-engineer, sre\\\\\\\\nWorkload: networking, security, hybrid, disaster-recovery\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"route53\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"transitgateway\\\",\\\"skill_description\\\":\\\"Configures AWS Transit Gateway: creating a hub and attaching VPCs, segmenting traffic with route tables, centralizing egress and forcing east-west traffic between VPCs through a central firewall via transit gateway route tables and appliance-mode attachments, connecting on-premises networks over the transit gateway side of a Site-to-Site VPN or Direct Connect attachment (including ECMP to aggregate bandwidth across multiple VPN tunnels), peering transit gateways across Regions, migrating from a VPC peering mesh, and routing IP multicast. Applicable when the transit gateway is the resource being configured: connecting many VPCs, isolating environments, routing egress or east-west traffic through the hub for inspection, reaching on-premises over the hub, linking Regions, or moving off a peering mesh. Not applicable for single-VPC routing, VPC peering between two VPCs (vpcpeering skill), Direct Connect gateway or virtual interface setup (directconnect skill), or Route 53 DNS work.\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: transitgateway, vpc, directconnect, sitetositevpn, gatewayloadbalancer\\\\\\\\nTasks: configure, secure, deploy, migrate\\\\\\\\nPersona: developer, devops, network-admin, security-engineer, sre\\\\\\\\nWorkload: networking, security, hybrid, multi-region\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"transitgateway\\\"}]}}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:22.010000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "a110b576-edea-40ad-a583-332cfd2f2396", + "executionId": "85f0df89-08a7-4486-a233-eea2ec390f97", + "recordId": "fe38a838-2fdc-461f-bcbd-3a2ec7986e09", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.7, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:22.111000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "a110b576-edea-40ad-a583-332cfd2f2396", + "executionId": "85f0df89-08a7-4486-a233-eea2ec390f97", + "recordId": "3bc9535b-c823-4506-b9c7-0193e3d6439f", + "content": "{\"id\": \"3bc9535b-c823-4506-b9c7-0193e3d6439f\", \"role\": \"assistant\", \"content\": [{\"text\": \"## Why this happens\\n\\nCloud WAN evaluates routes at each core network edge in a strict order, and it's step 1 that's almost always the culprit here:\\n\\n1. **Most specific prefix wins, period.** If your aggregate (e.g. a /16) is advertised over Direct Connect in the local region, but a more specific prefix (e.g. a /24 subset of it) is being learned from somewhere else \\u2014 another region's attachment, a VPN, a TGW peering \\u2014 longest prefix match sends that traffic there instead, regardless of path type or BGP attributes. This is the single most common reason \\\"aggregate traffic leaves the region\\\" \\u2014 a more specific route is winning before Direct Connect is even considered.\\n2. Only when destinations tie on specificity does Cloud WAN break the tie by path priority:\\n - Static routes\\n - System routes (1:1 NAT)\\n - VPC-propagated routes in the same region\\n - Unequal AS-path length / MED (shorter AS-path, then lower MED wins)\\n - If AS-path and MED are *equal*, then it finally prefers: **Direct Connect gateway-propagated routes** \\u2192 Connect-propagated \\u2192 VPN-propagated (same region) \\u2192 other sources (TGW peering, remote-region CNEs)\\n - If still tied, selection is deterministically random per segment\\n\\nSo for an aggregate prefix to \\\"lose\\\" to a remote-region path despite DX being present locally, one of these is usually true:\\n- A more specific route for part of that aggregate is coming in from the other region (step 1 overrides everything else).\\n- The AS-path or MED on the DX-advertised route is longer/higher than the competing route, so it loses at the BGP-attribute tiebreak before DX preference is even applied.\\n- The Direct Connect gateway attachment isn't associated with the core network edge (CNE) in that region, so no local DX-sourced route exists at all to compete.\\n\\n## How to make the local DX win\\n\\n- **Tighten what on-prem advertises.** If the real goal is per-region traffic steering, advertise more specific prefixes per DC/region rather than one flat aggregate everywhere \\u2014 this is the AWS-recommended pattern (per-DC prefix ownership) and avoids relying on tiebreaks entirely.\\n- **If you must keep the aggregate**, use Cloud WAN routing policies to set **local-preference** per attachment so the local DX path is explicitly preferred over the same prefix learned elsewhere \\u2014 local-preference is evaluated ahead of the DX-priority tiebreak implicitly via AS-path/MED, so set-med or AS-path manipulation on the policy is the lever, since local-preference itself isn't user-settable directly \\u2014 practically this means using the routing policy's MED/AS-path-length actions to make the local-region advertisement look \\\"better\\\" to the evaluation algorithm.\\n- **Confirm the DX gateway attachment is associated with the CNE in that region.** If it's only associated with a subset of edges, the local edge may have no local DX route to offer at all.\\n- **Check for a more-specific competing route first** \\u2014 that's the highest-leverage fix since it overrides every other rule.\\n\\nIf you'd like, I can look at your actual core network's route tables and the Direct Connect gateway attachment's associated edges to pinpoint which of these is happening in your account.\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:22.186000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "a110b576-edea-40ad-a583-332cfd2f2396", + "executionId": "85f0df89-08a7-4486-a233-eea2ec390f97", + "recordId": "9a923182-c1bd-4163-90b2-eaa0ffe49205", + "content": "{\"id\": \"52850d42-c41e-4f68-b4e3-11cce7c44c43\", \"role\": \"assistant\", \"content\": [{\"type\": \"text\", \"text\": \"## Why this happens\\n\\nCloud WAN evaluates routes at each core network edge in a strict order, and it's step 1 that's almost always the culprit here:\\n\\n1. **Most specific prefix wins, period.** If your aggregate (e.g. a /16) is advertised over Direct Connect in the local region, but a more specific prefix (e.g. a /24 subset of it) is being learned from somewhere else \\u2014 another region's attachment, a VPN, a TGW peering \\u2014 longest prefix match sends that traffic there instead, regardless of path type or BGP attributes. This is the single most common reason \\\"aggregate traffic leaves the region\\\" \\u2014 a more specific route is winning before Direct Connect is even considered.\\n2. Only when destinations tie on specificity does Cloud WAN break the tie by path priority:\\n - Static routes\\n - System routes (1:1 NAT)\\n - VPC-propagated routes in the same region\\n - Unequal AS-path length / MED (shorter AS-path, then lower MED wins)\\n - If AS-path and MED are *equal*, then it finally prefers: **Direct Connect gateway-propagated routes** \\u2192 Connect-propagated \\u2192 VPN-propagated (same region) \\u2192 other sources (TGW peering, remote-region CNEs)\\n - If still tied, selection is deterministically random per segment\\n\\nSo for an aggregate prefix to \\\"lose\\\" to a remote-region path despite DX being present locally, one of these is usually true:\\n- A more specific route for part of that aggregate is coming in from the other region (step 1 overrides everything else).\\n- The AS-path or MED on the DX-advertised route is longer/higher than the competing route, so it loses at the BGP-attribute tiebreak before DX preference is even applied.\\n- The Direct Connect gateway attachment isn't associated with the core network edge (CNE) in that region, so no local DX-sourced route exists at all to compete.\\n\\n## How to make the local DX win\\n\\n- **Tighten what on-prem advertises.** If the real goal is per-region traffic steering, advertise more specific prefixes per DC/region rather than one flat aggregate everywhere \\u2014 this is the AWS-recommended pattern (per-DC prefix ownership) and avoids relying on tiebreaks entirely.\\n- **If you must keep the aggregate**, use Cloud WAN routing policies to set **local-preference** per attachment so the local DX path is explicitly preferred over the same prefix learned elsewhere \\u2014 local-preference is evaluated ahead of the DX-priority tiebreak implicitly via AS-path/MED, so set-med or AS-path manipulation on the policy is the lever, since local-preference itself isn't user-settable directly \\u2014 practically this means using the routing policy's MED/AS-path-length actions to make the local-region advertisement look \\\"better\\\" to the evaluation algorithm.\\n- **Confirm the DX gateway attachment is associated with the CNE in that region.** If it's only associated with a subset of edges, the local edge may have no local DX route to offer at all.\\n- **Check for a more-specific competing route first** \\u2014 that's the highest-leverage fix since it overrides every other rule.\\n\\nIf you'd like, I can look at your actual core network's route tables and the Direct Connect gateway attachment's associated edges to pinpoint which of these is happening in your account.\"}]}", + "createdAt": "2026-10-27T17:05:22.259000-05:00", + "recordType": "final_response" + } +] \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-overlapping-supernet/with_skill/functional-tests-results.json b/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-overlapping-supernet/with_skill/functional-tests-results.json new file mode 100644 index 00000000..4613bc0f --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-overlapping-supernet/with_skill/functional-tests-results.json @@ -0,0 +1,74 @@ +{ + "version": "v1", + "iteration": 1, + "eval_id": "cloudwan-overlapping-supernet", + "run_type": "with_skill", + "success": true, + "error": null, + "understanding_agent_space_skill": null, + "tests": { + "trigger": { + "result": "passed", + "message": "Skill 'aws-routing' was triggered as expected", + "skill_loads_found": 1 + }, + "expected_output": { + "result": "failed", + "reasoning": "The expected output has several specific technical claims:\n1. LPM means VPC /16s win and on-prem hosts inside a VPC CIDR are shadowed/unreachable\n2. Best fix is non-overlapping address planning\n3. Cloud WAN Routing Policies can drop the inbound supernet AND suppress VPC prefixes outbound on the DX attachment\n4. BGP communities cannot be matched on DX attachments\n\nThe agent's response covers points 1 and 2 reasonably well: it explains LPM correctly (VPC /16s win over the /8 when propagated), discusses the shadowing concept in a more nuanced way (noting that if on-prem has real hosts in that range it's a duplicate-IP conflict, not fully framing it as \"shadowed/unreachable\" for on-prem hosts specifically). The expected output frames the issue as on-prem hosts inside the VPC CIDR becoming unreachable due to LPM favoring VPC routes - the agent does touch this with point 4 and 5 but doesn't explicitly state \"on-prem hosts inside a VPC CIDR become unreachable/shadowed\" as directly as the expected answer. It discusses it as \"duplicate addressing\" which is a related but slightly different framing - though it does capture the same underlying issue.\n\nPoint 2 (best fix is non-overlapping address planning) - the agent does state \"Best fix \u2014 stop over-advertising at the source... advertise only the ranges it legitimately owns\" which aligns with non-overlapping address planning, though phrased as filtering at source rather than address planning per se. Still reasonably aligned.\n\nPoint 3 - the agent only mentions inbound filtering (\"add an inbound policy matching prefix-in-cidr... action drop\") but does NOT mention outbound suppression of VPC prefixes on the DX attachment. This is a specific capability mentioned in the expected answer that's missing from the agent's response.\n\nPoint 4 - BGP communities cannot be matched on DX attachments - this specific point is entirely absent from the agent's response. The agent doesn't mention BGP communities limitations on DX attachments at all.\n\nSo the agent's response is strong on general issue explanation and partially addresses the routing policy fix (inbound only, missing outbound), and completely misses the specific technical detail about BGP communities not being matchable on DX attachments. This is a notable gap matching the expected output's specific technical criteria.\n\nGiven the agent captures most of the conceptual content (LPM favoring VPC CIDRs, routing policy as a fix, address planning as best practice) but misses two specific technical details (outbound suppression capability and BGP community limitation on DX), this is a partial match. Given the specificity of the expected output mentioning these technical nuances, and the agent's omission of the BGP community point entirely, I lean towards this being a partial failure on technical completeness, though the core conceptual understanding is solid.", + "evidence": "Agent states: \"add an inbound policy matching prefix-in-cidr for 10.1.0.0/16 and 10.2.0.0/16 with action drop\" (only inbound, not outbound suppression) and does not mention BGP communities limitation on DX attachments anywhere in the response.", + "confidence": "medium" + }, + "assertions": { + "assertion_results": [ + { + "text": "explains longest-prefix-match shadows on-prem hosts inside a VPC CIDR", + "evaluator": "llm", + "passed": true, + "evidence": "\"Step 1 of CNE evaluation is longest-prefix-match (LPM), which is absolute \u2014 so traffic to an address literally inside 10.1.0.0/16 or 10.2.0.0/16 will correctly prefer the VPC-propagated /16 over the DX-sourced /8\" and later: \"If on-prem actually has live hosts inside 10.1.0.0/16 or 10.2.0.0/16 (not just an oversized aggregate), that's a real IP conflict\" and \"In any fallback case where the /8 wins, return traffic gets sent to on-prem for addresses on-prem doesn't actually own, producing drops or NAKs\" - this explains that LPM normally protects VPC CIDRs but if propagation fails, on-prem hosts get shadowed/traffic is misrouted, addressing the shadowing scenario.", + "reasoning": "The output explicitly discusses LPM behavior and how on-prem hosts inside the VPC CIDR ranges would create conflicts or shadowing issues when propagation fails or when duplicate addressing exists, satisfying the assertion's substance.", + "confidence": "high" + }, + { + "text": "recommends non-overlapping address planning as the best fix", + "evaluator": "llm", + "passed": true, + "evidence": "\"Best fix \u2014 stop over-advertising at the source. Reconfigure the on-prem router to not originate 10.0.0.0/8 as a flat aggregate; advertise only the ranges it legitimately owns, explicitly excluding 10.1.0.0/16 and 10.2.0.0/16. This removes both the catch-all exposure and any genuine conflict, and doesn't depend on AWS-side filtering.\"", + "reasoning": "The output explicitly labels reconfiguring on-prem advertisement to avoid overlap as the 'Best fix', which is a form of non-overlapping address planning at the source.", + "confidence": "high" + }, + { + "text": "mentions Cloud WAN Routing Policies can filter (drop) prefixes on the Direct Connect attachment", + "evaluator": "llm", + "passed": true, + "evidence": "\"Cloud WAN Routing Policies on the DX attachment. Cloud WAN supports prefix-based route filtering on DX attachments: add an inbound policy matching prefix-in-cidr for 10.1.0.0/16 and 10.2.0.0/16 with action drop.\"", + "reasoning": "The output explicitly mentions Cloud WAN Routing Policies with a drop action filtering prefixes on the DX attachment, directly satisfying this assertion.", + "confidence": "high" + }, + { + "text": "does not contain 'error'", + "evaluator": "llm", + "passed": true, + "evidence": "No occurrence of the word 'error' found anywhere in the output text.", + "reasoning": "A scan of the output shows no instance of the word 'error' (case-insensitive) appearing in the text.", + "confidence": "high" + } + ], + "summary": { + "passed": 4, + "failed": 0, + "errored": 0, + "low_confidence": 0, + "total": 4, + "pass_rate": 1.0 + } + } + }, + "metrics": { + "runtime": "1m17s", + "cost": "$0.64", + "context_window": { + "utilization": "5.1%", + "compaction_count": 0 + } + } +} \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-overlapping-supernet/with_skill/outputs/journal_records.json b/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-overlapping-supernet/with_skill/outputs/journal_records.json new file mode 100644 index 00000000..aee4aaad --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-overlapping-supernet/with_skill/outputs/journal_records.json @@ -0,0 +1,154 @@ +[ + { + "agentSpaceId": "865d0604-96c8-40d1-9fb5-3bb62df7a5c5", + "executionId": "688b6a99-26fd-46cd-b681-630b9d5e9105", + "recordId": "8d962600-53e1-4abc-af5f-560e7b3e64f0", + "content": "{\"id\": \"8d962600-53e1-4abc-af5f-560e7b3e64f0\", \"role\": \"user\", \"content\": [{\"text\": \"A customer advertises a supernet (10.0.0.0/8) from Direct Connect into Cloud WAN that overlaps VPC CIDRs (10.1.0.0/16, 10.2.0.0/16). What issues would this cause and how do they fix it?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:08.964000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "865d0604-96c8-40d1-9fb5-3bb62df7a5c5", + "executionId": "688b6a99-26fd-46cd-b681-630b9d5e9105", + "recordId": "a525c130-a097-4659-af32-bc1aa27071c4", + "content": "{\"id\": \"980035f5-5a58-48e3-a4d1-8636bf5d2c73\", \"role\": \"assistant\", \"content\": [{\"type\": \"chat_title\", \"title\": \"AWS networking supernet overlap with VPC CIDR conflicts\"}]}", + "createdAt": "2026-10-27T17:05:09.598000-05:00", + "recordType": "chat_title" + }, + { + "agentSpaceId": "865d0604-96c8-40d1-9fb5-3bb62df7a5c5", + "executionId": "688b6a99-26fd-46cd-b681-630b9d5e9105", + "recordId": "312772ad-2b5d-4251-9730-9e0517e73e98", + "content": "{\"id\": \"312772ad-2b5d-4251-9730-9e0517e73e98\", \"role\": \"assistant\", \"content\": [{\"id\": \"tooluse_MsTgKWtlJi7SmLdm4h3HYW\", \"tool_name\": \"skill_read\", \"input\": {\"skill_id\": \"aws-routing\"}, \"type\": \"tool_use\", \"display_intent\": \"Reading AWS routing skill for Cloud WAN / Direct Connect CIDR overlap guidance\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:16.363000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "865d0604-96c8-40d1-9fb5-3bb62df7a5c5", + "executionId": "688b6a99-26fd-46cd-b681-630b9d5e9105", + "recordId": "d2bc1bf9-17f6-4e1b-8752-1527de119189", + "content": "{\"id\": \"34b2895f-bc22-49cd-8eef-941236f27b3d\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This question matches the aws-routing skill closely, so I should check that skill for grounded guidance before answering.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:16.363000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "865d0604-96c8-40d1-9fb5-3bb62df7a5c5", + "executionId": "688b6a99-26fd-46cd-b681-630b9d5e9105", + "recordId": "a1910d3e-d5c5-446c-ba5f-3e22b36d2621", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 3.8, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:16.463000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "865d0604-96c8-40d1-9fb5-3bb62df7a5c5", + "executionId": "688b6a99-26fd-46cd-b681-630b9d5e9105", + "recordId": "25549d22-2fac-4d56-a45e-93a08e9dfb0a", + "content": "{\"id\": \"6eabaf8b-7a20-4e02-8999-b5e043f389c4\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"29\"}]}", + "createdAt": "2026-10-27T17:05:16.467000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "865d0604-96c8-40d1-9fb5-3bb62df7a5c5", + "executionId": "688b6a99-26fd-46cd-b681-630b9d5e9105", + "recordId": "d87f923e-66c1-4631-beb9-8fd9d1341c20", + "content": "{\"id\": \"5172292b-fee3-4147-9207-47373ebf2d5b\", \"role\": \"assistant\", \"block_type\": \"load_skill\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_MsTgKWtlJi7SmLdm4h3HYW\", \"status\": \"success\", \"content\": [{\"text\": \"Done\"}]}], \"label\": \"Reading AWS routing skill for Cloud WAN / Direct Connect CIDR overlap guidance\"}", + "createdAt": "2026-10-27T17:05:16.551000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "865d0604-96c8-40d1-9fb5-3bb62df7a5c5", + "executionId": "688b6a99-26fd-46cd-b681-630b9d5e9105", + "recordId": "168bb71b-6792-4265-bc63-b31fa5ca036c", + "content": "{\"id\": \"4807148a-9f75-4c9c-b482-a187b77f860a\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This is about Cloud WAN CNE route evaluation with overlapping CIDRs, so I should check the cloudwan-dx-routing-patterns.md reference for details on longest prefix match and overlap scenarios.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:20.364000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "865d0604-96c8-40d1-9fb5-3bb62df7a5c5", + "executionId": "688b6a99-26fd-46cd-b681-630b9d5e9105", + "recordId": "9bf71dda-99f3-4429-a9ba-f0fc3b41cc7f", + "content": "{\"id\": \"9bf71dda-99f3-4429-a9ba-f0fc3b41cc7f\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_MsTgKWtlJi7SmLdm4h3HYW\", \"content\": \"[{'text': '{\\\"content\\\": \\\"---\\\\\\\\r\\\\\\\\nname: aws-routing\\\\\\\\r\\\\\\\\ndescription: \\\\'Use this skill when the user wants to know which path AWS will pick for their traffic, why traffic took an unexpected route, or how to make AWS prefer a path. Covers Cloud WAN CNE route evaluation, Direct Connect Gateway local vs remote region preference, Transit Gateway route tables, ECMP and peering, VPC longest-prefix-match, BGP traffic engineering (LP communities 7224:7100/7200/7300, AS-path prepending, MED), DX + VPN redundancy and failover, and asymmetric routing. Use it even when the user only describes a symptom, such as \\\\\\\\\\\"traffic is leaving the region\\\\\\\\\\\", \\\\\\\\\\\"not using my local Direct Connect\\\\\\\\\\\", \\\\\\\\\\\"routing to the wrong VIF/location\\\\\\\\\\\", or \\\\\\\\\\\"path is asymmetric\\\\\\\\\\\". Read-only: analysis, recommendations, and describe/get/list validation commands grounded in public AWS docs.\\\\'\\\\\\\\r\\\\\\\\nmetadata:\\\\\\\\r\\\\\\\\n version: \\\\\\\\\\\"1.2.5\\\\\\\\\\\"\\\\\\\\r\\\\\\\\n author: midakin\\\\\\\\r\\\\\\\\n aws-devops-agent-skills.agent-types: \\\\\\\\\\\"CHAT, INCIDENT_RCA\\\\\\\\\\\"\\\\\\\\r\\\\\\\\n---\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Overview\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nThis skill provides routing-first analysis for AWS networking. It reasons about the full traffic\\\\\\\\r\\\\\\\\npath, applies the documented route-evaluation logic for each service, and grounds recommendations\\\\\\\\r\\\\\\\\nin public AWS documentation. It is **read-only**: it produces analysis, recommendations, and\\\\\\\\r\\\\\\\\nread-only validation commands (`describe-*`, `get-*`, `list-*`) - never mutating actions.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Verification & Anti-Hallucination Directives (Mandatory)\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- Do not generate any fact, citation, or source that is not fully verifiable. If you lack a verified\\\\\\\\r\\\\\\\\n source, state \\\\\\\\\\\"I cannot verify this\\\\\\\\\\\" rather than inventing details.\\\\\\\\r\\\\\\\\n- State assumptions explicitly before answering.\\\\\\\\r\\\\\\\\n- Cross-check each claim against public AWS documentation or the `references/` files. Omit claims you\\\\\\\\r\\\\\\\\n cannot confirm.\\\\\\\\r\\\\\\\\n- Only cite AWS documentation URLs you have actually retrieved or confirmed. Do not fabricate links.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## When to use this skill\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nActivate when the user asks about:\\\\\\\\r\\\\\\\\n- Cloud WAN route evaluation / Core Network Edge (CNE) path selection\\\\\\\\r\\\\\\\\n- Direct Connect Gateway path selection, local-region preference, or LP communities\\\\\\\\r\\\\\\\\n- Transit Gateway route evaluation, ECMP, or peering routing\\\\\\\\r\\\\\\\\n- VPC route tables and longest-prefix-match behavior\\\\\\\\r\\\\\\\\n- BGP traffic engineering (communities, AS-path prepending, MED)\\\\\\\\r\\\\\\\\n- DX + VPN redundancy, active/active vs active/passive, or failover design\\\\\\\\r\\\\\\\\n- Why traffic is taking an unexpected path\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## What this skill does NOT do\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nThis skill is **read-only and advisory**. It will not:\\\\\\\\r\\\\\\\\n- **Design or apply infrastructure changes.** It does not create, modify, or delete AWS resources,\\\\\\\\r\\\\\\\\n and it does not run mutating CLI/API calls (`create-*`, `modify-*`, `associate-*`, `delete-*`,\\\\\\\\r\\\\\\\\n `put-*`, `update-*`) or write Infrastructure-as-Code to be deployed. Any change it describes is a\\\\\\\\r\\\\\\\\n *recommendation* for you to review and implement yourself.\\\\\\\\r\\\\\\\\n- **Configure BGP or push routing policy.** It explains which communities, AS-path, or MED settings\\\\\\\\r\\\\\\\\n to use, but it does not apply them to VIFs, route tables, or Cloud WAN policy documents.\\\\\\\\r\\\\\\\\n- **Execute anything against a live account beyond read-only validation.** Commands it provides are\\\\\\\\r\\\\\\\\n limited to `describe-*`, `get-*`, and `list-*` for confirming state.\\\\\\\\r\\\\\\\\n- **Guarantee production outcomes.** It recommends non-production testing and blue/green rollout;\\\\\\\\r\\\\\\\\n it does not perform the cutover or validate the result for you.\\\\\\\\r\\\\\\\\n- **Provide compliance, security, or contractual sign-off**, or make claims it cannot ground in\\\\\\\\r\\\\\\\\n public AWS documentation or the `references/` files.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nFor designing and applying changes, hand the recommendation to a change-capable workflow (with\\\\\\\\r\\\\\\\\nappropriate review, approvals, and testing).\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## How to analyze a routing problem\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nWork through this checklist in order; each step depends on the one before it.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- [ ] **1. Establish the path.** Identify source, destination, and every routing construct in\\\\\\\\r\\\\\\\\n between (VPC route table \\\\\\\\u2192 TGW/Cloud WAN CNE \\\\\\\\u2192 DXGW \\\\\\\\u2192 VIF \\\\\\\\u2192 on-prem, or the relevant subset).\\\\\\\\r\\\\\\\\n- [ ] **2. Name the route table.** Be explicit about which table makes each decision (VPC RT, TGW RT,\\\\\\\\r\\\\\\\\n CNE RT, DXGW internal selection).\\\\\\\\r\\\\\\\\n- [ ] **3. Apply the evaluation order for that construct** (see below).\\\\\\\\r\\\\\\\\n- [ ] **4. Identify the deciding attribute** (longest prefix, local preference, AS-path, MED, source\\\\\\\\r\\\\\\\\n type).\\\\\\\\r\\\\\\\\n- [ ] **5. Flag non-determinism** (e.g., ECMP or \\\\\\\\\\\"deterministically random\\\\\\\\\\\" tiebreakers) and\\\\\\\\r\\\\\\\\n recommend a deterministic alternative.\\\\\\\\r\\\\\\\\n- [ ] **6. Cite the AWS documentation** that supports the behavior.\\\\\\\\r\\\\\\\\n- [ ] **7. Provide read-only validation commands** the user can run to confirm.\\\\\\\\r\\\\\\\\n- [ ] **8. Self-check before answering.** Re-read the answer against steps 1-7: every route table is\\\\\\\\r\\\\\\\\n named, the evaluation order matches the construct, each cited URL was actually retrieved, and every\\\\\\\\r\\\\\\\\n command is read-only. Fix any gap before presenting.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Key evaluation rules\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Cloud WAN (per CNE)\\\\\\\\r\\\\\\\\nLongest prefix match \\\\\\\\u2192 static \\\\\\\\u2192 VPC-propagated (same region) \\\\\\\\u2192 unequal AS-path/MED (shortest wins) \\\\\\\\u2192\\\\\\\\r\\\\\\\\nequal AS-path & MED source preference: DXGW-propagated \\\\\\\\u2192 Cloud WAN Connect \\\\\\\\u2192 S2S VPN \\\\\\\\u2192 other\\\\\\\\r\\\\\\\\n(TGW peering, remote CNEs; identical from 2+ sources = deterministically random).\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Direct Connect (private/transit VIF, outbound AWS\\\\\\\\u2192on-prem)\\\\\\\\r\\\\\\\\nLongest prefix match \\\\\\\\u2192 local preference \\\\\\\\u2192 AS_PATH length \\\\\\\\u2192 MED \\\\\\\\u2192 ECMP (equal AS_PATH and BGP\\\\\\\\r\\\\\\\\nattributes). LP communities `7224:7300/7200/7100` (High/Medium/Low) set local preference and are\\\\\\\\r\\\\\\\\nevaluated before AS-path. DXGW prefers the local associated-region DX by default.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Direction & tooling guidance\\\\\\\\r\\\\\\\\n- **AWS \\\\\\\\u2192 on-prem (egress):** control with DX LP communities.\\\\\\\\r\\\\\\\\n- **On-prem \\\\\\\\u2192 AWS (ingress):** control with the customer router\\\\'s local-pref/weight/MED.\\\\\\\\r\\\\\\\\n- **AS-path prepending:** within a region only - unreliable across regions because DXGW local-region\\\\\\\\r\\\\\\\\n LP overrides it. **LP communities:** work within-region and cross-region.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Reference material\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nDetailed mechanics, patterns, and caveats are in `references/`. Load only the file that matches the\\\\\\\\r\\\\\\\\nconstruct deciding the path:\\\\\\\\r\\\\\\\\n- [Cloud WAN + DX routing patterns](references/cloudwan-dx-routing-patterns.md) - load when the path\\\\\\\\r\\\\\\\\n crosses a Cloud WAN CNE: CNE route evaluation, DXGW path selection into Cloud WAN, community\\\\\\\\r\\\\\\\\n traffic-engineering patterns, regional inspection, multi-region egress.\\\\\\\\r\\\\\\\\n- [Direct Connect routing patterns](references/dx-routing-patterns.md) - load when the question is\\\\\\\\r\\\\\\\\n about a DX VIF or DXGW without Cloud WAN: BGP mechanics, BFD, MED, ASN ranges, VIF considerations,\\\\\\\\r\\\\\\\\n CloudHub, when VIFs can talk, active/active vs active/passive.\\\\\\\\r\\\\\\\\n- [Transit Gateway routing patterns](references/tgw-routing-patterns.md) - load when a TGW route\\\\\\\\r\\\\\\\\n table makes the decision: TGW route evaluation, ECMP limits, DXGW-to-TGW behavior, DX/VPN caveats,\\\\\\\\r\\\\\\\\n failover, peering.\\\\\\\\r\\\\\\\\n- [VPN + DX redundancy patterns](references/vpn-dx-redundancy-patterns.md) - load when VPN backs up\\\\\\\\r\\\\\\\\n or runs alongside DX: route preference between them, ECMP limits, asymmetric return paths.\\\\\\\\r\\\\\\\\n- [Networking strategic questions](references/networking-strategic-questions.md) - load when key\\\\\\\\r\\\\\\\\n facts are missing (regions, attachments, ASNs, communities) and you need discovery questions to ask\\\\\\\\r\\\\\\\\n before analyzing.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Output expectations\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- Explain the route-evaluation logic step by step.\\\\\\\\r\\\\\\\\n- Name the specific route table involved.\\\\\\\\r\\\\\\\\n- Give the end-to-end traffic path.\\\\\\\\r\\\\\\\\n- Compare options with tradeoffs (determinism, complexity, failover) when multiple solutions exist.\\\\\\\\r\\\\\\\\n- Include read-only AWS CLI validation commands.\\\\\\\\r\\\\\\\\n- Recommend testing in a non-production environment and blue/green for migrations.\\\\\\\\r\\\\\\\\n\\\", \\\"resource_manifest\\\": [\\\"SKILL.md\\\", \\\"references/cloudwan-dx-routing-patterns.md\\\", \\\"references/dx-routing-patterns.md\\\", \\\"references/networking-strategic-questions.md\\\", \\\"references/tgw-routing-patterns.md\\\", \\\"references/vpn-dx-redundancy-patterns.md\\\"]}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:20.364000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "865d0604-96c8-40d1-9fb5-3bb62df7a5c5", + "executionId": "688b6a99-26fd-46cd-b681-630b9d5e9105", + "recordId": "34bb661f-e8e2-44fb-a011-4c8c3f48a39e", + "content": "{\"id\": \"00534f98-f479-4d39-844f-e4e18fe61f82\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"872\"}]}", + "createdAt": "2026-10-27T17:05:20.442000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "865d0604-96c8-40d1-9fb5-3bb62df7a5c5", + "executionId": "688b6a99-26fd-46cd-b681-630b9d5e9105", + "recordId": "b3ae3c05-cd48-44f6-a46d-28e4c5bfd894", + "content": "{\"id\": \"b3ae3c05-cd48-44f6-a46d-28e4c5bfd894\", \"role\": \"assistant\", \"content\": [{\"id\": \"tooluse_h8WwJS5GGEhnSlAl7QOBEP\", \"tool_name\": \"read_skill_resource\", \"input\": {\"skill_id\": \"aws-routing\", \"path\": \"references/cloudwan-dx-routing-patterns.md\"}, \"type\": \"tool_use\", \"display_intent\": \"Reading Cloud WAN + Direct Connect routing patterns reference doc\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:20.444000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "865d0604-96c8-40d1-9fb5-3bb62df7a5c5", + "executionId": "688b6a99-26fd-46cd-b681-630b9d5e9105", + "recordId": "42fc0ebd-37d4-4f4c-9772-a9a835c5ce35", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.2, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:20.513000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "865d0604-96c8-40d1-9fb5-3bb62df7a5c5", + "executionId": "688b6a99-26fd-46cd-b681-630b9d5e9105", + "recordId": "e965b622-ffcd-4585-a174-e78cbe6913a2", + "content": "{\"id\": \"a8f595dd-8861-47ef-b3ee-b8d576a864c1\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_h8WwJS5GGEhnSlAl7QOBEP\", \"status\": \"success\", \"content\": [{\"text\": \"Done\"}]}], \"label\": \"Reading Cloud WAN + Direct Connect routing patterns reference doc\"}", + "createdAt": "2026-10-27T17:05:20.776000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "865d0604-96c8-40d1-9fb5-3bb62df7a5c5", + "executionId": "688b6a99-26fd-46cd-b681-630b9d5e9105", + "recordId": "b6068ef6-9af3-4e59-a153-fdc66bfe66c8", + "content": "{\"id\": \"d2fb7f8e-4655-459a-85de-babc44aa9088\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"With this grounding, I can start crafting the answer about the overlapping supernet 10.0.0.0/8 versus the VPC CIDRs 10.1.0.0/16 and 10.2.0.0/16 \\u2014 since route tables use longest prefix match, the more specific VPC CIDRs will always take precedence over the broader supernet route regardless of ordering.\\n\\nBut there's a real risk: if 10.0.0.0/8 gets advertised via DX into every CNE, any portion of that address space not covered by a propagated VPC CIDR would now route to on-prem instead of being dropped, creating potential blackholing or address space leakage. There's also exposure if a particular CNE never receives the VPC's /16 propagation (segment or region mismatch) \\u2014 in that case the /8 becomes the only match and traffic meant for the VPC gets misrouted to on-prem instead.\\n\\nBeyond that, asymmetric return-path issues could surface if on-prem infrastructure doesn't recognize the specific /16 it's suddenly receiving traffic for, and advertising such a broad supernet over private VPC space is generally considered a routing anti-pattern from a security standpoint \\u2014 though I'm confirming that longest-prefix-match still protects the exact overlapping /16s themselves.\\n\\nThe real exposure is for any address in 10.0.0.0/8 that isn't one of those defined /16s \\u2014 space that was previously unreachable now suddenly gets routed toward on-prem. This creates risk of future VPCs or peer-account CIDRs within that supernet getting hijacked toward on-prem instead of staying local, potential blackholes if on-prem has no actual answer for those addresses, and an operational trap where any newly created VPC automatically inherits a competing route from the supernet advertisement.\\n\\nThis is really a route-table hygiene anti-pattern: while longest-prefix-match keeps the currently-defined overlapping /16s correct, advertising the full /8 effectively injects a catch-all for the entire private address space into AWS, silently exposing roughly 65,000 other undefined /16s to leakage toward on-prem.\\n\\nThat's dangerous because traffic that should fail fast (misconfigured IPs or typos within 10.0.0.0/8) now gets routed externally instead of erroring, which masks bugs, creates security exposure, and adds unnecessary cost or latency. There's also a real conflict risk if the DX side has its own more specific subnets defined for unrelated on-prem apps \\u2014 not just excess scope, but an actual overlapping-use problem. The fix is to only advertise the specific non-overlapping ranges from on-prem rather than broad supernets.\\n\\nThis matches the skill guidance exactly \\u2014 using drop policies to block the overlapping supernet inbound while allowing the rest of /8 through. As a complementary fix, the on-prem router could avoid originating those specific /16s if they aren't real on-prem subnets, though if they are genuinely used on-prem that's a hard conflict requiring renumbering.\\n\\nSince longest-prefix-match is absolute at the Core Network Edge, I need to confirm the VPC-propagated /16 routes are actually present in every CNE route table that needs reachability, auditing segment sharing and propagation so nothing relies solely on the overlapping /8 from Direct Connect. I'd validate this with read-only commands like get-network-routes to confirm installed routes win over the /8, and check existing Cloud WAN Routing Policies via get-core-network-policy.\\n\\nIf genuine address overlap exists -- on-prem truly using those /16 ranges for real hosts -- that's an actual IP conflict requiring readdressing, not a routing fix; filtering just prevents AWS confusion but makes that on-prem range unreachable from AWS, which may be acceptable depending on whether it's real infrastructure. I'd also reference commands like describe-direct-connect-gateway-association and list-virtual-interface-routes, sticking to the describe-*/get-*/list-* read-only pattern.\\n\\nThe get-network-routes command checks the Core Network Edge's route table, and I'm fairly confident that's the correct name since the skill's reference material cites it directly. I'll also suggest using list-virtual-interface-routes against the DX virtual interface to confirm what AS-path and communities are actually being received.\\n\\nNow I'm drafting the final answer: walking through CNE route evaluation order with longest-prefix match, naming the specific CNE route table, and tracing the end-to-end traffic path from on-prem through Direct Connect into AWS.\\n\\nI'll organize the response as Issues followed by Fixes, stating my assumptions clearly \\u2014 that the DX attachment and both VPCs share the same Cloud WAN segment, that their CIDRs (10.1.0.0/16 and 10.2.0.0/16) sit inside a broader 10.0.0.0/8 supernet, and that longest-prefix match still governs any exact overlaps at the CNE regardless of the supernet relationship.\\n\\nThe real problems I see are: first, the rest of 10.0.0.0/8 that isn't covered by those two /16s now unexpectedly gets routed on-prem via DXGW/CNE, silently absorbing typos or future misuse of that address space rather than failing closed as unreachable. Second, this creates a fragile dependency where correct routing only holds if propagation stays perfectly consistent across every CNE involved.\\n\\nAny CNE where the VPC propagation is missing or delayed \\u2014 due to attachment gaps, route limits, or rollout timing \\u2014 falls back to the overlapping /8 route and misroutes traffic toward on-prem instead of failing fast, making the error much harder to detect since it looks like success. On top of that, adding new VPC CIDRs later inside 10.0.0.0/8 doesn't remove this risk until propagation is fully in place everywhere, so any gap keeps pulling new VPCs toward on-prem too.\\n\\nIt's also possible this isn't purely a routing artifact \\u2014 if on-prem genuinely has hosts using addresses inside those /16s, that's a real duplicate-addressing conflict, not just an over-broad advertisement, and no routing fix alone will resolve it. There's also a risk of asymmetric return traffic: if on-prem doesn't actually own those ranges but is advertising reachability, AWS could send return traffic there in fallback cases, and on-prem would silently drop it, producing intermittent failures that are hard to pin down.\\n\\nThe best fix is to stop on-prem from over-advertising: reconfigure its BGP so it only originates the specific subnets it actually owns, explicitly excluding 10.1.0.0/16 and 10.2.0.0/16 rather than flatly aggregating 10.0.0.0/8. If that config can't be changed right away, a fallback is to use Cloud WAN Routing Policies on the DX Gateway attachment to filter out the overlapping inbound prefixes.\\n\\nSo the second fix drops the specific /16 prefixes inbound from the DX attachment, forcing those two ranges to always resolve via VPC-propagated routes instead \\u2014 this stops the missing-propagation fragility but only covers those two /16s, leaving the rest of 10.0.0.0/8 still exposed as a stopgap rather than a full fix. I also want to separately audit propagation gaps, confirming every CNE that needs those VPCs actually has the /16 routes shared via attachment policy so the longest-prefix-match logic reliably wins.\\n\\nI should also flag that if on-prem genuinely has live systems using those same /16 ranges, that's a real duplicate-IP conflict needing renumbering, not a routing fix. To validate all this, I'd check installed routes per core network edge, confirm what prefixes the DX virtual interface is actually advertising, and review the current core network policy document.\\n\\nFor testing, I'd recommend applying any drop-rule policy change in a non-production segment or sandbox first, then confirming via get-network-routes that the two /16s are sourced only from VPC propagation before rolling out broadly.\\n\\nI'm assuming the DX attachment and both VPCs share the same Cloud WAN segment, since that's what determines whether they appear in the same CNE route tables -- if they're in different segments, the overlap concern wouldn't apply, so I'll flag that assumption explicitly.\\n\\nNow I'll draft the response: concise but thorough, covering the route table in question, the end-to-end path, comparing the available options, and including the relevant CLI commands.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:06:21.518000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "865d0604-96c8-40d1-9fb5-3bb62df7a5c5", + "executionId": "688b6a99-26fd-46cd-b681-630b9d5e9105", + "recordId": "c60e4966-a5ae-44f2-91c4-aa16e5577fa0", + "content": "{\"id\": \"5d245a33-4ba9-4885-af61-2b4a35963548\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"57261\"}]}", + "createdAt": "2026-10-27T17:06:21.628000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "865d0604-96c8-40d1-9fb5-3bb62df7a5c5", + "executionId": "688b6a99-26fd-46cd-b681-630b9d5e9105", + "recordId": "66b9d7d7-04ef-4806-b9ab-1743e992d681", + "content": "{\"id\": \"66b9d7d7-04ef-4806-b9ab-1743e992d681\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_h8WwJS5GGEhnSlAl7QOBEP\", \"content\": \"[{'text': '\\\\r\\\\n\\\\r\\\\n# Cloud WAN + Direct Connect Routing Patterns & Considerations\\\\r\\\\n\\\\r\\\\nThis document captures key routing mechanics and traffic engineering patterns for Cloud WAN architectures with Direct Connect egress. Use this knowledge when advising on Cloud WAN route evaluation, DX Gateway behavior, BGP community usage, and regional failover design.\\\\r\\\\n\\\\r\\\\n## Reference Documentation\\\\r\\\\n\\\\r\\\\n- [Cloud WAN Route Evaluation](https://docs.aws.amazon.com/network-manager/latest/cloudwan/cloudwan-route-evaluation.html)\\\\r\\\\n- [Cloud WAN Routing Policies](https://docs.aws.amazon.com/network-manager/latest/cloudwan/cloudwan-routing-policies.html)\\\\r\\\\n- [Direct Connect Routing Policies and BGP Communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html)\\\\r\\\\n\\\\r\\\\n## Cloud WAN Route Evaluation Order (Per CNE)\\\\r\\\\n\\\\r\\\\nAt each Core Network Edge, Cloud WAN evaluates routes in this order:\\\\r\\\\n\\\\r\\\\n1. **Most specific route** (longest prefix match) \\u2014 absolute, wins before everything\\\\r\\\\n2. For same-destination routes with different targets:\\\\r\\\\n 1. Static routes\\\\r\\\\n 2. VPC-propagated routes (same region)\\\\r\\\\n 3. **Unequal AS-path length and/or MED** \\u2014 shortest wins\\\\r\\\\n 4. **Equal AS-path and MED** \\u2014 preference order:\\\\r\\\\n 1. Direct Connect Gateway-propagated routes\\\\r\\\\n 2. Cloud WAN Connect (same region)\\\\r\\\\n 3. Site-to-Site VPN (same region)\\\\r\\\\n 4. Other sources (TGW peering, CNEs in other regions) \\u2014 if identical from 2+ sources: **deterministically random**\\\\r\\\\n\\\\r\\\\n## DX Gateway Path Selection Mechanics\\\\r\\\\n\\\\r\\\\n### Key Behaviors\\\\r\\\\n\\\\r\\\\n1. **DXGW propagates only ONE path per prefix to each CNE route table** \\u2014 it performs internal path selection and sends only the winner. CNEs do not see all available DX paths.\\\\r\\\\n\\\\r\\\\n2. **DXGW uses Local Preference to prefer local-region DX over remote regions** \\u2014 if a DX VIF exists in the same associated AWS region as the target CNE, DXGW prefers it regardless of AS-path length.\\\\r\\\\n\\\\r\\\\n **\\u26a0\\ufe0f Always verify the Associated Region for each DX location** \\u2014 some DX locations can associate to different regions than expected based on geography. For example, London-area locations associate to different regions:\\\\r\\\\n - Equinix LD5 (Slough) \\u2192 **eu-west-1** (Ireland)\\\\r\\\\n - Digital Realty LHR20 (London) \\u2192 **eu-west-1** (Ireland)\\\\r\\\\n - Telehouse (London Docklands) \\u2192 **eu-west-2** (London)\\\\r\\\\n - Equinix MA3 (Manchester) \\u2192 **eu-west-2** (London)\\\\r\\\\n \\\\r\\\\n The associated region determines which CNE/TGW considers the DX \\\"local\\\" for LP preference. Verify a location\\\\'s associated region with the `DescribeLocations` API/CLI (`aws directconnect describe-locations`) or the official AWS Direct Connect locations reference: https://aws.amazon.com/directconnect/locations/\\\\r\\\\n\\\\r\\\\n3. **Explicit LP communities override DXGW\\\\'s default local-region preference** \\u2014 applying `7224:7300/7200/7100` on advertised prefixes forces DXGW to honor the explicit LP before applying its local-vs-remote default.\\\\r\\\\n\\\\r\\\\n4. **LP communities are evaluated before AS-path** \\u2014 per DX documentation: \\\"Local preference BGP community tags are evaluated before any AS_PATH attribute.\\\" This applies at the DXGW path selection stage.\\\\r\\\\n\\\\r\\\\n5. **LP communities from DX VIFs are NOT visible as community tags in Cloud WAN** \\u2014 the community tags themselves don\\\\'t propagate into Cloud WAN routing policies. However, the LP value they set at the DXGW level influences which path gets propagated to the CNE route table.\\\\r\\\\n\\\\r\\\\n6. **DXGW ECMPs across equal remote paths** \\u2014 when the local-region DX fails and multiple remote DX locations have equal attributes (same LP, same AS-path), DXGW will **ECMP across them** rather than picking one non-deterministically. This provides load-balanced failover across all remaining DX locations.\\\\r\\\\n\\\\r\\\\n7. **Route filtering on the native Cloud WAN DXGW attachment (Routing Policies)** \\u2014 Cloud WAN Routing Policies support **prefix-based route filtering (drop) and summarization, inbound and outbound, on Direct Connect attachments**. This is a **separate, supported feature** from the (unsupported) DXGW allowed-prefixes list. Use it to drop an overlapping inbound supernet or to suppress specific VPC prefixes advertised outbound to on-prem.\\\\r\\\\n - Match prefixes via `prefix-in-cidr` or `prefix-in-prefix-list`; actions are `allow`/`drop`; `routing-policy-direction` is `inbound` or `outbound`.\\\\r\\\\n - **DX caveat:** BGP communities **cannot** be matched or set on Direct Connect attachments in Cloud WAN \\u2014 prefix/CIDR matching only (consistent with point 5). Community-based routing policies work on other attachment types but not DX.\\\\r\\\\n - Source: [Cloud WAN Routing Policies](https://docs.aws.amazon.com/network-manager/latest/cloudwan/cloudwan-routing-policies.html); the DXGW-attachment \\\"no allowed-prefixes list\\\" limitation ([DX gateway attachments in Cloud WAN](https://docs.aws.amazon.com/network-manager/latest/cloudwan/cloudwan-dxattach-about.html)) refers to the legacy allowed-prefixes construct, not routing-policy filtering.\\\\r\\\\n\\\\r\\\\n**Important distinction \\u2014 path selection behavior by service:**\\\\r\\\\n- **DXGW:** ECMPs across equal remote paths (load-balanced failover)\\\\r\\\\n- **TGW:** Deterministic \\u2014 selects oldest route for equal paths. Consistent but not customer-controllable.\\\\r\\\\n- **Cloud WAN CNE:** Deterministic \\u2014 \\\"deterministically random\\\" at Step 2.4.4 for equal remote CNE paths. Consistent but not customer-controllable.\\\\r\\\\n\\\\r\\\\n### Common Pitfall: AS-Path Prepending Does Not Override Local-Region LP at the DXGW\\\\r\\\\n\\\\r\\\\n**The mistaken assumption:** \\\"If I prepend a DX VIF\\\\'s prefix enough, I can steer that region\\\\'s traffic away from (or toward) it across regions.\\\"\\\\r\\\\n\\\\r\\\\n**Why it\\\\'s wrong:** DX egress selection happens at the **DXGW lookup** (see \\\"Two Route Lookups\\\" below), where **local-region Local Preference is evaluated before AS-path**. For a region that has a **local** DX advertisement, the DXGW picks that local path on LP \\u2014 AS-path (and therefore any prepend) is never consulted. So prepending a local VIF does **not** move that region\\\\'s traffic off its local DX.\\\\r\\\\n\\\\r\\\\n**Where prepending actually acts:** only as the **tiebreaker among remote advertisements** for regions that have **no local** advertisement for the prefix. There, all candidates are remote (LP equal), so the DXGW falls through to AS-path length and the shortest remote advertisement wins. Prepending shapes *which remote DX* those regions use \\u2014 nothing more.\\\\r\\\\n\\\\r\\\\n**Consequence:** Over-prepending a local VIF is not a cross-region steering lever; it only changes that location\\\\'s standing in the remote-tiebreak for *other* regions. To steer cross-region DX preference, use LP communities (which are evaluated before AS-path at the DXGW), not prepending. Confirm the installed per-region result with `get-network-routes` and the advertised paths with `list-virtual-interface-routes`.\\\\r\\\\n\\\\r\\\\n### Foundational Principle: Every Route Table Does Its Own Independent Lookup\\\\r\\\\n\\\\r\\\\nThe single most important mental model for reasoning about AWS routing: **every construct that has its own route table performs its own independent route lookup**, and a packet is evaluated **hop-by-hop** \\u2014 each route table it traverses makes a fresh, local forwarding decision using only the routes present in *that* table. There is no single end-to-end path calculation; the path is the concatenation of per-table decisions.\\\\r\\\\n\\\\r\\\\nRoute-table-owning constructs each do this independently, including:\\\\r\\\\n- **VPC route tables** (subnet \\u2192 destination: local, IGW, NAT, ENI, TGW/VGW/CWAN attachment, peering, endpoint)\\\\r\\\\n- **Transit Gateway route tables** (per-attachment association/propagation; oldest-route tiebreak on equal paths)\\\\r\\\\n- **DX Gateway** (its own BGP path selection; installs one winning path per prefix into each attached CNE/VGW)\\\\r\\\\n- **Cloud WAN Core Network Edge (CNE) route tables** (the CNE evaluation order over propagated routes)\\\\r\\\\n- **On-prem / customer routers** (their own BGP/IGP tables)\\\\r\\\\n\\\\r\\\\nThe routing \\\"outcome\\\" is what you get when you chain these together. Each table\\\\'s chosen next-hop simply hands the packet to the next construct, which then runs *its own* lookup. So to predict or debug a path, walk it table by table: at each hop, ask \\\"what does *this* route table select for this destination, given only the routes it has?\\\" The output (installed route) of one table\\\\'s lookup becomes an input (a propagated/attached route) to the next.\\\\r\\\\n\\\\r\\\\n**Common analysis error:** treating an end-to-end path as one decision, or assuming an attribute (like a DX BGP community or a local preference) that decided at one table still applies at the next. It does not automatically \\u2014 each table applies only its own selection rules over its own routes. (Example: DX LP communities decide at the DXGW\\\\'s table but are not visible inside the Cloud WAN CNE table.)\\\\r\\\\n\\\\r\\\\n### Applying the Principle: DXGW Lookup then CNE Lookup\\\\r\\\\n\\\\r\\\\nIn a DX \\u2192 Cloud WAN egress path, two of these independent lookups sit back-to-back, for the two directions. The DXGW\\\\'s lookup output is the CNE\\\\'s lookup input.\\\\r\\\\n\\\\r\\\\n**Lookup 1 \\u2014 DXGW path selection (on-prem \\u2192 AWS; decides what is INSTALLED into each CNE).**\\\\r\\\\nThe DXGW performs its own selection for a prefix and **installs one winning path into each CNE\\\\'s route table**. Its selection order is:\\\\r\\\\n1. **Local-region Local Preference first** \\u2014 if a DX VIF advertising the prefix is in the same associated region as the CNE, the DXGW prefers it **regardless of AS-path**. That local copy is installed into that region\\\\'s CNE.\\\\r\\\\n2. **If no local advertisement exists for that CNE**, every candidate is remote, so they share the **same (remote) LP** \\u2014 LP is equal. The DXGW then falls through to **AS-path length**, and the **shortest AS-path wins** and is installed into that CNE.\\\\r\\\\n\\\\r\\\\n**Lookup 2 \\u2014 Cloud WAN CNE route lookup (AWS \\u2192 on-prem return path).**\\\\r\\\\nFor return traffic, each CNE runs its own lookup over the routes **propagated into its table** (what Lookup 1 installed, plus routes propagated from other CNEs). This is where the CNE evaluation order (longest-prefix \\u2192 AS-path \\u2192 source preference) applies, operating on the already-propagated entries.\\\\r\\\\n\\\\r\\\\n### Worked Example (from the field diagram)\\\\r\\\\n\\\\r\\\\nOnly **FR5/Frankfurt (eu-central-1)** and **LD5/London (eu-west-1)** advertise `11.0.0.0/8`. Applying **Lookup 1 (DXGW)** per region:\\\\r\\\\n\\\\r\\\\n| CNE region | Has local `/8` advertisement? | DXGW decision | Installed `/8` path |\\\\r\\\\n|---|---|---|---|\\\\r\\\\n| eu-central-1 (FR5) | Yes (FR5 local) | Local-region LP wins \\u2014 AS-path not consulted | **FR5 (local DX)** |\\\\r\\\\n| eu-west-1 (LD5) | Yes (LD5 local) | Local-region LP wins \\u2014 AS-path not consulted | **LD5 (local DX)** |\\\\r\\\\n| ap-southeast-1 | No | FR5 and LD5 **both remote \\u2192 LP equal** \\u2192 fall through to AS-path \\u2192 **FR5 shortest wins** | **FR5** |\\\\r\\\\n| us-east-1 | No | both remote \\u2192 LP equal \\u2192 AS-path \\u2192 **FR5 shortest** | **FR5** |\\\\r\\\\n| us-east-2 | No | both remote \\u2192 LP equal \\u2192 AS-path \\u2192 **FR5 shortest** | **FR5** |\\\\r\\\\n\\\\r\\\\n**Why the non-LD5 regions use FR5\\\\'s advertisement:** they have no local `/8`, so at the DXGW both FR5 and LD5 are **remote** and carry the **same (remote) LP** \\u2014 LP is equal. The DXGW therefore falls to the next attribute, **AS-path length**, where **FR5 is shorter than the (heavily prepended) LD5 advertisement**, so FR5 wins and is installed. This is a straightforward DXGW LP-then-AS-path decision \\u2014 **not** a CNE-to-CNE backbone comparison.\\\\r\\\\n\\\\r\\\\n**Role of the AS-path prepends:** the prepends matter at **Lookup 1 (DXGW)**, specifically as the **tiebreaker when LP is equal** (i.e., only for regions with no local advertisement). They set which remote advertisement (FR5 vs LD5) wins for those regions. They do **not** override the local-region LP for regions that *do* have a local advertisement \\u2014 eu-central-1 and eu-west-1 pick their local DX on LP before AS-path is ever considered. So over-prepending LD5 does not stop eu-west-1 from choosing LD5 locally; it affects LD5\\\\'s standing in the **AS-path tiebreak for the other regions**, where FR5\\\\'s shorter path beats it.\\\\r\\\\n\\\\r\\\\n**Rule of thumb:** Because local-region LP is evaluated before AS-path at the DXGW, a region with a local `/8` advertisement egresses locally regardless of prepends. Use AS-path prepending only to shape which **remote** advertisement wins for regions that have **no local** advertisement (the LP-equal tiebreak). Verify the installed per-region result with `get-network-routes` (Lookup 2\\\\'s table) and the per-VIF advertised paths with `list-virtual-interface-routes` (Lookup 1\\\\'s inputs).\\\\r\\\\n\\\\r\\\\n### Common Pitfall: Confusing Prepend Count with Total AS-Path Length (Origination vs. Transit)\\\\r\\\\n\\\\r\\\\nWhen comparing AS-path lengths at a CNE (Step 2.3), the number that matters is the **total AS-path AWS receives on the VIF**, not the prepend count configured on a site. These are only the same when the advertising site is the **originator** of the prefix.\\\\r\\\\n\\\\r\\\\n**The distinction:**\\\\r\\\\n- **Originating site** \\u2014 the site owns/aggregates the prefix (e.g., `aggregate-address 11.0.0.0/8`). The AS-path AWS sees is just that site\\\\'s ASN, repeated by its prepend count. A `prepend 60208 2` on an originated route \\u2192 AS-path `60208 60208 60208` (length 3).\\\\r\\\\n- **Transiting site** \\u2014 the site does NOT originate the prefix; it **learns it from another site over the on-prem WAN** and re-advertises it to its DX. The AS-path already contains the origin\\\\'s ASN, so the transiting site\\\\'s ASN and prepends stack **on top of** the origin ASN. A `prepend 60208 2` applied to a `/8` learned from origin AS 56608 \\u2192 AS-path `60208 60208 60208 56608` (length **4**, not 3).\\\\r\\\\n\\\\r\\\\n**Config signature to tell them apart:** A site that **matches-and-prepends** a prefix in a route-policy but has **no `aggregate-address`/origination** for it is almost certainly **transiting** a route learned from elsewhere. Do not assume it originates the prefix just because the prefix appears in its outbound policy.\\\\r\\\\n\\\\r\\\\n**Why this matters for path selection:** Step 2.3 compares total path length. If you model a transiting site\\\\'s length as \\\"prepend count + 1\\\" (treating it as an originator) you will miscompute which copy wins \\u2014 often flipping the predicted winner and the failover order. Example: three sites advertise a `/8` that **originates at only one** of them; the two that transit-and-prepend it are longer than a naive prepend-count model suggests, and can tie with the origin.\\\\r\\\\n\\\\r\\\\n**How to get the ground truth:** Never infer AS-path length from prepend config alone when a prefix may be transited. Read the literal path AWS receives per VIF:\\\\r\\\\n```\\\\r\\\\naws directconnect list-virtual-interface-routes --virtual-interface-id \\\\r\\\\n```\\\\r\\\\nThis shows the exact AS-path (and communities) AWS accepted, settling origination-vs-transit and the true length used at Step 2.3.\\\\r\\\\n\\\\r\\\\n## DX Routing Selection \\u2014 General Best Practice\\\\r\\\\n\\\\r\\\\n- **Always use LP communities (`7224:7300/7200/7100`) for expected egress routing** \\u2014 High Preference BGP communities are the recommended mechanism to achieve predictable DX egress behavior.\\\\r\\\\n- **AS-path prepending: within a region ONLY** \\u2014 AS-path prepending can be used to influence egress traffic between DX connections belonging to the **same associated AWS region** (e.g., two DX locations both in the same region). It should NOT be used to influence traffic across regions \\u2014 DXGW\\\\'s local-region LP behavior makes cross-region AS-path prepending unreliable.\\\\r\\\\n- **BGP communities: within region OR across regions** \\u2014 LP communities work for both intra-region path selection (between VIFs in the same region) and cross-region preference (overriding DXGW\\\\'s default local-region LP to prefer a remote DX location globally).\\\\r\\\\n\\\\r\\\\n| Scope | Recommended Mechanism | Why |\\\\r\\\\n|---|---|---|\\\\r\\\\n| Cross-region DX preference | LP Communities (`7224:7300/7200/7100`) | Overrides DXGW local-region LP; evaluated before AS-path |\\\\r\\\\n| Within-region DX preference (same associated region) | AS-path prepending OR LP communities | Both work; AS-path provides granular control between same-region VIFs |\\\\r\\\\n| Never use for cross-region | AS-path prepending alone | DXGW local-region LP overrides AS-path at the DXGW level, making prepending unreliable across regions |\\\\r\\\\n\\\\r\\\\n## Traffic Engineering Patterns\\\\r\\\\n\\\\r\\\\n### Pattern 1: Global DX Preference (Single Primary)\\\\r\\\\n\\\\r\\\\n**Use case:** One DX location should be globally preferred for a prefix, with another as backup.\\\\r\\\\n\\\\r\\\\n**Solution:** Apply tiered LP communities from on-premises routers:\\\\r\\\\n\\\\r\\\\n| DX Location | Community | Role |\\\\r\\\\n|---|---|---|\\\\r\\\\n| Primary DX location | `7224:7300` | HIGH \\u2014 globally preferred |\\\\r\\\\n| Secondary DX location | `7224:7200` | MEDIUM \\u2014 failover |\\\\r\\\\n| All other DX locations | `7224:7100` | LOW \\u2014 last resort |\\\\r\\\\n\\\\r\\\\n**Result:** DXGW selects the primary for ALL CNEs (overriding local-region preference). If the primary fails, the medium-priority location (`7224:7200`) takes over automatically. (Note: \\\"medium\\\" here refers to the medium local-preference community, not the BGP MED attribute.)\\\\r\\\\n\\\\r\\\\n**Tradeoff:** Regions with a local DX that isn\\\\'t the primary will cross the AWS backbone to reach the preferred DX. Traffic still traverses the local-region firewall (service insertion happens at segment level, before the CNE egress decision).\\\\r\\\\n\\\\r\\\\n### Pattern 2: Per-Region DX Preference (Split DXGW)\\\\r\\\\n\\\\r\\\\n**Use case:** Different regions need different primary DX locations (e.g., one region group prefers DX-A, another region group prefers DX-B).\\\\r\\\\n\\\\r\\\\n**Problem:** Communities from a single DXGW are global \\u2014 cannot provide per-region preferences.\\\\r\\\\n\\\\r\\\\n**Solution:** Create separate DXGWs per region group, each with their own VIFs:\\\\r\\\\n- DXGW-A: Region-A DX locations (primary `7224:7300`, secondary `7224:7200`)\\\\r\\\\n- DXGW-B: Region-B DX locations (`7224:7300`)\\\\r\\\\n\\\\r\\\\nBoth DXGWs propagate their path to all CNEs. The CNE route table evaluates them at Step 2.3 (AS-path length). Control which DXGW wins per region by differentiating AS-path length on the VIFs between the two DXGWs.\\\\r\\\\n\\\\r\\\\nUse communities within each DXGW for internal failover. No Cloud WAN Routing Policies needed \\u2014 VIF-level BGP attributes provide per-region control at the CNE route table.\\\\r\\\\n\\\\r\\\\n### Pattern 3: Per-Region Control via More Specific Routes\\\\r\\\\n\\\\r\\\\n**Use case:** On-prem address space is regionally segmented.\\\\r\\\\n\\\\r\\\\n**Solution:** Advertise more-specific prefixes from the corresponding DX location:\\\\r\\\\n- DX location A advertises a more-specific covering Region-A on-prem hosts\\\\r\\\\n- DX location B advertises a more-specific covering Region-B on-prem hosts\\\\r\\\\n- All locations advertise the aggregate as a failover catch-all\\\\r\\\\n\\\\r\\\\n**Result:** Step 1 (longest prefix match) is absolute \\u2014 each CNE route table forwards to the correct DX based on destination address.\\\\r\\\\n\\\\r\\\\n**Requirement:** Only works if on-prem address space is subdivided into regional ranges. Does NOT work for flat supernets.\\\\r\\\\n\\\\r\\\\n## Multi-Region Centralized Internet Egress\\\\r\\\\n\\\\r\\\\n### Non-Local Exit Point Selection\\\\r\\\\n\\\\r\\\\nIn architectures where internet egress is centralized (e.g., a single region hosts the NAT/internet gateway or firewall for outbound traffic), Cloud WAN must route traffic from remote regions to the centralized exit point. Understanding how non-local exit points are selected is critical:\\\\r\\\\n\\\\r\\\\n- Each CNE evaluates routes per the standard route evaluation order\\\\r\\\\n- If the default route (0.0.0.0/0) or internet-bound prefix is only advertised from the centralized region\\\\'s attachment (e.g., via TGW Connect or VPN), all CNEs will install that path\\\\r\\\\n- Remote CNEs forward internet-bound traffic across the AWS backbone to the centralized region\\\\'s CNE\\\\r\\\\n\\\\r\\\\n### AS-Path Prepending for Deterministic Exit (via TGW/VPN)\\\\r\\\\n\\\\r\\\\nWhen routes are propagated from DX or VPN **via TGW** to Cloud WAN (TGW peering attachment), AS-path length information is preserved. This means:\\\\r\\\\n\\\\r\\\\n- **AS-path prepending can define a deterministic exit path** \\u2014 if multiple regions advertise the same prefix via TGW attachments to Cloud WAN, prepending on less-preferred paths ensures remote CNEs choose the shorter AS-path (preferred exit)\\\\r\\\\n- **More-specific routing also applies** \\u2014 advertising a more-specific prefix from the preferred exit region wins at Step 1 (longest prefix match) regardless of AS-path\\\\r\\\\n\\\\r\\\\n**Note:** This behavior differs from DX-attached prefixes where DXGW\\\\'s local-region LP can override AS-path. When routes come via TGW attachments to Cloud WAN, AS-path evaluation at the CNE operates without DXGW LP interference \\u2014 making prepending a valid and effective traffic engineering tool in this context.\\\\r\\\\n\\\\r\\\\n### Example: Centralized Internet Egress\\\\r\\\\n\\\\r\\\\n- Region-A TGW advertises `0.0.0.0/0` to Cloud WAN via TGW peering (short AS-path)\\\\r\\\\n- Region-B TGW also advertises `0.0.0.0/0` but with prepending (longer AS-path)\\\\r\\\\n- All CNEs prefer Region-A as the internet exit (Step 2.3: shorter AS-path wins)\\\\r\\\\n- If Region-A fails, Region-B\\\\'s longer path takes over as backup\\\\r\\\\n\\\\r\\\\n## Regional Firewall Inspection\\\\r\\\\n\\\\r\\\\n- Cloud WAN segment policy / service insertion routes traffic through the **same-region firewall** before it reaches the CNE\\\\r\\\\n- Traffic flow: VPC \\u2192 local firewall (service insertion) \\u2192 returns to local CNE \\u2192 CNE route table \\u2192 DXGW \\u2192 DX\\\\r\\\\n- With a local DX advertisement present, the DXGW installs the local path into the local CNE (local-region LP, Lookup 1) \\u2014 traffic stays at the local CNE for egress, preserving the local inspection chain\\\\r\\\\n- If the region has no local DX advertisement (or a community forces a remote DX to win), the installed path points out of region, and traffic leaves the local region for egress \\u2014 potentially bypassing local firewall inspection\\\\r\\\\n- **Requirement:** Firewall VPCs must exist in every region with workloads\\\\r\\\\n\\\\r\\\\n## Directional Control (Mapped to the Two Lookups)\\\\r\\\\n\\\\r\\\\nThe two route lookups map to the two directions. Keep these straight \\u2014 they are governed by different services and different mechanisms.\\\\r\\\\n\\\\r\\\\n| Direction | Governing lookup | Controlled by | Mechanism |\\\\r\\\\n|---|---|---|---|\\\\r\\\\n| **On-prem \\u2192 AWS** (which DX path is INSTALLED into each CNE) | **Lookup 1 \\u2014 DXGW path selection** | On-prem BGP attributes the DXGW sees: DX LP communities (`7224:7300/7200/7100`), then AS-path as the LP-equal tiebreak | DXGW evaluates local-region LP first, then AS-path; installs one winning path per prefix into each CNE |\\\\r\\\\n| **AWS \\u2192 On-prem** (return path from each CNE) | **Lookup 2 \\u2014 Cloud WAN CNE route lookup** | The routes propagated into the CNE table (from Lookup 1 + other CNEs), evaluated by the CNE order | CNE evaluation: longest-prefix \\u2192 AS-path \\u2192 source preference, over already-propagated routes |\\\\r\\\\n\\\\r\\\\nNotes:\\\\r\\\\n- DX LP communities set the LP the **DXGW** uses in Lookup 1 \\u2014 they are the recommended lever for steering which DX path is installed. They are **not visible inside Cloud WAN** and do not act at Lookup 2.\\\\r\\\\n- If on-prem AS-path prepending was being used to influence selection, prefer LP communities at the DXGW (Lookup 1); over-prepending a local VIF does not achieve cross-region steering (see the AS-Path Prepending pitfall above).\\\\r\\\\n\\\\r\\\\n## Cleanup Recommendations\\\\r\\\\n\\\\r\\\\n- **Prefer LP communities over prepending for DX egress steering.** Communities set the LP the DXGW uses in Lookup 1 and are evaluated before AS-path; they can steer cross-region preference, which prepending cannot (local-region LP overrides AS-path for any region with a local advertisement).\\\\r\\\\n- **Do not use prepending on a local VIF as a cross-region lever.** At the DXGW, a region with a local advertisement selects it on LP regardless of prepends. Prepending only affects the AS-path tiebreak among **remote** advertisements for regions that have no local advertisement \\u2014 so its only legitimate use here is choosing which remote DX those regions fall to.\\\\r\\\\n- **Once communities are in place, remove leftover prepends** as a cleanup step, keeping only any within-region or remote-tiebreak prepends that serve a confirmed purpose.\\\\r\\\\n- Always confirm the actual outcome with `get-network-routes` (installed per-CNE result) and `list-virtual-interface-routes` (the AS-path/communities the DXGW actually received) rather than reasoning from configured prepend counts alone.\\\\r\\\\n'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:06:33.434000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "865d0604-96c8-40d1-9fb5-3bb62df7a5c5", + "executionId": "688b6a99-26fd-46cd-b681-630b9d5e9105", + "recordId": "0ddd552e-b423-4be0-a927-e9d6659398c5", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 5.1, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:06:33.531000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "865d0604-96c8-40d1-9fb5-3bb62df7a5c5", + "executionId": "688b6a99-26fd-46cd-b681-630b9d5e9105", + "recordId": "39fd27c6-d0e4-44ed-8f4c-17cad25f04c6", + "content": "{\"id\": \"39fd27c6-d0e4-44ed-8f4c-17cad25f04c6\", \"role\": \"assistant\", \"content\": [{\"text\": \"Advertising `10.0.0.0/8` from on-prem into Cloud WAN when you already have `10.1.0.0/16` and `10.2.0.0/16` as VPC CIDRs is a classic supernet-overlap scenario. Here's how it plays out and what to do about it.\\n\\n**Assumption:** the DX attachment and both VPCs share the same Cloud WAN segment (so their routes land in the same CNE route tables). If they're in different, non-sharing segments, the overlap never actually gets evaluated against the VPC routes \\u2014 worth confirming first.\\n\\n## Why it's a problem\\n\\nThe decision point is each **Core Network Edge (CNE) route table**. Step 1 of CNE evaluation is longest-prefix-match (LPM), which is absolute \\u2014 so traffic to an address literally inside `10.1.0.0/16` or `10.2.0.0/16` *will* correctly prefer the VPC-propagated `/16` over the DX-sourced `/8`, as long as that `/16` is actually propagated into the CNE in question. The real issues are:\\n\\n1. **Catch-all leak for the rest of `10.0.0.0/8`.** Everything in that supernet *not* covered by your two `/16`s previously had no route at all. Now it silently gets a path out to on-prem via the DX Gateway \\u2192 CNE. Typos, future accidental use of that space, or misconfigured workloads that should fail fast instead succeed toward on-prem \\u2014 harder to detect than a clean blackhole.\\n2. **Fragile dependency on propagation.** Correctness for the two `/16`s only holds if VPC-propagated routes reach *every* CNE that needs them. Any gap \\u2014 a segment-sharing misconfiguration, a delayed attachment, a rollout window \\u2014 and that CNE falls back to the overlapping `/8` from DX, silently misrouting a whole VPC's traffic to on-prem instead of failing obviously.\\n3. **Future trap.** Any new VPC CIDR you add later inside `10.0.0.0/8` is exposed to the same race until its propagation is confirmed everywhere.\\n4. **Possible genuine duplicate addressing.** If on-prem actually has live hosts inside `10.1.0.0/16` or `10.2.0.0/16` (not just an oversized aggregate), that's a real IP conflict \\u2014 no routing policy fixes true duplicate addressing, only renumbering does.\\n5. **Asymmetric/intermittent failures.** In any fallback case where the `/8` wins, return traffic gets sent to on-prem for addresses on-prem doesn't actually own, producing drops or NAKs that look intermittent rather than consistent.\\n\\n## How to fix it\\n\\n**Best fix \\u2014 stop over-advertising at the source.** Reconfigure the on-prem router to not originate `10.0.0.0/8` as a flat aggregate; advertise only the ranges it legitimately owns, explicitly excluding `10.1.0.0/16` and `10.2.0.0/16`. This removes both the catch-all exposure and any genuine conflict, and doesn't depend on AWS-side filtering.\\n\\n**Stopgap if on-prem config can't change immediately \\u2014 Cloud WAN Routing Policies on the DX attachment.** Cloud WAN supports prefix-based route filtering on DX attachments: add an **inbound** policy matching `prefix-in-cidr` for `10.1.0.0/16` and `10.2.0.0/16` with action `drop`. That guarantees VPC-propagated routes are the *only* source for those two `/16`s in every CNE (closing issue #2), but it doesn't address the broader catch-all exposure for the rest of `10.0.0.0/8` \\u2014 it's a scoped stopgap, not a full fix.\\n\\n**Regardless of which fix you pick \\u2014 audit propagation gaps.** Confirm every CNE that needs the two VPCs actually has their `/16` routes propagated into it, so LPM reliably wins before relying on it.\\n\\n**If on-prem genuinely hosts systems in those ranges**, that's a duplicate-IP conflict requiring renumbering on one side \\u2014 routing fixes alone won't resolve it.\\n\\n### Read-only validation\\n\\n- `aws networkmanager get-network-routes --core-network-id ...` \\u2014 confirm, per CNE, whether `10.1.0.0/16`/`10.2.0.0/16` are winning via VPC propagation and whether `10.0.0.0/8` is installed anywhere\\n- `aws directconnect list-virtual-interface-routes --virtual-interface-id ` \\u2014 confirm exactly what's being advertised from on-prem (prefix, AS-path, communities)\\n- `aws networkmanager get-core-network-policy --core-network-id ` \\u2014 check current segment/routing-policy config for existing filters\\n- `aws directconnect describe-direct-connect-gateway-associations` \\u2014 confirm which CNEs/segments the DXGW is actually associated with\\n\\nI'd test any policy change (the filter or the on-prem re-advertisement) in a non-production segment or sandbox core network first, verify with `get-network-routes` that the two `/16`s resolve only from VPC propagation, then roll out to production.\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:06:33.651000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "865d0604-96c8-40d1-9fb5-3bb62df7a5c5", + "executionId": "688b6a99-26fd-46cd-b681-630b9d5e9105", + "recordId": "1307d85a-4f0c-46bc-8f78-5bbd662a13f4", + "content": "{\"id\": \"3bb0e24e-15ca-42b9-9c9e-71a9b7e40efd\", \"role\": \"assistant\", \"content\": [{\"type\": \"text\", \"text\": \"Advertising `10.0.0.0/8` from on-prem into Cloud WAN when you already have `10.1.0.0/16` and `10.2.0.0/16` as VPC CIDRs is a classic supernet-overlap scenario. Here's how it plays out and what to do about it.\\n\\n**Assumption:** the DX attachment and both VPCs share the same Cloud WAN segment (so their routes land in the same CNE route tables). If they're in different, non-sharing segments, the overlap never actually gets evaluated against the VPC routes \\u2014 worth confirming first.\\n\\n## Why it's a problem\\n\\nThe decision point is each **Core Network Edge (CNE) route table**. Step 1 of CNE evaluation is longest-prefix-match (LPM), which is absolute \\u2014 so traffic to an address literally inside `10.1.0.0/16` or `10.2.0.0/16` *will* correctly prefer the VPC-propagated `/16` over the DX-sourced `/8`, as long as that `/16` is actually propagated into the CNE in question. The real issues are:\\n\\n1. **Catch-all leak for the rest of `10.0.0.0/8`.** Everything in that supernet *not* covered by your two `/16`s previously had no route at all. Now it silently gets a path out to on-prem via the DX Gateway \\u2192 CNE. Typos, future accidental use of that space, or misconfigured workloads that should fail fast instead succeed toward on-prem \\u2014 harder to detect than a clean blackhole.\\n2. **Fragile dependency on propagation.** Correctness for the two `/16`s only holds if VPC-propagated routes reach *every* CNE that needs them. Any gap \\u2014 a segment-sharing misconfiguration, a delayed attachment, a rollout window \\u2014 and that CNE falls back to the overlapping `/8` from DX, silently misrouting a whole VPC's traffic to on-prem instead of failing obviously.\\n3. **Future trap.** Any new VPC CIDR you add later inside `10.0.0.0/8` is exposed to the same race until its propagation is confirmed everywhere.\\n4. **Possible genuine duplicate addressing.** If on-prem actually has live hosts inside `10.1.0.0/16` or `10.2.0.0/16` (not just an oversized aggregate), that's a real IP conflict \\u2014 no routing policy fixes true duplicate addressing, only renumbering does.\\n5. **Asymmetric/intermittent failures.** In any fallback case where the `/8` wins, return traffic gets sent to on-prem for addresses on-prem doesn't actually own, producing drops or NAKs that look intermittent rather than consistent.\\n\\n## How to fix it\\n\\n**Best fix \\u2014 stop over-advertising at the source.** Reconfigure the on-prem router to not originate `10.0.0.0/8` as a flat aggregate; advertise only the ranges it legitimately owns, explicitly excluding `10.1.0.0/16` and `10.2.0.0/16`. This removes both the catch-all exposure and any genuine conflict, and doesn't depend on AWS-side filtering.\\n\\n**Stopgap if on-prem config can't change immediately \\u2014 Cloud WAN Routing Policies on the DX attachment.** Cloud WAN supports prefix-based route filtering on DX attachments: add an **inbound** policy matching `prefix-in-cidr` for `10.1.0.0/16` and `10.2.0.0/16` with action `drop`. That guarantees VPC-propagated routes are the *only* source for those two `/16`s in every CNE (closing issue #2), but it doesn't address the broader catch-all exposure for the rest of `10.0.0.0/8` \\u2014 it's a scoped stopgap, not a full fix.\\n\\n**Regardless of which fix you pick \\u2014 audit propagation gaps.** Confirm every CNE that needs the two VPCs actually has their `/16` routes propagated into it, so LPM reliably wins before relying on it.\\n\\n**If on-prem genuinely hosts systems in those ranges**, that's a duplicate-IP conflict requiring renumbering on one side \\u2014 routing fixes alone won't resolve it.\\n\\n### Read-only validation\\n\\n- `aws networkmanager get-network-routes --core-network-id ...` \\u2014 confirm, per CNE, whether `10.1.0.0/16`/`10.2.0.0/16` are winning via VPC propagation and whether `10.0.0.0/8` is installed anywhere\\n- `aws directconnect list-virtual-interface-routes --virtual-interface-id ` \\u2014 confirm exactly what's being advertised from on-prem (prefix, AS-path, communities)\\n- `aws networkmanager get-core-network-policy --core-network-id ` \\u2014 check current segment/routing-policy config for existing filters\\n- `aws directconnect describe-direct-connect-gateway-associations` \\u2014 confirm which CNEs/segments the DXGW is actually associated with\\n\\nI'd test any policy change (the filter or the on-prem re-advertisement) in a non-production segment or sandbox core network first, verify with `get-network-routes` that the two `/16`s resolve only from VPC propagation, then roll out to production.\"}]}", + "createdAt": "2026-10-27T17:06:33.746000-05:00", + "recordType": "final_response" + } +] \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-overlapping-supernet/without_skill/functional-tests-results.json b/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-overlapping-supernet/without_skill/functional-tests-results.json new file mode 100644 index 00000000..08aadbf7 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-overlapping-supernet/without_skill/functional-tests-results.json @@ -0,0 +1,70 @@ +{ + "version": "v1", + "iteration": 1, + "eval_id": "cloudwan-overlapping-supernet", + "run_type": "without_skill", + "success": true, + "error": null, + "understanding_agent_space_skill": null, + "tests": { + "trigger": null, + "expected_output": { + "result": "passed", + "reasoning": "The agent's response correctly identifies the core technical mechanism: longest-prefix-match means the VPC /16 propagated routes win over the DX-advertised /8, so VPC-to-VPC traffic stays local. It also correctly flags the inverse/shadowing problem \u2014 on-prem hosts whose addresses fall inside the VPC CIDR ranges become unreachable from the VPC side because AWS always prefers the local VPC route ('AWS will always prefer the local VPC route, so on-prem hosts in that exact range become permanently unreachable from the VPC side'), which matches the expected 'shadowed/unreachable' concept, though phrased as a future risk rather than the primary current issue. The response recommends non-overlapping address planning as a clean fix ('Have the customer de-aggregate the advertisement' and 'carve out non-overlapping VPC CIDR blocks'), matching the expected best-fix guidance. It also explicitly mentions Cloud WAN Routing Policies as a purpose-built fix for prefix filtering/summarization, aligning with the expected answer's mention of Routing Policies to drop inbound supernet and suppress outbound VPC prefixes, though it doesn't go into the same level of detail (inbound drop + outbound suppress specifically on DX attachment) nor does it mention the BGP communities point about DX attachments. Overall, the response covers the key technical substance (LPM behavior, shadowing risk, non-overlapping planning as best fix, Cloud WAN Routing Policy as fix) even though it frames the shadowing issue somewhat differently (as a 'future risk' rather than current state) and omits the specific BGP communities/DX attachment limitation detail. Given the substantial overlap in core concepts and correct recommended fixes, this merits a pass with medium confidence due to minor framing differences and lack of the BGP communities footnote.", + "evidence": "\"If the customer later spins up a new VPC with a CIDR inside 10.0.0.0/8 that happens to literally duplicate an on-prem subnet... AWS will always prefer the local VPC route, so on-prem hosts in that exact range become permanently unreachable from the VPC side.\" and \"Have the customer de-aggregate the advertisement... This is the cleanest fix\" and \"Apply AWS Cloud WAN's new Routing Policy feature... it adds route filtering/summarization controls... to prevent this kind of over-broad reachability\"", + "confidence": "medium" + }, + "assertions": { + "assertion_results": [ + { + "text": "explains longest-prefix-match shadows on-prem hosts inside a VPC CIDR", + "evaluator": "llm", + "passed": true, + "evidence": "\"AWS will always prefer the local VPC route, so on-prem hosts in that exact range become permanently unreachable from the VPC side. That's the dangerous failure mode worth protecting against proactively.\" and also \"the most specific route always wins first... VPC-propagated routes within the segment take priority over dynamic (BGP/Direct Connect) routes when prefix lengths tie\"", + "reasoning": "The output explicitly explains that longest-prefix-match (VPC /16 or exact overlap) means on-prem hosts within that exact overlapping range become permanently unreachable/shadowed from the VPC side, which is the described phenomenon.", + "confidence": "high" + }, + { + "text": "recommends non-overlapping address planning as the best fix", + "evaluator": "llm", + "passed": true, + "evidence": "\"Have the customer de-aggregate the advertisement \u2014 advertise only the actual on-premises subnets they use... This is the cleanest fix and removes the ambiguity entirely.\" and also point 3 about carving non-overlapping VPC CIDR blocks.", + "reasoning": "The agent explicitly recommends de-aggregation/non-overlapping address planning and labels it 'the cleanest fix', which aligns with recommending non-overlapping address planning as a best fix, though it is presented as one of several fixes rather than singularly 'the best' - but it is clearly marked as cleanest/most durable.", + "confidence": "high" + }, + { + "text": "mentions Cloud WAN Routing Policies can filter (drop) prefixes on the Direct Connect attachment", + "evaluator": "llm", + "passed": false, + "evidence": "\"Apply AWS Cloud WAN's new Routing Policy feature (GA as of November 2025) \u2014 it adds route filtering/summarization controls, BGP attribute tuning, and visibility into route tables specifically to prevent this kind of over-broad reachability and asymmetric/sub-optimal path issues across Cloud WAN segments.\"", + "reasoning": "The output mentions Cloud WAN Routing Policy feature providing 'route filtering' in general terms, but it does not specifically state that Routing Policies can filter (drop) prefixes on the Direct Connect attachment specifically - it's a generic mention of route filtering/summarization controls without tying it specifically to dropping prefixes on the DX attachment. The more specific fix mentioned for DX is 'allowed prefixes on the Direct Connect Gateway', not Routing Policies.", + "confidence": "medium" + }, + { + "text": "does not contain 'error'", + "evaluator": "llm", + "passed": true, + "evidence": "No instance of the word 'error' appears anywhere in the output text.", + "reasoning": "Scanning the full response, the word 'error' is not used.", + "confidence": "high" + } + ], + "summary": { + "passed": 3, + "failed": 1, + "errored": 0, + "low_confidence": 0, + "total": 4, + "pass_rate": 0.75 + } + } + }, + "metrics": { + "runtime": "23s", + "cost": "$0.20", + "context_window": { + "utilization": "4.6%", + "compaction_count": 0 + } + } +} \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-overlapping-supernet/without_skill/outputs/journal_records.json b/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-overlapping-supernet/without_skill/outputs/journal_records.json new file mode 100644 index 00000000..8515b709 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/cloudwan-overlapping-supernet/without_skill/outputs/journal_records.json @@ -0,0 +1,98 @@ +[ + { + "agentSpaceId": "19bc8d9a-9ae4-4d8d-95a7-a26e21716bce", + "executionId": "54324125-9c9d-4e65-bae0-4d82831992b7", + "recordId": "07a6b542-caf7-485a-b773-8bf3fa938a14", + "content": "{\"id\": \"07a6b542-caf7-485a-b773-8bf3fa938a14\", \"role\": \"user\", \"content\": [{\"text\": \"A customer advertises a supernet (10.0.0.0/8) from Direct Connect into Cloud WAN that overlaps VPC CIDRs (10.1.0.0/16, 10.2.0.0/16). What issues would this cause and how do they fix it?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:06.880000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "19bc8d9a-9ae4-4d8d-95a7-a26e21716bce", + "executionId": "54324125-9c9d-4e65-bae0-4d82831992b7", + "recordId": "981a02e0-7e6a-4e90-be54-b2304052f99a", + "content": "{\"id\": \"0ddada7f-ae0a-4fa7-a3d4-5bfe7769eb00\", \"role\": \"assistant\", \"content\": [{\"type\": \"chat_title\", \"title\": \"Supernet overlap issues AWS networking architecture\"}]}", + "createdAt": "2026-10-27T17:05:07.674000-05:00", + "recordType": "chat_title" + }, + { + "agentSpaceId": "19bc8d9a-9ae4-4d8d-95a7-a26e21716bce", + "executionId": "54324125-9c9d-4e65-bae0-4d82831992b7", + "recordId": "f98f7a29-dfef-4fb1-bf70-afee91c7bea7", + "content": "{\"id\": \"88dac70d-2dd8-4027-a368-0cc7f195f777\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This is a conceptual AWS networking question about Direct Connect, Cloud WAN, and routing behavior with overlapping CIDRs, rather than something tied to the user's actual infrastructure or the product's specific features.\\n\\nSince this touches on technical specifics like longest prefix match and Cloud WAN routing behavior that my training data may not fully capture, I should consider verifying some of these claims rather than relying purely on recall.\\n\\nI'll verify a couple of specific points: Cloud WAN's static route preference over propagated routes, and Direct Connect Gateway handling of overlapping CIDRs.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:19.998000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "19bc8d9a-9ae4-4d8d-95a7-a26e21716bce", + "executionId": "54324125-9c9d-4e65-bae0-4d82831992b7", + "recordId": "84616416-bdff-414e-929a-8088aed0fe38", + "content": "{\"id\": \"0fea1c38-eb8f-404b-84bd-56e581f057ff\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"8009\"}]}", + "createdAt": "2026-10-27T17:05:20.074000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "19bc8d9a-9ae4-4d8d-95a7-a26e21716bce", + "executionId": "54324125-9c9d-4e65-bae0-4d82831992b7", + "recordId": "b11988ad-799c-4cea-8843-5fd971ec6c32", + "content": "{\"id\": \"b11988ad-799c-4cea-8843-5fd971ec6c32\", \"role\": \"assistant\", \"content\": [{\"text\": \"This is a classic routing conflict scenario. Let me verify a couple of specific technical details about how Cloud WAN and Direct Connect handle this before giving you the full picture.\", \"type\": \"text\"}, {\"id\": \"tooluse_N5nxMfyxPRdkRoku4FVG1b\", \"tool_name\": \"verify_aws_claim\", \"input\": {\"claim\": \"AWS Cloud WAN route table resolution uses longest prefix match, and static/local VPC routes take precedence over propagated routes of the same or shorter prefix length\"}, \"type\": \"tool_use\", \"display_intent\": \"Verify Cloud WAN route preference and longest-prefix-match behavior in docs.\"}, {\"id\": \"tooluse_zkZKQIWghefaNCZ5wYQerl\", \"tool_name\": \"verify_aws_claim\", \"input\": {\"claim\": \"Direct Connect Gateway and Transit Gateway route propagation behavior when advertised on-premises CIDR overlaps with VPC CIDR blocks\"}, \"type\": \"tool_use\", \"display_intent\": \"Verify Direct Connect route propagation behavior with overlapping VPC CIDRs.\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:21.602000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "19bc8d9a-9ae4-4d8d-95a7-a26e21716bce", + "executionId": "54324125-9c9d-4e65-bae0-4d82831992b7", + "recordId": "66965c38-4fbf-4919-97b1-84267e9291fd", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 3.8, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:21.680000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "19bc8d9a-9ae4-4d8d-95a7-a26e21716bce", + "executionId": "54324125-9c9d-4e65-bae0-4d82831992b7", + "recordId": "f21c28d4-75f6-4a25-88c8-a74b54541a09", + "content": "{\"id\": \"c3aaffaf-fd6a-4d57-a701-4fdba41b539e\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_N5nxMfyxPRdkRoku4FVG1b\", \"status\": \"success\", \"content\": [{\"text\": \"## Documentation search results\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"How route priority works\\\",\\\"context\\\":\\\"# How route priority works\\\\n\\\\nIn general, we direct traffic using the most specific route that matches the traffic.\\\\nThis is known as the longest prefix match. If your route table has overlapping or\\\\nmatching routes, additional rules apply.\\\\n\\\\nThe following list shows a route priority summary with links to sections below with more detailed information and examples:\\\\n\\\\n1. Longest prefix (for example, 10.10.2.15/32 has priority over 10.10.2.0/24)\\\\n2. Static routes (like VPC peering and internet gateway connections)\\\\n3. Prefix list routes\\\\n4. Propagated routes\\\\n\\\\n 1. Direct Connect BGP routes (dynamic routes)\\\\n 2. VPN static routes\\\\n 3. VPN BGP routes (dynamic routes) (like virtual private gateways)\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/vpc/latest/userguide/route-tables-priority.html\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"Route evaluation\\\",\\\"context\\\":\\\"# Route evaluation\\\\n\\\\nCloud WAN evaluates routes at each core network edge in the following order:\\\\n\\\\n1. The most specific route for the destination\\\\n2. For routes with the same destination IP address, but different targets, the\\\\n following route priority is used:\\\\n\\\\n 1. Static routes\\\\n 2. System routes. For more information, see Routing behavior for 1:1 NAT in AWS Cloud WAN.\\\\n 3. VPC-propagated routes in the same Region.\\\\n 4. For dynamic routes received at the core network with an\\\\n *unequal* AS path length and/or MED BGP\\\\n attributes, Cloud WAN evaluates them in the following order:\\\\n\\\\n 1. AS path length\\\\n 2. MED\\\\n 5. For dynamic routes received at the core network with\\\\n *equal* AS path length and MED BGP attributes,\\\\n Cloud WAN evaluates them in the following order:\\\\n\\\\n 1. Direct Connect gateway-propagated routes.\\\\n 2. Cloud WAN Connect-propagated routes in the same\\\\n Region.\\\\n 3. Site-to-Site VPN-propagated routes in the same\\\\n Region.\\\\n 4. Routes propagated from other sources, such as transit gateway peering\\\\n and core network edges in other remote Regions\\\\n over the AWS global infrastructure. If identical routes are\\\\n received from two or more sources, a single attachment will be\\\\n chosen in a deterministically random manner (this occurs on a per\\\\n segment/network function group basis so you could see different destinations\\\\n for the same route across different segment/network function groups)\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/network-manager/latest/cloudwan/cloudwan-route-evaluation.html\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"How do I manage asymmetric routing with Direct Connect?\\\",\\\"context\\\":\\\"Resolution\\\\n----------\\\\n\\\\nThe following resolution explains how to manage outbound traffic flow from AWS to your on-premises network. For inbound traffic flow from your on-premises network to AWS, you must use preferred routing options to set the preference to a particular path.\\\\n\\\\nTo manage asymmetric routing with Direct Connect, use prefix length and Border Gateway Protocol (BGP) attributes to determine path selection for different types of virtual interfaces.\\\\n\\\\n**Note:** For public virtual interfaces, AWS uses AS\\\\\\\\_PATH and the [longest prefix match](https://aws.amazon.com/blogs/networking-and-content-delivery/influencing-traffic-over-hybrid-networks-using-longest-prefix-match/) to determine the routing path\\\",\\\"url\\\":\\\"https://repost.aws/knowledge-center/direct-connect-manage-asymmetric-routing\\\"}]}}\\n\\n## Recent AWS announcements (What's New)\\n\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"AWS Cloud WAN adds Routing Policy for advanced traffic control and flexible network deployments - AWS\\\",\\\"context\\\":\\\"AWS Cloud WAN has introduced a new Routing Policy feature, now generally available, which allows customers to optimize route management, control traffic patterns, and customize network behavior across their global wide-area networks. This feature enables advanced routing techniques such as route filtering and summarization, minimizing route reachability blast radius, preventing sub-optimal or asymmetric connectivity patterns, and avoiding over-running of route-tables. Customers can set advanced BGP attributes to customize network traffic behavior and build resilient hybrid-cloud network architectures. The new Routing Policy feature is accessible in all AWS Regions where AWS Cloud WAN is available, and can be enabled using the AWS Management Console, AWS CLI, and AWS SDK, with no additional charge. The feature provides advanced visibility in routing databases for rapid troubleshooting of network issues in complex multi-path environments\\\",\\\"url\\\":\\\"https://aws.amazon.com/about-aws/whats-new/2025/11/aws-cloud-wan-routing-policy/\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"route53\\\",\\\"skill_description\\\":\\\"Configures Amazon Route 53 DNS: public and private records, traffic-steering routing policies, health checks, DNS Firewall, Route 53 Profiles, VPC Resolver (also known as Route 53 Resolver) for hybrid and Outposts networks, and Global Resolver. Applicable when the customer wants to point a hostname at a target, split or fail over traffic across endpoints, monitor an endpoint, block malicious domains, centralize DNS across accounts, or resolve private DNS across a hybrid network. Routes to the right per-task procedure in references. Does not cover CloudFront-specific setup or non-DNS networking.\\\\n\\\\n\\\\nServices: route53, vpc, outposts, cloudwatch, sns\\\\nTasks: configure, secure, monitor, deploy\\\\nPersona: developer, devops, network-admin, security-engineer, sre\\\\nWorkload: networking, security, hybrid, disaster-recovery\\\\n*\\\",\\\"skill_name\\\":\\\"route53\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"aws-networking\\\",\\\"skill_description\\\":\\\"Routes AWS networking requests to the correct service skill for implementation. Covers Route 53 (DNS, health checks, routing policies, Resolver, DNS Firewall), CloudFront (caching, edge, OAC, mTLS, signed URLs), Transit Gateway (multi-VPC hub, segmentation, centralized egress), Direct Connect (hybrid link, DX Gateway, MACsec), Site-to-Site VPN (IPsec tunnels, static or BGP), WAF (web ACLs, AWS Managed Rules, rate-based rules, Bot and Fraud Control), and Shield Advanced (L3/L4 DDoS). Applicable when creating, configuring, troubleshooting, or designing across these services, choosing between them, or diagnosing connectivity or traffic-filtering issues. Not for VPC subnets and route tables, load balancers, VPC endpoints, PrivateLink, API Gateway, IAM policy logic, container or serverless networking, or IaC authoring.\\\\n\\\\nServices: route53, route53resolver, cloudfront, directconnect, transitgateway, vpn, waf, shield\\\\nTasks: configure, troubleshoot, debug, deploy, secure, migrate\\\\nPersona: developer, network-engineer, devops, security-engineer\\\\nWorkload: networking, security\\\\n*\\\",\\\"skill_name\\\":\\\"aws-networking\\\"},{\\\"rank_order\\\":4,\\\"title\\\":\\\"transitgateway\\\",\\\"skill_description\\\":\\\"Configures AWS Transit Gateway: creating a hub and attaching VPCs, segmenting traffic with route tables, centralizing egress and forcing east-west traffic between VPCs through a central firewall via transit gateway route tables and appliance-mode attachments, connecting on-premises networks over the transit gateway side of a Site-to-Site VPN or Direct Connect attachment (including ECMP to aggregate bandwidth across multiple VPN tunnels), peering transit gateways across Regions, migrating from a VPC peering mesh, and routing IP multicast. Applicable when the transit gateway is the resource being configured: connecting many VPCs, isolating environments, routing egress or east-west traffic through the hub for inspection, reaching on-premises over the hub, linking Regions, or moving off a peering mesh. Not applicable for single-VPC routing, VPC peering between two VPCs (vpcpeering skill), Direct Connect gateway or virtual interface setup (directconnect skill), or Route 53 DNS work.\\\\n\\\\n\\\\nServices: transitgateway, vpc, directconnect, sitetositevpn, gatewayloadbalancer\\\\nTasks: configure, secure, deploy, migrate\\\\nPersona: developer, devops, network-admin, security-engineer, sre\\\\nWorkload: networking, security, hybrid, multi-region\\\\n*\\\",\\\"skill_name\\\":\\\"transitgateway\\\"}]}}\"}]}], \"label\": \"Verify Cloud WAN route preference and longest-prefix-match behavior in docs.\"}", + "createdAt": "2026-10-27T17:05:23.466000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "19bc8d9a-9ae4-4d8d-95a7-a26e21716bce", + "executionId": "54324125-9c9d-4e65-bae0-4d82831992b7", + "recordId": "779873a4-586e-448b-b959-e9210c7813b5", + "content": "{\"id\": \"da59a91b-e8d4-4ddb-92c2-454c47fa5986\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_zkZKQIWghefaNCZ5wYQerl\", \"status\": \"success\", \"content\": [{\"text\": \"## Documentation search results\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"How AWS Transit Gateway works\\\",\\\"context\\\":\\\"### Route propagation\\\\n\\\\nEach attachment comes with routes that can be installed in one or more transit gateway route\\\\ntables. When an attachment is propagated to a transit gateway route table, these routes are\\\\ninstalled in the route table. You can't filter on advertised routes.\\\\n\\\\nFor a VPC attachment, the CIDR blocks of the VPC are propagated to the transit\\\\ngateway route table.\\\\n\\\\nWhen dynamic routing is used with a VPN attachment, VPN Concentrator attachment or a Direct Connect gateway\\\\nattachment, you can propagate the routes learned from the on-premises router through\\\\nBGP to any of the transit gateway route tables.\\\\n\\\\nWhen dynamic routing is used with a VPN attachment or a VPN Concentrator attachment, the routes in the route table\\\\nassociated with the VPN attachment or VPN Concentrator attachment are advertised to the customer gateway through\\\\nBGP.\\\\n\\\\nFor a Connect attachment, routes in the route table associated with the Connect\\\\nattachment are advertised to the third-party virtual appliances, such as SD-WAN\\\\nappliances, running in a VPC through BGP.\\\\n\\\\nFor a Direct Connect gateway attachment, allowed prefixes interactions\\\\ncontrol which routes are advertised to the customer network from AWS.\\\\n\\\\nWhen a static route and a propagated route have the same destination, the static\\\\nroute has the higher priority, so the propagated route is not included in the route\\\\ntable. If you remove the static route, the overlapping propagated route is included\\\\nin the route table\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/vpc/latest/tgw/how-transit-gateways-work.html\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"How can I advertise VPC routes over a Direct Connect connection to a on-premises network over BGP?\\\",\\\"context\\\":\\\"### Direct Connect transit VIF connecting to a Direct Connect gateway associated with a transit gateway\\\\n\\\\nYou can associate one Direct Connect gateway to up to six transit gateways. Hundreds of VPCs can send traffic across the transit gateway and through the Direct Connect connection. The on-premises network must have the routes for all individual VPCs or use a summarized route. Routes that are advertised from the transit gateway towards on-premises with Direct Connect are defined in the allowed prefixes.\\\\n\\\\nAll prefixes advertise to the on-premises BGP peer. The allowed prefixes list advertises from the transit gateway to the on-premises Direct Connect peer. You can advertise a route for any IP address like 8.8.8.8/32 even though it's not a VPC CIDR connected to the transit gateway.\\\\n\\\\nThe allowed prefix list for the transit gateway has a [200 prefix limit](https://docs.aws.amazon.com/directconnect/latest/UserGuide/limits.html) combined total for IPv4 and IPv6. In the following example, VPC-A CIDR 10.77.0.0/16, VPC-B CIDR 10.66.0.0/16 and VPC-C 192.168.0.0/16 are attached to a transit gateway that connects to a Direct Connect gateway. If the allowed prefixes list is set to allow 10.0.0.0/8 and 192.168.5.0/24, then you don't receive the three VPC CIDR prefixes on the on-premises network. Instead, you receive the prefixes 10.0.0.0/8 and 192.168.5.0/24 advertised over the BGP.\\\\n\\\\nIf the allowed prefixes list is set to allow 10.0.0.0/8 and 192.168.0.0/16, then you receive the prefixes 10.0.0.0/8 and 192.168.0.0/16 advertised over the BGP.\\\\n\\\\nIf the allowed prefixes list is set to allow only 0.0.0.0/0, then you receive only the default route 0.0.0.0/0 advertised over the BGP.\\\\n\\\\nChanges to allowed prefixes on a VGW or a transit gateway association with a Direct Connect gateway update for the routes. They don't bring down the BGP session.\\\\n\\\\n**Note:** Changes made to\\\\n\\\\n*[truncated \\u2014 additional content omitted]*\\\",\\\"url\\\":\\\"https://repost.aws/knowledge-center/direct-connect-vpc-bgp\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"How do I configure Direct Connect and VPN failover with Transit Gateway?\\\",\\\"context\\\":\\\"### (Optional) Prevent asymmetric routing when VPN routes are more specific\\\\n\\\\nVPC CIDRs that you advertise from Transit Gateway VPN route tables are more specific than CIDRs that you advertise over a transit virtual interface. As a result, the customer gateway can prioritize the Site-to-Site VPN over Direct Connect and cause asymmetric routing.\\\\n\\\\n**Note:** When you create summarized routes for Amazon VPC CIDRs in the **Direct Connect Gateway allowed prefix** field, AWS VPN to on-premises network advertises the Amazon VPC CIDRs.\\\\n\\\\nTo resolve asymmetric routing, complete the following steps:\\\\n\\\\n1. Add the summarized routes that you associated with the Direct Connect gateway to the Site-to-Site VPN attachment that you associated with the transit gateway route table. For the target attachment in the route table, select an Amazon VPC with a CIDR. The CIDR must be part of the summarized route to the Site-to-Site VPN attachment transit gateway route table. You must advertise the summarized route and specific routes over the Site-to-Site VPN.\\\\n2. In the Site-to-Site VPN customer gateway, filter out the routes that advertise more specific CIDR prefixes over the VPN. The customer gateway must have the same summarized routes over both connections. The gateway prefers the AWS Direct Connect connection\\\",\\\"url\\\":\\\"https://repost.aws/knowledge-center/direct-connect-configure-vpn-failover\\\"}]}}\\n\\n## Recent AWS announcements (What's New)\\n\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"transitgateway\\\",\\\"skill_description\\\":\\\"Configures AWS Transit Gateway: creating a hub and attaching VPCs, segmenting traffic with route tables, centralizing egress and forcing east-west traffic between VPCs through a central firewall via transit gateway route tables and appliance-mode attachments, connecting on-premises networks over the transit gateway side of a Site-to-Site VPN or Direct Connect attachment (including ECMP to aggregate bandwidth across multiple VPN tunnels), peering transit gateways across Regions, migrating from a VPC peering mesh, and routing IP multicast. Applicable when the transit gateway is the resource being configured: connecting many VPCs, isolating environments, routing egress or east-west traffic through the hub for inspection, reaching on-premises over the hub, linking Regions, or moving off a peering mesh. Not applicable for single-VPC routing, VPC peering between two VPCs (vpcpeering skill), Direct Connect gateway or virtual interface setup (directconnect skill), or Route 53 DNS work.\\\\n\\\\n\\\\nServices: transitgateway, vpc, directconnect, sitetositevpn, gatewayloadbalancer\\\\nTasks: configure, secure, deploy, migrate\\\\nPersona: developer, devops, network-admin, security-engineer, sre\\\\nWorkload: networking, security, hybrid, multi-region\\\\n*\\\",\\\"skill_name\\\":\\\"transitgateway\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\n\\\\n\\\\nServices: directconnect, vpc, transitgateway\\\\nTasks: configure, secure, deploy\\\\nPersona: network-admin, devops, developer, sre\\\\nWorkload: networking, hybrid-cloud\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"aws-networking\\\",\\\"skill_description\\\":\\\"Routes AWS networking requests to the correct service skill for implementation. Covers Route 53 (DNS, health checks, routing policies, Resolver, DNS Firewall), CloudFront (caching, edge, OAC, mTLS, signed URLs), Transit Gateway (multi-VPC hub, segmentation, centralized egress), Direct Connect (hybrid link, DX Gateway, MACsec), Site-to-Site VPN (IPsec tunnels, static or BGP), WAF (web ACLs, AWS Managed Rules, rate-based rules, Bot and Fraud Control), and Shield Advanced (L3/L4 DDoS). Applicable when creating, configuring, troubleshooting, or designing across these services, choosing between them, or diagnosing connectivity or traffic-filtering issues. Not for VPC subnets and route tables, load balancers, VPC endpoints, PrivateLink, API Gateway, IAM policy logic, container or serverless networking, or IaC authoring.\\\\n\\\\nServices: route53, route53resolver, cloudfront, directconnect, transitgateway, vpn, waf, shield\\\\nTasks: configure, troubleshoot, debug, deploy, secure, migrate\\\\nPersona: developer, network-engineer, devops, security-engineer\\\\nWorkload: networking, security\\\\n*\\\",\\\"skill_name\\\":\\\"aws-networking\\\"}]}}\"}]}], \"label\": \"Verify Direct Connect route propagation behavior with overlapping VPC CIDRs.\"}", + "createdAt": "2026-10-27T17:05:23.570000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "19bc8d9a-9ae4-4d8d-95a7-a26e21716bce", + "executionId": "54324125-9c9d-4e65-bae0-4d82831992b7", + "recordId": "4a3bc019-2777-4825-b539-2b069694d06d", + "content": "{\"id\": \"4a3bc019-2777-4825-b539-2b069694d06d\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_N5nxMfyxPRdkRoku4FVG1b\", \"content\": \"[{'text': '## Documentation search results\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"How route priority works\\\",\\\"context\\\":\\\"# How route priority works\\\\\\\\n\\\\\\\\nIn general, we direct traffic using the most specific route that matches the traffic.\\\\\\\\nThis is known as the longest prefix match. If your route table has overlapping or\\\\\\\\nmatching routes, additional rules apply.\\\\\\\\n\\\\\\\\nThe following list shows a route priority summary with links to sections below with more detailed information and examples:\\\\\\\\n\\\\\\\\n1. Longest prefix (for example, 10.10.2.15/32 has priority over 10.10.2.0/24)\\\\\\\\n2. Static routes (like VPC peering and internet gateway connections)\\\\\\\\n3. Prefix list routes\\\\\\\\n4. Propagated routes\\\\\\\\n\\\\\\\\n 1. Direct Connect BGP routes (dynamic routes)\\\\\\\\n 2. VPN static routes\\\\\\\\n 3. VPN BGP routes (dynamic routes) (like virtual private gateways)\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/vpc/latest/userguide/route-tables-priority.html\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"Route evaluation\\\",\\\"context\\\":\\\"# Route evaluation\\\\\\\\n\\\\\\\\nCloud WAN evaluates routes at each core network edge in the following order:\\\\\\\\n\\\\\\\\n1. The most specific route for the destination\\\\\\\\n2. For routes with the same destination IP address, but different targets, the\\\\\\\\n following route priority is used:\\\\\\\\n\\\\\\\\n 1. Static routes\\\\\\\\n 2. System routes. For more information, see Routing behavior for 1:1 NAT in AWS Cloud WAN.\\\\\\\\n 3. VPC-propagated routes in the same Region.\\\\\\\\n 4. For dynamic routes received at the core network with an\\\\\\\\n *unequal* AS path length and/or MED BGP\\\\\\\\n attributes, Cloud WAN evaluates them in the following order:\\\\\\\\n\\\\\\\\n 1. AS path length\\\\\\\\n 2. MED\\\\\\\\n 5. For dynamic routes received at the core network with\\\\\\\\n *equal* AS path length and MED BGP attributes,\\\\\\\\n Cloud WAN evaluates them in the following order:\\\\\\\\n\\\\\\\\n 1. Direct Connect gateway-propagated routes.\\\\\\\\n 2. Cloud WAN Connect-propagated routes in the same\\\\\\\\n Region.\\\\\\\\n 3. Site-to-Site VPN-propagated routes in the same\\\\\\\\n Region.\\\\\\\\n 4. Routes propagated from other sources, such as transit gateway peering\\\\\\\\n and core network edges in other remote Regions\\\\\\\\n over the AWS global infrastructure. If identical routes are\\\\\\\\n received from two or more sources, a single attachment will be\\\\\\\\n chosen in a deterministically random manner (this occurs on a per\\\\\\\\n segment/network function group basis so you could see different destinations\\\\\\\\n for the same route across different segment/network function groups)\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/network-manager/latest/cloudwan/cloudwan-route-evaluation.html\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"How do I manage asymmetric routing with Direct Connect?\\\",\\\"context\\\":\\\"Resolution\\\\\\\\n----------\\\\\\\\n\\\\\\\\nThe following resolution explains how to manage outbound traffic flow from AWS to your on-premises network. For inbound traffic flow from your on-premises network to AWS, you must use preferred routing options to set the preference to a particular path.\\\\\\\\n\\\\\\\\nTo manage asymmetric routing with Direct Connect, use prefix length and Border Gateway Protocol (BGP) attributes to determine path selection for different types of virtual interfaces.\\\\\\\\n\\\\\\\\n**Note:** For public virtual interfaces, AWS uses AS\\\\\\\\\\\\\\\\_PATH and the [longest prefix match](https://aws.amazon.com/blogs/networking-and-content-delivery/influencing-traffic-over-hybrid-networks-using-longest-prefix-match/) to determine the routing path\\\",\\\"url\\\":\\\"https://repost.aws/knowledge-center/direct-connect-manage-asymmetric-routing\\\"}]}}\\\\n\\\\n## Recent AWS announcements (What\\\\'s New)\\\\n\\\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"AWS Cloud WAN adds Routing Policy for advanced traffic control and flexible network deployments - AWS\\\",\\\"context\\\":\\\"AWS Cloud WAN has introduced a new Routing Policy feature, now generally available, which allows customers to optimize route management, control traffic patterns, and customize network behavior across their global wide-area networks. This feature enables advanced routing techniques such as route filtering and summarization, minimizing route reachability blast radius, preventing sub-optimal or asymmetric connectivity patterns, and avoiding over-running of route-tables. Customers can set advanced BGP attributes to customize network traffic behavior and build resilient hybrid-cloud network architectures. The new Routing Policy feature is accessible in all AWS Regions where AWS Cloud WAN is available, and can be enabled using the AWS Management Console, AWS CLI, and AWS SDK, with no additional charge. The feature provides advanced visibility in routing databases for rapid troubleshooting of network issues in complex multi-path environments\\\",\\\"url\\\":\\\"https://aws.amazon.com/about-aws/whats-new/2025/11/aws-cloud-wan-routing-policy/\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"route53\\\",\\\"skill_description\\\":\\\"Configures Amazon Route 53 DNS: public and private records, traffic-steering routing policies, health checks, DNS Firewall, Route 53 Profiles, VPC Resolver (also known as Route 53 Resolver) for hybrid and Outposts networks, and Global Resolver. Applicable when the customer wants to point a hostname at a target, split or fail over traffic across endpoints, monitor an endpoint, block malicious domains, centralize DNS across accounts, or resolve private DNS across a hybrid network. Routes to the right per-task procedure in references. Does not cover CloudFront-specific setup or non-DNS networking.\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: route53, vpc, outposts, cloudwatch, sns\\\\\\\\nTasks: configure, secure, monitor, deploy\\\\\\\\nPersona: developer, devops, network-admin, security-engineer, sre\\\\\\\\nWorkload: networking, security, hybrid, disaster-recovery\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"route53\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"aws-networking\\\",\\\"skill_description\\\":\\\"Routes AWS networking requests to the correct service skill for implementation. Covers Route 53 (DNS, health checks, routing policies, Resolver, DNS Firewall), CloudFront (caching, edge, OAC, mTLS, signed URLs), Transit Gateway (multi-VPC hub, segmentation, centralized egress), Direct Connect (hybrid link, DX Gateway, MACsec), Site-to-Site VPN (IPsec tunnels, static or BGP), WAF (web ACLs, AWS Managed Rules, rate-based rules, Bot and Fraud Control), and Shield Advanced (L3/L4 DDoS). Applicable when creating, configuring, troubleshooting, or designing across these services, choosing between them, or diagnosing connectivity or traffic-filtering issues. Not for VPC subnets and route tables, load balancers, VPC endpoints, PrivateLink, API Gateway, IAM policy logic, container or serverless networking, or IaC authoring.\\\\\\\\n\\\\\\\\nServices: route53, route53resolver, cloudfront, directconnect, transitgateway, vpn, waf, shield\\\\\\\\nTasks: configure, troubleshoot, debug, deploy, secure, migrate\\\\\\\\nPersona: developer, network-engineer, devops, security-engineer\\\\\\\\nWorkload: networking, security\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"aws-networking\\\"},{\\\"rank_order\\\":4,\\\"title\\\":\\\"transitgateway\\\",\\\"skill_description\\\":\\\"Configures AWS Transit Gateway: creating a hub and attaching VPCs, segmenting traffic with route tables, centralizing egress and forcing east-west traffic between VPCs through a central firewall via transit gateway route tables and appliance-mode attachments, connecting on-premises networks over the transit gateway side of a Site-to-Site VPN or Direct Connect attachment (including ECMP to aggregate bandwidth across multiple VPN tunnels), peering transit gateways across Regions, migrating from a VPC peering mesh, and routing IP multicast. Applicable when the transit gateway is the resource being configured: connecting many VPCs, isolating environments, routing egress or east-west traffic through the hub for inspection, reaching on-premises over the hub, linking Regions, or moving off a peering mesh. Not applicable for single-VPC routing, VPC peering between two VPCs (vpcpeering skill), Direct Connect gateway or virtual interface setup (directconnect skill), or Route 53 DNS work.\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: transitgateway, vpc, directconnect, sitetositevpn, gatewayloadbalancer\\\\\\\\nTasks: configure, secure, deploy, migrate\\\\\\\\nPersona: developer, devops, network-admin, security-engineer, sre\\\\\\\\nWorkload: networking, security, hybrid, multi-region\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"transitgateway\\\"}]}}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}, {\"id\": \"tooluse_zkZKQIWghefaNCZ5wYQerl\", \"content\": \"[{'text': '## Documentation search results\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"How AWS Transit Gateway works\\\",\\\"context\\\":\\\"### Route propagation\\\\\\\\n\\\\\\\\nEach attachment comes with routes that can be installed in one or more transit gateway route\\\\\\\\ntables. When an attachment is propagated to a transit gateway route table, these routes are\\\\\\\\ninstalled in the route table. You can\\\\'t filter on advertised routes.\\\\\\\\n\\\\\\\\nFor a VPC attachment, the CIDR blocks of the VPC are propagated to the transit\\\\\\\\ngateway route table.\\\\\\\\n\\\\\\\\nWhen dynamic routing is used with a VPN attachment, VPN Concentrator attachment or a Direct Connect gateway\\\\\\\\nattachment, you can propagate the routes learned from the on-premises router through\\\\\\\\nBGP to any of the transit gateway route tables.\\\\\\\\n\\\\\\\\nWhen dynamic routing is used with a VPN attachment or a VPN Concentrator attachment, the routes in the route table\\\\\\\\nassociated with the VPN attachment or VPN Concentrator attachment are advertised to the customer gateway through\\\\\\\\nBGP.\\\\\\\\n\\\\\\\\nFor a Connect attachment, routes in the route table associated with the Connect\\\\\\\\nattachment are advertised to the third-party virtual appliances, such as SD-WAN\\\\\\\\nappliances, running in a VPC through BGP.\\\\\\\\n\\\\\\\\nFor a Direct Connect gateway attachment, allowed prefixes interactions\\\\\\\\ncontrol which routes are advertised to the customer network from AWS.\\\\\\\\n\\\\\\\\nWhen a static route and a propagated route have the same destination, the static\\\\\\\\nroute has the higher priority, so the propagated route is not included in the route\\\\\\\\ntable. If you remove the static route, the overlapping propagated route is included\\\\\\\\nin the route table\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/vpc/latest/tgw/how-transit-gateways-work.html\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"How can I advertise VPC routes over a Direct Connect connection to a on-premises network over BGP?\\\",\\\"context\\\":\\\"### Direct Connect transit VIF connecting to a Direct Connect gateway associated with a transit gateway\\\\\\\\n\\\\\\\\nYou can associate one Direct Connect gateway to up to six transit gateways. Hundreds of VPCs can send traffic across the transit gateway and through the Direct Connect connection. The on-premises network must have the routes for all individual VPCs or use a summarized route. Routes that are advertised from the transit gateway towards on-premises with Direct Connect are defined in the allowed prefixes.\\\\\\\\n\\\\\\\\nAll prefixes advertise to the on-premises BGP peer. The allowed prefixes list advertises from the transit gateway to the on-premises Direct Connect peer. You can advertise a route for any IP address like 8.8.8.8/32 even though it\\\\'s not a VPC CIDR connected to the transit gateway.\\\\\\\\n\\\\\\\\nThe allowed prefix list for the transit gateway has a [200 prefix limit](https://docs.aws.amazon.com/directconnect/latest/UserGuide/limits.html) combined total for IPv4 and IPv6. In the following example, VPC-A CIDR 10.77.0.0/16, VPC-B CIDR 10.66.0.0/16 and VPC-C 192.168.0.0/16 are attached to a transit gateway that connects to a Direct Connect gateway. If the allowed prefixes list is set to allow 10.0.0.0/8 and 192.168.5.0/24, then you don\\\\'t receive the three VPC CIDR prefixes on the on-premises network. Instead, you receive the prefixes 10.0.0.0/8 and 192.168.5.0/24 advertised over the BGP.\\\\\\\\n\\\\\\\\nIf the allowed prefixes list is set to allow 10.0.0.0/8 and 192.168.0.0/16, then you receive the prefixes 10.0.0.0/8 and 192.168.0.0/16 advertised over the BGP.\\\\\\\\n\\\\\\\\nIf the allowed prefixes list is set to allow only 0.0.0.0/0, then you receive only the default route 0.0.0.0/0 advertised over the BGP.\\\\\\\\n\\\\\\\\nChanges to allowed prefixes on a VGW or a transit gateway association with a Direct Connect gateway update for the routes. They don\\\\'t bring down the BGP session.\\\\\\\\n\\\\\\\\n**Note:** Changes made to\\\\\\\\n\\\\\\\\n*[truncated \\u2014 additional content omitted]*\\\",\\\"url\\\":\\\"https://repost.aws/knowledge-center/direct-connect-vpc-bgp\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"How do I configure Direct Connect and VPN failover with Transit Gateway?\\\",\\\"context\\\":\\\"### (Optional) Prevent asymmetric routing when VPN routes are more specific\\\\\\\\n\\\\\\\\nVPC CIDRs that you advertise from Transit Gateway VPN route tables are more specific than CIDRs that you advertise over a transit virtual interface. As a result, the customer gateway can prioritize the Site-to-Site VPN over Direct Connect and cause asymmetric routing.\\\\\\\\n\\\\\\\\n**Note:** When you create summarized routes for Amazon VPC CIDRs in the **Direct Connect Gateway allowed prefix** field, AWS VPN to on-premises network advertises the Amazon VPC CIDRs.\\\\\\\\n\\\\\\\\nTo resolve asymmetric routing, complete the following steps:\\\\\\\\n\\\\\\\\n1. Add the summarized routes that you associated with the Direct Connect gateway to the Site-to-Site VPN attachment that you associated with the transit gateway route table. For the target attachment in the route table, select an Amazon VPC with a CIDR. The CIDR must be part of the summarized route to the Site-to-Site VPN attachment transit gateway route table. You must advertise the summarized route and specific routes over the Site-to-Site VPN.\\\\\\\\n2. In the Site-to-Site VPN customer gateway, filter out the routes that advertise more specific CIDR prefixes over the VPN. The customer gateway must have the same summarized routes over both connections. The gateway prefers the AWS Direct Connect connection\\\",\\\"url\\\":\\\"https://repost.aws/knowledge-center/direct-connect-configure-vpn-failover\\\"}]}}\\\\n\\\\n## Recent AWS announcements (What\\\\'s New)\\\\n\\\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"transitgateway\\\",\\\"skill_description\\\":\\\"Configures AWS Transit Gateway: creating a hub and attaching VPCs, segmenting traffic with route tables, centralizing egress and forcing east-west traffic between VPCs through a central firewall via transit gateway route tables and appliance-mode attachments, connecting on-premises networks over the transit gateway side of a Site-to-Site VPN or Direct Connect attachment (including ECMP to aggregate bandwidth across multiple VPN tunnels), peering transit gateways across Regions, migrating from a VPC peering mesh, and routing IP multicast. Applicable when the transit gateway is the resource being configured: connecting many VPCs, isolating environments, routing egress or east-west traffic through the hub for inspection, reaching on-premises over the hub, linking Regions, or moving off a peering mesh. Not applicable for single-VPC routing, VPC peering between two VPCs (vpcpeering skill), Direct Connect gateway or virtual interface setup (directconnect skill), or Route 53 DNS work.\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: transitgateway, vpc, directconnect, sitetositevpn, gatewayloadbalancer\\\\\\\\nTasks: configure, secure, deploy, migrate\\\\\\\\nPersona: developer, devops, network-admin, security-engineer, sre\\\\\\\\nWorkload: networking, security, hybrid, multi-region\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"transitgateway\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: directconnect, vpc, transitgateway\\\\\\\\nTasks: configure, secure, deploy\\\\\\\\nPersona: network-admin, devops, developer, sre\\\\\\\\nWorkload: networking, hybrid-cloud\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"aws-networking\\\",\\\"skill_description\\\":\\\"Routes AWS networking requests to the correct service skill for implementation. Covers Route 53 (DNS, health checks, routing policies, Resolver, DNS Firewall), CloudFront (caching, edge, OAC, mTLS, signed URLs), Transit Gateway (multi-VPC hub, segmentation, centralized egress), Direct Connect (hybrid link, DX Gateway, MACsec), Site-to-Site VPN (IPsec tunnels, static or BGP), WAF (web ACLs, AWS Managed Rules, rate-based rules, Bot and Fraud Control), and Shield Advanced (L3/L4 DDoS). Applicable when creating, configuring, troubleshooting, or designing across these services, choosing between them, or diagnosing connectivity or traffic-filtering issues. Not for VPC subnets and route tables, load balancers, VPC endpoints, PrivateLink, API Gateway, IAM policy logic, container or serverless networking, or IaC authoring.\\\\\\\\n\\\\\\\\nServices: route53, route53resolver, cloudfront, directconnect, transitgateway, vpn, waf, shield\\\\\\\\nTasks: configure, troubleshoot, debug, deploy, secure, migrate\\\\\\\\nPersona: developer, network-engineer, devops, security-engineer\\\\\\\\nWorkload: networking, security\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"aws-networking\\\"}]}}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:45.084000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "19bc8d9a-9ae4-4d8d-95a7-a26e21716bce", + "executionId": "54324125-9c9d-4e65-bae0-4d82831992b7", + "recordId": "b60c6d66-217c-496f-be8b-2462cb7ce8f5", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.6, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:45.169000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "19bc8d9a-9ae4-4d8d-95a7-a26e21716bce", + "executionId": "54324125-9c9d-4e65-bae0-4d82831992b7", + "recordId": "091a90cf-9906-4219-b39d-b273d9f86df7", + "content": "{\"id\": \"091a90cf-9906-4219-b39d-b273d9f86df7\", \"role\": \"assistant\", \"content\": [{\"text\": \"Confirmed what I expected \\u2014 here's the breakdown.\\n\\n## What goes wrong\\n\\n**1. The supernet doesn't actually override VPC-local routing (good news/bad news)**\\n\\nIn Cloud WAN's route evaluation order, the most specific route always wins first, and VPC-propagated routes within the segment take priority over dynamic (BGP/Direct Connect) routes when prefix lengths tie. So traffic from 10.1.0.0/16 to 10.2.0.0/16 (VPC-to-VPC, both attached to the same core network segment) still resolves locally \\u2014 it won't get hijacked by the 10.0.0.0/8 route *as long as the VPC CIDRs are more specific* (they are: /16 beats /8).\\n\\n**2. The real problems show up for addresses *inside* 10.0.0.0/8 that aren't covered by a VPC**\\n\\n- **Traffic black-holing / misrouting:** Any destination inside 10.0.0.0/8 that isn't one of your actual VPC CIDRs (e.g., 10.3.5.10, which doesn't belong to any VPC) matches the /8 and gets sent toward on-premises via Direct Connect \\u2014 even if that address was meant to be \\\"free\\\" address space for future VPCs, or is simply mistyped/unassigned. This is usually fine if that's the intent, but it becomes a problem the moment you add a new VPC (say 10.3.0.0/16) \\u2014 its propagated /16 will correctly win over the /8, but only once it's actually attached and propagating. Until then, traffic to that range silently vanishes toward on-premises.\\n- **Asymmetric routing:** On-premises, by advertising a coarse /8, is telling its own routers \\\"all of 10.0.0.0/8 is reachable via Direct Connect.\\\" If the customer's on-prem network *also* has other paths (e.g., a VPN back to AWS, or another DX circuit) with more specific routes for 10.1.0.0/16 and 10.2.0.0/16, inbound and outbound paths can diverge \\u2014 return traffic takes a different path than the outbound request took. This is the classic asymmetric-routing failure mode Direct Connect docs call out explicitly.\\n- **Blast radius / troubleshooting pain:** A single /8 summary hides which specific subnet is actually failing. If on-prem has a routing or firewall problem for one piece of 10.0.0.0/8, it's hard to tell from the AWS side which specific range is affected, since everything funnels through one coarse route.\\n- **Future overlap risk:** If the customer later spins up a new VPC with a CIDR inside 10.0.0.0/8 that happens to literally duplicate an on-prem subnet (not just be a subset of the advertised supernet, but an actual identical range), you get a true conflict \\u2014 AWS will always prefer the local VPC route, so on-prem hosts in that exact range become permanently unreachable from the VPC side. That's the dangerous failure mode worth protecting against proactively.\\n\\n## How to fix it\\n\\n1. **Use allowed prefixes on the Direct Connect Gateway (or equivalent prefix filtering into Cloud WAN) to prevent the summary from masking anything unexpected.** Rather than accepting whatever the customer advertises, control what Cloud WAN/DX Gateway accepts inbound and what gets advertised outbound \\u2014 don't let a bare /8 flow in unchallenged.\\n2. **Have the customer de-aggregate the advertisement** \\u2014 advertise only the actual on-premises subnets they use (e.g., 10.0.0.0/16, 10.10.0.0/16, etc.), not a blanket supernet that happens to include your VPC space. This is the cleanest fix and removes the ambiguity entirely.\\n3. **If they must keep summarizing, carve out non-overlapping VPC CIDR blocks.** Keep your VPCs out of whatever supernet range on-prem claims to own (e.g., VPCs use 10.100.0.0/16+ while on-prem owns 10.0.0.0\\u201310.99.255.255 under the /8 summary). This avoids ambiguity even though AWS's route priority would protect you today.\\n4. **Apply AWS Cloud WAN's new Routing Policy feature** (GA as of November 2025) \\u2014 it adds route filtering/summarization controls, BGP attribute tuning, and visibility into route tables specifically to prevent this kind of over-broad reachability and asymmetric/sub-optimal path issues across Cloud WAN segments. This is the most direct, purpose-built fix if you're already on Cloud WAN.\\n5. **Fix asymmetric routing with BGP attributes (AS_PATH / local preference / MED) or more specific VPN routes**, so inbound and outbound paths for the same traffic class take the same link, per Direct Connect's standard asymmetric-routing guidance.\\n\\nThe short version: your VPC traffic is probably safe today because longest-prefix-match protects it, but the real risk is unassigned/future address space inside that /8 silently routing to on-premises, plus asymmetric paths from the overly broad advertisement. Tightening the allowed-prefix list and asking the customer to de-aggregate are the most durable fixes.\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:45.256000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "19bc8d9a-9ae4-4d8d-95a7-a26e21716bce", + "executionId": "54324125-9c9d-4e65-bae0-4d82831992b7", + "recordId": "cf6dfd46-6746-40d7-bcec-645dd146cb20", + "content": "{\"id\": \"bb88c979-eb18-49f7-8eb4-18d3cb3d5d3d\", \"role\": \"assistant\", \"content\": [{\"type\": \"text\", \"text\": \"Confirmed what I expected \\u2014 here's the breakdown.\\n\\n## What goes wrong\\n\\n**1. The supernet doesn't actually override VPC-local routing (good news/bad news)**\\n\\nIn Cloud WAN's route evaluation order, the most specific route always wins first, and VPC-propagated routes within the segment take priority over dynamic (BGP/Direct Connect) routes when prefix lengths tie. So traffic from 10.1.0.0/16 to 10.2.0.0/16 (VPC-to-VPC, both attached to the same core network segment) still resolves locally \\u2014 it won't get hijacked by the 10.0.0.0/8 route *as long as the VPC CIDRs are more specific* (they are: /16 beats /8).\\n\\n**2. The real problems show up for addresses *inside* 10.0.0.0/8 that aren't covered by a VPC**\\n\\n- **Traffic black-holing / misrouting:** Any destination inside 10.0.0.0/8 that isn't one of your actual VPC CIDRs (e.g., 10.3.5.10, which doesn't belong to any VPC) matches the /8 and gets sent toward on-premises via Direct Connect \\u2014 even if that address was meant to be \\\"free\\\" address space for future VPCs, or is simply mistyped/unassigned. This is usually fine if that's the intent, but it becomes a problem the moment you add a new VPC (say 10.3.0.0/16) \\u2014 its propagated /16 will correctly win over the /8, but only once it's actually attached and propagating. Until then, traffic to that range silently vanishes toward on-premises.\\n- **Asymmetric routing:** On-premises, by advertising a coarse /8, is telling its own routers \\\"all of 10.0.0.0/8 is reachable via Direct Connect.\\\" If the customer's on-prem network *also* has other paths (e.g., a VPN back to AWS, or another DX circuit) with more specific routes for 10.1.0.0/16 and 10.2.0.0/16, inbound and outbound paths can diverge \\u2014 return traffic takes a different path than the outbound request took. This is the classic asymmetric-routing failure mode Direct Connect docs call out explicitly.\\n- **Blast radius / troubleshooting pain:** A single /8 summary hides which specific subnet is actually failing. If on-prem has a routing or firewall problem for one piece of 10.0.0.0/8, it's hard to tell from the AWS side which specific range is affected, since everything funnels through one coarse route.\\n- **Future overlap risk:** If the customer later spins up a new VPC with a CIDR inside 10.0.0.0/8 that happens to literally duplicate an on-prem subnet (not just be a subset of the advertised supernet, but an actual identical range), you get a true conflict \\u2014 AWS will always prefer the local VPC route, so on-prem hosts in that exact range become permanently unreachable from the VPC side. That's the dangerous failure mode worth protecting against proactively.\\n\\n## How to fix it\\n\\n1. **Use allowed prefixes on the Direct Connect Gateway (or equivalent prefix filtering into Cloud WAN) to prevent the summary from masking anything unexpected.** Rather than accepting whatever the customer advertises, control what Cloud WAN/DX Gateway accepts inbound and what gets advertised outbound \\u2014 don't let a bare /8 flow in unchallenged.\\n2. **Have the customer de-aggregate the advertisement** \\u2014 advertise only the actual on-premises subnets they use (e.g., 10.0.0.0/16, 10.10.0.0/16, etc.), not a blanket supernet that happens to include your VPC space. This is the cleanest fix and removes the ambiguity entirely.\\n3. **If they must keep summarizing, carve out non-overlapping VPC CIDR blocks.** Keep your VPCs out of whatever supernet range on-prem claims to own (e.g., VPCs use 10.100.0.0/16+ while on-prem owns 10.0.0.0\\u201310.99.255.255 under the /8 summary). This avoids ambiguity even though AWS's route priority would protect you today.\\n4. **Apply AWS Cloud WAN's new Routing Policy feature** (GA as of November 2025) \\u2014 it adds route filtering/summarization controls, BGP attribute tuning, and visibility into route tables specifically to prevent this kind of over-broad reachability and asymmetric/sub-optimal path issues across Cloud WAN segments. This is the most direct, purpose-built fix if you're already on Cloud WAN.\\n5. **Fix asymmetric routing with BGP attributes (AS_PATH / local preference / MED) or more specific VPN routes**, so inbound and outbound paths for the same traffic class take the same link, per Direct Connect's standard asymmetric-routing guidance.\\n\\nThe short version: your VPC traffic is probably safe today because longest-prefix-match protects it, but the real risk is unassigned/future address space inside that /8 silently routing to on-premises, plus asymmetric paths from the overly broad advertisement. Tightening the allowed-prefix list and asking the customer to de-aggregate are the most durable fixes.\"}]}", + "createdAt": "2026-10-27T17:05:45.358000-05:00", + "recordType": "final_response" + } +] \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/dx-path-selection-order/with_skill/functional-tests-results.json b/skills/aws-routing/evals/functional/v1/iteration-1/dx-path-selection-order/with_skill/functional-tests-results.json new file mode 100644 index 00000000..a429df7e --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/dx-path-selection-order/with_skill/functional-tests-results.json @@ -0,0 +1,74 @@ +{ + "version": "v1", + "iteration": 1, + "eval_id": "dx-path-selection-order", + "run_type": "with_skill", + "success": true, + "error": null, + "understanding_agent_space_skill": null, + "tests": { + "trigger": { + "result": "passed", + "message": "Skill 'aws-routing' was triggered as expected", + "skill_loads_found": 1 + }, + "expected_output": { + "result": "passed", + "reasoning": "The agent's response matches the expected order exactly: longest prefix match \u2192 local preference \u2192 AS_PATH length \u2192 MED \u2192 ECMP. It correctly identifies MED as a low-priority tiebreaker and explicitly states AWS does not recommend relying on MED. It also correctly lists the LP communities 7224:7100 (Low), 7224:7200 (Medium), 7224:7300 (High) for setting local preference, matching the expected output's mention of these exact community values.", + "evidence": "\"1. Longest prefix match... 2. Local preference \u2014 set via DX LP BGP communities: 7224:7300 (High), 7224:7200 (Medium), 7224:7100 (Low)... 3. AS_PATH length... 4. MED (Multi-Exit Discriminator) \u2014 lower value wins... 5. ECMP... AWS documentation explicitly does not recommend relying on MED for predictable DX egress control.\"", + "confidence": "high" + }, + "assertions": { + "assertion_results": [ + { + "text": "states the order longest prefix match, local preference, AS_PATH, MED, ECMP", + "evaluator": "llm", + "passed": true, + "evidence": "The agent lists: '1. Longest prefix match ... 2. Local preference ... 3. AS_PATH length ... 4. MED ... 5. ECMP'", + "reasoning": "The output explicitly states the order longest prefix match, local preference, AS_PATH, MED, ECMP in that sequence.", + "confidence": "high" + }, + { + "text": "notes MED is a low-priority tiebreaker", + "evaluator": "llm", + "passed": true, + "evidence": "'MED sits second-to-last in the evaluation order \u2014 it's only consulted as a tiebreaker after prefix length, local preference, and AS_PATH length have all come out equal... Because it's so far down the priority list, AWS documentation explicitly does not recommend relying on MED for predictable DX egress control.'", + "reasoning": "The agent clearly frames MED as a low-priority tiebreaker, used only after other attributes are equal, and notes it's not recommended for reliable control.", + "confidence": "high" + }, + { + "text": "mentions LP communities 7224:7100/7200/7300", + "evaluator": "llm", + "passed": true, + "evidence": "'Local preference \u2014 set via DX LP BGP communities: 7224:7300 (High), 7224:7200 (Medium), 7224:7100 (Low)'", + "reasoning": "The output explicitly mentions all three LP communities 7224:7100, 7224:7200, and 7224:7300.", + "confidence": "high" + }, + { + "text": "does not contain 'error'", + "evaluator": "llm", + "passed": true, + "evidence": "No occurrence of the word 'error' found anywhere in the output text.", + "reasoning": "Scanning the full response, the word 'error' does not appear.", + "confidence": "high" + } + ], + "summary": { + "passed": 4, + "failed": 0, + "errored": 0, + "low_confidence": 0, + "total": 4, + "pass_rate": 1.0 + } + } + }, + "metrics": { + "runtime": "12s", + "cost": "$0.10", + "context_window": { + "utilization": "4.9%", + "compaction_count": 0 + } + } +} \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/dx-path-selection-order/with_skill/outputs/journal_records.json b/skills/aws-routing/evals/functional/v1/iteration-1/dx-path-selection-order/with_skill/outputs/journal_records.json new file mode 100644 index 00000000..039173b6 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/dx-path-selection-order/with_skill/outputs/journal_records.json @@ -0,0 +1,138 @@ +[ + { + "agentSpaceId": "afca24ed-acb9-4aa1-a565-3abfab2c554b", + "executionId": "05fe08d0-8f28-41bf-9889-5328d93019ca", + "recordId": "2fb7dcdb-9adc-4d29-ba79-c4b534121f85", + "content": "{\"id\": \"2fb7dcdb-9adc-4d29-ba79-c4b534121f85\", \"role\": \"user\", \"content\": [{\"text\": \"What is the outbound path-selection order for a Direct Connect private VIF, and where does MED fit?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:04.600000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "afca24ed-acb9-4aa1-a565-3abfab2c554b", + "executionId": "05fe08d0-8f28-41bf-9889-5328d93019ca", + "recordId": "f6178719-a936-4306-8deb-bb6452a5369f", + "content": "{\"id\": \"7f934208-d94c-42ab-ac5b-0daafb31a825\", \"role\": \"assistant\", \"content\": [{\"type\": \"chat_title\", \"title\": \"Direct Connect Private VIF Path Selection Order\"}]}", + "createdAt": "2026-10-27T17:05:05.202000-05:00", + "recordType": "chat_title" + }, + { + "agentSpaceId": "afca24ed-acb9-4aa1-a565-3abfab2c554b", + "executionId": "05fe08d0-8f28-41bf-9889-5328d93019ca", + "recordId": "d685b6ed-020d-46fe-b140-50ac4a21eabd", + "content": "{\"id\": \"c1c4374b-c4ea-4c60-b31d-9886cbe0fe28\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This is about Direct Connect path selection order and MED, which falls under the aws-routing skill, so I'll check that skill for guidance.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:08.570000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "afca24ed-acb9-4aa1-a565-3abfab2c554b", + "executionId": "05fe08d0-8f28-41bf-9889-5328d93019ca", + "recordId": "149eab54-6864-4edf-a144-785397d0824a", + "content": "{\"id\": \"149eab54-6864-4edf-a144-785397d0824a\", \"role\": \"assistant\", \"content\": [{\"text\": \"Let me pull up the routing skill for Direct Connect path selection.\", \"type\": \"text\"}, {\"id\": \"tooluse_dbiw8pFnZ1ulNaWgnZXXas\", \"tool_name\": \"skill_read\", \"input\": {\"skill_id\": \"aws-routing\"}, \"type\": \"tool_use\", \"display_intent\": \"Read the AWS routing skill for Direct Connect path-selection guidance.\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:08.587000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "afca24ed-acb9-4aa1-a565-3abfab2c554b", + "executionId": "05fe08d0-8f28-41bf-9889-5328d93019ca", + "recordId": "8dc3a937-46ae-4c7c-aae7-f95117c5adf8", + "content": "{\"id\": \"3791b756-8e03-405f-9546-6b12c6bc1ca5\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"4\"}]}", + "createdAt": "2026-10-27T17:05:08.641000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "afca24ed-acb9-4aa1-a565-3abfab2c554b", + "executionId": "05fe08d0-8f28-41bf-9889-5328d93019ca", + "recordId": "6d26917f-448a-4f93-a37c-e333a8c861df", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 3.8, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:08.664000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "afca24ed-acb9-4aa1-a565-3abfab2c554b", + "executionId": "05fe08d0-8f28-41bf-9889-5328d93019ca", + "recordId": "00d4ec01-153e-4cb9-bba2-40faf44fa40c", + "content": "{\"id\": \"0648bf7f-dd3e-425c-bf65-4dd3998c4458\", \"role\": \"assistant\", \"block_type\": \"load_skill\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_dbiw8pFnZ1ulNaWgnZXXas\", \"status\": \"success\", \"content\": [{\"text\": \"Done\"}]}], \"label\": \"Read the AWS routing skill for Direct Connect path-selection guidance.\"}", + "createdAt": "2026-10-27T17:05:08.757000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "afca24ed-acb9-4aa1-a565-3abfab2c554b", + "executionId": "05fe08d0-8f28-41bf-9889-5328d93019ca", + "recordId": "37f4aa88-bb03-4801-978f-a18fef56525b", + "content": "{\"id\": \"edce9c5f-3501-46d9-9a69-bc7aa2f13ea2\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"I should check the dx-routing-patterns.md reference for more details specifically on MED.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:12.211000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "afca24ed-acb9-4aa1-a565-3abfab2c554b", + "executionId": "05fe08d0-8f28-41bf-9889-5328d93019ca", + "recordId": "bbbe2707-f064-4898-905e-8a68e7d38fb7", + "content": "{\"id\": \"bbbe2707-f064-4898-905e-8a68e7d38fb7\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_dbiw8pFnZ1ulNaWgnZXXas\", \"content\": \"[{'text': '{\\\"content\\\": \\\"---\\\\\\\\r\\\\\\\\nname: aws-routing\\\\\\\\r\\\\\\\\ndescription: \\\\'Use this skill when the user wants to know which path AWS will pick for their traffic, why traffic took an unexpected route, or how to make AWS prefer a path. Covers Cloud WAN CNE route evaluation, Direct Connect Gateway local vs remote region preference, Transit Gateway route tables, ECMP and peering, VPC longest-prefix-match, BGP traffic engineering (LP communities 7224:7100/7200/7300, AS-path prepending, MED), DX + VPN redundancy and failover, and asymmetric routing. Use it even when the user only describes a symptom, such as \\\\\\\\\\\"traffic is leaving the region\\\\\\\\\\\", \\\\\\\\\\\"not using my local Direct Connect\\\\\\\\\\\", \\\\\\\\\\\"routing to the wrong VIF/location\\\\\\\\\\\", or \\\\\\\\\\\"path is asymmetric\\\\\\\\\\\". Read-only: analysis, recommendations, and describe/get/list validation commands grounded in public AWS docs.\\\\'\\\\\\\\r\\\\\\\\nmetadata:\\\\\\\\r\\\\\\\\n version: \\\\\\\\\\\"1.2.5\\\\\\\\\\\"\\\\\\\\r\\\\\\\\n author: midakin\\\\\\\\r\\\\\\\\n aws-devops-agent-skills.agent-types: \\\\\\\\\\\"CHAT, INCIDENT_RCA\\\\\\\\\\\"\\\\\\\\r\\\\\\\\n---\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Overview\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nThis skill provides routing-first analysis for AWS networking. It reasons about the full traffic\\\\\\\\r\\\\\\\\npath, applies the documented route-evaluation logic for each service, and grounds recommendations\\\\\\\\r\\\\\\\\nin public AWS documentation. It is **read-only**: it produces analysis, recommendations, and\\\\\\\\r\\\\\\\\nread-only validation commands (`describe-*`, `get-*`, `list-*`) - never mutating actions.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Verification & Anti-Hallucination Directives (Mandatory)\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- Do not generate any fact, citation, or source that is not fully verifiable. If you lack a verified\\\\\\\\r\\\\\\\\n source, state \\\\\\\\\\\"I cannot verify this\\\\\\\\\\\" rather than inventing details.\\\\\\\\r\\\\\\\\n- State assumptions explicitly before answering.\\\\\\\\r\\\\\\\\n- Cross-check each claim against public AWS documentation or the `references/` files. Omit claims you\\\\\\\\r\\\\\\\\n cannot confirm.\\\\\\\\r\\\\\\\\n- Only cite AWS documentation URLs you have actually retrieved or confirmed. Do not fabricate links.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## When to use this skill\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nActivate when the user asks about:\\\\\\\\r\\\\\\\\n- Cloud WAN route evaluation / Core Network Edge (CNE) path selection\\\\\\\\r\\\\\\\\n- Direct Connect Gateway path selection, local-region preference, or LP communities\\\\\\\\r\\\\\\\\n- Transit Gateway route evaluation, ECMP, or peering routing\\\\\\\\r\\\\\\\\n- VPC route tables and longest-prefix-match behavior\\\\\\\\r\\\\\\\\n- BGP traffic engineering (communities, AS-path prepending, MED)\\\\\\\\r\\\\\\\\n- DX + VPN redundancy, active/active vs active/passive, or failover design\\\\\\\\r\\\\\\\\n- Why traffic is taking an unexpected path\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## What this skill does NOT do\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nThis skill is **read-only and advisory**. It will not:\\\\\\\\r\\\\\\\\n- **Design or apply infrastructure changes.** It does not create, modify, or delete AWS resources,\\\\\\\\r\\\\\\\\n and it does not run mutating CLI/API calls (`create-*`, `modify-*`, `associate-*`, `delete-*`,\\\\\\\\r\\\\\\\\n `put-*`, `update-*`) or write Infrastructure-as-Code to be deployed. Any change it describes is a\\\\\\\\r\\\\\\\\n *recommendation* for you to review and implement yourself.\\\\\\\\r\\\\\\\\n- **Configure BGP or push routing policy.** It explains which communities, AS-path, or MED settings\\\\\\\\r\\\\\\\\n to use, but it does not apply them to VIFs, route tables, or Cloud WAN policy documents.\\\\\\\\r\\\\\\\\n- **Execute anything against a live account beyond read-only validation.** Commands it provides are\\\\\\\\r\\\\\\\\n limited to `describe-*`, `get-*`, and `list-*` for confirming state.\\\\\\\\r\\\\\\\\n- **Guarantee production outcomes.** It recommends non-production testing and blue/green rollout;\\\\\\\\r\\\\\\\\n it does not perform the cutover or validate the result for you.\\\\\\\\r\\\\\\\\n- **Provide compliance, security, or contractual sign-off**, or make claims it cannot ground in\\\\\\\\r\\\\\\\\n public AWS documentation or the `references/` files.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nFor designing and applying changes, hand the recommendation to a change-capable workflow (with\\\\\\\\r\\\\\\\\nappropriate review, approvals, and testing).\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## How to analyze a routing problem\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nWork through this checklist in order; each step depends on the one before it.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- [ ] **1. Establish the path.** Identify source, destination, and every routing construct in\\\\\\\\r\\\\\\\\n between (VPC route table \\\\\\\\u2192 TGW/Cloud WAN CNE \\\\\\\\u2192 DXGW \\\\\\\\u2192 VIF \\\\\\\\u2192 on-prem, or the relevant subset).\\\\\\\\r\\\\\\\\n- [ ] **2. Name the route table.** Be explicit about which table makes each decision (VPC RT, TGW RT,\\\\\\\\r\\\\\\\\n CNE RT, DXGW internal selection).\\\\\\\\r\\\\\\\\n- [ ] **3. Apply the evaluation order for that construct** (see below).\\\\\\\\r\\\\\\\\n- [ ] **4. Identify the deciding attribute** (longest prefix, local preference, AS-path, MED, source\\\\\\\\r\\\\\\\\n type).\\\\\\\\r\\\\\\\\n- [ ] **5. Flag non-determinism** (e.g., ECMP or \\\\\\\\\\\"deterministically random\\\\\\\\\\\" tiebreakers) and\\\\\\\\r\\\\\\\\n recommend a deterministic alternative.\\\\\\\\r\\\\\\\\n- [ ] **6. Cite the AWS documentation** that supports the behavior.\\\\\\\\r\\\\\\\\n- [ ] **7. Provide read-only validation commands** the user can run to confirm.\\\\\\\\r\\\\\\\\n- [ ] **8. Self-check before answering.** Re-read the answer against steps 1-7: every route table is\\\\\\\\r\\\\\\\\n named, the evaluation order matches the construct, each cited URL was actually retrieved, and every\\\\\\\\r\\\\\\\\n command is read-only. Fix any gap before presenting.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Key evaluation rules\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Cloud WAN (per CNE)\\\\\\\\r\\\\\\\\nLongest prefix match \\\\\\\\u2192 static \\\\\\\\u2192 VPC-propagated (same region) \\\\\\\\u2192 unequal AS-path/MED (shortest wins) \\\\\\\\u2192\\\\\\\\r\\\\\\\\nequal AS-path & MED source preference: DXGW-propagated \\\\\\\\u2192 Cloud WAN Connect \\\\\\\\u2192 S2S VPN \\\\\\\\u2192 other\\\\\\\\r\\\\\\\\n(TGW peering, remote CNEs; identical from 2+ sources = deterministically random).\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Direct Connect (private/transit VIF, outbound AWS\\\\\\\\u2192on-prem)\\\\\\\\r\\\\\\\\nLongest prefix match \\\\\\\\u2192 local preference \\\\\\\\u2192 AS_PATH length \\\\\\\\u2192 MED \\\\\\\\u2192 ECMP (equal AS_PATH and BGP\\\\\\\\r\\\\\\\\nattributes). LP communities `7224:7300/7200/7100` (High/Medium/Low) set local preference and are\\\\\\\\r\\\\\\\\nevaluated before AS-path. DXGW prefers the local associated-region DX by default.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Direction & tooling guidance\\\\\\\\r\\\\\\\\n- **AWS \\\\\\\\u2192 on-prem (egress):** control with DX LP communities.\\\\\\\\r\\\\\\\\n- **On-prem \\\\\\\\u2192 AWS (ingress):** control with the customer router\\\\'s local-pref/weight/MED.\\\\\\\\r\\\\\\\\n- **AS-path prepending:** within a region only - unreliable across regions because DXGW local-region\\\\\\\\r\\\\\\\\n LP overrides it. **LP communities:** work within-region and cross-region.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Reference material\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nDetailed mechanics, patterns, and caveats are in `references/`. Load only the file that matches the\\\\\\\\r\\\\\\\\nconstruct deciding the path:\\\\\\\\r\\\\\\\\n- [Cloud WAN + DX routing patterns](references/cloudwan-dx-routing-patterns.md) - load when the path\\\\\\\\r\\\\\\\\n crosses a Cloud WAN CNE: CNE route evaluation, DXGW path selection into Cloud WAN, community\\\\\\\\r\\\\\\\\n traffic-engineering patterns, regional inspection, multi-region egress.\\\\\\\\r\\\\\\\\n- [Direct Connect routing patterns](references/dx-routing-patterns.md) - load when the question is\\\\\\\\r\\\\\\\\n about a DX VIF or DXGW without Cloud WAN: BGP mechanics, BFD, MED, ASN ranges, VIF considerations,\\\\\\\\r\\\\\\\\n CloudHub, when VIFs can talk, active/active vs active/passive.\\\\\\\\r\\\\\\\\n- [Transit Gateway routing patterns](references/tgw-routing-patterns.md) - load when a TGW route\\\\\\\\r\\\\\\\\n table makes the decision: TGW route evaluation, ECMP limits, DXGW-to-TGW behavior, DX/VPN caveats,\\\\\\\\r\\\\\\\\n failover, peering.\\\\\\\\r\\\\\\\\n- [VPN + DX redundancy patterns](references/vpn-dx-redundancy-patterns.md) - load when VPN backs up\\\\\\\\r\\\\\\\\n or runs alongside DX: route preference between them, ECMP limits, asymmetric return paths.\\\\\\\\r\\\\\\\\n- [Networking strategic questions](references/networking-strategic-questions.md) - load when key\\\\\\\\r\\\\\\\\n facts are missing (regions, attachments, ASNs, communities) and you need discovery questions to ask\\\\\\\\r\\\\\\\\n before analyzing.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Output expectations\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- Explain the route-evaluation logic step by step.\\\\\\\\r\\\\\\\\n- Name the specific route table involved.\\\\\\\\r\\\\\\\\n- Give the end-to-end traffic path.\\\\\\\\r\\\\\\\\n- Compare options with tradeoffs (determinism, complexity, failover) when multiple solutions exist.\\\\\\\\r\\\\\\\\n- Include read-only AWS CLI validation commands.\\\\\\\\r\\\\\\\\n- Recommend testing in a non-production environment and blue/green for migrations.\\\\\\\\r\\\\\\\\n\\\", \\\"resource_manifest\\\": [\\\"SKILL.md\\\", \\\"references/cloudwan-dx-routing-patterns.md\\\", \\\"references/dx-routing-patterns.md\\\", \\\"references/networking-strategic-questions.md\\\", \\\"references/tgw-routing-patterns.md\\\", \\\"references/vpn-dx-redundancy-patterns.md\\\"]}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:12.211000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "afca24ed-acb9-4aa1-a565-3abfab2c554b", + "executionId": "05fe08d0-8f28-41bf-9889-5328d93019ca", + "recordId": "a77dcba2-166b-4e52-b265-43d2bbf847be", + "content": "{\"id\": \"a77dcba2-166b-4e52-b265-43d2bbf847be\", \"role\": \"assistant\", \"content\": [{\"id\": \"tooluse_XY66zTvfkIK1b7wo33wwZC\", \"tool_name\": \"read_skill_resource\", \"input\": {\"skill_id\": \"aws-routing\", \"path\": \"references/dx-routing-patterns.md\"}, \"type\": \"tool_use\", \"display_intent\": \"Read Direct Connect routing patterns reference for MED details.\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:12.294000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "afca24ed-acb9-4aa1-a565-3abfab2c554b", + "executionId": "05fe08d0-8f28-41bf-9889-5328d93019ca", + "recordId": "26914ed5-87ec-4c76-984b-f10217b96fdf", + "content": "{\"id\": \"5f1d966e-cd3e-41a0-b471-e6e9009856d1\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"364\"}]}", + "createdAt": "2026-10-27T17:05:12.296000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "afca24ed-acb9-4aa1-a565-3abfab2c554b", + "executionId": "05fe08d0-8f28-41bf-9889-5328d93019ca", + "recordId": "487ec950-6a47-4bba-9f33-92efce3f0f4b", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.2, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:12.371000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "afca24ed-acb9-4aa1-a565-3abfab2c554b", + "executionId": "05fe08d0-8f28-41bf-9889-5328d93019ca", + "recordId": "bfe26463-3344-4035-9939-4bd936d35bae", + "content": "{\"id\": \"ad4ef145-2dbf-44e7-8ba3-363e5096a286\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_XY66zTvfkIK1b7wo33wwZC\", \"status\": \"success\", \"content\": [{\"text\": \"Done\"}]}], \"label\": \"Read Direct Connect routing patterns reference for MED details.\"}", + "createdAt": "2026-10-27T17:05:12.660000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "afca24ed-acb9-4aa1-a565-3abfab2c554b", + "executionId": "05fe08d0-8f28-41bf-9889-5328d93019ca", + "recordId": "8b569e21-e146-49a0-afd7-c0367c94ffd9", + "content": "{\"id\": \"8b569e21-e146-49a0-afd7-c0367c94ffd9\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_XY66zTvfkIK1b7wo33wwZC\", \"content\": \"[{'text': '\\\\r\\\\n\\\\r\\\\n# Direct Connect Routing Patterns & Considerations\\\\r\\\\n\\\\r\\\\nThis document captures key BGP routing mechanics, path selection behavior, VIF-specific considerations, and operational caveats for AWS Direct Connect architectures. Use this knowledge when advising on DX design (active/active, active/passive), BGP community behavior, BFD configuration, and public/private/transit VIF routing.\\\\r\\\\n\\\\r\\\\n## Reference Documentation\\\\r\\\\n\\\\r\\\\n- [Direct Connect Routing Policies and BGP Communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html)\\\\r\\\\n- [Direct Connect Path Selection (routing policies for private/transit VIFs)](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html)\\\\r\\\\n- [Direct Connect Virtual Interfaces](https://docs.aws.amazon.com/directconnect/latest/UserGuide/WorkingWithVirtualInterfaces.html)\\\\r\\\\n\\\\r\\\\n## BGP & Routing\\\\r\\\\n\\\\r\\\\n### BGP Route Visibility (ListVirtualInterfaceRoutes)\\\\r\\\\n\\\\r\\\\nAs of July 2026, DX supports **BGP route visibility** on virtual interfaces via the `ListVirtualInterfaceRoutes` API and the DX console:\\\\r\\\\n\\\\r\\\\n- **Accepted routes:** Routes AWS received from your on-prem router\\\\r\\\\n- **Advertised routes:** Routes AWS is sending to your on-prem router\\\\r\\\\n- Each route shows: prefix, address family, AS-path, community values, installation timestamp\\\\r\\\\n- Can filter by: prefix, AS-path, community, or address family\\\\r\\\\n- Available on: private, transit, and public VIFs\\\\r\\\\n- Available in: all AWS commercial regions\\\\r\\\\n\\\\r\\\\n**Use for troubleshooting:**\\\\r\\\\n- Verify route propagation between on-prem and AWS\\\\r\\\\n- Validate BGP policy configurations (communities, AS-path prepending)\\\\r\\\\n- Diagnose unexpected traffic patterns or asymmetric routing\\\\r\\\\n- Confirm which routes DXGW is accepting vs. advertising\\\\r\\\\n- Validate LP community tags are being received correctly\\\\r\\\\n\\\\r\\\\n**Reference:** [AWS Direct Connect BGP Route Visibility announcement](https://aws.amazon.com/about-aws/whats-new/2026/07/aws-direct-connect-bgp-visibility/)\\\\r\\\\n\\\\r\\\\n### CloudWatch BGP Metrics for DX Virtual Interfaces\\\\r\\\\n\\\\r\\\\nAs of March 2026, DX publishes **3 BGP metrics** to CloudWatch for all VIF types (private, transit, public):\\\\r\\\\n\\\\r\\\\n| Metric | What It Reports |\\\\r\\\\n|---|---|\\\\r\\\\n| `VirtualInterfaceBgpStatus` | BGP session state: 1 = up, 0 = down |\\\\r\\\\n| `VirtualInterfaceBgpPrefixesAccepted` | Number of prefixes AWS accepted from on-prem |\\\\r\\\\n| `VirtualInterfaceBgpPrefixesAdvertised` | Number of prefixes AWS advertises to on-prem |\\\\r\\\\n\\\\r\\\\n**Key details:**\\\\r\\\\n- Updated every 5 minutes (BGP flaps between intervals may not be captured)\\\\r\\\\n- Published at no extra cost in the region where the DX location is associated\\\\r\\\\n- Available via CloudWatch MCP server (`get_metric_data`) for programmatic access\\\\r\\\\n\\\\r\\\\n**Use cases:**\\\\r\\\\n- **BGP session monitoring:** Alarm on `VirtualInterfaceBgpStatus < 1` to detect session failures without API polling\\\\r\\\\n- **Prefix limit protection:** Alarm when `PrefixesAccepted` approaches VIF quota to prevent session idle state\\\\r\\\\n- **Silent route withdrawal detection:** BGP session stays UP but prefix count drops \\u2192 indicates routes withdrawn without session going down\\\\r\\\\n- **DR/failover validation:** Compare prefix counts between primary and standby VIFs \\u2014 a standby with fewer prefixes indicates a broken failover path\\\\r\\\\n- **Route change correlation:** Sudden drop in accepted/advertised prefixes correlates with traffic impact events\\\\r\\\\n\\\\r\\\\n**Alarm examples:**\\\\r\\\\n- `VirtualInterfaceBgpStatus` \\u2014 Statistic: Minimum, Period: 5min, Threshold: Lower than 1\\\\r\\\\n- `VirtualInterfaceBgpPrefixesAccepted` \\u2014 Statistic: Maximum, Period: 5min, Threshold: Greater/Equal to (near quota limit)\\\\r\\\\n- `VirtualInterfaceBgpPrefixesAccepted` \\u2014 Statistic: Maximum, Period: 5min, Threshold: Lower/Equal to (below expected minimum)\\\\r\\\\n\\\\r\\\\n**Reference:** [CloudWatch Metrics for DX VIF BGP Health and Prefix Count](https://aws.amazon.com/blogs/networking-and-content-delivery/introducing-cloudwatch-metrics-for-aws-direct-connect-virtual-interface-bgp-health-and-prefix-count/)\\\\r\\\\n\\\\r\\\\n### General Routing Behavior\\\\r\\\\n\\\\r\\\\n- When designing active/active or active/passive DX (within or across DX locations), apply the documented **Private/Transit VIF outbound path-selection order**: longest prefix match \\u2192 local preference \\u2192 AS_PATH length \\u2192 MED \\u2192 ECMP (equal AS_PATH length and BGP attributes). See [Direct Connect routing policies and BGP communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html).\\\\r\\\\n- Consider recommendations for **migrating from DX + VGW/Private VIF to DX + TGW/Transit VIF** when customers are modernizing their architecture\\\\r\\\\n\\\\r\\\\n### MED Support\\\\r\\\\n\\\\r\\\\n- DX **supports Multi-Exit Discriminator (MED) on Private and Transit VIFs** \\u2014 MED can influence the outbound (AWS \\u2192 on-prem) routing path, but only as a **tiebreaker after prefix length, local preference, and AS_PATH are equal**. AWS explicitly **does not recommend relying on MED** given its lower priority in the evaluation order.\\\\r\\\\n- For **Public VIFs**, the documented outbound routing policy uses **AS_PATH and longest prefix match** \\u2014 MED is not listed among the supported public VIF path-selection attributes.\\\\r\\\\n- **Outbound (AWS \\u2192 on-prem) path selection order for Private/Transit VIF:** longest prefix match \\u2192 local preference \\u2192 AS_PATH length \\u2192 MED \\u2192 ECMP (equal AS_PATH length and BGP attributes).\\\\r\\\\n\\\\r\\\\n**Reference:** [Direct Connect routing policies and BGP communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html)\\\\r\\\\n\\\\r\\\\n### BFD (Bidirectional Forwarding Detection)\\\\r\\\\n\\\\r\\\\n**\\u26a0\\ufe0f Always verify the Associated Region for each DX location** \\u2014 some DX locations can associate to different AWS regions than expected based on their physical geography. For example:\\\\r\\\\n- Equinix LD5 (Slough, UK) \\u2192 **eu-west-1** (Ireland)\\\\r\\\\n- Telehouse (London Docklands, UK) \\u2192 **eu-west-2** (London)\\\\r\\\\n- Equinix MA3 (Manchester, UK) \\u2192 **eu-west-2** (London)\\\\r\\\\n\\\\r\\\\nThe associated region determines which TGW/DXGW considers the DX \\\"local\\\" for LP preference and routing behavior. Verify a location\\\\'s associated region with the `DescribeLocations` API/CLI (`aws directconnect describe-locations`) or the official AWS Direct Connect locations reference: https://aws.amazon.com/directconnect/locations/\\\\r\\\\n\\\\r\\\\n### BFD (Bidirectional Forwarding Detection)\\\\r\\\\n\\\\r\\\\n- **Asynchronous BFD is automatically enabled** on the AWS side for all DX virtual interfaces\\\\r\\\\n- Customer must configure asynchronous BFD on their router (refer to vendor documentation)\\\\r\\\\n- BFD provides fast link failure detection \\u2014 ensures BGP neighbor relationship is torn down quickly for faster failover\\\\r\\\\n- **Do NOT configure both BFD for BGP and BGP Graceful Restart simultaneously** \\u2014 this results in sub-optimal routing performance (conflicting goals: BFD wants fast teardown, Graceful Restart wants to preserve routes during failure)\\\\r\\\\n\\\\r\\\\n### ASN Support\\\\r\\\\n\\\\r\\\\n| Context | Supported ASN Range |\\\\r\\\\n|---|---|\\\\r\\\\n| AWS side via DX-GW | Full 32-bit: 4,200,000,000 to 4,294,967,294 (private) |\\\\r\\\\n| AWS side via VGW | 32-bit: 1 to 2,147,483,647 only |\\\\r\\\\n| Customer side (Private/Public/Transit VIF) | 32-bit: 1 to 2,147,483,647 only |\\\\r\\\\n\\\\r\\\\n- Customers can request a Private/Transit VIF ASN from the 4,200,000,000\\u20134,294,967,294 range via a support case (non-standard, requires internal approval)\\\\r\\\\n\\\\r\\\\n### SiteLink & BGP Communities\\\\r\\\\n\\\\r\\\\n- SiteLink **maintains BGP community tags** \\u2014 tags are passed through from one connected peer to another\\\\r\\\\n- This means communities set on one DX location will be visible at another DX location connected via SiteLink\\\\r\\\\n\\\\r\\\\n## VIF-Specific Considerations\\\\r\\\\n\\\\r\\\\n### Public VIF\\\\r\\\\n\\\\r\\\\n- **Reverse-path forwarding (RPF) strict mode is enforced** \\u2014 validates that the source of traffic originated from the customer\\\\'s advertised prefix (prevents address spoofing)\\\\r\\\\n- **Do NOT use EC2 EIPs for Public VIF BGP peering** \\u2014 can cause routing issues. Customers should use:\\\\r\\\\n - Public IPs they own, OR\\\\r\\\\n - IPs assigned by their provider/AWS support\\\\r\\\\n- **EC2 BYOIP CIDRs are NOT in ip-ranges.json** (not part of Amazon address space), but they ARE advertised over DX Public VIF to customers\\\\r\\\\n- **S3-only Public VIF**: BGP prefix lists are available (GitHub repository) for filtering S3 traffic over DX Public VIF \\u2014 allows routing only S3 traffic through DX while sending other AWS traffic over internet to reduce costs\\\\r\\\\n\\\\r\\\\n### Private / Transit VIF\\\\r\\\\n\\\\r\\\\n- For Private/Transit VIFs, customers can use any IPv4 address (if not using AWS auto-generated IPs), but must ensure those IPs **do not conflict** with on-prem and/or VPC networks\\\\r\\\\n- When attaching VPCs with **AWS-assigned IPv6** to a TGW, these are discontinuous by default and **cannot be aggregated** for Direct Connect \\u2014 leads to DX prefix quota limitations\\\\r\\\\n - **Workarounds:** Use Amazon-provided contiguous IPv6 CIDR blocks, or EC2 BYOIP\\\\r\\\\n\\\\r\\\\n### VPC-to-VPC Routing via DXGW (Regression)\\\\r\\\\n\\\\r\\\\n- As of November 2021, **VPC-to-VPC routing** connected to the same DXGW via VGW/TGW which are also connected over Private/Transit VIF **is possible due to a regression**\\\\r\\\\n- This was previously blocked \\u2014 be aware of this change\\\\r\\\\n- Workarounds exist to block this unintended traffic flow if isolation is required\\\\r\\\\n\\\\r\\\\n### DX VIFs and VGW \\u2014 CloudHub Behavior\\\\r\\\\n\\\\r\\\\n- **VPN CloudHub is a Site-to-Site VPN feature**: multiple VPN connections terminating on the same VGW can exchange routes with each other, letting remote sites communicate through AWS. See [AWS VPN CloudHub](https://docs.aws.amazon.com/vpc/latest/userguide/vpn-connections.html).\\\\r\\\\n- **DX can participate in a CloudHub with VPN**: when a DX private VIF and one or more VPN connections share the same VGW, prefixes learned from a VPN customer gateway are advertised to the DX customer gateway and vice versa \\u2014 so a DX site and VPN site(s) on the same VGW can reach each other.\\\\r\\\\n- **DX-to-DX does NOT hub through a VGW**: if two DX VIFs from different on-prem sites terminate on the same VGW (or DX Gateway), AWS does not re-advertise one site\\\\'s on-prem prefixes to the other. AWS advertises only the VPC CIDR(s) toward each VIF, so the two on-prem sites cannot reach each other through AWS. Site-to-site transit requires the customer\\\\'s own router (hairpin) or a design that isn\\\\'t a single shared gateway.\\\\r\\\\n- **DX Gateway (VGW association) advertises only VPC CIDRs to on-prem** \\u2014 it does not provide CloudHub between gateway associations; associations on the same DXGW cannot send traffic to each other.\\\\r\\\\n- **Floating VGW (VGW not attached to a VPC):**\\\\r\\\\n - *With VPN* \\u2014 a floating VGW can still act as a VPN CloudHub: multiple VPN connections on it exchange routes, so remote sites reach each other through AWS with no VPC involved.\\\\r\\\\n - *With DXGW* \\u2014 not a usable scenario. A DXGW advertises associated VPC CIDRs to on-prem; a floating VGW has no VPC, so there is nothing to advertise. DXGW-over-VGW is only meaningful when the VGW is attached to a VPC.\\\\r\\\\n\\\\r\\\\n**Note:** This corrects an earlier claim that DX VIFs on a shared VGW form a DX-to-DX hub-and-spoke mesh. Route sharing between on-prem sites on a shared gateway is a VPN CloudHub behavior (VPN connections), not a DX-to-DX behavior. Validate specific flows in a lab before relying on them.\\\\r\\\\n\\\\r\\\\n### When Can VIFs Talk to Each Other? (Site-to-Site via AWS)\\\\r\\\\n\\\\r\\\\n**Governing principle:** AWS does not act as a transit hub between VIFs by default. Over a VGW or a single DXGW, AWS advertises only the **VPC CIDRs** toward each VIF, so two on-prem sites (each on its own VIF) do not learn each other\\\\'s prefixes unless something explicitly enables it.\\\\r\\\\n\\\\r\\\\n**Supported \\u2014 VIFs (sites) CAN reach each other:**\\\\r\\\\n\\\\r\\\\n| Scenario | How it works |\\\\r\\\\n|---|---|\\\\r\\\\n| VPN CloudHub on a VGW | Multiple VPN connections on the same VGW exchange routes; remote sites reach each other. See [AWS VPN CloudHub](https://docs.aws.amazon.com/vpc/latest/userguide/vpn-connections.html). |\\\\r\\\\n| DX + VPN on the same VGW | A DX private VIF sharing a VGW with VPN connection(s) can talk to the VPN site(s) \\u2014 DX \\u2194 VPN. |\\\\r\\\\n| TGW + Allowed Prefix List | The DXGW\\u2192TGW attachment advertises whatever is in the Allowed Prefix List (no VPC-CIDR-only restriction), so remote-site prefixes can be advertised intentionally. Enables DX \\u2194 VPN CloudHub via TGW. |\\\\r\\\\n| DX \\u2194 DX via two DXGWs into one TGW | Two DXGWs attached to the same TGW (CGW1\\u2013DXGW1\\u2013TGW\\u2013DXGW2\\u2013CGW2) allows DX site-to-site. |\\\\r\\\\n| Customer-side hairpin | Always available: advertise a supernet/default from on-prem and let the customer router be the hub. AWS is not the transit point. |\\\\r\\\\n\\\\r\\\\n**Not supported \\u2014 sites CANNOT reach each other through AWS:**\\\\r\\\\n\\\\r\\\\n| Scenario | Why |\\\\r\\\\n|---|---|\\\\r\\\\n| Two DX VIFs on the same VGW | AWS advertises only VPC CIDRs to each VIF; no site-to-site. |\\\\r\\\\n| Two DX VIFs on the same single DXGW | Gateway associations on one DXGW cannot send traffic to each other. |\\\\r\\\\n| Inter-site over a single DXGW with transit VIFs | Routes may install but the flow loops; not supported. Use two DXGWs. |\\\\r\\\\n| Inter-site via two DXGWs with VGW associations | Only VPC CIDRs are advertised, so the sites never learn each other. |\\\\r\\\\n\\\\r\\\\n**Summary:**\\\\r\\\\n- **VGW / single DXGW** \\u2192 advertises VPC CIDRs only \\u2192 no VIF-to-VIF transit (except true VPN CloudHub, which is a VPN feature).\\\\r\\\\n- **TGW** \\u2192 Allowed Prefix List with no route validation \\u2192 VIF-to-VIF transit can be built intentionally, using two DXGWs for DX-to-DX.\\\\r\\\\n\\\\r\\\\n## DX vs Peering (Public VIF vs PNI)\\\\r\\\\n\\\\r\\\\n### Key Differences\\\\r\\\\n\\\\r\\\\n| Attribute | DX Public VIF | Public/Private Peering (PNI) |\\\\r\\\\n|---|---|---|\\\\r\\\\n| Official AWS service | Yes | No |\\\\r\\\\n| SLA | Yes | No |\\\\r\\\\n| AWS Support coverage | Yes | No (handled by Global Interconnect Strategy team) |\\\\r\\\\n| Internal LOCAL_PREF | Higher | Lower |\\\\r\\\\n| Path preference | **Preferred** (higher LP) | Lower priority |\\\\r\\\\n\\\\r\\\\n### Important Guidance\\\\r\\\\n\\\\r\\\\n- If the **same prefix** is received by AWS over both DX Public VIF and Peering (PNI, IXP, or Transit), **DX Public VIF is the preferred path** (higher internal LOCAL_PREF)\\\\r\\\\n- Public Peering / PNI is NOT an official AWS service \\u2014 no SLA, no support expectation from the technical field\\\\r\\\\n- **Steer customers toward DX** when they need reliability and support coverage\\\\r\\\\n- There is no expectation for the technical field to support Public Peering / PNI engagements\\\\r\\\\n\\\\r\\\\n## Design Considerations for Active/Active & Active/Passive\\\\r\\\\n\\\\r\\\\n### General Best Practice for DX Routing Selection\\\\r\\\\n\\\\r\\\\n- **Always use LP communities (`7224:7300/7200/7100`) for expected egress routing** \\u2014 High Preference BGP communities are the recommended mechanism for predictable DX egress.\\\\r\\\\n- **AS-path prepending: within a region ONLY** \\u2014 Use AS-path prepending to influence egress between DX connections belonging to the **same associated AWS region**. Do NOT use AS-path prepending to influence traffic across regions \\u2014 DXGW\\\\'s local-region LP makes cross-region prepending unreliable.\\\\r\\\\n - **When comparing AS-path lengths, distinguish an originating site from a transiting one** \\u2014 a site that learns a prefix over the on-prem WAN and re-advertises it carries the origin ASN in its path, so its total length is prepend-count + origin ASN(s), not prepend-count alone. Confirm the real path with `list-virtual-interface-routes`. See \\\"Confusing Prepend Count with Total AS-Path Length\\\" in cloudwan-dx-routing-patterns.md.\\\\r\\\\n- **BGP communities: within region OR across regions** \\u2014 LP communities work for both intra-region path selection and cross-region preference (overriding DXGW\\\\'s default local-region LP).\\\\r\\\\n\\\\r\\\\n### Active/Active (within or across DX locations)\\\\r\\\\n\\\\r\\\\n- Understand how AWS selects between multiple active paths using the Private/Transit VIF outbound path-selection order (longest prefix match \\u2192 local preference \\u2192 AS_PATH length \\u2192 MED \\u2192 ECMP). See [Direct Connect routing policies and BGP communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html).\\\\r\\\\n- ECMP is available at the TGW level across multiple DX-GW attachments (same or different DX-GWs)\\\\r\\\\n- LP communities (`7224:7300/7200/7100`) can influence preference but behavior varies between VGW/TGW/Cloud WAN \\u2014 always test\\\\r\\\\n- **DXGW ECMPs across equal remote paths** \\u2014 when a local DX fails and multiple remote DX locations have matching attributes (same LP, same AS-path), DXGW will ECMP across them. This provides load-balanced failover rather than single-path selection.\\\\r\\\\n\\\\r\\\\n**Path selection behavior by service (NOT at DXGW):**\\\\r\\\\n- **DXGW:** ECMPs across equal paths \\u2014 load-balanced failover\\\\r\\\\n- **TGW:** Deterministic \\u2014 selects oldest route for equal paths. Consistent but not customer-controllable.\\\\r\\\\n- **Cloud WAN CNE:** Deterministic \\u2014 \\\"deterministically random\\\" at Step 2.4.4 for equal remote CNE paths. Consistent but not customer-controllable.\\\\r\\\\n\\\\r\\\\n### Active/Passive\\\\r\\\\n\\\\r\\\\n- Use LP communities or AS-path prepending to create primary/secondary paths\\\\r\\\\n- For Cloud WAN: communities override DXGW local-region preference (see cloudwan-dx-routing-patterns.md)\\\\r\\\\n- For TGW: LP communities are NOT honored across multiple DX-GW attachments (see tgw-routing-patterns.md)\\\\r\\\\n- BFD should be enabled for fast failure detection on the active path\\\\r\\\\n'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:20.398000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "afca24ed-acb9-4aa1-a565-3abfab2c554b", + "executionId": "05fe08d0-8f28-41bf-9889-5328d93019ca", + "recordId": "48b3f2da-ca69-4a8b-a4e8-35379590c557", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.9, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:20.490000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "afca24ed-acb9-4aa1-a565-3abfab2c554b", + "executionId": "05fe08d0-8f28-41bf-9889-5328d93019ca", + "recordId": "b06c200d-4531-494f-99a9-e2a96b84143e", + "content": "{\"id\": \"b06c200d-4531-494f-99a9-e2a96b84143e\", \"role\": \"assistant\", \"content\": [{\"text\": \"## Outbound (AWS \\u2192 on-prem) path-selection order for a Private/Transit VIF\\n\\nFor traffic leaving AWS toward on-prem over a Direct Connect private (or transit) VIF, AWS evaluates attributes in this order:\\n\\n1. **Longest prefix match** \\u2014 most specific route wins, regardless of other attributes\\n2. **Local preference** \\u2014 set via DX LP BGP communities: `7224:7300` (High), `7224:7200` (Medium), `7224:7100` (Low)\\n3. **AS_PATH length** \\u2014 shorter path wins\\n4. **MED (Multi-Exit Discriminator)** \\u2014 lower value wins\\n5. **ECMP** \\u2014 used only when AS_PATH length and all BGP attributes above are equal\\n\\n## Where MED fits\\n\\nMED sits **second-to-last** in the evaluation order \\u2014 it's only consulted as a tiebreaker **after** prefix length, local preference, and AS_PATH length have all come out equal between candidate paths. Because it's so far down the priority list, AWS documentation explicitly **does not recommend relying on MED** for predictable DX egress control. If you need deterministic steering, LP communities (step 2) are the right lever \\u2014 they work both within a region and across regions, whereas AS-path prepending (step 3) only reliably works within the same associated region (DXGW's local-region LP preference overrides prepending across regions).\\n\\nWorth noting: this order applies to **private/transit VIFs**. Public VIFs use a different, narrower policy \\u2014 longest prefix match and AS_PATH only; MED isn't part of the public VIF path-selection attributes.\\n\\nLet me know if you want me to walk through where a specific path-selection decision is landing in your setup, or check the LP communities/AS-path currently advertised on a VIF.\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:20.563000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "afca24ed-acb9-4aa1-a565-3abfab2c554b", + "executionId": "05fe08d0-8f28-41bf-9889-5328d93019ca", + "recordId": "e27717dc-80bd-4b99-9461-98988871bbf9", + "content": "{\"id\": \"120a66b3-d258-4517-acdc-4e20d919af75\", \"role\": \"assistant\", \"content\": [{\"type\": \"text\", \"text\": \"## Outbound (AWS \\u2192 on-prem) path-selection order for a Private/Transit VIF\\n\\nFor traffic leaving AWS toward on-prem over a Direct Connect private (or transit) VIF, AWS evaluates attributes in this order:\\n\\n1. **Longest prefix match** \\u2014 most specific route wins, regardless of other attributes\\n2. **Local preference** \\u2014 set via DX LP BGP communities: `7224:7300` (High), `7224:7200` (Medium), `7224:7100` (Low)\\n3. **AS_PATH length** \\u2014 shorter path wins\\n4. **MED (Multi-Exit Discriminator)** \\u2014 lower value wins\\n5. **ECMP** \\u2014 used only when AS_PATH length and all BGP attributes above are equal\\n\\n## Where MED fits\\n\\nMED sits **second-to-last** in the evaluation order \\u2014 it's only consulted as a tiebreaker **after** prefix length, local preference, and AS_PATH length have all come out equal between candidate paths. Because it's so far down the priority list, AWS documentation explicitly **does not recommend relying on MED** for predictable DX egress control. If you need deterministic steering, LP communities (step 2) are the right lever \\u2014 they work both within a region and across regions, whereas AS-path prepending (step 3) only reliably works within the same associated region (DXGW's local-region LP preference overrides prepending across regions).\\n\\nWorth noting: this order applies to **private/transit VIFs**. Public VIFs use a different, narrower policy \\u2014 longest prefix match and AS_PATH only; MED isn't part of the public VIF path-selection attributes.\\n\\nLet me know if you want me to walk through where a specific path-selection decision is landing in your setup, or check the LP communities/AS-path currently advertised on a VIF.\"}]}", + "createdAt": "2026-10-27T17:05:20.658000-05:00", + "recordType": "final_response" + } +] \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/dx-path-selection-order/without_skill/functional-tests-results.json b/skills/aws-routing/evals/functional/v1/iteration-1/dx-path-selection-order/without_skill/functional-tests-results.json new file mode 100644 index 00000000..bda07ced --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/dx-path-selection-order/without_skill/functional-tests-results.json @@ -0,0 +1,70 @@ +{ + "version": "v1", + "iteration": 1, + "eval_id": "dx-path-selection-order", + "run_type": "without_skill", + "success": true, + "error": null, + "understanding_agent_space_skill": null, + "tests": { + "trigger": null, + "expected_output": { + "result": "passed", + "reasoning": "The agent correctly states the path-selection order: longest prefix match, local preference, AS_PATH length, MED, then ECMP \u2014 matching the expected order exactly. It correctly notes MED is a low-priority tiebreaker that AWS does not recommend relying on. The expected output also mentions LP communities 7224:7100/7200/7300 set local preference; the agent mentions 7224:7200 specifically but does not list the full set of community tags (7100/7200/7300). This is a minor omission/incompleteness, but the core substantive content (order, MED's position, recommendation) is fully correct and matches. The community tag detail is a minor supporting fact not the central ask of the question.", + "evidence": "\"1. Longest prefix match ... 2. Local preference ... (7224:7200 = medium local preference...) 3. AS_PATH length ... 4. MED ... 5. ECMP ... AWS explicitly notes it doesn't recommend relying on MED given how low it ranks\"", + "confidence": "high" + }, + "assertions": { + "assertion_results": [ + { + "text": "states the order longest prefix match, local preference, AS_PATH, MED, ECMP", + "evaluator": "llm", + "passed": true, + "evidence": "The response lists: '1. Longest prefix match ... 2. Local preference ... 3. AS_PATH length ... 4. MED ... 5. ECMP' which matches the exact order asserted.", + "reasoning": "The output explicitly enumerates the five-step order in the exact sequence specified.", + "confidence": "high" + }, + { + "text": "notes MED is a low-priority tiebreaker", + "evaluator": "llm", + "passed": true, + "evidence": "'MED ... only comes into play if prefix length, local preference, and AS_PATH length are all equal. It's the last tiebreaker before ECMP.' and 'AWS explicitly notes it doesn't recommend relying on MED given how low it ranks'", + "reasoning": "The output clearly characterizes MED as a low-priority, last-resort tiebreaker before ECMP.", + "confidence": "high" + }, + { + "text": "mentions LP communities 7224:7100/7200/7300", + "evaluator": "llm", + "passed": false, + "evidence": "The output only mentions '7224:7200 = medium local preference for same-Region locations, lower otherwise' and does not mention 7224:7100 or 7224:7300.", + "reasoning": "The assertion requires mention of all three community values 7224:7100/7200/7300, but only 7224:7200 is mentioned explicitly; the others are only vaguely referenced as 'lower otherwise' without their specific tag values.", + "confidence": "high" + }, + { + "text": "does not contain 'error'", + "evaluator": "llm", + "passed": true, + "evidence": "No occurrence of the word 'error' anywhere in the response text.", + "reasoning": "A straightforward text check confirms the word 'error' does not appear in the output.", + "confidence": "high" + } + ], + "summary": { + "passed": 3, + "failed": 1, + "errored": 0, + "low_confidence": 0, + "total": 4, + "pass_rate": 0.75 + } + } + }, + "metrics": { + "runtime": "10s", + "cost": "$0.09", + "context_window": { + "utilization": "4.2%", + "compaction_count": 0 + } + } +} \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/dx-path-selection-order/without_skill/outputs/journal_records.json b/skills/aws-routing/evals/functional/v1/iteration-1/dx-path-selection-order/without_skill/outputs/journal_records.json new file mode 100644 index 00000000..c67d254a --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/dx-path-selection-order/without_skill/outputs/journal_records.json @@ -0,0 +1,90 @@ +[ + { + "agentSpaceId": "ae05a109-e337-48d2-b41b-d8cbdc61e782", + "executionId": "6d3e6776-bffc-47ca-b313-0ef96db94544", + "recordId": "2a9313cc-9dc6-4040-bfe0-a4c118ad0dc5", + "content": "{\"id\": \"2a9313cc-9dc6-4040-bfe0-a4c118ad0dc5\", \"role\": \"user\", \"content\": [{\"text\": \"What is the outbound path-selection order for a Direct Connect private VIF, and where does MED fit?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:04.568000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "ae05a109-e337-48d2-b41b-d8cbdc61e782", + "executionId": "6d3e6776-bffc-47ca-b313-0ef96db94544", + "recordId": "c17fdcfe-17b0-4305-bfeb-1c9669476982", + "content": "{\"id\": \"e285ea2e-7bae-48f6-8494-45a1ba759b51\", \"role\": \"assistant\", \"content\": [{\"type\": \"chat_title\", \"title\": \"Direct Connect private VIF path selection MED\"}]}", + "createdAt": "2026-10-27T17:05:05.379000-05:00", + "recordType": "chat_title" + }, + { + "agentSpaceId": "ae05a109-e337-48d2-b41b-d8cbdc61e782", + "executionId": "6d3e6776-bffc-47ca-b313-0ef96db94544", + "recordId": "b4d69ad2-7912-440f-889f-b23bc99131a8", + "content": "{\"id\": \"b57a1b90-bf12-4364-8869-a35b694a3feb\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This is a general AWS technical question about Direct Connect BGP path selection and MED, not specific to the AWS DevOps Agent product itself, so I should verify this claim using verify_aws_claim.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:10.047000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "ae05a109-e337-48d2-b41b-d8cbdc61e782", + "executionId": "6d3e6776-bffc-47ca-b313-0ef96db94544", + "recordId": "3a209457-eb62-467b-bdaf-22dddc780d4e", + "content": "{\"id\": \"3a209457-eb62-467b-bdaf-22dddc780d4e\", \"role\": \"assistant\", \"content\": [{\"text\": \"Let me verify the Direct Connect path-selection order and MED details against AWS documentation.\", \"type\": \"text\"}, {\"id\": \"tooluse_q744hLOH67HlaJo78pbK3C\", \"tool_name\": \"verify_aws_claim\", \"input\": {\"claim\": \"Direct Connect private VIF outbound path selection order and where MED (Multi-Exit Discriminator) fits in BGP path selection\"}, \"type\": \"tool_use\", \"display_intent\": \"Verify AWS Direct Connect BGP path selection order and MED documentation\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:10.073000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "ae05a109-e337-48d2-b41b-d8cbdc61e782", + "executionId": "6d3e6776-bffc-47ca-b313-0ef96db94544", + "recordId": "2d4dfa80-f27a-4b41-96ea-f27d66b980be", + "content": "{\"id\": \"36e79749-f5ce-484d-bbbe-eeeba52b7dbf\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"1343\"}]}", + "createdAt": "2026-10-27T17:05:10.129000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "ae05a109-e337-48d2-b41b-d8cbdc61e782", + "executionId": "6d3e6776-bffc-47ca-b313-0ef96db94544", + "recordId": "3fe1be70-61fd-483c-8957-07e354c506b2", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 3.8, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:10.180000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "ae05a109-e337-48d2-b41b-d8cbdc61e782", + "executionId": "6d3e6776-bffc-47ca-b313-0ef96db94544", + "recordId": "edee31aa-1519-41f2-8e31-a51c24148a71", + "content": "{\"id\": \"122f24c8-1f0b-49de-914f-a93acbc8ae7e\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_q744hLOH67HlaJo78pbK3C\", \"status\": \"success\", \"content\": [{\"text\": \"## Documentation search results\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"Direct Connect routing policies and BGP communities\\\",\\\"context\\\":\\\"## Private virtual interface and transit virtual interface routing policies\\\\n\\\\nIf you're using AWS Direct Connect to access your private AWS resources,\\\\nyou must specify the IPv4 or IPv6 prefixes to advertise over BGP. These prefixes can be\\\\npublic or private.\\\\n\\\\nThe following outbound routing rules apply based on the prefixes advertised:\\\\n\\\\n* AWS evaluates the longest prefix length first. AWS recommends advertising\\\\n more specific routes using multiple Direct Connect virtual interfaces if the\\\\n desired routing paths are meant for active/passive connections. See Influencing Traffic over Hybrid Networks using Longest Prefix\\\\n Match for more information.\\\\n* Local preference is the BGP attribute recommended to use when desired routing\\\\n paths are meant for active/passive connections and the prefix lengths advertised\\\\n are the same. This value is set per Region to prefer Direct\\\\n Connect locations that have the same associated AWS Region using the\\\\n `7224:7200`\\u2014Medium local preference community value. Where\\\\n the local Region is not associated with the Direct Connect\\\\n location, it is set to a lower value. This applies only if no local preference\\\\n community tags are assigned. For more information about associated Regions, see\\\\n Associated Region.\\\\n* AS_PATH length can be used to determine the routing path when the prefix\\\\n length and local preference are the same.\\\\n* Multi-Exit Discriminator (MED) can be used to determine the routing path when\\\\n prefix length, local preference, and AS_PATH are the same. AWS does not\\\\n recommend using MED values given their lower priority in evaluation.\\\\n* AWS uses equal-cost multi-path (ECMP) routing across multiple transit or\\\\n private virtual interfaces when prefixes have the same AS_PATH length and BGP\\\\n attributes. The ASNs in the AS_PATH of the prefixes do not need to match\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"How do I manage asymmetric routing with Direct Connect?\\\",\\\"context\\\":\\\"Resolution\\\\n----------\\\\n\\\\nThe following resolution explains how to manage outbound traffic flow from AWS to your on-premises network. For inbound traffic flow from your on-premises network to AWS, you must use preferred routing options to set the preference to a particular path.\\\\n\\\\nTo manage asymmetric routing with Direct Connect, use prefix length and Border Gateway Protocol (BGP) attributes to determine path selection for different types of virtual interfaces.\\\\n\\\\n**Note:** For public virtual interfaces, AWS uses AS\\\\\\\\_PATH and the [longest prefix match](https://aws.amazon.com/blogs/networking-and-content-delivery/influencing-traffic-over-hybrid-networks-using-longest-prefix-match/) to determine the routing path\\\",\\\"url\\\":\\\"https://repost.aws/knowledge-center/direct-connect-manage-asymmetric-routing\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"Creating active/passive BGP connections over AWS Direct Connect | Networking & Content Delivery\\\",\\\"context\\\":\\\"### **Let\\u2019s talk datacenter routing tables!**\\\\n\\\\nWithout any tuning whatsoever, this is what the route tables would look like:\\\\n\\\\n* DC1/DC2 would have routes to 10.0.1.0/24 and 10.0.2.0/24, the AS_Path for these routes would have 65001 in the AS_Path list \\u2013 the outgoing interface would the eBGP interfaces for the PrivateVIF or TransitVIF.\\\\n* DXGW would have routes to 172.16.1.0/24 and 172.16.2.0/24, the AS_Path for these routes would have 65000 in the list \\u2013 the outgoing interfaces could be either of the associated PrivateVIFs or TransitVIFs from DC1 or DC2.\\\\n\\\\nNow then, I\\u2019m sure that you can see some of the considerations here:\\\\n\\\\n* Without any tuning, we can\\u2019t influence outbound traffic towards AWS for the production or development networks from the perspective of DC1/DC2.\\\\n* We can\\u2019t influence return traffic from AWS via DXGW so that traffic doesn\\u2019t cross our inter-dc link unnecessarily.\\\\n* We introduce asymmetric routing, which in itself isn\\u2019t necessarily an issue but it could be if there were security devices in the path.\\\\n\\\\nFor both the PrivateVIF and TransitVIF scenarios there is additional configuration that is required beyond the DX BGP configuration in order for traffic to flow end to end. If you are new to DXGW or TGW, check out these great blogs by Jeff Barr:\\\\n\\\\nhttps://aws.amazon.com/blogs/aws/new-aws-direct-connect-gateway-inter-region-vpc-access/\\\\n\\\\nhttps://aws.amazon.com/blogs/aws/new-use-an-aws-transit-gateway-to-simplify-your-network-architecture/\\\",\\\"url\\\":\\\"https://aws.amazon.com/blogs/networking-and-content-delivery/creating-active-passive-bgp-connections-over-aws-direct-connect/\\\"}]}}\\n\\n## Recent AWS announcements (What's New)\\n\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"AWS Direct Connect now supports BGP route visibility on Virtual Interfaces - AWS\\\",\\\"context\\\":\\\"AWS Direct Connect has introduced BGP route visibility on Virtual Interfaces, enabling network administrators to view routes exchanged between AWS and their on-premises routers. This feature allows users to see accepted routes (routes AWS received from their router) and advertised routes (routes AWS sends to their router), along with their AS path and BGP community values. The visibility helps in troubleshooting routing issues, verifying route propagation, and monitoring hybrid network connectivity. Users can access this information through the Direct Connect console or programmatically using the ListVirtualInterfaceRoutes API action. Each route displays its prefix, address family, AS path, community values, and installation timestamp. The feature is available in all AWS commercial Regions and the AWS China Regions. This capability is beneficial for managing complex multi-region architectures, validating BGP policy configurations, or diagnosing unexpected traffic patterns\\\",\\\"url\\\":\\\"https://aws.amazon.com/about-aws/whats-new/2026/07/aws-direct-connect-bgp-visibility/\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\n\\\\n\\\\nServices: directconnect, vpc, transitgateway\\\\nTasks: configure, secure, deploy\\\\nPersona: network-admin, devops, developer, sre\\\\nWorkload: networking, hybrid-cloud\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"AWS Direct Connect introduces inbound prefix controls and higher prefix scale - AWS\\\",\\\"context\\\":\\\"AWS Direct Connect has introduced new capabilities in the form of inbound prefix controls and higher prefix scale. This update allows users to allocate and manage inbound route-prefix allocations for private and transit virtual interfaces (VIFs) based on their workload needs. The maximum number of prefixes that can be accepted on a VIF has increased from 100 to 1,000 for both IPv4 and IPv6. This change eliminates the need for architectural workarounds like summarizing routes or segmenting across multiple VIFs or connections.\\\\n\\\\nInbound prefix controls have introduced new prefix capacity pools at the dedicated connection level and at the Direct Connect gateway (DXGW) level. When creating or updating a VIF, users allocate a specific number of prefixes to it, which draws from the dedicated connection's pool and the DXGW's pool when attached. This allows users to right-size prefix capacity per workload. Connection pool sizes scale with connection speed, and link aggregation group (LAG) pools scale with the number of member connections.\\\\n\\\\nUsers can configure prefix allocations using the AWS Direct Connect console or CLI/API. Inbound prefix controls are available at no additional cost in all commercial AWS Regions where AWS Direct Connect is available, AWS GovCloud Regions (US-East and US-West), as well as the Amazon Web Services China (Beijing) Region, operated by Sinnet, and the Amazon Web Services China (Ningxia) Region, operated by NWCD\\\",\\\"url\\\":\\\"https://aws.amazon.com/about-aws/whats-new/2026/08/aws-direct-connect-new-prefix-controls/\\\"},{\\\"rank_order\\\":4,\\\"title\\\":\\\"connecting-vpcs-with-peering\\\",\\\"skill_description\\\":\\\"Establishes VPC peering connections between two VPCs for direct private network connectivity. Always use this skill when creating or managing VPC peering \\u2014 it validates CIDR overlap, updates all route tables in both VPCs, configures DNS resolution, and provides security group guidance that are critical for correct connectivity.\\\\n\\\\nServices: ec2, vpc\\\\nTasks: configure, deploy\\\\nPersona: developer, devops\\\\nWorkload: networking\\\\n*\\\",\\\"skill_name\\\":\\\"connecting-vpcs-with-peering\\\"},{\\\"rank_order\\\":5,\\\"title\\\":\\\"configuring-vpc-endpoints-for-private-aws-service-access\\\",\\\"skill_description\\\":\\\"Configures VPC endpoints (interface and gateway) for private AWS service access using AWS PrivateLink. Use when setting up secure private connectivity to S3, DynamoDB, and other AWS services without internet gateway, NAT device, or public IP addresses. Covers endpoint creation, security groups, route tables, and DNS configuration.\\\\n\\\\nServices: ec2, vpc, s3, dynamodb\\\\nTasks: configure\\\\nPersona: developer, devops\\\\nWorkload: networking\\\\n*\\\",\\\"skill_name\\\":\\\"configuring-vpc-endpoints-for-private-aws-service-access\\\"}]}}\"}]}], \"label\": \"Verify AWS Direct Connect BGP path selection order and MED documentation\"}", + "createdAt": "2026-10-27T17:05:11.827000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "ae05a109-e337-48d2-b41b-d8cbdc61e782", + "executionId": "6d3e6776-bffc-47ca-b313-0ef96db94544", + "recordId": "4154d050-84ef-453a-a599-e106c6983359", + "content": "{\"id\": \"4154d050-84ef-453a-a599-e106c6983359\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_q744hLOH67HlaJo78pbK3C\", \"content\": \"[{'text': '## Documentation search results\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"Direct Connect routing policies and BGP communities\\\",\\\"context\\\":\\\"## Private virtual interface and transit virtual interface routing policies\\\\\\\\n\\\\\\\\nIf you\\\\'re using AWS Direct Connect to access your private AWS resources,\\\\\\\\nyou must specify the IPv4 or IPv6 prefixes to advertise over BGP. These prefixes can be\\\\\\\\npublic or private.\\\\\\\\n\\\\\\\\nThe following outbound routing rules apply based on the prefixes advertised:\\\\\\\\n\\\\\\\\n* AWS evaluates the longest prefix length first. AWS recommends advertising\\\\\\\\n more specific routes using multiple Direct Connect virtual interfaces if the\\\\\\\\n desired routing paths are meant for active/passive connections. See Influencing Traffic over Hybrid Networks using Longest Prefix\\\\\\\\n Match for more information.\\\\\\\\n* Local preference is the BGP attribute recommended to use when desired routing\\\\\\\\n paths are meant for active/passive connections and the prefix lengths advertised\\\\\\\\n are the same. This value is set per Region to prefer Direct\\\\\\\\n Connect locations that have the same associated AWS Region using the\\\\\\\\n `7224:7200`\\u2014Medium local preference community value. Where\\\\\\\\n the local Region is not associated with the Direct Connect\\\\\\\\n location, it is set to a lower value. This applies only if no local preference\\\\\\\\n community tags are assigned. For more information about associated Regions, see\\\\\\\\n Associated Region.\\\\\\\\n* AS_PATH length can be used to determine the routing path when the prefix\\\\\\\\n length and local preference are the same.\\\\\\\\n* Multi-Exit Discriminator (MED) can be used to determine the routing path when\\\\\\\\n prefix length, local preference, and AS_PATH are the same. AWS does not\\\\\\\\n recommend using MED values given their lower priority in evaluation.\\\\\\\\n* AWS uses equal-cost multi-path (ECMP) routing across multiple transit or\\\\\\\\n private virtual interfaces when prefixes have the same AS_PATH length and BGP\\\\\\\\n attributes. The ASNs in the AS_PATH of the prefixes do not need to match\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"How do I manage asymmetric routing with Direct Connect?\\\",\\\"context\\\":\\\"Resolution\\\\\\\\n----------\\\\\\\\n\\\\\\\\nThe following resolution explains how to manage outbound traffic flow from AWS to your on-premises network. For inbound traffic flow from your on-premises network to AWS, you must use preferred routing options to set the preference to a particular path.\\\\\\\\n\\\\\\\\nTo manage asymmetric routing with Direct Connect, use prefix length and Border Gateway Protocol (BGP) attributes to determine path selection for different types of virtual interfaces.\\\\\\\\n\\\\\\\\n**Note:** For public virtual interfaces, AWS uses AS\\\\\\\\\\\\\\\\_PATH and the [longest prefix match](https://aws.amazon.com/blogs/networking-and-content-delivery/influencing-traffic-over-hybrid-networks-using-longest-prefix-match/) to determine the routing path\\\",\\\"url\\\":\\\"https://repost.aws/knowledge-center/direct-connect-manage-asymmetric-routing\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"Creating active/passive BGP connections over AWS Direct Connect | Networking & Content Delivery\\\",\\\"context\\\":\\\"### **Let\\u2019s talk datacenter routing tables!**\\\\\\\\n\\\\\\\\nWithout any tuning whatsoever, this is what the route tables would look like:\\\\\\\\n\\\\\\\\n* DC1/DC2 would have routes to 10.0.1.0/24 and 10.0.2.0/24, the AS_Path for these routes would have 65001 in the AS_Path list \\u2013 the outgoing interface would the eBGP interfaces for the PrivateVIF or TransitVIF.\\\\\\\\n* DXGW would have routes to 172.16.1.0/24 and 172.16.2.0/24, the AS_Path for these routes would have 65000 in the list \\u2013 the outgoing interfaces could be either of the associated PrivateVIFs or TransitVIFs from DC1 or DC2.\\\\\\\\n\\\\\\\\nNow then, I\\u2019m sure that you can see some of the considerations here:\\\\\\\\n\\\\\\\\n* Without any tuning, we can\\u2019t influence outbound traffic towards AWS for the production or development networks from the perspective of DC1/DC2.\\\\\\\\n* We can\\u2019t influence return traffic from AWS via DXGW so that traffic doesn\\u2019t cross our inter-dc link unnecessarily.\\\\\\\\n* We introduce asymmetric routing, which in itself isn\\u2019t necessarily an issue but it could be if there were security devices in the path.\\\\\\\\n\\\\\\\\nFor both the PrivateVIF and TransitVIF scenarios there is additional configuration that is required beyond the DX BGP configuration in order for traffic to flow end to end. If you are new to DXGW or TGW, check out these great blogs by Jeff Barr:\\\\\\\\n\\\\\\\\nhttps://aws.amazon.com/blogs/aws/new-aws-direct-connect-gateway-inter-region-vpc-access/\\\\\\\\n\\\\\\\\nhttps://aws.amazon.com/blogs/aws/new-use-an-aws-transit-gateway-to-simplify-your-network-architecture/\\\",\\\"url\\\":\\\"https://aws.amazon.com/blogs/networking-and-content-delivery/creating-active-passive-bgp-connections-over-aws-direct-connect/\\\"}]}}\\\\n\\\\n## Recent AWS announcements (What\\\\'s New)\\\\n\\\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"AWS Direct Connect now supports BGP route visibility on Virtual Interfaces - AWS\\\",\\\"context\\\":\\\"AWS Direct Connect has introduced BGP route visibility on Virtual Interfaces, enabling network administrators to view routes exchanged between AWS and their on-premises routers. This feature allows users to see accepted routes (routes AWS received from their router) and advertised routes (routes AWS sends to their router), along with their AS path and BGP community values. The visibility helps in troubleshooting routing issues, verifying route propagation, and monitoring hybrid network connectivity. Users can access this information through the Direct Connect console or programmatically using the ListVirtualInterfaceRoutes API action. Each route displays its prefix, address family, AS path, community values, and installation timestamp. The feature is available in all AWS commercial Regions and the AWS China Regions. This capability is beneficial for managing complex multi-region architectures, validating BGP policy configurations, or diagnosing unexpected traffic patterns\\\",\\\"url\\\":\\\"https://aws.amazon.com/about-aws/whats-new/2026/07/aws-direct-connect-bgp-visibility/\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: directconnect, vpc, transitgateway\\\\\\\\nTasks: configure, secure, deploy\\\\\\\\nPersona: network-admin, devops, developer, sre\\\\\\\\nWorkload: networking, hybrid-cloud\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"AWS Direct Connect introduces inbound prefix controls and higher prefix scale - AWS\\\",\\\"context\\\":\\\"AWS Direct Connect has introduced new capabilities in the form of inbound prefix controls and higher prefix scale. This update allows users to allocate and manage inbound route-prefix allocations for private and transit virtual interfaces (VIFs) based on their workload needs. The maximum number of prefixes that can be accepted on a VIF has increased from 100 to 1,000 for both IPv4 and IPv6. This change eliminates the need for architectural workarounds like summarizing routes or segmenting across multiple VIFs or connections.\\\\\\\\n\\\\\\\\nInbound prefix controls have introduced new prefix capacity pools at the dedicated connection level and at the Direct Connect gateway (DXGW) level. When creating or updating a VIF, users allocate a specific number of prefixes to it, which draws from the dedicated connection\\\\'s pool and the DXGW\\\\'s pool when attached. This allows users to right-size prefix capacity per workload. Connection pool sizes scale with connection speed, and link aggregation group (LAG) pools scale with the number of member connections.\\\\\\\\n\\\\\\\\nUsers can configure prefix allocations using the AWS Direct Connect console or CLI/API. Inbound prefix controls are available at no additional cost in all commercial AWS Regions where AWS Direct Connect is available, AWS GovCloud Regions (US-East and US-West), as well as the Amazon Web Services China (Beijing) Region, operated by Sinnet, and the Amazon Web Services China (Ningxia) Region, operated by NWCD\\\",\\\"url\\\":\\\"https://aws.amazon.com/about-aws/whats-new/2026/08/aws-direct-connect-new-prefix-controls/\\\"},{\\\"rank_order\\\":4,\\\"title\\\":\\\"connecting-vpcs-with-peering\\\",\\\"skill_description\\\":\\\"Establishes VPC peering connections between two VPCs for direct private network connectivity. Always use this skill when creating or managing VPC peering \\u2014 it validates CIDR overlap, updates all route tables in both VPCs, configures DNS resolution, and provides security group guidance that are critical for correct connectivity.\\\\\\\\n\\\\\\\\nServices: ec2, vpc\\\\\\\\nTasks: configure, deploy\\\\\\\\nPersona: developer, devops\\\\\\\\nWorkload: networking\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"connecting-vpcs-with-peering\\\"},{\\\"rank_order\\\":5,\\\"title\\\":\\\"configuring-vpc-endpoints-for-private-aws-service-access\\\",\\\"skill_description\\\":\\\"Configures VPC endpoints (interface and gateway) for private AWS service access using AWS PrivateLink. Use when setting up secure private connectivity to S3, DynamoDB, and other AWS services without internet gateway, NAT device, or public IP addresses. Covers endpoint creation, security groups, route tables, and DNS configuration.\\\\\\\\n\\\\\\\\nServices: ec2, vpc, s3, dynamodb\\\\\\\\nTasks: configure\\\\\\\\nPersona: developer, devops\\\\\\\\nWorkload: networking\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"configuring-vpc-endpoints-for-private-aws-service-access\\\"}]}}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:20.534000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "ae05a109-e337-48d2-b41b-d8cbdc61e782", + "executionId": "6d3e6776-bffc-47ca-b313-0ef96db94544", + "recordId": "d48793cb-79db-42e0-87fc-74751e4b9801", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.2, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:20.614000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "ae05a109-e337-48d2-b41b-d8cbdc61e782", + "executionId": "6d3e6776-bffc-47ca-b313-0ef96db94544", + "recordId": "5ca6c47c-04c3-4c20-af75-8759b5565d42", + "content": "{\"id\": \"5ca6c47c-04c3-4c20-af75-8759b5565d42\", \"role\": \"assistant\", \"content\": [{\"text\": \"For an outbound private VIF (traffic going from AWS toward your on-prem network), AWS evaluates the path in this order:\\n\\n1. **Longest prefix match** \\u2014 the most specific advertised prefix wins first, regardless of other attributes. AWS recommends using this (via multiple VIFs) for active/passive designs.\\n2. **Local preference** \\u2014 used as a tiebreaker when prefix lengths match. AWS sets this automatically based on whether the Direct Connect location shares an associated Region with the VIF (`7224:7200` = medium local preference for same-Region locations, lower otherwise) \\u2014 unless you've set it yourself via local preference community tags.\\n3. **AS_PATH length** \\u2014 shorter AS_PATH wins if prefix length and local preference are tied.\\n4. **MED (Multi-Exit Discriminator)** \\u2014 only comes into play if prefix length, local preference, and AS_PATH length are *all* equal. It's the last tiebreaker before ECMP.\\n5. **ECMP** \\u2014 if everything above (including MED) is equal across multiple VIFs/connections, AWS load-balances across them via equal-cost multi-path routing. ASNs in the AS_PATH don't need to match for ECMP to apply.\\n\\nSo MED sits second-to-last \\u2014 AWS explicitly notes it doesn't recommend relying on MED given how low it ranks; longest prefix match or local preference are the preferred levers for steering active/passive paths.\\n\\nWorth noting: this order is specific to **private/transit VIFs**. Public VIFs use a different evaluation (AS_PATH + longest prefix match, no local preference step) if that's relevant to what you're configuring.\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:20.700000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "ae05a109-e337-48d2-b41b-d8cbdc61e782", + "executionId": "6d3e6776-bffc-47ca-b313-0ef96db94544", + "recordId": "41f27b30-a762-4ffa-81ef-08680a7fefe8", + "content": "{\"id\": \"dcdd7e0d-e7a6-4e1e-942f-f90f505ab139\", \"role\": \"assistant\", \"content\": [{\"type\": \"text\", \"text\": \"For an outbound private VIF (traffic going from AWS toward your on-prem network), AWS evaluates the path in this order:\\n\\n1. **Longest prefix match** \\u2014 the most specific advertised prefix wins first, regardless of other attributes. AWS recommends using this (via multiple VIFs) for active/passive designs.\\n2. **Local preference** \\u2014 used as a tiebreaker when prefix lengths match. AWS sets this automatically based on whether the Direct Connect location shares an associated Region with the VIF (`7224:7200` = medium local preference for same-Region locations, lower otherwise) \\u2014 unless you've set it yourself via local preference community tags.\\n3. **AS_PATH length** \\u2014 shorter AS_PATH wins if prefix length and local preference are tied.\\n4. **MED (Multi-Exit Discriminator)** \\u2014 only comes into play if prefix length, local preference, and AS_PATH length are *all* equal. It's the last tiebreaker before ECMP.\\n5. **ECMP** \\u2014 if everything above (including MED) is equal across multiple VIFs/connections, AWS load-balances across them via equal-cost multi-path routing. ASNs in the AS_PATH don't need to match for ECMP to apply.\\n\\nSo MED sits second-to-last \\u2014 AWS explicitly notes it doesn't recommend relying on MED given how low it ranks; longest prefix match or local preference are the preferred levers for steering active/passive paths.\\n\\nWorth noting: this order is specific to **private/transit VIFs**. Public VIFs use a different evaluation (AS_PATH + longest prefix match, no local preference step) if that's relevant to what you're configuring.\"}]}", + "createdAt": "2026-10-27T17:05:20.779000-05:00", + "recordType": "final_response" + } +] \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/dx-prefer-one-location/with_skill/functional-tests-results.json b/skills/aws-routing/evals/functional/v1/iteration-1/dx-prefer-one-location/with_skill/functional-tests-results.json new file mode 100644 index 00000000..b47c488c --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/dx-prefer-one-location/with_skill/functional-tests-results.json @@ -0,0 +1,64 @@ +{ + "version": "v1", + "iteration": 1, + "eval_id": "dx-prefer-one-location", + "run_type": "with_skill", + "success": true, + "error": null, + "understanding_agent_space_skill": null, + "tests": { + "trigger": { + "result": "passed", + "message": "Skill 'aws-routing' was triggered as expected", + "skill_loads_found": 1 + }, + "expected_output": { + "result": "failed", + "reasoning": "No FinalResponse found in chat output. The agent may not have produced a final response." + }, + "assertions": { + "assertion_results": [ + { + "text": "recommends LP communities with a higher preference on the primary", + "evaluator": "llm", + "passed": null, + "error": "No FinalResponse found in chat output, so no assertion could be checked." + }, + { + "text": "explains AS-path prepending is reliable only within the same region", + "evaluator": "llm", + "passed": null, + "error": "No FinalResponse found in chat output, so no assertion could be checked." + }, + { + "text": "warns cross-region AS-path prepending is unreliable", + "evaluator": "llm", + "passed": null, + "error": "No FinalResponse found in chat output, so no assertion could be checked." + }, + { + "text": "does not contain 'error'", + "evaluator": "llm", + "passed": null, + "error": "No FinalResponse found in chat output, so no assertion could be checked." + } + ], + "summary": { + "passed": 0, + "failed": 0, + "errored": 4, + "low_confidence": 0, + "total": 0, + "pass_rate": 0.0 + } + } + }, + "metrics": { + "runtime": "16s", + "cost": "$0.14", + "context_window": { + "utilization": "4.9%", + "compaction_count": 0 + } + } +} \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/dx-prefer-one-location/with_skill/outputs/journal_records.json b/skills/aws-routing/evals/functional/v1/iteration-1/dx-prefer-one-location/with_skill/outputs/journal_records.json new file mode 100644 index 00000000..7e296d65 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/dx-prefer-one-location/with_skill/outputs/journal_records.json @@ -0,0 +1,146 @@ +[ + { + "agentSpaceId": "c535feca-7122-4e76-a70f-34dc696d9683", + "executionId": "cb665d99-08df-4a4d-ad7f-af59d394c298", + "recordId": "deb37750-4cdc-44b1-b1e1-32f785d3d27b", + "content": "{\"id\": \"deb37750-4cdc-44b1-b1e1-32f785d3d27b\", \"role\": \"user\", \"content\": [{\"text\": \"I have two Direct Connect locations and want traffic to prefer location A over location B, with B as backup. How do I do this and what should I avoid?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:04.082000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "c535feca-7122-4e76-a70f-34dc696d9683", + "executionId": "cb665d99-08df-4a4d-ad7f-af59d394c298", + "recordId": "7c56eb0f-acb1-4cc1-aa32-69ec35a84360", + "content": "{\"id\": \"048a7030-7e0e-4912-8331-0542a8d0ef4c\", \"role\": \"assistant\", \"content\": [{\"type\": \"chat_title\", \"title\": \"AWS Direct Connect traffic preference location A backup\"}]}", + "createdAt": "2026-10-27T17:05:04.780000-05:00", + "recordType": "chat_title" + }, + { + "agentSpaceId": "c535feca-7122-4e76-a70f-34dc696d9683", + "executionId": "cb665d99-08df-4a4d-ad7f-af59d394c298", + "recordId": "51cbb775-c3e0-4e45-8fc8-0cd49ec9aa27", + "content": "{\"id\": \"94350113-1fab-4e5d-b38c-b3559ddd9aba\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This question involves Direct Connect traffic engineering, so I should use the aws-routing skill for this.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:07.406000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "c535feca-7122-4e76-a70f-34dc696d9683", + "executionId": "cb665d99-08df-4a4d-ad7f-af59d394c298", + "recordId": "0520f059-e6fb-401c-a413-33615321b388", + "content": "{\"id\": \"0520f059-e6fb-401c-a413-33615321b388\", \"role\": \"assistant\", \"content\": [{\"id\": \"tooluse_5iRpoKnAJopJk9JKw0ZUAJ\", \"tool_name\": \"skill_read\", \"input\": {\"skill_id\": \"aws-routing\"}, \"type\": \"tool_use\", \"display_intent\": \"Reading AWS routing skill for Direct Connect traffic engineering guidance\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:07.406000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "c535feca-7122-4e76-a70f-34dc696d9683", + "executionId": "cb665d99-08df-4a4d-ad7f-af59d394c298", + "recordId": "981ba009-38e5-4da0-af07-8e40f0e218cc", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 3.8, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:07.481000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "c535feca-7122-4e76-a70f-34dc696d9683", + "executionId": "cb665d99-08df-4a4d-ad7f-af59d394c298", + "recordId": "47dea729-0623-4d41-87c5-e3be3e808664", + "content": "{\"id\": \"cc28d5d0-97f5-40d5-8822-43166ce75fb8\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"460\"}]}", + "createdAt": "2026-10-27T17:05:07.484000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "c535feca-7122-4e76-a70f-34dc696d9683", + "executionId": "cb665d99-08df-4a4d-ad7f-af59d394c298", + "recordId": "a3780938-4728-4252-ae2b-385de2a4ae4f", + "content": "{\"id\": \"d42648af-fd4d-453c-8e46-6c9258e89766\", \"role\": \"assistant\", \"block_type\": \"load_skill\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_5iRpoKnAJopJk9JKw0ZUAJ\", \"status\": \"success\", \"content\": [{\"text\": \"Done\"}]}], \"label\": \"Reading AWS routing skill for Direct Connect traffic engineering guidance\"}", + "createdAt": "2026-10-27T17:05:07.593000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "c535feca-7122-4e76-a70f-34dc696d9683", + "executionId": "cb665d99-08df-4a4d-ad7f-af59d394c298", + "recordId": "cca5695c-8af4-4f76-95bb-152bf9eb68dc", + "content": "{\"id\": \"70c43e84-6e6f-407c-880a-f222816ed435\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"I should pull up the dx-routing-patterns reference to check BGP local preference communities and the active/active versus active/passive design tradeoffs and caveats.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:11.172000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "c535feca-7122-4e76-a70f-34dc696d9683", + "executionId": "cb665d99-08df-4a4d-ad7f-af59d394c298", + "recordId": "91f3f657-f398-4104-97c9-9cad2e02f3a5", + "content": "{\"id\": \"91f3f657-f398-4104-97c9-9cad2e02f3a5\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_5iRpoKnAJopJk9JKw0ZUAJ\", \"content\": \"[{'text': '{\\\"content\\\": \\\"---\\\\\\\\r\\\\\\\\nname: aws-routing\\\\\\\\r\\\\\\\\ndescription: \\\\'Use this skill when the user wants to know which path AWS will pick for their traffic, why traffic took an unexpected route, or how to make AWS prefer a path. Covers Cloud WAN CNE route evaluation, Direct Connect Gateway local vs remote region preference, Transit Gateway route tables, ECMP and peering, VPC longest-prefix-match, BGP traffic engineering (LP communities 7224:7100/7200/7300, AS-path prepending, MED), DX + VPN redundancy and failover, and asymmetric routing. Use it even when the user only describes a symptom, such as \\\\\\\\\\\"traffic is leaving the region\\\\\\\\\\\", \\\\\\\\\\\"not using my local Direct Connect\\\\\\\\\\\", \\\\\\\\\\\"routing to the wrong VIF/location\\\\\\\\\\\", or \\\\\\\\\\\"path is asymmetric\\\\\\\\\\\". Read-only: analysis, recommendations, and describe/get/list validation commands grounded in public AWS docs.\\\\'\\\\\\\\r\\\\\\\\nmetadata:\\\\\\\\r\\\\\\\\n version: \\\\\\\\\\\"1.2.5\\\\\\\\\\\"\\\\\\\\r\\\\\\\\n author: midakin\\\\\\\\r\\\\\\\\n aws-devops-agent-skills.agent-types: \\\\\\\\\\\"CHAT, INCIDENT_RCA\\\\\\\\\\\"\\\\\\\\r\\\\\\\\n---\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Overview\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nThis skill provides routing-first analysis for AWS networking. It reasons about the full traffic\\\\\\\\r\\\\\\\\npath, applies the documented route-evaluation logic for each service, and grounds recommendations\\\\\\\\r\\\\\\\\nin public AWS documentation. It is **read-only**: it produces analysis, recommendations, and\\\\\\\\r\\\\\\\\nread-only validation commands (`describe-*`, `get-*`, `list-*`) - never mutating actions.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Verification & Anti-Hallucination Directives (Mandatory)\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- Do not generate any fact, citation, or source that is not fully verifiable. If you lack a verified\\\\\\\\r\\\\\\\\n source, state \\\\\\\\\\\"I cannot verify this\\\\\\\\\\\" rather than inventing details.\\\\\\\\r\\\\\\\\n- State assumptions explicitly before answering.\\\\\\\\r\\\\\\\\n- Cross-check each claim against public AWS documentation or the `references/` files. Omit claims you\\\\\\\\r\\\\\\\\n cannot confirm.\\\\\\\\r\\\\\\\\n- Only cite AWS documentation URLs you have actually retrieved or confirmed. Do not fabricate links.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## When to use this skill\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nActivate when the user asks about:\\\\\\\\r\\\\\\\\n- Cloud WAN route evaluation / Core Network Edge (CNE) path selection\\\\\\\\r\\\\\\\\n- Direct Connect Gateway path selection, local-region preference, or LP communities\\\\\\\\r\\\\\\\\n- Transit Gateway route evaluation, ECMP, or peering routing\\\\\\\\r\\\\\\\\n- VPC route tables and longest-prefix-match behavior\\\\\\\\r\\\\\\\\n- BGP traffic engineering (communities, AS-path prepending, MED)\\\\\\\\r\\\\\\\\n- DX + VPN redundancy, active/active vs active/passive, or failover design\\\\\\\\r\\\\\\\\n- Why traffic is taking an unexpected path\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## What this skill does NOT do\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nThis skill is **read-only and advisory**. It will not:\\\\\\\\r\\\\\\\\n- **Design or apply infrastructure changes.** It does not create, modify, or delete AWS resources,\\\\\\\\r\\\\\\\\n and it does not run mutating CLI/API calls (`create-*`, `modify-*`, `associate-*`, `delete-*`,\\\\\\\\r\\\\\\\\n `put-*`, `update-*`) or write Infrastructure-as-Code to be deployed. Any change it describes is a\\\\\\\\r\\\\\\\\n *recommendation* for you to review and implement yourself.\\\\\\\\r\\\\\\\\n- **Configure BGP or push routing policy.** It explains which communities, AS-path, or MED settings\\\\\\\\r\\\\\\\\n to use, but it does not apply them to VIFs, route tables, or Cloud WAN policy documents.\\\\\\\\r\\\\\\\\n- **Execute anything against a live account beyond read-only validation.** Commands it provides are\\\\\\\\r\\\\\\\\n limited to `describe-*`, `get-*`, and `list-*` for confirming state.\\\\\\\\r\\\\\\\\n- **Guarantee production outcomes.** It recommends non-production testing and blue/green rollout;\\\\\\\\r\\\\\\\\n it does not perform the cutover or validate the result for you.\\\\\\\\r\\\\\\\\n- **Provide compliance, security, or contractual sign-off**, or make claims it cannot ground in\\\\\\\\r\\\\\\\\n public AWS documentation or the `references/` files.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nFor designing and applying changes, hand the recommendation to a change-capable workflow (with\\\\\\\\r\\\\\\\\nappropriate review, approvals, and testing).\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## How to analyze a routing problem\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nWork through this checklist in order; each step depends on the one before it.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- [ ] **1. Establish the path.** Identify source, destination, and every routing construct in\\\\\\\\r\\\\\\\\n between (VPC route table \\\\\\\\u2192 TGW/Cloud WAN CNE \\\\\\\\u2192 DXGW \\\\\\\\u2192 VIF \\\\\\\\u2192 on-prem, or the relevant subset).\\\\\\\\r\\\\\\\\n- [ ] **2. Name the route table.** Be explicit about which table makes each decision (VPC RT, TGW RT,\\\\\\\\r\\\\\\\\n CNE RT, DXGW internal selection).\\\\\\\\r\\\\\\\\n- [ ] **3. Apply the evaluation order for that construct** (see below).\\\\\\\\r\\\\\\\\n- [ ] **4. Identify the deciding attribute** (longest prefix, local preference, AS-path, MED, source\\\\\\\\r\\\\\\\\n type).\\\\\\\\r\\\\\\\\n- [ ] **5. Flag non-determinism** (e.g., ECMP or \\\\\\\\\\\"deterministically random\\\\\\\\\\\" tiebreakers) and\\\\\\\\r\\\\\\\\n recommend a deterministic alternative.\\\\\\\\r\\\\\\\\n- [ ] **6. Cite the AWS documentation** that supports the behavior.\\\\\\\\r\\\\\\\\n- [ ] **7. Provide read-only validation commands** the user can run to confirm.\\\\\\\\r\\\\\\\\n- [ ] **8. Self-check before answering.** Re-read the answer against steps 1-7: every route table is\\\\\\\\r\\\\\\\\n named, the evaluation order matches the construct, each cited URL was actually retrieved, and every\\\\\\\\r\\\\\\\\n command is read-only. Fix any gap before presenting.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Key evaluation rules\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Cloud WAN (per CNE)\\\\\\\\r\\\\\\\\nLongest prefix match \\\\\\\\u2192 static \\\\\\\\u2192 VPC-propagated (same region) \\\\\\\\u2192 unequal AS-path/MED (shortest wins) \\\\\\\\u2192\\\\\\\\r\\\\\\\\nequal AS-path & MED source preference: DXGW-propagated \\\\\\\\u2192 Cloud WAN Connect \\\\\\\\u2192 S2S VPN \\\\\\\\u2192 other\\\\\\\\r\\\\\\\\n(TGW peering, remote CNEs; identical from 2+ sources = deterministically random).\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Direct Connect (private/transit VIF, outbound AWS\\\\\\\\u2192on-prem)\\\\\\\\r\\\\\\\\nLongest prefix match \\\\\\\\u2192 local preference \\\\\\\\u2192 AS_PATH length \\\\\\\\u2192 MED \\\\\\\\u2192 ECMP (equal AS_PATH and BGP\\\\\\\\r\\\\\\\\nattributes). LP communities `7224:7300/7200/7100` (High/Medium/Low) set local preference and are\\\\\\\\r\\\\\\\\nevaluated before AS-path. DXGW prefers the local associated-region DX by default.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Direction & tooling guidance\\\\\\\\r\\\\\\\\n- **AWS \\\\\\\\u2192 on-prem (egress):** control with DX LP communities.\\\\\\\\r\\\\\\\\n- **On-prem \\\\\\\\u2192 AWS (ingress):** control with the customer router\\\\'s local-pref/weight/MED.\\\\\\\\r\\\\\\\\n- **AS-path prepending:** within a region only - unreliable across regions because DXGW local-region\\\\\\\\r\\\\\\\\n LP overrides it. **LP communities:** work within-region and cross-region.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Reference material\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nDetailed mechanics, patterns, and caveats are in `references/`. Load only the file that matches the\\\\\\\\r\\\\\\\\nconstruct deciding the path:\\\\\\\\r\\\\\\\\n- [Cloud WAN + DX routing patterns](references/cloudwan-dx-routing-patterns.md) - load when the path\\\\\\\\r\\\\\\\\n crosses a Cloud WAN CNE: CNE route evaluation, DXGW path selection into Cloud WAN, community\\\\\\\\r\\\\\\\\n traffic-engineering patterns, regional inspection, multi-region egress.\\\\\\\\r\\\\\\\\n- [Direct Connect routing patterns](references/dx-routing-patterns.md) - load when the question is\\\\\\\\r\\\\\\\\n about a DX VIF or DXGW without Cloud WAN: BGP mechanics, BFD, MED, ASN ranges, VIF considerations,\\\\\\\\r\\\\\\\\n CloudHub, when VIFs can talk, active/active vs active/passive.\\\\\\\\r\\\\\\\\n- [Transit Gateway routing patterns](references/tgw-routing-patterns.md) - load when a TGW route\\\\\\\\r\\\\\\\\n table makes the decision: TGW route evaluation, ECMP limits, DXGW-to-TGW behavior, DX/VPN caveats,\\\\\\\\r\\\\\\\\n failover, peering.\\\\\\\\r\\\\\\\\n- [VPN + DX redundancy patterns](references/vpn-dx-redundancy-patterns.md) - load when VPN backs up\\\\\\\\r\\\\\\\\n or runs alongside DX: route preference between them, ECMP limits, asymmetric return paths.\\\\\\\\r\\\\\\\\n- [Networking strategic questions](references/networking-strategic-questions.md) - load when key\\\\\\\\r\\\\\\\\n facts are missing (regions, attachments, ASNs, communities) and you need discovery questions to ask\\\\\\\\r\\\\\\\\n before analyzing.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Output expectations\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- Explain the route-evaluation logic step by step.\\\\\\\\r\\\\\\\\n- Name the specific route table involved.\\\\\\\\r\\\\\\\\n- Give the end-to-end traffic path.\\\\\\\\r\\\\\\\\n- Compare options with tradeoffs (determinism, complexity, failover) when multiple solutions exist.\\\\\\\\r\\\\\\\\n- Include read-only AWS CLI validation commands.\\\\\\\\r\\\\\\\\n- Recommend testing in a non-production environment and blue/green for migrations.\\\\\\\\r\\\\\\\\n\\\", \\\"resource_manifest\\\": [\\\"SKILL.md\\\", \\\"references/cloudwan-dx-routing-patterns.md\\\", \\\"references/dx-routing-patterns.md\\\", \\\"references/networking-strategic-questions.md\\\", \\\"references/tgw-routing-patterns.md\\\", \\\"references/vpn-dx-redundancy-patterns.md\\\"]}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:11.172000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "c535feca-7122-4e76-a70f-34dc696d9683", + "executionId": "cb665d99-08df-4a4d-ad7f-af59d394c298", + "recordId": "7e7bc02e-b239-4cbc-b66b-cab1baa139db", + "content": "{\"id\": \"7e7bc02e-b239-4cbc-b66b-cab1baa139db\", \"role\": \"assistant\", \"content\": [{\"id\": \"tooluse_FZsZ3J3ToTmp3dr2VYl1XA\", \"tool_name\": \"read_skill_resource\", \"input\": {\"skill_id\": \"aws-routing\", \"path\": \"references/dx-routing-patterns.md\"}, \"type\": \"tool_use\", \"display_intent\": \"Reading Direct Connect routing patterns reference for active/passive design guidance\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:11.252000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "c535feca-7122-4e76-a70f-34dc696d9683", + "executionId": "cb665d99-08df-4a4d-ad7f-af59d394c298", + "recordId": "4c3447eb-cc16-47d4-8e90-99ab5ba81d3e", + "content": "{\"id\": \"dfe74061-fdf0-4cfa-89bf-55b2edef5219\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"804\"}]}", + "createdAt": "2026-10-27T17:05:11.258000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "c535feca-7122-4e76-a70f-34dc696d9683", + "executionId": "cb665d99-08df-4a4d-ad7f-af59d394c298", + "recordId": "14ef9d48-90b6-45ca-a686-b7ca0130fac5", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.2, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:11.363000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "c535feca-7122-4e76-a70f-34dc696d9683", + "executionId": "cb665d99-08df-4a4d-ad7f-af59d394c298", + "recordId": "82f63005-76c0-4852-9494-d5a55cf51bc0", + "content": "{\"id\": \"746272c4-b101-4910-b949-258f35e685d4\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_FZsZ3J3ToTmp3dr2VYl1XA\", \"status\": \"success\", \"content\": [{\"text\": \"Done\"}]}], \"label\": \"Reading Direct Connect routing patterns reference for active/passive design guidance\"}", + "createdAt": "2026-10-27T17:05:11.675000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "c535feca-7122-4e76-a70f-34dc696d9683", + "executionId": "cb665d99-08df-4a4d-ad7f-af59d394c298", + "recordId": "dd84dfe9-9a30-44f5-8116-64b1a59d1aea", + "content": "{\"id\": \"b5bc7cab-7e54-4e44-89f8-69ed9121e4a6\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This is a general conceptual question, so I can answer directly with advisory guidance rather than pulling account-specific data \\u2014 recommending LP communities for the route with high LP on one location and lower on the other, while flagging what to avoid.\\n\\nBut the exact recommendation varies significantly depending on their gateway architecture \\u2014 DX Gateway + VGW, DX Gateway + TGW, or Cloud WAN behave differently with LP communities, and notably LP isn't honored the same way across TGW DX-GW attachments. This is worth clarifying with the user before giving a precise answer, so I'll ask which construct they're using.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:22.911000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "c535feca-7122-4e76-a70f-34dc696d9683", + "executionId": "cb665d99-08df-4a4d-ad7f-af59d394c298", + "recordId": "3b6d65f1-b9c3-4422-8fca-ae485c51fa33", + "content": "{\"id\": \"eda63a8e-dcb9-4697-b100-3a393fa14f50\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"5862\"}]}", + "createdAt": "2026-10-27T17:05:23-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "c535feca-7122-4e76-a70f-34dc696d9683", + "executionId": "cb665d99-08df-4a4d-ad7f-af59d394c298", + "recordId": "4c3f398b-d8d8-4eb3-837c-233cb07eeb82", + "content": "{\"id\": \"4c3f398b-d8d8-4eb3-837c-233cb07eeb82\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_FZsZ3J3ToTmp3dr2VYl1XA\", \"content\": \"[{'text': '\\\\r\\\\n\\\\r\\\\n# Direct Connect Routing Patterns & Considerations\\\\r\\\\n\\\\r\\\\nThis document captures key BGP routing mechanics, path selection behavior, VIF-specific considerations, and operational caveats for AWS Direct Connect architectures. Use this knowledge when advising on DX design (active/active, active/passive), BGP community behavior, BFD configuration, and public/private/transit VIF routing.\\\\r\\\\n\\\\r\\\\n## Reference Documentation\\\\r\\\\n\\\\r\\\\n- [Direct Connect Routing Policies and BGP Communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html)\\\\r\\\\n- [Direct Connect Path Selection (routing policies for private/transit VIFs)](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html)\\\\r\\\\n- [Direct Connect Virtual Interfaces](https://docs.aws.amazon.com/directconnect/latest/UserGuide/WorkingWithVirtualInterfaces.html)\\\\r\\\\n\\\\r\\\\n## BGP & Routing\\\\r\\\\n\\\\r\\\\n### BGP Route Visibility (ListVirtualInterfaceRoutes)\\\\r\\\\n\\\\r\\\\nAs of July 2026, DX supports **BGP route visibility** on virtual interfaces via the `ListVirtualInterfaceRoutes` API and the DX console:\\\\r\\\\n\\\\r\\\\n- **Accepted routes:** Routes AWS received from your on-prem router\\\\r\\\\n- **Advertised routes:** Routes AWS is sending to your on-prem router\\\\r\\\\n- Each route shows: prefix, address family, AS-path, community values, installation timestamp\\\\r\\\\n- Can filter by: prefix, AS-path, community, or address family\\\\r\\\\n- Available on: private, transit, and public VIFs\\\\r\\\\n- Available in: all AWS commercial regions\\\\r\\\\n\\\\r\\\\n**Use for troubleshooting:**\\\\r\\\\n- Verify route propagation between on-prem and AWS\\\\r\\\\n- Validate BGP policy configurations (communities, AS-path prepending)\\\\r\\\\n- Diagnose unexpected traffic patterns or asymmetric routing\\\\r\\\\n- Confirm which routes DXGW is accepting vs. advertising\\\\r\\\\n- Validate LP community tags are being received correctly\\\\r\\\\n\\\\r\\\\n**Reference:** [AWS Direct Connect BGP Route Visibility announcement](https://aws.amazon.com/about-aws/whats-new/2026/07/aws-direct-connect-bgp-visibility/)\\\\r\\\\n\\\\r\\\\n### CloudWatch BGP Metrics for DX Virtual Interfaces\\\\r\\\\n\\\\r\\\\nAs of March 2026, DX publishes **3 BGP metrics** to CloudWatch for all VIF types (private, transit, public):\\\\r\\\\n\\\\r\\\\n| Metric | What It Reports |\\\\r\\\\n|---|---|\\\\r\\\\n| `VirtualInterfaceBgpStatus` | BGP session state: 1 = up, 0 = down |\\\\r\\\\n| `VirtualInterfaceBgpPrefixesAccepted` | Number of prefixes AWS accepted from on-prem |\\\\r\\\\n| `VirtualInterfaceBgpPrefixesAdvertised` | Number of prefixes AWS advertises to on-prem |\\\\r\\\\n\\\\r\\\\n**Key details:**\\\\r\\\\n- Updated every 5 minutes (BGP flaps between intervals may not be captured)\\\\r\\\\n- Published at no extra cost in the region where the DX location is associated\\\\r\\\\n- Available via CloudWatch MCP server (`get_metric_data`) for programmatic access\\\\r\\\\n\\\\r\\\\n**Use cases:**\\\\r\\\\n- **BGP session monitoring:** Alarm on `VirtualInterfaceBgpStatus < 1` to detect session failures without API polling\\\\r\\\\n- **Prefix limit protection:** Alarm when `PrefixesAccepted` approaches VIF quota to prevent session idle state\\\\r\\\\n- **Silent route withdrawal detection:** BGP session stays UP but prefix count drops \\u2192 indicates routes withdrawn without session going down\\\\r\\\\n- **DR/failover validation:** Compare prefix counts between primary and standby VIFs \\u2014 a standby with fewer prefixes indicates a broken failover path\\\\r\\\\n- **Route change correlation:** Sudden drop in accepted/advertised prefixes correlates with traffic impact events\\\\r\\\\n\\\\r\\\\n**Alarm examples:**\\\\r\\\\n- `VirtualInterfaceBgpStatus` \\u2014 Statistic: Minimum, Period: 5min, Threshold: Lower than 1\\\\r\\\\n- `VirtualInterfaceBgpPrefixesAccepted` \\u2014 Statistic: Maximum, Period: 5min, Threshold: Greater/Equal to (near quota limit)\\\\r\\\\n- `VirtualInterfaceBgpPrefixesAccepted` \\u2014 Statistic: Maximum, Period: 5min, Threshold: Lower/Equal to (below expected minimum)\\\\r\\\\n\\\\r\\\\n**Reference:** [CloudWatch Metrics for DX VIF BGP Health and Prefix Count](https://aws.amazon.com/blogs/networking-and-content-delivery/introducing-cloudwatch-metrics-for-aws-direct-connect-virtual-interface-bgp-health-and-prefix-count/)\\\\r\\\\n\\\\r\\\\n### General Routing Behavior\\\\r\\\\n\\\\r\\\\n- When designing active/active or active/passive DX (within or across DX locations), apply the documented **Private/Transit VIF outbound path-selection order**: longest prefix match \\u2192 local preference \\u2192 AS_PATH length \\u2192 MED \\u2192 ECMP (equal AS_PATH length and BGP attributes). See [Direct Connect routing policies and BGP communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html).\\\\r\\\\n- Consider recommendations for **migrating from DX + VGW/Private VIF to DX + TGW/Transit VIF** when customers are modernizing their architecture\\\\r\\\\n\\\\r\\\\n### MED Support\\\\r\\\\n\\\\r\\\\n- DX **supports Multi-Exit Discriminator (MED) on Private and Transit VIFs** \\u2014 MED can influence the outbound (AWS \\u2192 on-prem) routing path, but only as a **tiebreaker after prefix length, local preference, and AS_PATH are equal**. AWS explicitly **does not recommend relying on MED** given its lower priority in the evaluation order.\\\\r\\\\n- For **Public VIFs**, the documented outbound routing policy uses **AS_PATH and longest prefix match** \\u2014 MED is not listed among the supported public VIF path-selection attributes.\\\\r\\\\n- **Outbound (AWS \\u2192 on-prem) path selection order for Private/Transit VIF:** longest prefix match \\u2192 local preference \\u2192 AS_PATH length \\u2192 MED \\u2192 ECMP (equal AS_PATH length and BGP attributes).\\\\r\\\\n\\\\r\\\\n**Reference:** [Direct Connect routing policies and BGP communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html)\\\\r\\\\n\\\\r\\\\n### BFD (Bidirectional Forwarding Detection)\\\\r\\\\n\\\\r\\\\n**\\u26a0\\ufe0f Always verify the Associated Region for each DX location** \\u2014 some DX locations can associate to different AWS regions than expected based on their physical geography. For example:\\\\r\\\\n- Equinix LD5 (Slough, UK) \\u2192 **eu-west-1** (Ireland)\\\\r\\\\n- Telehouse (London Docklands, UK) \\u2192 **eu-west-2** (London)\\\\r\\\\n- Equinix MA3 (Manchester, UK) \\u2192 **eu-west-2** (London)\\\\r\\\\n\\\\r\\\\nThe associated region determines which TGW/DXGW considers the DX \\\"local\\\" for LP preference and routing behavior. Verify a location\\\\'s associated region with the `DescribeLocations` API/CLI (`aws directconnect describe-locations`) or the official AWS Direct Connect locations reference: https://aws.amazon.com/directconnect/locations/\\\\r\\\\n\\\\r\\\\n### BFD (Bidirectional Forwarding Detection)\\\\r\\\\n\\\\r\\\\n- **Asynchronous BFD is automatically enabled** on the AWS side for all DX virtual interfaces\\\\r\\\\n- Customer must configure asynchronous BFD on their router (refer to vendor documentation)\\\\r\\\\n- BFD provides fast link failure detection \\u2014 ensures BGP neighbor relationship is torn down quickly for faster failover\\\\r\\\\n- **Do NOT configure both BFD for BGP and BGP Graceful Restart simultaneously** \\u2014 this results in sub-optimal routing performance (conflicting goals: BFD wants fast teardown, Graceful Restart wants to preserve routes during failure)\\\\r\\\\n\\\\r\\\\n### ASN Support\\\\r\\\\n\\\\r\\\\n| Context | Supported ASN Range |\\\\r\\\\n|---|---|\\\\r\\\\n| AWS side via DX-GW | Full 32-bit: 4,200,000,000 to 4,294,967,294 (private) |\\\\r\\\\n| AWS side via VGW | 32-bit: 1 to 2,147,483,647 only |\\\\r\\\\n| Customer side (Private/Public/Transit VIF) | 32-bit: 1 to 2,147,483,647 only |\\\\r\\\\n\\\\r\\\\n- Customers can request a Private/Transit VIF ASN from the 4,200,000,000\\u20134,294,967,294 range via a support case (non-standard, requires internal approval)\\\\r\\\\n\\\\r\\\\n### SiteLink & BGP Communities\\\\r\\\\n\\\\r\\\\n- SiteLink **maintains BGP community tags** \\u2014 tags are passed through from one connected peer to another\\\\r\\\\n- This means communities set on one DX location will be visible at another DX location connected via SiteLink\\\\r\\\\n\\\\r\\\\n## VIF-Specific Considerations\\\\r\\\\n\\\\r\\\\n### Public VIF\\\\r\\\\n\\\\r\\\\n- **Reverse-path forwarding (RPF) strict mode is enforced** \\u2014 validates that the source of traffic originated from the customer\\\\'s advertised prefix (prevents address spoofing)\\\\r\\\\n- **Do NOT use EC2 EIPs for Public VIF BGP peering** \\u2014 can cause routing issues. Customers should use:\\\\r\\\\n - Public IPs they own, OR\\\\r\\\\n - IPs assigned by their provider/AWS support\\\\r\\\\n- **EC2 BYOIP CIDRs are NOT in ip-ranges.json** (not part of Amazon address space), but they ARE advertised over DX Public VIF to customers\\\\r\\\\n- **S3-only Public VIF**: BGP prefix lists are available (GitHub repository) for filtering S3 traffic over DX Public VIF \\u2014 allows routing only S3 traffic through DX while sending other AWS traffic over internet to reduce costs\\\\r\\\\n\\\\r\\\\n### Private / Transit VIF\\\\r\\\\n\\\\r\\\\n- For Private/Transit VIFs, customers can use any IPv4 address (if not using AWS auto-generated IPs), but must ensure those IPs **do not conflict** with on-prem and/or VPC networks\\\\r\\\\n- When attaching VPCs with **AWS-assigned IPv6** to a TGW, these are discontinuous by default and **cannot be aggregated** for Direct Connect \\u2014 leads to DX prefix quota limitations\\\\r\\\\n - **Workarounds:** Use Amazon-provided contiguous IPv6 CIDR blocks, or EC2 BYOIP\\\\r\\\\n\\\\r\\\\n### VPC-to-VPC Routing via DXGW (Regression)\\\\r\\\\n\\\\r\\\\n- As of November 2021, **VPC-to-VPC routing** connected to the same DXGW via VGW/TGW which are also connected over Private/Transit VIF **is possible due to a regression**\\\\r\\\\n- This was previously blocked \\u2014 be aware of this change\\\\r\\\\n- Workarounds exist to block this unintended traffic flow if isolation is required\\\\r\\\\n\\\\r\\\\n### DX VIFs and VGW \\u2014 CloudHub Behavior\\\\r\\\\n\\\\r\\\\n- **VPN CloudHub is a Site-to-Site VPN feature**: multiple VPN connections terminating on the same VGW can exchange routes with each other, letting remote sites communicate through AWS. See [AWS VPN CloudHub](https://docs.aws.amazon.com/vpc/latest/userguide/vpn-connections.html).\\\\r\\\\n- **DX can participate in a CloudHub with VPN**: when a DX private VIF and one or more VPN connections share the same VGW, prefixes learned from a VPN customer gateway are advertised to the DX customer gateway and vice versa \\u2014 so a DX site and VPN site(s) on the same VGW can reach each other.\\\\r\\\\n- **DX-to-DX does NOT hub through a VGW**: if two DX VIFs from different on-prem sites terminate on the same VGW (or DX Gateway), AWS does not re-advertise one site\\\\'s on-prem prefixes to the other. AWS advertises only the VPC CIDR(s) toward each VIF, so the two on-prem sites cannot reach each other through AWS. Site-to-site transit requires the customer\\\\'s own router (hairpin) or a design that isn\\\\'t a single shared gateway.\\\\r\\\\n- **DX Gateway (VGW association) advertises only VPC CIDRs to on-prem** \\u2014 it does not provide CloudHub between gateway associations; associations on the same DXGW cannot send traffic to each other.\\\\r\\\\n- **Floating VGW (VGW not attached to a VPC):**\\\\r\\\\n - *With VPN* \\u2014 a floating VGW can still act as a VPN CloudHub: multiple VPN connections on it exchange routes, so remote sites reach each other through AWS with no VPC involved.\\\\r\\\\n - *With DXGW* \\u2014 not a usable scenario. A DXGW advertises associated VPC CIDRs to on-prem; a floating VGW has no VPC, so there is nothing to advertise. DXGW-over-VGW is only meaningful when the VGW is attached to a VPC.\\\\r\\\\n\\\\r\\\\n**Note:** This corrects an earlier claim that DX VIFs on a shared VGW form a DX-to-DX hub-and-spoke mesh. Route sharing between on-prem sites on a shared gateway is a VPN CloudHub behavior (VPN connections), not a DX-to-DX behavior. Validate specific flows in a lab before relying on them.\\\\r\\\\n\\\\r\\\\n### When Can VIFs Talk to Each Other? (Site-to-Site via AWS)\\\\r\\\\n\\\\r\\\\n**Governing principle:** AWS does not act as a transit hub between VIFs by default. Over a VGW or a single DXGW, AWS advertises only the **VPC CIDRs** toward each VIF, so two on-prem sites (each on its own VIF) do not learn each other\\\\'s prefixes unless something explicitly enables it.\\\\r\\\\n\\\\r\\\\n**Supported \\u2014 VIFs (sites) CAN reach each other:**\\\\r\\\\n\\\\r\\\\n| Scenario | How it works |\\\\r\\\\n|---|---|\\\\r\\\\n| VPN CloudHub on a VGW | Multiple VPN connections on the same VGW exchange routes; remote sites reach each other. See [AWS VPN CloudHub](https://docs.aws.amazon.com/vpc/latest/userguide/vpn-connections.html). |\\\\r\\\\n| DX + VPN on the same VGW | A DX private VIF sharing a VGW with VPN connection(s) can talk to the VPN site(s) \\u2014 DX \\u2194 VPN. |\\\\r\\\\n| TGW + Allowed Prefix List | The DXGW\\u2192TGW attachment advertises whatever is in the Allowed Prefix List (no VPC-CIDR-only restriction), so remote-site prefixes can be advertised intentionally. Enables DX \\u2194 VPN CloudHub via TGW. |\\\\r\\\\n| DX \\u2194 DX via two DXGWs into one TGW | Two DXGWs attached to the same TGW (CGW1\\u2013DXGW1\\u2013TGW\\u2013DXGW2\\u2013CGW2) allows DX site-to-site. |\\\\r\\\\n| Customer-side hairpin | Always available: advertise a supernet/default from on-prem and let the customer router be the hub. AWS is not the transit point. |\\\\r\\\\n\\\\r\\\\n**Not supported \\u2014 sites CANNOT reach each other through AWS:**\\\\r\\\\n\\\\r\\\\n| Scenario | Why |\\\\r\\\\n|---|---|\\\\r\\\\n| Two DX VIFs on the same VGW | AWS advertises only VPC CIDRs to each VIF; no site-to-site. |\\\\r\\\\n| Two DX VIFs on the same single DXGW | Gateway associations on one DXGW cannot send traffic to each other. |\\\\r\\\\n| Inter-site over a single DXGW with transit VIFs | Routes may install but the flow loops; not supported. Use two DXGWs. |\\\\r\\\\n| Inter-site via two DXGWs with VGW associations | Only VPC CIDRs are advertised, so the sites never learn each other. |\\\\r\\\\n\\\\r\\\\n**Summary:**\\\\r\\\\n- **VGW / single DXGW** \\u2192 advertises VPC CIDRs only \\u2192 no VIF-to-VIF transit (except true VPN CloudHub, which is a VPN feature).\\\\r\\\\n- **TGW** \\u2192 Allowed Prefix List with no route validation \\u2192 VIF-to-VIF transit can be built intentionally, using two DXGWs for DX-to-DX.\\\\r\\\\n\\\\r\\\\n## DX vs Peering (Public VIF vs PNI)\\\\r\\\\n\\\\r\\\\n### Key Differences\\\\r\\\\n\\\\r\\\\n| Attribute | DX Public VIF | Public/Private Peering (PNI) |\\\\r\\\\n|---|---|---|\\\\r\\\\n| Official AWS service | Yes | No |\\\\r\\\\n| SLA | Yes | No |\\\\r\\\\n| AWS Support coverage | Yes | No (handled by Global Interconnect Strategy team) |\\\\r\\\\n| Internal LOCAL_PREF | Higher | Lower |\\\\r\\\\n| Path preference | **Preferred** (higher LP) | Lower priority |\\\\r\\\\n\\\\r\\\\n### Important Guidance\\\\r\\\\n\\\\r\\\\n- If the **same prefix** is received by AWS over both DX Public VIF and Peering (PNI, IXP, or Transit), **DX Public VIF is the preferred path** (higher internal LOCAL_PREF)\\\\r\\\\n- Public Peering / PNI is NOT an official AWS service \\u2014 no SLA, no support expectation from the technical field\\\\r\\\\n- **Steer customers toward DX** when they need reliability and support coverage\\\\r\\\\n- There is no expectation for the technical field to support Public Peering / PNI engagements\\\\r\\\\n\\\\r\\\\n## Design Considerations for Active/Active & Active/Passive\\\\r\\\\n\\\\r\\\\n### General Best Practice for DX Routing Selection\\\\r\\\\n\\\\r\\\\n- **Always use LP communities (`7224:7300/7200/7100`) for expected egress routing** \\u2014 High Preference BGP communities are the recommended mechanism for predictable DX egress.\\\\r\\\\n- **AS-path prepending: within a region ONLY** \\u2014 Use AS-path prepending to influence egress between DX connections belonging to the **same associated AWS region**. Do NOT use AS-path prepending to influence traffic across regions \\u2014 DXGW\\\\'s local-region LP makes cross-region prepending unreliable.\\\\r\\\\n - **When comparing AS-path lengths, distinguish an originating site from a transiting one** \\u2014 a site that learns a prefix over the on-prem WAN and re-advertises it carries the origin ASN in its path, so its total length is prepend-count + origin ASN(s), not prepend-count alone. Confirm the real path with `list-virtual-interface-routes`. See \\\"Confusing Prepend Count with Total AS-Path Length\\\" in cloudwan-dx-routing-patterns.md.\\\\r\\\\n- **BGP communities: within region OR across regions** \\u2014 LP communities work for both intra-region path selection and cross-region preference (overriding DXGW\\\\'s default local-region LP).\\\\r\\\\n\\\\r\\\\n### Active/Active (within or across DX locations)\\\\r\\\\n\\\\r\\\\n- Understand how AWS selects between multiple active paths using the Private/Transit VIF outbound path-selection order (longest prefix match \\u2192 local preference \\u2192 AS_PATH length \\u2192 MED \\u2192 ECMP). See [Direct Connect routing policies and BGP communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html).\\\\r\\\\n- ECMP is available at the TGW level across multiple DX-GW attachments (same or different DX-GWs)\\\\r\\\\n- LP communities (`7224:7300/7200/7100`) can influence preference but behavior varies between VGW/TGW/Cloud WAN \\u2014 always test\\\\r\\\\n- **DXGW ECMPs across equal remote paths** \\u2014 when a local DX fails and multiple remote DX locations have matching attributes (same LP, same AS-path), DXGW will ECMP across them. This provides load-balanced failover rather than single-path selection.\\\\r\\\\n\\\\r\\\\n**Path selection behavior by service (NOT at DXGW):**\\\\r\\\\n- **DXGW:** ECMPs across equal paths \\u2014 load-balanced failover\\\\r\\\\n- **TGW:** Deterministic \\u2014 selects oldest route for equal paths. Consistent but not customer-controllable.\\\\r\\\\n- **Cloud WAN CNE:** Deterministic \\u2014 \\\"deterministically random\\\" at Step 2.4.4 for equal remote CNE paths. Consistent but not customer-controllable.\\\\r\\\\n\\\\r\\\\n### Active/Passive\\\\r\\\\n\\\\r\\\\n- Use LP communities or AS-path prepending to create primary/secondary paths\\\\r\\\\n- For Cloud WAN: communities override DXGW local-region preference (see cloudwan-dx-routing-patterns.md)\\\\r\\\\n- For TGW: LP communities are NOT honored across multiple DX-GW attachments (see tgw-routing-patterns.md)\\\\r\\\\n- BFD should be enabled for fast failure detection on the active path\\\\r\\\\n'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:24.227000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "c535feca-7122-4e76-a70f-34dc696d9683", + "executionId": "cb665d99-08df-4a4d-ad7f-af59d394c298", + "recordId": "8b1b84c2-8cba-480b-8132-ae7cdc8a30a5", + "content": "{\"id\": \"8b1b84c2-8cba-480b-8132-ae7cdc8a30a5\", \"role\": \"assistant\", \"content\": [{\"text\": \"How you set this up depends on which construct is evaluating the route (VGW, TGW, or Cloud WAN), since the recommended mechanism and its caveats differ meaningfully between them. Let me make sure I point you at the right approach.\", \"type\": \"text\"}, {\"id\": \"tooluse_nIBgZkMg0xAOWT00bCMh7r\", \"tool_name\": \"ask_user\", \"input\": {\"question\": \"Which AWS construct are your two Direct Connect locations attached to?\", \"options\": [{\"label\": \"Direct Connect Gateway + VGW\", \"description\": \"VIFs associate to a Virtual Private Gateway (classic DXGW-to-VGW setup)\", \"recommended\": true, \"recommended_reason\": \"Most common setup; LP communities work reliably here for A/B preference\"}, {\"label\": \"Direct Connect Gateway + Transit Gateway\", \"description\": \"VIFs associate to a DX Gateway attached to a Transit Gateway\"}, {\"label\": \"Cloud WAN (Core Network Edge)\", \"description\": \"VIFs feed into a Cloud WAN core network edge rather than a TGW or VGW\"}]}, \"type\": \"tool_use\", \"display_intent\": \"Ask which gateway construct terminates the two Direct Connect VIFs\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:24.303000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "c535feca-7122-4e76-a70f-34dc696d9683", + "executionId": "cb665d99-08df-4a4d-ad7f-af59d394c298", + "recordId": "5af820eb-d760-4fca-95e4-fef068e819f5", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.9, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:24.396000-05:00", + "recordType": "utilization" + } +] \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/dx-prefer-one-location/without_skill/functional-tests-results.json b/skills/aws-routing/evals/functional/v1/iteration-1/dx-prefer-one-location/without_skill/functional-tests-results.json new file mode 100644 index 00000000..53bba399 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/dx-prefer-one-location/without_skill/functional-tests-results.json @@ -0,0 +1,70 @@ +{ + "version": "v1", + "iteration": 1, + "eval_id": "dx-prefer-one-location", + "run_type": "without_skill", + "success": true, + "error": null, + "understanding_agent_space_skill": null, + "tests": { + "trigger": null, + "expected_output": { + "result": "passed", + "reasoning": "The agent's response correctly recommends using BGP local preference community tags (7224:7300 for primary/high, 7224:7100 for backup/low) as the recommended primary method for traffic preference. It also correctly explains the cross-region caveat: when VIFs span different AWS Regions attached to one Direct Connect Gateway, AWS always prefers the same-Region connection regardless of AS-PATH prepending, and recommends using local preference community tags instead in that topology. This matches the expected output's core points: LP communities (7224:7300 primary, lower backup) and the unreliability of cross-region AS-path prepending due to DXGW local-region LP preference.", + "evidence": "\"Tag prefixes advertised from Location A with `7224:7300` (high) and from Location B with `7224:7100` (low).\" and \"Don't rely on AS-PATH prepending alone if VIFs span different AWS Regions attached to one Direct Connect Gateway... AWS will always prefer the same-Region connection regardless of AS-PATH \u2014 prepending won't override this. Use local preference community tags instead in this topology.\"", + "confidence": "high" + }, + "assertions": { + "assertion_results": [ + { + "text": "recommends LP communities with a higher preference on the primary", + "evaluator": "llm", + "passed": true, + "evidence": "\"Tag prefixes advertised from Location A with `7224:7300` (high) and from Location B with `7224:7100` (low). AWS evaluates local preference first, so this wins regardless of path length.\"", + "reasoning": "The agent explicitly recommends using local preference BGP community tags with a higher value (7224:7300) on Location A (the primary) and lower (7224:7100) on Location B (the backup).", + "confidence": "high" + }, + { + "text": "explains AS-path prepending is reliable only within the same region", + "evaluator": "llm", + "passed": true, + "evidence": "\"Don't rely on AS-PATH prepending alone if VIFs span different AWS Regions attached to one Direct Connect Gateway... AWS will always prefer the same-Region connection regardless of AS-PATH \u2014 prepending won't override this. Use local preference community tags instead in this topology.\"", + "reasoning": "The agent explains that AS-path prepending is overridden by regional proximity when VIFs span different regions, implying it is only reliable when the comparison is within the same region (same-region connections are preferred regardless of AS-path). This matches the substance of the assertion.", + "confidence": "medium" + }, + { + "text": "warns cross-region AS-path prepending is unreliable", + "evaluator": "llm", + "passed": true, + "evidence": "\"Don't rely on AS-PATH prepending alone if VIFs span different AWS Regions attached to one Direct Connect Gateway... prepending won't override this.\"", + "reasoning": "The agent directly warns that cross-region AS-path prepending is unreliable, since AWS prefers same-region connections regardless of AS-path in that scenario.", + "confidence": "high" + }, + { + "text": "does not contain 'error'", + "evaluator": "llm", + "passed": true, + "evidence": "No occurrence of the word 'error' found anywhere in the output text.", + "reasoning": "A review of the full output text shows no instance of the word 'error'.", + "confidence": "high" + } + ], + "summary": { + "passed": 4, + "failed": 0, + "errored": 0, + "low_confidence": 0, + "total": 4, + "pass_rate": 1.0 + } + } + }, + "metrics": { + "runtime": "15s", + "cost": "$0.13", + "context_window": { + "utilization": "4.2%", + "compaction_count": 0 + } + } +} \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/dx-prefer-one-location/without_skill/outputs/journal_records.json b/skills/aws-routing/evals/functional/v1/iteration-1/dx-prefer-one-location/without_skill/outputs/journal_records.json new file mode 100644 index 00000000..fd65454f --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/dx-prefer-one-location/without_skill/outputs/journal_records.json @@ -0,0 +1,90 @@ +[ + { + "agentSpaceId": "d91153a7-b5ec-45e4-8906-f91a68222ea6", + "executionId": "5c997e74-7770-4743-997d-87cafcca60e3", + "recordId": "355dd3cf-2de8-48a4-a2c0-6e9c5edb7815", + "content": "{\"id\": \"355dd3cf-2de8-48a4-a2c0-6e9c5edb7815\", \"role\": \"user\", \"content\": [{\"text\": \"I have two Direct Connect locations and want traffic to prefer location A over location B, with B as backup. How do I do this and what should I avoid?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:04.973000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "d91153a7-b5ec-45e4-8906-f91a68222ea6", + "executionId": "5c997e74-7770-4743-997d-87cafcca60e3", + "recordId": "66721f82-ec5d-4019-b977-11c87671ef57", + "content": "{\"id\": \"817218b5-20a2-4ccd-9b1d-b8b9afe1c404\", \"role\": \"assistant\", \"content\": [{\"type\": \"chat_title\", \"title\": \"AWS Direct Connect traffic preference and failover\"}]}", + "createdAt": "2026-10-27T17:05:05.407000-05:00", + "recordType": "chat_title" + }, + { + "agentSpaceId": "d91153a7-b5ec-45e4-8906-f91a68222ea6", + "executionId": "5c997e74-7770-4743-997d-87cafcca60e3", + "recordId": "7ad6697b-2521-44f6-9af4-9cd66a92f57b", + "content": "{\"id\": \"e5461b0d-e16c-4936-9812-249ad115588f\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This is actually a general AWS Direct Connect networking question rather than something specific to the DevOps Agent product itself \\u2014 it's asking about BGP routing preference mechanisms like AS_PATH prepending, Local Preference, MED, and BGP Communities.\\n\\nSince this doesn't fit under the product_knowledge category, I should verify the accuracy of my answer against current AWS documentation using verify_aws_claim rather than relying on potentially outdated training data about Direct Connect routing preference best practices.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:11.755000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "d91153a7-b5ec-45e4-8906-f91a68222ea6", + "executionId": "5c997e74-7770-4743-997d-87cafcca60e3", + "recordId": "6afce6ba-a7d7-4539-9304-8a46170975b5", + "content": "{\"id\": \"6afce6ba-a7d7-4539-9304-8a46170975b5\", \"role\": \"assistant\", \"content\": [{\"text\": \"Let me check the latest AWS Direct Connect guidance on routing preference between two connections.\", \"type\": \"text\"}, {\"id\": \"tooluse_78TVgme6xL7IBOD6905ow5\", \"tool_name\": \"verify_aws_claim\", \"input\": {\"claim\": \"AWS Direct Connect active/passive routing preference between two locations using BGP AS-PATH prepending and local preference\"}, \"type\": \"tool_use\", \"display_intent\": \"Look up AWS docs on Direct Connect active/passive BGP routing preference\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:11.781000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "d91153a7-b5ec-45e4-8906-f91a68222ea6", + "executionId": "5c997e74-7770-4743-997d-87cafcca60e3", + "recordId": "d90bea2f-54b8-48c9-beaf-9ee484840147", + "content": "{\"id\": \"e9f63c7d-3f08-4771-9b21-54762411bfbc\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"2422\"}]}", + "createdAt": "2026-10-27T17:05:11.859000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "d91153a7-b5ec-45e4-8906-f91a68222ea6", + "executionId": "5c997e74-7770-4743-997d-87cafcca60e3", + "recordId": "eb3a049d-e657-4ce4-ad94-72fb1c766dc0", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 3.8, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:11.874000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "d91153a7-b5ec-45e4-8906-f91a68222ea6", + "executionId": "5c997e74-7770-4743-997d-87cafcca60e3", + "recordId": "0e92b97d-d33a-42ea-8a80-2b6226cfd2d1", + "content": "{\"id\": \"06f84e45-25a7-4cc5-979e-60beb2e6cc5d\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_78TVgme6xL7IBOD6905ow5\", \"status\": \"success\", \"content\": [{\"text\": \"## Documentation search results\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"How do I set up an Active/Passive Direct Connect connection to AWS?\\\",\\\"context\\\":\\\"### Influencing inbound traffic to an on-premises location from AWS\\\\n\\\\nFor an **Active/Passive** configuration of Direct Connect connections:\\\\n\\\\n* Apply the [local preference BGP community tag](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html#bgp-communities-private-transit). Set a higher preference to the advertised prefixes for the primary or active connection. Then, set a medium or lower preference for the passive connection.\\\\n* AS Path prepend using a shorter AS path on the active connection and a longer AS path on the passive connection.\\\\n* Advertise the most specific route using BGP on the active connection.\\\\n\\\\n**Note:**\\u00a0AS Path prepending can't be used to configure Active/Passive connections in the following scenarios:\\\\n\\\\n* Connection A (virtual interface VIF-A) is in Region 1.\\\\n* Connection B (virtual interface VIF-B) is in Region 2.\\\\n* Both virtual interfaces connect to a VPC in Region 1 using a Direct Connect gateway.\\\\n* Both virtual interfaces advertise the same prefixes with the same BGP attributes (such as AS Path and MED) on both connections from the on-premises location.\\\\n\\\\nFor these scenarios, outbound traffic from the VPC to the on-premises location prefers Connection A due to being in the same Region as the VPC. To setup Active/Passive Direct Connect connections in this scenario, apply a [local preference BGP community tag](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html#bgp-communities-private-transit). Also, you can advertise a more specific route on an active connection\\\",\\\"url\\\":\\\"https://repost.aws/knowledge-center/direct-connect-active-passive-connection\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"Active/Active and Active/Passive Configurations in AWS Direct Connect\\\",\\\"context\\\":\\\"## Active/Passive with Private/Transit VIF Diagram\\\\n\\\\nBuild Active/Passive configuration with Transit/Private VIF for max\\\\nresiliency. Have redundant Direct Connect connections inside each\\\\nDirect Connect location as well as across locations, customer\\\\ndata centers and devices. This configuration offers customers\\\\nmax resilience to failure. Such a topology ensures resilience\\\\nto connectivity failure due to a fiber cut or a device\\\\nfailure as well as a complete location failure.\\\\n\\\\n* **Transit/Private VIF**: You can\\\\n create Active/Passive setups in multiple ways by controlling\\\\n advertised prefixes, local preference, AS path and MED value.\\\\n With that you influence incoming traffic from AWS. These options\\\\n are not mutually exclusive and can be used together.\\\\n* **Active/Passive**: One connection\\\\n handles traffic, and the others are on standby. If the active\\\\n connection becomes unavailable, then all traffic is routed\\\\n through the passive connections.\\\\n\\\\n1. **Direct Connect** gateway is a global construct\\\\n and AWS takes care of its availability.\\\\n2. When you request multiple ports at the same **Direct Connect**\\\\n location, they will be provisioned on redundant AWS equipment.\\\\n3. Have the redundant **Direct Connect** connections\\\\n terminate on different customer/partner routers inside the\\\\n **Direct Connect** location.\\\\n4. **Direct Connect** connections have to be\\\\n brought down for regular scheduled maintenance. Therefore, use\\\\n multiple connections.\\\\n5. Have the redundant **Direct Connect** connections\\\\n terminate on different routers inside your data center.\\\\n6. Leverage multiple **Direct Connect** connections\\\\n across multiple **Direct Connect** locations.\\\\n7. Leverage multiple **Direct Connect** connections\\\\n across multiple customer sites.\\\\n8. By advertising more specific prefixes you can prefer\\\\n one **Direct Connect** connection over the others.\\\\n9. If prefixes are the same, a higher local preference is preferred.\\\\n By using BGP communities you can set a local preference\\\\n from 7224:7100/7200/7300 where 7100 is low, 7200 is medium,\\\\n and 7300 is high preference. Use a higher local preference to\\\\n prefer a specific **Direct Connect**\\\\n connection over others.\\\\n10. If local preference is the same, shortest AS paths are\\\\n preferred. You can use AS path prepending for\\\\n **Direct Connect** connections\\\\n associated with\\\\n\\\\n*[truncated \\u2014 additional content omitted]*\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/reference-architecture-diagrams/latest/active-active-and-active-passive-configurations-in-aws-direct-connect/active-active-and-active-passive-configurations-in-aws-direct-connect.html\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"How can I use BGP communities to influence the preferred routing path on Direct Connect links from AWS to my network?\\\",\\\"context\\\":\\\"### Apply community tags and prepend AS\\\\\\\\_PATH attributes\\\\n\\\\nDirect Connect supports the following local preference BGP community tags:\\\\n\\\\n* 7224:7100 Low preference\\\\n* 7224:7200 Medium preference\\\\n* 7224:7300 High preference\\\\n\\\\nDirect Connect evaluates [local preference BGP community tags](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html#local-pref-bgp-communities) from lowest to highest preference.\\\\n\\\\nTo use community tags and AS\\\\\\\\_PATH attributes, take the following actions:\\\\n\\\\n* For each prefix that you advertise over a BGP session, apply a community tag to show the associated path's priority for returning traffic.\\\\n* When you advertise from an on-premises router to AWS, prepend additional AS\\\\\\\\_PATH attributes to the prefixes. Direct Connect prioritizes the path with the fewest AS\\\\\\\\_PATH attributes for returning traffic.\\\\n\\\\nFor example, you have virtual interfaces A and B, and you're advertising prefixes 10.10.10.0/24 and 10.20.20.0/24 to AWS. To make virtual interface A the preferred routing path for traffic to network 10.10.10.0/24, apply higher preference tag 7224:7300 to prefix 10.10.10.0/24 on the BGP session. To make virtual interface B the preferred routing path for traffic to network 10.20.20.0/24, apply higher preference tag 7224:7300 to prefix 10.20.20.0/24 on the BGP session.\\\\n\\\\nIf you don't use BGP community tags to specify local preference for private virtual interfaces, then the outbound routing behavior defaults to an AWS Region. In this case, the behavior is based on the Direct Connect location's relative distance to the originating Region. If the relative distance costs aren't equal, then [Autonomous System (AS) prepending](https://docs.aws.amazon.com/directconnect/latest/UserGuide/WorkingWithVirtualInterfaces.html#vif-prerequisites) can't influence routing behavior. Direct Connect evaluates the lowest distance relative cost before you prepend AS\\\\\\\\_PATH.\\\\n\\\\nTo load balance traffic across multiple AWS Direct Connect connections, apply the same community tag across the prefixes for the connections\\\",\\\"url\\\":\\\"https://repost.aws/knowledge-center/direct-connect-bgp-communities\\\"}]}}\\n\\n## Recent AWS announcements (What's New)\\n\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\n\\\\n\\\\nServices: directconnect, vpc, transitgateway\\\\nTasks: configure, secure, deploy\\\\nPersona: network-admin, devops, developer, sre\\\\nWorkload: networking, hybrid-cloud\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"aws-networking\\\",\\\"skill_description\\\":\\\"Routes AWS networking requests to the correct service skill for implementation. Covers Route 53 (DNS, health checks, routing policies, Resolver, DNS Firewall), CloudFront (caching, edge, OAC, mTLS, signed URLs), Transit Gateway (multi-VPC hub, segmentation, centralized egress), Direct Connect (hybrid link, DX Gateway, MACsec), Site-to-Site VPN (IPsec tunnels, static or BGP), WAF (web ACLs, AWS Managed Rules, rate-based rules, Bot and Fraud Control), and Shield Advanced (L3/L4 DDoS). Applicable when creating, configuring, troubleshooting, or designing across these services, choosing between them, or diagnosing connectivity or traffic-filtering issues. Not for VPC subnets and route tables, load balancers, VPC endpoints, PrivateLink, API Gateway, IAM policy logic, container or serverless networking, or IaC authoring.\\\\n\\\\nServices: route53, route53resolver, cloudfront, directconnect, transitgateway, vpn, waf, shield\\\\nTasks: configure, troubleshoot, debug, deploy, secure, migrate\\\\nPersona: developer, network-engineer, devops, security-engineer\\\\nWorkload: networking, security\\\\n*\\\",\\\"skill_name\\\":\\\"aws-networking\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"transitgateway\\\",\\\"skill_description\\\":\\\"Configures AWS Transit Gateway: creating a hub and attaching VPCs, segmenting traffic with route tables, centralizing egress and forcing east-west traffic between VPCs through a central firewall via transit gateway route tables and appliance-mode attachments, connecting on-premises networks over the transit gateway side of a Site-to-Site VPN or Direct Connect attachment (including ECMP to aggregate bandwidth across multiple VPN tunnels), peering transit gateways across Regions, migrating from a VPC peering mesh, and routing IP multicast. Applicable when the transit gateway is the resource being configured: connecting many VPCs, isolating environments, routing egress or east-west traffic through the hub for inspection, reaching on-premises over the hub, linking Regions, or moving off a peering mesh. Not applicable for single-VPC routing, VPC peering between two VPCs (vpcpeering skill), Direct Connect gateway or virtual interface setup (directconnect skill), or Route 53 DNS work.\\\\n\\\\n\\\\nServices: transitgateway, vpc, directconnect, sitetositevpn, gatewayloadbalancer\\\\nTasks: configure, secure, deploy, migrate\\\\nPersona: developer, devops, network-admin, security-engineer, sre\\\\nWorkload: networking, security, hybrid, multi-region\\\\n*\\\",\\\"skill_name\\\":\\\"transitgateway\\\"}]}}\"}]}], \"label\": \"Look up AWS docs on Direct Connect active/passive BGP routing preference\"}", + "createdAt": "2026-10-27T17:05:13.451000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "d91153a7-b5ec-45e4-8906-f91a68222ea6", + "executionId": "5c997e74-7770-4743-997d-87cafcca60e3", + "recordId": "01dda08c-c5aa-4086-82c6-17ff9a9a478d", + "content": "{\"id\": \"01dda08c-c5aa-4086-82c6-17ff9a9a478d\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_78TVgme6xL7IBOD6905ow5\", \"content\": \"[{'text': '## Documentation search results\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"How do I set up an Active/Passive Direct Connect connection to AWS?\\\",\\\"context\\\":\\\"### Influencing inbound traffic to an on-premises location from AWS\\\\\\\\n\\\\\\\\nFor an **Active/Passive** configuration of Direct Connect connections:\\\\\\\\n\\\\\\\\n* Apply the [local preference BGP community tag](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html#bgp-communities-private-transit). Set a higher preference to the advertised prefixes for the primary or active connection. Then, set a medium or lower preference for the passive connection.\\\\\\\\n* AS Path prepend using a shorter AS path on the active connection and a longer AS path on the passive connection.\\\\\\\\n* Advertise the most specific route using BGP on the active connection.\\\\\\\\n\\\\\\\\n**Note:**\\\\xa0AS Path prepending can\\\\'t be used to configure Active/Passive connections in the following scenarios:\\\\\\\\n\\\\\\\\n* Connection A (virtual interface VIF-A) is in Region 1.\\\\\\\\n* Connection B (virtual interface VIF-B) is in Region 2.\\\\\\\\n* Both virtual interfaces connect to a VPC in Region 1 using a Direct Connect gateway.\\\\\\\\n* Both virtual interfaces advertise the same prefixes with the same BGP attributes (such as AS Path and MED) on both connections from the on-premises location.\\\\\\\\n\\\\\\\\nFor these scenarios, outbound traffic from the VPC to the on-premises location prefers Connection A due to being in the same Region as the VPC. To setup Active/Passive Direct Connect connections in this scenario, apply a [local preference BGP community tag](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html#bgp-communities-private-transit). Also, you can advertise a more specific route on an active connection\\\",\\\"url\\\":\\\"https://repost.aws/knowledge-center/direct-connect-active-passive-connection\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"Active/Active and Active/Passive Configurations in AWS Direct Connect\\\",\\\"context\\\":\\\"## Active/Passive with Private/Transit VIF Diagram\\\\\\\\n\\\\\\\\nBuild Active/Passive configuration with Transit/Private VIF for max\\\\\\\\nresiliency. Have redundant Direct Connect connections inside each\\\\\\\\nDirect Connect location as well as across locations, customer\\\\\\\\ndata centers and devices. This configuration offers customers\\\\\\\\nmax resilience to failure. Such a topology ensures resilience\\\\\\\\nto connectivity failure due to a fiber cut or a device\\\\\\\\nfailure as well as a complete location failure.\\\\\\\\n\\\\\\\\n* **Transit/Private VIF**: You can\\\\\\\\n create Active/Passive setups in multiple ways by controlling\\\\\\\\n advertised prefixes, local preference, AS path and MED value.\\\\\\\\n With that you influence incoming traffic from AWS. These options\\\\\\\\n are not mutually exclusive and can be used together.\\\\\\\\n* **Active/Passive**: One connection\\\\\\\\n handles traffic, and the others are on standby. If the active\\\\\\\\n connection becomes unavailable, then all traffic is routed\\\\\\\\n through the passive connections.\\\\\\\\n\\\\\\\\n1. **Direct Connect** gateway is a global construct\\\\\\\\n and AWS takes care of its availability.\\\\\\\\n2. When you request multiple ports at the same **Direct Connect**\\\\\\\\n location, they will be provisioned on redundant AWS equipment.\\\\\\\\n3. Have the redundant **Direct Connect** connections\\\\\\\\n terminate on different customer/partner routers inside the\\\\\\\\n **Direct Connect** location.\\\\\\\\n4. **Direct Connect** connections have to be\\\\\\\\n brought down for regular scheduled maintenance. Therefore, use\\\\\\\\n multiple connections.\\\\\\\\n5. Have the redundant **Direct Connect** connections\\\\\\\\n terminate on different routers inside your data center.\\\\\\\\n6. Leverage multiple **Direct Connect** connections\\\\\\\\n across multiple **Direct Connect** locations.\\\\\\\\n7. Leverage multiple **Direct Connect** connections\\\\\\\\n across multiple customer sites.\\\\\\\\n8. By advertising more specific prefixes you can prefer\\\\\\\\n one **Direct Connect** connection over the others.\\\\\\\\n9. If prefixes are the same, a higher local preference is preferred.\\\\\\\\n By using BGP communities you can set a local preference\\\\\\\\n from 7224:7100/7200/7300 where 7100 is low, 7200 is medium,\\\\\\\\n and 7300 is high preference. Use a higher local preference to\\\\\\\\n prefer a specific **Direct Connect**\\\\\\\\n connection over others.\\\\\\\\n10. If local preference is the same, shortest AS paths are\\\\\\\\n preferred. You can use AS path prepending for\\\\\\\\n **Direct Connect** connections\\\\\\\\n associated with\\\\\\\\n\\\\\\\\n*[truncated \\u2014 additional content omitted]*\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/reference-architecture-diagrams/latest/active-active-and-active-passive-configurations-in-aws-direct-connect/active-active-and-active-passive-configurations-in-aws-direct-connect.html\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"How can I use BGP communities to influence the preferred routing path on Direct Connect links from AWS to my network?\\\",\\\"context\\\":\\\"### Apply community tags and prepend AS\\\\\\\\\\\\\\\\_PATH attributes\\\\\\\\n\\\\\\\\nDirect Connect supports the following local preference BGP community tags:\\\\\\\\n\\\\\\\\n* 7224:7100 Low preference\\\\\\\\n* 7224:7200 Medium preference\\\\\\\\n* 7224:7300 High preference\\\\\\\\n\\\\\\\\nDirect Connect evaluates [local preference BGP community tags](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html#local-pref-bgp-communities) from lowest to highest preference.\\\\\\\\n\\\\\\\\nTo use community tags and AS\\\\\\\\\\\\\\\\_PATH attributes, take the following actions:\\\\\\\\n\\\\\\\\n* For each prefix that you advertise over a BGP session, apply a community tag to show the associated path\\\\'s priority for returning traffic.\\\\\\\\n* When you advertise from an on-premises router to AWS, prepend additional AS\\\\\\\\\\\\\\\\_PATH attributes to the prefixes. Direct Connect prioritizes the path with the fewest AS\\\\\\\\\\\\\\\\_PATH attributes for returning traffic.\\\\\\\\n\\\\\\\\nFor example, you have virtual interfaces A and B, and you\\\\'re advertising prefixes 10.10.10.0/24 and 10.20.20.0/24 to AWS. To make virtual interface A the preferred routing path for traffic to network 10.10.10.0/24, apply higher preference tag 7224:7300 to prefix 10.10.10.0/24 on the BGP session. To make virtual interface B the preferred routing path for traffic to network 10.20.20.0/24, apply higher preference tag 7224:7300 to prefix 10.20.20.0/24 on the BGP session.\\\\\\\\n\\\\\\\\nIf you don\\\\'t use BGP community tags to specify local preference for private virtual interfaces, then the outbound routing behavior defaults to an AWS Region. In this case, the behavior is based on the Direct Connect location\\\\'s relative distance to the originating Region. If the relative distance costs aren\\\\'t equal, then [Autonomous System (AS) prepending](https://docs.aws.amazon.com/directconnect/latest/UserGuide/WorkingWithVirtualInterfaces.html#vif-prerequisites) can\\\\'t influence routing behavior. Direct Connect evaluates the lowest distance relative cost before you prepend AS\\\\\\\\\\\\\\\\_PATH.\\\\\\\\n\\\\\\\\nTo load balance traffic across multiple AWS Direct Connect connections, apply the same community tag across the prefixes for the connections\\\",\\\"url\\\":\\\"https://repost.aws/knowledge-center/direct-connect-bgp-communities\\\"}]}}\\\\n\\\\n## Recent AWS announcements (What\\\\'s New)\\\\n\\\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: directconnect, vpc, transitgateway\\\\\\\\nTasks: configure, secure, deploy\\\\\\\\nPersona: network-admin, devops, developer, sre\\\\\\\\nWorkload: networking, hybrid-cloud\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"aws-networking\\\",\\\"skill_description\\\":\\\"Routes AWS networking requests to the correct service skill for implementation. Covers Route 53 (DNS, health checks, routing policies, Resolver, DNS Firewall), CloudFront (caching, edge, OAC, mTLS, signed URLs), Transit Gateway (multi-VPC hub, segmentation, centralized egress), Direct Connect (hybrid link, DX Gateway, MACsec), Site-to-Site VPN (IPsec tunnels, static or BGP), WAF (web ACLs, AWS Managed Rules, rate-based rules, Bot and Fraud Control), and Shield Advanced (L3/L4 DDoS). Applicable when creating, configuring, troubleshooting, or designing across these services, choosing between them, or diagnosing connectivity or traffic-filtering issues. Not for VPC subnets and route tables, load balancers, VPC endpoints, PrivateLink, API Gateway, IAM policy logic, container or serverless networking, or IaC authoring.\\\\\\\\n\\\\\\\\nServices: route53, route53resolver, cloudfront, directconnect, transitgateway, vpn, waf, shield\\\\\\\\nTasks: configure, troubleshoot, debug, deploy, secure, migrate\\\\\\\\nPersona: developer, network-engineer, devops, security-engineer\\\\\\\\nWorkload: networking, security\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"aws-networking\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"transitgateway\\\",\\\"skill_description\\\":\\\"Configures AWS Transit Gateway: creating a hub and attaching VPCs, segmenting traffic with route tables, centralizing egress and forcing east-west traffic between VPCs through a central firewall via transit gateway route tables and appliance-mode attachments, connecting on-premises networks over the transit gateway side of a Site-to-Site VPN or Direct Connect attachment (including ECMP to aggregate bandwidth across multiple VPN tunnels), peering transit gateways across Regions, migrating from a VPC peering mesh, and routing IP multicast. Applicable when the transit gateway is the resource being configured: connecting many VPCs, isolating environments, routing egress or east-west traffic through the hub for inspection, reaching on-premises over the hub, linking Regions, or moving off a peering mesh. Not applicable for single-VPC routing, VPC peering between two VPCs (vpcpeering skill), Direct Connect gateway or virtual interface setup (directconnect skill), or Route 53 DNS work.\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: transitgateway, vpc, directconnect, sitetositevpn, gatewayloadbalancer\\\\\\\\nTasks: configure, secure, deploy, migrate\\\\\\\\nPersona: developer, devops, network-admin, security-engineer, sre\\\\\\\\nWorkload: networking, security, hybrid, multi-region\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"transitgateway\\\"}]}}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:26.793000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "d91153a7-b5ec-45e4-8906-f91a68222ea6", + "executionId": "5c997e74-7770-4743-997d-87cafcca60e3", + "recordId": "25da268b-da73-43a9-9210-7f760c9b727e", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.2, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:26.877000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "d91153a7-b5ec-45e4-8906-f91a68222ea6", + "executionId": "5c997e74-7770-4743-997d-87cafcca60e3", + "recordId": "ab4317f5-fbd9-46ca-9842-f2d43086acc1", + "content": "{\"id\": \"ab4317f5-fbd9-46ca-9842-f2d43086acc1\", \"role\": \"assistant\", \"content\": [{\"text\": \"## Making Location A primary, Location B backup\\n\\nDirect Connect doesn't have an \\\"active/backup\\\" toggle \\u2014 you influence path selection with BGP attributes on the virtual interfaces at each location. Any of these (not mutually exclusive) will do it:\\n\\n**For outbound traffic from AWS to your network** (which path AWS uses to reach you):\\n- **Local preference BGP community tags** \\u2014 this is the recommended primary method. Tag prefixes advertised from Location A with `7224:7300` (high) and from Location B with `7224:7100` (low). AWS evaluates local preference first, so this wins regardless of path length.\\n- **AS-PATH prepending** \\u2014 as a secondary/fallback mechanism, prepend extra AS numbers on the routes advertised from Location B so its path looks longer. Only matters if local preference ties.\\n- **More specific prefixes** on the active connection also win, independent of the above.\\n\\n**For inbound traffic to your network from AWS** (which path your network prefers for return traffic):\\n- Advertise with local preference community tags and/or AS-PATH prepending from your router at Location B so AWS prefers sending to A.\\n- Advertise a more specific route from Location A.\\n\\nOrder of precedence Direct Connect uses: **most specific prefix \\u2192 local preference \\u2192 shortest AS path.** So local preference is the reliable lever; AS-PATH prepending alone can get overridden by prefix specificity or regional distance.\\n\\n## What to avoid\\n\\n- **Don't rely on AS-PATH prepending alone if VIFs span different AWS Regions attached to one Direct Connect Gateway.** If VIF-A is in Region 1 and VIF-B is in Region 2, but both reach a VPC in Region 1, AWS will always prefer the same-Region connection regardless of AS-PATH \\u2014 prepending won't override this. Use local preference community tags instead in this topology.\\n- **Don't advertise identical prefixes with identical attributes (AS path, MED, community) on both connections** \\u2014 that creates unintended load-balancing (ECMP) across both paths instead of a real active/passive split, which can cause asymmetric routing and odd troubleshooting if B is meant to be pure backup.\\n- **Don't mix inconsistent signals** \\u2014 e.g., tagging local preference high on B while also advertising a more-specific prefix on B. More-specific-prefix wins first, so it'll silently override your local preference intent.\\n- **Don't forget both directions** \\u2014 you need to set preference for outbound-from-AWS (community tags on your advertisements) AND for your own router's outbound-to-AWS path selection (your local BGP policy / AS-PATH on what you accept from AWS). Setting only one side gives asymmetric routing.\\n- **Test failover explicitly** \\u2014 bring down Location A's connection (or BGP session) and confirm traffic actually shifts to B and back cleanly, rather than assuming the config works.\\n\\nWant me to check your actual Direct Connect virtual interfaces and BGP configuration in this account to see what's currently advertised at each location?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:26.962000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "d91153a7-b5ec-45e4-8906-f91a68222ea6", + "executionId": "5c997e74-7770-4743-997d-87cafcca60e3", + "recordId": "4f99883d-1224-430b-bebe-5cb4695e0b4b", + "content": "{\"id\": \"e4e0978b-55cc-4d16-bb13-cde410e01fa0\", \"role\": \"assistant\", \"content\": [{\"type\": \"text\", \"text\": \"## Making Location A primary, Location B backup\\n\\nDirect Connect doesn't have an \\\"active/backup\\\" toggle \\u2014 you influence path selection with BGP attributes on the virtual interfaces at each location. Any of these (not mutually exclusive) will do it:\\n\\n**For outbound traffic from AWS to your network** (which path AWS uses to reach you):\\n- **Local preference BGP community tags** \\u2014 this is the recommended primary method. Tag prefixes advertised from Location A with `7224:7300` (high) and from Location B with `7224:7100` (low). AWS evaluates local preference first, so this wins regardless of path length.\\n- **AS-PATH prepending** \\u2014 as a secondary/fallback mechanism, prepend extra AS numbers on the routes advertised from Location B so its path looks longer. Only matters if local preference ties.\\n- **More specific prefixes** on the active connection also win, independent of the above.\\n\\n**For inbound traffic to your network from AWS** (which path your network prefers for return traffic):\\n- Advertise with local preference community tags and/or AS-PATH prepending from your router at Location B so AWS prefers sending to A.\\n- Advertise a more specific route from Location A.\\n\\nOrder of precedence Direct Connect uses: **most specific prefix \\u2192 local preference \\u2192 shortest AS path.** So local preference is the reliable lever; AS-PATH prepending alone can get overridden by prefix specificity or regional distance.\\n\\n## What to avoid\\n\\n- **Don't rely on AS-PATH prepending alone if VIFs span different AWS Regions attached to one Direct Connect Gateway.** If VIF-A is in Region 1 and VIF-B is in Region 2, but both reach a VPC in Region 1, AWS will always prefer the same-Region connection regardless of AS-PATH \\u2014 prepending won't override this. Use local preference community tags instead in this topology.\\n- **Don't advertise identical prefixes with identical attributes (AS path, MED, community) on both connections** \\u2014 that creates unintended load-balancing (ECMP) across both paths instead of a real active/passive split, which can cause asymmetric routing and odd troubleshooting if B is meant to be pure backup.\\n- **Don't mix inconsistent signals** \\u2014 e.g., tagging local preference high on B while also advertising a more-specific prefix on B. More-specific-prefix wins first, so it'll silently override your local preference intent.\\n- **Don't forget both directions** \\u2014 you need to set preference for outbound-from-AWS (community tags on your advertisements) AND for your own router's outbound-to-AWS path selection (your local BGP policy / AS-PATH on what you accept from AWS). Setting only one side gives asymmetric routing.\\n- **Test failover explicitly** \\u2014 bring down Location A's connection (or BGP session) and confirm traffic actually shifts to B and back cleanly, rather than assuming the config works.\\n\\nWant me to check your actual Direct Connect virtual interfaces and BGP configuration in this account to see what's currently advertised at each location?\"}]}", + "createdAt": "2026-10-27T17:05:27.048000-05:00", + "recordType": "final_response" + } +] \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/dx-vpn-redundancy-preference/with_skill/functional-tests-results.json b/skills/aws-routing/evals/functional/v1/iteration-1/dx-vpn-redundancy-preference/with_skill/functional-tests-results.json new file mode 100644 index 00000000..d0878089 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/dx-vpn-redundancy-preference/with_skill/functional-tests-results.json @@ -0,0 +1,66 @@ +{ + "version": "v1", + "iteration": 1, + "eval_id": "dx-vpn-redundancy-preference", + "run_type": "with_skill", + "success": true, + "error": null, + "understanding_agent_space_skill": null, + "tests": { + "trigger": { + "result": "passed", + "message": "Skill 'aws-routing' was triggered as expected", + "skill_loads_found": 1 + }, + "expected_output": { + "result": "passed", + "reasoning": "The agent correctly states that Direct Connect is always preferred over VPN for the same prefix at a VGW (\"Direct Connect always wins over VPN for same-prefix return traffic\"), explicitly notes no ECMP between DX and VPN (\"VGW doesn't support ECMP\"), identifies asymmetric routing as the main failure mode and explains it in detail, and provides actionable guidance to match ingress/egress preference (make CGW prefer DX whenever it's up, since AWS always returns via DX) to avoid asymmetric routing. This substantively matches all elements of the expected output.", + "evidence": "\"Bottom line: Direct Connect always wins over VPN for same-prefix return traffic.\" ... \"Also note: VGW doesn't support ECMP, so even on the VPN alone, return traffic always rides a single tunnel\" ... \"Make your customer gateway router send on-prem\u2192AWS traffic over DX whenever DX is up. Since AWS will always return via DX anyway, if your CGW prefers VPN while DX is healthy, you get asymmetric routing (egress on VPN, ingress on DX).\"", + "confidence": "high" + }, + "assertions": { + "assertion_results": [ + { + "text": "states DX is always preferred over VPN for the same prefix at a VGW", + "evaluator": "llm", + "passed": true, + "evidence": "'Bottom line: Direct Connect always wins over VPN for same-prefix return traffic.' and ranking list shows 'BGP propagated routes from Direct Connect \u2190 your DX wins here' as highest priority.", + "reasoning": "The output explicitly states DX is always preferred over VPN for the same prefix at the VGW.", + "confidence": "high" + }, + { + "text": "identifies asymmetric routing as the main risk", + "evaluator": "llm", + "passed": true, + "evidence": "The second major section is titled 'How to avoid asymmetric routing' and the body repeatedly discusses asymmetric routing risk: 'if your CGW prefers VPN while DX is healthy, you get asymmetric routing (egress on VPN, ingress on DX).'", + "reasoning": "The output clearly frames asymmetric routing as the central risk to be managed and dedicates most of its guidance to avoiding it.", + "confidence": "high" + }, + { + "text": "does not contain 'error'", + "evaluator": "llm", + "passed": true, + "evidence": "No occurrence of the word 'error' appears anywhere in the response text.", + "reasoning": "A scan of the output shows no instance of the literal word 'error'.", + "confidence": "high" + } + ], + "summary": { + "passed": 3, + "failed": 0, + "errored": 0, + "low_confidence": 0, + "total": 3, + "pass_rate": 1.0 + } + } + }, + "metrics": { + "runtime": "21s", + "cost": "$0.18", + "context_window": { + "utilization": "4.7%", + "compaction_count": 0 + } + } +} \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/dx-vpn-redundancy-preference/with_skill/outputs/journal_records.json b/skills/aws-routing/evals/functional/v1/iteration-1/dx-vpn-redundancy-preference/with_skill/outputs/journal_records.json new file mode 100644 index 00000000..84f951fc --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/dx-vpn-redundancy-preference/with_skill/outputs/journal_records.json @@ -0,0 +1,186 @@ +[ + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "877b92c1-3f95-4042-a04b-0fcf32e6cfc9", + "content": "{\"id\": \"877b92c1-3f95-4042-a04b-0fcf32e6cfc9\", \"role\": \"user\", \"content\": [{\"text\": \"I have a Direct Connect and a backup Site-to-Site VPN to the same VGW. Which path does AWS prefer and how do I avoid asymmetric routing?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:05.034000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "3932eced-f1b7-4d9b-9aef-88da08437c64", + "content": "{\"id\": \"16a175fc-c66e-4497-a12c-8b74da51dd85\", \"role\": \"assistant\", \"content\": [{\"type\": \"chat_title\", \"title\": \"Direct Connect vs Site-to-Site VPN routing preferences\"}]}", + "createdAt": "2026-10-27T17:05:05.647000-05:00", + "recordType": "chat_title" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "2012d168-67aa-4efe-b0eb-9232da4474ec", + "content": "{\"id\": \"2012d168-67aa-4efe-b0eb-9232da4474ec\", \"role\": \"assistant\", \"content\": [{\"id\": \"tooluse_SqEM6DyK14lwJn3UiMT1hy\", \"tool_name\": \"skill_read\", \"input\": {\"skill_id\": \"aws-routing\"}, \"type\": \"tool_use\", \"display_intent\": \"Reading AWS routing skill for Direct Connect vs VPN preference guidance.\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:08.252000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "5ffdbc10-62e1-459d-ad5d-f5bc0798bb56", + "content": "{\"id\": \"05526710-da74-4b78-81cd-3786a5047c6b\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This question is about AWS routing, so I should check the aws-routing skill for guidance.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:08.252000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "b9559d6e-64cc-478f-8def-965c98fbea0d", + "content": "{\"id\": \"eb608d7a-1084-46fc-a9b8-925d6787ccc4\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"316\"}]}", + "createdAt": "2026-10-27T17:05:08.325000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "398f6e1c-adcf-42bf-b2ee-5f7d6a053207", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 3.8, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:08.349000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "43780661-bae6-473b-a0e2-ec3a1e6518bd", + "content": "{\"id\": \"58c38ab0-bfa5-423b-aa19-46d08830d799\", \"role\": \"assistant\", \"block_type\": \"load_skill\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_SqEM6DyK14lwJn3UiMT1hy\", \"status\": \"success\", \"content\": [{\"text\": \"Done\"}]}], \"label\": \"Reading AWS routing skill for Direct Connect vs VPN preference guidance.\"}", + "createdAt": "2026-10-27T17:05:08.467000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "19327165-74db-4421-826c-575494caa52d", + "content": "{\"id\": \"19327165-74db-4421-826c-575494caa52d\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_SqEM6DyK14lwJn3UiMT1hy\", \"content\": \"[{'text': '{\\\"content\\\": \\\"---\\\\\\\\r\\\\\\\\nname: aws-routing\\\\\\\\r\\\\\\\\ndescription: \\\\'Use this skill when the user wants to know which path AWS will pick for their traffic, why traffic took an unexpected route, or how to make AWS prefer a path. Covers Cloud WAN CNE route evaluation, Direct Connect Gateway local vs remote region preference, Transit Gateway route tables, ECMP and peering, VPC longest-prefix-match, BGP traffic engineering (LP communities 7224:7100/7200/7300, AS-path prepending, MED), DX + VPN redundancy and failover, and asymmetric routing. Use it even when the user only describes a symptom, such as \\\\\\\\\\\"traffic is leaving the region\\\\\\\\\\\", \\\\\\\\\\\"not using my local Direct Connect\\\\\\\\\\\", \\\\\\\\\\\"routing to the wrong VIF/location\\\\\\\\\\\", or \\\\\\\\\\\"path is asymmetric\\\\\\\\\\\". Read-only: analysis, recommendations, and describe/get/list validation commands grounded in public AWS docs.\\\\'\\\\\\\\r\\\\\\\\nmetadata:\\\\\\\\r\\\\\\\\n version: \\\\\\\\\\\"1.2.5\\\\\\\\\\\"\\\\\\\\r\\\\\\\\n author: midakin\\\\\\\\r\\\\\\\\n aws-devops-agent-skills.agent-types: \\\\\\\\\\\"CHAT, INCIDENT_RCA\\\\\\\\\\\"\\\\\\\\r\\\\\\\\n---\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Overview\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nThis skill provides routing-first analysis for AWS networking. It reasons about the full traffic\\\\\\\\r\\\\\\\\npath, applies the documented route-evaluation logic for each service, and grounds recommendations\\\\\\\\r\\\\\\\\nin public AWS documentation. It is **read-only**: it produces analysis, recommendations, and\\\\\\\\r\\\\\\\\nread-only validation commands (`describe-*`, `get-*`, `list-*`) - never mutating actions.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Verification & Anti-Hallucination Directives (Mandatory)\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- Do not generate any fact, citation, or source that is not fully verifiable. If you lack a verified\\\\\\\\r\\\\\\\\n source, state \\\\\\\\\\\"I cannot verify this\\\\\\\\\\\" rather than inventing details.\\\\\\\\r\\\\\\\\n- State assumptions explicitly before answering.\\\\\\\\r\\\\\\\\n- Cross-check each claim against public AWS documentation or the `references/` files. Omit claims you\\\\\\\\r\\\\\\\\n cannot confirm.\\\\\\\\r\\\\\\\\n- Only cite AWS documentation URLs you have actually retrieved or confirmed. Do not fabricate links.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## When to use this skill\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nActivate when the user asks about:\\\\\\\\r\\\\\\\\n- Cloud WAN route evaluation / Core Network Edge (CNE) path selection\\\\\\\\r\\\\\\\\n- Direct Connect Gateway path selection, local-region preference, or LP communities\\\\\\\\r\\\\\\\\n- Transit Gateway route evaluation, ECMP, or peering routing\\\\\\\\r\\\\\\\\n- VPC route tables and longest-prefix-match behavior\\\\\\\\r\\\\\\\\n- BGP traffic engineering (communities, AS-path prepending, MED)\\\\\\\\r\\\\\\\\n- DX + VPN redundancy, active/active vs active/passive, or failover design\\\\\\\\r\\\\\\\\n- Why traffic is taking an unexpected path\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## What this skill does NOT do\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nThis skill is **read-only and advisory**. It will not:\\\\\\\\r\\\\\\\\n- **Design or apply infrastructure changes.** It does not create, modify, or delete AWS resources,\\\\\\\\r\\\\\\\\n and it does not run mutating CLI/API calls (`create-*`, `modify-*`, `associate-*`, `delete-*`,\\\\\\\\r\\\\\\\\n `put-*`, `update-*`) or write Infrastructure-as-Code to be deployed. Any change it describes is a\\\\\\\\r\\\\\\\\n *recommendation* for you to review and implement yourself.\\\\\\\\r\\\\\\\\n- **Configure BGP or push routing policy.** It explains which communities, AS-path, or MED settings\\\\\\\\r\\\\\\\\n to use, but it does not apply them to VIFs, route tables, or Cloud WAN policy documents.\\\\\\\\r\\\\\\\\n- **Execute anything against a live account beyond read-only validation.** Commands it provides are\\\\\\\\r\\\\\\\\n limited to `describe-*`, `get-*`, and `list-*` for confirming state.\\\\\\\\r\\\\\\\\n- **Guarantee production outcomes.** It recommends non-production testing and blue/green rollout;\\\\\\\\r\\\\\\\\n it does not perform the cutover or validate the result for you.\\\\\\\\r\\\\\\\\n- **Provide compliance, security, or contractual sign-off**, or make claims it cannot ground in\\\\\\\\r\\\\\\\\n public AWS documentation or the `references/` files.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nFor designing and applying changes, hand the recommendation to a change-capable workflow (with\\\\\\\\r\\\\\\\\nappropriate review, approvals, and testing).\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## How to analyze a routing problem\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nWork through this checklist in order; each step depends on the one before it.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- [ ] **1. Establish the path.** Identify source, destination, and every routing construct in\\\\\\\\r\\\\\\\\n between (VPC route table \\\\\\\\u2192 TGW/Cloud WAN CNE \\\\\\\\u2192 DXGW \\\\\\\\u2192 VIF \\\\\\\\u2192 on-prem, or the relevant subset).\\\\\\\\r\\\\\\\\n- [ ] **2. Name the route table.** Be explicit about which table makes each decision (VPC RT, TGW RT,\\\\\\\\r\\\\\\\\n CNE RT, DXGW internal selection).\\\\\\\\r\\\\\\\\n- [ ] **3. Apply the evaluation order for that construct** (see below).\\\\\\\\r\\\\\\\\n- [ ] **4. Identify the deciding attribute** (longest prefix, local preference, AS-path, MED, source\\\\\\\\r\\\\\\\\n type).\\\\\\\\r\\\\\\\\n- [ ] **5. Flag non-determinism** (e.g., ECMP or \\\\\\\\\\\"deterministically random\\\\\\\\\\\" tiebreakers) and\\\\\\\\r\\\\\\\\n recommend a deterministic alternative.\\\\\\\\r\\\\\\\\n- [ ] **6. Cite the AWS documentation** that supports the behavior.\\\\\\\\r\\\\\\\\n- [ ] **7. Provide read-only validation commands** the user can run to confirm.\\\\\\\\r\\\\\\\\n- [ ] **8. Self-check before answering.** Re-read the answer against steps 1-7: every route table is\\\\\\\\r\\\\\\\\n named, the evaluation order matches the construct, each cited URL was actually retrieved, and every\\\\\\\\r\\\\\\\\n command is read-only. Fix any gap before presenting.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Key evaluation rules\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Cloud WAN (per CNE)\\\\\\\\r\\\\\\\\nLongest prefix match \\\\\\\\u2192 static \\\\\\\\u2192 VPC-propagated (same region) \\\\\\\\u2192 unequal AS-path/MED (shortest wins) \\\\\\\\u2192\\\\\\\\r\\\\\\\\nequal AS-path & MED source preference: DXGW-propagated \\\\\\\\u2192 Cloud WAN Connect \\\\\\\\u2192 S2S VPN \\\\\\\\u2192 other\\\\\\\\r\\\\\\\\n(TGW peering, remote CNEs; identical from 2+ sources = deterministically random).\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Direct Connect (private/transit VIF, outbound AWS\\\\\\\\u2192on-prem)\\\\\\\\r\\\\\\\\nLongest prefix match \\\\\\\\u2192 local preference \\\\\\\\u2192 AS_PATH length \\\\\\\\u2192 MED \\\\\\\\u2192 ECMP (equal AS_PATH and BGP\\\\\\\\r\\\\\\\\nattributes). LP communities `7224:7300/7200/7100` (High/Medium/Low) set local preference and are\\\\\\\\r\\\\\\\\nevaluated before AS-path. DXGW prefers the local associated-region DX by default.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Direction & tooling guidance\\\\\\\\r\\\\\\\\n- **AWS \\\\\\\\u2192 on-prem (egress):** control with DX LP communities.\\\\\\\\r\\\\\\\\n- **On-prem \\\\\\\\u2192 AWS (ingress):** control with the customer router\\\\'s local-pref/weight/MED.\\\\\\\\r\\\\\\\\n- **AS-path prepending:** within a region only - unreliable across regions because DXGW local-region\\\\\\\\r\\\\\\\\n LP overrides it. **LP communities:** work within-region and cross-region.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Reference material\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nDetailed mechanics, patterns, and caveats are in `references/`. Load only the file that matches the\\\\\\\\r\\\\\\\\nconstruct deciding the path:\\\\\\\\r\\\\\\\\n- [Cloud WAN + DX routing patterns](references/cloudwan-dx-routing-patterns.md) - load when the path\\\\\\\\r\\\\\\\\n crosses a Cloud WAN CNE: CNE route evaluation, DXGW path selection into Cloud WAN, community\\\\\\\\r\\\\\\\\n traffic-engineering patterns, regional inspection, multi-region egress.\\\\\\\\r\\\\\\\\n- [Direct Connect routing patterns](references/dx-routing-patterns.md) - load when the question is\\\\\\\\r\\\\\\\\n about a DX VIF or DXGW without Cloud WAN: BGP mechanics, BFD, MED, ASN ranges, VIF considerations,\\\\\\\\r\\\\\\\\n CloudHub, when VIFs can talk, active/active vs active/passive.\\\\\\\\r\\\\\\\\n- [Transit Gateway routing patterns](references/tgw-routing-patterns.md) - load when a TGW route\\\\\\\\r\\\\\\\\n table makes the decision: TGW route evaluation, ECMP limits, DXGW-to-TGW behavior, DX/VPN caveats,\\\\\\\\r\\\\\\\\n failover, peering.\\\\\\\\r\\\\\\\\n- [VPN + DX redundancy patterns](references/vpn-dx-redundancy-patterns.md) - load when VPN backs up\\\\\\\\r\\\\\\\\n or runs alongside DX: route preference between them, ECMP limits, asymmetric return paths.\\\\\\\\r\\\\\\\\n- [Networking strategic questions](references/networking-strategic-questions.md) - load when key\\\\\\\\r\\\\\\\\n facts are missing (regions, attachments, ASNs, communities) and you need discovery questions to ask\\\\\\\\r\\\\\\\\n before analyzing.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Output expectations\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- Explain the route-evaluation logic step by step.\\\\\\\\r\\\\\\\\n- Name the specific route table involved.\\\\\\\\r\\\\\\\\n- Give the end-to-end traffic path.\\\\\\\\r\\\\\\\\n- Compare options with tradeoffs (determinism, complexity, failover) when multiple solutions exist.\\\\\\\\r\\\\\\\\n- Include read-only AWS CLI validation commands.\\\\\\\\r\\\\\\\\n- Recommend testing in a non-production environment and blue/green for migrations.\\\\\\\\r\\\\\\\\n\\\", \\\"resource_manifest\\\": [\\\"SKILL.md\\\", \\\"references/cloudwan-dx-routing-patterns.md\\\", \\\"references/dx-routing-patterns.md\\\", \\\"references/networking-strategic-questions.md\\\", \\\"references/tgw-routing-patterns.md\\\", \\\"references/vpn-dx-redundancy-patterns.md\\\"]}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:12.577000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "43f95fa3-fdd8-4994-889e-b13d351eadf0", + "content": "{\"id\": \"fa643724-087d-47ca-961c-a71e4986562a\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This setup is a direct DX plus VPN connection to a VGW, without DXGW or TGW involved, so I should check the vpn-dx-redundancy-patterns reference for guidance.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:12.577000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "f734f8ef-eef8-4f50-a130-716dfadb5924", + "content": "{\"id\": \"8eff7d2c-35b0-43e1-ba14-93673b30ff86\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"732\"}]}", + "createdAt": "2026-10-27T17:05:12.660000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "4820650e-a65f-4d6f-ae8c-f03bb8d917f7", + "content": "{\"id\": \"4820650e-a65f-4d6f-ae8c-f03bb8d917f7\", \"role\": \"assistant\", \"content\": [{\"id\": \"tooluse_8OT8oGFimkEU6uVl4ZpdkZ\", \"tool_name\": \"read_skill_resource\", \"input\": {\"skill_id\": \"aws-routing\", \"path\": \"references/vpn-dx-redundancy-patterns.md\"}, \"type\": \"tool_use\", \"display_intent\": \"Reading VPN plus Direct Connect redundancy patterns reference doc.\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:12.665000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "6c7569fc-c34e-4f9c-a822-413b03084cb4", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.2, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:12.738000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "f5097747-d5cc-4735-b417-109cfd78e95b", + "content": "{\"id\": \"0d352d57-7a1f-4e39-9da3-0263a55819e5\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_8OT8oGFimkEU6uVl4ZpdkZ\", \"status\": \"success\", \"content\": [{\"text\": \"Done\"}]}], \"label\": \"Reading VPN plus Direct Connect redundancy patterns reference doc.\"}", + "createdAt": "2026-10-27T17:05:13.087000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "bf511218-a64b-4781-9bd0-c4988a37937f", + "content": "{\"id\": \"bf759bbf-ba5d-4ade-a554-c06a85a530fa\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"I should double-check the key claim that Direct Connect always takes priority over VPN at the virtual gateway with no ECMP between them, since the skill calls for grounding assertions in AWS documentation before stating them confidently.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:18.483000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "134cdc61-cfdb-435a-b0bd-5a395a164a12", + "content": "{\"id\": \"134cdc61-cfdb-435a-b0bd-5a395a164a12\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_8OT8oGFimkEU6uVl4ZpdkZ\", \"content\": \"[{'text': '\\\\r\\\\n\\\\r\\\\n# VPN and Direct Connect Redundancy Patterns\\\\r\\\\n\\\\r\\\\nThis document captures route preference behavior for VPN and DX redundancy designs with VGW. Use this knowledge when advising on active/active, active/passive, and failover architectures.\\\\r\\\\n\\\\r\\\\n## Reference Documentation\\\\r\\\\n\\\\r\\\\n- [AWS Site-to-Site VPN routing](https://docs.aws.amazon.com/vpn/latest/s2svpn/VPNRoutingTypes.html)\\\\r\\\\n- [Direct Connect routing policies and BGP communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html)\\\\r\\\\n- [DX Resiliency Recommendations](https://aws.amazon.com/directconnect/resiliency-recommendation/)\\\\r\\\\n\\\\r\\\\n## Assumptions\\\\r\\\\n\\\\r\\\\n- Customer\\\\'s network does not overlap with VPC CIDR\\\\r\\\\n- Customer Gateway (CGW) device is configured to handle asymmetric routing\\\\r\\\\n- Static VPN tunnels are Route-Based only (not Policy-Based)\\\\r\\\\n- Customer advertises the same on-prem routes through all redundant connections (DX and VPN)\\\\r\\\\n\\\\r\\\\n## Key Behavior: VGW Does NOT Support ECMP\\\\r\\\\n\\\\r\\\\nVGW does **not** support Equal-Cost Multi-Path routing (ECMP). Return traffic from VPC to on-prem always takes a **single tunnel/path** \\u2014 the \\\"Preferred Tunnel.\\\" This is a fundamental constraint that affects all redundancy designs.\\\\r\\\\n\\\\r\\\\n---\\\\r\\\\n\\\\r\\\\n## Part 1: Redundancy Only on AWS End\\\\r\\\\n\\\\r\\\\n### VPN Connection \\u2014 2 Active Tunnels (Static)\\\\r\\\\n\\\\r\\\\n- Customers can have static route-based tunnels in Active/Active mode\\\\r\\\\n- **Customer \\u2192 VPC:** Can load balance (equal route metrics) or prefer one tunnel (unequal metrics)\\\\r\\\\n- **VPC \\u2192 Customer:** Always takes a single tunnel (Preferred Tunnel) \\u2014 VGW does not support ECMP\\\\r\\\\n\\\\r\\\\n- Customers can have BGP tunnels in Active/Active mode\\\\r\\\\n- **Customer \\u2192 VPC:** Load balance (equal LP) or prefer one (unequal LP)\\\\r\\\\n- **VPC \\u2192 Customer:** Always a single tunnel. Customer can influence by:\\\\r\\\\n - Advertising **lower AS-path** to VGW over the desired tunnel\\\\r\\\\n - Advertising **lower MED** to VGW over the desired tunnel\\\\r\\\\n\\\\r\\\\n### DX Connection with Backup VPN\\\\r\\\\n\\\\r\\\\n- DX is **always preferred over VPN** for return traffic to the same on-prem subnet\\\\r\\\\n- Customer must ensure on-prem traffic is sent over DX when both are UP\\\\r\\\\n- Attempting to prefer VPN over DX will cause asymmetric routing (VGW always returns via DX)\\\\r\\\\n\\\\r\\\\n### 2 DX Connections (Same VGW)\\\\r\\\\n\\\\r\\\\n- 2 private VIFs to the same VGW over 2 DX connections\\\\r\\\\n- **Active/Active:** Equal LP + AS-path on CGW \\u2192 return traffic is flow-hashed across both DX connections\\\\r\\\\n- **Active/Passive:** Unequal LP + AS-path \\u2192 customer-to-VPC uses higher LP; VPC-to-customer uses lower AS-path\\\\r\\\\n\\\\r\\\\n**\\u26a0\\ufe0f ECMP only works when both DX connections terminate at the SAME DX location:**\\\\r\\\\n- Same location (e.g., both at Equinix DC, Ashburn) \\u2192 flow-hashed (ECMP)\\\\r\\\\n- Different locations in same region (e.g., Equinix DC Ashburn + CoreSite NYC) \\u2192 **single path only**\\\\r\\\\n\\\\r\\\\n---\\\\r\\\\n\\\\r\\\\n## Part 2: Redundancy on AWS and Customer Ends\\\\r\\\\n\\\\r\\\\n### 2 VPN Connections from 2 CGW Devices (Recommended Design)\\\\r\\\\n\\\\r\\\\n- 2 BGP VPN connections (4 active tunnels) to same VGW from different on-prem CGW devices\\\\r\\\\n- **Recommended:** Use higher LP + lower AS-path on preferred VPN connection to avoid routing issues\\\\r\\\\n- **Customer \\u2192 VPC:** Traffic uses tunnel(s) with higher LP\\\\r\\\\n- **VPC \\u2192 Customer:** Single tunnel (Preferred Tunnel). Customer influences via lower AS-path on desired connection.\\\\r\\\\n\\\\r\\\\n### DX + Backup VPN from 2 CGW Devices\\\\r\\\\n\\\\r\\\\n- DX and VPN from different on-prem CGW devices\\\\r\\\\n- When both UP: customer must ensure on-prem traffic goes over DX only\\\\r\\\\n- **VPC \\u2192 Customer:** Always sent over DX (DX always preferred over VPN by VGW)\\\\r\\\\n- Design works correctly as active/passive (DX active, VPN passive)\\\\r\\\\n\\\\r\\\\n### 2 DX Connections from 2 CGW Devices\\\\r\\\\n\\\\r\\\\n- 2 DX connections from different on-prem CGW devices, 2 private VIFs to same VGW\\\\r\\\\n- **Active/Active:** Equal LP + AS-path \\u2192 flow-hashed return traffic\\\\r\\\\n- **Active/Passive:** Unequal LP + AS-path \\u2192 deterministic path preference\\\\r\\\\n\\\\r\\\\n**Same ECMP caveat applies:** Flow-hashing only works when both DX connections are at the same DX location. Different locations \\u2192 single path selected.\\\\r\\\\n\\\\r\\\\n---\\\\r\\\\n\\\\r\\\\n## Part 3: Designs That Cause Routing Issues\\\\r\\\\n\\\\r\\\\n### \\u274c 2 Static VPN Connections from 2 CGW Devices\\\\r\\\\n\\\\r\\\\n- VPC \\u2192 Customer always traverses single tunnel (Preferred Tunnel)\\\\r\\\\n- If customer sends traffic on CGW 2, return may come back on CGW 1 \\u2192 **asymmetric routing**\\\\r\\\\n- Only works if CGW devices have synchronization (ECMP) and accept asymmetric traffic\\\\r\\\\n\\\\r\\\\n### \\u274c 2 BGP VPN Connections with Same LP and AS-Path from 2 CGW Devices\\\\r\\\\n\\\\r\\\\n- Same problem as static: VPC \\u2192 Customer takes single tunnel, may not match inbound path\\\\r\\\\n- **Fix:** Use unequal LP + AS-path to prefer one VPN connection over the other\\\\r\\\\n- Or ensure CGW devices handle asymmetric routing\\\\r\\\\n\\\\r\\\\n### \\u274c DX + VPN \\u2014 Attempting to Prefer VPN Over DX\\\\r\\\\n\\\\r\\\\n- If customer tries to prefer VPN over DX, return traffic (VPC \\u2192 on-prem) STILL goes over DX\\\\r\\\\n- VGW always prefers DX over VPN regardless of BGP attributes\\\\r\\\\n- **Result:** Asymmetric routing \\u2014 customer sends via VPN, AWS returns via DX\\\\r\\\\n- This is true whether using same CGW device or different CGW devices\\\\r\\\\n- **Fix:** Advertise more-specific routes on VPN \\u2014 longest prefix match wins before attachment-type preference, so a /24 via VPN will be preferred over a /16 via DX for matching traffic\\\\r\\\\n\\\\r\\\\n### \\u274c 2 VPN Connections from 2 CGW Devices with Different ASNs\\\\r\\\\n\\\\r\\\\n- 2 BGP VPN connections from 2 data centers (different ASNs) advertising same networks\\\\r\\\\n- VPC \\u2192 Customer: single tunnel, may not match inbound path \\u2192 asymmetric\\\\r\\\\n- **Fix:** Use unequal AS-path values to prefer one VPN connection\\\\r\\\\n- Cannot achieve ECMP across different ASNs\\\\r\\\\n\\\\r\\\\n### \\u274c DX + VPN from 2 CGW Devices with Different ASNs\\\\r\\\\n\\\\r\\\\n- If customer sends traffic over VPN, return goes over DX (DX always wins)\\\\r\\\\n- **Must avoid using same networks in multiple datacenters** to prevent routing issues\\\\r\\\\n- Or accept that DX is always the return path regardless of inbound path\\\\r\\\\n\\\\r\\\\n---\\\\r\\\\n\\\\r\\\\n## Summary: Route Preference Rules at VGW\\\\r\\\\n\\\\r\\\\n| Traffic Direction | Rule |\\\\r\\\\n|---|---|\\\\r\\\\n| VPC \\u2192 On-prem (same prefix) | DX **always** preferred over VPN |\\\\r\\\\n| VPC \\u2192 On-prem (VPN only) | Single tunnel \\u2014 lowest AS-path or MED, no ECMP |\\\\r\\\\n| VPC \\u2192 On-prem (2 DX, same location) | Flow-hashed (ECMP) |\\\\r\\\\n| VPC \\u2192 On-prem (2 DX, different locations) | Single path |\\\\r\\\\n| On-prem \\u2192 VPC (VPN) | Customer controls via LP (higher wins) |\\\\r\\\\n| On-prem \\u2192 VPC (DX) | Customer controls via LP (higher wins) |\\\\r\\\\n| On-prem \\u2192 VPC (DX + VPN both UP) | Customer must direct to DX |\\\\r\\\\n\\\\r\\\\n## Key Takeaways for Customer Guidance\\\\r\\\\n\\\\r\\\\n1. **VGW does not support ECMP** \\u2014 return traffic always takes one path\\\\r\\\\n2. **DX always wins over VPN** at the VGW \\u2014 cannot prefer VPN for return traffic\\\\r\\\\n3. **Same DX location required for flow-hashing** \\u2014 different locations in same region get single-path selection\\\\r\\\\n4. **Use unequal LP + AS-path** for active/passive designs to avoid asymmetric routing\\\\r\\\\n5. **Asymmetric routing is the primary failure mode** \\u2014 designs with equal attributes from multiple CGW devices without synchronization will cause issues\\\\r\\\\n6. **MED can influence VGW path selection** \\u2014 documented at [VPN route priority](https://docs.aws.amazon.com/vpn/latest/s2svpn/vpn-route-priority.html)\\\\r\\\\n'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:18.483000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "7ca1e88f-63f0-4717-8429-c215c2fe8641", + "content": "{\"id\": \"7ca1e88f-63f0-4717-8429-c215c2fe8641\", \"role\": \"assistant\", \"content\": [{\"id\": \"tooluse_x0LUu0nkFAEchdsCgz5z5v\", \"tool_name\": \"verify_aws_claim\", \"input\": {\"claim\": \"Direct Connect is always preferred over Site-to-Site VPN for routes of the same prefix length at a Virtual Private Gateway (VGW)\", \"url\": \"https://docs.aws.amazon.com/vpn/latest/s2svpn/vpn-route-priority.html\"}, \"type\": \"tool_use\", \"display_intent\": \"Verifying AWS doc on VPN route priority vs Direct Connect at a VGW.\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:18.560000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "f646cd38-ffd1-4a61-aea0-52ac6b10626b", + "content": "{\"id\": \"ffd5c079-bbb3-4dfd-9827-e8a6b6c658e2\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"848\"}]}", + "createdAt": "2026-10-27T17:05:18.602000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "ff21c24e-a27f-44e5-aa7b-a8ebbeabb4c5", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.5, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:18.636000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "f9328eda-7d3b-4c4c-9713-6f2f47d77096", + "content": "{\"id\": \"a2bd3488-5713-4fff-9b53-7ab2b7ee7a01\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_x0LUu0nkFAEchdsCgz5z5v\", \"status\": \"success\", \"content\": [{\"text\": \"{\\\"content\\\":{\\\"result\\\":[{\\\"status\\\":\\\"SUCCESS\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/vpn/latest/s2svpn/vpn-route-priority.html\\\",\\\"content\\\":\\\"# Route tables and AWS Site-to-Site VPN route priority\\\\n\\\\nRoute tables determine where\\\\nnetwork traffic from your VPC is directed. In your VPC route table, you must add a route\\\\nfor your remote network and specify the virtual private gateway as the target. This\\\\nenables traffic from your VPC that's destined for your remote network to route via the\\\\nvirtual private gateway and over one of the VPN tunnels. You can enable route\\\\npropagation for your route table to automatically propagate your network routes to the\\\\ntable for you.\\\\n\\\\nWe use the most specific route in your route table that matches the traffic to\\\\ndetermine how to route the traffic (longest prefix match). If your route table has\\\\noverlapping or matching routes, the following rules apply:\\\\n\\\\n* If propagated routes from a Site-to-Site VPN connection or Direct Connect connection overlap\\\\n with the local route for your VPC, the local route is most preferred even if the\\\\n propagated routes are more specific.\\\\n* If propagated routes from a Site-to-Site VPN connection or Direct Connect connection have the\\\\n same destination CIDR block as other existing static routes (longest prefix\\\\n match cannot be applied), we prioritize the static routes whose targets are an\\\\n internet gateway, a virtual private gateway, a network interface, an instance\\\\n ID, a VPC peering connection, a NAT gateway, a transit gateway, or a gateway VPC\\\\n endpoint.\\\\n\\\\nFor example, the following route table has a static route to an internet gateway, and\\\\na propagated route to a virtual private gateway. Both routes have a destination of\\\\n`172.31.0.0/24`. In this case, all traffic destined for\\\\n`172.31.0.0/24` is routed to the internet gateway \\u2014 it is a static\\\\nroute and therefore takes priority over the propagated route.\\\\n\\\\n| Destination | Target |\\\\n| --- | --- |\\\\n| 10.0.0.0/16 | Local |\\\\n| 172.31.0.0/24 | vgw-11223344556677889 (propagated) |\\\\n| 172.31.0.0/24 | igw-12345678901234567 (static) |\\\\n\\\\nOnly IP prefixes that are known to the virtual private gateway, whether through BGP\\\\nadvertisements or a static route entry, can receive traffic from your VPC. The virtual\\\\nprivate gateway does not route any other traffic destined outside of received BGP\\\\nadvertisements, static route entries, or its attached VPC CIDR. Virtual private gateways\\\\ndo not support IPv6 traffic.\\\\n\\\\nWhen a virtual private gateway receives routing information, it uses path selection to\\\\ndetermine how to route traffic. Longest prefix match applies, if all endpoints are\\\\nhealthy. The health of a tunnel endpoint takes precedence over other routing attributes.\\\\nThis precedence applies to VPNs on virtual private gateways and Transit Gateways. If the\\\\nprefixes are the same, then the virtual private gateway prioritizes routes as follows,\\\\nfrom most preferred to least preferred:\\\\n\\\\n* BGP propagated routes from an Direct Connect connection\\\\n\\\\n Blackhole routes are not propagated to a Site-to-Site VPN customer gateway via BGP.\\\\n* Manually added static routes for a Site-to-Site VPN connection\\\\n* BGP propagated routes from a Site-to-Site VPN connection\\\\n* For matching prefixes where each Site-to-Site VPN connection uses BGP, the AS PATH is\\\\n compared and the prefix with the shortest AS PATH is preferred.\\\\n\\\\n **Note:**\\\\n\\\\n AWS strongly recommends using customer gateway devices that support\\\\n asymmetric routing.\\\\n\\\\n For customer gateway devices that support asymmetric routing, we\\\\n *do not* recommend using AS PATH prepending, to\\\\n ensure that both tunnels have equal AS PATH. This helps to ensure that the\\\\n multi-exit discriminator (MED) value that we set on a\\\\n tunnel during VPN tunnel endpoint\\\\n updates is used to determine tunnel priority.\\\\n\\\\n For customer gateway devices that do not support asymmetric routing, you\\\\n can use AS PATH prepending and Local Preference to prefer one tunnel over\\\\n the other. However, when the egress path changes, this may cause traffic to\\\\n drop.\\\\n* When the AS PATHs are the same length and if the first AS in the AS_SEQUENCE\\\\n is the same across multiple paths, multi-exit discriminators\\\\n (MEDs) are compared. The path with the lowest MED value is preferred.\\\\n\\\\nRoute priority is affected during VPN\\\\ntunnel endpoint updates.\\\\n\\\\nOn a Site-to-Site VPN connection, AWS selects one of the two redundant tunnels as the primary\\\\negress path. This selection may change at times, and we strongly recommend that you\\\\nconfigure both tunnels for high availability, and allow asymmetric routing. The health\\\\nof a tunnel endpoint takes precedence over other routing attributes. This precedence\\\\napplies to VPNs on virtual private gateways and Transit Gateways.\\\\n\\\\nFor a virtual private gateway, one tunnel across all Site-to-Site VPN connections on the gateway\\\\nwill be selected. To use more than one tunnel, we recommend exploring Equal Cost\\\\nMultipath (ECMP), which is supported for Site-to-Site VPN connections on a transit gateway. For\\\\nmore information, see Transit gateways in\\\\n*Amazon VPC Transit Gateways*. ECMP is not supported for Site-to-Site VPN connections on\\\\na virtual private gateway.\\\\n\\\\nFor Site-to-Site VPN connections that use BGP, the primary tunnel can be identified by the\\\\nmulti-exit discriminator (MED) value. We recommend advertising more\\\\nspecific BGP routes to influence routing decisions.\\\\n\\\\nFor Site-to-Site VPN connections that use static routing, the primary tunnel can be identified by\\\\ntraffic statistics or metrics.\\\",\\\"total_length\\\":5340,\\\"start_index\\\":0,\\\"end_index\\\":5340,\\\"truncated\\\":false,\\\"redirected_url\\\":null,\\\"error_code\\\":null}]}}\"}]}], \"label\": \"Verifying AWS doc on VPN route priority vs Direct Connect at a VGW.\"}", + "createdAt": "2026-10-27T17:05:19.247000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "a5287e3d-3378-44fb-8127-b667a88e1087", + "content": "{\"id\": \"a5287e3d-3378-44fb-8127-b667a88e1087\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_x0LUu0nkFAEchdsCgz5z5v\", \"content\": \"[{'text': '{\\\"content\\\":{\\\"result\\\":[{\\\"status\\\":\\\"SUCCESS\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/vpn/latest/s2svpn/vpn-route-priority.html\\\",\\\"content\\\":\\\"# Route tables and AWS Site-to-Site VPN route priority\\\\\\\\n\\\\\\\\nRoute tables determine where\\\\\\\\nnetwork traffic from your VPC is directed. In your VPC route table, you must add a route\\\\\\\\nfor your remote network and specify the virtual private gateway as the target. This\\\\\\\\nenables traffic from your VPC that\\\\'s destined for your remote network to route via the\\\\\\\\nvirtual private gateway and over one of the VPN tunnels. You can enable route\\\\\\\\npropagation for your route table to automatically propagate your network routes to the\\\\\\\\ntable for you.\\\\\\\\n\\\\\\\\nWe use the most specific route in your route table that matches the traffic to\\\\\\\\ndetermine how to route the traffic (longest prefix match). If your route table has\\\\\\\\noverlapping or matching routes, the following rules apply:\\\\\\\\n\\\\\\\\n* If propagated routes from a Site-to-Site VPN connection or Direct Connect connection overlap\\\\\\\\n with the local route for your VPC, the local route is most preferred even if the\\\\\\\\n propagated routes are more specific.\\\\\\\\n* If propagated routes from a Site-to-Site VPN connection or Direct Connect connection have the\\\\\\\\n same destination CIDR block as other existing static routes (longest prefix\\\\\\\\n match cannot be applied), we prioritize the static routes whose targets are an\\\\\\\\n internet gateway, a virtual private gateway, a network interface, an instance\\\\\\\\n ID, a VPC peering connection, a NAT gateway, a transit gateway, or a gateway VPC\\\\\\\\n endpoint.\\\\\\\\n\\\\\\\\nFor example, the following route table has a static route to an internet gateway, and\\\\\\\\na propagated route to a virtual private gateway. Both routes have a destination of\\\\\\\\n`172.31.0.0/24`. In this case, all traffic destined for\\\\\\\\n`172.31.0.0/24` is routed to the internet gateway \\u2014 it is a static\\\\\\\\nroute and therefore takes priority over the propagated route.\\\\\\\\n\\\\\\\\n| Destination | Target |\\\\\\\\n| --- | --- |\\\\\\\\n| 10.0.0.0/16 | Local |\\\\\\\\n| 172.31.0.0/24 | vgw-11223344556677889 (propagated) |\\\\\\\\n| 172.31.0.0/24 | igw-12345678901234567 (static) |\\\\\\\\n\\\\\\\\nOnly IP prefixes that are known to the virtual private gateway, whether through BGP\\\\\\\\nadvertisements or a static route entry, can receive traffic from your VPC. The virtual\\\\\\\\nprivate gateway does not route any other traffic destined outside of received BGP\\\\\\\\nadvertisements, static route entries, or its attached VPC CIDR. Virtual private gateways\\\\\\\\ndo not support IPv6 traffic.\\\\\\\\n\\\\\\\\nWhen a virtual private gateway receives routing information, it uses path selection to\\\\\\\\ndetermine how to route traffic. Longest prefix match applies, if all endpoints are\\\\\\\\nhealthy. The health of a tunnel endpoint takes precedence over other routing attributes.\\\\\\\\nThis precedence applies to VPNs on virtual private gateways and Transit Gateways. If the\\\\\\\\nprefixes are the same, then the virtual private gateway prioritizes routes as follows,\\\\\\\\nfrom most preferred to least preferred:\\\\\\\\n\\\\\\\\n* BGP propagated routes from an Direct Connect connection\\\\\\\\n\\\\\\\\n Blackhole routes are not propagated to a Site-to-Site VPN customer gateway via BGP.\\\\\\\\n* Manually added static routes for a Site-to-Site VPN connection\\\\\\\\n* BGP propagated routes from a Site-to-Site VPN connection\\\\\\\\n* For matching prefixes where each Site-to-Site VPN connection uses BGP, the AS PATH is\\\\\\\\n compared and the prefix with the shortest AS PATH is preferred.\\\\\\\\n\\\\\\\\n **Note:**\\\\\\\\n\\\\\\\\n AWS strongly recommends using customer gateway devices that support\\\\\\\\n asymmetric routing.\\\\\\\\n\\\\\\\\n For customer gateway devices that support asymmetric routing, we\\\\\\\\n *do not* recommend using AS PATH prepending, to\\\\\\\\n ensure that both tunnels have equal AS PATH. This helps to ensure that the\\\\\\\\n multi-exit discriminator (MED) value that we set on a\\\\\\\\n tunnel during VPN tunnel endpoint\\\\\\\\n updates is used to determine tunnel priority.\\\\\\\\n\\\\\\\\n For customer gateway devices that do not support asymmetric routing, you\\\\\\\\n can use AS PATH prepending and Local Preference to prefer one tunnel over\\\\\\\\n the other. However, when the egress path changes, this may cause traffic to\\\\\\\\n drop.\\\\\\\\n* When the AS PATHs are the same length and if the first AS in the AS_SEQUENCE\\\\\\\\n is the same across multiple paths, multi-exit discriminators\\\\\\\\n (MEDs) are compared. The path with the lowest MED value is preferred.\\\\\\\\n\\\\\\\\nRoute priority is affected during VPN\\\\\\\\ntunnel endpoint updates.\\\\\\\\n\\\\\\\\nOn a Site-to-Site VPN connection, AWS selects one of the two redundant tunnels as the primary\\\\\\\\negress path. This selection may change at times, and we strongly recommend that you\\\\\\\\nconfigure both tunnels for high availability, and allow asymmetric routing. The health\\\\\\\\nof a tunnel endpoint takes precedence over other routing attributes. This precedence\\\\\\\\napplies to VPNs on virtual private gateways and Transit Gateways.\\\\\\\\n\\\\\\\\nFor a virtual private gateway, one tunnel across all Site-to-Site VPN connections on the gateway\\\\\\\\nwill be selected. To use more than one tunnel, we recommend exploring Equal Cost\\\\\\\\nMultipath (ECMP), which is supported for Site-to-Site VPN connections on a transit gateway. For\\\\\\\\nmore information, see Transit gateways in\\\\\\\\n*Amazon VPC Transit Gateways*. ECMP is not supported for Site-to-Site VPN connections on\\\\\\\\na virtual private gateway.\\\\\\\\n\\\\\\\\nFor Site-to-Site VPN connections that use BGP, the primary tunnel can be identified by the\\\\\\\\nmulti-exit discriminator (MED) value. We recommend advertising more\\\\\\\\nspecific BGP routes to influence routing decisions.\\\\\\\\n\\\\\\\\nFor Site-to-Site VPN connections that use static routing, the primary tunnel can be identified by\\\\\\\\ntraffic statistics or metrics.\\\",\\\"total_length\\\":5340,\\\"start_index\\\":0,\\\"end_index\\\":5340,\\\"truncated\\\":false,\\\"redirected_url\\\":null,\\\"error_code\\\":null}]}}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:29.974000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "149ade49-0682-4e3a-a60d-7bebe5c8b216", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.7, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:30.045000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "1bf4b205-f7a5-4949-9fce-0b9dff194d7b", + "content": "{\"id\": \"1bf4b205-f7a5-4949-9fce-0b9dff194d7b\", \"role\": \"assistant\", \"content\": [{\"text\": \"This confirms the behavior. Here's how it works for your setup:\\n\\n## Which path AWS prefers (VPC/VGW \\u2192 on-prem)\\n\\nFor the **same destination prefix**, the VGW prioritizes route sources in this order (highest to lowest):\\n\\n1. **BGP propagated routes from Direct Connect** \\u2190 your DX wins here\\n2. Manually added static VPN routes\\n3. BGP propagated routes from Site-to-Site VPN\\n4. Among VPN BGP routes with matching prefixes, shortest AS-path, then lowest MED\\n\\n**Bottom line: Direct Connect always wins over VPN for same-prefix return traffic.** This is a fixed VGW preference \\u2014 you cannot override it with BGP attributes (LP, AS-path, MED) on the VPN side. The only way to force traffic onto the VPN path is to advertise a **more specific (longer) prefix** over the VPN \\u2014 longest-prefix-match is evaluated *before* the attachment-type preference, so a /24 via VPN beats a /16 via DX for matching traffic.\\n\\nAlso note: **VGW doesn't support ECMP**, so even on the VPN alone, return traffic always rides a single tunnel (the \\\"Preferred Tunnel\\\") \\u2014 never load-balanced across both tunnels.\\n\\n## How to avoid asymmetric routing\\n\\n1. **Make your customer gateway router send on-prem\\u2192AWS traffic over DX whenever DX is up.** Since AWS will always return via DX anyway, if your CGW prefers VPN while DX is healthy, you get asymmetric routing (egress on VPN, ingress on DX).\\n2. **Don't try to prefer VPN over DX using BGP attributes** (LP/AS-path/MED) for the *same* prefix \\u2014 it won't work; the VGW's attachment-type preference overrides it.\\n3. **If you genuinely need VPN to carry specific traffic** (e.g., for testing or partial failover), advertise a more-specific prefix over the VPN than what's advertised over DX \\u2014 that's the one lever that beats the DX preference.\\n4. **Use a CGW device that explicitly supports asymmetric routing**, per AWS's recommendation \\u2014 this matters especially during automatic VPN tunnel endpoint maintenance, when AWS may shift the primary VPN tunnel without the health of DX changing.\\n5. **Treat the VPN purely as passive backup**: it should only carry traffic when DX is fully down (BGP session drops and the DX route disappears), at which point the VGW naturally falls back to the VPN route \\u2014 no manual intervention needed.\\n\\nIf you want, I can pull the current BGP state of your DX VIF and VPN tunnels (AS-path, advertised prefixes) to confirm they're configured as a clean active/passive pair.\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:30.123000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "2bc0d151-2022-412e-8749-c9fc39db7a0e", + "executionId": "42bc5a4a-e55e-4a5a-8130-bc8292ee1134", + "recordId": "39599304-5cf7-4a39-ac3f-5b2f658fcede", + "content": "{\"id\": \"09bdfe0a-95ab-43dd-bef7-71cbbf12eb4b\", \"role\": \"assistant\", \"content\": [{\"type\": \"text\", \"text\": \"This confirms the behavior. Here's how it works for your setup:\\n\\n## Which path AWS prefers (VPC/VGW \\u2192 on-prem)\\n\\nFor the **same destination prefix**, the VGW prioritizes route sources in this order (highest to lowest):\\n\\n1. **BGP propagated routes from Direct Connect** \\u2190 your DX wins here\\n2. Manually added static VPN routes\\n3. BGP propagated routes from Site-to-Site VPN\\n4. Among VPN BGP routes with matching prefixes, shortest AS-path, then lowest MED\\n\\n**Bottom line: Direct Connect always wins over VPN for same-prefix return traffic.** This is a fixed VGW preference \\u2014 you cannot override it with BGP attributes (LP, AS-path, MED) on the VPN side. The only way to force traffic onto the VPN path is to advertise a **more specific (longer) prefix** over the VPN \\u2014 longest-prefix-match is evaluated *before* the attachment-type preference, so a /24 via VPN beats a /16 via DX for matching traffic.\\n\\nAlso note: **VGW doesn't support ECMP**, so even on the VPN alone, return traffic always rides a single tunnel (the \\\"Preferred Tunnel\\\") \\u2014 never load-balanced across both tunnels.\\n\\n## How to avoid asymmetric routing\\n\\n1. **Make your customer gateway router send on-prem\\u2192AWS traffic over DX whenever DX is up.** Since AWS will always return via DX anyway, if your CGW prefers VPN while DX is healthy, you get asymmetric routing (egress on VPN, ingress on DX).\\n2. **Don't try to prefer VPN over DX using BGP attributes** (LP/AS-path/MED) for the *same* prefix \\u2014 it won't work; the VGW's attachment-type preference overrides it.\\n3. **If you genuinely need VPN to carry specific traffic** (e.g., for testing or partial failover), advertise a more-specific prefix over the VPN than what's advertised over DX \\u2014 that's the one lever that beats the DX preference.\\n4. **Use a CGW device that explicitly supports asymmetric routing**, per AWS's recommendation \\u2014 this matters especially during automatic VPN tunnel endpoint maintenance, when AWS may shift the primary VPN tunnel without the health of DX changing.\\n5. **Treat the VPN purely as passive backup**: it should only carry traffic when DX is fully down (BGP session drops and the DX route disappears), at which point the VGW naturally falls back to the VPN route \\u2014 no manual intervention needed.\\n\\nIf you want, I can pull the current BGP state of your DX VIF and VPN tunnels (AS-path, advertised prefixes) to confirm they're configured as a clean active/passive pair.\"}]}", + "createdAt": "2026-10-27T17:05:30.208000-05:00", + "recordType": "final_response" + } +] \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/dx-vpn-redundancy-preference/without_skill/functional-tests-results.json b/skills/aws-routing/evals/functional/v1/iteration-1/dx-vpn-redundancy-preference/without_skill/functional-tests-results.json new file mode 100644 index 00000000..d81a33b1 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/dx-vpn-redundancy-preference/without_skill/functional-tests-results.json @@ -0,0 +1,62 @@ +{ + "version": "v1", + "iteration": 1, + "eval_id": "dx-vpn-redundancy-preference", + "run_type": "without_skill", + "success": true, + "error": null, + "understanding_agent_space_skill": null, + "tests": { + "trigger": null, + "expected_output": { + "result": "passed", + "reasoning": "The agent correctly states that AWS always prefers DX over VPN for equal-prefix-length routes to a VGW, and that this is hard-coded (implying no ECMP between DX and VPN). It identifies asymmetric routing as the main failure mode arising from mismatched ingress/egress preference and longest-prefix-match issues, and provides concrete guidance to match prefix advertisement and local preference on both sides to validate/align both directions. This substantively matches the expected output's key points.", + "evidence": "\"AWS always prefers the Direct Connect path over the VPN path to the VGW \u2014 this is a hard-coded AWS routing preference... Where asymmetric routing actually creeps in is almost always on your side... Match prefix granularity on both paths... Set local preference on your on-prem BGP routers so outbound... traffic also prefers the path reachable via DX, mirroring AWS's own preference inbound.\"", + "confidence": "high" + }, + "assertions": { + "assertion_results": [ + { + "text": "states DX is always preferred over VPN for the same prefix at a VGW", + "evaluator": "llm", + "passed": true, + "evidence": "The agent states: 'with equal prefix length, AWS always prefers the Direct Connect path over the VPN path to the VGW \u2014 this is a hard-coded AWS routing preference... DX wins automatically whenever it's up and advertising the same or a more specific prefix.'", + "reasoning": "The output explicitly and unambiguously states that DX is always preferred over VPN for the same prefix length at the VGW, matching the assertion.", + "confidence": "high" + }, + { + "text": "identifies asymmetric routing as the main risk", + "evaluator": "llm", + "passed": true, + "evidence": "The agent says: 'Where asymmetric routing actually creeps in is almost always on your side...' and provides a dedicated 'Causes' and 'How to keep it symmetric' section addressing asymmetric routing directly as the central problem to solve.", + "reasoning": "The entire second half of the response is structured around identifying and mitigating asymmetric routing, treating it as the key risk the user needs to manage, satisfying the assertion.", + "confidence": "high" + }, + { + "text": "does not contain 'error'", + "evaluator": "llm", + "passed": true, + "evidence": "No occurrence of the word 'error' appears anywhere in the agent's response text.", + "reasoning": "A plain text scan of the output shows no instance of the word 'error', so the assertion holds true.", + "confidence": "high" + } + ], + "summary": { + "passed": 3, + "failed": 0, + "errored": 0, + "low_confidence": 0, + "total": 3, + "pass_rate": 1.0 + } + } + }, + "metrics": { + "runtime": "28s", + "cost": "$0.24", + "context_window": { + "utilization": "4.5%", + "compaction_count": 0 + } + } +} \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/dx-vpn-redundancy-preference/without_skill/outputs/journal_records.json b/skills/aws-routing/evals/functional/v1/iteration-1/dx-vpn-redundancy-preference/without_skill/outputs/journal_records.json new file mode 100644 index 00000000..bcaed113 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/dx-vpn-redundancy-preference/without_skill/outputs/journal_records.json @@ -0,0 +1,186 @@ +[ + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "6185e12c-d94a-4e2d-8f62-0461fdff04ac", + "content": "{\"id\": \"6185e12c-d94a-4e2d-8f62-0461fdff04ac\", \"role\": \"user\", \"content\": [{\"text\": \"I have a Direct Connect and a backup Site-to-Site VPN to the same VGW. Which path does AWS prefer and how do I avoid asymmetric routing?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:04.559000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "c4caf839-34c8-4e79-87cf-306ff21382f9", + "content": "{\"id\": \"9d76d391-bd66-42d2-8e3d-c87654d58934\", \"role\": \"assistant\", \"content\": [{\"type\": \"chat_title\", \"title\": \"Direct Connect vs VPN path preference AWS\"}]}", + "createdAt": "2026-10-27T17:05:05.266000-05:00", + "recordType": "chat_title" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "a68d29e7-cc5f-47b5-9b53-0f00858dd909", + "content": "{\"id\": \"d97b7eb2-3683-46f3-84be-c724ff6deff9\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This is actually a general AWS networking question about Direct Connect vs VPN routing preference to the same VGW, not specific to the AWS DevOps Agent itself. I should still verify this factual claim against AWS documentation to make sure I'm not giving outdated or incorrect information.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:11.216000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "228c8994-68c5-41c9-8daa-e06b5394f095", + "content": "{\"id\": \"228c8994-68c5-41c9-8daa-e06b5394f095\", \"role\": \"assistant\", \"content\": [{\"id\": \"tooluse_8pAOJvjqSbQN52e89vGSy3\", \"tool_name\": \"verify_aws_claim\", \"input\": {\"claim\": \"When a VGW has both a Direct Connect private VIF and a Site-to-Site VPN connection advertising the same prefixes, which path AWS prefers and how BGP path selection/local preference works\", \"topics\": [\"reference_documentation\"]}, \"type\": \"tool_use\", \"display_intent\": \"Verify AWS routing preference between Direct Connect and Site-to-Site VPN to same VGW\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:11.216000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "820efece-d62b-473e-8735-e0636f3cbb41", + "content": "{\"id\": \"3ae363aa-a392-458b-9ae8-a38625b3c6f5\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"1651\"}]}", + "createdAt": "2026-10-27T17:05:11.288000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "d97194da-f08e-4a6a-af5a-9a1b786c0628", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 3.8, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:11.291000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "794424f8-722c-4a8a-ba9f-0f526f1fc2da", + "content": "{\"id\": \"035d69af-7727-472d-bd82-96e234c1c5fc\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_8pAOJvjqSbQN52e89vGSy3\", \"status\": \"success\", \"content\": [{\"text\": \"{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"Direct Connect routing policies and BGP communities\\\",\\\"context\\\":\\\"## Private virtual interface and transit virtual interface routing policies\\\\n\\\\nIf you're using AWS Direct Connect to access your private AWS resources,\\\\nyou must specify the IPv4 or IPv6 prefixes to advertise over BGP. These prefixes can be\\\\npublic or private.\\\\n\\\\nThe following outbound routing rules apply based on the prefixes advertised:\\\\n\\\\n* AWS evaluates the longest prefix length first. AWS recommends advertising\\\\n more specific routes using multiple Direct Connect virtual interfaces if the\\\\n desired routing paths are meant for active/passive connections. See Influencing Traffic over Hybrid Networks using Longest Prefix\\\\n Match for more information.\\\\n* Local preference is the BGP attribute recommended to use when desired routing\\\\n paths are meant for active/passive connections and the prefix lengths advertised\\\\n are the same. This value is set per Region to prefer Direct\\\\n Connect locations that have the same associated AWS Region using the\\\\n `7224:7200`\\u2014Medium local preference community value. Where\\\\n the local Region is not associated with the Direct Connect\\\\n location, it is set to a lower value. This applies only if no local preference\\\\n community tags are assigned. For more information about associated Regions, see\\\\n Associated Region.\\\\n* AS_PATH length can be used to determine the routing path when the prefix\\\\n length and local preference are the same.\\\\n* Multi-Exit Discriminator (MED) can be used to determine the routing path when\\\\n prefix length, local preference, and AS_PATH are the same. AWS does not\\\\n recommend using MED values given their lower priority in evaluation.\\\\n* AWS uses equal-cost multi-path (ECMP) routing across multiple transit or\\\\n private virtual interfaces when prefixes have the same AS_PATH length and BGP\\\\n attributes. The ASNs in the AS_PATH of the prefixes do not need to match\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"Direct Connect private virtual interface routing example\\\",\\\"context\\\":\\\"# Direct Connect private virtual interface routing example\\\\n\\\\nConsider the configuration where the Direct Connect location 1 home Region is the same as\\\\nthe VPC home Region. There is a redundant Direct Connect location in a different Region There\\\\nare two private VIFs (VIF A and VIF B) from Direct Connect location 1 (us-east-1) to the\\\\nDirect Connect gateway. There is one private VIF (VIF C) from Direct Connect location\\\\n(us-west-1) to the Direct Connect gateway. To have AWS route traffic over VIF B before\\\\nVIF A, set the AS_PATH attribute of VIF B to be shorter than the VIF A AS_PATH\\\\nattribute.\\\\n\\\\nThe VIFs have the following configurations:\\\\n\\\\n* VIF A (in us-east-1) advertises 172.16.0.0/16 and has an AS_PATH attribute of\\\\n 65001, 65001, 65001\\\\n* VIF B (in us-east-1) advertises 172.16.0.0/16 and has an AS_PATH attribute of\\\\n 65001, 65001\\\\n* VIF C (in us-west-1) advertises 172.16.0.0/16 and has an AS_PATH attribute of\\\\n 65001\\\\n\\\\nIf you change the CIDR range configuration of VIF C, routes that fall in to the VIF C\\\\nCIDR range use VIF C because it has the longest prefix length.\\\\n\\\\n* VIF C (in us-west-1) advertises 172.16.0.0/24 and has an AS_PATH attribute of\\\\n 65001\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/directconnect/latest/UserGuide/private-transit-vif-example.html\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\n\\\\n\\\\nServices: directconnect, vpc, transitgateway\\\\nTasks: configure, secure, deploy\\\\nPersona: network-admin, devops, developer, sre\\\\nWorkload: networking, hybrid-cloud\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"}]}}\"}]}], \"label\": \"Verify AWS routing preference between Direct Connect and Site-to-Site VPN to same VGW\"}", + "createdAt": "2026-10-27T17:05:12.740000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "7e8c9c50-c2d5-4c5b-9335-5e9a4574bce4", + "content": "{\"id\": \"7e8c9c50-c2d5-4c5b-9335-5e9a4574bce4\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_8pAOJvjqSbQN52e89vGSy3\", \"content\": \"[{'text': '{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"Direct Connect routing policies and BGP communities\\\",\\\"context\\\":\\\"## Private virtual interface and transit virtual interface routing policies\\\\\\\\n\\\\\\\\nIf you\\\\'re using AWS Direct Connect to access your private AWS resources,\\\\\\\\nyou must specify the IPv4 or IPv6 prefixes to advertise over BGP. These prefixes can be\\\\\\\\npublic or private.\\\\\\\\n\\\\\\\\nThe following outbound routing rules apply based on the prefixes advertised:\\\\\\\\n\\\\\\\\n* AWS evaluates the longest prefix length first. AWS recommends advertising\\\\\\\\n more specific routes using multiple Direct Connect virtual interfaces if the\\\\\\\\n desired routing paths are meant for active/passive connections. See Influencing Traffic over Hybrid Networks using Longest Prefix\\\\\\\\n Match for more information.\\\\\\\\n* Local preference is the BGP attribute recommended to use when desired routing\\\\\\\\n paths are meant for active/passive connections and the prefix lengths advertised\\\\\\\\n are the same. This value is set per Region to prefer Direct\\\\\\\\n Connect locations that have the same associated AWS Region using the\\\\\\\\n `7224:7200`\\u2014Medium local preference community value. Where\\\\\\\\n the local Region is not associated with the Direct Connect\\\\\\\\n location, it is set to a lower value. This applies only if no local preference\\\\\\\\n community tags are assigned. For more information about associated Regions, see\\\\\\\\n Associated Region.\\\\\\\\n* AS_PATH length can be used to determine the routing path when the prefix\\\\\\\\n length and local preference are the same.\\\\\\\\n* Multi-Exit Discriminator (MED) can be used to determine the routing path when\\\\\\\\n prefix length, local preference, and AS_PATH are the same. AWS does not\\\\\\\\n recommend using MED values given their lower priority in evaluation.\\\\\\\\n* AWS uses equal-cost multi-path (ECMP) routing across multiple transit or\\\\\\\\n private virtual interfaces when prefixes have the same AS_PATH length and BGP\\\\\\\\n attributes. The ASNs in the AS_PATH of the prefixes do not need to match\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"Direct Connect private virtual interface routing example\\\",\\\"context\\\":\\\"# Direct Connect private virtual interface routing example\\\\\\\\n\\\\\\\\nConsider the configuration where the Direct Connect location 1 home Region is the same as\\\\\\\\nthe VPC home Region. There is a redundant Direct Connect location in a different Region There\\\\\\\\nare two private VIFs (VIF A and VIF B) from Direct Connect location 1 (us-east-1) to the\\\\\\\\nDirect Connect gateway. There is one private VIF (VIF C) from Direct Connect location\\\\\\\\n(us-west-1) to the Direct Connect gateway. To have AWS route traffic over VIF B before\\\\\\\\nVIF A, set the AS_PATH attribute of VIF B to be shorter than the VIF A AS_PATH\\\\\\\\nattribute.\\\\\\\\n\\\\\\\\nThe VIFs have the following configurations:\\\\\\\\n\\\\\\\\n* VIF A (in us-east-1) advertises 172.16.0.0/16 and has an AS_PATH attribute of\\\\\\\\n 65001, 65001, 65001\\\\\\\\n* VIF B (in us-east-1) advertises 172.16.0.0/16 and has an AS_PATH attribute of\\\\\\\\n 65001, 65001\\\\\\\\n* VIF C (in us-west-1) advertises 172.16.0.0/16 and has an AS_PATH attribute of\\\\\\\\n 65001\\\\\\\\n\\\\\\\\nIf you change the CIDR range configuration of VIF C, routes that fall in to the VIF C\\\\\\\\nCIDR range use VIF C because it has the longest prefix length.\\\\\\\\n\\\\\\\\n* VIF C (in us-west-1) advertises 172.16.0.0/24 and has an AS_PATH attribute of\\\\\\\\n 65001\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/directconnect/latest/UserGuide/private-transit-vif-example.html\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: directconnect, vpc, transitgateway\\\\\\\\nTasks: configure, secure, deploy\\\\\\\\nPersona: network-admin, devops, developer, sre\\\\\\\\nWorkload: networking, hybrid-cloud\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"}]}}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:20.668000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "3325a4ae-c325-46da-ae1d-bdb6f45cde40", + "content": "{\"id\": \"eca2d539-c785-4eaa-bac1-bc10b933f2b3\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This confirms AWS's well-documented preference: when the same prefix is learned via both Direct Connect and VPN to the same VGW, AWS assigns higher local preference to DX-learned BGP routes by default, so DX wins over VPN.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:20.668000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "9ba8f036-3f3f-4f9b-bc99-f2662cc62ce3", + "content": "{\"id\": \"4c98374b-43c1-49db-aeae-80457cde4b1b\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"3206\"}]}", + "createdAt": "2026-10-27T17:05:20.736000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "8ffc954b-012a-4b23-85c9-533899e0c3cd", + "content": "{\"id\": \"8ffc954b-012a-4b23-85c9-533899e0c3cd\", \"role\": \"assistant\", \"content\": [{\"id\": \"tooluse_4X0HyUzzIDh359r3hjV0iM\", \"tool_name\": \"verify_aws_claim\", \"input\": {\"claim\": \"A virtual private gateway (VGW) prefers routes learned via a Direct Connect BGP session over routes learned via a Site-to-Site VPN BGP session when prefix lengths are equal, due to a higher default local preference assigned to Direct Connect\", \"topics\": [\"reference_documentation\"]}, \"type\": \"tool_use\", \"display_intent\": \"Verify VGW path preference between Direct Connect BGP routes and VPN BGP routes\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:20.739000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "8842bb78-cfdc-4727-a358-0518ac15cff3", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.0, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:20.812000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "93369a40-dd25-40cd-b08b-271f65848c4e", + "content": "{\"id\": \"8f52eec2-72ba-45d0-aa8f-0155812d6a8b\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_4X0HyUzzIDh359r3hjV0iM\", \"status\": \"success\", \"content\": [{\"text\": \"{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"Direct Connect routing policies and BGP communities\\\",\\\"context\\\":\\\"## Private virtual interface and transit virtual interface routing policies\\\\n\\\\nIf you're using AWS Direct Connect to access your private AWS resources,\\\\nyou must specify the IPv4 or IPv6 prefixes to advertise over BGP. These prefixes can be\\\\npublic or private.\\\\n\\\\nThe following outbound routing rules apply based on the prefixes advertised:\\\\n\\\\n* AWS evaluates the longest prefix length first. AWS recommends advertising\\\\n more specific routes using multiple Direct Connect virtual interfaces if the\\\\n desired routing paths are meant for active/passive connections. See Influencing Traffic over Hybrid Networks using Longest Prefix\\\\n Match for more information.\\\\n* Local preference is the BGP attribute recommended to use when desired routing\\\\n paths are meant for active/passive connections and the prefix lengths advertised\\\\n are the same. This value is set per Region to prefer Direct\\\\n Connect locations that have the same associated AWS Region using the\\\\n `7224:7200`\\u2014Medium local preference community value. Where\\\\n the local Region is not associated with the Direct Connect\\\\n location, it is set to a lower value. This applies only if no local preference\\\\n community tags are assigned. For more information about associated Regions, see\\\\n Associated Region.\\\\n* AS_PATH length can be used to determine the routing path when the prefix\\\\n length and local preference are the same.\\\\n* Multi-Exit Discriminator (MED) can be used to determine the routing path when\\\\n prefix length, local preference, and AS_PATH are the same. AWS does not\\\\n recommend using MED values given their lower priority in evaluation.\\\\n* AWS uses equal-cost multi-path (ECMP) routing across multiple transit or\\\\n private virtual interfaces when prefixes have the same AS_PATH length and BGP\\\\n attributes. The ASNs in the AS_PATH of the prefixes do not need to match\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\n\\\\n\\\\nServices: directconnect, vpc, transitgateway\\\\nTasks: configure, secure, deploy\\\\nPersona: network-admin, devops, developer, sre\\\\nWorkload: networking, hybrid-cloud\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"sitetositevpn\\\",\\\"skill_description\\\":\\\"Configures AWS Site-to-Site VPN: creating an IPsec VPN connection between an on-premises network and a VPC, choosing the target gateway (virtual private gateway, transit gateway, or AWS Cloud WAN), choosing static or dynamic (BGP) routing, sizing tunnel bandwidth (Standard 1.25 Gbps or Large 5 Gbps), connecting many sites through a VPN Concentrator, applying the customer gateway device configuration, making a connection highly available, and monitoring tunnels with CloudWatch. Applicable when the user wants to connect a data center or branch office to AWS over an encrypted tunnel, choose how routes are exchanged, scale throughput, consolidate sites, or diagnose a down tunnel. Routes to the right per-task procedure in references. Not for AWS Direct Connect (its own service), Client VPN for individual remote users, the transit gateway side of a VPN attachment (transitgateway skill), or Route 53 DNS work.\\\\n\\\\n\\\\nServices: vpn, vpc, transitgateway, cloudwatch\\\\nTasks: configure, secure, monitor, deploy\\\\nPersona: developer, devops, network-admin, sre\\\\nWorkload: networking, hybrid, disaster-recovery\\\\n*\\\",\\\"skill_name\\\":\\\"sitetositevpn\\\"}]}}\"}]}], \"label\": \"Verify VGW path preference between Direct Connect BGP routes and VPN BGP routes\"}", + "createdAt": "2026-10-27T17:05:22.179000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "b5501802-e0a0-4e4e-b640-8f6e4bad381e", + "content": "{\"id\": \"f736d360-eb32-47fc-87cf-e79a2e941d82\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"I'm recalling that AWS documentation confirms general BGP path-selection rules, and specifically that AWS always prefers Direct Connect over VPN when the same prefix has equal AS_PATH length\\u2014this is documented behavior in the Direct Connect/VPN backup guide I should verify directly.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:27.498000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "38d4878c-bae3-4482-a56b-ecf8797b5672", + "content": "{\"id\": \"38d4878c-bae3-4482-a56b-ecf8797b5672\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_4X0HyUzzIDh359r3hjV0iM\", \"content\": \"[{'text': '{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"Direct Connect routing policies and BGP communities\\\",\\\"context\\\":\\\"## Private virtual interface and transit virtual interface routing policies\\\\\\\\n\\\\\\\\nIf you\\\\'re using AWS Direct Connect to access your private AWS resources,\\\\\\\\nyou must specify the IPv4 or IPv6 prefixes to advertise over BGP. These prefixes can be\\\\\\\\npublic or private.\\\\\\\\n\\\\\\\\nThe following outbound routing rules apply based on the prefixes advertised:\\\\\\\\n\\\\\\\\n* AWS evaluates the longest prefix length first. AWS recommends advertising\\\\\\\\n more specific routes using multiple Direct Connect virtual interfaces if the\\\\\\\\n desired routing paths are meant for active/passive connections. See Influencing Traffic over Hybrid Networks using Longest Prefix\\\\\\\\n Match for more information.\\\\\\\\n* Local preference is the BGP attribute recommended to use when desired routing\\\\\\\\n paths are meant for active/passive connections and the prefix lengths advertised\\\\\\\\n are the same. This value is set per Region to prefer Direct\\\\\\\\n Connect locations that have the same associated AWS Region using the\\\\\\\\n `7224:7200`\\u2014Medium local preference community value. Where\\\\\\\\n the local Region is not associated with the Direct Connect\\\\\\\\n location, it is set to a lower value. This applies only if no local preference\\\\\\\\n community tags are assigned. For more information about associated Regions, see\\\\\\\\n Associated Region.\\\\\\\\n* AS_PATH length can be used to determine the routing path when the prefix\\\\\\\\n length and local preference are the same.\\\\\\\\n* Multi-Exit Discriminator (MED) can be used to determine the routing path when\\\\\\\\n prefix length, local preference, and AS_PATH are the same. AWS does not\\\\\\\\n recommend using MED values given their lower priority in evaluation.\\\\\\\\n* AWS uses equal-cost multi-path (ECMP) routing across multiple transit or\\\\\\\\n private virtual interfaces when prefixes have the same AS_PATH length and BGP\\\\\\\\n attributes. The ASNs in the AS_PATH of the prefixes do not need to match\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: directconnect, vpc, transitgateway\\\\\\\\nTasks: configure, secure, deploy\\\\\\\\nPersona: network-admin, devops, developer, sre\\\\\\\\nWorkload: networking, hybrid-cloud\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"sitetositevpn\\\",\\\"skill_description\\\":\\\"Configures AWS Site-to-Site VPN: creating an IPsec VPN connection between an on-premises network and a VPC, choosing the target gateway (virtual private gateway, transit gateway, or AWS Cloud WAN), choosing static or dynamic (BGP) routing, sizing tunnel bandwidth (Standard 1.25 Gbps or Large 5 Gbps), connecting many sites through a VPN Concentrator, applying the customer gateway device configuration, making a connection highly available, and monitoring tunnels with CloudWatch. Applicable when the user wants to connect a data center or branch office to AWS over an encrypted tunnel, choose how routes are exchanged, scale throughput, consolidate sites, or diagnose a down tunnel. Routes to the right per-task procedure in references. Not for AWS Direct Connect (its own service), Client VPN for individual remote users, the transit gateway side of a VPN attachment (transitgateway skill), or Route 53 DNS work.\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: vpn, vpc, transitgateway, cloudwatch\\\\\\\\nTasks: configure, secure, monitor, deploy\\\\\\\\nPersona: developer, devops, network-admin, sre\\\\\\\\nWorkload: networking, hybrid, disaster-recovery\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"sitetositevpn\\\"}]}}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:27.498000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "d8e3b0a2-3abb-442a-be88-54336649c04d", + "content": "{\"id\": \"1a9229dd-d32a-4b82-bf20-f0c5e58ee0b1\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"1593\"}]}", + "createdAt": "2026-10-27T17:05:27.598000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "ad71ef9d-9931-480a-b851-b216a74a0d3c", + "content": "{\"id\": \"ad71ef9d-9931-480a-b851-b216a74a0d3c\", \"role\": \"assistant\", \"content\": [{\"id\": \"tooluse_eG9cYz7lekG5EM2GGc6LJd\", \"tool_name\": \"verify_aws_claim\", \"input\": {\"claim\": \"AWS Site-to-Site VPN as a backup for Direct Connect: how AWS selects Direct Connect over VPN and how to avoid asymmetric routing\", \"topics\": [\"reference_documentation\"]}, \"type\": \"tool_use\", \"display_intent\": \"Check AWS doc on using Site-to-Site VPN as backup for Direct Connect failover behavior\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:27.627000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "9a1ec9b1-eb15-4529-9137-f7180c3354d4", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.2, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:27.701000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "f32149fc-a672-4036-825a-37044404902c", + "content": "{\"id\": \"0e220d38-74bd-4028-9836-ddaea1f3c8b8\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_eG9cYz7lekG5EM2GGc6LJd\", \"status\": \"success\", \"content\": [{\"text\": \"{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"AWS Direct Connect as Primary and AWS Site-to-Site VPN as Backup\\\",\\\"context\\\":\\\"## AWS Direct Connect primary and VPN backup architecture\\\\n\\\\nThe following steps describe the on-premises to AWS traffic flow:\\\\n\\\\n1. Traffic from the office branch destined to the **Spoke VPC B** is forwarded to the Transit Gateway through the AWS Direct Connect link. This behavior can be achieved by configuring the office branch devices with higher BGP local preference pointing to the DX peer. The Transit Gateway is connected to Direct Connect by using a Transit VIF and a Direct Connect Gateway.\\\\n2. As per the **Transit Gateway on-premises route table**, the traffic is forwarded to the **spoke VPC B** attachment.\\\\n3. The TGW ENI of the **spoke VPC B** forwards the traffic to the destination.\\\\n\\\\nIn the event of an AWS Direct Connect failure:\\\\n\\\\n1. Traffic from the office branch destined to the **spoke VPC B** is forwarded to the Transit Gateway through the AWS Site-to-Site VPN connection.\\\\n2. As per the **Transit Gateway on-premises route table**, the traffic is forwarded to the **spoke VPC B** attachment.\\\\n3. The TGW ENI of the **spoke VPC B** forwards the traffic to the destination.\\\\n\\\\nFor more information about how to configure AWS Direct Connect with AWS Site-to-Site VPN as a backup, see Using a VPN connection as a backup to Direct Connect.\\\\n\\\\nFor more information about asymmetric routing, see Resolve asymmetric routing issues on AWS re:Post\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/reference-architecture-diagrams/latest/hybrid-connectivity-transit-gateway/hybrid-dx-primary-vpn-backup.html\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"Reliability\\\",\\\"context\\\":\\\"## VPN connection as a backup to AWS DX connection example\\\\n\\\\nVPN can be selected to provide a backup network connection to an AWS Direct Connect\\\\nconnection. Typically, this type of connectivity model is driven by cost, because it\\\\nprovides a lower level of reliability to the overall hybrid connectivity solution due to\\\\nindeterministic performance over the internet, and there is no SLA that can be obtained for\\\\na connection over the public internet. It is a valid and cost-effective connectivity model,\\\\nand should be used when cost is the top priority consideration and there is a limited\\\\nbudget, or possibly as an interim solution until a secondary DX can be provisioned. Figure\\\\n12 illustrates the design of this connectivity model. One key consideration with this\\\\ndesign, where both the VPN and DX connections are terminating at the AWS Transit Gateway, is that\\\\nthe VPN connection can advertise higher number of routes compared to the ones that can be\\\\nadvertised over a DX connection connected to AWS Transit Gateway. This may cause a suboptimal\\\\nrouting situation. An option to resolve this issue is to configure route filtering at the\\\\ncustomer gateway device (CGW) for the routes received from the VPN connection, allowing only\\\\nthe summary routes to be accepted.\\\\n\\\\nNote: To create the summary route on the AWS Transit Gateway, you need to specify a static\\\\nroute to an arbitrary attachment in the AWS Transit Gateway route table so that the summary is sent\\\\nalong the more specific route.\\\\n\\\\nFrom the AWS Transit Gateway routing table\\u2019s point of view, the routes for the on-premises\\\\nprefix are received both from the AWS DX connection (via DXGW) and from VPN, with the same\\\\nprefix length. Following the route\\\\npriority logic of AWS Transit Gateway, routes received over Direct Connect have a higher\\\\npreference than the ones received over Site-to-Site VPN, and thus the path over the AWS Direct Connect will\\\\nbe the preferred to reach the on-premises network(s).\\\\n\\\\n*Figure 12 \\u2013 VPN connection as a backup to AWS DX connection example*\\\\n\\\\nThe following decision tree guides you through making the desired decision for\\\\nachieving a resilient (which will result in a reliably) hybrid network connectivity. For\\\\nmore information, refer to AWS Direct Connect Resiliency\\\\nToolkit.\\\\n\\\\n*Figure 13 \\u2013 Reliability decision tree*\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/whitepapers/latest/hybrid-connectivity/reliability.html\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\n\\\\n\\\\nServices: directconnect, vpc, transitgateway\\\\nTasks: configure, secure, deploy\\\\nPersona: network-admin, devops, developer, sre\\\\nWorkload: networking, hybrid-cloud\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"}]}}\"}]}], \"label\": \"Check AWS doc on using Site-to-Site VPN as backup for Direct Connect failover behavior\"}", + "createdAt": "2026-10-27T17:05:28.894000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "1419e26e-9995-4d31-a3ff-233c2d8ba923", + "content": "{\"id\": \"1419e26e-9995-4d31-a3ff-233c2d8ba923\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_eG9cYz7lekG5EM2GGc6LJd\", \"content\": \"[{'text': '{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"AWS Direct Connect as Primary and AWS Site-to-Site VPN as Backup\\\",\\\"context\\\":\\\"## AWS Direct Connect primary and VPN backup architecture\\\\\\\\n\\\\\\\\nThe following steps describe the on-premises to AWS traffic flow:\\\\\\\\n\\\\\\\\n1. Traffic from the office branch destined to the **Spoke VPC B** is forwarded to the Transit Gateway through the AWS Direct Connect link. This behavior can be achieved by configuring the office branch devices with higher BGP local preference pointing to the DX peer. The Transit Gateway is connected to Direct Connect by using a Transit VIF and a Direct Connect Gateway.\\\\\\\\n2. As per the **Transit Gateway on-premises route table**, the traffic is forwarded to the **spoke VPC B** attachment.\\\\\\\\n3. The TGW ENI of the **spoke VPC B** forwards the traffic to the destination.\\\\\\\\n\\\\\\\\nIn the event of an AWS Direct Connect failure:\\\\\\\\n\\\\\\\\n1. Traffic from the office branch destined to the **spoke VPC B** is forwarded to the Transit Gateway through the AWS Site-to-Site VPN connection.\\\\\\\\n2. As per the **Transit Gateway on-premises route table**, the traffic is forwarded to the **spoke VPC B** attachment.\\\\\\\\n3. The TGW ENI of the **spoke VPC B** forwards the traffic to the destination.\\\\\\\\n\\\\\\\\nFor more information about how to configure AWS Direct Connect with AWS Site-to-Site VPN as a backup, see Using a VPN connection as a backup to Direct Connect.\\\\\\\\n\\\\\\\\nFor more information about asymmetric routing, see Resolve asymmetric routing issues on AWS re:Post\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/reference-architecture-diagrams/latest/hybrid-connectivity-transit-gateway/hybrid-dx-primary-vpn-backup.html\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"Reliability\\\",\\\"context\\\":\\\"## VPN connection as a backup to AWS DX connection example\\\\\\\\n\\\\\\\\nVPN can be selected to provide a backup network connection to an AWS Direct Connect\\\\\\\\nconnection. Typically, this type of connectivity model is driven by cost, because it\\\\\\\\nprovides a lower level of reliability to the overall hybrid connectivity solution due to\\\\\\\\nindeterministic performance over the internet, and there is no SLA that can be obtained for\\\\\\\\na connection over the public internet. It is a valid and cost-effective connectivity model,\\\\\\\\nand should be used when cost is the top priority consideration and there is a limited\\\\\\\\nbudget, or possibly as an interim solution until a secondary DX can be provisioned. Figure\\\\\\\\n12 illustrates the design of this connectivity model. One key consideration with this\\\\\\\\ndesign, where both the VPN and DX connections are terminating at the AWS Transit Gateway, is that\\\\\\\\nthe VPN connection can advertise higher number of routes compared to the ones that can be\\\\\\\\nadvertised over a DX connection connected to AWS Transit Gateway. This may cause a suboptimal\\\\\\\\nrouting situation. An option to resolve this issue is to configure route filtering at the\\\\\\\\ncustomer gateway device (CGW) for the routes received from the VPN connection, allowing only\\\\\\\\nthe summary routes to be accepted.\\\\\\\\n\\\\\\\\nNote: To create the summary route on the AWS Transit Gateway, you need to specify a static\\\\\\\\nroute to an arbitrary attachment in the AWS Transit Gateway route table so that the summary is sent\\\\\\\\nalong the more specific route.\\\\\\\\n\\\\\\\\nFrom the AWS Transit Gateway routing table\\u2019s point of view, the routes for the on-premises\\\\\\\\nprefix are received both from the AWS DX connection (via DXGW) and from VPN, with the same\\\\\\\\nprefix length. Following the route\\\\\\\\npriority logic of AWS Transit Gateway, routes received over Direct Connect have a higher\\\\\\\\npreference than the ones received over Site-to-Site VPN, and thus the path over the AWS Direct Connect will\\\\\\\\nbe the preferred to reach the on-premises network(s).\\\\\\\\n\\\\\\\\n*Figure 12 \\u2013 VPN connection as a backup to AWS DX connection example*\\\\\\\\n\\\\\\\\nThe following decision tree guides you through making the desired decision for\\\\\\\\nachieving a resilient (which will result in a reliably) hybrid network connectivity. For\\\\\\\\nmore information, refer to AWS Direct Connect Resiliency\\\\\\\\nToolkit.\\\\\\\\n\\\\\\\\n*Figure 13 \\u2013 Reliability decision tree*\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/whitepapers/latest/hybrid-connectivity/reliability.html\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: directconnect, vpc, transitgateway\\\\\\\\nTasks: configure, secure, deploy\\\\\\\\nPersona: network-admin, devops, developer, sre\\\\\\\\nWorkload: networking, hybrid-cloud\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"}]}}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:41.315000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "c6f969fc-dc53-46de-95a7-060b3d318276", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.5, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:41.403000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "ca50bd7c-0bc3-4198-8c3f-d88386a4a28d", + "content": "{\"id\": \"ca50bd7c-0bc3-4198-8c3f-d88386a4a28d\", \"role\": \"assistant\", \"content\": [{\"text\": \"This confirms exactly what you'd expect: with equal prefix length, **AWS always prefers the Direct Connect path over the VPN path** to the VGW \\u2014 this is a hard-coded AWS routing preference, not something you configure via local preference or AS_PATH on the AWS side. You don't need to (and can't) tune AWS's inbound preference between the two; DX wins automatically whenever it's up and advertising the same or a more specific prefix.\\n\\nWhere asymmetric routing actually creeps in is almost always on **your side** \\u2014 the path AWS picks outbound (DX, by default) doesn't match the path your on-prem network picks inbound toward AWS. A few concrete causes and fixes:\\n\\n**Causes**\\n- Your on-prem/CGW router doesn't give DX a clear preference, so return traffic (toward AWS) sometimes goes out the VPN while traffic from AWS comes back over DX, or vice versa.\\n- The VPN connection advertises a longer/more specific prefix than DX (longest-prefix-match always wins regardless of local preference), accidentally making VPN the active path for some subnets.\\n- Stateful devices (firewalls, NAT) on-prem seeing only one leg of a flow and dropping it because the return path differs.\\n\\n**How to keep it symmetric**\\n1. **Match prefix granularity on both paths.** Advertise the *same* prefixes (same length) over DX and VPN. Don't let the VPN side get more specific \\u2014 that silently overrides AWS's DX preference via longest-prefix-match.\\n2. **Set local preference on your on-prem BGP routers** so outbound (on-prem \\u2192 AWS) traffic also prefers the path reachable via DX, mirroring AWS's own preference inbound. Since AWS already prefers DX automatically, aligning your CGW to do the same keeps both directions on DX.\\n3. **Use consistent AS_PATH length** for the same prefix on both circuits if you're not using local preference, since AS_PATH is the next tiebreaker after local preference.\\n4. If you're running this through a Transit Gateway rather than directly at the VGW, apply route filtering on the CGW so the VPN only advertises summary routes \\u2014 otherwise VPN's more specific routes can win on longest-prefix-match even with DX up.\\n5. Avoid flow-breaking stateful devices across the two paths, or make sure they can tolerate asymmetric return traffic during failover.\\n\\nIf you want, I can take a look at your actual VGW/Transit Gateway route tables and the prefixes each connection is advertising to check for a longest-prefix-match mismatch \\u2014 that's the most common real-world cause of this.\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:41.483000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "61faea88-7a8a-49a1-bf3e-bdcbb07a5948", + "executionId": "f5067c13-7ce0-42a9-b745-8934b0727bc5", + "recordId": "0c158d81-ecb0-4c2a-b5a2-7be199fa5a91", + "content": "{\"id\": \"9753e31e-209a-479a-90db-f185d5bf5926\", \"role\": \"assistant\", \"content\": [{\"type\": \"text\", \"text\": \"This confirms exactly what you'd expect: with equal prefix length, **AWS always prefers the Direct Connect path over the VPN path** to the VGW \\u2014 this is a hard-coded AWS routing preference, not something you configure via local preference or AS_PATH on the AWS side. You don't need to (and can't) tune AWS's inbound preference between the two; DX wins automatically whenever it's up and advertising the same or a more specific prefix.\\n\\nWhere asymmetric routing actually creeps in is almost always on **your side** \\u2014 the path AWS picks outbound (DX, by default) doesn't match the path your on-prem network picks inbound toward AWS. A few concrete causes and fixes:\\n\\n**Causes**\\n- Your on-prem/CGW router doesn't give DX a clear preference, so return traffic (toward AWS) sometimes goes out the VPN while traffic from AWS comes back over DX, or vice versa.\\n- The VPN connection advertises a longer/more specific prefix than DX (longest-prefix-match always wins regardless of local preference), accidentally making VPN the active path for some subnets.\\n- Stateful devices (firewalls, NAT) on-prem seeing only one leg of a flow and dropping it because the return path differs.\\n\\n**How to keep it symmetric**\\n1. **Match prefix granularity on both paths.** Advertise the *same* prefixes (same length) over DX and VPN. Don't let the VPN side get more specific \\u2014 that silently overrides AWS's DX preference via longest-prefix-match.\\n2. **Set local preference on your on-prem BGP routers** so outbound (on-prem \\u2192 AWS) traffic also prefers the path reachable via DX, mirroring AWS's own preference inbound. Since AWS already prefers DX automatically, aligning your CGW to do the same keeps both directions on DX.\\n3. **Use consistent AS_PATH length** for the same prefix on both circuits if you're not using local preference, since AS_PATH is the next tiebreaker after local preference.\\n4. If you're running this through a Transit Gateway rather than directly at the VGW, apply route filtering on the CGW so the VPN only advertises summary routes \\u2014 otherwise VPN's more specific routes can win on longest-prefix-match even with DX up.\\n5. Avoid flow-breaking stateful devices across the two paths, or make sure they can tolerate asymmetric return traffic during failover.\\n\\nIf you want, I can take a look at your actual VGW/Transit Gateway route tables and the prefixes each connection is advertising to check for a longest-prefix-match mismatch \\u2014 that's the most common real-world cause of this.\"}]}", + "createdAt": "2026-10-27T17:05:41.555000-05:00", + "recordType": "final_response" + } +] \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/negative-dynamodb-iam-policy/with_skill/functional-tests-results.json b/skills/aws-routing/evals/functional/v1/iteration-1/negative-dynamodb-iam-policy/with_skill/functional-tests-results.json new file mode 100644 index 00000000..da714ab5 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/negative-dynamodb-iam-policy/with_skill/functional-tests-results.json @@ -0,0 +1,19 @@ +{ + "version": "v1", + "iteration": 1, + "eval_id": "negative-dynamodb-iam-policy", + "run_type": "with_skill", + "success": true, + "error": null, + "understanding_agent_space_skill": null, + "tests": { + "trigger": { + "result": "passed", + "message": "Skill 'aws-routing' was correctly NOT triggered", + "skill_loads_found": 0 + }, + "expected_output": null, + "assertions": null + }, + "metrics": null +} \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/negative-dynamodb-iam-policy/with_skill/outputs/journal_records.json b/skills/aws-routing/evals/functional/v1/iteration-1/negative-dynamodb-iam-policy/with_skill/outputs/journal_records.json new file mode 100644 index 00000000..13a89c40 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/negative-dynamodb-iam-policy/with_skill/outputs/journal_records.json @@ -0,0 +1,90 @@ +[ + { + "agentSpaceId": "5ebb3d7a-03c9-40ed-9991-aadbac3bbe2a", + "executionId": "a28bae56-e59e-43e9-8426-510b6ccc8a61", + "recordId": "4f3ff629-7968-48a9-a1e2-0d80fa4858ec", + "content": "{\"id\": \"4f3ff629-7968-48a9-a1e2-0d80fa4858ec\", \"role\": \"user\", \"content\": [{\"text\": \"What IAM policy do I need to allow read-only access to DynamoDB?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:04.172000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "5ebb3d7a-03c9-40ed-9991-aadbac3bbe2a", + "executionId": "a28bae56-e59e-43e9-8426-510b6ccc8a61", + "recordId": "8e990e24-3f6c-407e-b86e-109f74b8da84", + "content": "{\"id\": \"156b310d-03eb-4d68-988c-4bdaf65cf8b8\", \"role\": \"assistant\", \"content\": [{\"type\": \"chat_title\", \"title\": \"DynamoDB read-only IAM policy configuration\"}]}", + "createdAt": "2026-10-27T17:05:04.670000-05:00", + "recordType": "chat_title" + }, + { + "agentSpaceId": "5ebb3d7a-03c9-40ed-9991-aadbac3bbe2a", + "executionId": "a28bae56-e59e-43e9-8426-510b6ccc8a61", + "recordId": "13833a37-b116-4111-a6f8-4b6787e2d11a", + "content": "{\"id\": \"baae2e50-0e9a-449a-bebf-ee90ff382a77\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This is really a general AWS question rather than something specific to the product or the user's infrastructure\\u2014it's asking about IAM policy structure for read-only DynamoDB access, which I could answer from general knowledge or check against AWS's managed policy examples.\\n\\nI recall there's an AWS managed policy called \\\"AmazonDynamoDBReadOnlyAccess\\\" I should verify exists via verify_aws_claim, and I'll also put together a custom policy example for more granular control.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:11.350000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "5ebb3d7a-03c9-40ed-9991-aadbac3bbe2a", + "executionId": "a28bae56-e59e-43e9-8426-510b6ccc8a61", + "recordId": "e5c66b55-296b-4d2f-a943-6a2b707d32e7", + "content": "{\"id\": \"e5c66b55-296b-4d2f-a943-6a2b707d32e7\", \"role\": \"assistant\", \"content\": [{\"text\": \"I'll quickly confirm the AWS-managed policy details before giving you the exact policy.\", \"type\": \"text\"}, {\"id\": \"tooluse_4I2yZwp35KSoGzZCBy0ELg\", \"tool_name\": \"verify_aws_claim\", \"input\": {\"claim\": \"AWS managed policy AmazonDynamoDBReadOnlyAccess grants read-only access to DynamoDB\"}, \"type\": \"tool_use\", \"display_intent\": \"Verify AWS managed IAM policy for DynamoDB read-only access\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:11.377000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "5ebb3d7a-03c9-40ed-9991-aadbac3bbe2a", + "executionId": "a28bae56-e59e-43e9-8426-510b6ccc8a61", + "recordId": "f66408b2-81d3-4f65-9ac6-8cf26e3a5115", + "content": "{\"id\": \"7664d1a6-cedd-47fe-8208-821210a62f06\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"2396\"}]}", + "createdAt": "2026-10-27T17:05:11.426000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "5ebb3d7a-03c9-40ed-9991-aadbac3bbe2a", + "executionId": "a28bae56-e59e-43e9-8426-510b6ccc8a61", + "recordId": "e375ec19-e9fb-46e2-bac1-1fb7907c00af", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 3.8, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:11.460000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "5ebb3d7a-03c9-40ed-9991-aadbac3bbe2a", + "executionId": "a28bae56-e59e-43e9-8426-510b6ccc8a61", + "recordId": "67bf98d3-9342-4b3a-9780-a51ca90a109f", + "content": "{\"id\": \"b07a77d6-0ce6-4b84-8ecf-2d75f3d273b8\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_4I2yZwp35KSoGzZCBy0ELg\", \"status\": \"success\", \"content\": [{\"text\": \"## Documentation search results\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"AmazonDynamoDBReadOnlyAccess\\\",\\\"context\\\":\\\"# AmazonDynamoDBReadOnlyAccess\\\\n\\\\n**Description**: Provides read only access to Amazon DynamoDB via the AWS Management Console.\\\\n\\\\n`AmazonDynamoDBReadOnlyAccess` is an AWS managed policy\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/aws-managed-policy/latest/reference/AmazonDynamoDBReadOnlyAccess.html\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"DynamoDB FSI Service Spotlight | AWS for Industries\\\",\\\"context\\\":\\\"#### Identity-based Policy\\\\n\\\\nYou need an IAM user or IAM role to authenticate your DynamoDB access. Using identity-based policies (IAM policies), you can grant permissions to a person or group in their account to create or access Amazon DynamoDB resources, such as reading from an Amazon DynamoDB table. DynamoDB primary resources are *tables, indexes, and streams*.\\\\n\\\\nFor each DynamoDB resource, the service defines a set of API operations. To grant permissions for these API operations, DynamoDB defines a set of actions that you can specify in a policy. For a list of available operations, see Amazon DynamoDB Actions.\\\\n\\\\nAWS customers in the financial services industry may require its trading desk could only access its own trades within the `Trades` table. The following is an example of identity-based policy that grants permissions for the read-only actions on his/her own trades. You can find more DynamoDB custom policy examples here.\\\\n\\\\n```\\\\n{\\\\n \\\\\\\"Version\\\\\\\": \\\\\\\"2012-10-17\\\\\\\",\\\\n \\\\\\\"Statement\\\\\\\": [\\\\n {\\\\n \\\\\\\"Sid\\\\\\\": \\\\\\\"AllowReadOnlyAccessToOnlyTradesMatchingTraderID\\\\\\\",\\\\n \\\\\\\"Effect\\\\\\\": \\\\\\\"Allow\\\\\\\",\\\\n \\\\\\\"Action\\\\\\\": [\\\\n \\\\\\\"dynamodb:GetItem\\\\\\\",\\\\n \\\\\\\"dynamodb:BatchGetItem\\\\\\\",\\\\n \\\\\\\"dynamodb:Query\\\\\\\",\\\\n \\\\\\\"dynamodb:Scan\\\\\\\",\\\\n \\\\\\\"dynamodb:ConditionCheckItem\\\\\\\"\\\\n ],\\\\n \\\\\\\"Resource\\\\\\\": [\\\\n \\\\\\\"arn:aws:dynamodb:us-east-1:ACCOUNT_ID:table/Trades\\\\\\\"\\\\n ],\\\\n \\\\\\\"Condition\\\\\\\": {\\\\n \\\\\\\"ForAllValues:StringEquals\\\\\\\": {\\\\n \\\\\\\"dynamodb:LeadingKeys\\\\\\\": [\\\\n \\\\\\\"${www.amazon.com:user_id}\\\\\\\"\\\\n ],\\\\n \\\\\\\"dynamodb:Attributes\\\\\\\": [\\\\n \\\\\\\"TraderId\\\\\\\",\\\\n \\\\\\\"TradeId\\\\\\\",\\\\n \\\\\\\"Stock\\\\\\\",\\\\n \\\\\\\"Price\\\\\\\",\\\\n \\\\\\\"Quantity\\\\\\\",\\\\n \\\\\\\"TradeDateTime\\\\\\\"\\\\n ]\\\\n }, \\\\\\\"StringEqualsIfExists\\\\\\\": {\\\\n \\\\\\\"dynamodb:Select\\\\\\\": \\\\\\\"SPECIFIC_ATTRIBUTES\\\\\\\"\\\\n }\\\\n }\\\\n }\\\\n ]\\\\n}\\\\n\\\\n```\\\\n\\\\nTo help our customers avoid having to investigate which permissions are needed for some common use cases, AWS provides the following AWS managed policies:\\\\n\\\\n* **AmazonDynamoDBReadOnlyAccess** \\u2013 Grants read-only access to DynamoDB resources.\\\\n* **AmazonDynamoDBFullAccess** \\u2013 Grants full access to DynamoDB resources.\\\\n\\\\nIt\\u2019s important to implement IAM policies that follow the principle of least privilege and enforce separation of duties with the appropriate authorization for each role. The typical roles can be:\\\\n\\\\n*[truncated \\u2014 additional content omitted]*\\\",\\\"url\\\":\\\"https://aws.amazon.com/blogs/industries/dynamodb-fsi-service-spotlight/\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"How to Create an AWS IAM Policy to Grant AWS Lambda Access to an Amazon DynamoDB Table | AWS Security Blog\\\",\\\"context\\\":\\\"### IAM access policies\\\\n\\\\nI have authored an IAM access policy with JSON to grant the required permissions to the DynamoDB table and CloudWatch Logs. I will attach this policy to a role, and this role will then be attached to a Lambda function, which will assume the required access to DynamoDB and CloudWatch Logs\\\\n\\\\nI will walk through this policy, and explain its elements and how to create the policy in the IAM console.\\\\n\\\\nThe following policy grants a Lambda function read and write access to a DynamoDB table and writes log files to CloudWatch Logs. This policy is called `MyLambdaPolicy`. The following is the full JSON document of this policy (the **AWS account ID** is a placeholder value that you would replace with your own account ID).\\\\n\\\\n```\\\\n{\\\\n\\\\t\\\\\\\"Version\\\\\\\": \\\\\\\"2012-10-17\\\\\\\",\\\\n\\\\t\\\\\\\"Statement\\\\\\\": [{\\\\n\\\\t\\\\t\\\\t\\\\\\\"Effect\\\\\\\": \\\\\\\"Allow\\\\\\\",\\\\n\\\\t\\\\t\\\\t\\\\\\\"Action\\\\\\\": [\\\\n\\\\t\\\\t\\\\t\\\\t\\\\\\\"dynamodb:BatchGetItem\\\\\\\",\\\\n\\\\t\\\\t\\\\t\\\\t\\\\\\\"dynamodb:GetItem\\\\\\\",\\\\n\\\\t\\\\t\\\\t\\\\t\\\\\\\"dynamodb:Query\\\\\\\",\\\\n\\\\t\\\\t\\\\t\\\\t\\\\\\\"dynamodb:Scan\\\\\\\",\\\\n\\\\t\\\\t\\\\t\\\\t\\\\\\\"dynamodb:BatchWriteItem\\\\\\\",\\\\n\\\\t\\\\t\\\\t\\\\t\\\\\\\"dynamodb:PutItem\\\\\\\",\\\\n\\\\t\\\\t\\\\t\\\\t\\\\\\\"dynamodb:UpdateItem\\\\\\\"\\\\n\\\\t\\\\t\\\\t],\\\\n\\\\t\\\\t\\\\t\\\\\\\"Resource\\\\\\\": \\\\\\\"[redacted ARN]\\\\\\\"\\\\n\\\\t\\\\t},\\\\n\\\\t\\\\t{\\\\n\\\\t\\\\t\\\\t\\\\\\\"Effect\\\\\\\": \\\\\\\"Allow\\\\\\\",\\\\n\\\\t\\\\t\\\\t\\\\\\\"Action\\\\\\\": [\\\\n\\\\t\\\\t\\\\t\\\\t\\\\\\\"logs:CreateLogStream\\\\\\\",\\\\n\\\\t\\\\t\\\\t\\\\t\\\\\\\"logs:PutLogEvents\\\\\\\"\\\\n\\\\t\\\\t\\\\t],\\\\n\\\\t\\\\t\\\\t\\\\\\\"Resource\\\\\\\": \\\\\\\"[redacted ARN]\\\\\\\"\\\\n\\\\t\\\\t},\\\\n\\\\t\\\\t{\\\\n\\\\t\\\\t\\\\t\\\\\\\"Effect\\\\\\\": \\\\\\\"Allow\\\\\\\",\\\\n\\\\t\\\\t\\\\t\\\\\\\"Action\\\\\\\": \\\\\\\"logs:CreateLogGroup\\\\\\\",\\\\n\\\\t\\\\t\\\\t\\\\\\\"Resource\\\\\\\": \\\\\\\"*\\\\\\\"\\\\n\\\\t\\\\t}\\\\n\\\\t]\\\\n}\\\\n```\\\\n\\\\nThe first element in this policy is the `Version`, which defines the JSON version. At the time of this post\\u2019s publication, the most recent version of JSON is `2012-10-17`.\\\\n\\\\nThe next element in this first policy is a `Statement`. This is the main section of the policy and includes multiple elements. This first statement is to `Allow` access to DynamoDB, and in this example, the elements I use are:\\\\n\\\\n* An Effect element \\u2013 Specifies whether the statement results in an `Allow` or an explicit `Deny`. By default, access to resources is implicitly denied. In this example\\\\n\\\\n*[truncated \\u2014 additional content omitted]*\\\",\\\"url\\\":\\\"https://aws.amazon.com/blogs/security/how-to-create-an-aws-iam-policy-to-grant-aws-lambda-access-to-an-amazon-dynamodb-table/\\\"}]}}\\n\\n## Recent AWS announcements (What's New)\\n\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"AWS expands Resource Control Policies support to Amazon DynamoDB - AWS\\\",\\\"context\\\":\\\"AWS Resource Control Policies (RCPs), a type of organization policy used to manage permissions in an AWS environment, now support Amazon DynamoDB. RCPs enable central control over the maximum available permissions for resources in an organization. With this expansion, users can manage DynamoDB permissions using RCPs, preventing identities outside the organization from accessing it and enforcing baseline security standards. RCPs are available in all AWS commercial and AWS GovCloud (US) Regions. For more information on RCPs and the list of supported AWS services, visit the Resource control policies (RCPs) documentation in the AWS Organizations User Guide\\\",\\\"url\\\":\\\"https://aws.amazon.com/about-aws/whats-new/2026/02/aws-expands-resource-control-policies-amazon/\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"connecting-lambda-to-dynamodb\\\",\\\"skill_description\\\":\\\"Connects an AWS Lambda function to DynamoDB with IAM roles, stream event source mapping, and read/write permissions. Use when setting up Lambda-DynamoDB integration, processing DynamoDB stream events, or deploying serverless event-driven architectures.\\\\n\\\\nServices: lambda, dynamodb, iam, cloudwatch\\\\nTasks: deploy, configure\\\\nPersona: developer, devops\\\\nWorkload: serverless, database\\\\n*\\\",\\\"skill_name\\\":\\\"connecting-lambda-to-dynamodb\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"configuring-vpc-endpoints-for-private-aws-service-access\\\",\\\"skill_description\\\":\\\"Configures VPC endpoints (interface and gateway) for private AWS service access using AWS PrivateLink. Use when setting up secure private connectivity to S3, DynamoDB, and other AWS services without internet gateway, NAT device, or public IP addresses. Covers endpoint creation, security groups, route tables, and DNS configuration.\\\\n\\\\nServices: ec2, vpc, s3, dynamodb\\\\nTasks: configure\\\\nPersona: developer, devops\\\\nWorkload: networking\\\\n*\\\",\\\"skill_name\\\":\\\"configuring-vpc-endpoints-for-private-aws-service-access\\\"},{\\\"rank_order\\\":4,\\\"title\\\":\\\"amazon-documentdb\\\",\\\"skill_description\\\":\\\"Manages Amazon DocumentDB end-to-end \\u2014 serverless-on-8.0 cluster setup, TLS/VPC/driver config, flexible-schema and vector-search data modeling, MongoDB compatibility assessment, DMS-based migration, slow-query diagnosis, major version upgrades (4.0->5.0->8.0), Well-Architected reviews (41-check wa_review.py), cost estimation, and security hardening. Retrieve for every DocumentDB question and when the user asks to set up or migrate MongoDB to AWS \\u2014 DocumentDB is AWS's MongoDB-compatible managed database. Triggers: JSON document store, document database, MongoDB on AWS, Nested fields, Lambda cannot connect, TLS handshake, VPC port 27017, IAM auth, Secrets Manager, encryption at rest, $graphLookup, flexible schema, COLLSCAN, compound index, DMS migration, CDC cutover, $vectorSearch, RAG, Global Clusters, DR replication, cost sizing, audit, health check, production-readiness.\\\\n\\\\nServices: documentdb, dms, cloudwatch, secrets-manager\\\\nTasks: setup, migrate, optimize, upgrade, troubleshoot, review, create, modify\\\\nPersona: developer, devops, architect, dba\\\\nWorkload: document-database, mongodb-compatible, vector-search, rag\\\\n*\\\",\\\"skill_name\\\":\\\"amazon-documentdb\\\"}]}}\"}]}], \"label\": \"Verify AWS managed IAM policy for DynamoDB read-only access\"}", + "createdAt": "2026-10-27T17:05:12.968000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "5ebb3d7a-03c9-40ed-9991-aadbac3bbe2a", + "executionId": "a28bae56-e59e-43e9-8426-510b6ccc8a61", + "recordId": "767453ef-c3f5-4b97-9173-d78ac4e45ab3", + "content": "{\"id\": \"767453ef-c3f5-4b97-9173-d78ac4e45ab3\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_4I2yZwp35KSoGzZCBy0ELg\", \"content\": \"[{'text': '## Documentation search results\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"AmazonDynamoDBReadOnlyAccess\\\",\\\"context\\\":\\\"# AmazonDynamoDBReadOnlyAccess\\\\\\\\n\\\\\\\\n**Description**: Provides read only access to Amazon DynamoDB via the AWS Management Console.\\\\\\\\n\\\\\\\\n`AmazonDynamoDBReadOnlyAccess` is an AWS managed policy\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/aws-managed-policy/latest/reference/AmazonDynamoDBReadOnlyAccess.html\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"DynamoDB FSI Service Spotlight | AWS for Industries\\\",\\\"context\\\":\\\"#### Identity-based Policy\\\\\\\\n\\\\\\\\nYou need an IAM user or IAM role to authenticate your DynamoDB access. Using identity-based policies (IAM policies), you can grant permissions to a person or group in their account to create or access Amazon DynamoDB resources, such as reading from an Amazon DynamoDB table. DynamoDB primary resources are *tables, indexes, and streams*.\\\\\\\\n\\\\\\\\nFor each DynamoDB resource, the service defines a set of API operations. To grant permissions for these API operations, DynamoDB defines a set of actions that you can specify in a policy. For a list of available operations, see Amazon DynamoDB Actions.\\\\\\\\n\\\\\\\\nAWS customers in the financial services industry may require its trading desk could only access its own trades within the `Trades` table. The following is an example of identity-based policy that grants permissions for the read-only actions on his/her own trades. You can find more DynamoDB custom policy examples here.\\\\\\\\n\\\\\\\\n```\\\\\\\\n{\\\\\\\\n \\\\\\\\\\\"Version\\\\\\\\\\\": \\\\\\\\\\\"2012-10-17\\\\\\\\\\\",\\\\\\\\n \\\\\\\\\\\"Statement\\\\\\\\\\\": [\\\\\\\\n {\\\\\\\\n \\\\\\\\\\\"Sid\\\\\\\\\\\": \\\\\\\\\\\"AllowReadOnlyAccessToOnlyTradesMatchingTraderID\\\\\\\\\\\",\\\\\\\\n \\\\\\\\\\\"Effect\\\\\\\\\\\": \\\\\\\\\\\"Allow\\\\\\\\\\\",\\\\\\\\n \\\\\\\\\\\"Action\\\\\\\\\\\": [\\\\\\\\n \\\\\\\\\\\"dynamodb:GetItem\\\\\\\\\\\",\\\\\\\\n \\\\\\\\\\\"dynamodb:BatchGetItem\\\\\\\\\\\",\\\\\\\\n \\\\\\\\\\\"dynamodb:Query\\\\\\\\\\\",\\\\\\\\n \\\\\\\\\\\"dynamodb:Scan\\\\\\\\\\\",\\\\\\\\n \\\\\\\\\\\"dynamodb:ConditionCheckItem\\\\\\\\\\\"\\\\\\\\n ],\\\\\\\\n \\\\\\\\\\\"Resource\\\\\\\\\\\": [\\\\\\\\n \\\\\\\\\\\"arn:aws:dynamodb:us-east-1:ACCOUNT_ID:table/Trades\\\\\\\\\\\"\\\\\\\\n ],\\\\\\\\n \\\\\\\\\\\"Condition\\\\\\\\\\\": {\\\\\\\\n \\\\\\\\\\\"ForAllValues:StringEquals\\\\\\\\\\\": {\\\\\\\\n \\\\\\\\\\\"dynamodb:LeadingKeys\\\\\\\\\\\": [\\\\\\\\n \\\\\\\\\\\"${www.amazon.com:user_id}\\\\\\\\\\\"\\\\\\\\n ],\\\\\\\\n \\\\\\\\\\\"dynamodb:Attributes\\\\\\\\\\\": [\\\\\\\\n \\\\\\\\\\\"TraderId\\\\\\\\\\\",\\\\\\\\n \\\\\\\\\\\"TradeId\\\\\\\\\\\",\\\\\\\\n \\\\\\\\\\\"Stock\\\\\\\\\\\",\\\\\\\\n \\\\\\\\\\\"Price\\\\\\\\\\\",\\\\\\\\n \\\\\\\\\\\"Quantity\\\\\\\\\\\",\\\\\\\\n \\\\\\\\\\\"TradeDateTime\\\\\\\\\\\"\\\\\\\\n ]\\\\\\\\n }, \\\\\\\\\\\"StringEqualsIfExists\\\\\\\\\\\": {\\\\\\\\n \\\\\\\\\\\"dynamodb:Select\\\\\\\\\\\": \\\\\\\\\\\"SPECIFIC_ATTRIBUTES\\\\\\\\\\\"\\\\\\\\n }\\\\\\\\n }\\\\\\\\n }\\\\\\\\n ]\\\\\\\\n}\\\\\\\\n\\\\\\\\n```\\\\\\\\n\\\\\\\\nTo help our customers avoid having to investigate which permissions are needed for some common use cases, AWS provides the following AWS managed policies:\\\\\\\\n\\\\\\\\n* **AmazonDynamoDBReadOnlyAccess** \\u2013 Grants read-only access to DynamoDB resources.\\\\\\\\n* **AmazonDynamoDBFullAccess** \\u2013 Grants full access to DynamoDB resources.\\\\\\\\n\\\\\\\\nIt\\u2019s important to implement IAM policies that follow the principle of least privilege and enforce separation of duties with the appropriate authorization for each role. The typical roles can be:\\\\\\\\n\\\\\\\\n*[truncated \\u2014 additional content omitted]*\\\",\\\"url\\\":\\\"https://aws.amazon.com/blogs/industries/dynamodb-fsi-service-spotlight/\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"How to Create an AWS IAM Policy to Grant AWS Lambda Access to an Amazon DynamoDB Table | AWS Security Blog\\\",\\\"context\\\":\\\"### IAM access policies\\\\\\\\n\\\\\\\\nI have authored an IAM access policy with JSON to grant the required permissions to the DynamoDB table and CloudWatch Logs. I will attach this policy to a role, and this role will then be attached to a Lambda function, which will assume the required access to DynamoDB and CloudWatch Logs\\\\\\\\n\\\\\\\\nI will walk through this policy, and explain its elements and how to create the policy in the IAM console.\\\\\\\\n\\\\\\\\nThe following policy grants a Lambda function read and write access to a DynamoDB table and writes log files to CloudWatch Logs. This policy is called `MyLambdaPolicy`. The following is the full JSON document of this policy (the **AWS account ID** is a placeholder value that you would replace with your own account ID).\\\\\\\\n\\\\\\\\n```\\\\\\\\n{\\\\\\\\n\\\\\\\\t\\\\\\\\\\\"Version\\\\\\\\\\\": \\\\\\\\\\\"2012-10-17\\\\\\\\\\\",\\\\\\\\n\\\\\\\\t\\\\\\\\\\\"Statement\\\\\\\\\\\": [{\\\\\\\\n\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\\\\"Effect\\\\\\\\\\\": \\\\\\\\\\\"Allow\\\\\\\\\\\",\\\\\\\\n\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\\\\"Action\\\\\\\\\\\": [\\\\\\\\n\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\\\\"dynamodb:BatchGetItem\\\\\\\\\\\",\\\\\\\\n\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\\\\"dynamodb:GetItem\\\\\\\\\\\",\\\\\\\\n\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\\\\"dynamodb:Query\\\\\\\\\\\",\\\\\\\\n\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\\\\"dynamodb:Scan\\\\\\\\\\\",\\\\\\\\n\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\\\\"dynamodb:BatchWriteItem\\\\\\\\\\\",\\\\\\\\n\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\\\\"dynamodb:PutItem\\\\\\\\\\\",\\\\\\\\n\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\\\\"dynamodb:UpdateItem\\\\\\\\\\\"\\\\\\\\n\\\\\\\\t\\\\\\\\t\\\\\\\\t],\\\\\\\\n\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\\\\"Resource\\\\\\\\\\\": \\\\\\\\\\\"[redacted ARN]\\\\\\\\\\\"\\\\\\\\n\\\\\\\\t\\\\\\\\t},\\\\\\\\n\\\\\\\\t\\\\\\\\t{\\\\\\\\n\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\\\\"Effect\\\\\\\\\\\": \\\\\\\\\\\"Allow\\\\\\\\\\\",\\\\\\\\n\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\\\\"Action\\\\\\\\\\\": [\\\\\\\\n\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\\\\"logs:CreateLogStream\\\\\\\\\\\",\\\\\\\\n\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\\\\"logs:PutLogEvents\\\\\\\\\\\"\\\\\\\\n\\\\\\\\t\\\\\\\\t\\\\\\\\t],\\\\\\\\n\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\\\\"Resource\\\\\\\\\\\": \\\\\\\\\\\"[redacted ARN]\\\\\\\\\\\"\\\\\\\\n\\\\\\\\t\\\\\\\\t},\\\\\\\\n\\\\\\\\t\\\\\\\\t{\\\\\\\\n\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\\\\"Effect\\\\\\\\\\\": \\\\\\\\\\\"Allow\\\\\\\\\\\",\\\\\\\\n\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\\\\"Action\\\\\\\\\\\": \\\\\\\\\\\"logs:CreateLogGroup\\\\\\\\\\\",\\\\\\\\n\\\\\\\\t\\\\\\\\t\\\\\\\\t\\\\\\\\\\\"Resource\\\\\\\\\\\": \\\\\\\\\\\"*\\\\\\\\\\\"\\\\\\\\n\\\\\\\\t\\\\\\\\t}\\\\\\\\n\\\\\\\\t]\\\\\\\\n}\\\\\\\\n```\\\\\\\\n\\\\\\\\nThe first element in this policy is the `Version`, which defines the JSON version. At the time of this post\\u2019s publication, the most recent version of JSON is `2012-10-17`.\\\\\\\\n\\\\\\\\nThe next element in this first policy is a `Statement`. This is the main section of the policy and includes multiple elements. This first statement is to `Allow` access to DynamoDB, and in this example, the elements I use are:\\\\\\\\n\\\\\\\\n* An Effect element \\u2013 Specifies whether the statement results in an `Allow` or an explicit `Deny`. By default, access to resources is implicitly denied. In this example\\\\\\\\n\\\\\\\\n*[truncated \\u2014 additional content omitted]*\\\",\\\"url\\\":\\\"https://aws.amazon.com/blogs/security/how-to-create-an-aws-iam-policy-to-grant-aws-lambda-access-to-an-amazon-dynamodb-table/\\\"}]}}\\\\n\\\\n## Recent AWS announcements (What\\\\'s New)\\\\n\\\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"AWS expands Resource Control Policies support to Amazon DynamoDB - AWS\\\",\\\"context\\\":\\\"AWS Resource Control Policies (RCPs), a type of organization policy used to manage permissions in an AWS environment, now support Amazon DynamoDB. RCPs enable central control over the maximum available permissions for resources in an organization. With this expansion, users can manage DynamoDB permissions using RCPs, preventing identities outside the organization from accessing it and enforcing baseline security standards. RCPs are available in all AWS commercial and AWS GovCloud (US) Regions. For more information on RCPs and the list of supported AWS services, visit the Resource control policies (RCPs) documentation in the AWS Organizations User Guide\\\",\\\"url\\\":\\\"https://aws.amazon.com/about-aws/whats-new/2026/02/aws-expands-resource-control-policies-amazon/\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"connecting-lambda-to-dynamodb\\\",\\\"skill_description\\\":\\\"Connects an AWS Lambda function to DynamoDB with IAM roles, stream event source mapping, and read/write permissions. Use when setting up Lambda-DynamoDB integration, processing DynamoDB stream events, or deploying serverless event-driven architectures.\\\\\\\\n\\\\\\\\nServices: lambda, dynamodb, iam, cloudwatch\\\\\\\\nTasks: deploy, configure\\\\\\\\nPersona: developer, devops\\\\\\\\nWorkload: serverless, database\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"connecting-lambda-to-dynamodb\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"configuring-vpc-endpoints-for-private-aws-service-access\\\",\\\"skill_description\\\":\\\"Configures VPC endpoints (interface and gateway) for private AWS service access using AWS PrivateLink. Use when setting up secure private connectivity to S3, DynamoDB, and other AWS services without internet gateway, NAT device, or public IP addresses. Covers endpoint creation, security groups, route tables, and DNS configuration.\\\\\\\\n\\\\\\\\nServices: ec2, vpc, s3, dynamodb\\\\\\\\nTasks: configure\\\\\\\\nPersona: developer, devops\\\\\\\\nWorkload: networking\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"configuring-vpc-endpoints-for-private-aws-service-access\\\"},{\\\"rank_order\\\":4,\\\"title\\\":\\\"amazon-documentdb\\\",\\\"skill_description\\\":\\\"Manages Amazon DocumentDB end-to-end \\u2014 serverless-on-8.0 cluster setup, TLS/VPC/driver config, flexible-schema and vector-search data modeling, MongoDB compatibility assessment, DMS-based migration, slow-query diagnosis, major version upgrades (4.0->5.0->8.0), Well-Architected reviews (41-check wa_review.py), cost estimation, and security hardening. Retrieve for every DocumentDB question and when the user asks to set up or migrate MongoDB to AWS \\u2014 DocumentDB is AWS\\\\'s MongoDB-compatible managed database. Triggers: JSON document store, document database, MongoDB on AWS, Nested fields, Lambda cannot connect, TLS handshake, VPC port 27017, IAM auth, Secrets Manager, encryption at rest, $graphLookup, flexible schema, COLLSCAN, compound index, DMS migration, CDC cutover, $vectorSearch, RAG, Global Clusters, DR replication, cost sizing, audit, health check, production-readiness.\\\\\\\\n\\\\\\\\nServices: documentdb, dms, cloudwatch, secrets-manager\\\\\\\\nTasks: setup, migrate, optimize, upgrade, troubleshoot, review, create, modify\\\\\\\\nPersona: developer, devops, architect, dba\\\\\\\\nWorkload: document-database, mongodb-compatible, vector-search, rag\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"amazon-documentdb\\\"}]}}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:19.075000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "5ebb3d7a-03c9-40ed-9991-aadbac3bbe2a", + "executionId": "a28bae56-e59e-43e9-8426-510b6ccc8a61", + "recordId": "6dff0fc5-3743-444d-816e-780d48981bc6", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.2, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:19.158000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "5ebb3d7a-03c9-40ed-9991-aadbac3bbe2a", + "executionId": "a28bae56-e59e-43e9-8426-510b6ccc8a61", + "recordId": "e9444b9f-df79-4ce4-bc74-f80ecc585892", + "content": "{\"id\": \"e9444b9f-df79-4ce4-bc74-f80ecc585892\", \"role\": \"assistant\", \"content\": [{\"text\": \"Confirmed \\u2014 AWS already provides a managed policy for this, so you don't need to write your own in most cases.\\n\\n## Option 1: AWS managed policy (simplest)\\n\\nAttach **`AmazonDynamoDBReadOnlyAccess`** to your IAM user or role. It grants read-only access to DynamoDB resources across the account via the console and API.\\n\\n```bash\\naws iam attach-role-policy \\\\\\n --role-name YourRoleName \\\\\\n --policy-arn [redacted ARN]\\n```\\n\\n## Option 2: Custom least-privilege policy (scoped to specific tables)\\n\\nIf you want to limit access to one or more specific tables rather than the whole account:\\n\\n```json\\n{\\n \\\"Version\\\": \\\"2012-10-17\\\",\\n \\\"Statement\\\": [\\n {\\n \\\"Sid\\\": \\\"DynamoDBReadOnly\\\",\\n \\\"Effect\\\": \\\"Allow\\\",\\n \\\"Action\\\": [\\n \\\"dynamodb:GetItem\\\",\\n \\\"dynamodb:BatchGetItem\\\",\\n \\\"dynamodb:Query\\\",\\n \\\"dynamodb:Scan\\\",\\n \\\"dynamodb:ConditionCheckItem\\\",\\n \\\"dynamodb:DescribeTable\\\"\\n ],\\n \\\"Resource\\\": [\\n \\\"arn:aws:dynamodb:REGION:ACCOUNT_ID:table/YourTableName\\\",\\n \\\"arn:aws:dynamodb:REGION:ACCOUNT_ID:table/YourTableName/index/*\\\"\\n ]\\n }\\n ]\\n}\\n```\\n\\nReplace `REGION`, `ACCOUNT_ID`, and `YourTableName`. Including `/index/*` covers reads against global/local secondary indexes.\\n\\nWant me to attach the managed policy to a specific role or user in your account, or draft the scoped version for a particular table?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:19.228000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "5ebb3d7a-03c9-40ed-9991-aadbac3bbe2a", + "executionId": "a28bae56-e59e-43e9-8426-510b6ccc8a61", + "recordId": "1375ad48-9c87-4fc0-bd85-71e95bce1896", + "content": "{\"id\": \"7c1936a4-3828-4d43-b0f5-fd4ebb72109a\", \"role\": \"assistant\", \"content\": [{\"type\": \"text\", \"text\": \"Confirmed \\u2014 AWS already provides a managed policy for this, so you don't need to write your own in most cases.\\n\\n## Option 1: AWS managed policy (simplest)\\n\\nAttach **`AmazonDynamoDBReadOnlyAccess`** to your IAM user or role. It grants read-only access to DynamoDB resources across the account via the console and API.\\n\\n```bash\\naws iam attach-role-policy \\\\\\n --role-name YourRoleName \\\\\\n --policy-arn [redacted ARN]\\n```\\n\\n## Option 2: Custom least-privilege policy (scoped to specific tables)\\n\\nIf you want to limit access to one or more specific tables rather than the whole account:\\n\\n```json\\n{\\n \\\"Version\\\": \\\"2012-10-17\\\",\\n \\\"Statement\\\": [\\n {\\n \\\"Sid\\\": \\\"DynamoDBReadOnly\\\",\\n \\\"Effect\\\": \\\"Allow\\\",\\n \\\"Action\\\": [\\n \\\"dynamodb:GetItem\\\",\\n \\\"dynamodb:BatchGetItem\\\",\\n \\\"dynamodb:Query\\\",\\n \\\"dynamodb:Scan\\\",\\n \\\"dynamodb:ConditionCheckItem\\\",\\n \\\"dynamodb:DescribeTable\\\"\\n ],\\n \\\"Resource\\\": [\\n \\\"arn:aws:dynamodb:REGION:ACCOUNT_ID:table/YourTableName\\\",\\n \\\"arn:aws:dynamodb:REGION:ACCOUNT_ID:table/YourTableName/index/*\\\"\\n ]\\n }\\n ]\\n}\\n```\\n\\nReplace `REGION`, `ACCOUNT_ID`, and `YourTableName`. Including `/index/*` covers reads against global/local secondary indexes.\\n\\nWant me to attach the managed policy to a specific role or user in your account, or draft the scoped version for a particular table?\"}]}", + "createdAt": "2026-10-27T17:05:19.305000-05:00", + "recordType": "final_response" + } +] \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/negative-python-linked-list/with_skill/functional-tests-results.json b/skills/aws-routing/evals/functional/v1/iteration-1/negative-python-linked-list/with_skill/functional-tests-results.json new file mode 100644 index 00000000..1ead1260 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/negative-python-linked-list/with_skill/functional-tests-results.json @@ -0,0 +1,19 @@ +{ + "version": "v1", + "iteration": 1, + "eval_id": "negative-python-linked-list", + "run_type": "with_skill", + "success": true, + "error": null, + "understanding_agent_space_skill": null, + "tests": { + "trigger": { + "result": "passed", + "message": "Skill 'aws-routing' was correctly NOT triggered", + "skill_loads_found": 0 + }, + "expected_output": null, + "assertions": null + }, + "metrics": null +} \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/negative-python-linked-list/with_skill/outputs/journal_records.json b/skills/aws-routing/evals/functional/v1/iteration-1/negative-python-linked-list/with_skill/outputs/journal_records.json new file mode 100644 index 00000000..5a59cb8d --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/negative-python-linked-list/with_skill/outputs/journal_records.json @@ -0,0 +1,42 @@ +[ + { + "agentSpaceId": "acad88da-48bb-4398-a4d1-b0b0b31a1695", + "executionId": "86f4986f-d2f5-4f29-837b-78f42f350615", + "recordId": "fdc8282e-e07d-4ea5-8d62-0314571ce088", + "content": "{\"id\": \"fdc8282e-e07d-4ea5-8d62-0314571ce088\", \"role\": \"user\", \"content\": [{\"text\": \"Write a Python function to reverse a linked list.\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:03.850000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "acad88da-48bb-4398-a4d1-b0b0b31a1695", + "executionId": "86f4986f-d2f5-4f29-837b-78f42f350615", + "recordId": "02c3414f-f1f3-4480-8346-d2a77707d610", + "content": "{\"id\": \"a05b2be8-47f2-4777-b717-abe04e701a99\", \"role\": \"assistant\", \"content\": [{\"type\": \"chat_title\", \"title\": \"Python function to reverse a linked list\"}]}", + "createdAt": "2026-10-27T17:05:04.815000-05:00", + "recordType": "chat_title" + }, + { + "agentSpaceId": "acad88da-48bb-4398-a4d1-b0b0b31a1695", + "executionId": "86f4986f-d2f5-4f29-837b-78f42f350615", + "recordId": "e46c6329-1a90-47c9-a9f7-0b189d1cb962", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 3.8, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:09.489000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "acad88da-48bb-4398-a4d1-b0b0b31a1695", + "executionId": "86f4986f-d2f5-4f29-837b-78f42f350615", + "recordId": "940d9774-9625-4e66-8f9d-3b2e38346281", + "content": "{\"id\": \"940d9774-9625-4e66-8f9d-3b2e38346281\", \"role\": \"assistant\", \"content\": [{\"text\": \"Here's a Python function to reverse a singly linked list:\\n\\n```python\\nclass ListNode:\\n def __init__(self, val=0, next=None):\\n self.val = val\\n self.next = next\\n\\n\\ndef reverse_linked_list(head):\\n prev = None\\n current = head\\n\\n while current:\\n next_node = current.next # save the next node\\n current.next = prev # reverse the pointer\\n prev = current # move prev forward\\n current = next_node # move current forward\\n\\n return prev # new head of the reversed list\\n```\\n\\n**How it works:**\\n- `prev` starts as `None` (it'll become the new tail's `next`)\\n- At each step, you save `current.next` before overwriting it, flip `current.next` to point backward to `prev`, then shift both `prev` and `current` one step forward\\n- When `current` becomes `None`, `prev` is sitting on the last original node \\u2014 which is now the new head\\n\\n**Example:**\\n```python\\n# Build 1 -> 2 -> 3 -> None\\nhead = ListNode(1, ListNode(2, ListNode(3)))\\n\\nnew_head = reverse_linked_list(head)\\n\\n# Print result: 3 -> 2 -> 1 -> None\\nnode = new_head\\nwhile node:\\n print(node.val, end=\\\" -> \\\")\\n node = node.next\\nprint(\\\"None\\\")\\n```\\n\\nThis runs in O(n) time and O(1) extra space. If you'd like, I can also show a recursive version.\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:09.587000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "acad88da-48bb-4398-a4d1-b0b0b31a1695", + "executionId": "86f4986f-d2f5-4f29-837b-78f42f350615", + "recordId": "441da8dc-d93d-415a-a671-07969c7e546f", + "content": "{\"id\": \"8585cda5-8a06-4a63-9229-d790b383d0e5\", \"role\": \"assistant\", \"content\": [{\"type\": \"text\", \"text\": \"Here's a Python function to reverse a singly linked list:\\n\\n```python\\nclass ListNode:\\n def __init__(self, val=0, next=None):\\n self.val = val\\n self.next = next\\n\\n\\ndef reverse_linked_list(head):\\n prev = None\\n current = head\\n\\n while current:\\n next_node = current.next # save the next node\\n current.next = prev # reverse the pointer\\n prev = current # move prev forward\\n current = next_node # move current forward\\n\\n return prev # new head of the reversed list\\n```\\n\\n**How it works:**\\n- `prev` starts as `None` (it'll become the new tail's `next`)\\n- At each step, you save `current.next` before overwriting it, flip `current.next` to point backward to `prev`, then shift both `prev` and `current` one step forward\\n- When `current` becomes `None`, `prev` is sitting on the last original node \\u2014 which is now the new head\\n\\n**Example:**\\n```python\\n# Build 1 -> 2 -> 3 -> None\\nhead = ListNode(1, ListNode(2, ListNode(3)))\\n\\nnew_head = reverse_linked_list(head)\\n\\n# Print result: 3 -> 2 -> 1 -> None\\nnode = new_head\\nwhile node:\\n print(node.val, end=\\\" -> \\\")\\n node = node.next\\nprint(\\\"None\\\")\\n```\\n\\nThis runs in O(n) time and O(1) extra space. If you'd like, I can also show a recursive version.\"}]}", + "createdAt": "2026-10-27T17:05:09.721000-05:00", + "recordType": "final_response" + } +] \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/tgw-ecmp-behavior/with_skill/functional-tests-results.json b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-ecmp-behavior/with_skill/functional-tests-results.json new file mode 100644 index 00000000..d5c8e504 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-ecmp-behavior/with_skill/functional-tests-results.json @@ -0,0 +1,65 @@ +{ + "version": "v1", + "iteration": 1, + "eval_id": "tgw-ecmp-behavior", + "run_type": "with_skill", + "success": true, + "error": null, + "understanding_agent_space_skill": null, + "tests": { + "trigger": { + "result": "passed", + "message": "Skill 'aws-routing' was triggered as expected", + "skill_loads_found": 1 + }, + "expected_output": { + "result": "passed", + "reasoning": "The agent's response addresses the core requirements of the expected output: it explains that ECMP requires the same attachment type (explicitly stating TGW never ECMPs across different attachment types like DX vs VPN) and that AS-paths must match. It also correctly states that when conditions aren't met, TGW installs the oldest route as a single path, matching the expected output's 'single (oldest) route' criterion. The response goes into more granular detail (e.g., ASN values not just path length matching for DXGW attachments specifically), which is a more precise/nuanced version of 'equal AS-path length / BGP attributes'. While the response doesn't explicitly mention 'LP communities not being honored across multiple DXGW attachments,' it does capture the essential substance: same attachment type requirement, AS-path/attribute matching requirement for ECMP, and the fallback to oldest single route otherwise. The core technical conclusions align well with the expected output, even though the specific phrasing about LP communities is absent - the response conveys equivalent technical accuracy about why ECMP fails and what happens instead.", + "confidence": "medium" + }, + "assertions": { + "assertion_results": [ + { + "text": "states ECMP requires same attachment type and equal AS-path/BGP attributes", + "evaluator": "llm", + "passed": true, + "evidence": "'The two Direct Connect paths come from two separate DX Gateway attachments on the TGW... and Both DXGWs advertise the prefix with truly identical AS-paths \u2014 same length and same ASNs' and 'TGW never ECMPs across different attachment types'", + "reasoning": "The output explicitly states ECMP requires the same attachment type (not DX vs VPN) and equal AS-path/BGP attributes (same length and same ASNs), satisfying the assertion's substance.", + "confidence": "high" + }, + { + "text": "notes TGW picks a single route when ECMP conditions are not met", + "evaluator": "llm", + "passed": true, + "evidence": "'If the AS-paths aren't identical (including the actual ASN values, not just path length), ECMP is skipped and TGW installs the oldest route \u2014 a single path, not a blend.'", + "reasoning": "The agent clearly states that when ECMP conditions are not met, TGW picks just one route (the oldest route), directly satisfying the assertion.", + "confidence": "high" + }, + { + "text": "does not contain 'error'", + "evaluator": "llm", + "passed": true, + "evidence": "No instance of the word 'error' appears anywhere in the output.", + "reasoning": "A review of the full output text shows no occurrence of the word 'error'.", + "confidence": "high" + } + ], + "summary": { + "passed": 3, + "failed": 0, + "errored": 0, + "low_confidence": 0, + "total": 3, + "pass_rate": 1.0 + } + } + }, + "metrics": { + "runtime": "13s", + "cost": "$0.12", + "context_window": { + "utilization": "4.7%", + "compaction_count": 0 + } + } +} \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/tgw-ecmp-behavior/with_skill/outputs/journal_records.json b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-ecmp-behavior/with_skill/outputs/journal_records.json new file mode 100644 index 00000000..723fa949 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-ecmp-behavior/with_skill/outputs/journal_records.json @@ -0,0 +1,138 @@ +[ + { + "agentSpaceId": "ccbde5fc-16ba-414d-8d9e-ef9ddf6b26c8", + "executionId": "82ba7a6c-5308-4ed3-ac48-ee68be7bc349", + "recordId": "2239dba0-276d-48eb-a623-57f6d1a159b5", + "content": "{\"id\": \"2239dba0-276d-48eb-a623-57f6d1a159b5\", \"role\": \"user\", \"content\": [{\"text\": \"When does a Transit Gateway load-balance across two Direct Connect paths, and when does it pick just one?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:06.514000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "ccbde5fc-16ba-414d-8d9e-ef9ddf6b26c8", + "executionId": "82ba7a6c-5308-4ed3-ac48-ee68be7bc349", + "recordId": "adbf0846-a45c-4722-abf1-a4949ff3461b", + "content": "{\"id\": \"89e84bd9-8bde-447a-a00c-afae953553af\", \"role\": \"assistant\", \"content\": [{\"type\": \"chat_title\", \"title\": \"Transit Gateway load balancing across Direct Connect paths\"}]}", + "createdAt": "2026-10-27T17:05:06.941000-05:00", + "recordType": "chat_title" + }, + { + "agentSpaceId": "ccbde5fc-16ba-414d-8d9e-ef9ddf6b26c8", + "executionId": "82ba7a6c-5308-4ed3-ac48-ee68be7bc349", + "recordId": "7e853d06-561e-4354-9bdf-fe78cdeaf903", + "content": "{\"id\": \"991b4823-0180-4cb4-a953-580c9eda4832\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This question is about Transit Gateway and Direct Connect routing with ECMP, so I should check the aws-routing skill for guidance.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:10.791000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "ccbde5fc-16ba-414d-8d9e-ef9ddf6b26c8", + "executionId": "82ba7a6c-5308-4ed3-ac48-ee68be7bc349", + "recordId": "8eb02df2-1f45-4a72-9db7-99caee3b1904", + "content": "{\"id\": \"8eb02df2-1f45-4a72-9db7-99caee3b1904\", \"role\": \"assistant\", \"content\": [{\"text\": \"This is exactly the kind of routing behavior question the AWS routing skill covers \\u2014 let me pull that up.\", \"type\": \"text\"}, {\"id\": \"tooluse_P5zBYTWCPGAzJlL08zlNww\", \"tool_name\": \"skill_read\", \"input\": {\"skill_id\": \"aws-routing\"}, \"type\": \"tool_use\", \"display_intent\": \"Read the aws-routing skill for Transit Gateway ECMP guidance over Direct Connect.\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:10.816000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "ccbde5fc-16ba-414d-8d9e-ef9ddf6b26c8", + "executionId": "82ba7a6c-5308-4ed3-ac48-ee68be7bc349", + "recordId": "9201b546-70e6-4cfb-abe9-0545e7cd37ae", + "content": "{\"id\": \"52a6e89f-3bd7-4efc-948f-fcd2ce14ea15\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"404\"}]}", + "createdAt": "2026-10-27T17:05:10.862000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "ccbde5fc-16ba-414d-8d9e-ef9ddf6b26c8", + "executionId": "82ba7a6c-5308-4ed3-ac48-ee68be7bc349", + "recordId": "2ff849d7-c149-495d-a044-5440ae8f5875", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 3.8, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:10.890000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "ccbde5fc-16ba-414d-8d9e-ef9ddf6b26c8", + "executionId": "82ba7a6c-5308-4ed3-ac48-ee68be7bc349", + "recordId": "e9abae43-699f-43e2-b28e-edfaa93f6858", + "content": "{\"id\": \"028576bc-38f6-4245-a7aa-3b097e2c8934\", \"role\": \"assistant\", \"block_type\": \"load_skill\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_P5zBYTWCPGAzJlL08zlNww\", \"status\": \"success\", \"content\": [{\"text\": \"Done\"}]}], \"label\": \"Read the aws-routing skill for Transit Gateway ECMP guidance over Direct Connect.\"}", + "createdAt": "2026-10-27T17:05:10.982000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "ccbde5fc-16ba-414d-8d9e-ef9ddf6b26c8", + "executionId": "82ba7a6c-5308-4ed3-ac48-ee68be7bc349", + "recordId": "6baddc69-4fd2-47e9-abc5-3f33b79a870a", + "content": "{\"id\": \"da0670b7-dda2-40d6-be9d-71b9ac7d8c7a\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This is about Transit Gateway route table design, so I should pull up the tgw-routing-patterns.md reference to inform the decision.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:14.905000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "ccbde5fc-16ba-414d-8d9e-ef9ddf6b26c8", + "executionId": "82ba7a6c-5308-4ed3-ac48-ee68be7bc349", + "recordId": "1041bcff-83d3-474f-82da-12e4daa62faa", + "content": "{\"id\": \"1041bcff-83d3-474f-82da-12e4daa62faa\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_P5zBYTWCPGAzJlL08zlNww\", \"content\": \"[{'text': '{\\\"content\\\": \\\"---\\\\\\\\r\\\\\\\\nname: aws-routing\\\\\\\\r\\\\\\\\ndescription: \\\\'Use this skill when the user wants to know which path AWS will pick for their traffic, why traffic took an unexpected route, or how to make AWS prefer a path. Covers Cloud WAN CNE route evaluation, Direct Connect Gateway local vs remote region preference, Transit Gateway route tables, ECMP and peering, VPC longest-prefix-match, BGP traffic engineering (LP communities 7224:7100/7200/7300, AS-path prepending, MED), DX + VPN redundancy and failover, and asymmetric routing. Use it even when the user only describes a symptom, such as \\\\\\\\\\\"traffic is leaving the region\\\\\\\\\\\", \\\\\\\\\\\"not using my local Direct Connect\\\\\\\\\\\", \\\\\\\\\\\"routing to the wrong VIF/location\\\\\\\\\\\", or \\\\\\\\\\\"path is asymmetric\\\\\\\\\\\". Read-only: analysis, recommendations, and describe/get/list validation commands grounded in public AWS docs.\\\\'\\\\\\\\r\\\\\\\\nmetadata:\\\\\\\\r\\\\\\\\n version: \\\\\\\\\\\"1.2.5\\\\\\\\\\\"\\\\\\\\r\\\\\\\\n author: midakin\\\\\\\\r\\\\\\\\n aws-devops-agent-skills.agent-types: \\\\\\\\\\\"CHAT, INCIDENT_RCA\\\\\\\\\\\"\\\\\\\\r\\\\\\\\n---\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Overview\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nThis skill provides routing-first analysis for AWS networking. It reasons about the full traffic\\\\\\\\r\\\\\\\\npath, applies the documented route-evaluation logic for each service, and grounds recommendations\\\\\\\\r\\\\\\\\nin public AWS documentation. It is **read-only**: it produces analysis, recommendations, and\\\\\\\\r\\\\\\\\nread-only validation commands (`describe-*`, `get-*`, `list-*`) - never mutating actions.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Verification & Anti-Hallucination Directives (Mandatory)\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- Do not generate any fact, citation, or source that is not fully verifiable. If you lack a verified\\\\\\\\r\\\\\\\\n source, state \\\\\\\\\\\"I cannot verify this\\\\\\\\\\\" rather than inventing details.\\\\\\\\r\\\\\\\\n- State assumptions explicitly before answering.\\\\\\\\r\\\\\\\\n- Cross-check each claim against public AWS documentation or the `references/` files. Omit claims you\\\\\\\\r\\\\\\\\n cannot confirm.\\\\\\\\r\\\\\\\\n- Only cite AWS documentation URLs you have actually retrieved or confirmed. Do not fabricate links.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## When to use this skill\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nActivate when the user asks about:\\\\\\\\r\\\\\\\\n- Cloud WAN route evaluation / Core Network Edge (CNE) path selection\\\\\\\\r\\\\\\\\n- Direct Connect Gateway path selection, local-region preference, or LP communities\\\\\\\\r\\\\\\\\n- Transit Gateway route evaluation, ECMP, or peering routing\\\\\\\\r\\\\\\\\n- VPC route tables and longest-prefix-match behavior\\\\\\\\r\\\\\\\\n- BGP traffic engineering (communities, AS-path prepending, MED)\\\\\\\\r\\\\\\\\n- DX + VPN redundancy, active/active vs active/passive, or failover design\\\\\\\\r\\\\\\\\n- Why traffic is taking an unexpected path\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## What this skill does NOT do\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nThis skill is **read-only and advisory**. It will not:\\\\\\\\r\\\\\\\\n- **Design or apply infrastructure changes.** It does not create, modify, or delete AWS resources,\\\\\\\\r\\\\\\\\n and it does not run mutating CLI/API calls (`create-*`, `modify-*`, `associate-*`, `delete-*`,\\\\\\\\r\\\\\\\\n `put-*`, `update-*`) or write Infrastructure-as-Code to be deployed. Any change it describes is a\\\\\\\\r\\\\\\\\n *recommendation* for you to review and implement yourself.\\\\\\\\r\\\\\\\\n- **Configure BGP or push routing policy.** It explains which communities, AS-path, or MED settings\\\\\\\\r\\\\\\\\n to use, but it does not apply them to VIFs, route tables, or Cloud WAN policy documents.\\\\\\\\r\\\\\\\\n- **Execute anything against a live account beyond read-only validation.** Commands it provides are\\\\\\\\r\\\\\\\\n limited to `describe-*`, `get-*`, and `list-*` for confirming state.\\\\\\\\r\\\\\\\\n- **Guarantee production outcomes.** It recommends non-production testing and blue/green rollout;\\\\\\\\r\\\\\\\\n it does not perform the cutover or validate the result for you.\\\\\\\\r\\\\\\\\n- **Provide compliance, security, or contractual sign-off**, or make claims it cannot ground in\\\\\\\\r\\\\\\\\n public AWS documentation or the `references/` files.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nFor designing and applying changes, hand the recommendation to a change-capable workflow (with\\\\\\\\r\\\\\\\\nappropriate review, approvals, and testing).\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## How to analyze a routing problem\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nWork through this checklist in order; each step depends on the one before it.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- [ ] **1. Establish the path.** Identify source, destination, and every routing construct in\\\\\\\\r\\\\\\\\n between (VPC route table \\\\\\\\u2192 TGW/Cloud WAN CNE \\\\\\\\u2192 DXGW \\\\\\\\u2192 VIF \\\\\\\\u2192 on-prem, or the relevant subset).\\\\\\\\r\\\\\\\\n- [ ] **2. Name the route table.** Be explicit about which table makes each decision (VPC RT, TGW RT,\\\\\\\\r\\\\\\\\n CNE RT, DXGW internal selection).\\\\\\\\r\\\\\\\\n- [ ] **3. Apply the evaluation order for that construct** (see below).\\\\\\\\r\\\\\\\\n- [ ] **4. Identify the deciding attribute** (longest prefix, local preference, AS-path, MED, source\\\\\\\\r\\\\\\\\n type).\\\\\\\\r\\\\\\\\n- [ ] **5. Flag non-determinism** (e.g., ECMP or \\\\\\\\\\\"deterministically random\\\\\\\\\\\" tiebreakers) and\\\\\\\\r\\\\\\\\n recommend a deterministic alternative.\\\\\\\\r\\\\\\\\n- [ ] **6. Cite the AWS documentation** that supports the behavior.\\\\\\\\r\\\\\\\\n- [ ] **7. Provide read-only validation commands** the user can run to confirm.\\\\\\\\r\\\\\\\\n- [ ] **8. Self-check before answering.** Re-read the answer against steps 1-7: every route table is\\\\\\\\r\\\\\\\\n named, the evaluation order matches the construct, each cited URL was actually retrieved, and every\\\\\\\\r\\\\\\\\n command is read-only. Fix any gap before presenting.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Key evaluation rules\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Cloud WAN (per CNE)\\\\\\\\r\\\\\\\\nLongest prefix match \\\\\\\\u2192 static \\\\\\\\u2192 VPC-propagated (same region) \\\\\\\\u2192 unequal AS-path/MED (shortest wins) \\\\\\\\u2192\\\\\\\\r\\\\\\\\nequal AS-path & MED source preference: DXGW-propagated \\\\\\\\u2192 Cloud WAN Connect \\\\\\\\u2192 S2S VPN \\\\\\\\u2192 other\\\\\\\\r\\\\\\\\n(TGW peering, remote CNEs; identical from 2+ sources = deterministically random).\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Direct Connect (private/transit VIF, outbound AWS\\\\\\\\u2192on-prem)\\\\\\\\r\\\\\\\\nLongest prefix match \\\\\\\\u2192 local preference \\\\\\\\u2192 AS_PATH length \\\\\\\\u2192 MED \\\\\\\\u2192 ECMP (equal AS_PATH and BGP\\\\\\\\r\\\\\\\\nattributes). LP communities `7224:7300/7200/7100` (High/Medium/Low) set local preference and are\\\\\\\\r\\\\\\\\nevaluated before AS-path. DXGW prefers the local associated-region DX by default.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Direction & tooling guidance\\\\\\\\r\\\\\\\\n- **AWS \\\\\\\\u2192 on-prem (egress):** control with DX LP communities.\\\\\\\\r\\\\\\\\n- **On-prem \\\\\\\\u2192 AWS (ingress):** control with the customer router\\\\'s local-pref/weight/MED.\\\\\\\\r\\\\\\\\n- **AS-path prepending:** within a region only - unreliable across regions because DXGW local-region\\\\\\\\r\\\\\\\\n LP overrides it. **LP communities:** work within-region and cross-region.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Reference material\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nDetailed mechanics, patterns, and caveats are in `references/`. Load only the file that matches the\\\\\\\\r\\\\\\\\nconstruct deciding the path:\\\\\\\\r\\\\\\\\n- [Cloud WAN + DX routing patterns](references/cloudwan-dx-routing-patterns.md) - load when the path\\\\\\\\r\\\\\\\\n crosses a Cloud WAN CNE: CNE route evaluation, DXGW path selection into Cloud WAN, community\\\\\\\\r\\\\\\\\n traffic-engineering patterns, regional inspection, multi-region egress.\\\\\\\\r\\\\\\\\n- [Direct Connect routing patterns](references/dx-routing-patterns.md) - load when the question is\\\\\\\\r\\\\\\\\n about a DX VIF or DXGW without Cloud WAN: BGP mechanics, BFD, MED, ASN ranges, VIF considerations,\\\\\\\\r\\\\\\\\n CloudHub, when VIFs can talk, active/active vs active/passive.\\\\\\\\r\\\\\\\\n- [Transit Gateway routing patterns](references/tgw-routing-patterns.md) - load when a TGW route\\\\\\\\r\\\\\\\\n table makes the decision: TGW route evaluation, ECMP limits, DXGW-to-TGW behavior, DX/VPN caveats,\\\\\\\\r\\\\\\\\n failover, peering.\\\\\\\\r\\\\\\\\n- [VPN + DX redundancy patterns](references/vpn-dx-redundancy-patterns.md) - load when VPN backs up\\\\\\\\r\\\\\\\\n or runs alongside DX: route preference between them, ECMP limits, asymmetric return paths.\\\\\\\\r\\\\\\\\n- [Networking strategic questions](references/networking-strategic-questions.md) - load when key\\\\\\\\r\\\\\\\\n facts are missing (regions, attachments, ASNs, communities) and you need discovery questions to ask\\\\\\\\r\\\\\\\\n before analyzing.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Output expectations\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- Explain the route-evaluation logic step by step.\\\\\\\\r\\\\\\\\n- Name the specific route table involved.\\\\\\\\r\\\\\\\\n- Give the end-to-end traffic path.\\\\\\\\r\\\\\\\\n- Compare options with tradeoffs (determinism, complexity, failover) when multiple solutions exist.\\\\\\\\r\\\\\\\\n- Include read-only AWS CLI validation commands.\\\\\\\\r\\\\\\\\n- Recommend testing in a non-production environment and blue/green for migrations.\\\\\\\\r\\\\\\\\n\\\", \\\"resource_manifest\\\": [\\\"SKILL.md\\\", \\\"references/cloudwan-dx-routing-patterns.md\\\", \\\"references/dx-routing-patterns.md\\\", \\\"references/networking-strategic-questions.md\\\", \\\"references/tgw-routing-patterns.md\\\", \\\"references/vpn-dx-redundancy-patterns.md\\\"]}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:14.905000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "ccbde5fc-16ba-414d-8d9e-ef9ddf6b26c8", + "executionId": "82ba7a6c-5308-4ed3-ac48-ee68be7bc349", + "recordId": "5b1e7d4b-8a6b-4884-a926-b1086ac91573", + "content": "{\"id\": \"832ae6c9-2347-4eda-a793-f0e4208f8606\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"395\"}]}", + "createdAt": "2026-10-27T17:05:14.996000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "ccbde5fc-16ba-414d-8d9e-ef9ddf6b26c8", + "executionId": "82ba7a6c-5308-4ed3-ac48-ee68be7bc349", + "recordId": "1fb71228-fbc1-47d3-bb98-0dd81f3e9c9a", + "content": "{\"id\": \"1fb71228-fbc1-47d3-bb98-0dd81f3e9c9a\", \"role\": \"assistant\", \"content\": [{\"id\": \"tooluse_Eg8EnoY7ZuvtrMB2LT6mrr\", \"tool_name\": \"read_skill_resource\", \"input\": {\"skill_id\": \"aws-routing\", \"path\": \"references/tgw-routing-patterns.md\"}, \"type\": \"tool_use\", \"display_intent\": \"Read Transit Gateway routing patterns reference for ECMP over Direct Connect details.\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:15.013000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "ccbde5fc-16ba-414d-8d9e-ef9ddf6b26c8", + "executionId": "82ba7a6c-5308-4ed3-ac48-ee68be7bc349", + "recordId": "016709f1-74d1-4c97-b285-551cdba9b8bd", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.2, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:15.119000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "ccbde5fc-16ba-414d-8d9e-ef9ddf6b26c8", + "executionId": "82ba7a6c-5308-4ed3-ac48-ee68be7bc349", + "recordId": "5bde3990-1650-4c92-b2a2-35129ca20d3c", + "content": "{\"id\": \"715dc819-927f-4ffc-9120-6b21f47d29d4\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_Eg8EnoY7ZuvtrMB2LT6mrr\", \"status\": \"success\", \"content\": [{\"text\": \"Done\"}]}], \"label\": \"Read Transit Gateway routing patterns reference for ECMP over Direct Connect details.\"}", + "createdAt": "2026-10-27T17:05:15.382000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "ccbde5fc-16ba-414d-8d9e-ef9ddf6b26c8", + "executionId": "82ba7a6c-5308-4ed3-ac48-ee68be7bc349", + "recordId": "84cad2a5-e448-42a2-8574-78eb78c8be95", + "content": "{\"id\": \"84cad2a5-e448-42a2-8574-78eb78c8be95\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_Eg8EnoY7ZuvtrMB2LT6mrr\", \"content\": \"[{'text': '\\\\r\\\\n\\\\r\\\\n# Transit Gateway Routing Patterns & Considerations\\\\r\\\\n\\\\r\\\\nThis document captures key routing mechanics, ECMP behavior, failover characteristics, and operational considerations for Transit Gateway architectures. Use this knowledge when advising on TGW route evaluation, multi-attachment designs, inter-region peering, and DX/VPN integration.\\\\r\\\\n\\\\r\\\\n## Reference Documentation\\\\r\\\\n\\\\r\\\\n- [Transit Gateway Route Evaluation Order](https://docs.aws.amazon.com/vpc/latest/tgw/how-transit-gateways-work.html#tgw-route-table-evaluation)\\\\r\\\\n- [Transit Gateway VPN ECMP](https://docs.aws.amazon.com/vpc/latest/tgw/transit-gateway-share.html)\\\\r\\\\n- [Direct Connect + Transit Gateway](https://docs.aws.amazon.com/directconnect/latest/UserGuide/direct-connect-transit-gateways.html)\\\\r\\\\n\\\\r\\\\n## Route Evaluation & Priority\\\\r\\\\n\\\\r\\\\n### Static VPN Has Higher Priority Than DX\\\\r\\\\n\\\\r\\\\n- Static VPN routes have a **higher routing priority** at the TGW than Direct Connect routes\\\\r\\\\n- Always refer to the TGW Route evaluation order when determining which attachment path will win for a given prefix\\\\r\\\\n- This can cause unexpected behavior if a customer has both a static VPN and DX advertising the same prefix \\u2014 the static VPN will be preferred\\\\r\\\\n\\\\r\\\\n## ECMP Behavior\\\\r\\\\n\\\\r\\\\n### DXGW Advertises Only ONE Best Path Per Prefix to TGW\\\\r\\\\n\\\\r\\\\nA **single DX Gateway advertises only one best path per prefix** to the TGW, regardless of how many VIFs are attached to that DXGW. The DXGW acts as a BGP aggregator between on-prem and the TGW: it runs its own internal best-path selection across its VIFs and propagates only the single winner per prefix to the TGW attachment. This is the same \\\"DXGW propagates one path per prefix\\\" behavior seen on the Cloud WAN side (DXGW \\u2192 CNE); the downstream being a TGW does not change it.\\\\r\\\\n\\\\r\\\\nConsequences:\\\\r\\\\n- **No active/active ECMP across multiple VIFs on the *same* DXGW** \\u2014 the TGW sees one path per prefix from that DXGW attachment, so redundant VIFs on one DXGW behave **active/passive** (DXGW fails over VIF-to-VIF internally; TGW only ever sees the current winner).\\\\r\\\\n- **To get multiple paths per prefix at the TGW, use multiple DXGWs** \\u2014 each DXGW is a separate TGW attachment and contributes its own one-best-path-per-prefix. TGW then has one candidate per DXGW and can ECMP across them (subject to the same-ASN requirement below).\\\\r\\\\n\\\\r\\\\n### Supported ECMP Scenarios\\\\r\\\\n\\\\r\\\\n- **VPN attachments**: TGW supports ECMP across VPN attachments. A TGW-level toggle (\\\"VPN ECMP support\\\") must be enabled.\\\\r\\\\n- **Multiple different DX-GWs**: TGW will ECMP across the per-DXGW best paths received from different DX Gateways for a matching prefix \\u2014 **but only when the AS-paths are identical, which requires the DXGWs to advertise with the SAME ASN** (see \\\"Same-ASN requirement\\\" below). Note this is ECMP across *separate DXGW attachments*, not across VIFs within one DXGW (one DXGW only ever offers a single path per prefix \\u2014 see above).\\\\r\\\\n- **Multiple TGW Connect attachments**: TGW will ECMP across matching prefixes\\\\r\\\\n\\\\r\\\\n### NOT Supported\\\\r\\\\n\\\\r\\\\n- **ECMP across different attachment types** \\u2014 TGW does NOT ECMP between, for example, a DX-GW attachment and a VPN attachment for the same prefix. Instead, the Route evaluation order determines the winner.\\\\r\\\\n- **Multipath AS-path relaxed** \\u2014 ECMP for paths with different AS-paths is NOT supported. If matching prefixes are received with different AS-paths, only the **oldest route is installed** (not load-balanced).\\\\r\\\\n- If prefixes match but ECMP conditions aren\\\\'t met, the **oldest route** is installed.\\\\r\\\\n\\\\r\\\\n### Same-ASN Requirement for TGW ECMP Across DXGWs\\\\r\\\\n\\\\r\\\\n- Because TGW does **not** support AS-path relax, ECMP across multiple DXGW attachments requires the AS-paths to be **truly identical \\u2014 same length AND same ASNs**. Equal *length* alone is not sufficient if the ASNs differ.\\\\r\\\\n- In practice this means the **DXGWs must advertise the prefix with the same ASN** for TGW to ECMP across them. Different ASNs \\u2192 no ECMP \\u2192 TGW installs the **oldest route** (single path).\\\\r\\\\n- **This differs from DX-side ECMP.** Per AWS DX docs, ECMP *across VIFs for a Region* works \\\"when prefixes have the same AS_PATH length and BGP attributes\\\" and the ASNs in the AS_PATH **do not need to match** \\u2014 that is a Region/DX-side behavior. The **same-ASN requirement here is specific to TGW ECMP across separate DXGW attachments** \\u2014 do not conflate the two selection points.\\\\r\\\\n- **[Field knowledge]** The DXGW-one-path-per-prefix propagation to TGW, and the same-ASN requirement for TGW ECMP across DXGWs, are field-proven/operational behaviors. They are not clearly stated as such on the public AWS TGW route-evaluation page \\u2014 validate in-account with `search-transit-gateway-routes` (per-prefix candidates and which attachment) and `list-virtual-interface-routes` (what each DXGW\\\\'s VIF advertised).\\\\r\\\\n\\\\r\\\\n## Direct Connect + TGW Considerations\\\\r\\\\n\\\\r\\\\n### LP Communities Not Honored Across Multiple DX-GW Attachments\\\\r\\\\n\\\\r\\\\n- Direct Connect Local Preference BGP communities (`7224:7300/7200/7100`) are **NOT honored** across multiple DX Gateway attachments on TGW\\\\r\\\\n- This differs from Cloud WAN where communities influence DXGW path selection\\\\r\\\\n- For TGW with multiple DX-GW attachments advertising the same prefix, ECMP or oldest-route logic applies \\u2014 NOT community-based preference\\\\r\\\\n\\\\r\\\\n### IPv6 Addressing Limitation\\\\r\\\\n\\\\r\\\\n- When attaching VPCs with **AWS-assigned IPv6** addresses to a TGW, these IPv6 CIDRs are discontinuous by default and **cannot be aggregated** for use with Direct Connect\\\\r\\\\n- This leads to DX prefix quota limitations\\\\r\\\\n- **Workarounds:**\\\\r\\\\n - Use Amazon-provided **contiguous** IPv6 CIDR blocks\\\\r\\\\n - Use **EC2 BYOIP** (Bring Your Own IP) for aggregatable address space\\\\r\\\\n\\\\r\\\\n### Allowed Prefixes Update Behavior (AWS \\u2192 On-Prem)\\\\r\\\\n\\\\r\\\\n- Updates (Add and Remove) to the \\\"Allowed prefixes\\\" in the TGW \\u2194 DX-GW association are **executed at the same time** on the AWS side\\\\r\\\\n- However, from an **on-premises perspective**, it appears as prefixes added first, then removed later\\\\r\\\\n- This is because on-prem learns the new route once added to the first device, and only loses the old route once withdrawn from all devices\\\\r\\\\n- Transition window: typically **1 to 6 minutes**\\\\r\\\\n\\\\r\\\\n## Route Advertisement Limits & Summarization\\\\r\\\\n\\\\r\\\\n### TGW-to-VPN 5,000 Route Advertisement Limit\\\\r\\\\n\\\\r\\\\n- The limit of **5,000 routes advertised from TGW to VPN** (toward the customer gateway) is **not adjustable**\\\\r\\\\n- The TGW route table itself supports **10,000 routes** \\u2014 it can hold more than the VPN advertisement limit\\\\r\\\\n- **Advertisement-selection order when the attachment\\\\'s advertised-route limit is exceeded** (e.g., TGW\\u2192VPN at 5,000): the routes that get advertised are chosen by (1) **static takes precedence over propagated**, then (2) among routes of the **same type** (both static or both propagated), the **least-specific** is advertised first. This is what lets a static summary survive the cap while the propagated specifics it covers are the ones dropped.\\\\r\\\\n - **[Field knowledge]** This limit-hit advertisement-selection order (static-over-propagated, then least-specific-first) is field-proven behavior and is **not stated on the public AWS TGW route-evaluation page** (which documents most-specific-first forwarding and static-over-propagated for same-CIDR). Treat as operational guidance; validate in-account.\\\\r\\\\n\\\\r\\\\n### Correct Summarization Approach\\\\r\\\\n\\\\r\\\\n**Do NOT remove VPC propagation** \\u2014 propagated routes are needed for the return path (VPN \\u2192 individual VPCs via longest prefix match within the TGW data plane).\\\\r\\\\n\\\\r\\\\n**How it works:**\\\\r\\\\n1. **Keep VPC propagation enabled** \\u2014 individual VPC routes (/16s, /24s) remain in the TGW route table for return path forwarding\\\\r\\\\n2. **Add static summary routes** covering the VPC CIDR ranges (e.g., 10.0.0.0/14 covering 10.0-10.3.x.x)\\\\r\\\\n3. **Advertisement-selection under the cap** favors the static summary \\u2014 static beats propagated, and among same-type routes the least-specific (the summary) is advertised first\\\\r\\\\n4. Once the advertised-route limit is reached, remaining propagated specifics are cut off \\u2014 but the summary already covers them, so reachability is preserved\\\\r\\\\n5. **Return traffic** from VPN to individual VPCs still works because TGW uses longest prefix match internally \\u2014 the more-specific propagated /16 routes win over the /14 summary for data plane forwarding\\\\r\\\\n\\\\r\\\\n**Key points:**\\\\r\\\\n- TGW route table holds BOTH the static summary AND the propagated specifics (up to 10,000 total)\\\\r\\\\n- The static summary consumes 1 route in the 5,000 VPN advertisement budget (instead of hundreds of individual VPC routes)\\\\r\\\\n- VPC propagation is NOT removed \\u2014 it provides the granular return path\\\\r\\\\n- This pattern leverages the difference between the advertisement limit (5,000) and the route table capacity (10,000)\\\\r\\\\n- **Static summary routes must point to a valid attachment** \\u2014 if the attachment is deleted, the route becomes blackholed, and blackholed routes are NOT propagated/advertised. Point summaries to a persistent attachment:\\\\r\\\\n - A **shared-services or firewall VPC** (recommended \\u2014 already exists, won\\\\'t be deleted)\\\\r\\\\n - A **dedicated \\\"dummy\\\" VPC** \\u2014 a minimal VPC created specifically to anchor static summary routes. Disable propagation on this VPC (no traffic originates from it), label it clearly as non-deletable, and tag it for operational protection. This ensures summary routes remain active even if workload VPCs are added/removed.\\\\r\\\\n\\\\r\\\\n### Alternatives When Summarization Isn\\\\'t Sufficient\\\\r\\\\n\\\\r\\\\n- **TGW Connect (GRE+BGP):** Each Connect peer has its own 5,000-route advertisement quota. With 4 peers per attachment, scales horizontally.\\\\r\\\\n- **Multiple VPN connections:** Segment route tables per VPN so each stays under 5,000 advertised. Adds on-prem complexity.\\\\r\\\\n- **Cloud WAN:** Native outbound route summarization via routing policies.\\\\r\\\\n\\\\r\\\\n## Failover & Convergence\\\\r\\\\n\\\\r\\\\n### On-Prem \\u2192 AWS Convergence\\\\r\\\\n\\\\r\\\\n- Route table updates for **propagated routes from BGP peers** can take **up to 60 seconds** to take effect within TGW between attachment paths\\\\r\\\\n- Customers needing faster failover should:\\\\r\\\\n - Route traffic over the **same/single attachment** (avoid cross-attachment failover dependency)\\\\r\\\\n - Consider **bypassing TGW** for high-priority workloads that cannot tolerate 60s convergence\\\\r\\\\n\\\\r\\\\n## TGW Peering\\\\r\\\\n\\\\r\\\\n### Policy-Based Routing (PBR) \\u2014 Transit Gateway Policy Tables\\\\r\\\\n\\\\r\\\\nTGW now supports **Policy-Based Routing** via policy tables, enabling forwarding decisions based on packet attributes beyond just destination IP:\\\\r\\\\n\\\\r\\\\n- Forwarding decisions can be based on: **source IP, destination IP, source port, destination port, and protocol**\\\\r\\\\n- Configured through **policy tables** \\u2014 ordered lists of rules that classify traffic and direct matching packets to a specified TGW route table\\\\r\\\\n- A policy table is associated with a TGW attachment \\u2014 it **replaces** the standard route table on that attachment (attachment can have either a policy table OR a route table, not both)\\\\r\\\\n- Available in all regions at no additional charge beyond standard TGW fees\\\\r\\\\n\\\\r\\\\n**Use cases:**\\\\r\\\\n- Steer traffic selectively to security appliances or inspection services based on source, port, or protocol\\\\r\\\\n- Route traffic from different applications or user populations over different network paths\\\\r\\\\n- Isolate network traffic into separate routing domains for security segmentation\\\\r\\\\n\\\\r\\\\n**Reference:** [Transit Gateway Policy Tables](https://docs.aws.amazon.com/vpc/latest/tgw/tgw-policy-tables.html)\\\\r\\\\n\\\\r\\\\n### Inter-Region Peering\\\\r\\\\n\\\\r\\\\n- TGW Inter-Region Peering (across regions) is **not yet supported in all regions**\\\\r\\\\n- **BGP dynamic route propagation across TGW peering is not yet supported** \\u2014 routes must be statically configured on peered TGW route tables\\\\r\\\\n- This limitation does NOT apply to Cloud WAN dynamic peering\\\\r\\\\n\\\\r\\\\n### Intra-Region Peering\\\\r\\\\n\\\\r\\\\n- TGW Intra-Region Peering (within a region) **is supported**\\\\r\\\\n\\\\r\\\\n### Alternative to Multiple TGWs\\\\r\\\\n\\\\r\\\\n- Instead of deploying multiple TGWs, customers can use **Route Tables within a single TGW** to segregate traffic between different domains/environments\\\\r\\\\n- This simplifies management and avoids peering complexity\\\\r\\\\n\\\\r\\\\n## Multi-Region Traffic & AZ Awareness\\\\r\\\\n\\\\r\\\\n### Gray Failure Risk with Static TGW Peering\\\\r\\\\n\\\\r\\\\n- Traffic between peered TGWs **does not have AZ awareness**\\\\r\\\\n- When traffic arrives from a remote region over **static TGW peering**, it could first land in an AZ different from the destination AZ before being routed to its final destination\\\\r\\\\n- During **gray failures** (partial AZ degradation): packet loss can be observed even though the destination AZ is not impacted \\u2014 because traffic may transit the degraded AZ\\\\r\\\\n- **This limitation only applies to static TGW Peerings**\\\\r\\\\n- It will NOT exist once BGP dynamic route propagation across TGW peering becomes available\\\\r\\\\n- It does NOT apply to **Cloud WAN dynamic peering** (which has AZ awareness)\\\\r\\\\n'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:24.516000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "ccbde5fc-16ba-414d-8d9e-ef9ddf6b26c8", + "executionId": "82ba7a6c-5308-4ed3-ac48-ee68be7bc349", + "recordId": "b161932d-f84c-43cf-8d68-87f7b59ce16b", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.7, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:24.600000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "ccbde5fc-16ba-414d-8d9e-ef9ddf6b26c8", + "executionId": "82ba7a6c-5308-4ed3-ac48-ee68be7bc349", + "recordId": "fe3e17ba-005d-42c3-9132-029674be512d", + "content": "{\"id\": \"fe3e17ba-005d-42c3-9132-029674be512d\", \"role\": \"assistant\", \"content\": [{\"text\": \"## Load-balances (ECMP) when:\\n\\n- The two Direct Connect paths come from **two separate DX Gateway attachments** on the TGW (not two VIFs on the same DXGW \\u2014 more on that below), **and**\\n- Both DXGWs advertise the prefix with **truly identical AS-paths \\u2014 same length *and* same ASNs**.\\n\\nIf both hold, the TGW installs both as ECMP candidates and load-balances traffic across them.\\n\\n## Picks just one when:\\n\\n- **Two VIFs, one DXGW:** A single DX Gateway only ever advertises **one best path per prefix** to the TGW, no matter how many VIFs are attached to it. It does its own internal best-path selection across its VIFs first, so the TGW never sees more than one candidate from that DXGW \\u2014 the redundancy is effectively **active/passive** (DXGW fails over internally; TGW just sees whichever VIF currently wins).\\n- **Two DXGWs, but ASNs differ:** TGW doesn't support AS-path-relaxed multipath. If the AS-paths aren't identical (including the actual ASN values, not just path length), ECMP is skipped and TGW installs the **oldest route** \\u2014 a single path, not a blend.\\n- **DX vs. a different attachment type for the same prefix** (e.g., DX-GW vs. VPN): TGW never ECMPs across different attachment types. The standard route-evaluation order picks a winner instead (notably, static VPN outranks DX by default).\\n\\nOne nuance worth flagging: the \\\"ASNs must match, not just AS-path length\\\" requirement is specific to **TGW ECMP across DXGW attachments**. It's different from DX-side ECMP across VIFs within a region, where AS_PATH length and BGP attributes matching is enough and the ASNs themselves don't need to match \\u2014 so don't conflate the two decision points when troubleshooting.\\n\\n**To validate in your account** (read-only):\\n- `search-transit-gateway-routes` \\u2014 see the per-prefix candidates and which attachment(s) are active/ECMP'd\\n- `list-virtual-interface-routes` \\u2014 see what each DXGW's VIF is actually advertising internally\\n\\nWant me to check a specific TGW's route table or DXGW attachments against this logic?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:24.695000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "ccbde5fc-16ba-414d-8d9e-ef9ddf6b26c8", + "executionId": "82ba7a6c-5308-4ed3-ac48-ee68be7bc349", + "recordId": "3d6beea9-a810-4a54-8a98-3972197cd917", + "content": "{\"id\": \"c558cfbb-392e-497f-b96d-f815e79a4b7d\", \"role\": \"assistant\", \"content\": [{\"type\": \"text\", \"text\": \"## Load-balances (ECMP) when:\\n\\n- The two Direct Connect paths come from **two separate DX Gateway attachments** on the TGW (not two VIFs on the same DXGW \\u2014 more on that below), **and**\\n- Both DXGWs advertise the prefix with **truly identical AS-paths \\u2014 same length *and* same ASNs**.\\n\\nIf both hold, the TGW installs both as ECMP candidates and load-balances traffic across them.\\n\\n## Picks just one when:\\n\\n- **Two VIFs, one DXGW:** A single DX Gateway only ever advertises **one best path per prefix** to the TGW, no matter how many VIFs are attached to it. It does its own internal best-path selection across its VIFs first, so the TGW never sees more than one candidate from that DXGW \\u2014 the redundancy is effectively **active/passive** (DXGW fails over internally; TGW just sees whichever VIF currently wins).\\n- **Two DXGWs, but ASNs differ:** TGW doesn't support AS-path-relaxed multipath. If the AS-paths aren't identical (including the actual ASN values, not just path length), ECMP is skipped and TGW installs the **oldest route** \\u2014 a single path, not a blend.\\n- **DX vs. a different attachment type for the same prefix** (e.g., DX-GW vs. VPN): TGW never ECMPs across different attachment types. The standard route-evaluation order picks a winner instead (notably, static VPN outranks DX by default).\\n\\nOne nuance worth flagging: the \\\"ASNs must match, not just AS-path length\\\" requirement is specific to **TGW ECMP across DXGW attachments**. It's different from DX-side ECMP across VIFs within a region, where AS_PATH length and BGP attributes matching is enough and the ASNs themselves don't need to match \\u2014 so don't conflate the two decision points when troubleshooting.\\n\\n**To validate in your account** (read-only):\\n- `search-transit-gateway-routes` \\u2014 see the per-prefix candidates and which attachment(s) are active/ECMP'd\\n- `list-virtual-interface-routes` \\u2014 see what each DXGW's VIF is actually advertising internally\\n\\nWant me to check a specific TGW's route table or DXGW attachments against this logic?\"}]}", + "createdAt": "2026-10-27T17:05:24.778000-05:00", + "recordType": "final_response" + } +] \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/tgw-ecmp-behavior/without_skill/functional-tests-results.json b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-ecmp-behavior/without_skill/functional-tests-results.json new file mode 100644 index 00000000..b456d4ad --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-ecmp-behavior/without_skill/functional-tests-results.json @@ -0,0 +1,61 @@ +{ + "version": "v1", + "iteration": 1, + "eval_id": "tgw-ecmp-behavior", + "run_type": "without_skill", + "success": true, + "error": null, + "understanding_agent_space_skill": null, + "tests": { + "trigger": null, + "expected_output": { + "result": "passed", + "reasoning": "The expected output requires: (1) ECMP requires same attachment type and equal AS-path length/BGP attributes, otherwise TGW installs a single route, and (2) LP communities not honored across multiple DXGW attachments. The agent's response covers point (1) well - it explains ECMP occurs when paths have identical prefix, prefix length, and AS_PATH over transit VIFs attached to the same DX Gateway, and that differing AS_PATH/prefix/ASN causes single-path selection via best-path selection (implying a single route is chosen, consistent with 'single (oldest) route' though it doesn't use the word 'oldest'). The response also touches on the DXGW-related nuance by noting that separate DX Gateways break ECMP and AWS recommends a single DXGW - this is directionally consistent with the LP communities point, though it doesn't explicitly mention 'Local Preference communities' by name. The core technical substance (attachment symmetry, equal BGP attributes required for ECMP, otherwise single path) is substantively present and correct, even though some specific terminology (oldest route, LP communities) isn't used verbatim.", + "confidence": "medium" + }, + "assertions": { + "assertion_results": [ + { + "text": "states ECMP requires same attachment type and equal AS-path/BGP attributes", + "evaluator": "llm", + "passed": true, + "evidence": "The output states ECMP occurs when 'Both Direct Connect paths terminate on transit virtual interfaces attached to the same Direct Connect Gateway' (same attachment type) and 'The routes learned over BGP on each path have identical network prefix, prefix length, and AS_PATH' (equal AS-path/BGP attributes).", + "reasoning": "The response explicitly covers both the attachment type condition (same DX Gateway/transit VIFs) and the requirement for equal AS_PATH and other BGP attributes (prefix, prefix length) for ECMP to occur.", + "confidence": "high" + }, + { + "text": "notes TGW picks a single route when ECMP conditions are not met", + "evaluator": "llm", + "passed": true, + "evidence": "The output states 'The AS_PATH, prefix, or prefix length differ between the two paths \u2014 Transit Gateway picks the preferred route using normal BGP best-path selection rather than splitting traffic' and concludes 'Anything that breaks that symmetry ... \u2192 Transit Gateway picks a single best path instead.'", + "reasoning": "The response clearly and explicitly notes that when ECMP conditions are not met (asymmetry in BGP attributes, different DX gateways, different ASNs), TGW falls back to selecting a single best path via normal BGP best-path selection.", + "confidence": "high" + }, + { + "text": "does not contain 'error'", + "evaluator": "llm", + "passed": true, + "evidence": "No instance of the word 'error' appears anywhere in the output text.", + "reasoning": "A full scan of the response text shows no occurrence of the word 'error' in any form.", + "confidence": "high" + } + ], + "summary": { + "passed": 3, + "failed": 0, + "errored": 0, + "low_confidence": 0, + "total": 3, + "pass_rate": 1.0 + } + } + }, + "metrics": { + "runtime": "7s", + "cost": "$0.07", + "context_window": { + "utilization": "4.1%", + "compaction_count": 0 + } + } +} \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/tgw-ecmp-behavior/without_skill/outputs/journal_records.json b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-ecmp-behavior/without_skill/outputs/journal_records.json new file mode 100644 index 00000000..0c3ea830 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-ecmp-behavior/without_skill/outputs/journal_records.json @@ -0,0 +1,90 @@ +[ + { + "agentSpaceId": "8789f1a2-4be3-4fb8-91d1-90f9f9746251", + "executionId": "9829c626-898b-4d99-a560-ebfdb350c0df", + "recordId": "25dbe261-e5e7-4d4e-9b7e-41f7b52ca896", + "content": "{\"id\": \"25dbe261-e5e7-4d4e-9b7e-41f7b52ca896\", \"role\": \"user\", \"content\": [{\"text\": \"When does a Transit Gateway load-balance across two Direct Connect paths, and when does it pick just one?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:04.213000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "8789f1a2-4be3-4fb8-91d1-90f9f9746251", + "executionId": "9829c626-898b-4d99-a560-ebfdb350c0df", + "recordId": "6c703595-eb87-4e5f-b26b-4ca9fc3ce5c6", + "content": "{\"id\": \"e5d9ab7b-57e4-412c-a015-f80d1bd67c12\", \"role\": \"assistant\", \"content\": [{\"type\": \"chat_title\", \"title\": \"Transit Gateway load balancing Direct Connect paths\"}]}", + "createdAt": "2026-10-27T17:05:04.778000-05:00", + "recordType": "chat_title" + }, + { + "agentSpaceId": "8789f1a2-4be3-4fb8-91d1-90f9f9746251", + "executionId": "9829c626-898b-4d99-a560-ebfdb350c0df", + "recordId": "01271668-02ae-4fcd-949e-0e37b38de198", + "content": "{\"id\": \"ff4cecfc-ab63-4c87-85cb-cb14af253d14\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This is actually a general AWS networking question about Transit Gateway and Direct Connect ECMP behavior, not something specific to the DevOps Agent product. Since it's a specific technical claim about AWS behavior, I should verify it via verify_aws_claim given my limited confidence here.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:10.621000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "8789f1a2-4be3-4fb8-91d1-90f9f9746251", + "executionId": "9829c626-898b-4d99-a560-ebfdb350c0df", + "recordId": "86165193-3c45-4bfc-9b93-5337be1a3ad5", + "content": "{\"id\": \"86165193-3c45-4bfc-9b93-5337be1a3ad5\", \"role\": \"assistant\", \"content\": [{\"id\": \"tooluse_rpXGXGlppCFvSPU5XmUaCR\", \"tool_name\": \"verify_aws_claim\", \"input\": {\"claim\": \"Transit Gateway ECMP load balancing across multiple Direct Connect paths - when it load balances vs picks a single path\"}, \"type\": \"tool_use\", \"display_intent\": \"Look up AWS docs on Transit Gateway ECMP behavior with Direct Connect paths\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:10.621000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "8789f1a2-4be3-4fb8-91d1-90f9f9746251", + "executionId": "9829c626-898b-4d99-a560-ebfdb350c0df", + "recordId": "e65d7778-e4f5-4c66-9ec1-b205da38188e", + "content": "{\"id\": \"193ece97-945d-48c6-a675-bf827ab8ae28\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"1522\"}]}", + "createdAt": "2026-10-27T17:05:10.715000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "8789f1a2-4be3-4fb8-91d1-90f9f9746251", + "executionId": "9829c626-898b-4d99-a560-ebfdb350c0df", + "recordId": "b5bb35be-a31c-4231-860e-00fbd3db3135", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 3.8, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:10.715000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "8789f1a2-4be3-4fb8-91d1-90f9f9746251", + "executionId": "9829c626-898b-4d99-a560-ebfdb350c0df", + "recordId": "b4daca98-7bca-484f-a15a-48a2213b15b2", + "content": "{\"id\": \"faec4d18-323e-43a0-8d9d-a5afbe2b399e\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_rpXGXGlppCFvSPU5XmUaCR\", \"status\": \"success\", \"content\": [{\"text\": \"## Documentation search results\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"features\\\",\\\"context\\\":\\\"### Edge connectivity\\\\n\\\\nYou can create VPN connections between your AWS Transit Gateway and on-premises gateways using VPN. You can create multiple VPN connections that announce the same prefixes and enable Equal Cost Multipath (ECMP) between these connections. By load-balancing traffic over multiple paths, ECMP can increase the bandwidth\\\",\\\"url\\\":\\\"https://aws.amazon.com/transit-gateway/features/\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"How AWS Transit Gateway works\\\",\\\"context\\\":\\\"## Equal Cost Multipath routing\\\\n\\\\nAWS Transit Gateway supports Equal Cost Multipath (ECMP) routing for most attachments. For a\\\\nVPN attachment, you can enable or disable ECMP support using the console when\\\\ncreating or modifying a transit gateway. For all other attachment types, the following ECMP\\\\nrestrictions apply:\\\\n\\\\n* VPC - VPC does not support ECMP since CIDR blocks cannot overlap. For example,\\\\n you can't attach a VPC with a CIDR 10.1.0.0/16 with a second VPC using the same\\\\n CIDR to a transit gateway, and then set up routing to load balance the traffic between\\\\n them.\\\\n* VPN - When the **VPN ECMP support** option is disabled, a\\\\n transit gateway uses internal metrics to determine the preferred path in the event of\\\\n equal prefixes across multiple paths. For more information on enabling or\\\\n disabling ECMP for a VPN attachment, see Transit gateways in AWS Transit Gateway.\\\\n* AWS Transit Gateway Connect - AWS Transit Gateway Connect attachments automatically support\\\\n ECMP.\\\\n* AWS Direct Connect Gateway - AWS Direct Connect Gateway attachments automatically\\\\n support ECMP across multiple Direct Connect Gateway attachments when the network\\\\n prefix, prefix length, and AS_PATH are exactly the same.\\\\n* Transit gateway peering - Transit gateway peering does not support ECMP since\\\\n it neither supports dynamic routing nor can you configure the same static route\\\\n against two different targets.\\\\n* VPN Concentrator - VPN Concentrator does not support ECMP.\\\\n\\\\n**Note:**\\\\n\\\\n* BGP Multipath AS-Path Relax is not supported, so you can't use ECMP over\\\\n different Autonomous System Numbers (ASNs).\\\\n* ECMP is not supported between different attachment types. For example, you\\\\n can't enable ECMP between a VPN and a VPC attachment. Instead, transit gateway routes\\\\n are evaluated and traffic routed accordingly to the evaluated route. For\\\\n more information, see Route evaluation order.\\\\n* A single Direct Connect gateway supports ECMP across multiple transit\\\\n virtual interfaces. Therefore, we recommended that you set up and use only a\\\\n single Direct Connect gateway and to not set up and use multiple gateways to\\\\n take advantage of ECMP. For more information about Direct Connect gateways\\\\n and public virtual interfaces, see How do I set up an Active/Active or Active/Passive Direct Connect\\\\n connection to AWS from a public virtual interface?\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/vpc/latest/tgw/how-transit-gateways-work.html\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"AWS Transit Gateway Documentation\\\",\\\"context\\\":\\\"## Edge connectivity\\\\n\\\\nYou can create VPN connections between your AWS Transit Gateway and on-premises gateways using a VPN. You can create multiple VPN connections that announce the same prefixes and enable Equal Cost Multipath (ECMP) between these connections. By load-balancing traffic over multiple paths, ECMP can increase the bandwidth\\\",\\\"url\\\":\\\"https://aws.amazon.com/documentation-overview/transit-gateway/\\\"}]}}\\n\\n## Recent AWS announcements (What's New)\\n\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"transitgateway\\\",\\\"skill_description\\\":\\\"Configures AWS Transit Gateway: creating a hub and attaching VPCs, segmenting traffic with route tables, centralizing egress and forcing east-west traffic between VPCs through a central firewall via transit gateway route tables and appliance-mode attachments, connecting on-premises networks over the transit gateway side of a Site-to-Site VPN or Direct Connect attachment (including ECMP to aggregate bandwidth across multiple VPN tunnels), peering transit gateways across Regions, migrating from a VPC peering mesh, and routing IP multicast. Applicable when the transit gateway is the resource being configured: connecting many VPCs, isolating environments, routing egress or east-west traffic through the hub for inspection, reaching on-premises over the hub, linking Regions, or moving off a peering mesh. Not applicable for single-VPC routing, VPC peering between two VPCs (vpcpeering skill), Direct Connect gateway or virtual interface setup (directconnect skill), or Route 53 DNS work.\\\\n\\\\n\\\\nServices: transitgateway, vpc, directconnect, sitetositevpn, gatewayloadbalancer\\\\nTasks: configure, secure, deploy, migrate\\\\nPersona: developer, devops, network-admin, security-engineer, sre\\\\nWorkload: networking, security, hybrid, multi-region\\\\n*\\\",\\\"skill_name\\\":\\\"transitgateway\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\n\\\\n\\\\nServices: directconnect, vpc, transitgateway\\\\nTasks: configure, secure, deploy\\\\nPersona: network-admin, devops, developer, sre\\\\nWorkload: networking, hybrid-cloud\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"aws-networking\\\",\\\"skill_description\\\":\\\"Routes AWS networking requests to the correct service skill for implementation. Covers Route 53 (DNS, health checks, routing policies, Resolver, DNS Firewall), CloudFront (caching, edge, OAC, mTLS, signed URLs), Transit Gateway (multi-VPC hub, segmentation, centralized egress), Direct Connect (hybrid link, DX Gateway, MACsec), Site-to-Site VPN (IPsec tunnels, static or BGP), WAF (web ACLs, AWS Managed Rules, rate-based rules, Bot and Fraud Control), and Shield Advanced (L3/L4 DDoS). Applicable when creating, configuring, troubleshooting, or designing across these services, choosing between them, or diagnosing connectivity or traffic-filtering issues. Not for VPC subnets and route tables, load balancers, VPC endpoints, PrivateLink, API Gateway, IAM policy logic, container or serverless networking, or IaC authoring.\\\\n\\\\nServices: route53, route53resolver, cloudfront, directconnect, transitgateway, vpn, waf, shield\\\\nTasks: configure, troubleshoot, debug, deploy, secure, migrate\\\\nPersona: developer, network-engineer, devops, security-engineer\\\\nWorkload: networking, security\\\\n*\\\",\\\"skill_name\\\":\\\"aws-networking\\\"}]}}\"}]}], \"label\": \"Look up AWS docs on Transit Gateway ECMP behavior with Direct Connect paths\"}", + "createdAt": "2026-10-27T17:05:12.499000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "8789f1a2-4be3-4fb8-91d1-90f9f9746251", + "executionId": "9829c626-898b-4d99-a560-ebfdb350c0df", + "recordId": "d2398da7-fdf3-4d04-b5fb-ddc4cb86f424", + "content": "{\"id\": \"d2398da7-fdf3-4d04-b5fb-ddc4cb86f424\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_rpXGXGlppCFvSPU5XmUaCR\", \"content\": \"[{'text': '## Documentation search results\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"features\\\",\\\"context\\\":\\\"### Edge connectivity\\\\\\\\n\\\\\\\\nYou can create VPN connections between your AWS Transit Gateway and on-premises gateways using VPN. You can create multiple VPN connections that announce the same prefixes and enable Equal Cost Multipath (ECMP) between these connections. By load-balancing traffic over multiple paths, ECMP can increase the bandwidth\\\",\\\"url\\\":\\\"https://aws.amazon.com/transit-gateway/features/\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"How AWS Transit Gateway works\\\",\\\"context\\\":\\\"## Equal Cost Multipath routing\\\\\\\\n\\\\\\\\nAWS Transit Gateway supports Equal Cost Multipath (ECMP) routing for most attachments. For a\\\\\\\\nVPN attachment, you can enable or disable ECMP support using the console when\\\\\\\\ncreating or modifying a transit gateway. For all other attachment types, the following ECMP\\\\\\\\nrestrictions apply:\\\\\\\\n\\\\\\\\n* VPC - VPC does not support ECMP since CIDR blocks cannot overlap. For example,\\\\\\\\n you can\\\\'t attach a VPC with a CIDR 10.1.0.0/16 with a second VPC using the same\\\\\\\\n CIDR to a transit gateway, and then set up routing to load balance the traffic between\\\\\\\\n them.\\\\\\\\n* VPN - When the **VPN ECMP support** option is disabled, a\\\\\\\\n transit gateway uses internal metrics to determine the preferred path in the event of\\\\\\\\n equal prefixes across multiple paths. For more information on enabling or\\\\\\\\n disabling ECMP for a VPN attachment, see Transit gateways in AWS Transit Gateway.\\\\\\\\n* AWS Transit Gateway Connect - AWS Transit Gateway Connect attachments automatically support\\\\\\\\n ECMP.\\\\\\\\n* AWS Direct Connect Gateway - AWS Direct Connect Gateway attachments automatically\\\\\\\\n support ECMP across multiple Direct Connect Gateway attachments when the network\\\\\\\\n prefix, prefix length, and AS_PATH are exactly the same.\\\\\\\\n* Transit gateway peering - Transit gateway peering does not support ECMP since\\\\\\\\n it neither supports dynamic routing nor can you configure the same static route\\\\\\\\n against two different targets.\\\\\\\\n* VPN Concentrator - VPN Concentrator does not support ECMP.\\\\\\\\n\\\\\\\\n**Note:**\\\\\\\\n\\\\\\\\n* BGP Multipath AS-Path Relax is not supported, so you can\\\\'t use ECMP over\\\\\\\\n different Autonomous System Numbers (ASNs).\\\\\\\\n* ECMP is not supported between different attachment types. For example, you\\\\\\\\n can\\\\'t enable ECMP between a VPN and a VPC attachment. Instead, transit gateway routes\\\\\\\\n are evaluated and traffic routed accordingly to the evaluated route. For\\\\\\\\n more information, see Route evaluation order.\\\\\\\\n* A single Direct Connect gateway supports ECMP across multiple transit\\\\\\\\n virtual interfaces. Therefore, we recommended that you set up and use only a\\\\\\\\n single Direct Connect gateway and to not set up and use multiple gateways to\\\\\\\\n take advantage of ECMP. For more information about Direct Connect gateways\\\\\\\\n and public virtual interfaces, see How do I set up an Active/Active or Active/Passive Direct Connect\\\\\\\\n connection to AWS from a public virtual interface?\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/vpc/latest/tgw/how-transit-gateways-work.html\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"AWS Transit Gateway Documentation\\\",\\\"context\\\":\\\"## Edge connectivity\\\\\\\\n\\\\\\\\nYou can create VPN connections between your AWS Transit Gateway and on-premises gateways using a VPN. You can create multiple VPN connections that announce the same prefixes and enable Equal Cost Multipath (ECMP) between these connections. By load-balancing traffic over multiple paths, ECMP can increase the bandwidth\\\",\\\"url\\\":\\\"https://aws.amazon.com/documentation-overview/transit-gateway/\\\"}]}}\\\\n\\\\n## Recent AWS announcements (What\\\\'s New)\\\\n\\\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"transitgateway\\\",\\\"skill_description\\\":\\\"Configures AWS Transit Gateway: creating a hub and attaching VPCs, segmenting traffic with route tables, centralizing egress and forcing east-west traffic between VPCs through a central firewall via transit gateway route tables and appliance-mode attachments, connecting on-premises networks over the transit gateway side of a Site-to-Site VPN or Direct Connect attachment (including ECMP to aggregate bandwidth across multiple VPN tunnels), peering transit gateways across Regions, migrating from a VPC peering mesh, and routing IP multicast. Applicable when the transit gateway is the resource being configured: connecting many VPCs, isolating environments, routing egress or east-west traffic through the hub for inspection, reaching on-premises over the hub, linking Regions, or moving off a peering mesh. Not applicable for single-VPC routing, VPC peering between two VPCs (vpcpeering skill), Direct Connect gateway or virtual interface setup (directconnect skill), or Route 53 DNS work.\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: transitgateway, vpc, directconnect, sitetositevpn, gatewayloadbalancer\\\\\\\\nTasks: configure, secure, deploy, migrate\\\\\\\\nPersona: developer, devops, network-admin, security-engineer, sre\\\\\\\\nWorkload: networking, security, hybrid, multi-region\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"transitgateway\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: directconnect, vpc, transitgateway\\\\\\\\nTasks: configure, secure, deploy\\\\\\\\nPersona: network-admin, devops, developer, sre\\\\\\\\nWorkload: networking, hybrid-cloud\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"aws-networking\\\",\\\"skill_description\\\":\\\"Routes AWS networking requests to the correct service skill for implementation. Covers Route 53 (DNS, health checks, routing policies, Resolver, DNS Firewall), CloudFront (caching, edge, OAC, mTLS, signed URLs), Transit Gateway (multi-VPC hub, segmentation, centralized egress), Direct Connect (hybrid link, DX Gateway, MACsec), Site-to-Site VPN (IPsec tunnels, static or BGP), WAF (web ACLs, AWS Managed Rules, rate-based rules, Bot and Fraud Control), and Shield Advanced (L3/L4 DDoS). Applicable when creating, configuring, troubleshooting, or designing across these services, choosing between them, or diagnosing connectivity or traffic-filtering issues. Not for VPC subnets and route tables, load balancers, VPC endpoints, PrivateLink, API Gateway, IAM policy logic, container or serverless networking, or IaC authoring.\\\\\\\\n\\\\\\\\nServices: route53, route53resolver, cloudfront, directconnect, transitgateway, vpn, waf, shield\\\\\\\\nTasks: configure, troubleshoot, debug, deploy, secure, migrate\\\\\\\\nPersona: developer, network-engineer, devops, security-engineer\\\\\\\\nWorkload: networking, security\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"aws-networking\\\"}]}}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:20.115000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "8789f1a2-4be3-4fb8-91d1-90f9f9746251", + "executionId": "9829c626-898b-4d99-a560-ebfdb350c0df", + "recordId": "d4602968-cc56-4427-8dec-c7694017419c", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.1, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:20.180000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "8789f1a2-4be3-4fb8-91d1-90f9f9746251", + "executionId": "9829c626-898b-4d99-a560-ebfdb350c0df", + "recordId": "359e5e1c-b8e8-4799-b2fd-7de09a35b9f1", + "content": "{\"id\": \"359e5e1c-b8e8-4799-b2fd-7de09a35b9f1\", \"role\": \"assistant\", \"content\": [{\"text\": \"## Transit Gateway ECMP over Direct Connect\\n\\n**It load-balances (ECMP) when both paths are advertised as \\\"equal\\\":**\\n- Both Direct Connect paths terminate on **transit virtual interfaces attached to the same Direct Connect Gateway**, which is then associated to the Transit Gateway\\n- The routes learned over BGP on each path have **identical network prefix, prefix length, and AS_PATH**\\n- In this setup, Transit Gateway automatically ECMPs traffic across the multiple transit VIFs \\u2014 no toggle needed, it just works when the paths are truly equal\\n\\n**It picks just one path when:**\\n- The AS_PATH, prefix, or prefix length differ between the two paths \\u2014 Transit Gateway picks the preferred route using normal BGP best-path selection rather than splitting traffic\\n- The two paths are on **separate Direct Connect Gateways** rather than a single shared one \\u2014 AWS explicitly recommends using a single Direct Connect Gateway so ECMP kicks in, since ECMP across *multiple* DX Gateways isn't the supported pattern\\n- BGP Multipath AS-Path Relax isn't supported, so two paths over **different ASNs** won't ECMP even if everything else matches\\n- More generally, any asymmetry that makes one path look \\\"more preferred\\\" in route evaluation causes single-path selection rather than load-balancing\\n\\nSo the practical rule of thumb: one DX Gateway, two transit VIFs, identical advertised routes \\u2192 ECMP load-balancing. Anything that breaks that symmetry (different ASN, different AS_PATH, separate DX Gateways) \\u2192 Transit Gateway picks a single best path instead.\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:20.258000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "8789f1a2-4be3-4fb8-91d1-90f9f9746251", + "executionId": "9829c626-898b-4d99-a560-ebfdb350c0df", + "recordId": "a000743d-dd5a-466d-911c-b1316a0fea8d", + "content": "{\"id\": \"cb6d25e6-94d2-4fc8-b15e-68d0f511e17a\", \"role\": \"assistant\", \"content\": [{\"type\": \"text\", \"text\": \"## Transit Gateway ECMP over Direct Connect\\n\\n**It load-balances (ECMP) when both paths are advertised as \\\"equal\\\":**\\n- Both Direct Connect paths terminate on **transit virtual interfaces attached to the same Direct Connect Gateway**, which is then associated to the Transit Gateway\\n- The routes learned over BGP on each path have **identical network prefix, prefix length, and AS_PATH**\\n- In this setup, Transit Gateway automatically ECMPs traffic across the multiple transit VIFs \\u2014 no toggle needed, it just works when the paths are truly equal\\n\\n**It picks just one path when:**\\n- The AS_PATH, prefix, or prefix length differ between the two paths \\u2014 Transit Gateway picks the preferred route using normal BGP best-path selection rather than splitting traffic\\n- The two paths are on **separate Direct Connect Gateways** rather than a single shared one \\u2014 AWS explicitly recommends using a single Direct Connect Gateway so ECMP kicks in, since ECMP across *multiple* DX Gateways isn't the supported pattern\\n- BGP Multipath AS-Path Relax isn't supported, so two paths over **different ASNs** won't ECMP even if everything else matches\\n- More generally, any asymmetry that makes one path look \\\"more preferred\\\" in route evaluation causes single-path selection rather than load-balancing\\n\\nSo the practical rule of thumb: one DX Gateway, two transit VIFs, identical advertised routes \\u2192 ECMP load-balancing. Anything that breaks that symmetry (different ASN, different AS_PATH, separate DX Gateways) \\u2192 Transit Gateway picks a single best path instead.\"}]}", + "createdAt": "2026-10-27T17:05:20.337000-05:00", + "recordType": "final_response" + } +] \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vpn-summarization/with_skill/functional-tests-results.json b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vpn-summarization/with_skill/functional-tests-results.json new file mode 100644 index 00000000..cae0b4cb --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vpn-summarization/with_skill/functional-tests-results.json @@ -0,0 +1,74 @@ +{ + "version": "v1", + "iteration": 1, + "eval_id": "tgw-vpn-summarization", + "run_type": "with_skill", + "success": true, + "error": null, + "understanding_agent_space_skill": null, + "tests": { + "trigger": { + "result": "passed", + "message": "Skill 'aws-routing' was triggered as expected", + "skill_loads_found": 1 + }, + "expected_output": { + "result": "passed", + "reasoning": "The agent's response directly matches all key elements of the expected output: (1) explicitly states to keep VPC propagation enabled for longest-prefix-match return path forwarding, (2) instructs adding a static summary route to the TGW route table, (3) explains the route selection order - static beats propagated, and among same-type least-specific is advertised first - meaning the summary survives the advertisement cap, and (4) recommends anchoring the summary route to a persistent attachment (shared-services VPC or non-deletable dummy VPC) to avoid blackholing. All core technical points align precisely with the expected output description.", + "evidence": "\"Keep VPC propagation enabled... VPN \u2192 individual VPCs is resolved by longest-prefix-match in the TGW data plane... Add static summary route(s) covering the VPC CIDR ranges... the selection order is: static beats propagated, then among same-type routes least-specific advertised first... Point the static summary at a persistent attachment \u2014 a shared-services/firewall VPC, or a dedicated non-deletable 'dummy' VPC\"", + "confidence": "high" + }, + "assertions": { + "assertion_results": [ + { + "text": "keeps VPC route propagation enabled for the return path", + "evaluator": "llm", + "passed": true, + "evidence": "\"Keep VPC propagation enabled \u2014 all the granular /16s/24s stay in the TGW route table.\" and \"Do not disable VPC propagation. The individual propagated VPC routes ... are what the TGW uses for the return path\"", + "reasoning": "The agent explicitly instructs keeping VPC propagation enabled and explains why it preserves the return path.", + "confidence": "high" + }, + { + "text": "adds a static summary route in the TGW route table associated with the VPN attachment", + "evaluator": "llm", + "passed": true, + "evidence": "\"Add static summary route(s) covering the VPC CIDR ranges (e.g., 10.0.0.0/14 over 10.0\u201310.3.x.x).\"", + "reasoning": "The agent explicitly recommends adding a static summary route; while it doesn't use the exact phrase 'TGW route table associated with the VPN attachment', the context (TGW route table discussion, pointing to a persistent attachment) implies this is the static route added to the relevant TGW route table.", + "confidence": "medium" + }, + { + "text": "explains static-over-propagated and least-specific selection under the advertisement cap", + "evaluator": "llm", + "passed": true, + "evidence": "\"the selection order is: static beats propagated, then among same-type routes least-specific advertised first. That means the static summary is advertised and survives the cap, while the propagated specifics it already covers are the ones trimmed off the advertisement\"", + "reasoning": "The agent explicitly explains the static-over-propagated priority and least-specific tiebreak rule under the advertisement cap.", + "confidence": "high" + }, + { + "text": "does not contain 'error'", + "evaluator": "llm", + "passed": true, + "evidence": "No instance of the word 'error' appears anywhere in the output.", + "reasoning": "A full scan of the text shows no occurrence of 'error'.", + "confidence": "high" + } + ], + "summary": { + "passed": 4, + "failed": 0, + "errored": 0, + "low_confidence": 0, + "total": 4, + "pass_rate": 1.0 + } + } + }, + "metrics": { + "runtime": "17s", + "cost": "$0.14", + "context_window": { + "utilization": "5.0%", + "compaction_count": 0 + } + } +} \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vpn-summarization/with_skill/outputs/journal_records.json b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vpn-summarization/with_skill/outputs/journal_records.json new file mode 100644 index 00000000..1bdbbb08 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vpn-summarization/with_skill/outputs/journal_records.json @@ -0,0 +1,146 @@ +[ + { + "agentSpaceId": "03fc9fc9-11c2-443e-81da-17388310a337", + "executionId": "b47f9985-842b-4193-9a12-1c28adcfc238", + "recordId": "aa7394f8-c8d0-4e99-a1b4-83fdebea4ddf", + "content": "{\"id\": \"aa7394f8-c8d0-4e99-a1b4-83fdebea4ddf\", \"role\": \"user\", \"content\": [{\"text\": \"A customer is at the TGW-to-VPN advertised-route limit. How do they summarize without breaking the return path?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:04.016000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "03fc9fc9-11c2-443e-81da-17388310a337", + "executionId": "b47f9985-842b-4193-9a12-1c28adcfc238", + "recordId": "fc1f529f-6e7a-4f65-a287-62ce74aa2a30", + "content": "{\"id\": \"69ca546f-a136-4d7a-a4c4-23637e593a80\", \"role\": \"assistant\", \"content\": [{\"type\": \"chat_title\", \"title\": \"TGW-to-VPN route limit summarization best practices\"}]}", + "createdAt": "2026-10-27T17:05:04.576000-05:00", + "recordType": "chat_title" + }, + { + "agentSpaceId": "03fc9fc9-11c2-443e-81da-17388310a337", + "executionId": "b47f9985-842b-4193-9a12-1c28adcfc238", + "recordId": "9e0cc6ad-9011-41aa-bc71-7add34b2e57b", + "content": "{\"id\": \"64beaa4a-1bbf-42b3-87f6-b322df118b87\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This is a networking question about Transit Gateway VPN route limits and summarization risks, specifically the danger of asymmetric routing if the return path isn't handled consistently. This squarely fits the aws-routing skill's coverage of Transit Gateway route tables and asymmetric routing concerns.\\n\\nThis skill isn't one of the tool-enabling ones in the capability map\\u2014it's a methodology-only skill grounded in documentation, so I should read it myself rather than looking for an associated tool.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:11.087000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "03fc9fc9-11c2-443e-81da-17388310a337", + "executionId": "b47f9985-842b-4193-9a12-1c28adcfc238", + "recordId": "e67dab33-9052-48d2-9d0f-4fb5b4d6bf87", + "content": "{\"id\": \"e67dab33-9052-48d2-9d0f-4fb5b4d6bf87\", \"role\": \"assistant\", \"content\": [{\"text\": \"Let me pull up the routing skill for TGW/VPN route summarization guidance.\", \"type\": \"text\"}, {\"id\": \"tooluse_7udWYjqgJXw8gYZss8nbPZ\", \"tool_name\": \"skill_read\", \"input\": {\"skill_id\": \"aws-routing\"}, \"type\": \"tool_use\", \"display_intent\": \"Read the aws-routing skill for TGW/VPN route summarization guidance.\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:11.099000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "03fc9fc9-11c2-443e-81da-17388310a337", + "executionId": "b47f9985-842b-4193-9a12-1c28adcfc238", + "recordId": "b68b57e0-dc8d-4279-a432-ae778b8bfe11", + "content": "{\"id\": \"2bbc555d-3a82-4e53-ba8d-11e7838b34af\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"2307\"}]}", + "createdAt": "2026-10-27T17:05:11.179000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "03fc9fc9-11c2-443e-81da-17388310a337", + "executionId": "b47f9985-842b-4193-9a12-1c28adcfc238", + "recordId": "50b55ae2-bb97-458f-a692-acd88d169a08", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 3.8, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:11.189000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "03fc9fc9-11c2-443e-81da-17388310a337", + "executionId": "b47f9985-842b-4193-9a12-1c28adcfc238", + "recordId": "a07b6871-093b-4ff5-8059-2ba03d937474", + "content": "{\"id\": \"31b2ee02-9d84-4383-820b-b00ad581de9b\", \"role\": \"assistant\", \"block_type\": \"load_skill\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_7udWYjqgJXw8gYZss8nbPZ\", \"status\": \"success\", \"content\": [{\"text\": \"Done\"}]}], \"label\": \"Read the aws-routing skill for TGW/VPN route summarization guidance.\"}", + "createdAt": "2026-10-27T17:05:11.297000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "03fc9fc9-11c2-443e-81da-17388310a337", + "executionId": "b47f9985-842b-4193-9a12-1c28adcfc238", + "recordId": "7d0d8d44-a45d-4db0-b45c-aa487c91f347", + "content": "{\"id\": \"7d0d8d44-a45d-4db0-b45c-aa487c91f347\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_7udWYjqgJXw8gYZss8nbPZ\", \"content\": \"[{'text': '{\\\"content\\\": \\\"---\\\\\\\\r\\\\\\\\nname: aws-routing\\\\\\\\r\\\\\\\\ndescription: \\\\'Use this skill when the user wants to know which path AWS will pick for their traffic, why traffic took an unexpected route, or how to make AWS prefer a path. Covers Cloud WAN CNE route evaluation, Direct Connect Gateway local vs remote region preference, Transit Gateway route tables, ECMP and peering, VPC longest-prefix-match, BGP traffic engineering (LP communities 7224:7100/7200/7300, AS-path prepending, MED), DX + VPN redundancy and failover, and asymmetric routing. Use it even when the user only describes a symptom, such as \\\\\\\\\\\"traffic is leaving the region\\\\\\\\\\\", \\\\\\\\\\\"not using my local Direct Connect\\\\\\\\\\\", \\\\\\\\\\\"routing to the wrong VIF/location\\\\\\\\\\\", or \\\\\\\\\\\"path is asymmetric\\\\\\\\\\\". Read-only: analysis, recommendations, and describe/get/list validation commands grounded in public AWS docs.\\\\'\\\\\\\\r\\\\\\\\nmetadata:\\\\\\\\r\\\\\\\\n version: \\\\\\\\\\\"1.2.5\\\\\\\\\\\"\\\\\\\\r\\\\\\\\n author: midakin\\\\\\\\r\\\\\\\\n aws-devops-agent-skills.agent-types: \\\\\\\\\\\"CHAT, INCIDENT_RCA\\\\\\\\\\\"\\\\\\\\r\\\\\\\\n---\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Overview\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nThis skill provides routing-first analysis for AWS networking. It reasons about the full traffic\\\\\\\\r\\\\\\\\npath, applies the documented route-evaluation logic for each service, and grounds recommendations\\\\\\\\r\\\\\\\\nin public AWS documentation. It is **read-only**: it produces analysis, recommendations, and\\\\\\\\r\\\\\\\\nread-only validation commands (`describe-*`, `get-*`, `list-*`) - never mutating actions.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Verification & Anti-Hallucination Directives (Mandatory)\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- Do not generate any fact, citation, or source that is not fully verifiable. If you lack a verified\\\\\\\\r\\\\\\\\n source, state \\\\\\\\\\\"I cannot verify this\\\\\\\\\\\" rather than inventing details.\\\\\\\\r\\\\\\\\n- State assumptions explicitly before answering.\\\\\\\\r\\\\\\\\n- Cross-check each claim against public AWS documentation or the `references/` files. Omit claims you\\\\\\\\r\\\\\\\\n cannot confirm.\\\\\\\\r\\\\\\\\n- Only cite AWS documentation URLs you have actually retrieved or confirmed. Do not fabricate links.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## When to use this skill\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nActivate when the user asks about:\\\\\\\\r\\\\\\\\n- Cloud WAN route evaluation / Core Network Edge (CNE) path selection\\\\\\\\r\\\\\\\\n- Direct Connect Gateway path selection, local-region preference, or LP communities\\\\\\\\r\\\\\\\\n- Transit Gateway route evaluation, ECMP, or peering routing\\\\\\\\r\\\\\\\\n- VPC route tables and longest-prefix-match behavior\\\\\\\\r\\\\\\\\n- BGP traffic engineering (communities, AS-path prepending, MED)\\\\\\\\r\\\\\\\\n- DX + VPN redundancy, active/active vs active/passive, or failover design\\\\\\\\r\\\\\\\\n- Why traffic is taking an unexpected path\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## What this skill does NOT do\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nThis skill is **read-only and advisory**. It will not:\\\\\\\\r\\\\\\\\n- **Design or apply infrastructure changes.** It does not create, modify, or delete AWS resources,\\\\\\\\r\\\\\\\\n and it does not run mutating CLI/API calls (`create-*`, `modify-*`, `associate-*`, `delete-*`,\\\\\\\\r\\\\\\\\n `put-*`, `update-*`) or write Infrastructure-as-Code to be deployed. Any change it describes is a\\\\\\\\r\\\\\\\\n *recommendation* for you to review and implement yourself.\\\\\\\\r\\\\\\\\n- **Configure BGP or push routing policy.** It explains which communities, AS-path, or MED settings\\\\\\\\r\\\\\\\\n to use, but it does not apply them to VIFs, route tables, or Cloud WAN policy documents.\\\\\\\\r\\\\\\\\n- **Execute anything against a live account beyond read-only validation.** Commands it provides are\\\\\\\\r\\\\\\\\n limited to `describe-*`, `get-*`, and `list-*` for confirming state.\\\\\\\\r\\\\\\\\n- **Guarantee production outcomes.** It recommends non-production testing and blue/green rollout;\\\\\\\\r\\\\\\\\n it does not perform the cutover or validate the result for you.\\\\\\\\r\\\\\\\\n- **Provide compliance, security, or contractual sign-off**, or make claims it cannot ground in\\\\\\\\r\\\\\\\\n public AWS documentation or the `references/` files.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nFor designing and applying changes, hand the recommendation to a change-capable workflow (with\\\\\\\\r\\\\\\\\nappropriate review, approvals, and testing).\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## How to analyze a routing problem\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nWork through this checklist in order; each step depends on the one before it.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- [ ] **1. Establish the path.** Identify source, destination, and every routing construct in\\\\\\\\r\\\\\\\\n between (VPC route table \\\\\\\\u2192 TGW/Cloud WAN CNE \\\\\\\\u2192 DXGW \\\\\\\\u2192 VIF \\\\\\\\u2192 on-prem, or the relevant subset).\\\\\\\\r\\\\\\\\n- [ ] **2. Name the route table.** Be explicit about which table makes each decision (VPC RT, TGW RT,\\\\\\\\r\\\\\\\\n CNE RT, DXGW internal selection).\\\\\\\\r\\\\\\\\n- [ ] **3. Apply the evaluation order for that construct** (see below).\\\\\\\\r\\\\\\\\n- [ ] **4. Identify the deciding attribute** (longest prefix, local preference, AS-path, MED, source\\\\\\\\r\\\\\\\\n type).\\\\\\\\r\\\\\\\\n- [ ] **5. Flag non-determinism** (e.g., ECMP or \\\\\\\\\\\"deterministically random\\\\\\\\\\\" tiebreakers) and\\\\\\\\r\\\\\\\\n recommend a deterministic alternative.\\\\\\\\r\\\\\\\\n- [ ] **6. Cite the AWS documentation** that supports the behavior.\\\\\\\\r\\\\\\\\n- [ ] **7. Provide read-only validation commands** the user can run to confirm.\\\\\\\\r\\\\\\\\n- [ ] **8. Self-check before answering.** Re-read the answer against steps 1-7: every route table is\\\\\\\\r\\\\\\\\n named, the evaluation order matches the construct, each cited URL was actually retrieved, and every\\\\\\\\r\\\\\\\\n command is read-only. Fix any gap before presenting.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Key evaluation rules\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Cloud WAN (per CNE)\\\\\\\\r\\\\\\\\nLongest prefix match \\\\\\\\u2192 static \\\\\\\\u2192 VPC-propagated (same region) \\\\\\\\u2192 unequal AS-path/MED (shortest wins) \\\\\\\\u2192\\\\\\\\r\\\\\\\\nequal AS-path & MED source preference: DXGW-propagated \\\\\\\\u2192 Cloud WAN Connect \\\\\\\\u2192 S2S VPN \\\\\\\\u2192 other\\\\\\\\r\\\\\\\\n(TGW peering, remote CNEs; identical from 2+ sources = deterministically random).\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Direct Connect (private/transit VIF, outbound AWS\\\\\\\\u2192on-prem)\\\\\\\\r\\\\\\\\nLongest prefix match \\\\\\\\u2192 local preference \\\\\\\\u2192 AS_PATH length \\\\\\\\u2192 MED \\\\\\\\u2192 ECMP (equal AS_PATH and BGP\\\\\\\\r\\\\\\\\nattributes). LP communities `7224:7300/7200/7100` (High/Medium/Low) set local preference and are\\\\\\\\r\\\\\\\\nevaluated before AS-path. DXGW prefers the local associated-region DX by default.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Direction & tooling guidance\\\\\\\\r\\\\\\\\n- **AWS \\\\\\\\u2192 on-prem (egress):** control with DX LP communities.\\\\\\\\r\\\\\\\\n- **On-prem \\\\\\\\u2192 AWS (ingress):** control with the customer router\\\\'s local-pref/weight/MED.\\\\\\\\r\\\\\\\\n- **AS-path prepending:** within a region only - unreliable across regions because DXGW local-region\\\\\\\\r\\\\\\\\n LP overrides it. **LP communities:** work within-region and cross-region.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Reference material\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nDetailed mechanics, patterns, and caveats are in `references/`. Load only the file that matches the\\\\\\\\r\\\\\\\\nconstruct deciding the path:\\\\\\\\r\\\\\\\\n- [Cloud WAN + DX routing patterns](references/cloudwan-dx-routing-patterns.md) - load when the path\\\\\\\\r\\\\\\\\n crosses a Cloud WAN CNE: CNE route evaluation, DXGW path selection into Cloud WAN, community\\\\\\\\r\\\\\\\\n traffic-engineering patterns, regional inspection, multi-region egress.\\\\\\\\r\\\\\\\\n- [Direct Connect routing patterns](references/dx-routing-patterns.md) - load when the question is\\\\\\\\r\\\\\\\\n about a DX VIF or DXGW without Cloud WAN: BGP mechanics, BFD, MED, ASN ranges, VIF considerations,\\\\\\\\r\\\\\\\\n CloudHub, when VIFs can talk, active/active vs active/passive.\\\\\\\\r\\\\\\\\n- [Transit Gateway routing patterns](references/tgw-routing-patterns.md) - load when a TGW route\\\\\\\\r\\\\\\\\n table makes the decision: TGW route evaluation, ECMP limits, DXGW-to-TGW behavior, DX/VPN caveats,\\\\\\\\r\\\\\\\\n failover, peering.\\\\\\\\r\\\\\\\\n- [VPN + DX redundancy patterns](references/vpn-dx-redundancy-patterns.md) - load when VPN backs up\\\\\\\\r\\\\\\\\n or runs alongside DX: route preference between them, ECMP limits, asymmetric return paths.\\\\\\\\r\\\\\\\\n- [Networking strategic questions](references/networking-strategic-questions.md) - load when key\\\\\\\\r\\\\\\\\n facts are missing (regions, attachments, ASNs, communities) and you need discovery questions to ask\\\\\\\\r\\\\\\\\n before analyzing.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Output expectations\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- Explain the route-evaluation logic step by step.\\\\\\\\r\\\\\\\\n- Name the specific route table involved.\\\\\\\\r\\\\\\\\n- Give the end-to-end traffic path.\\\\\\\\r\\\\\\\\n- Compare options with tradeoffs (determinism, complexity, failover) when multiple solutions exist.\\\\\\\\r\\\\\\\\n- Include read-only AWS CLI validation commands.\\\\\\\\r\\\\\\\\n- Recommend testing in a non-production environment and blue/green for migrations.\\\\\\\\r\\\\\\\\n\\\", \\\"resource_manifest\\\": [\\\"SKILL.md\\\", \\\"references/cloudwan-dx-routing-patterns.md\\\", \\\"references/dx-routing-patterns.md\\\", \\\"references/networking-strategic-questions.md\\\", \\\"references/tgw-routing-patterns.md\\\", \\\"references/vpn-dx-redundancy-patterns.md\\\"]}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:17.129000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "03fc9fc9-11c2-443e-81da-17388310a337", + "executionId": "b47f9985-842b-4193-9a12-1c28adcfc238", + "recordId": "9f80c5a6-127c-4856-a1e4-94658d87ec65", + "content": "{\"id\": \"c5a93883-69c9-4b57-9c75-b100fc2b00e5\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This question touches TGW-to-VPN routing, so I should pull in the tgw-routing-patterns reference for route evaluation and DX/VPN caveats, plus the vpn-dx-redundancy-patterns doc for asymmetric return path risks.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:17.129000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "03fc9fc9-11c2-443e-81da-17388310a337", + "executionId": "b47f9985-842b-4193-9a12-1c28adcfc238", + "recordId": "54d008ba-db63-4482-8599-3c29de1f6d29", + "content": "{\"id\": \"54d008ba-db63-4482-8599-3c29de1f6d29\", \"role\": \"assistant\", \"content\": [{\"id\": \"tooluse_bycTByWzN7cv5GOaNWLLo2\", \"tool_name\": \"read_skill_resource\", \"input\": {\"skill_id\": \"aws-routing\", \"path\": \"references/tgw-routing-patterns.md\"}, \"type\": \"tool_use\", \"display_intent\": \"Read TGW routing patterns reference for route summarization guidance.\"}, {\"id\": \"tooluse_YC18TMU8MS7loxUQW5nbGq\", \"tool_name\": \"read_skill_resource\", \"input\": {\"skill_id\": \"aws-routing\", \"path\": \"references/vpn-dx-redundancy-patterns.md\"}, \"type\": \"tool_use\", \"display_intent\": \"Read VPN+DX redundancy reference for asymmetric return path guidance.\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:17.224000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "03fc9fc9-11c2-443e-81da-17388310a337", + "executionId": "b47f9985-842b-4193-9a12-1c28adcfc238", + "recordId": "8ab8ab15-92a4-4fa4-bf06-9bf7de2ddc2a", + "content": "{\"id\": \"84ec5ab9-8fbf-4a31-976f-07c6331f18ed\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"887\"}]}", + "createdAt": "2026-10-27T17:05:17.231000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "03fc9fc9-11c2-443e-81da-17388310a337", + "executionId": "b47f9985-842b-4193-9a12-1c28adcfc238", + "recordId": "9760e195-8964-4252-aa96-45606caaeba3", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.2, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:17.317000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "03fc9fc9-11c2-443e-81da-17388310a337", + "executionId": "b47f9985-842b-4193-9a12-1c28adcfc238", + "recordId": "c5426b53-6e87-4d36-9cb4-6c176c1ccb44", + "content": "{\"id\": \"1505a681-39fb-48c8-b49f-44637b0b9361\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_bycTByWzN7cv5GOaNWLLo2\", \"status\": \"success\", \"content\": [{\"text\": \"Done\"}]}], \"label\": \"Read TGW routing patterns reference for route summarization guidance.\"}", + "createdAt": "2026-10-27T17:05:17.619000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "03fc9fc9-11c2-443e-81da-17388310a337", + "executionId": "b47f9985-842b-4193-9a12-1c28adcfc238", + "recordId": "cb5e34e4-d89b-4be2-b6b2-c6717737bb88", + "content": "{\"id\": \"ee8dd355-6f71-480a-9633-36d1a56aeecc\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_YC18TMU8MS7loxUQW5nbGq\", \"status\": \"success\", \"content\": [{\"text\": \"Done\"}]}], \"label\": \"Read VPN+DX redundancy reference for asymmetric return path guidance.\"}", + "createdAt": "2026-10-27T17:05:17.718000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "03fc9fc9-11c2-443e-81da-17388310a337", + "executionId": "b47f9985-842b-4193-9a12-1c28adcfc238", + "recordId": "28692b68-128f-44c7-a53f-112237f50838", + "content": "{\"id\": \"28692b68-128f-44c7-a53f-112237f50838\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_bycTByWzN7cv5GOaNWLLo2\", \"content\": \"[{'text': '\\\\r\\\\n\\\\r\\\\n# Transit Gateway Routing Patterns & Considerations\\\\r\\\\n\\\\r\\\\nThis document captures key routing mechanics, ECMP behavior, failover characteristics, and operational considerations for Transit Gateway architectures. Use this knowledge when advising on TGW route evaluation, multi-attachment designs, inter-region peering, and DX/VPN integration.\\\\r\\\\n\\\\r\\\\n## Reference Documentation\\\\r\\\\n\\\\r\\\\n- [Transit Gateway Route Evaluation Order](https://docs.aws.amazon.com/vpc/latest/tgw/how-transit-gateways-work.html#tgw-route-table-evaluation)\\\\r\\\\n- [Transit Gateway VPN ECMP](https://docs.aws.amazon.com/vpc/latest/tgw/transit-gateway-share.html)\\\\r\\\\n- [Direct Connect + Transit Gateway](https://docs.aws.amazon.com/directconnect/latest/UserGuide/direct-connect-transit-gateways.html)\\\\r\\\\n\\\\r\\\\n## Route Evaluation & Priority\\\\r\\\\n\\\\r\\\\n### Static VPN Has Higher Priority Than DX\\\\r\\\\n\\\\r\\\\n- Static VPN routes have a **higher routing priority** at the TGW than Direct Connect routes\\\\r\\\\n- Always refer to the TGW Route evaluation order when determining which attachment path will win for a given prefix\\\\r\\\\n- This can cause unexpected behavior if a customer has both a static VPN and DX advertising the same prefix \\u2014 the static VPN will be preferred\\\\r\\\\n\\\\r\\\\n## ECMP Behavior\\\\r\\\\n\\\\r\\\\n### DXGW Advertises Only ONE Best Path Per Prefix to TGW\\\\r\\\\n\\\\r\\\\nA **single DX Gateway advertises only one best path per prefix** to the TGW, regardless of how many VIFs are attached to that DXGW. The DXGW acts as a BGP aggregator between on-prem and the TGW: it runs its own internal best-path selection across its VIFs and propagates only the single winner per prefix to the TGW attachment. This is the same \\\"DXGW propagates one path per prefix\\\" behavior seen on the Cloud WAN side (DXGW \\u2192 CNE); the downstream being a TGW does not change it.\\\\r\\\\n\\\\r\\\\nConsequences:\\\\r\\\\n- **No active/active ECMP across multiple VIFs on the *same* DXGW** \\u2014 the TGW sees one path per prefix from that DXGW attachment, so redundant VIFs on one DXGW behave **active/passive** (DXGW fails over VIF-to-VIF internally; TGW only ever sees the current winner).\\\\r\\\\n- **To get multiple paths per prefix at the TGW, use multiple DXGWs** \\u2014 each DXGW is a separate TGW attachment and contributes its own one-best-path-per-prefix. TGW then has one candidate per DXGW and can ECMP across them (subject to the same-ASN requirement below).\\\\r\\\\n\\\\r\\\\n### Supported ECMP Scenarios\\\\r\\\\n\\\\r\\\\n- **VPN attachments**: TGW supports ECMP across VPN attachments. A TGW-level toggle (\\\"VPN ECMP support\\\") must be enabled.\\\\r\\\\n- **Multiple different DX-GWs**: TGW will ECMP across the per-DXGW best paths received from different DX Gateways for a matching prefix \\u2014 **but only when the AS-paths are identical, which requires the DXGWs to advertise with the SAME ASN** (see \\\"Same-ASN requirement\\\" below). Note this is ECMP across *separate DXGW attachments*, not across VIFs within one DXGW (one DXGW only ever offers a single path per prefix \\u2014 see above).\\\\r\\\\n- **Multiple TGW Connect attachments**: TGW will ECMP across matching prefixes\\\\r\\\\n\\\\r\\\\n### NOT Supported\\\\r\\\\n\\\\r\\\\n- **ECMP across different attachment types** \\u2014 TGW does NOT ECMP between, for example, a DX-GW attachment and a VPN attachment for the same prefix. Instead, the Route evaluation order determines the winner.\\\\r\\\\n- **Multipath AS-path relaxed** \\u2014 ECMP for paths with different AS-paths is NOT supported. If matching prefixes are received with different AS-paths, only the **oldest route is installed** (not load-balanced).\\\\r\\\\n- If prefixes match but ECMP conditions aren\\\\'t met, the **oldest route** is installed.\\\\r\\\\n\\\\r\\\\n### Same-ASN Requirement for TGW ECMP Across DXGWs\\\\r\\\\n\\\\r\\\\n- Because TGW does **not** support AS-path relax, ECMP across multiple DXGW attachments requires the AS-paths to be **truly identical \\u2014 same length AND same ASNs**. Equal *length* alone is not sufficient if the ASNs differ.\\\\r\\\\n- In practice this means the **DXGWs must advertise the prefix with the same ASN** for TGW to ECMP across them. Different ASNs \\u2192 no ECMP \\u2192 TGW installs the **oldest route** (single path).\\\\r\\\\n- **This differs from DX-side ECMP.** Per AWS DX docs, ECMP *across VIFs for a Region* works \\\"when prefixes have the same AS_PATH length and BGP attributes\\\" and the ASNs in the AS_PATH **do not need to match** \\u2014 that is a Region/DX-side behavior. The **same-ASN requirement here is specific to TGW ECMP across separate DXGW attachments** \\u2014 do not conflate the two selection points.\\\\r\\\\n- **[Field knowledge]** The DXGW-one-path-per-prefix propagation to TGW, and the same-ASN requirement for TGW ECMP across DXGWs, are field-proven/operational behaviors. They are not clearly stated as such on the public AWS TGW route-evaluation page \\u2014 validate in-account with `search-transit-gateway-routes` (per-prefix candidates and which attachment) and `list-virtual-interface-routes` (what each DXGW\\\\'s VIF advertised).\\\\r\\\\n\\\\r\\\\n## Direct Connect + TGW Considerations\\\\r\\\\n\\\\r\\\\n### LP Communities Not Honored Across Multiple DX-GW Attachments\\\\r\\\\n\\\\r\\\\n- Direct Connect Local Preference BGP communities (`7224:7300/7200/7100`) are **NOT honored** across multiple DX Gateway attachments on TGW\\\\r\\\\n- This differs from Cloud WAN where communities influence DXGW path selection\\\\r\\\\n- For TGW with multiple DX-GW attachments advertising the same prefix, ECMP or oldest-route logic applies \\u2014 NOT community-based preference\\\\r\\\\n\\\\r\\\\n### IPv6 Addressing Limitation\\\\r\\\\n\\\\r\\\\n- When attaching VPCs with **AWS-assigned IPv6** addresses to a TGW, these IPv6 CIDRs are discontinuous by default and **cannot be aggregated** for use with Direct Connect\\\\r\\\\n- This leads to DX prefix quota limitations\\\\r\\\\n- **Workarounds:**\\\\r\\\\n - Use Amazon-provided **contiguous** IPv6 CIDR blocks\\\\r\\\\n - Use **EC2 BYOIP** (Bring Your Own IP) for aggregatable address space\\\\r\\\\n\\\\r\\\\n### Allowed Prefixes Update Behavior (AWS \\u2192 On-Prem)\\\\r\\\\n\\\\r\\\\n- Updates (Add and Remove) to the \\\"Allowed prefixes\\\" in the TGW \\u2194 DX-GW association are **executed at the same time** on the AWS side\\\\r\\\\n- However, from an **on-premises perspective**, it appears as prefixes added first, then removed later\\\\r\\\\n- This is because on-prem learns the new route once added to the first device, and only loses the old route once withdrawn from all devices\\\\r\\\\n- Transition window: typically **1 to 6 minutes**\\\\r\\\\n\\\\r\\\\n## Route Advertisement Limits & Summarization\\\\r\\\\n\\\\r\\\\n### TGW-to-VPN 5,000 Route Advertisement Limit\\\\r\\\\n\\\\r\\\\n- The limit of **5,000 routes advertised from TGW to VPN** (toward the customer gateway) is **not adjustable**\\\\r\\\\n- The TGW route table itself supports **10,000 routes** \\u2014 it can hold more than the VPN advertisement limit\\\\r\\\\n- **Advertisement-selection order when the attachment\\\\'s advertised-route limit is exceeded** (e.g., TGW\\u2192VPN at 5,000): the routes that get advertised are chosen by (1) **static takes precedence over propagated**, then (2) among routes of the **same type** (both static or both propagated), the **least-specific** is advertised first. This is what lets a static summary survive the cap while the propagated specifics it covers are the ones dropped.\\\\r\\\\n - **[Field knowledge]** This limit-hit advertisement-selection order (static-over-propagated, then least-specific-first) is field-proven behavior and is **not stated on the public AWS TGW route-evaluation page** (which documents most-specific-first forwarding and static-over-propagated for same-CIDR). Treat as operational guidance; validate in-account.\\\\r\\\\n\\\\r\\\\n### Correct Summarization Approach\\\\r\\\\n\\\\r\\\\n**Do NOT remove VPC propagation** \\u2014 propagated routes are needed for the return path (VPN \\u2192 individual VPCs via longest prefix match within the TGW data plane).\\\\r\\\\n\\\\r\\\\n**How it works:**\\\\r\\\\n1. **Keep VPC propagation enabled** \\u2014 individual VPC routes (/16s, /24s) remain in the TGW route table for return path forwarding\\\\r\\\\n2. **Add static summary routes** covering the VPC CIDR ranges (e.g., 10.0.0.0/14 covering 10.0-10.3.x.x)\\\\r\\\\n3. **Advertisement-selection under the cap** favors the static summary \\u2014 static beats propagated, and among same-type routes the least-specific (the summary) is advertised first\\\\r\\\\n4. Once the advertised-route limit is reached, remaining propagated specifics are cut off \\u2014 but the summary already covers them, so reachability is preserved\\\\r\\\\n5. **Return traffic** from VPN to individual VPCs still works because TGW uses longest prefix match internally \\u2014 the more-specific propagated /16 routes win over the /14 summary for data plane forwarding\\\\r\\\\n\\\\r\\\\n**Key points:**\\\\r\\\\n- TGW route table holds BOTH the static summary AND the propagated specifics (up to 10,000 total)\\\\r\\\\n- The static summary consumes 1 route in the 5,000 VPN advertisement budget (instead of hundreds of individual VPC routes)\\\\r\\\\n- VPC propagation is NOT removed \\u2014 it provides the granular return path\\\\r\\\\n- This pattern leverages the difference between the advertisement limit (5,000) and the route table capacity (10,000)\\\\r\\\\n- **Static summary routes must point to a valid attachment** \\u2014 if the attachment is deleted, the route becomes blackholed, and blackholed routes are NOT propagated/advertised. Point summaries to a persistent attachment:\\\\r\\\\n - A **shared-services or firewall VPC** (recommended \\u2014 already exists, won\\\\'t be deleted)\\\\r\\\\n - A **dedicated \\\"dummy\\\" VPC** \\u2014 a minimal VPC created specifically to anchor static summary routes. Disable propagation on this VPC (no traffic originates from it), label it clearly as non-deletable, and tag it for operational protection. This ensures summary routes remain active even if workload VPCs are added/removed.\\\\r\\\\n\\\\r\\\\n### Alternatives When Summarization Isn\\\\'t Sufficient\\\\r\\\\n\\\\r\\\\n- **TGW Connect (GRE+BGP):** Each Connect peer has its own 5,000-route advertisement quota. With 4 peers per attachment, scales horizontally.\\\\r\\\\n- **Multiple VPN connections:** Segment route tables per VPN so each stays under 5,000 advertised. Adds on-prem complexity.\\\\r\\\\n- **Cloud WAN:** Native outbound route summarization via routing policies.\\\\r\\\\n\\\\r\\\\n## Failover & Convergence\\\\r\\\\n\\\\r\\\\n### On-Prem \\u2192 AWS Convergence\\\\r\\\\n\\\\r\\\\n- Route table updates for **propagated routes from BGP peers** can take **up to 60 seconds** to take effect within TGW between attachment paths\\\\r\\\\n- Customers needing faster failover should:\\\\r\\\\n - Route traffic over the **same/single attachment** (avoid cross-attachment failover dependency)\\\\r\\\\n - Consider **bypassing TGW** for high-priority workloads that cannot tolerate 60s convergence\\\\r\\\\n\\\\r\\\\n## TGW Peering\\\\r\\\\n\\\\r\\\\n### Policy-Based Routing (PBR) \\u2014 Transit Gateway Policy Tables\\\\r\\\\n\\\\r\\\\nTGW now supports **Policy-Based Routing** via policy tables, enabling forwarding decisions based on packet attributes beyond just destination IP:\\\\r\\\\n\\\\r\\\\n- Forwarding decisions can be based on: **source IP, destination IP, source port, destination port, and protocol**\\\\r\\\\n- Configured through **policy tables** \\u2014 ordered lists of rules that classify traffic and direct matching packets to a specified TGW route table\\\\r\\\\n- A policy table is associated with a TGW attachment \\u2014 it **replaces** the standard route table on that attachment (attachment can have either a policy table OR a route table, not both)\\\\r\\\\n- Available in all regions at no additional charge beyond standard TGW fees\\\\r\\\\n\\\\r\\\\n**Use cases:**\\\\r\\\\n- Steer traffic selectively to security appliances or inspection services based on source, port, or protocol\\\\r\\\\n- Route traffic from different applications or user populations over different network paths\\\\r\\\\n- Isolate network traffic into separate routing domains for security segmentation\\\\r\\\\n\\\\r\\\\n**Reference:** [Transit Gateway Policy Tables](https://docs.aws.amazon.com/vpc/latest/tgw/tgw-policy-tables.html)\\\\r\\\\n\\\\r\\\\n### Inter-Region Peering\\\\r\\\\n\\\\r\\\\n- TGW Inter-Region Peering (across regions) is **not yet supported in all regions**\\\\r\\\\n- **BGP dynamic route propagation across TGW peering is not yet supported** \\u2014 routes must be statically configured on peered TGW route tables\\\\r\\\\n- This limitation does NOT apply to Cloud WAN dynamic peering\\\\r\\\\n\\\\r\\\\n### Intra-Region Peering\\\\r\\\\n\\\\r\\\\n- TGW Intra-Region Peering (within a region) **is supported**\\\\r\\\\n\\\\r\\\\n### Alternative to Multiple TGWs\\\\r\\\\n\\\\r\\\\n- Instead of deploying multiple TGWs, customers can use **Route Tables within a single TGW** to segregate traffic between different domains/environments\\\\r\\\\n- This simplifies management and avoids peering complexity\\\\r\\\\n\\\\r\\\\n## Multi-Region Traffic & AZ Awareness\\\\r\\\\n\\\\r\\\\n### Gray Failure Risk with Static TGW Peering\\\\r\\\\n\\\\r\\\\n- Traffic between peered TGWs **does not have AZ awareness**\\\\r\\\\n- When traffic arrives from a remote region over **static TGW peering**, it could first land in an AZ different from the destination AZ before being routed to its final destination\\\\r\\\\n- During **gray failures** (partial AZ degradation): packet loss can be observed even though the destination AZ is not impacted \\u2014 because traffic may transit the degraded AZ\\\\r\\\\n- **This limitation only applies to static TGW Peerings**\\\\r\\\\n- It will NOT exist once BGP dynamic route propagation across TGW peering becomes available\\\\r\\\\n- It does NOT apply to **Cloud WAN dynamic peering** (which has AZ awareness)\\\\r\\\\n'}]\", \"status\": \"success\", \"type\": \"tool_result\"}, {\"id\": \"tooluse_YC18TMU8MS7loxUQW5nbGq\", \"content\": \"[{'text': '\\\\r\\\\n\\\\r\\\\n# VPN and Direct Connect Redundancy Patterns\\\\r\\\\n\\\\r\\\\nThis document captures route preference behavior for VPN and DX redundancy designs with VGW. Use this knowledge when advising on active/active, active/passive, and failover architectures.\\\\r\\\\n\\\\r\\\\n## Reference Documentation\\\\r\\\\n\\\\r\\\\n- [AWS Site-to-Site VPN routing](https://docs.aws.amazon.com/vpn/latest/s2svpn/VPNRoutingTypes.html)\\\\r\\\\n- [Direct Connect routing policies and BGP communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html)\\\\r\\\\n- [DX Resiliency Recommendations](https://aws.amazon.com/directconnect/resiliency-recommendation/)\\\\r\\\\n\\\\r\\\\n## Assumptions\\\\r\\\\n\\\\r\\\\n- Customer\\\\'s network does not overlap with VPC CIDR\\\\r\\\\n- Customer Gateway (CGW) device is configured to handle asymmetric routing\\\\r\\\\n- Static VPN tunnels are Route-Based only (not Policy-Based)\\\\r\\\\n- Customer advertises the same on-prem routes through all redundant connections (DX and VPN)\\\\r\\\\n\\\\r\\\\n## Key Behavior: VGW Does NOT Support ECMP\\\\r\\\\n\\\\r\\\\nVGW does **not** support Equal-Cost Multi-Path routing (ECMP). Return traffic from VPC to on-prem always takes a **single tunnel/path** \\u2014 the \\\"Preferred Tunnel.\\\" This is a fundamental constraint that affects all redundancy designs.\\\\r\\\\n\\\\r\\\\n---\\\\r\\\\n\\\\r\\\\n## Part 1: Redundancy Only on AWS End\\\\r\\\\n\\\\r\\\\n### VPN Connection \\u2014 2 Active Tunnels (Static)\\\\r\\\\n\\\\r\\\\n- Customers can have static route-based tunnels in Active/Active mode\\\\r\\\\n- **Customer \\u2192 VPC:** Can load balance (equal route metrics) or prefer one tunnel (unequal metrics)\\\\r\\\\n- **VPC \\u2192 Customer:** Always takes a single tunnel (Preferred Tunnel) \\u2014 VGW does not support ECMP\\\\r\\\\n\\\\r\\\\n- Customers can have BGP tunnels in Active/Active mode\\\\r\\\\n- **Customer \\u2192 VPC:** Load balance (equal LP) or prefer one (unequal LP)\\\\r\\\\n- **VPC \\u2192 Customer:** Always a single tunnel. Customer can influence by:\\\\r\\\\n - Advertising **lower AS-path** to VGW over the desired tunnel\\\\r\\\\n - Advertising **lower MED** to VGW over the desired tunnel\\\\r\\\\n\\\\r\\\\n### DX Connection with Backup VPN\\\\r\\\\n\\\\r\\\\n- DX is **always preferred over VPN** for return traffic to the same on-prem subnet\\\\r\\\\n- Customer must ensure on-prem traffic is sent over DX when both are UP\\\\r\\\\n- Attempting to prefer VPN over DX will cause asymmetric routing (VGW always returns via DX)\\\\r\\\\n\\\\r\\\\n### 2 DX Connections (Same VGW)\\\\r\\\\n\\\\r\\\\n- 2 private VIFs to the same VGW over 2 DX connections\\\\r\\\\n- **Active/Active:** Equal LP + AS-path on CGW \\u2192 return traffic is flow-hashed across both DX connections\\\\r\\\\n- **Active/Passive:** Unequal LP + AS-path \\u2192 customer-to-VPC uses higher LP; VPC-to-customer uses lower AS-path\\\\r\\\\n\\\\r\\\\n**\\u26a0\\ufe0f ECMP only works when both DX connections terminate at the SAME DX location:**\\\\r\\\\n- Same location (e.g., both at Equinix DC, Ashburn) \\u2192 flow-hashed (ECMP)\\\\r\\\\n- Different locations in same region (e.g., Equinix DC Ashburn + CoreSite NYC) \\u2192 **single path only**\\\\r\\\\n\\\\r\\\\n---\\\\r\\\\n\\\\r\\\\n## Part 2: Redundancy on AWS and Customer Ends\\\\r\\\\n\\\\r\\\\n### 2 VPN Connections from 2 CGW Devices (Recommended Design)\\\\r\\\\n\\\\r\\\\n- 2 BGP VPN connections (4 active tunnels) to same VGW from different on-prem CGW devices\\\\r\\\\n- **Recommended:** Use higher LP + lower AS-path on preferred VPN connection to avoid routing issues\\\\r\\\\n- **Customer \\u2192 VPC:** Traffic uses tunnel(s) with higher LP\\\\r\\\\n- **VPC \\u2192 Customer:** Single tunnel (Preferred Tunnel). Customer influences via lower AS-path on desired connection.\\\\r\\\\n\\\\r\\\\n### DX + Backup VPN from 2 CGW Devices\\\\r\\\\n\\\\r\\\\n- DX and VPN from different on-prem CGW devices\\\\r\\\\n- When both UP: customer must ensure on-prem traffic goes over DX only\\\\r\\\\n- **VPC \\u2192 Customer:** Always sent over DX (DX always preferred over VPN by VGW)\\\\r\\\\n- Design works correctly as active/passive (DX active, VPN passive)\\\\r\\\\n\\\\r\\\\n### 2 DX Connections from 2 CGW Devices\\\\r\\\\n\\\\r\\\\n- 2 DX connections from different on-prem CGW devices, 2 private VIFs to same VGW\\\\r\\\\n- **Active/Active:** Equal LP + AS-path \\u2192 flow-hashed return traffic\\\\r\\\\n- **Active/Passive:** Unequal LP + AS-path \\u2192 deterministic path preference\\\\r\\\\n\\\\r\\\\n**Same ECMP caveat applies:** Flow-hashing only works when both DX connections are at the same DX location. Different locations \\u2192 single path selected.\\\\r\\\\n\\\\r\\\\n---\\\\r\\\\n\\\\r\\\\n## Part 3: Designs That Cause Routing Issues\\\\r\\\\n\\\\r\\\\n### \\u274c 2 Static VPN Connections from 2 CGW Devices\\\\r\\\\n\\\\r\\\\n- VPC \\u2192 Customer always traverses single tunnel (Preferred Tunnel)\\\\r\\\\n- If customer sends traffic on CGW 2, return may come back on CGW 1 \\u2192 **asymmetric routing**\\\\r\\\\n- Only works if CGW devices have synchronization (ECMP) and accept asymmetric traffic\\\\r\\\\n\\\\r\\\\n### \\u274c 2 BGP VPN Connections with Same LP and AS-Path from 2 CGW Devices\\\\r\\\\n\\\\r\\\\n- Same problem as static: VPC \\u2192 Customer takes single tunnel, may not match inbound path\\\\r\\\\n- **Fix:** Use unequal LP + AS-path to prefer one VPN connection over the other\\\\r\\\\n- Or ensure CGW devices handle asymmetric routing\\\\r\\\\n\\\\r\\\\n### \\u274c DX + VPN \\u2014 Attempting to Prefer VPN Over DX\\\\r\\\\n\\\\r\\\\n- If customer tries to prefer VPN over DX, return traffic (VPC \\u2192 on-prem) STILL goes over DX\\\\r\\\\n- VGW always prefers DX over VPN regardless of BGP attributes\\\\r\\\\n- **Result:** Asymmetric routing \\u2014 customer sends via VPN, AWS returns via DX\\\\r\\\\n- This is true whether using same CGW device or different CGW devices\\\\r\\\\n- **Fix:** Advertise more-specific routes on VPN \\u2014 longest prefix match wins before attachment-type preference, so a /24 via VPN will be preferred over a /16 via DX for matching traffic\\\\r\\\\n\\\\r\\\\n### \\u274c 2 VPN Connections from 2 CGW Devices with Different ASNs\\\\r\\\\n\\\\r\\\\n- 2 BGP VPN connections from 2 data centers (different ASNs) advertising same networks\\\\r\\\\n- VPC \\u2192 Customer: single tunnel, may not match inbound path \\u2192 asymmetric\\\\r\\\\n- **Fix:** Use unequal AS-path values to prefer one VPN connection\\\\r\\\\n- Cannot achieve ECMP across different ASNs\\\\r\\\\n\\\\r\\\\n### \\u274c DX + VPN from 2 CGW Devices with Different ASNs\\\\r\\\\n\\\\r\\\\n- If customer sends traffic over VPN, return goes over DX (DX always wins)\\\\r\\\\n- **Must avoid using same networks in multiple datacenters** to prevent routing issues\\\\r\\\\n- Or accept that DX is always the return path regardless of inbound path\\\\r\\\\n\\\\r\\\\n---\\\\r\\\\n\\\\r\\\\n## Summary: Route Preference Rules at VGW\\\\r\\\\n\\\\r\\\\n| Traffic Direction | Rule |\\\\r\\\\n|---|---|\\\\r\\\\n| VPC \\u2192 On-prem (same prefix) | DX **always** preferred over VPN |\\\\r\\\\n| VPC \\u2192 On-prem (VPN only) | Single tunnel \\u2014 lowest AS-path or MED, no ECMP |\\\\r\\\\n| VPC \\u2192 On-prem (2 DX, same location) | Flow-hashed (ECMP) |\\\\r\\\\n| VPC \\u2192 On-prem (2 DX, different locations) | Single path |\\\\r\\\\n| On-prem \\u2192 VPC (VPN) | Customer controls via LP (higher wins) |\\\\r\\\\n| On-prem \\u2192 VPC (DX) | Customer controls via LP (higher wins) |\\\\r\\\\n| On-prem \\u2192 VPC (DX + VPN both UP) | Customer must direct to DX |\\\\r\\\\n\\\\r\\\\n## Key Takeaways for Customer Guidance\\\\r\\\\n\\\\r\\\\n1. **VGW does not support ECMP** \\u2014 return traffic always takes one path\\\\r\\\\n2. **DX always wins over VPN** at the VGW \\u2014 cannot prefer VPN for return traffic\\\\r\\\\n3. **Same DX location required for flow-hashing** \\u2014 different locations in same region get single-path selection\\\\r\\\\n4. **Use unequal LP + AS-path** for active/passive designs to avoid asymmetric routing\\\\r\\\\n5. **Asymmetric routing is the primary failure mode** \\u2014 designs with equal attributes from multiple CGW devices without synchronization will cause issues\\\\r\\\\n6. **MED can influence VGW path selection** \\u2014 documented at [VPN route priority](https://docs.aws.amazon.com/vpn/latest/s2svpn/vpn-route-priority.html)\\\\r\\\\n'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:27.928000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "03fc9fc9-11c2-443e-81da-17388310a337", + "executionId": "b47f9985-842b-4193-9a12-1c28adcfc238", + "recordId": "7d2420cb-b200-4dbb-b23f-b0bb71596248", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 5.0, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:28.013000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "03fc9fc9-11c2-443e-81da-17388310a337", + "executionId": "b47f9985-842b-4193-9a12-1c28adcfc238", + "recordId": "81dacff4-ee45-4dff-833e-aa813d69b7fe", + "content": "{\"id\": \"81dacff4-ee45-4dff-833e-aa813d69b7fe\", \"role\": \"assistant\", \"content\": [{\"text\": \"## The problem\\n\\nTGW\\u2192VPN (customer gateway) advertisement is capped at **5,000 routes \\u2014 not adjustable**. The TGW route table itself holds up to 10,000 routes, so there's headroom there; the cap is specifically on what gets *advertised* to the VPN side.\\n\\n## The fix: summarize the advertisement, not the route table\\n\\n**Do not disable VPC propagation.** The individual propagated VPC routes (/16s, /24s, etc.) are what the TGW uses for the *return path* \\u2014 VPN \\u2192 individual VPCs is resolved by longest-prefix-match in the TGW data plane. Remove them and inbound-to-VPC forwarding breaks.\\n\\nInstead:\\n\\n1. **Keep VPC propagation enabled** \\u2014 all the granular /16s/24s stay in the TGW route table.\\n2. **Add static summary route(s)** covering the VPC CIDR ranges (e.g., `10.0.0.0/14` over `10.0\\u201310.3.x.x`).\\n3. When the attachment hits its advertised-route limit, the selection order is: **static beats propagated**, then among same-type routes **least-specific advertised first**. That means the static summary is advertised and survives the cap, while the propagated specifics it already covers are the ones trimmed off the advertisement \\u2014 but they're still sitting in the TGW route table doing the forwarding work.\\n4. **Point the static summary at a persistent attachment** \\u2014 a shared-services/firewall VPC, or a dedicated non-deletable \\\"dummy\\\" VPC with propagation disabled. If the attachment is ever deleted, the summary route blackholes (and blackholed routes aren't advertised), silently breaking reachability.\\n\\nSo: outbound advertisement shrinks to one summary route (barely touching the 5,000 budget), while the return path keeps working because forwarding still consults the specific propagated routes, not the summary.\\n\\n**If 5,000 still isn't enough even after summarizing** (e.g., many non-contiguous blocks that can't be covered by one or two summaries): segment across multiple VPN connections (separate route tables, each under 5,000), or move to TGW Connect (GRE+BGP), where each Connect peer carries its own 5,000-route budget and scales horizontally with more peers.\\n\\nWorth validating in-account afterward with read-only checks (`search-transit-gateway-routes` for the per-prefix winner, and confirming the VPN's actually-advertised set) since the limit-hit selection order isn't fully documented on the public TGW page \\u2014 it's field-verified behavior. Want me to pull those checks for a specific TGW/attachment?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:28.096000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "03fc9fc9-11c2-443e-81da-17388310a337", + "executionId": "b47f9985-842b-4193-9a12-1c28adcfc238", + "recordId": "52facb0a-4bb0-4c24-b352-710535647dd1", + "content": "{\"id\": \"bb383b5d-3dc3-4b09-9a30-ba9d47eb046b\", \"role\": \"assistant\", \"content\": [{\"type\": \"text\", \"text\": \"## The problem\\n\\nTGW\\u2192VPN (customer gateway) advertisement is capped at **5,000 routes \\u2014 not adjustable**. The TGW route table itself holds up to 10,000 routes, so there's headroom there; the cap is specifically on what gets *advertised* to the VPN side.\\n\\n## The fix: summarize the advertisement, not the route table\\n\\n**Do not disable VPC propagation.** The individual propagated VPC routes (/16s, /24s, etc.) are what the TGW uses for the *return path* \\u2014 VPN \\u2192 individual VPCs is resolved by longest-prefix-match in the TGW data plane. Remove them and inbound-to-VPC forwarding breaks.\\n\\nInstead:\\n\\n1. **Keep VPC propagation enabled** \\u2014 all the granular /16s/24s stay in the TGW route table.\\n2. **Add static summary route(s)** covering the VPC CIDR ranges (e.g., `10.0.0.0/14` over `10.0\\u201310.3.x.x`).\\n3. When the attachment hits its advertised-route limit, the selection order is: **static beats propagated**, then among same-type routes **least-specific advertised first**. That means the static summary is advertised and survives the cap, while the propagated specifics it already covers are the ones trimmed off the advertisement \\u2014 but they're still sitting in the TGW route table doing the forwarding work.\\n4. **Point the static summary at a persistent attachment** \\u2014 a shared-services/firewall VPC, or a dedicated non-deletable \\\"dummy\\\" VPC with propagation disabled. If the attachment is ever deleted, the summary route blackholes (and blackholed routes aren't advertised), silently breaking reachability.\\n\\nSo: outbound advertisement shrinks to one summary route (barely touching the 5,000 budget), while the return path keeps working because forwarding still consults the specific propagated routes, not the summary.\\n\\n**If 5,000 still isn't enough even after summarizing** (e.g., many non-contiguous blocks that can't be covered by one or two summaries): segment across multiple VPN connections (separate route tables, each under 5,000), or move to TGW Connect (GRE+BGP), where each Connect peer carries its own 5,000-route budget and scales horizontally with more peers.\\n\\nWorth validating in-account afterward with read-only checks (`search-transit-gateway-routes` for the per-prefix winner, and confirming the VPN's actually-advertised set) since the limit-hit selection order isn't fully documented on the public TGW page \\u2014 it's field-verified behavior. Want me to pull those checks for a specific TGW/attachment?\"}]}", + "createdAt": "2026-10-27T17:05:28.190000-05:00", + "recordType": "final_response" + } +] \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vpn-summarization/without_skill/functional-tests-results.json b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vpn-summarization/without_skill/functional-tests-results.json new file mode 100644 index 00000000..3de05c93 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vpn-summarization/without_skill/functional-tests-results.json @@ -0,0 +1,70 @@ +{ + "version": "v1", + "iteration": 1, + "eval_id": "tgw-vpn-summarization", + "run_type": "without_skill", + "success": true, + "error": null, + "understanding_agent_space_skill": null, + "tests": { + "trigger": null, + "expected_output": { + "result": "failed", + "reasoning": "The expected output describes a specific technical mechanism: keep VPC propagation ENABLED (because specifics are needed for longest-prefix-match return forwarding), add a static summary route in the VPN-associated TGW route table, and rely on the fact that under the advertised-route cap, static routes beat propagated routes and the least-specific route is advertised first \u2014 so the summary survives and gets advertised while specifics remain for internal LPM routing. The summary should be anchored to a persistent attachment.\n\nThe agent's response gets this fundamentally backwards in a key respect: it explicitly instructs the user to SUPPRESS/disable propagation of the individual VPC-propagated routes once the summary is in place ('make sure the individual VPC-propagated routes it replaces aren't still being propagated into that same route table'). This is the opposite of the expected guidance, which says to keep propagation enabled because the specifics are needed for correct longest-prefix-match return forwarding inside the TGW route table. The agent's approach could actually break the return path internally (the very thing the user asked to avoid), which contradicts the expected output's core technical insight about why specifics must stay enabled.\n\nWhile the agent correctly identifies asymmetric routing as a risk and gives good general advice about CIDR planning, BGP attribute consistency, and validation, it misses the precise mechanism described in the expected answer (static-beats-propagated advertisement preference under the cap) and gives an instruction that directly contradicts the recommended approach (disable vs. keep propagation enabled). This is a substantive technical discrepancy, not just a wording difference.", + "evidence": "\"After adding the static aggregate route in the TGW route table associated with the VPN attachment, make sure the individual VPC-propagated routes it replaces aren't still being propagated into that same route table \u2014 leftover specifics will both re-inflate your advertised count and... can silently override the summary\"", + "confidence": "high" + }, + "assertions": { + "assertion_results": [ + { + "text": "keeps VPC route propagation enabled for the return path", + "evaluator": "llm", + "passed": false, + "evidence": "\"Suppress the propagated specifics once the summary is in place... make sure the individual VPC-propagated routes it replaces aren't still being propagated into that same route table\"", + "reasoning": "The agent explicitly recommends suppressing/removing propagation of the individual VPC routes once the summary is in place, which is the opposite of keeping propagation enabled.", + "confidence": "high" + }, + { + "text": "adds a static summary route in the TGW route table associated with the VPN attachment", + "evaluator": "llm", + "passed": true, + "evidence": "\"After adding the static aggregate route in the TGW route table associated with the VPN attachment...\" and \"Once the static aggregate is in place, confirm end-to-end reachability...\"", + "reasoning": "The agent explicitly instructs adding a static aggregate/summary route in the TGW route table associated with the VPN attachment.", + "confidence": "high" + }, + { + "text": "explains static-over-propagated and least-specific selection under the advertisement cap", + "evaluator": "llm", + "passed": false, + "evidence": "\"leftover specifics will both re-inflate your advertised count and, because of longest-prefix-match, can silently override the summary for a subset of addresses and send them to the wrong next hop.\" - this touches on longest-prefix-match but does not explicitly state static routes override propagated routes regardless of prefix length (TGW's actual route selection order), nor explain least-specific selection under the cap in detail.", + "reasoning": "The agent discusses suppressing specifics to avoid longest-prefix-match issues and re-inflating counts, but does not explicitly explain that static routes take precedence over propagated routes in TGW route tables, nor does it explain how least-specific route selection interacts with the advertisement cap in a structured way. It mentions longest-prefix-match briefly but not the static-over-propagated precedence rule or least-specific selection under the cap explicitly.", + "confidence": "medium" + }, + { + "text": "does not contain 'error'", + "evaluator": "llm", + "passed": true, + "evidence": "No occurrence of the word 'error' found in the response text.", + "reasoning": "The output does not contain the literal word 'error' anywhere in the text.", + "confidence": "high" + } + ], + "summary": { + "passed": 2, + "failed": 2, + "errored": 0, + "low_confidence": 0, + "total": 4, + "pass_rate": 0.5 + } + } + }, + "metrics": { + "runtime": "39s", + "cost": "$0.33", + "context_window": { + "utilization": "4.0%", + "compaction_count": 0 + } + } +} \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vpn-summarization/without_skill/outputs/journal_records.json b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vpn-summarization/without_skill/outputs/journal_records.json new file mode 100644 index 00000000..1220c70b --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vpn-summarization/without_skill/outputs/journal_records.json @@ -0,0 +1,106 @@ +[ + { + "agentSpaceId": "b05bab6b-8817-4787-885d-468f7c2286a4", + "executionId": "9b61aafd-8bf3-468b-8d36-77359164f8d1", + "recordId": "4452e794-9a87-4519-ba80-04e7d3790469", + "content": "{\"id\": \"4452e794-9a87-4519-ba80-04e7d3790469\", \"role\": \"user\", \"content\": [{\"text\": \"A customer is at the TGW-to-VPN advertised-route limit. How do they summarize without breaking the return path?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:03.037000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "b05bab6b-8817-4787-885d-468f7c2286a4", + "executionId": "9b61aafd-8bf3-468b-8d36-77359164f8d1", + "recordId": "9558fc32-f6ab-4c1c-b69d-c0bc1e3b05be", + "content": "{\"id\": \"b4135ff4-3599-4030-941b-434f1e47446a\", \"role\": \"assistant\", \"content\": [{\"type\": \"chat_title\", \"title\": \"TGW-to-VPN advertised route limit summary\"}]}", + "createdAt": "2026-10-27T17:05:04.774000-05:00", + "recordType": "chat_title" + }, + { + "agentSpaceId": "b05bab6b-8817-4787-885d-468f7c2286a4", + "executionId": "9b61aafd-8bf3-468b-8d36-77359164f8d1", + "recordId": "53287543-b307-4db3-964d-5a9e9836602d", + "content": "{\"id\": \"71bc4b8a-bf36-4e1e-8c95-d6b0a4ce4547\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This looks like a general AWS networking question about Transit Gateway VPN route limits and summarization strategy, without any specific account context tied to the user's infrastructure, so I'm weighing whether verify_aws_claim is even needed here.\\n\\nI'm trying to recall the specific quota: the number of routes TGW can advertise over BGP to an on-premises router via Site-to-Site VPN. Since this is a factual, changeable AWS limit, I should verify it with the tool rather than rely on memory.\\n\\nSince there's no specific account or TGW ID mentioned, this reads as a general guidance request about route summarization strategy to avoid breaking the return path when that advertised route limit is hit, so I'll verify the quota first before giving best-practice advice.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:15.795000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "b05bab6b-8817-4787-885d-468f7c2286a4", + "executionId": "9b61aafd-8bf3-468b-8d36-77359164f8d1", + "recordId": "2d2e91e5-045c-4472-a477-c1707c61b266", + "content": "{\"id\": \"2d2e91e5-045c-4472-a477-c1707c61b266\", \"role\": \"assistant\", \"content\": [{\"id\": \"tooluse_8NWAQ9FXoHijPdMgVLGukK\", \"tool_name\": \"verify_aws_claim\", \"input\": {\"claim\": \"Transit Gateway VPN attachment maximum number of routes advertised via BGP\", \"topics\": [\"reference_documentation\"]}, \"type\": \"tool_use\", \"display_intent\": \"Verify AWS Transit Gateway VPN advertised route limit via BGP documentation.\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:15.795000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "b05bab6b-8817-4787-885d-468f7c2286a4", + "executionId": "9b61aafd-8bf3-468b-8d36-77359164f8d1", + "recordId": "169b2d05-7c8b-408e-8aa4-ac77f8b0acd6", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 3.8, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:15.893000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "b05bab6b-8817-4787-885d-468f7c2286a4", + "executionId": "9b61aafd-8bf3-468b-8d36-77359164f8d1", + "recordId": "1683c3aa-0fb5-4210-add8-1af3d367170f", + "content": "{\"id\": \"e31997ae-b46f-427a-80d4-b89b00926c4b\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"8759\"}]}", + "createdAt": "2026-10-27T17:05:15.905000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "b05bab6b-8817-4787-885d-468f7c2286a4", + "executionId": "9b61aafd-8bf3-468b-8d36-77359164f8d1", + "recordId": "0ba865f2-6fe7-4c80-b7aa-377426026b16", + "content": "{\"id\": \"383cb9d8-cb8f-49bb-8549-93cab7069a39\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_8NWAQ9FXoHijPdMgVLGukK\", \"status\": \"success\", \"content\": [{\"text\": \"{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"AWS Site-to-Site VPN quotas\\\",\\\"context\\\":\\\"## Routes\\\\n\\\\nAdvertised route sources include VPC routes, other VPN routes, and routes from Direct Connect\\\\nvirtual interfaces. Advertised routes come from the route table that's associated with the\\\\nVPN attachment.\\\\n\\\\n**Note:**\\\\n\\\\nIf you are using a virtual private gateway and route propagation is enabled on your VPC route table, both dynamic and static routes will automatically be added for your VPN connection, up to the limit of the VPC's route table. See Amazon VPC quotas in the *Amazon VPC User Guide* for further details.\\\\n\\\\n| Name | Default | Adjustable |\\\\n| --- | --- | --- |\\\\n| Dynamic routes advertised from a customer gateway device to a Site-to-Site VPN connection on a virtual private gateway | 100 | No |\\\\n| Routes advertised from a Site-to-Site VPN connection on a virtual private gateway to a customer gateway device | 1,000 | No |\\\\n| Dynamic routes advertised from a customer gateway device to a Site-to-Site VPN connection on a transit gateway | 1,000 | No |\\\\n| Routes advertised from a Site-to-Site VPN connection on a transit gateway to a customer gateway device | 5,000 | No |\\\\n| Static routes from a customer gateway device to a Site-to-Site VPN connection on a virtual private gateway | 100 | No |\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/vpn/latest/s2svpn/vpn-limits.html\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"AWS Transit Gateway Quotas\\\",\\\"context\\\":\\\"## Routing\\\\n\\\\n| Name | Default | Adjustable |\\\\n| --- | --- | --- |\\\\n| Transit gateway route tables per transit gateway | 20 | Yes |\\\\n| Transit gateway policy tables per transit gateway | 20 | No |\\\\n| Policy table entries per transit gateway (customer-managed entries, across all policy tables on a single transit gateway) | 200 | Yes |\\\\n| Total combined routes (dynamic and static) across all route tables for a single transit gateway | 10,000 | Contact your Solutions Architect (SA) or Technical Account Manager (TAM) for further assistance. |\\\\n| Dynamic routes advertised from a virtual router appliance to a Connect peer | 1,000 | Contact your Solutions Architect (SA) or Technical Account Manager (TAM) for further assistance. |\\\\n| Routes advertised from a Connect peer on a transit gateway to a virtual router appliance | 5,000 | No |\\\\n| Static routes for a prefix to a single attachment | 1 | No |\\\\n\\\\nAdvertised routes come from the route table that's associated with the Connect attachment\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/vpc/latest/tgw/transit-gateway-quotas.html\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"AWS Site-to-Site VPN Concentrators\\\",\\\"context\\\":\\\"## Routing\\\\n\\\\nSite-to-Site VPN Concentrators support BGP (Border Gateway Protocol) routing only. Static routing is not supported.\\\\n\\\\nAll customer gateways connected to the Site-to-Site VPNConcentrator use the same Site-to-Site VPN\\\\nConcentrator attachment to the transit gateway for routing. Each site connecting to the Site-to-Site VPN\\\\nConcentrator can send a maximum of 5,000 routes from the transit gateway to a customer gateway and 1,000\\\\nroutes from the customer gateway to the transit gateway\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/vpn/latest/s2svpn/vpn-concentrator.html\\\"}]}}\"}]}], \"label\": \"Verify AWS Transit Gateway VPN advertised route limit via BGP documentation.\"}", + "createdAt": "2026-10-27T17:05:17.148000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "b05bab6b-8817-4787-885d-468f7c2286a4", + "executionId": "9b61aafd-8bf3-468b-8d36-77359164f8d1", + "recordId": "7013b25b-361c-4b57-a52f-98cd46dccfd1", + "content": "{\"id\": \"72997761-8899-458f-a464-16cd275314cf\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"So this confirms the fixed limits: 5,000 routes TGW VPN advertises to the customer gateway, and 1,000 routes from the customer gateway to the TGW VPN side. The customer's issue is likely tied to one of these advertised route limits, probably the TGW-to-VPN side given the context.\\n\\nNow I'm thinking through how to advise on route summarization without breaking the return path - a classic routing concern. The key risk is that summarizing CIDRs at the TGW route table level could create asymmetric routing or blackholing if a sub-prefix within the summary actually belongs to a different attachment than the one the summary route points to, so any summarized block needs to map cleanly to the correct attachment(s) on the AWS side.\\n\\nThe best practice is hierarchical IPAM-style CIDR allocation ahead of time so VPCs share contiguous blocks that can be cleanly summarized, then using TGW static routes to advertise one aggregate route per attachment while suppressing dynamic propagation of the individual VPC routes toward the CGW, so the summary route maps exactly to the intended attachment or inspection appliance.\\n\\nFor symmetry, the aggregation point handling the summary route needs full specific routing knowledge of every real subnet it covers, so it doesn't drop packets for valid addresses\\u2014blackholing only truly unused space within the block. This also helps on the customer gateway side, since summarization cuts down the route count advertised outward toward the 5,000-route VPN limit.\\n\\nI need to verify the on-prem router actually uses the summary route for return traffic, and watch out for a more specific but conflicting route via a different path that could cause asymmetric routing\\u2014which breaks stateful firewalls or NAT. If a legitimate secondary path does exist, I should align BGP attributes like AS-PATH, local-pref, or MED (or use route-maps/communities) to keep both specific and summary routes consistently preferred on matching legs.\\n\\nFor redundant tunnels or CGWs, I need the summary advertised uniformly across all paths so failover doesn't break return routing, then confirm on the AWS side that the TGW route table only has the aggregate static route with no leftover specific propagated routes that could override it via longest-prefix-match. Finally, I'd validate the full round trip using traceroute, VPC Reachability Analyzer, or TGW Route Analyzer.\\n\\nSince this is a conceptual best-practices question with no specific account or customer context, I can answer directly without needing tool calls -- I'll give concise, practical guidance in bullet form.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:45.594000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "b05bab6b-8817-4787-885d-468f7c2286a4", + "executionId": "9b61aafd-8bf3-468b-8d36-77359164f8d1", + "recordId": "98098502-e9ec-4ef0-9657-b8bae10f9206", + "content": "{\"id\": \"d3b7e798-2268-4343-b5cc-61eff099d08f\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"24752\"}]}", + "createdAt": "2026-10-27T17:05:45.681000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "b05bab6b-8817-4787-885d-468f7c2286a4", + "executionId": "9b61aafd-8bf3-468b-8d36-77359164f8d1", + "recordId": "dc1d943a-1bf1-4490-a7ae-d2e840339b56", + "content": "{\"id\": \"dc1d943a-1bf1-4490-a7ae-d2e840339b56\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_8NWAQ9FXoHijPdMgVLGukK\", \"content\": \"[{'text': '{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"AWS Site-to-Site VPN quotas\\\",\\\"context\\\":\\\"## Routes\\\\\\\\n\\\\\\\\nAdvertised route sources include VPC routes, other VPN routes, and routes from Direct Connect\\\\\\\\nvirtual interfaces. Advertised routes come from the route table that\\\\'s associated with the\\\\\\\\nVPN attachment.\\\\\\\\n\\\\\\\\n**Note:**\\\\\\\\n\\\\\\\\nIf you are using a virtual private gateway and route propagation is enabled on your VPC route table, both dynamic and static routes will automatically be added for your VPN connection, up to the limit of the VPC\\\\'s route table. See Amazon VPC quotas in the *Amazon VPC User Guide* for further details.\\\\\\\\n\\\\\\\\n| Name | Default | Adjustable |\\\\\\\\n| --- | --- | --- |\\\\\\\\n| Dynamic routes advertised from a customer gateway device to a Site-to-Site VPN connection on a virtual private gateway | 100 | No |\\\\\\\\n| Routes advertised from a Site-to-Site VPN connection on a virtual private gateway to a customer gateway device | 1,000 | No |\\\\\\\\n| Dynamic routes advertised from a customer gateway device to a Site-to-Site VPN connection on a transit gateway | 1,000 | No |\\\\\\\\n| Routes advertised from a Site-to-Site VPN connection on a transit gateway to a customer gateway device | 5,000 | No |\\\\\\\\n| Static routes from a customer gateway device to a Site-to-Site VPN connection on a virtual private gateway | 100 | No |\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/vpn/latest/s2svpn/vpn-limits.html\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"AWS Transit Gateway Quotas\\\",\\\"context\\\":\\\"## Routing\\\\\\\\n\\\\\\\\n| Name | Default | Adjustable |\\\\\\\\n| --- | --- | --- |\\\\\\\\n| Transit gateway route tables per transit gateway | 20 | Yes |\\\\\\\\n| Transit gateway policy tables per transit gateway | 20 | No |\\\\\\\\n| Policy table entries per transit gateway (customer-managed entries, across all policy tables on a single transit gateway) | 200 | Yes |\\\\\\\\n| Total combined routes (dynamic and static) across all route tables for a single transit gateway | 10,000 | Contact your Solutions Architect (SA) or Technical Account Manager (TAM) for further assistance. |\\\\\\\\n| Dynamic routes advertised from a virtual router appliance to a Connect peer | 1,000 | Contact your Solutions Architect (SA) or Technical Account Manager (TAM) for further assistance. |\\\\\\\\n| Routes advertised from a Connect peer on a transit gateway to a virtual router appliance | 5,000 | No |\\\\\\\\n| Static routes for a prefix to a single attachment | 1 | No |\\\\\\\\n\\\\\\\\nAdvertised routes come from the route table that\\\\'s associated with the Connect attachment\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/vpc/latest/tgw/transit-gateway-quotas.html\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"AWS Site-to-Site VPN Concentrators\\\",\\\"context\\\":\\\"## Routing\\\\\\\\n\\\\\\\\nSite-to-Site VPN Concentrators support BGP (Border Gateway Protocol) routing only. Static routing is not supported.\\\\\\\\n\\\\\\\\nAll customer gateways connected to the Site-to-Site VPNConcentrator use the same Site-to-Site VPN\\\\\\\\nConcentrator attachment to the transit gateway for routing. Each site connecting to the Site-to-Site VPN\\\\\\\\nConcentrator can send a maximum of 5,000 routes from the transit gateway to a customer gateway and 1,000\\\\\\\\nroutes from the customer gateway to the transit gateway\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/vpn/latest/s2svpn/vpn-concentrator.html\\\"}]}}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:56.309000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "b05bab6b-8817-4787-885d-468f7c2286a4", + "executionId": "9b61aafd-8bf3-468b-8d36-77359164f8d1", + "recordId": "81cd0eb0-08ad-46bd-a638-a515d035dfc1", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.0, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:56.393000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "b05bab6b-8817-4787-885d-468f7c2286a4", + "executionId": "9b61aafd-8bf3-468b-8d36-77359164f8d1", + "recordId": "4ff3061c-6405-44e3-b197-31d21776b5e3", + "content": "{\"id\": \"4ff3061c-6405-44e3-b197-31d21776b5e3\", \"role\": \"assistant\", \"content\": [{\"text\": \"Quick fact-check first: on a Transit Gateway VPN attachment, AWS can advertise up to **5,000 routes outbound to the customer gateway** (TGW \\u2192 on-prem) and accept up to **1,000 routes inbound from the customer gateway** (on-prem \\u2192 TGW) \\u2014 both are hard limits, not adjustable. If you're pinned at the outbound 5,000, summarizing is the right move, but it's easy to break the return path if you do it carelessly. Here's how to do it safely:\\n\\n**1. Aggregate at a point that can actually reach everything inside the summary**\\nDon't just slap a supernet static route in the TGW route table pointing at one VPC attachment unless that attachment (or a hub/inspection VPC behind it) genuinely has reachability to every subnet the summary covers. If the summary spans multiple VPCs/attachments but you point it at only one, traffic for the others in that range will blackhole.\\n\\n**2. Suppress the propagated specifics once the summary is in place**\\nAfter adding the static aggregate route in the TGW route table associated with the VPN attachment, make sure the individual VPC-propagated routes it replaces aren't still being propagated into that same route table \\u2014 leftover specifics will both re-inflate your advertised count and, because of longest-prefix-match, can silently override the summary for a subset of addresses and send them to the wrong next hop.\\n\\n**3. This is the part that actually breaks return paths \\u2014 watch for asymmetry**\\nSummarizing shrinks what TGW advertises *out*, but it doesn't change anything on the other leg. If there's any other path (a second VPN, a DX circuit, a backup CGW) that still advertises a more-specific route for a prefix inside your new summary, on-prem routing will prefer the specific route outbound while AWS still only has the summary \\u2014 traffic goes out one path and the return hits a different attachment. For anything stateful (NAT gateways, firewalls, inspection appliances) that will drop the session.\\n - Keep BGP attributes (AS-PATH length, local-pref/MED, communities) consistent across all paths so the same path wins in both directions for both the summary and any remaining specifics.\\n - If you run active/active tunnels or dual CGWs, advertise the same summary with matching attributes on all of them.\\n\\n**4. Plan the CIDR layout before you aggregate, not after**\\nSummarization only works cleanly if the VPCs in question were allocated from a contiguous parent block. If they weren't (common when VPCs were created ad hoc), you'll need either an IPAM-driven re-allocation or to insert a hub VPC that owns the summarized block and does the fan-out internally.\\n\\n**5. Validate before you trust it**\\nOnce the static aggregate is in place, confirm end-to-end reachability per subnet inside the summary and check that no stray specific route reappears later (e.g., someone re-enables propagation during a maintenance). A route analyzer / reachability check against the TGW route table is worth running after any change here, since a bad aggregate fails silently until a specific flow breaks.\\n\\nIf you want, I can pull up the actual TGW route tables and VPN attachment config for a specific Transit Gateway in your account and check whether a summary route is safe to add \\u2014 just point me at the TGW ID.\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:56.475000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "b05bab6b-8817-4787-885d-468f7c2286a4", + "executionId": "9b61aafd-8bf3-468b-8d36-77359164f8d1", + "recordId": "fab938ae-4084-4012-8c0b-7e244d73aded", + "content": "{\"id\": \"20a66248-30b3-4f3d-9a2f-f3cb17eef3ec\", \"role\": \"assistant\", \"content\": [{\"type\": \"text\", \"text\": \"Quick fact-check first: on a Transit Gateway VPN attachment, AWS can advertise up to **5,000 routes outbound to the customer gateway** (TGW \\u2192 on-prem) and accept up to **1,000 routes inbound from the customer gateway** (on-prem \\u2192 TGW) \\u2014 both are hard limits, not adjustable. If you're pinned at the outbound 5,000, summarizing is the right move, but it's easy to break the return path if you do it carelessly. Here's how to do it safely:\\n\\n**1. Aggregate at a point that can actually reach everything inside the summary**\\nDon't just slap a supernet static route in the TGW route table pointing at one VPC attachment unless that attachment (or a hub/inspection VPC behind it) genuinely has reachability to every subnet the summary covers. If the summary spans multiple VPCs/attachments but you point it at only one, traffic for the others in that range will blackhole.\\n\\n**2. Suppress the propagated specifics once the summary is in place**\\nAfter adding the static aggregate route in the TGW route table associated with the VPN attachment, make sure the individual VPC-propagated routes it replaces aren't still being propagated into that same route table \\u2014 leftover specifics will both re-inflate your advertised count and, because of longest-prefix-match, can silently override the summary for a subset of addresses and send them to the wrong next hop.\\n\\n**3. This is the part that actually breaks return paths \\u2014 watch for asymmetry**\\nSummarizing shrinks what TGW advertises *out*, but it doesn't change anything on the other leg. If there's any other path (a second VPN, a DX circuit, a backup CGW) that still advertises a more-specific route for a prefix inside your new summary, on-prem routing will prefer the specific route outbound while AWS still only has the summary \\u2014 traffic goes out one path and the return hits a different attachment. For anything stateful (NAT gateways, firewalls, inspection appliances) that will drop the session.\\n - Keep BGP attributes (AS-PATH length, local-pref/MED, communities) consistent across all paths so the same path wins in both directions for both the summary and any remaining specifics.\\n - If you run active/active tunnels or dual CGWs, advertise the same summary with matching attributes on all of them.\\n\\n**4. Plan the CIDR layout before you aggregate, not after**\\nSummarization only works cleanly if the VPCs in question were allocated from a contiguous parent block. If they weren't (common when VPCs were created ad hoc), you'll need either an IPAM-driven re-allocation or to insert a hub VPC that owns the summarized block and does the fan-out internally.\\n\\n**5. Validate before you trust it**\\nOnce the static aggregate is in place, confirm end-to-end reachability per subnet inside the summary and check that no stray specific route reappears later (e.g., someone re-enables propagation during a maintenance). A route analyzer / reachability check against the TGW route table is worth running after any change here, since a bad aggregate fails silently until a specific flow breaks.\\n\\nIf you want, I can pull up the actual TGW route tables and VPN attachment config for a specific Transit Gateway in your account and check whether a summary route is safe to add \\u2014 just point me at the TGW ID.\"}]}", + "createdAt": "2026-10-27T17:05:56.568000-05:00", + "recordType": "final_response" + } +] \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vs-vgw-vif-transit/with_skill/functional-tests-results.json b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vs-vgw-vif-transit/with_skill/functional-tests-results.json new file mode 100644 index 00000000..2a942625 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vs-vgw-vif-transit/with_skill/functional-tests-results.json @@ -0,0 +1,74 @@ +{ + "version": "v1", + "iteration": 1, + "eval_id": "tgw-vs-vgw-vif-transit", + "run_type": "with_skill", + "success": true, + "error": null, + "understanding_agent_space_skill": null, + "tests": { + "trigger": { + "result": "passed", + "message": "Skill 'aws-routing' was triggered as expected", + "skill_loads_found": 1 + }, + "expected_output": { + "result": "passed", + "reasoning": "The agent correctly states AWS doesn't act as a transit hub by default (only advertises VPC CIDRs), explains VPN CloudHub-like behavior for DX+VPN on same VGW, and describes TGW with two DXGWs plus Allowed Prefix List as the intentional way to enable DX-to-DX communication. This matches all key points of the expected output.", + "evidence": "\"AWS doesn't act as a transit hub between Direct Connect VIFs automatically... AWS only advertises VPC CIDRs toward each VIF\" and \"Two DXGWs into one TGW... DX \u2194 DX works\" and \"TGW + Allowed Prefix List... remote-site prefixes can be deliberately advertised\" and \"DX + VPN on the same VGW... true VPN CloudHub behavior\"", + "confidence": "high" + }, + "assertions": { + "assertion_results": [ + { + "text": "explains AWS does not act as a transit hub by default", + "evaluator": "llm", + "passed": true, + "evidence": "\"AWS doesn't act as a transit hub between Direct Connect VIFs automatically.\" This directly states the assertion.", + "reasoning": "The output explicitly states this principle as the key opening explanation.", + "confidence": "high" + }, + { + "text": "mentions VPN CloudHub for VPN connections", + "evaluator": "llm", + "passed": true, + "evidence": "\"DX + VPN on the same VGW | DX site can talk to a VPN site on that VGW (true VPN CloudHub behavior)\"", + "reasoning": "The output explicitly mentions VPN CloudHub behavior in the context of DX+VPN on same VGW.", + "confidence": "high" + }, + { + "text": "mentions TGW or two DXGWs as a way to enable DX-to-DX intentionally", + "evaluator": "llm", + "passed": true, + "evidence": "\"Two DXGWs into one TGW (CGW1\u2013DXGW1\u2013TGW\u2013DXGW2\u2013CGW2) | Each DX VIF attaches via its own DXGW to a shared TGW \u2014 DX \u2194 DX works\" and \"TGW + Allowed Prefix List\"", + "reasoning": "The output clearly describes using two DXGWs attached to a single TGW as the recommended way to enable DX-to-DX communication, and reiterates this in the concluding summary paragraph.", + "confidence": "high" + }, + { + "text": "does not contain 'error'", + "evaluator": "llm", + "passed": true, + "evidence": "No occurrence of the word 'error' anywhere in the response text.", + "reasoning": "A scan of the output shows no instance of the literal string 'error'.", + "confidence": "high" + } + ], + "summary": { + "passed": 4, + "failed": 0, + "errored": 0, + "low_confidence": 0, + "total": 4, + "pass_rate": 1.0 + } + } + }, + "metrics": { + "runtime": "11s", + "cost": "$0.09", + "context_window": { + "utilization": "4.9%", + "compaction_count": 0 + } + } +} \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vs-vgw-vif-transit/with_skill/outputs/journal_records.json b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vs-vgw-vif-transit/with_skill/outputs/journal_records.json new file mode 100644 index 00000000..86623769 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vs-vgw-vif-transit/with_skill/outputs/journal_records.json @@ -0,0 +1,122 @@ +[ + { + "agentSpaceId": "5257e7cf-1dc5-4af2-b50d-da93fc40ae71", + "executionId": "8795326d-cc03-4f4c-acc3-8cd0706148d6", + "recordId": "de511f5b-a525-4abc-8823-8c723ce3bf1d", + "content": "{\"id\": \"de511f5b-a525-4abc-8823-8c723ce3bf1d\", \"role\": \"user\", \"content\": [{\"text\": \"Can two on-premises sites on separate Direct Connect VIFs talk to each other through AWS?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:02.128000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "5257e7cf-1dc5-4af2-b50d-da93fc40ae71", + "executionId": "8795326d-cc03-4f4c-acc3-8cd0706148d6", + "recordId": "0f4510ac-e759-485d-97f0-6690f4bf1d9c", + "content": "{\"id\": \"388c2d3f-81d3-4b84-9e14-cd9e2e29f0b2\", \"role\": \"assistant\", \"content\": [{\"type\": \"chat_title\", \"title\": \"On-premises sites communicating through AWS Direct Connect\"}]}", + "createdAt": "2026-10-27T17:05:02.754000-05:00", + "recordType": "chat_title" + }, + { + "agentSpaceId": "5257e7cf-1dc5-4af2-b50d-da93fc40ae71", + "executionId": "8795326d-cc03-4f4c-acc3-8cd0706148d6", + "recordId": "7f8e0042-e33e-4a8f-8eee-0847b258850f", + "content": "{\"id\": \"7f8e0042-e33e-4a8f-8eee-0847b258850f\", \"role\": \"assistant\", \"content\": [{\"id\": \"tooluse_z1S23KysipBvSOekbNB5Ay\", \"tool_name\": \"skill_read\", \"input\": {\"skill_id\": \"aws-routing\"}, \"type\": \"tool_use\", \"display_intent\": \"Reading AWS routing skill for Direct Connect VIF-to-VIF connectivity guidance.\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:05.805000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "5257e7cf-1dc5-4af2-b50d-da93fc40ae71", + "executionId": "8795326d-cc03-4f4c-acc3-8cd0706148d6", + "recordId": "626edb60-3140-49d9-b7ce-cd7d169a0a41", + "content": "{\"id\": \"9b5a5d70-2eb6-4605-b40c-3b70d899b001\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This question about Direct Connect VIF routing fits the aws-routing skill well, so I'll check that skill for guidance.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:05.805000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "5257e7cf-1dc5-4af2-b50d-da93fc40ae71", + "executionId": "8795326d-cc03-4f4c-acc3-8cd0706148d6", + "recordId": "9f645db7-29d2-402d-8c9c-077eee47e291", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 3.8, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:05.875000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "5257e7cf-1dc5-4af2-b50d-da93fc40ae71", + "executionId": "8795326d-cc03-4f4c-acc3-8cd0706148d6", + "recordId": "65f94b15-3bfc-493b-bd1f-b2917391fedd", + "content": "{\"id\": \"4931fe4a-26ed-46a6-8e50-15054dfee303\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"526\"}]}", + "createdAt": "2026-10-27T17:05:05.910000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "5257e7cf-1dc5-4af2-b50d-da93fc40ae71", + "executionId": "8795326d-cc03-4f4c-acc3-8cd0706148d6", + "recordId": "d20e62fb-7968-438b-80c3-6d578f6bb122", + "content": "{\"id\": \"9d28ec89-edf2-4362-9cfe-671c576229b5\", \"role\": \"assistant\", \"block_type\": \"load_skill\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_z1S23KysipBvSOekbNB5Ay\", \"status\": \"success\", \"content\": [{\"text\": \"Done\"}]}], \"label\": \"Reading AWS routing skill for Direct Connect VIF-to-VIF connectivity guidance.\"}", + "createdAt": "2026-10-27T17:05:05.978000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "5257e7cf-1dc5-4af2-b50d-da93fc40ae71", + "executionId": "8795326d-cc03-4f4c-acc3-8cd0706148d6", + "recordId": "d97ba5cd-dc56-4282-b6a4-eb7d5da2da4a", + "content": "{\"id\": \"d97ba5cd-dc56-4282-b6a4-eb7d5da2da4a\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_z1S23KysipBvSOekbNB5Ay\", \"content\": \"[{'text': '{\\\"content\\\": \\\"---\\\\\\\\r\\\\\\\\nname: aws-routing\\\\\\\\r\\\\\\\\ndescription: \\\\'Use this skill when the user wants to know which path AWS will pick for their traffic, why traffic took an unexpected route, or how to make AWS prefer a path. Covers Cloud WAN CNE route evaluation, Direct Connect Gateway local vs remote region preference, Transit Gateway route tables, ECMP and peering, VPC longest-prefix-match, BGP traffic engineering (LP communities 7224:7100/7200/7300, AS-path prepending, MED), DX + VPN redundancy and failover, and asymmetric routing. Use it even when the user only describes a symptom, such as \\\\\\\\\\\"traffic is leaving the region\\\\\\\\\\\", \\\\\\\\\\\"not using my local Direct Connect\\\\\\\\\\\", \\\\\\\\\\\"routing to the wrong VIF/location\\\\\\\\\\\", or \\\\\\\\\\\"path is asymmetric\\\\\\\\\\\". Read-only: analysis, recommendations, and describe/get/list validation commands grounded in public AWS docs.\\\\'\\\\\\\\r\\\\\\\\nmetadata:\\\\\\\\r\\\\\\\\n version: \\\\\\\\\\\"1.2.5\\\\\\\\\\\"\\\\\\\\r\\\\\\\\n author: midakin\\\\\\\\r\\\\\\\\n aws-devops-agent-skills.agent-types: \\\\\\\\\\\"CHAT, INCIDENT_RCA\\\\\\\\\\\"\\\\\\\\r\\\\\\\\n---\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Overview\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nThis skill provides routing-first analysis for AWS networking. It reasons about the full traffic\\\\\\\\r\\\\\\\\npath, applies the documented route-evaluation logic for each service, and grounds recommendations\\\\\\\\r\\\\\\\\nin public AWS documentation. It is **read-only**: it produces analysis, recommendations, and\\\\\\\\r\\\\\\\\nread-only validation commands (`describe-*`, `get-*`, `list-*`) - never mutating actions.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Verification & Anti-Hallucination Directives (Mandatory)\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- Do not generate any fact, citation, or source that is not fully verifiable. If you lack a verified\\\\\\\\r\\\\\\\\n source, state \\\\\\\\\\\"I cannot verify this\\\\\\\\\\\" rather than inventing details.\\\\\\\\r\\\\\\\\n- State assumptions explicitly before answering.\\\\\\\\r\\\\\\\\n- Cross-check each claim against public AWS documentation or the `references/` files. Omit claims you\\\\\\\\r\\\\\\\\n cannot confirm.\\\\\\\\r\\\\\\\\n- Only cite AWS documentation URLs you have actually retrieved or confirmed. Do not fabricate links.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## When to use this skill\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nActivate when the user asks about:\\\\\\\\r\\\\\\\\n- Cloud WAN route evaluation / Core Network Edge (CNE) path selection\\\\\\\\r\\\\\\\\n- Direct Connect Gateway path selection, local-region preference, or LP communities\\\\\\\\r\\\\\\\\n- Transit Gateway route evaluation, ECMP, or peering routing\\\\\\\\r\\\\\\\\n- VPC route tables and longest-prefix-match behavior\\\\\\\\r\\\\\\\\n- BGP traffic engineering (communities, AS-path prepending, MED)\\\\\\\\r\\\\\\\\n- DX + VPN redundancy, active/active vs active/passive, or failover design\\\\\\\\r\\\\\\\\n- Why traffic is taking an unexpected path\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## What this skill does NOT do\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nThis skill is **read-only and advisory**. It will not:\\\\\\\\r\\\\\\\\n- **Design or apply infrastructure changes.** It does not create, modify, or delete AWS resources,\\\\\\\\r\\\\\\\\n and it does not run mutating CLI/API calls (`create-*`, `modify-*`, `associate-*`, `delete-*`,\\\\\\\\r\\\\\\\\n `put-*`, `update-*`) or write Infrastructure-as-Code to be deployed. Any change it describes is a\\\\\\\\r\\\\\\\\n *recommendation* for you to review and implement yourself.\\\\\\\\r\\\\\\\\n- **Configure BGP or push routing policy.** It explains which communities, AS-path, or MED settings\\\\\\\\r\\\\\\\\n to use, but it does not apply them to VIFs, route tables, or Cloud WAN policy documents.\\\\\\\\r\\\\\\\\n- **Execute anything against a live account beyond read-only validation.** Commands it provides are\\\\\\\\r\\\\\\\\n limited to `describe-*`, `get-*`, and `list-*` for confirming state.\\\\\\\\r\\\\\\\\n- **Guarantee production outcomes.** It recommends non-production testing and blue/green rollout;\\\\\\\\r\\\\\\\\n it does not perform the cutover or validate the result for you.\\\\\\\\r\\\\\\\\n- **Provide compliance, security, or contractual sign-off**, or make claims it cannot ground in\\\\\\\\r\\\\\\\\n public AWS documentation or the `references/` files.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nFor designing and applying changes, hand the recommendation to a change-capable workflow (with\\\\\\\\r\\\\\\\\nappropriate review, approvals, and testing).\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## How to analyze a routing problem\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nWork through this checklist in order; each step depends on the one before it.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- [ ] **1. Establish the path.** Identify source, destination, and every routing construct in\\\\\\\\r\\\\\\\\n between (VPC route table \\\\\\\\u2192 TGW/Cloud WAN CNE \\\\\\\\u2192 DXGW \\\\\\\\u2192 VIF \\\\\\\\u2192 on-prem, or the relevant subset).\\\\\\\\r\\\\\\\\n- [ ] **2. Name the route table.** Be explicit about which table makes each decision (VPC RT, TGW RT,\\\\\\\\r\\\\\\\\n CNE RT, DXGW internal selection).\\\\\\\\r\\\\\\\\n- [ ] **3. Apply the evaluation order for that construct** (see below).\\\\\\\\r\\\\\\\\n- [ ] **4. Identify the deciding attribute** (longest prefix, local preference, AS-path, MED, source\\\\\\\\r\\\\\\\\n type).\\\\\\\\r\\\\\\\\n- [ ] **5. Flag non-determinism** (e.g., ECMP or \\\\\\\\\\\"deterministically random\\\\\\\\\\\" tiebreakers) and\\\\\\\\r\\\\\\\\n recommend a deterministic alternative.\\\\\\\\r\\\\\\\\n- [ ] **6. Cite the AWS documentation** that supports the behavior.\\\\\\\\r\\\\\\\\n- [ ] **7. Provide read-only validation commands** the user can run to confirm.\\\\\\\\r\\\\\\\\n- [ ] **8. Self-check before answering.** Re-read the answer against steps 1-7: every route table is\\\\\\\\r\\\\\\\\n named, the evaluation order matches the construct, each cited URL was actually retrieved, and every\\\\\\\\r\\\\\\\\n command is read-only. Fix any gap before presenting.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Key evaluation rules\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Cloud WAN (per CNE)\\\\\\\\r\\\\\\\\nLongest prefix match \\\\\\\\u2192 static \\\\\\\\u2192 VPC-propagated (same region) \\\\\\\\u2192 unequal AS-path/MED (shortest wins) \\\\\\\\u2192\\\\\\\\r\\\\\\\\nequal AS-path & MED source preference: DXGW-propagated \\\\\\\\u2192 Cloud WAN Connect \\\\\\\\u2192 S2S VPN \\\\\\\\u2192 other\\\\\\\\r\\\\\\\\n(TGW peering, remote CNEs; identical from 2+ sources = deterministically random).\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Direct Connect (private/transit VIF, outbound AWS\\\\\\\\u2192on-prem)\\\\\\\\r\\\\\\\\nLongest prefix match \\\\\\\\u2192 local preference \\\\\\\\u2192 AS_PATH length \\\\\\\\u2192 MED \\\\\\\\u2192 ECMP (equal AS_PATH and BGP\\\\\\\\r\\\\\\\\nattributes). LP communities `7224:7300/7200/7100` (High/Medium/Low) set local preference and are\\\\\\\\r\\\\\\\\nevaluated before AS-path. DXGW prefers the local associated-region DX by default.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n### Direction & tooling guidance\\\\\\\\r\\\\\\\\n- **AWS \\\\\\\\u2192 on-prem (egress):** control with DX LP communities.\\\\\\\\r\\\\\\\\n- **On-prem \\\\\\\\u2192 AWS (ingress):** control with the customer router\\\\'s local-pref/weight/MED.\\\\\\\\r\\\\\\\\n- **AS-path prepending:** within a region only - unreliable across regions because DXGW local-region\\\\\\\\r\\\\\\\\n LP overrides it. **LP communities:** work within-region and cross-region.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Reference material\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nDetailed mechanics, patterns, and caveats are in `references/`. Load only the file that matches the\\\\\\\\r\\\\\\\\nconstruct deciding the path:\\\\\\\\r\\\\\\\\n- [Cloud WAN + DX routing patterns](references/cloudwan-dx-routing-patterns.md) - load when the path\\\\\\\\r\\\\\\\\n crosses a Cloud WAN CNE: CNE route evaluation, DXGW path selection into Cloud WAN, community\\\\\\\\r\\\\\\\\n traffic-engineering patterns, regional inspection, multi-region egress.\\\\\\\\r\\\\\\\\n- [Direct Connect routing patterns](references/dx-routing-patterns.md) - load when the question is\\\\\\\\r\\\\\\\\n about a DX VIF or DXGW without Cloud WAN: BGP mechanics, BFD, MED, ASN ranges, VIF considerations,\\\\\\\\r\\\\\\\\n CloudHub, when VIFs can talk, active/active vs active/passive.\\\\\\\\r\\\\\\\\n- [Transit Gateway routing patterns](references/tgw-routing-patterns.md) - load when a TGW route\\\\\\\\r\\\\\\\\n table makes the decision: TGW route evaluation, ECMP limits, DXGW-to-TGW behavior, DX/VPN caveats,\\\\\\\\r\\\\\\\\n failover, peering.\\\\\\\\r\\\\\\\\n- [VPN + DX redundancy patterns](references/vpn-dx-redundancy-patterns.md) - load when VPN backs up\\\\\\\\r\\\\\\\\n or runs alongside DX: route preference between them, ECMP limits, asymmetric return paths.\\\\\\\\r\\\\\\\\n- [Networking strategic questions](references/networking-strategic-questions.md) - load when key\\\\\\\\r\\\\\\\\n facts are missing (regions, attachments, ASNs, communities) and you need discovery questions to ask\\\\\\\\r\\\\\\\\n before analyzing.\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n## Output expectations\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\n- Explain the route-evaluation logic step by step.\\\\\\\\r\\\\\\\\n- Name the specific route table involved.\\\\\\\\r\\\\\\\\n- Give the end-to-end traffic path.\\\\\\\\r\\\\\\\\n- Compare options with tradeoffs (determinism, complexity, failover) when multiple solutions exist.\\\\\\\\r\\\\\\\\n- Include read-only AWS CLI validation commands.\\\\\\\\r\\\\\\\\n- Recommend testing in a non-production environment and blue/green for migrations.\\\\\\\\r\\\\\\\\n\\\", \\\"resource_manifest\\\": [\\\"SKILL.md\\\", \\\"references/cloudwan-dx-routing-patterns.md\\\", \\\"references/dx-routing-patterns.md\\\", \\\"references/networking-strategic-questions.md\\\", \\\"references/tgw-routing-patterns.md\\\", \\\"references/vpn-dx-redundancy-patterns.md\\\"]}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:08.577000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "5257e7cf-1dc5-4af2-b50d-da93fc40ae71", + "executionId": "8795326d-cc03-4f4c-acc3-8cd0706148d6", + "recordId": "e073955a-0d38-4006-9fb2-656256c3cb27", + "content": "{\"id\": \"e073955a-0d38-4006-9fb2-656256c3cb27\", \"role\": \"assistant\", \"content\": [{\"text\": \"This is exactly the \\\"CloudHub\\\" scenario \\u2014 let me pull the Direct Connect reference for the specific mechanics.\", \"type\": \"text\"}, {\"id\": \"tooluse_WzIezM46XDlACCdFzhKEcw\", \"tool_name\": \"read_skill_resource\", \"input\": {\"skill_id\": \"aws-routing\", \"path\": \"references/dx-routing-patterns.md\"}, \"type\": \"tool_use\", \"display_intent\": \"Reading Direct Connect routing patterns reference for VIF-to-VIF connectivity details.\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:08.656000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "5257e7cf-1dc5-4af2-b50d-da93fc40ae71", + "executionId": "8795326d-cc03-4f4c-acc3-8cd0706148d6", + "recordId": "4573feb9-b27a-4c59-9274-fd0b564a2775", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.2, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:08.744000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "5257e7cf-1dc5-4af2-b50d-da93fc40ae71", + "executionId": "8795326d-cc03-4f4c-acc3-8cd0706148d6", + "recordId": "8c55ec9b-81ca-40c5-82fb-a49a28787423", + "content": "{\"id\": \"2bb43aba-66d3-4a69-b400-20a1547daf1f\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_WzIezM46XDlACCdFzhKEcw\", \"status\": \"success\", \"content\": [{\"text\": \"Done\"}]}], \"label\": \"Reading Direct Connect routing patterns reference for VIF-to-VIF connectivity details.\"}", + "createdAt": "2026-10-27T17:05:09.026000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "5257e7cf-1dc5-4af2-b50d-da93fc40ae71", + "executionId": "8795326d-cc03-4f4c-acc3-8cd0706148d6", + "recordId": "9be0be5f-5c46-4a58-b012-ebaa058c307a", + "content": "{\"id\": \"9be0be5f-5c46-4a58-b012-ebaa058c307a\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_WzIezM46XDlACCdFzhKEcw\", \"content\": \"[{'text': '\\\\r\\\\n\\\\r\\\\n# Direct Connect Routing Patterns & Considerations\\\\r\\\\n\\\\r\\\\nThis document captures key BGP routing mechanics, path selection behavior, VIF-specific considerations, and operational caveats for AWS Direct Connect architectures. Use this knowledge when advising on DX design (active/active, active/passive), BGP community behavior, BFD configuration, and public/private/transit VIF routing.\\\\r\\\\n\\\\r\\\\n## Reference Documentation\\\\r\\\\n\\\\r\\\\n- [Direct Connect Routing Policies and BGP Communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html)\\\\r\\\\n- [Direct Connect Path Selection (routing policies for private/transit VIFs)](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html)\\\\r\\\\n- [Direct Connect Virtual Interfaces](https://docs.aws.amazon.com/directconnect/latest/UserGuide/WorkingWithVirtualInterfaces.html)\\\\r\\\\n\\\\r\\\\n## BGP & Routing\\\\r\\\\n\\\\r\\\\n### BGP Route Visibility (ListVirtualInterfaceRoutes)\\\\r\\\\n\\\\r\\\\nAs of July 2026, DX supports **BGP route visibility** on virtual interfaces via the `ListVirtualInterfaceRoutes` API and the DX console:\\\\r\\\\n\\\\r\\\\n- **Accepted routes:** Routes AWS received from your on-prem router\\\\r\\\\n- **Advertised routes:** Routes AWS is sending to your on-prem router\\\\r\\\\n- Each route shows: prefix, address family, AS-path, community values, installation timestamp\\\\r\\\\n- Can filter by: prefix, AS-path, community, or address family\\\\r\\\\n- Available on: private, transit, and public VIFs\\\\r\\\\n- Available in: all AWS commercial regions\\\\r\\\\n\\\\r\\\\n**Use for troubleshooting:**\\\\r\\\\n- Verify route propagation between on-prem and AWS\\\\r\\\\n- Validate BGP policy configurations (communities, AS-path prepending)\\\\r\\\\n- Diagnose unexpected traffic patterns or asymmetric routing\\\\r\\\\n- Confirm which routes DXGW is accepting vs. advertising\\\\r\\\\n- Validate LP community tags are being received correctly\\\\r\\\\n\\\\r\\\\n**Reference:** [AWS Direct Connect BGP Route Visibility announcement](https://aws.amazon.com/about-aws/whats-new/2026/07/aws-direct-connect-bgp-visibility/)\\\\r\\\\n\\\\r\\\\n### CloudWatch BGP Metrics for DX Virtual Interfaces\\\\r\\\\n\\\\r\\\\nAs of March 2026, DX publishes **3 BGP metrics** to CloudWatch for all VIF types (private, transit, public):\\\\r\\\\n\\\\r\\\\n| Metric | What It Reports |\\\\r\\\\n|---|---|\\\\r\\\\n| `VirtualInterfaceBgpStatus` | BGP session state: 1 = up, 0 = down |\\\\r\\\\n| `VirtualInterfaceBgpPrefixesAccepted` | Number of prefixes AWS accepted from on-prem |\\\\r\\\\n| `VirtualInterfaceBgpPrefixesAdvertised` | Number of prefixes AWS advertises to on-prem |\\\\r\\\\n\\\\r\\\\n**Key details:**\\\\r\\\\n- Updated every 5 minutes (BGP flaps between intervals may not be captured)\\\\r\\\\n- Published at no extra cost in the region where the DX location is associated\\\\r\\\\n- Available via CloudWatch MCP server (`get_metric_data`) for programmatic access\\\\r\\\\n\\\\r\\\\n**Use cases:**\\\\r\\\\n- **BGP session monitoring:** Alarm on `VirtualInterfaceBgpStatus < 1` to detect session failures without API polling\\\\r\\\\n- **Prefix limit protection:** Alarm when `PrefixesAccepted` approaches VIF quota to prevent session idle state\\\\r\\\\n- **Silent route withdrawal detection:** BGP session stays UP but prefix count drops \\u2192 indicates routes withdrawn without session going down\\\\r\\\\n- **DR/failover validation:** Compare prefix counts between primary and standby VIFs \\u2014 a standby with fewer prefixes indicates a broken failover path\\\\r\\\\n- **Route change correlation:** Sudden drop in accepted/advertised prefixes correlates with traffic impact events\\\\r\\\\n\\\\r\\\\n**Alarm examples:**\\\\r\\\\n- `VirtualInterfaceBgpStatus` \\u2014 Statistic: Minimum, Period: 5min, Threshold: Lower than 1\\\\r\\\\n- `VirtualInterfaceBgpPrefixesAccepted` \\u2014 Statistic: Maximum, Period: 5min, Threshold: Greater/Equal to (near quota limit)\\\\r\\\\n- `VirtualInterfaceBgpPrefixesAccepted` \\u2014 Statistic: Maximum, Period: 5min, Threshold: Lower/Equal to (below expected minimum)\\\\r\\\\n\\\\r\\\\n**Reference:** [CloudWatch Metrics for DX VIF BGP Health and Prefix Count](https://aws.amazon.com/blogs/networking-and-content-delivery/introducing-cloudwatch-metrics-for-aws-direct-connect-virtual-interface-bgp-health-and-prefix-count/)\\\\r\\\\n\\\\r\\\\n### General Routing Behavior\\\\r\\\\n\\\\r\\\\n- When designing active/active or active/passive DX (within or across DX locations), apply the documented **Private/Transit VIF outbound path-selection order**: longest prefix match \\u2192 local preference \\u2192 AS_PATH length \\u2192 MED \\u2192 ECMP (equal AS_PATH length and BGP attributes). See [Direct Connect routing policies and BGP communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html).\\\\r\\\\n- Consider recommendations for **migrating from DX + VGW/Private VIF to DX + TGW/Transit VIF** when customers are modernizing their architecture\\\\r\\\\n\\\\r\\\\n### MED Support\\\\r\\\\n\\\\r\\\\n- DX **supports Multi-Exit Discriminator (MED) on Private and Transit VIFs** \\u2014 MED can influence the outbound (AWS \\u2192 on-prem) routing path, but only as a **tiebreaker after prefix length, local preference, and AS_PATH are equal**. AWS explicitly **does not recommend relying on MED** given its lower priority in the evaluation order.\\\\r\\\\n- For **Public VIFs**, the documented outbound routing policy uses **AS_PATH and longest prefix match** \\u2014 MED is not listed among the supported public VIF path-selection attributes.\\\\r\\\\n- **Outbound (AWS \\u2192 on-prem) path selection order for Private/Transit VIF:** longest prefix match \\u2192 local preference \\u2192 AS_PATH length \\u2192 MED \\u2192 ECMP (equal AS_PATH length and BGP attributes).\\\\r\\\\n\\\\r\\\\n**Reference:** [Direct Connect routing policies and BGP communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html)\\\\r\\\\n\\\\r\\\\n### BFD (Bidirectional Forwarding Detection)\\\\r\\\\n\\\\r\\\\n**\\u26a0\\ufe0f Always verify the Associated Region for each DX location** \\u2014 some DX locations can associate to different AWS regions than expected based on their physical geography. For example:\\\\r\\\\n- Equinix LD5 (Slough, UK) \\u2192 **eu-west-1** (Ireland)\\\\r\\\\n- Telehouse (London Docklands, UK) \\u2192 **eu-west-2** (London)\\\\r\\\\n- Equinix MA3 (Manchester, UK) \\u2192 **eu-west-2** (London)\\\\r\\\\n\\\\r\\\\nThe associated region determines which TGW/DXGW considers the DX \\\"local\\\" for LP preference and routing behavior. Verify a location\\\\'s associated region with the `DescribeLocations` API/CLI (`aws directconnect describe-locations`) or the official AWS Direct Connect locations reference: https://aws.amazon.com/directconnect/locations/\\\\r\\\\n\\\\r\\\\n### BFD (Bidirectional Forwarding Detection)\\\\r\\\\n\\\\r\\\\n- **Asynchronous BFD is automatically enabled** on the AWS side for all DX virtual interfaces\\\\r\\\\n- Customer must configure asynchronous BFD on their router (refer to vendor documentation)\\\\r\\\\n- BFD provides fast link failure detection \\u2014 ensures BGP neighbor relationship is torn down quickly for faster failover\\\\r\\\\n- **Do NOT configure both BFD for BGP and BGP Graceful Restart simultaneously** \\u2014 this results in sub-optimal routing performance (conflicting goals: BFD wants fast teardown, Graceful Restart wants to preserve routes during failure)\\\\r\\\\n\\\\r\\\\n### ASN Support\\\\r\\\\n\\\\r\\\\n| Context | Supported ASN Range |\\\\r\\\\n|---|---|\\\\r\\\\n| AWS side via DX-GW | Full 32-bit: 4,200,000,000 to 4,294,967,294 (private) |\\\\r\\\\n| AWS side via VGW | 32-bit: 1 to 2,147,483,647 only |\\\\r\\\\n| Customer side (Private/Public/Transit VIF) | 32-bit: 1 to 2,147,483,647 only |\\\\r\\\\n\\\\r\\\\n- Customers can request a Private/Transit VIF ASN from the 4,200,000,000\\u20134,294,967,294 range via a support case (non-standard, requires internal approval)\\\\r\\\\n\\\\r\\\\n### SiteLink & BGP Communities\\\\r\\\\n\\\\r\\\\n- SiteLink **maintains BGP community tags** \\u2014 tags are passed through from one connected peer to another\\\\r\\\\n- This means communities set on one DX location will be visible at another DX location connected via SiteLink\\\\r\\\\n\\\\r\\\\n## VIF-Specific Considerations\\\\r\\\\n\\\\r\\\\n### Public VIF\\\\r\\\\n\\\\r\\\\n- **Reverse-path forwarding (RPF) strict mode is enforced** \\u2014 validates that the source of traffic originated from the customer\\\\'s advertised prefix (prevents address spoofing)\\\\r\\\\n- **Do NOT use EC2 EIPs for Public VIF BGP peering** \\u2014 can cause routing issues. Customers should use:\\\\r\\\\n - Public IPs they own, OR\\\\r\\\\n - IPs assigned by their provider/AWS support\\\\r\\\\n- **EC2 BYOIP CIDRs are NOT in ip-ranges.json** (not part of Amazon address space), but they ARE advertised over DX Public VIF to customers\\\\r\\\\n- **S3-only Public VIF**: BGP prefix lists are available (GitHub repository) for filtering S3 traffic over DX Public VIF \\u2014 allows routing only S3 traffic through DX while sending other AWS traffic over internet to reduce costs\\\\r\\\\n\\\\r\\\\n### Private / Transit VIF\\\\r\\\\n\\\\r\\\\n- For Private/Transit VIFs, customers can use any IPv4 address (if not using AWS auto-generated IPs), but must ensure those IPs **do not conflict** with on-prem and/or VPC networks\\\\r\\\\n- When attaching VPCs with **AWS-assigned IPv6** to a TGW, these are discontinuous by default and **cannot be aggregated** for Direct Connect \\u2014 leads to DX prefix quota limitations\\\\r\\\\n - **Workarounds:** Use Amazon-provided contiguous IPv6 CIDR blocks, or EC2 BYOIP\\\\r\\\\n\\\\r\\\\n### VPC-to-VPC Routing via DXGW (Regression)\\\\r\\\\n\\\\r\\\\n- As of November 2021, **VPC-to-VPC routing** connected to the same DXGW via VGW/TGW which are also connected over Private/Transit VIF **is possible due to a regression**\\\\r\\\\n- This was previously blocked \\u2014 be aware of this change\\\\r\\\\n- Workarounds exist to block this unintended traffic flow if isolation is required\\\\r\\\\n\\\\r\\\\n### DX VIFs and VGW \\u2014 CloudHub Behavior\\\\r\\\\n\\\\r\\\\n- **VPN CloudHub is a Site-to-Site VPN feature**: multiple VPN connections terminating on the same VGW can exchange routes with each other, letting remote sites communicate through AWS. See [AWS VPN CloudHub](https://docs.aws.amazon.com/vpc/latest/userguide/vpn-connections.html).\\\\r\\\\n- **DX can participate in a CloudHub with VPN**: when a DX private VIF and one or more VPN connections share the same VGW, prefixes learned from a VPN customer gateway are advertised to the DX customer gateway and vice versa \\u2014 so a DX site and VPN site(s) on the same VGW can reach each other.\\\\r\\\\n- **DX-to-DX does NOT hub through a VGW**: if two DX VIFs from different on-prem sites terminate on the same VGW (or DX Gateway), AWS does not re-advertise one site\\\\'s on-prem prefixes to the other. AWS advertises only the VPC CIDR(s) toward each VIF, so the two on-prem sites cannot reach each other through AWS. Site-to-site transit requires the customer\\\\'s own router (hairpin) or a design that isn\\\\'t a single shared gateway.\\\\r\\\\n- **DX Gateway (VGW association) advertises only VPC CIDRs to on-prem** \\u2014 it does not provide CloudHub between gateway associations; associations on the same DXGW cannot send traffic to each other.\\\\r\\\\n- **Floating VGW (VGW not attached to a VPC):**\\\\r\\\\n - *With VPN* \\u2014 a floating VGW can still act as a VPN CloudHub: multiple VPN connections on it exchange routes, so remote sites reach each other through AWS with no VPC involved.\\\\r\\\\n - *With DXGW* \\u2014 not a usable scenario. A DXGW advertises associated VPC CIDRs to on-prem; a floating VGW has no VPC, so there is nothing to advertise. DXGW-over-VGW is only meaningful when the VGW is attached to a VPC.\\\\r\\\\n\\\\r\\\\n**Note:** This corrects an earlier claim that DX VIFs on a shared VGW form a DX-to-DX hub-and-spoke mesh. Route sharing between on-prem sites on a shared gateway is a VPN CloudHub behavior (VPN connections), not a DX-to-DX behavior. Validate specific flows in a lab before relying on them.\\\\r\\\\n\\\\r\\\\n### When Can VIFs Talk to Each Other? (Site-to-Site via AWS)\\\\r\\\\n\\\\r\\\\n**Governing principle:** AWS does not act as a transit hub between VIFs by default. Over a VGW or a single DXGW, AWS advertises only the **VPC CIDRs** toward each VIF, so two on-prem sites (each on its own VIF) do not learn each other\\\\'s prefixes unless something explicitly enables it.\\\\r\\\\n\\\\r\\\\n**Supported \\u2014 VIFs (sites) CAN reach each other:**\\\\r\\\\n\\\\r\\\\n| Scenario | How it works |\\\\r\\\\n|---|---|\\\\r\\\\n| VPN CloudHub on a VGW | Multiple VPN connections on the same VGW exchange routes; remote sites reach each other. See [AWS VPN CloudHub](https://docs.aws.amazon.com/vpc/latest/userguide/vpn-connections.html). |\\\\r\\\\n| DX + VPN on the same VGW | A DX private VIF sharing a VGW with VPN connection(s) can talk to the VPN site(s) \\u2014 DX \\u2194 VPN. |\\\\r\\\\n| TGW + Allowed Prefix List | The DXGW\\u2192TGW attachment advertises whatever is in the Allowed Prefix List (no VPC-CIDR-only restriction), so remote-site prefixes can be advertised intentionally. Enables DX \\u2194 VPN CloudHub via TGW. |\\\\r\\\\n| DX \\u2194 DX via two DXGWs into one TGW | Two DXGWs attached to the same TGW (CGW1\\u2013DXGW1\\u2013TGW\\u2013DXGW2\\u2013CGW2) allows DX site-to-site. |\\\\r\\\\n| Customer-side hairpin | Always available: advertise a supernet/default from on-prem and let the customer router be the hub. AWS is not the transit point. |\\\\r\\\\n\\\\r\\\\n**Not supported \\u2014 sites CANNOT reach each other through AWS:**\\\\r\\\\n\\\\r\\\\n| Scenario | Why |\\\\r\\\\n|---|---|\\\\r\\\\n| Two DX VIFs on the same VGW | AWS advertises only VPC CIDRs to each VIF; no site-to-site. |\\\\r\\\\n| Two DX VIFs on the same single DXGW | Gateway associations on one DXGW cannot send traffic to each other. |\\\\r\\\\n| Inter-site over a single DXGW with transit VIFs | Routes may install but the flow loops; not supported. Use two DXGWs. |\\\\r\\\\n| Inter-site via two DXGWs with VGW associations | Only VPC CIDRs are advertised, so the sites never learn each other. |\\\\r\\\\n\\\\r\\\\n**Summary:**\\\\r\\\\n- **VGW / single DXGW** \\u2192 advertises VPC CIDRs only \\u2192 no VIF-to-VIF transit (except true VPN CloudHub, which is a VPN feature).\\\\r\\\\n- **TGW** \\u2192 Allowed Prefix List with no route validation \\u2192 VIF-to-VIF transit can be built intentionally, using two DXGWs for DX-to-DX.\\\\r\\\\n\\\\r\\\\n## DX vs Peering (Public VIF vs PNI)\\\\r\\\\n\\\\r\\\\n### Key Differences\\\\r\\\\n\\\\r\\\\n| Attribute | DX Public VIF | Public/Private Peering (PNI) |\\\\r\\\\n|---|---|---|\\\\r\\\\n| Official AWS service | Yes | No |\\\\r\\\\n| SLA | Yes | No |\\\\r\\\\n| AWS Support coverage | Yes | No (handled by Global Interconnect Strategy team) |\\\\r\\\\n| Internal LOCAL_PREF | Higher | Lower |\\\\r\\\\n| Path preference | **Preferred** (higher LP) | Lower priority |\\\\r\\\\n\\\\r\\\\n### Important Guidance\\\\r\\\\n\\\\r\\\\n- If the **same prefix** is received by AWS over both DX Public VIF and Peering (PNI, IXP, or Transit), **DX Public VIF is the preferred path** (higher internal LOCAL_PREF)\\\\r\\\\n- Public Peering / PNI is NOT an official AWS service \\u2014 no SLA, no support expectation from the technical field\\\\r\\\\n- **Steer customers toward DX** when they need reliability and support coverage\\\\r\\\\n- There is no expectation for the technical field to support Public Peering / PNI engagements\\\\r\\\\n\\\\r\\\\n## Design Considerations for Active/Active & Active/Passive\\\\r\\\\n\\\\r\\\\n### General Best Practice for DX Routing Selection\\\\r\\\\n\\\\r\\\\n- **Always use LP communities (`7224:7300/7200/7100`) for expected egress routing** \\u2014 High Preference BGP communities are the recommended mechanism for predictable DX egress.\\\\r\\\\n- **AS-path prepending: within a region ONLY** \\u2014 Use AS-path prepending to influence egress between DX connections belonging to the **same associated AWS region**. Do NOT use AS-path prepending to influence traffic across regions \\u2014 DXGW\\\\'s local-region LP makes cross-region prepending unreliable.\\\\r\\\\n - **When comparing AS-path lengths, distinguish an originating site from a transiting one** \\u2014 a site that learns a prefix over the on-prem WAN and re-advertises it carries the origin ASN in its path, so its total length is prepend-count + origin ASN(s), not prepend-count alone. Confirm the real path with `list-virtual-interface-routes`. See \\\"Confusing Prepend Count with Total AS-Path Length\\\" in cloudwan-dx-routing-patterns.md.\\\\r\\\\n- **BGP communities: within region OR across regions** \\u2014 LP communities work for both intra-region path selection and cross-region preference (overriding DXGW\\\\'s default local-region LP).\\\\r\\\\n\\\\r\\\\n### Active/Active (within or across DX locations)\\\\r\\\\n\\\\r\\\\n- Understand how AWS selects between multiple active paths using the Private/Transit VIF outbound path-selection order (longest prefix match \\u2192 local preference \\u2192 AS_PATH length \\u2192 MED \\u2192 ECMP). See [Direct Connect routing policies and BGP communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html).\\\\r\\\\n- ECMP is available at the TGW level across multiple DX-GW attachments (same or different DX-GWs)\\\\r\\\\n- LP communities (`7224:7300/7200/7100`) can influence preference but behavior varies between VGW/TGW/Cloud WAN \\u2014 always test\\\\r\\\\n- **DXGW ECMPs across equal remote paths** \\u2014 when a local DX fails and multiple remote DX locations have matching attributes (same LP, same AS-path), DXGW will ECMP across them. This provides load-balanced failover rather than single-path selection.\\\\r\\\\n\\\\r\\\\n**Path selection behavior by service (NOT at DXGW):**\\\\r\\\\n- **DXGW:** ECMPs across equal paths \\u2014 load-balanced failover\\\\r\\\\n- **TGW:** Deterministic \\u2014 selects oldest route for equal paths. Consistent but not customer-controllable.\\\\r\\\\n- **Cloud WAN CNE:** Deterministic \\u2014 \\\"deterministically random\\\" at Step 2.4.4 for equal remote CNE paths. Consistent but not customer-controllable.\\\\r\\\\n\\\\r\\\\n### Active/Passive\\\\r\\\\n\\\\r\\\\n- Use LP communities or AS-path prepending to create primary/secondary paths\\\\r\\\\n- For Cloud WAN: communities override DXGW local-region preference (see cloudwan-dx-routing-patterns.md)\\\\r\\\\n- For TGW: LP communities are NOT honored across multiple DX-GW attachments (see tgw-routing-patterns.md)\\\\r\\\\n- BFD should be enabled for fast failure detection on the active path\\\\r\\\\n'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:17.112000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "5257e7cf-1dc5-4af2-b50d-da93fc40ae71", + "executionId": "8795326d-cc03-4f4c-acc3-8cd0706148d6", + "recordId": "83ca36f7-58d2-40c1-80b5-d00b92f40f77", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.9, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:17.189000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "5257e7cf-1dc5-4af2-b50d-da93fc40ae71", + "executionId": "8795326d-cc03-4f4c-acc3-8cd0706148d6", + "recordId": "be1174cb-a944-46c9-bec3-00c879442b4c", + "content": "{\"id\": \"be1174cb-a944-46c9-bec3-00c879442b4c\", \"role\": \"assistant\", \"content\": [{\"text\": \"**Short answer: not by default, but it's achievable with the right design.**\\n\\nThe key principle: AWS doesn't act as a transit hub between Direct Connect VIFs automatically. Over a VGW or a single DXGW, AWS only advertises **VPC CIDRs** toward each VIF \\u2014 it doesn't re-advertise one on-prem site's prefixes to the other. So two DX sites sharing a gateway don't learn about each other.\\n\\n**It does NOT work when:**\\n- Both VIFs terminate on the same VGW \\u2014 AWS advertises only VPC CIDRs, no site-to-site\\n- Both VIFs are on the same single DXGW \\u2014 gateway associations on one DXGW can't send traffic to each other\\n\\n**It DOES work with these designs:**\\n\\n| Design | How it enables site-to-site |\\n|---|---|\\n| **Two DXGWs into one TGW** (CGW1\\u2013DXGW1\\u2013TGW\\u2013DXGW2\\u2013CGW2) | Each DX VIF attaches via its own DXGW to a shared TGW \\u2014 DX \\u2194 DX works |\\n| **TGW + Allowed Prefix List** | The DXGW\\u2192TGW attachment advertises whatever's in the Allowed Prefix List, not just VPC CIDRs, so remote-site prefixes can be deliberately advertised |\\n| **DX + VPN on the same VGW** | DX site can talk to a VPN site on that VGW (true VPN CloudHub behavior) |\\n| **Customer-side hairpin** | Advertise a supernet/default from on-prem and let your own router be the hub \\u2014 AWS isn't the transit point |\\n\\nSo if your two sites are each on their own DX VIF today and you want them to reach each other through AWS, the standard pattern is: put each VIF behind its own DXGW, attach both DXGWs to a common Transit Gateway, and use an Allowed Prefix List to control what gets advertised between them.\\n\\nWant me to walk through what that TGW route table would look like for your specific setup, or check the Allowed Prefix List config on an existing DXGW?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:17.259000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "5257e7cf-1dc5-4af2-b50d-da93fc40ae71", + "executionId": "8795326d-cc03-4f4c-acc3-8cd0706148d6", + "recordId": "609668e7-5cbe-4756-9b91-099f7bcca250", + "content": "{\"id\": \"04f52fbb-a06a-4b68-91ad-b36e6075e4f1\", \"role\": \"assistant\", \"content\": [{\"type\": \"text\", \"text\": \"**Short answer: not by default, but it's achievable with the right design.**\\n\\nThe key principle: AWS doesn't act as a transit hub between Direct Connect VIFs automatically. Over a VGW or a single DXGW, AWS only advertises **VPC CIDRs** toward each VIF \\u2014 it doesn't re-advertise one on-prem site's prefixes to the other. So two DX sites sharing a gateway don't learn about each other.\\n\\n**It does NOT work when:**\\n- Both VIFs terminate on the same VGW \\u2014 AWS advertises only VPC CIDRs, no site-to-site\\n- Both VIFs are on the same single DXGW \\u2014 gateway associations on one DXGW can't send traffic to each other\\n\\n**It DOES work with these designs:**\\n\\n| Design | How it enables site-to-site |\\n|---|---|\\n| **Two DXGWs into one TGW** (CGW1\\u2013DXGW1\\u2013TGW\\u2013DXGW2\\u2013CGW2) | Each DX VIF attaches via its own DXGW to a shared TGW \\u2014 DX \\u2194 DX works |\\n| **TGW + Allowed Prefix List** | The DXGW\\u2192TGW attachment advertises whatever's in the Allowed Prefix List, not just VPC CIDRs, so remote-site prefixes can be deliberately advertised |\\n| **DX + VPN on the same VGW** | DX site can talk to a VPN site on that VGW (true VPN CloudHub behavior) |\\n| **Customer-side hairpin** | Advertise a supernet/default from on-prem and let your own router be the hub \\u2014 AWS isn't the transit point |\\n\\nSo if your two sites are each on their own DX VIF today and you want them to reach each other through AWS, the standard pattern is: put each VIF behind its own DXGW, attach both DXGWs to a common Transit Gateway, and use an Allowed Prefix List to control what gets advertised between them.\\n\\nWant me to walk through what that TGW route table would look like for your specific setup, or check the Allowed Prefix List config on an existing DXGW?\"}]}", + "createdAt": "2026-10-27T17:05:17.324000-05:00", + "recordType": "final_response" + } +] \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vs-vgw-vif-transit/without_skill/functional-tests-results.json b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vs-vgw-vif-transit/without_skill/functional-tests-results.json new file mode 100644 index 00000000..eb340d96 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vs-vgw-vif-transit/without_skill/functional-tests-results.json @@ -0,0 +1,69 @@ +{ + "version": "v1", + "iteration": 1, + "eval_id": "tgw-vs-vgw-vif-transit", + "run_type": "without_skill", + "success": true, + "error": null, + "understanding_agent_space_skill": null, + "tests": { + "trigger": null, + "expected_output": { + "result": "passed", + "reasoning": "The expected output centers on two key points: (1) by default AWS does NOT act as a transit hub between on-premises sites - it only advertises VPC CIDRs, so private VIFs alone can't achieve site-to-site connectivity; and (2) this behavior can be enabled intentionally via specific configurations (TGW-based routing, or VPN CloudHub for VPN, or DXGWs for DX-to-DX).\n\nThe agent's response captures the core substance of this: it explicitly states \"Private VIFs alone won't do it\" and that \"traffic between two private VIFs doesn't route to each other by default \u2014 there's no path for site-to-site traffic through a VGW\" \u2014 this matches the 'AWS does not act as a transit hub by default' point. It then explains that Transit VIFs + Direct Connect Gateway + Transit Gateway can enable this intentionally, which aligns with the TGW-based solution mentioned in the expected output. It also notes \"this is a deliberate routing decision, not automatic\" which reinforces the default-off behavior.\n\nThe agent doesn't mention VPN CloudHub specifically (since the question was about DX VIFs, not VPN, this omission is reasonable) nor does it explicitly use the term \"Allowed Prefix List,\" but the core architectural guidance (TGW as the hub, DXGW associations) is present and correct. The response's substance aligns well with the expected output's key conclusions: no default transit behavior, intentional enablement via TGW/DXGW routing.\n\nWhile the explanation of \"VGW doesn't route between VIFs\" is slightly imprecise (the real reason is more about default route advertisement behavior), the practical guidance given \u2014 default non-transit behavior, and TGW/DXGW as the enabling mechanism \u2014 is substantively correct and matches the expected output's intent.", + "confidence": "medium" + }, + "assertions": { + "assertion_results": [ + { + "text": "explains AWS does not act as a transit hub by default", + "evaluator": "llm", + "passed": true, + "evidence": "'traffic between two private VIFs doesn't route to each other by default \u2014 there's no path for site-to-site traffic through a VGW' and 'this is a deliberate routing decision, not automatic'", + "reasoning": "The agent explicitly states that AWS does not transit traffic between VIFs by default, requiring deliberate setup (Transit VIF + DXGW + TGW) to enable it.", + "confidence": "high" + }, + { + "text": "mentions VPN CloudHub for VPN connections", + "evaluator": "llm", + "passed": false, + "evidence": "No mention of 'VPN CloudHub' anywhere in the response. The response only discusses Direct Connect VIFs, Direct Connect Gateway, and Transit Gateway.", + "reasoning": "The assertion requires mention of VPN CloudHub specifically for VPN connections, which is entirely absent from the output.", + "confidence": "high" + }, + { + "text": "mentions TGW or two DXGWs as a way to enable DX-to-DX intentionally", + "evaluator": "llm", + "passed": true, + "evidence": "'Transit VIFs + Direct Connect Gateway + Transit Gateway does work... the Transit Gateway can route traffic between the two sites (it acts as the hub)' and 'AWS's hybrid architectures let you use separate Direct Connect Gateways per VIF for traffic isolation between sites, or a shared one if you want them to reach each other'", + "reasoning": "The agent explicitly mentions using a Transit Gateway (TGW) combined with Direct Connect Gateway as the intentional mechanism to enable DX-to-DX communication, satisfying the assertion.", + "confidence": "high" + }, + { + "text": "does not contain 'error'", + "evaluator": "llm", + "passed": true, + "evidence": "No occurrence of the word 'error' found anywhere in the output text.", + "reasoning": "A full scan of the response shows no instance of the word 'error'.", + "confidence": "high" + } + ], + "summary": { + "passed": 3, + "failed": 1, + "errored": 0, + "low_confidence": 0, + "total": 4, + "pass_rate": 0.75 + } + } + }, + "metrics": { + "runtime": "6s", + "cost": "$0.05", + "context_window": { + "utilization": "4.2%", + "compaction_count": 0 + } + } +} \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vs-vgw-vif-transit/without_skill/outputs/journal_records.json b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vs-vgw-vif-transit/without_skill/outputs/journal_records.json new file mode 100644 index 00000000..3a325cb0 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/tgw-vs-vgw-vif-transit/without_skill/outputs/journal_records.json @@ -0,0 +1,90 @@ +[ + { + "agentSpaceId": "ca555d0d-44f0-4b80-94cc-5440c9638400", + "executionId": "d7e39191-959f-45d5-a2dc-a350320b2f2a", + "recordId": "22a79d14-e83f-4e0f-95c3-5ad281054af2", + "content": "{\"id\": \"22a79d14-e83f-4e0f-95c3-5ad281054af2\", \"role\": \"user\", \"content\": [{\"text\": \"Can two on-premises sites on separate Direct Connect VIFs talk to each other through AWS?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:05.150000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "ca555d0d-44f0-4b80-94cc-5440c9638400", + "executionId": "d7e39191-959f-45d5-a2dc-a350320b2f2a", + "recordId": "8435d0e7-0149-4ef1-bcaa-aa81f02f2bd5", + "content": "{\"id\": \"07354b58-c2d2-46c2-ac5a-68ac00fdce2b\", \"role\": \"assistant\", \"content\": [{\"type\": \"chat_title\", \"title\": \"On-premises sites connecting through AWS Direct Connect\"}]}", + "createdAt": "2026-10-27T17:05:05.663000-05:00", + "recordType": "chat_title" + }, + { + "agentSpaceId": "ca555d0d-44f0-4b80-94cc-5440c9638400", + "executionId": "d7e39191-959f-45d5-a2dc-a350320b2f2a", + "recordId": "ffae0119-2518-454a-bf52-0e28c7c02d98", + "content": "{\"id\": \"221ea4db-715b-4d39-9758-8133fd914049\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This is a factual AWS Direct Connect question about whether two on-premises sites on separate VIFs can communicate through AWS, which depends on nuances like private VIFs, Direct Connect Gateway, or Transit Gateway setups. I should verify these specifics against documentation rather than trust memory alone.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:11.589000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "ca555d0d-44f0-4b80-94cc-5440c9638400", + "executionId": "d7e39191-959f-45d5-a2dc-a350320b2f2a", + "recordId": "bd569cf6-0c7a-43de-870f-257297925a77", + "content": "{\"id\": \"bd569cf6-0c7a-43de-870f-257297925a77\", \"role\": \"assistant\", \"content\": [{\"id\": \"tooluse_fmIAS3blu2PzHtMVNJqIPQ\", \"tool_name\": \"verify_aws_claim\", \"input\": {\"claim\": \"Can two on-premises sites connected via separate Direct Connect virtual interfaces communicate with each other through AWS (e.g. via Direct Connect Gateway or Transit Gateway)?\"}, \"type\": \"tool_use\", \"display_intent\": \"Checking AWS docs on Direct Connect site-to-site connectivity via VIFs\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:11.589000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "ca555d0d-44f0-4b80-94cc-5440c9638400", + "executionId": "d7e39191-959f-45d5-a2dc-a350320b2f2a", + "recordId": "fd9dea1f-4f49-4112-a57b-f3b1b98534c6", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 3.8, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:11.661000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "ca555d0d-44f0-4b80-94cc-5440c9638400", + "executionId": "d7e39191-959f-45d5-a2dc-a350320b2f2a", + "recordId": "a5d5e9c2-4d5e-49e4-965f-a9a77d4cd0a8", + "content": "{\"id\": \"7fc7630d-8901-46cf-9ece-26cf9bff92b8\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"1780\"}]}", + "createdAt": "2026-10-27T17:05:11.664000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "ca555d0d-44f0-4b80-94cc-5440c9638400", + "executionId": "d7e39191-959f-45d5-a2dc-a350320b2f2a", + "recordId": "49c0b7b7-fc5c-4541-9431-e9339fe06235", + "content": "{\"id\": \"c7070220-4c92-411e-a8fb-5a3564e27b41\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_fmIAS3blu2PzHtMVNJqIPQ\", \"status\": \"success\", \"content\": [{\"text\": \"## Documentation search results\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"Hybrid cloud architectures using AWS Direct Connect gateway | Networking & Content Delivery\\\",\\\"context\\\":\\\"### Scenario 3: Extend logically segmented routing to on-premises networks for separation between Line of Business (LOB) or tenants\\\\n\\\\nIn this scenario, as depicted in the following diagram (figure 6), a customer has configured four Virtual Routing and Forwarding (VRFs) domains on the on-premises routers while using one Direct Connect connection and four Direct Connect gateways with a dedicated Transit Gateway. This creates end-to-end routing segmentation for each LOB. By provisioning separate Direct Connect gateways with their own Transit VIF, logical segmentation can be extended from a single Transit Gateway up to the customer gateway through the Virtual Interface.\\\\n\\\\n[](https://d2908q01vomqb2.cloudfront.net/5b384ce32d8cdef02bc3a139d4cac0a22bb029e8/2023/10/28/Segmentation.png)\\\\n\\\\nFigure 6: Multiple Direct Connect gateways and Transit VIFs for traffic isolation\\\",\\\"url\\\":\\\"https://aws.amazon.com/blogs/networking-and-content-delivery/hybrid-cloud-architectures-using-aws-direct-connect-gateway/\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"Hybrid Connectivity to AWS Transit Gateway with AWS Direct Connect\\\",\\\"context\\\":\\\"## Hybrid connectivity with AWS Direct Connect architecture\\\\n\\\\nThe following steps describe the on-premises to AWS traffic flow:\\\\n\\\\n1. Traffic from the corporate data center destined to the **spoke VPC A** is forwarded to AWS Transit Gateway through the AWS Direct Connect (DX) link. The Transit Gateway is connected to the Direct Connect link by using a Transit virtual interface (VIF) and a Direct Connect Gateway.\\\\n2. As per the **Transit Gateway on-premises route table**, the traffic is forwarded to the **spoke VPC A** attachment.\\\\n3. The Transit Gateway ENI of the **spoke VPC A** forwards the traffic to the destination.\\\\n\\\\nThe following steps describe the AWS to on-premises traffic flow:\\\\n\\\\n1. Traffic initiated from an Amazon Elastic Compute Cloud instance in the **spoke VPC B** and destined to the corporate data center is routed to the Transit Gateway ENI as per the **spoke VPC B** route table.\\\\n2. Traffic is forwarded to the AWS Transit Gateway. As per the **spoke VPC route table**, the traffic is routed to the office branch through the AWS Direct Connect Gateway attachment.\\\\n3. The traffic is routed to the destination through the AWS Direct Connect link.\\\\n\\\\nFor more information about how to create an AWS Direct Connect connection, see Create a Direct Connect dedicated connection\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/reference-architecture-diagrams/latest/hybrid-connectivity-transit-gateway/hybrid-dx.html\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"Advanced hybrid routing scenarios with AWS Cloud WAN and AWS Direct Connect | Networking & Content Delivery\\\",\\\"context\\\":\\\"### Scenario 3: Two on-premises locations, with two AWS Direct Connect connections each, associated with two AWS Regions. Each on-premises location advertises the same IP prefixes to AWS, and there is connectivity between on-premises locations.\\\\n\\\\nAdvertising the same summary IP prefix from multiple on-premises locations to AWS creates an \\u201canycast\\u201d routing setup. The connectivity between on-premises locations is important in this routing scenario, as it ensures traffic can be routed from AWS to on-premises destinations using the nearest AWS Direct Connect Point of Presence (POP). When needed, traffic can traverse the on-premises backbone to reach the final destination.\\\\n\\\\nWe introduce an additional AWS Direct Connect gateway to control how traffic flows from AWS to on premises:\\\\n\\\\n* We advertise 10.0.0.0/8 \\u2013 IPv4 and 2001:db8::/32 \\u2013 IPv6 from both on-premises locations on all four Transit VIFs. No other more specific routes are advertised from on premises to AWS.\\\\n* We associate VIFs in each AWS Direct Connect POP to a separate Direct Connect gateway \\u2013 VIF-1 and VIF-2 are associated with Direct Connect gateway A, while VIF-3 and VIF-4 are associated with Direct Connect gateway B.\\\\n* We configure BGP local preference communities on routes from on premises to ensure the Direct Connect gateways choose primary/backup VIFs. You can also load balance traffic across both VIFs in an AWS Direct Connect POP by setting identical BGP attributes.\\\\n\\\\nWe call these behavior-based Direct Connect gateways because they accommodate a specific routing configuration for traffic from AWS to on premises. You can define your own desired behavior for each Region through a dedicated behavior-based Direct Connect gateway.\\\\n\\\\nThe setup for this scenario is shown in the following diagram (Figure 9):\\\\n\\\\n[](https://d2908q01vomqb2.cloudfront.net/5b384ce32d8cdef02bc3a139d4cac0a22bb029e8/2024/03/29/Figure9-1-scaled.jpg)\\\\n\\\\nFigure 9: Scenario 3 \\u2013 Multi-Region AWS Cloud WAN setup with a transit gateway on each Region, two Direct Connect gateways, multiple AWS Direct Connect connections associated with two Regions and a backbone network connecting the colocation facilities\\\",\\\"url\\\":\\\"https://aws.amazon.com/blogs/networking-and-content-delivery/advanced-hybrid-routing-scenarios-with-aws-cloud-wan-and-aws-direct-connect/\\\"}]}}\\n\\n## Recent AWS announcements (What's New)\\n\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\n\\\\n\\\\nServices: directconnect, vpc, transitgateway\\\\nTasks: configure, secure, deploy\\\\nPersona: network-admin, devops, developer, sre\\\\nWorkload: networking, hybrid-cloud\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"transitgateway\\\",\\\"skill_description\\\":\\\"Configures AWS Transit Gateway: creating a hub and attaching VPCs, segmenting traffic with route tables, centralizing egress and forcing east-west traffic between VPCs through a central firewall via transit gateway route tables and appliance-mode attachments, connecting on-premises networks over the transit gateway side of a Site-to-Site VPN or Direct Connect attachment (including ECMP to aggregate bandwidth across multiple VPN tunnels), peering transit gateways across Regions, migrating from a VPC peering mesh, and routing IP multicast. Applicable when the transit gateway is the resource being configured: connecting many VPCs, isolating environments, routing egress or east-west traffic through the hub for inspection, reaching on-premises over the hub, linking Regions, or moving off a peering mesh. Not applicable for single-VPC routing, VPC peering between two VPCs (vpcpeering skill), Direct Connect gateway or virtual interface setup (directconnect skill), or Route 53 DNS work.\\\\n\\\\n\\\\nServices: transitgateway, vpc, directconnect, sitetositevpn, gatewayloadbalancer\\\\nTasks: configure, secure, deploy, migrate\\\\nPersona: developer, devops, network-admin, security-engineer, sre\\\\nWorkload: networking, security, hybrid, multi-region\\\\n*\\\",\\\"skill_name\\\":\\\"transitgateway\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"sitetositevpn\\\",\\\"skill_description\\\":\\\"Configures AWS Site-to-Site VPN: creating an IPsec VPN connection between an on-premises network and a VPC, choosing the target gateway (virtual private gateway, transit gateway, or AWS Cloud WAN), choosing static or dynamic (BGP) routing, sizing tunnel bandwidth (Standard 1.25 Gbps or Large 5 Gbps), connecting many sites through a VPN Concentrator, applying the customer gateway device configuration, making a connection highly available, and monitoring tunnels with CloudWatch. Applicable when the user wants to connect a data center or branch office to AWS over an encrypted tunnel, choose how routes are exchanged, scale throughput, consolidate sites, or diagnose a down tunnel. Routes to the right per-task procedure in references. Not for AWS Direct Connect (its own service), Client VPN for individual remote users, the transit gateway side of a VPN attachment (transitgateway skill), or Route 53 DNS work.\\\\n\\\\n\\\\nServices: vpn, vpc, transitgateway, cloudwatch\\\\nTasks: configure, secure, monitor, deploy\\\\nPersona: developer, devops, network-admin, sre\\\\nWorkload: networking, hybrid, disaster-recovery\\\\n*\\\",\\\"skill_name\\\":\\\"sitetositevpn\\\"}]}}\"}]}], \"label\": \"Checking AWS docs on Direct Connect site-to-site connectivity via VIFs\"}", + "createdAt": "2026-10-27T17:05:15.665000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "ca555d0d-44f0-4b80-94cc-5440c9638400", + "executionId": "d7e39191-959f-45d5-a2dc-a350320b2f2a", + "recordId": "c2863daa-c926-49e3-88f9-9fc11dd25574", + "content": "{\"id\": \"c2863daa-c926-49e3-88f9-9fc11dd25574\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_fmIAS3blu2PzHtMVNJqIPQ\", \"content\": \"[{'text': '## Documentation search results\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"Hybrid cloud architectures using AWS Direct Connect gateway | Networking & Content Delivery\\\",\\\"context\\\":\\\"### Scenario 3: Extend logically segmented routing to on-premises networks for separation between Line of Business (LOB) or tenants\\\\\\\\n\\\\\\\\nIn this scenario, as depicted in the following diagram (figure 6), a customer has configured four Virtual Routing and Forwarding (VRFs) domains on the on-premises routers while using one Direct Connect connection and four Direct Connect gateways with a dedicated Transit Gateway. This creates end-to-end routing segmentation for each LOB. By provisioning separate Direct Connect gateways with their own Transit VIF, logical segmentation can be extended from a single Transit Gateway up to the customer gateway through the Virtual Interface.\\\\\\\\n\\\\\\\\n[](https://d2908q01vomqb2.cloudfront.net/5b384ce32d8cdef02bc3a139d4cac0a22bb029e8/2023/10/28/Segmentation.png)\\\\\\\\n\\\\\\\\nFigure 6: Multiple Direct Connect gateways and Transit VIFs for traffic isolation\\\",\\\"url\\\":\\\"https://aws.amazon.com/blogs/networking-and-content-delivery/hybrid-cloud-architectures-using-aws-direct-connect-gateway/\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"Hybrid Connectivity to AWS Transit Gateway with AWS Direct Connect\\\",\\\"context\\\":\\\"## Hybrid connectivity with AWS Direct Connect architecture\\\\\\\\n\\\\\\\\nThe following steps describe the on-premises to AWS traffic flow:\\\\\\\\n\\\\\\\\n1. Traffic from the corporate data center destined to the **spoke VPC A** is forwarded to AWS Transit Gateway through the AWS Direct Connect (DX) link. The Transit Gateway is connected to the Direct Connect link by using a Transit virtual interface (VIF) and a Direct Connect Gateway.\\\\\\\\n2. As per the **Transit Gateway on-premises route table**, the traffic is forwarded to the **spoke VPC A** attachment.\\\\\\\\n3. The Transit Gateway ENI of the **spoke VPC A** forwards the traffic to the destination.\\\\\\\\n\\\\\\\\nThe following steps describe the AWS to on-premises traffic flow:\\\\\\\\n\\\\\\\\n1. Traffic initiated from an Amazon Elastic Compute Cloud instance in the **spoke VPC B** and destined to the corporate data center is routed to the Transit Gateway ENI as per the **spoke VPC B** route table.\\\\\\\\n2. Traffic is forwarded to the AWS Transit Gateway. As per the **spoke VPC route table**, the traffic is routed to the office branch through the AWS Direct Connect Gateway attachment.\\\\\\\\n3. The traffic is routed to the destination through the AWS Direct Connect link.\\\\\\\\n\\\\\\\\nFor more information about how to create an AWS Direct Connect connection, see Create a Direct Connect dedicated connection\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/reference-architecture-diagrams/latest/hybrid-connectivity-transit-gateway/hybrid-dx.html\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"Advanced hybrid routing scenarios with AWS Cloud WAN and AWS Direct Connect | Networking & Content Delivery\\\",\\\"context\\\":\\\"### Scenario 3: Two on-premises locations, with two AWS Direct Connect connections each, associated with two AWS Regions. Each on-premises location advertises the same IP prefixes to AWS, and there is connectivity between on-premises locations.\\\\\\\\n\\\\\\\\nAdvertising the same summary IP prefix from multiple on-premises locations to AWS creates an \\u201canycast\\u201d routing setup. The connectivity between on-premises locations is important in this routing scenario, as it ensures traffic can be routed from AWS to on-premises destinations using the nearest AWS Direct Connect Point of Presence (POP). When needed, traffic can traverse the on-premises backbone to reach the final destination.\\\\\\\\n\\\\\\\\nWe introduce an additional AWS Direct Connect gateway to control how traffic flows from AWS to on premises:\\\\\\\\n\\\\\\\\n* We advertise 10.0.0.0/8 \\u2013 IPv4 and 2001:db8::/32 \\u2013 IPv6 from both on-premises locations on all four Transit VIFs. No other more specific routes are advertised from on premises to AWS.\\\\\\\\n* We associate VIFs in each AWS Direct Connect POP to a separate Direct Connect gateway \\u2013 VIF-1 and VIF-2 are associated with Direct Connect gateway A, while VIF-3 and VIF-4 are associated with Direct Connect gateway B.\\\\\\\\n* We configure BGP local preference communities on routes from on premises to ensure the Direct Connect gateways choose primary/backup VIFs. You can also load balance traffic across both VIFs in an AWS Direct Connect POP by setting identical BGP attributes.\\\\\\\\n\\\\\\\\nWe call these behavior-based Direct Connect gateways because they accommodate a specific routing configuration for traffic from AWS to on premises. You can define your own desired behavior for each Region through a dedicated behavior-based Direct Connect gateway.\\\\\\\\n\\\\\\\\nThe setup for this scenario is shown in the following diagram (Figure 9):\\\\\\\\n\\\\\\\\n[](https://d2908q01vomqb2.cloudfront.net/5b384ce32d8cdef02bc3a139d4cac0a22bb029e8/2024/03/29/Figure9-1-scaled.jpg)\\\\\\\\n\\\\\\\\nFigure 9: Scenario 3 \\u2013 Multi-Region AWS Cloud WAN setup with a transit gateway on each Region, two Direct Connect gateways, multiple AWS Direct Connect connections associated with two Regions and a backbone network connecting the colocation facilities\\\",\\\"url\\\":\\\"https://aws.amazon.com/blogs/networking-and-content-delivery/advanced-hybrid-routing-scenarios-with-aws-cloud-wan-and-aws-direct-connect/\\\"}]}}\\\\n\\\\n## Recent AWS announcements (What\\\\'s New)\\\\n\\\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: directconnect, vpc, transitgateway\\\\\\\\nTasks: configure, secure, deploy\\\\\\\\nPersona: network-admin, devops, developer, sre\\\\\\\\nWorkload: networking, hybrid-cloud\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"transitgateway\\\",\\\"skill_description\\\":\\\"Configures AWS Transit Gateway: creating a hub and attaching VPCs, segmenting traffic with route tables, centralizing egress and forcing east-west traffic between VPCs through a central firewall via transit gateway route tables and appliance-mode attachments, connecting on-premises networks over the transit gateway side of a Site-to-Site VPN or Direct Connect attachment (including ECMP to aggregate bandwidth across multiple VPN tunnels), peering transit gateways across Regions, migrating from a VPC peering mesh, and routing IP multicast. Applicable when the transit gateway is the resource being configured: connecting many VPCs, isolating environments, routing egress or east-west traffic through the hub for inspection, reaching on-premises over the hub, linking Regions, or moving off a peering mesh. Not applicable for single-VPC routing, VPC peering between two VPCs (vpcpeering skill), Direct Connect gateway or virtual interface setup (directconnect skill), or Route 53 DNS work.\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: transitgateway, vpc, directconnect, sitetositevpn, gatewayloadbalancer\\\\\\\\nTasks: configure, secure, deploy, migrate\\\\\\\\nPersona: developer, devops, network-admin, security-engineer, sre\\\\\\\\nWorkload: networking, security, hybrid, multi-region\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"transitgateway\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"sitetositevpn\\\",\\\"skill_description\\\":\\\"Configures AWS Site-to-Site VPN: creating an IPsec VPN connection between an on-premises network and a VPC, choosing the target gateway (virtual private gateway, transit gateway, or AWS Cloud WAN), choosing static or dynamic (BGP) routing, sizing tunnel bandwidth (Standard 1.25 Gbps or Large 5 Gbps), connecting many sites through a VPN Concentrator, applying the customer gateway device configuration, making a connection highly available, and monitoring tunnels with CloudWatch. Applicable when the user wants to connect a data center or branch office to AWS over an encrypted tunnel, choose how routes are exchanged, scale throughput, consolidate sites, or diagnose a down tunnel. Routes to the right per-task procedure in references. Not for AWS Direct Connect (its own service), Client VPN for individual remote users, the transit gateway side of a VPN attachment (transitgateway skill), or Route 53 DNS work.\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: vpn, vpc, transitgateway, cloudwatch\\\\\\\\nTasks: configure, secure, monitor, deploy\\\\\\\\nPersona: developer, devops, network-admin, sre\\\\\\\\nWorkload: networking, hybrid, disaster-recovery\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"sitetositevpn\\\"}]}}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:21.877000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "ca555d0d-44f0-4b80-94cc-5440c9638400", + "executionId": "d7e39191-959f-45d5-a2dc-a350320b2f2a", + "recordId": "6a55967f-adbe-4a58-9b47-995483afec76", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.2, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:21.966000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "ca555d0d-44f0-4b80-94cc-5440c9638400", + "executionId": "d7e39191-959f-45d5-a2dc-a350320b2f2a", + "recordId": "10d36a7c-4002-433e-8af0-1df675e2ee63", + "content": "{\"id\": \"10d36a7c-4002-433e-8af0-1df675e2ee63\", \"role\": \"assistant\", \"content\": [{\"text\": \"Yes \\u2014 but it depends on how the VIFs are set up and routed:\\n\\n- **Private VIFs alone won't do it.** A private VIF connects to a single virtual private gateway (VGW)/VPC, and traffic between two private VIFs doesn't route to each other by default \\u2014 there's no path for site-to-site traffic through a VGW.\\n- **Transit VIFs + Direct Connect Gateway + Transit Gateway does work.** If each on-premises site connects via a Transit VIF to a Direct Connect Gateway, and that gateway is associated with a Transit Gateway, the Transit Gateway can route traffic between the two sites (it acts as the hub). This is the standard hybrid connectivity pattern AWS documents for site-to-site communication through AWS.\\n- **You can also segment or combine behavior.** AWS's hybrid architectures let you use separate Direct Connect Gateways per VIF for traffic isolation between sites, or a shared one if you want them to reach each other \\u2014 so this is a deliberate routing decision, not automatic.\\n\\nSo the short answer: two private VIFs by themselves \\u2014 no. Two sites on Transit VIFs tied into the same Direct Connect Gateway/Transit Gateway \\u2014 yes, AWS becomes the transit hub between them.\\n\\nWant me to walk through what this would look like for your specific setup (e.g., check your actual Direct Connect/Transit Gateway associations in this account)?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:22.036000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "ca555d0d-44f0-4b80-94cc-5440c9638400", + "executionId": "d7e39191-959f-45d5-a2dc-a350320b2f2a", + "recordId": "5ece8999-4cfe-40de-8afa-51c2e6992958", + "content": "{\"id\": \"a49e9337-b747-4eeb-9ef5-aec121625de4\", \"role\": \"assistant\", \"content\": [{\"type\": \"text\", \"text\": \"Yes \\u2014 but it depends on how the VIFs are set up and routed:\\n\\n- **Private VIFs alone won't do it.** A private VIF connects to a single virtual private gateway (VGW)/VPC, and traffic between two private VIFs doesn't route to each other by default \\u2014 there's no path for site-to-site traffic through a VGW.\\n- **Transit VIFs + Direct Connect Gateway + Transit Gateway does work.** If each on-premises site connects via a Transit VIF to a Direct Connect Gateway, and that gateway is associated with a Transit Gateway, the Transit Gateway can route traffic between the two sites (it acts as the hub). This is the standard hybrid connectivity pattern AWS documents for site-to-site communication through AWS.\\n- **You can also segment or combine behavior.** AWS's hybrid architectures let you use separate Direct Connect Gateways per VIF for traffic isolation between sites, or a shared one if you want them to reach each other \\u2014 so this is a deliberate routing decision, not automatic.\\n\\nSo the short answer: two private VIFs by themselves \\u2014 no. Two sites on Transit VIFs tied into the same Direct Connect Gateway/Transit Gateway \\u2014 yes, AWS becomes the transit hub between them.\\n\\nWant me to walk through what this would look like for your specific setup (e.g., check your actual Direct Connect/Transit Gateway associations in this account)?\"}]}", + "createdAt": "2026-10-27T17:05:22.166000-05:00", + "recordType": "final_response" + } +] \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/verification-unknown-source/with_skill/functional-tests-results.json b/skills/aws-routing/evals/functional/v1/iteration-1/verification-unknown-source/with_skill/functional-tests-results.json new file mode 100644 index 00000000..c55dd376 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/verification-unknown-source/with_skill/functional-tests-results.json @@ -0,0 +1,66 @@ +{ + "version": "v1", + "iteration": 1, + "eval_id": "verification-unknown-source", + "run_type": "with_skill", + "success": true, + "error": null, + "understanding_agent_space_skill": null, + "tests": { + "trigger": { + "result": "failed", + "message": "Skill 'aws-routing' was NOT triggered (no skills were loaded)", + "skill_loads_found": 0 + }, + "expected_output": { + "result": "passed", + "reasoning": "The agent explicitly states it does not find a document titled \"DX Path Selection Flow Chart\" in AWS official documentation, avoiding fabrication of a specific named artifact. It then points to the Direct Connect routing policies and BGP communities documentation (routing-and-bgp.html) as the authoritative reference for path-selection order (longest prefix match, local preference, AS_PATH length, MED, ECMP), which matches the expected behavior. It also provides supplementary real sources (a blog post and a re:Post article) without claiming they are the exact named document, and is transparent about there being no single named flow chart doc. This aligns well with the expected output criteria.", + "evidence": "\"I don't see a document specifically titled \\\"DX Path Selection Flow Chart\\\" in AWS's official documentation... Direct Connect routing policies and BGP communities \u2014 the authoritative reference. It lays out the evaluation order AWS uses for outbound path selection: longest prefix match first, then local preference..., then AS_PATH length, then MED, with ECMP applied when everything ties.\"", + "confidence": "high" + }, + "assertions": { + "assertion_results": [ + { + "text": "does not fabricate a document or URL", + "evaluator": "llm", + "passed": true, + "evidence": "The agent says 'I don't see a document specifically titled \"DX Path Selection Flow Chart\"' and instead cites real, identifiable AWS resources: docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html, the AWS blog post on Longest Prefix Match, and the re:Post knowledge center article.", + "reasoning": "The agent provides real, plausible URLs to AWS documentation, a known AWS blog post, and a re:Post knowledge center article, all of which correspond to real AWS content types. It explicitly states it does not find a document with the exact requested title, rather than inventing one.", + "confidence": "medium" + }, + { + "text": "states it cannot verify such a named artifact", + "evaluator": "llm", + "passed": true, + "evidence": "\"I don't see a document specifically titled \\\"DX Path Selection Flow Chart\\\" in AWS's official documentation. There's no single named flow chart doc for this\"", + "reasoning": "The agent explicitly states it cannot find a document with that exact name/title, directly addressing the user's question about a specifically named artifact.", + "confidence": "high" + }, + { + "text": "points to the Direct Connect routing policies documentation", + "evaluator": "llm", + "passed": true, + "evidence": "\"**[Direct Connect routing policies and BGP communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html)** \u2014 the authoritative reference. It lays out the evaluation order AWS uses for outbound path selection...\"", + "reasoning": "The agent explicitly links to and describes the Direct Connect routing policies and BGP communities documentation as the authoritative reference.", + "confidence": "high" + } + ], + "summary": { + "passed": 3, + "failed": 0, + "errored": 0, + "low_confidence": 0, + "total": 3, + "pass_rate": 1.0 + } + } + }, + "metrics": { + "runtime": "7s", + "cost": "$0.07", + "context_window": { + "utilization": "4.2%", + "compaction_count": 0 + } + } +} \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/verification-unknown-source/with_skill/outputs/journal_records.json b/skills/aws-routing/evals/functional/v1/iteration-1/verification-unknown-source/with_skill/outputs/journal_records.json new file mode 100644 index 00000000..40fbaf38 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/verification-unknown-source/with_skill/outputs/journal_records.json @@ -0,0 +1,90 @@ +[ + { + "agentSpaceId": "b3203492-a72d-48c3-9753-46d4c1f04992", + "executionId": "3ed8b3f3-d97b-424a-bef1-dd7a6727039d", + "recordId": "4bb93c91-24d6-4250-8c80-5b13254111b8", + "content": "{\"id\": \"4bb93c91-24d6-4250-8c80-5b13254111b8\", \"role\": \"user\", \"content\": [{\"text\": \"Is there an official AWS 'DX Path Selection Flow Chart' document I can cite?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:21.835000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "b3203492-a72d-48c3-9753-46d4c1f04992", + "executionId": "3ed8b3f3-d97b-424a-bef1-dd7a6727039d", + "recordId": "33d751d0-028b-4a0b-9055-1223ac7779b0", + "content": "{\"id\": \"81262987-b4d2-4da1-bd98-f0dadf56813c\", \"role\": \"assistant\", \"content\": [{\"type\": \"chat_title\", \"title\": \"Finding AWS DX Path Selection Flow Chart\"}]}", + "createdAt": "2026-10-27T17:05:22.298000-05:00", + "recordType": "chat_title" + }, + { + "agentSpaceId": "b3203492-a72d-48c3-9753-46d4c1f04992", + "executionId": "3ed8b3f3-d97b-424a-bef1-dd7a6727039d", + "recordId": "1940cbf3-b949-4f5f-8c1c-506916fe9c1f", + "content": "{\"id\": \"1940cbf3-b949-4f5f-8c1c-506916fe9c1f\", \"role\": \"assistant\", \"content\": [{\"id\": \"tooluse_eogW5l4gPkBuZPdhVJ3vQw\", \"tool_name\": \"verify_aws_claim\", \"input\": {\"claim\": \"AWS Direct Connect Path Selection Flow Chart official documentation\"}, \"type\": \"tool_use\", \"display_intent\": \"Checking AWS docs for an official Direct Connect path selection flow chart\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:26.840000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "b3203492-a72d-48c3-9753-46d4c1f04992", + "executionId": "3ed8b3f3-d97b-424a-bef1-dd7a6727039d", + "recordId": "f5062b8e-28b9-4aeb-928d-402c03e8b964", + "content": "{\"id\": \"6e492ca6-bea0-429e-b6ae-457b07a66fcf\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"The user's asking about an official AWS document, the \\\"DX Path Selection Flow Chart\\\" for Direct Connect routing. This is a straightforward factual verification, so I should use verify_aws_claim directly rather than delegating it elsewhere.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:26.841000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "b3203492-a72d-48c3-9753-46d4c1f04992", + "executionId": "3ed8b3f3-d97b-424a-bef1-dd7a6727039d", + "recordId": "4318e18e-12d5-4008-8d30-67278e03ef62", + "content": "{\"id\": \"42fb05e8-ccbc-4549-afea-e54b4d52b089\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"901\"}]}", + "createdAt": "2026-10-27T17:05:26.917000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "b3203492-a72d-48c3-9753-46d4c1f04992", + "executionId": "3ed8b3f3-d97b-424a-bef1-dd7a6727039d", + "recordId": "b01da708-fb37-4abe-a07c-bfd921336bb3", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 3.8, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:26.942000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "b3203492-a72d-48c3-9753-46d4c1f04992", + "executionId": "3ed8b3f3-d97b-424a-bef1-dd7a6727039d", + "recordId": "acf5d83f-e971-4309-96da-5351f155004f", + "content": "{\"id\": \"40bf617c-7f15-4175-a76e-ec070b07fe92\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_eogW5l4gPkBuZPdhVJ3vQw\", \"status\": \"success\", \"content\": [{\"text\": \"## Documentation search results\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"How do I manage asymmetric routing with Direct Connect?\\\",\\\"context\\\":\\\"Resolution\\\\n----------\\\\n\\\\nThe following resolution explains how to manage outbound traffic flow from AWS to your on-premises network. For inbound traffic flow from your on-premises network to AWS, you must use preferred routing options to set the preference to a particular path.\\\\n\\\\nTo manage asymmetric routing with Direct Connect, use prefix length and Border Gateway Protocol (BGP) attributes to determine path selection for different types of virtual interfaces.\\\\n\\\\n**Note:** For public virtual interfaces, AWS uses AS\\\\\\\\_PATH and the [longest prefix match](https://aws.amazon.com/blogs/networking-and-content-delivery/influencing-traffic-over-hybrid-networks-using-longest-prefix-match/) to determine the routing path\\\",\\\"url\\\":\\\"https://repost.aws/knowledge-center/direct-connect-manage-asymmetric-routing\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"Direct Connect routing policies and BGP communities\\\",\\\"context\\\":\\\"## Private virtual interface and transit virtual interface routing policies\\\\n\\\\nIf you're using AWS Direct Connect to access your private AWS resources,\\\\nyou must specify the IPv4 or IPv6 prefixes to advertise over BGP. These prefixes can be\\\\npublic or private.\\\\n\\\\nThe following outbound routing rules apply based on the prefixes advertised:\\\\n\\\\n* AWS evaluates the longest prefix length first. AWS recommends advertising\\\\n more specific routes using multiple Direct Connect virtual interfaces if the\\\\n desired routing paths are meant for active/passive connections. See Influencing Traffic over Hybrid Networks using Longest Prefix\\\\n Match for more information.\\\\n* Local preference is the BGP attribute recommended to use when desired routing\\\\n paths are meant for active/passive connections and the prefix lengths advertised\\\\n are the same. This value is set per Region to prefer Direct\\\\n Connect locations that have the same associated AWS Region using the\\\\n `7224:7200`\\u2014Medium local preference community value. Where\\\\n the local Region is not associated with the Direct Connect\\\\n location, it is set to a lower value. This applies only if no local preference\\\\n community tags are assigned. For more information about associated Regions, see\\\\n Associated Region.\\\\n* AS_PATH length can be used to determine the routing path when the prefix\\\\n length and local preference are the same.\\\\n* Multi-Exit Discriminator (MED) can be used to determine the routing path when\\\\n prefix length, local preference, and AS_PATH are the same. AWS does not\\\\n recommend using MED values given their lower priority in evaluation.\\\\n* AWS uses equal-cost multi-path (ECMP) routing across multiple transit or\\\\n private virtual interfaces when prefixes have the same AS_PATH length and BGP\\\\n attributes. The ASNs in the AS_PATH of the prefixes do not need to match\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"Influencing Traffic over Hybrid Networks using Longest Prefix Match | Networking & Content Delivery\\\",\\\"context\\\":\\\"### Data Center to AWS\\\\n\\\\nWe can also use LPM to influence traffic in the reverse direction. Let\\u2019s suppose we want traffic originating from PHX to reach a destination resource in US-West-1 (10.0.0.0/16) with the local Direct Connect as the primary path and the MPLS as the secondary. The destination resource is in a VPC with IP address of 10.0.0.101.\\\\n\\\\nIn this case, we want the routes advertised from the PHX Direct Connect Gateway to be more specific than the routes advertised across MPLS. To do this, we go to the AWS console and enter 10.0.0.0/17 and 10.0.128.17 as the IP prefixes to be advertised from AWS to PHX via Allowed prefixes under Direct Connect gateway association as shown in Figure 12:\\\\n\\\\nFigure 12: Use LPM to prefer local Direct Connect as Primary to US-West-1\\\\n\\\\nThe resulting route table in PHX customer router would then look as shown in Table 4:\\\\n\\\\n| | |\\\\n| --- | --- |\\\\n| **Destination (Prefix)** | **Target (Next Hop)** |\\\\n| 10.0.0.0/17 | Direct Connect |\\\\n| 10.0.128.0/17 | Direct Connect |\\\\n| 10.0.0.0/16 | MPLS |\\\\n\\\\nTable 4: PHX Route Table\\\\n\\\\nBecause the routes using the /17 prefix are more specific, the PHX router will use its local Direct Connect as its primary path to reach the 10.0.0.101 destination in US-West-1. If the local Direct Connect in PHX is disrupted, the PHX router will failover to the next matching path (10.0.0.0/16) over MPLS, and then through US-East-1 to reach its destination in US-West-1 as shown on Figure 13:\\\\n\\\\nFigure 13: Failover to MPLS to reach US-West-1\\\\n\\\\nThe return traffic from US-West-1 would follow the same path in the reverse direction as shown in Figure 14:\\\\n\\\\nFigure 14: Return Traffic from US-West-1 to PHX\\\\n\\\\nOne thing to remember is that BGP is used to advertise routes between the AWS and the data centers but not between the Transit Gateway peers. Therefore, static routes need to be manually entered into each Transit Gateway in order to route traffic between Transit Gateway peers\\\",\\\"url\\\":\\\"https://aws.amazon.com/blogs/networking-and-content-delivery/influencing-traffic-over-hybrid-networks-using-longest-prefix-match/\\\"}]}}\\n\\n## Recent AWS announcements (What's New)\\n\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\n\\\\n\\\\nServices: directconnect, vpc, transitgateway\\\\nTasks: configure, secure, deploy\\\\nPersona: network-admin, devops, developer, sre\\\\nWorkload: networking, hybrid-cloud\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"sitetositevpn\\\",\\\"skill_description\\\":\\\"Configures AWS Site-to-Site VPN: creating an IPsec VPN connection between an on-premises network and a VPC, choosing the target gateway (virtual private gateway, transit gateway, or AWS Cloud WAN), choosing static or dynamic (BGP) routing, sizing tunnel bandwidth (Standard 1.25 Gbps or Large 5 Gbps), connecting many sites through a VPN Concentrator, applying the customer gateway device configuration, making a connection highly available, and monitoring tunnels with CloudWatch. Applicable when the user wants to connect a data center or branch office to AWS over an encrypted tunnel, choose how routes are exchanged, scale throughput, consolidate sites, or diagnose a down tunnel. Routes to the right per-task procedure in references. Not for AWS Direct Connect (its own service), Client VPN for individual remote users, the transit gateway side of a VPN attachment (transitgateway skill), or Route 53 DNS work.\\\\n\\\\n\\\\nServices: vpn, vpc, transitgateway, cloudwatch\\\\nTasks: configure, secure, monitor, deploy\\\\nPersona: developer, devops, network-admin, sre\\\\nWorkload: networking, hybrid, disaster-recovery\\\\n*\\\",\\\"skill_name\\\":\\\"sitetositevpn\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"aws-networking\\\",\\\"skill_description\\\":\\\"Routes AWS networking requests to the correct service skill for implementation. Covers Route 53 (DNS, health checks, routing policies, Resolver, DNS Firewall), CloudFront (caching, edge, OAC, mTLS, signed URLs), Transit Gateway (multi-VPC hub, segmentation, centralized egress), Direct Connect (hybrid link, DX Gateway, MACsec), Site-to-Site VPN (IPsec tunnels, static or BGP), WAF (web ACLs, AWS Managed Rules, rate-based rules, Bot and Fraud Control), and Shield Advanced (L3/L4 DDoS). Applicable when creating, configuring, troubleshooting, or designing across these services, choosing between them, or diagnosing connectivity or traffic-filtering issues. Not for VPC subnets and route tables, load balancers, VPC endpoints, PrivateLink, API Gateway, IAM policy logic, container or serverless networking, or IaC authoring.\\\\n\\\\nServices: route53, route53resolver, cloudfront, directconnect, transitgateway, vpn, waf, shield\\\\nTasks: configure, troubleshoot, debug, deploy, secure, migrate\\\\nPersona: developer, network-engineer, devops, security-engineer\\\\nWorkload: networking, security\\\\n*\\\",\\\"skill_name\\\":\\\"aws-networking\\\"}]}}\"}]}], \"label\": \"Checking AWS docs for an official Direct Connect path selection flow chart\"}", + "createdAt": "2026-10-27T17:05:28.535000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "b3203492-a72d-48c3-9753-46d4c1f04992", + "executionId": "3ed8b3f3-d97b-424a-bef1-dd7a6727039d", + "recordId": "21ca960d-9a83-4789-9b90-12f8ea54052c", + "content": "{\"id\": \"21ca960d-9a83-4789-9b90-12f8ea54052c\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_eogW5l4gPkBuZPdhVJ3vQw\", \"content\": \"[{'text': '## Documentation search results\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"How do I manage asymmetric routing with Direct Connect?\\\",\\\"context\\\":\\\"Resolution\\\\\\\\n----------\\\\\\\\n\\\\\\\\nThe following resolution explains how to manage outbound traffic flow from AWS to your on-premises network. For inbound traffic flow from your on-premises network to AWS, you must use preferred routing options to set the preference to a particular path.\\\\\\\\n\\\\\\\\nTo manage asymmetric routing with Direct Connect, use prefix length and Border Gateway Protocol (BGP) attributes to determine path selection for different types of virtual interfaces.\\\\\\\\n\\\\\\\\n**Note:** For public virtual interfaces, AWS uses AS\\\\\\\\\\\\\\\\_PATH and the [longest prefix match](https://aws.amazon.com/blogs/networking-and-content-delivery/influencing-traffic-over-hybrid-networks-using-longest-prefix-match/) to determine the routing path\\\",\\\"url\\\":\\\"https://repost.aws/knowledge-center/direct-connect-manage-asymmetric-routing\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"Direct Connect routing policies and BGP communities\\\",\\\"context\\\":\\\"## Private virtual interface and transit virtual interface routing policies\\\\\\\\n\\\\\\\\nIf you\\\\'re using AWS Direct Connect to access your private AWS resources,\\\\\\\\nyou must specify the IPv4 or IPv6 prefixes to advertise over BGP. These prefixes can be\\\\\\\\npublic or private.\\\\\\\\n\\\\\\\\nThe following outbound routing rules apply based on the prefixes advertised:\\\\\\\\n\\\\\\\\n* AWS evaluates the longest prefix length first. AWS recommends advertising\\\\\\\\n more specific routes using multiple Direct Connect virtual interfaces if the\\\\\\\\n desired routing paths are meant for active/passive connections. See Influencing Traffic over Hybrid Networks using Longest Prefix\\\\\\\\n Match for more information.\\\\\\\\n* Local preference is the BGP attribute recommended to use when desired routing\\\\\\\\n paths are meant for active/passive connections and the prefix lengths advertised\\\\\\\\n are the same. This value is set per Region to prefer Direct\\\\\\\\n Connect locations that have the same associated AWS Region using the\\\\\\\\n `7224:7200`\\u2014Medium local preference community value. Where\\\\\\\\n the local Region is not associated with the Direct Connect\\\\\\\\n location, it is set to a lower value. This applies only if no local preference\\\\\\\\n community tags are assigned. For more information about associated Regions, see\\\\\\\\n Associated Region.\\\\\\\\n* AS_PATH length can be used to determine the routing path when the prefix\\\\\\\\n length and local preference are the same.\\\\\\\\n* Multi-Exit Discriminator (MED) can be used to determine the routing path when\\\\\\\\n prefix length, local preference, and AS_PATH are the same. AWS does not\\\\\\\\n recommend using MED values given their lower priority in evaluation.\\\\\\\\n* AWS uses equal-cost multi-path (ECMP) routing across multiple transit or\\\\\\\\n private virtual interfaces when prefixes have the same AS_PATH length and BGP\\\\\\\\n attributes. The ASNs in the AS_PATH of the prefixes do not need to match\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"Influencing Traffic over Hybrid Networks using Longest Prefix Match | Networking & Content Delivery\\\",\\\"context\\\":\\\"### Data Center to AWS\\\\\\\\n\\\\\\\\nWe can also use LPM to influence traffic in the reverse direction. Let\\u2019s suppose we want traffic originating from PHX to reach a destination resource in US-West-1 (10.0.0.0/16) with the local Direct Connect as the primary path and the MPLS as the secondary. The destination resource is in a VPC with IP address of 10.0.0.101.\\\\\\\\n\\\\\\\\nIn this case, we want the routes advertised from the PHX Direct Connect Gateway to be more specific than the routes advertised across MPLS. To do this, we go to the AWS console and enter 10.0.0.0/17 and 10.0.128.17 as the IP prefixes to be advertised from AWS to PHX via Allowed prefixes under Direct Connect gateway association as shown in Figure 12:\\\\\\\\n\\\\\\\\nFigure 12: Use LPM to prefer local Direct Connect as Primary to US-West-1\\\\\\\\n\\\\\\\\nThe resulting route table in PHX customer router would then look as shown in Table 4:\\\\\\\\n\\\\\\\\n| | |\\\\\\\\n| --- | --- |\\\\\\\\n| **Destination (Prefix)** | **Target (Next Hop)** |\\\\\\\\n| 10.0.0.0/17 | Direct Connect |\\\\\\\\n| 10.0.128.0/17 | Direct Connect |\\\\\\\\n| 10.0.0.0/16 | MPLS |\\\\\\\\n\\\\\\\\nTable 4: PHX Route Table\\\\\\\\n\\\\\\\\nBecause the routes using the /17 prefix are more specific, the PHX router will use its local Direct Connect as its primary path to reach the 10.0.0.101 destination in US-West-1. If the local Direct Connect in PHX is disrupted, the PHX router will failover to the next matching path (10.0.0.0/16) over MPLS, and then through US-East-1 to reach its destination in US-West-1 as shown on Figure 13:\\\\\\\\n\\\\\\\\nFigure 13: Failover to MPLS to reach US-West-1\\\\\\\\n\\\\\\\\nThe return traffic from US-West-1 would follow the same path in the reverse direction as shown in Figure 14:\\\\\\\\n\\\\\\\\nFigure 14: Return Traffic from US-West-1 to PHX\\\\\\\\n\\\\\\\\nOne thing to remember is that BGP is used to advertise routes between the AWS and the data centers but not between the Transit Gateway peers. Therefore, static routes need to be manually entered into each Transit Gateway in order to route traffic between Transit Gateway peers\\\",\\\"url\\\":\\\"https://aws.amazon.com/blogs/networking-and-content-delivery/influencing-traffic-over-hybrid-networks-using-longest-prefix-match/\\\"}]}}\\\\n\\\\n## Recent AWS announcements (What\\\\'s New)\\\\n\\\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: directconnect, vpc, transitgateway\\\\\\\\nTasks: configure, secure, deploy\\\\\\\\nPersona: network-admin, devops, developer, sre\\\\\\\\nWorkload: networking, hybrid-cloud\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"sitetositevpn\\\",\\\"skill_description\\\":\\\"Configures AWS Site-to-Site VPN: creating an IPsec VPN connection between an on-premises network and a VPC, choosing the target gateway (virtual private gateway, transit gateway, or AWS Cloud WAN), choosing static or dynamic (BGP) routing, sizing tunnel bandwidth (Standard 1.25 Gbps or Large 5 Gbps), connecting many sites through a VPN Concentrator, applying the customer gateway device configuration, making a connection highly available, and monitoring tunnels with CloudWatch. Applicable when the user wants to connect a data center or branch office to AWS over an encrypted tunnel, choose how routes are exchanged, scale throughput, consolidate sites, or diagnose a down tunnel. Routes to the right per-task procedure in references. Not for AWS Direct Connect (its own service), Client VPN for individual remote users, the transit gateway side of a VPN attachment (transitgateway skill), or Route 53 DNS work.\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: vpn, vpc, transitgateway, cloudwatch\\\\\\\\nTasks: configure, secure, monitor, deploy\\\\\\\\nPersona: developer, devops, network-admin, sre\\\\\\\\nWorkload: networking, hybrid, disaster-recovery\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"sitetositevpn\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"aws-networking\\\",\\\"skill_description\\\":\\\"Routes AWS networking requests to the correct service skill for implementation. Covers Route 53 (DNS, health checks, routing policies, Resolver, DNS Firewall), CloudFront (caching, edge, OAC, mTLS, signed URLs), Transit Gateway (multi-VPC hub, segmentation, centralized egress), Direct Connect (hybrid link, DX Gateway, MACsec), Site-to-Site VPN (IPsec tunnels, static or BGP), WAF (web ACLs, AWS Managed Rules, rate-based rules, Bot and Fraud Control), and Shield Advanced (L3/L4 DDoS). Applicable when creating, configuring, troubleshooting, or designing across these services, choosing between them, or diagnosing connectivity or traffic-filtering issues. Not for VPC subnets and route tables, load balancers, VPC endpoints, PrivateLink, API Gateway, IAM policy logic, container or serverless networking, or IaC authoring.\\\\\\\\n\\\\\\\\nServices: route53, route53resolver, cloudfront, directconnect, transitgateway, vpn, waf, shield\\\\\\\\nTasks: configure, troubleshoot, debug, deploy, secure, migrate\\\\\\\\nPersona: developer, network-engineer, devops, security-engineer\\\\\\\\nWorkload: networking, security\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"aws-networking\\\"}]}}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:36.171000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "b3203492-a72d-48c3-9753-46d4c1f04992", + "executionId": "3ed8b3f3-d97b-424a-bef1-dd7a6727039d", + "recordId": "210f16da-e7e3-4f65-8323-ec1f6b8db023", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.2, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:36.255000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "b3203492-a72d-48c3-9753-46d4c1f04992", + "executionId": "3ed8b3f3-d97b-424a-bef1-dd7a6727039d", + "recordId": "722f68fa-f604-4a5a-a106-11ace4dac158", + "content": "{\"id\": \"722f68fa-f604-4a5a-a106-11ace4dac158\", \"role\": \"assistant\", \"content\": [{\"text\": \"I don't see a document specifically titled \\\"DX Path Selection Flow Chart\\\" in AWS's official documentation. There's no single named flow chart doc for this \\u2014 AWS instead spreads the path-selection logic across a few sources:\\n\\n- **[Direct Connect routing policies and BGP communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html)** \\u2014 the authoritative reference. It lays out the evaluation order AWS uses for outbound path selection: longest prefix match first, then local preference (BGP community `7224:7200` for same-Region DX locations), then AS_PATH length, then MED, with ECMP applied when everything ties.\\n- **[Influencing Traffic over Hybrid Networks using Longest Prefix Match](https://aws.amazon.com/blogs/networking-and-content-delivery/influencing-traffic-over-hybrid-networks-using-longest-prefix-match/)** \\u2014 a Networking & Content Delivery blog post that does include actual flow diagrams/figures (e.g., Figure 12\\u201314) showing primary vs. failover path selection, which is probably the closest thing to a visual \\\"flow chart\\\" AWS publishes.\\n- **[How do I manage asymmetric routing with Direct Connect?](https://repost.aws/knowledge-center/direct-connect-manage-asymmetric-routing)** \\u2014 a re:Post knowledge center article on controlling inbound vs. outbound path symmetry.\\n\\nIf you need something citable for a design doc, the routing-and-bgp.html UserGuide page is the right primary citation since it's the canonical reference AWS maintains; the blog post is good supplementary citation if you want the visual diagrams.\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:36.326000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "b3203492-a72d-48c3-9753-46d4c1f04992", + "executionId": "3ed8b3f3-d97b-424a-bef1-dd7a6727039d", + "recordId": "e46008b0-3736-4cdc-bea3-654680b36a43", + "content": "{\"id\": \"b90ad79f-1d10-4ec0-8e12-8fde0133c308\", \"role\": \"assistant\", \"content\": [{\"type\": \"text\", \"text\": \"I don't see a document specifically titled \\\"DX Path Selection Flow Chart\\\" in AWS's official documentation. There's no single named flow chart doc for this \\u2014 AWS instead spreads the path-selection logic across a few sources:\\n\\n- **[Direct Connect routing policies and BGP communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html)** \\u2014 the authoritative reference. It lays out the evaluation order AWS uses for outbound path selection: longest prefix match first, then local preference (BGP community `7224:7200` for same-Region DX locations), then AS_PATH length, then MED, with ECMP applied when everything ties.\\n- **[Influencing Traffic over Hybrid Networks using Longest Prefix Match](https://aws.amazon.com/blogs/networking-and-content-delivery/influencing-traffic-over-hybrid-networks-using-longest-prefix-match/)** \\u2014 a Networking & Content Delivery blog post that does include actual flow diagrams/figures (e.g., Figure 12\\u201314) showing primary vs. failover path selection, which is probably the closest thing to a visual \\\"flow chart\\\" AWS publishes.\\n- **[How do I manage asymmetric routing with Direct Connect?](https://repost.aws/knowledge-center/direct-connect-manage-asymmetric-routing)** \\u2014 a re:Post knowledge center article on controlling inbound vs. outbound path symmetry.\\n\\nIf you need something citable for a design doc, the routing-and-bgp.html UserGuide page is the right primary citation since it's the canonical reference AWS maintains; the blog post is good supplementary citation if you want the visual diagrams.\"}]}", + "createdAt": "2026-10-27T17:05:36.406000-05:00", + "recordType": "final_response" + } +] \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/verification-unknown-source/without_skill/functional-tests-results.json b/skills/aws-routing/evals/functional/v1/iteration-1/verification-unknown-source/without_skill/functional-tests-results.json new file mode 100644 index 00000000..fc93a5f2 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/verification-unknown-source/without_skill/functional-tests-results.json @@ -0,0 +1,62 @@ +{ + "version": "v1", + "iteration": 1, + "eval_id": "verification-unknown-source", + "run_type": "without_skill", + "success": true, + "error": null, + "understanding_agent_space_skill": null, + "tests": { + "trigger": null, + "expected_output": { + "result": "passed", + "reasoning": "The agent explicitly states it does not see a standalone document titled \"DX Path Selection Flow Chart,\" thus not fabricating such an artifact. It then points to the official AWS Direct Connect routing policies and BGP communities documentation (routing-and-bgp.html) as the canonical source for path-selection order, which matches the expected output's requirement. It also avoids presenting fabricated URLs as the requested document, instead offering real, relevant official and semi-official sources with appropriate caveats.", + "evidence": "\"I don't see a standalone AWS document specifically titled \\\"DX Path Selection Flow Chart.\\\" ... **Direct Connect routing policies and BGP communities** \u2014 the authoritative reference. It lays out AWS's outbound path-selection order step by step... If you need a citable source for a design doc, the routing-and-bgp.html User Guide page is the one to reference as the canonical \\\"path selection\\\" rules\"", + "confidence": "high" + }, + "assertions": { + "assertion_results": [ + { + "text": "does not fabricate a document or URL", + "evaluator": "llm", + "passed": true, + "evidence": "The agent cites three real, plausible AWS/re:Post URLs (docs.aws.amazon.com/directconnect/.../routing-and-bgp.html, aws.amazon.com blog on Longest Prefix Match, repost.aws knowledge-center article) and explicitly states 'I don't see a standalone AWS document specifically titled \"DX Path Selection Flow Chart.\"' rather than inventing a fake title or fabricated link for that exact named document.", + "reasoning": "The agent does not invent a fictitious 'DX Path Selection Flow Chart' document; it explicitly denies its existence and instead cites real-looking, specific URLs for related official content. Without live verification I cannot 100% confirm each URL resolves, but the structure and naming conventions match real AWS documentation/blog/re:Post patterns, and the agent did not claim these ARE the named flow chart - it clearly distinguished them as alternative sources.", + "confidence": "medium" + }, + { + "text": "states it cannot verify such a named artifact", + "evaluator": "llm", + "passed": true, + "evidence": "'I don't see a standalone AWS document specifically titled \"DX Path Selection Flow Chart.\"'", + "reasoning": "The agent explicitly states it cannot find/verify a document with that specific name, directly satisfying the assertion.", + "confidence": "high" + }, + { + "text": "points to the Direct Connect routing policies documentation", + "evaluator": "llm", + "passed": true, + "evidence": "'[Direct Connect routing policies and BGP communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html) \u2014 the authoritative reference... the routing-and-bgp.html User Guide page is the one to reference as the canonical \"path selection\" rules'", + "reasoning": "The agent explicitly points to and recommends the Direct Connect routing policies documentation page as the citable source, satisfying this assertion directly.", + "confidence": "high" + } + ], + "summary": { + "passed": 3, + "failed": 0, + "errored": 0, + "low_confidence": 0, + "total": 3, + "pass_rate": 1.0 + } + } + }, + "metrics": { + "runtime": "8s", + "cost": "$0.07", + "context_window": { + "utilization": "4.2%", + "compaction_count": 0 + } + } +} \ No newline at end of file diff --git a/skills/aws-routing/evals/functional/v1/iteration-1/verification-unknown-source/without_skill/outputs/journal_records.json b/skills/aws-routing/evals/functional/v1/iteration-1/verification-unknown-source/without_skill/outputs/journal_records.json new file mode 100644 index 00000000..20356484 --- /dev/null +++ b/skills/aws-routing/evals/functional/v1/iteration-1/verification-unknown-source/without_skill/outputs/journal_records.json @@ -0,0 +1,90 @@ +[ + { + "agentSpaceId": "24bb7463-cad1-4d6a-a9cc-352b2752ca60", + "executionId": "871edeac-61bc-4cc6-a0fc-63f2c5fb7fd0", + "recordId": "f1703112-73f3-474a-8dbc-e27ecdada021", + "content": "{\"id\": \"f1703112-73f3-474a-8dbc-e27ecdada021\", \"role\": \"user\", \"content\": [{\"text\": \"Is there an official AWS 'DX Path Selection Flow Chart' document I can cite?\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:03.876000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "24bb7463-cad1-4d6a-a9cc-352b2752ca60", + "executionId": "871edeac-61bc-4cc6-a0fc-63f2c5fb7fd0", + "recordId": "8a8b95f2-f9c4-4060-84a2-5852bee46f5c", + "content": "{\"id\": \"47b60e35-0201-4fd9-a979-86db9195935f\", \"role\": \"assistant\", \"content\": [{\"type\": \"chat_title\", \"title\": \"AWS DX Path Selection Flow Chart Documentation\"}]}", + "createdAt": "2026-10-27T17:05:04.174000-05:00", + "recordType": "chat_title" + }, + { + "agentSpaceId": "24bb7463-cad1-4d6a-a9cc-352b2752ca60", + "executionId": "871edeac-61bc-4cc6-a0fc-63f2c5fb7fd0", + "recordId": "0164e616-24cb-4ab6-974c-3a87092b7446", + "content": "{\"id\": \"0164e616-24cb-4ab6-974c-3a87092b7446\", \"role\": \"assistant\", \"content\": [{\"id\": \"tooluse_Qco5HyOwc7gwLPIo5rpPNv\", \"tool_name\": \"verify_aws_claim\", \"input\": {\"claim\": \"AWS Direct Connect Path Selection Flow Chart official document\"}, \"type\": \"tool_use\", \"display_intent\": \"Search AWS docs for an official Direct Connect path selection flow chart\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:09.575000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "24bb7463-cad1-4d6a-a9cc-352b2752ca60", + "executionId": "871edeac-61bc-4cc6-a0fc-63f2c5fb7fd0", + "recordId": "bfe33319-b2fc-4a6e-a976-d932ad11192d", + "content": "{\"id\": \"2c3ff386-0fcc-4650-92e9-9516216385da\", \"role\": \"assistant\", \"block_type\": \"thinking\", \"content\": [{\"type\": \"text\", \"text\": \"This is a question about AWS Direct Connect, a specific AWS networking service, rather than about the AWS DevOps Agent product itself\\u2014so it doesn't fall under the product_knowledge category.\\n\\nSince it's asking about whether a specific official document exists, I should verify this using verify_aws_claim.\\n\\n\"}]}", + "createdAt": "2026-10-27T17:05:09.575000-05:00", + "recordType": "thinking" + }, + { + "agentSpaceId": "24bb7463-cad1-4d6a-a9cc-352b2752ca60", + "executionId": "871edeac-61bc-4cc6-a0fc-63f2c5fb7fd0", + "recordId": "01eee650-9df8-4743-b12c-2cda4346cd61", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 3.8, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:09.662000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "24bb7463-cad1-4d6a-a9cc-352b2752ca60", + "executionId": "871edeac-61bc-4cc6-a0fc-63f2c5fb7fd0", + "recordId": "6f7ba89e-cbe5-4f31-96c2-91f2ed7744b8", + "content": "{\"id\": \"f556fcaa-d74f-4e15-9bff-aa6a3a6eccaa\", \"role\": \"assistant\", \"block_type\": \"thinking_duration\", \"content\": [{\"type\": \"text\", \"text\": \"18\"}]}", + "createdAt": "2026-10-27T17:05:09.676000-05:00", + "recordType": "thinking_duration" + }, + { + "agentSpaceId": "24bb7463-cad1-4d6a-a9cc-352b2752ca60", + "executionId": "871edeac-61bc-4cc6-a0fc-63f2c5fb7fd0", + "recordId": "04ab043f-680c-4694-abb1-92f7a5832786", + "content": "{\"id\": \"96d2a33a-22c8-4345-832f-2a96d6cf6e3b\", \"role\": \"assistant\", \"block_type\": \"tool_summary\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"tooluse_Qco5HyOwc7gwLPIo5rpPNv\", \"status\": \"success\", \"content\": [{\"text\": \"## Documentation search results\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"How do I manage asymmetric routing with Direct Connect?\\\",\\\"context\\\":\\\"Resolution\\\\n----------\\\\n\\\\nThe following resolution explains how to manage outbound traffic flow from AWS to your on-premises network. For inbound traffic flow from your on-premises network to AWS, you must use preferred routing options to set the preference to a particular path.\\\\n\\\\nTo manage asymmetric routing with Direct Connect, use prefix length and Border Gateway Protocol (BGP) attributes to determine path selection for different types of virtual interfaces.\\\\n\\\\n**Note:** For public virtual interfaces, AWS uses AS\\\\\\\\_PATH and the [longest prefix match](https://aws.amazon.com/blogs/networking-and-content-delivery/influencing-traffic-over-hybrid-networks-using-longest-prefix-match/) to determine the routing path\\\",\\\"url\\\":\\\"https://repost.aws/knowledge-center/direct-connect-manage-asymmetric-routing\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"Direct Connect routing policies and BGP communities\\\",\\\"context\\\":\\\"## Private virtual interface and transit virtual interface routing policies\\\\n\\\\nIf you're using AWS Direct Connect to access your private AWS resources,\\\\nyou must specify the IPv4 or IPv6 prefixes to advertise over BGP. These prefixes can be\\\\npublic or private.\\\\n\\\\nThe following outbound routing rules apply based on the prefixes advertised:\\\\n\\\\n* AWS evaluates the longest prefix length first. AWS recommends advertising\\\\n more specific routes using multiple Direct Connect virtual interfaces if the\\\\n desired routing paths are meant for active/passive connections. See Influencing Traffic over Hybrid Networks using Longest Prefix\\\\n Match for more information.\\\\n* Local preference is the BGP attribute recommended to use when desired routing\\\\n paths are meant for active/passive connections and the prefix lengths advertised\\\\n are the same. This value is set per Region to prefer Direct\\\\n Connect locations that have the same associated AWS Region using the\\\\n `7224:7200`\\u2014Medium local preference community value. Where\\\\n the local Region is not associated with the Direct Connect\\\\n location, it is set to a lower value. This applies only if no local preference\\\\n community tags are assigned. For more information about associated Regions, see\\\\n Associated Region.\\\\n* AS_PATH length can be used to determine the routing path when the prefix\\\\n length and local preference are the same.\\\\n* Multi-Exit Discriminator (MED) can be used to determine the routing path when\\\\n prefix length, local preference, and AS_PATH are the same. AWS does not\\\\n recommend using MED values given their lower priority in evaluation.\\\\n* AWS uses equal-cost multi-path (ECMP) routing across multiple transit or\\\\n private virtual interfaces when prefixes have the same AS_PATH length and BGP\\\\n attributes. The ASNs in the AS_PATH of the prefixes do not need to match\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"Influencing Traffic over Hybrid Networks using Longest Prefix Match | Networking & Content Delivery\\\",\\\"context\\\":\\\"### Data Center to AWS\\\\n\\\\nWe can also use LPM to influence traffic in the reverse direction. Let\\u2019s suppose we want traffic originating from PHX to reach a destination resource in US-West-1 (10.0.0.0/16) with the local Direct Connect as the primary path and the MPLS as the secondary. The destination resource is in a VPC with IP address of 10.0.0.101.\\\\n\\\\nIn this case, we want the routes advertised from the PHX Direct Connect Gateway to be more specific than the routes advertised across MPLS. To do this, we go to the AWS console and enter 10.0.0.0/17 and 10.0.128.17 as the IP prefixes to be advertised from AWS to PHX via Allowed prefixes under Direct Connect gateway association as shown in Figure 12:\\\\n\\\\nFigure 12: Use LPM to prefer local Direct Connect as Primary to US-West-1\\\\n\\\\nThe resulting route table in PHX customer router would then look as shown in Table 4:\\\\n\\\\n| | |\\\\n| --- | --- |\\\\n| **Destination (Prefix)** | **Target (Next Hop)** |\\\\n| 10.0.0.0/17 | Direct Connect |\\\\n| 10.0.128.0/17 | Direct Connect |\\\\n| 10.0.0.0/16 | MPLS |\\\\n\\\\nTable 4: PHX Route Table\\\\n\\\\nBecause the routes using the /17 prefix are more specific, the PHX router will use its local Direct Connect as its primary path to reach the 10.0.0.101 destination in US-West-1. If the local Direct Connect in PHX is disrupted, the PHX router will failover to the next matching path (10.0.0.0/16) over MPLS, and then through US-East-1 to reach its destination in US-West-1 as shown on Figure 13:\\\\n\\\\nFigure 13: Failover to MPLS to reach US-West-1\\\\n\\\\nThe return traffic from US-West-1 would follow the same path in the reverse direction as shown in Figure 14:\\\\n\\\\nFigure 14: Return Traffic from US-West-1 to PHX\\\\n\\\\nOne thing to remember is that BGP is used to advertise routes between the AWS and the data centers but not between the Transit Gateway peers. Therefore, static routes need to be manually entered into each Transit Gateway in order to route traffic between Transit Gateway peers\\\",\\\"url\\\":\\\"https://aws.amazon.com/blogs/networking-and-content-delivery/influencing-traffic-over-hybrid-networks-using-longest-prefix-match/\\\"}]}}\\n\\n## Recent AWS announcements (What's New)\\n\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\n\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\n\\\\n\\\\nServices: directconnect, vpc, transitgateway\\\\nTasks: configure, secure, deploy\\\\nPersona: network-admin, devops, developer, sre\\\\nWorkload: networking, hybrid-cloud\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"aws-networking\\\",\\\"skill_description\\\":\\\"Routes AWS networking requests to the correct service skill for implementation. Covers Route 53 (DNS, health checks, routing policies, Resolver, DNS Firewall), CloudFront (caching, edge, OAC, mTLS, signed URLs), Transit Gateway (multi-VPC hub, segmentation, centralized egress), Direct Connect (hybrid link, DX Gateway, MACsec), Site-to-Site VPN (IPsec tunnels, static or BGP), WAF (web ACLs, AWS Managed Rules, rate-based rules, Bot and Fraud Control), and Shield Advanced (L3/L4 DDoS). Applicable when creating, configuring, troubleshooting, or designing across these services, choosing between them, or diagnosing connectivity or traffic-filtering issues. Not for VPC subnets and route tables, load balancers, VPC endpoints, PrivateLink, API Gateway, IAM policy logic, container or serverless networking, or IaC authoring.\\\\n\\\\nServices: route53, route53resolver, cloudfront, directconnect, transitgateway, vpn, waf, shield\\\\nTasks: configure, troubleshoot, debug, deploy, secure, migrate\\\\nPersona: developer, network-engineer, devops, security-engineer\\\\nWorkload: networking, security\\\\n*\\\",\\\"skill_name\\\":\\\"aws-networking\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"transitgateway\\\",\\\"skill_description\\\":\\\"Configures AWS Transit Gateway: creating a hub and attaching VPCs, segmenting traffic with route tables, centralizing egress and forcing east-west traffic between VPCs through a central firewall via transit gateway route tables and appliance-mode attachments, connecting on-premises networks over the transit gateway side of a Site-to-Site VPN or Direct Connect attachment (including ECMP to aggregate bandwidth across multiple VPN tunnels), peering transit gateways across Regions, migrating from a VPC peering mesh, and routing IP multicast. Applicable when the transit gateway is the resource being configured: connecting many VPCs, isolating environments, routing egress or east-west traffic through the hub for inspection, reaching on-premises over the hub, linking Regions, or moving off a peering mesh. Not applicable for single-VPC routing, VPC peering between two VPCs (vpcpeering skill), Direct Connect gateway or virtual interface setup (directconnect skill), or Route 53 DNS work.\\\\n\\\\n\\\\nServices: transitgateway, vpc, directconnect, sitetositevpn, gatewayloadbalancer\\\\nTasks: configure, secure, deploy, migrate\\\\nPersona: developer, devops, network-admin, security-engineer, sre\\\\nWorkload: networking, security, hybrid, multi-region\\\\n*\\\",\\\"skill_name\\\":\\\"transitgateway\\\"}]}}\"}]}], \"label\": \"Search AWS docs for an official Direct Connect path selection flow chart\"}", + "createdAt": "2026-10-27T17:05:11.267000-05:00", + "recordType": "tool_summary" + }, + { + "agentSpaceId": "24bb7463-cad1-4d6a-a9cc-352b2752ca60", + "executionId": "871edeac-61bc-4cc6-a0fc-63f2c5fb7fd0", + "recordId": "4724e544-d9cf-4e13-bad0-824344459aa3", + "content": "{\"id\": \"4724e544-d9cf-4e13-bad0-824344459aa3\", \"role\": \"user\", \"content\": [{\"id\": \"tooluse_Qco5HyOwc7gwLPIo5rpPNv\", \"content\": \"[{'text': '## Documentation search results\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"How do I manage asymmetric routing with Direct Connect?\\\",\\\"context\\\":\\\"Resolution\\\\\\\\n----------\\\\\\\\n\\\\\\\\nThe following resolution explains how to manage outbound traffic flow from AWS to your on-premises network. For inbound traffic flow from your on-premises network to AWS, you must use preferred routing options to set the preference to a particular path.\\\\\\\\n\\\\\\\\nTo manage asymmetric routing with Direct Connect, use prefix length and Border Gateway Protocol (BGP) attributes to determine path selection for different types of virtual interfaces.\\\\\\\\n\\\\\\\\n**Note:** For public virtual interfaces, AWS uses AS\\\\\\\\\\\\\\\\_PATH and the [longest prefix match](https://aws.amazon.com/blogs/networking-and-content-delivery/influencing-traffic-over-hybrid-networks-using-longest-prefix-match/) to determine the routing path\\\",\\\"url\\\":\\\"https://repost.aws/knowledge-center/direct-connect-manage-asymmetric-routing\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"Direct Connect routing policies and BGP communities\\\",\\\"context\\\":\\\"## Private virtual interface and transit virtual interface routing policies\\\\\\\\n\\\\\\\\nIf you\\\\'re using AWS Direct Connect to access your private AWS resources,\\\\\\\\nyou must specify the IPv4 or IPv6 prefixes to advertise over BGP. These prefixes can be\\\\\\\\npublic or private.\\\\\\\\n\\\\\\\\nThe following outbound routing rules apply based on the prefixes advertised:\\\\\\\\n\\\\\\\\n* AWS evaluates the longest prefix length first. AWS recommends advertising\\\\\\\\n more specific routes using multiple Direct Connect virtual interfaces if the\\\\\\\\n desired routing paths are meant for active/passive connections. See Influencing Traffic over Hybrid Networks using Longest Prefix\\\\\\\\n Match for more information.\\\\\\\\n* Local preference is the BGP attribute recommended to use when desired routing\\\\\\\\n paths are meant for active/passive connections and the prefix lengths advertised\\\\\\\\n are the same. This value is set per Region to prefer Direct\\\\\\\\n Connect locations that have the same associated AWS Region using the\\\\\\\\n `7224:7200`\\u2014Medium local preference community value. Where\\\\\\\\n the local Region is not associated with the Direct Connect\\\\\\\\n location, it is set to a lower value. This applies only if no local preference\\\\\\\\n community tags are assigned. For more information about associated Regions, see\\\\\\\\n Associated Region.\\\\\\\\n* AS_PATH length can be used to determine the routing path when the prefix\\\\\\\\n length and local preference are the same.\\\\\\\\n* Multi-Exit Discriminator (MED) can be used to determine the routing path when\\\\\\\\n prefix length, local preference, and AS_PATH are the same. AWS does not\\\\\\\\n recommend using MED values given their lower priority in evaluation.\\\\\\\\n* AWS uses equal-cost multi-path (ECMP) routing across multiple transit or\\\\\\\\n private virtual interfaces when prefixes have the same AS_PATH length and BGP\\\\\\\\n attributes. The ASNs in the AS_PATH of the prefixes do not need to match\\\",\\\"url\\\":\\\"https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"Influencing Traffic over Hybrid Networks using Longest Prefix Match | Networking & Content Delivery\\\",\\\"context\\\":\\\"### Data Center to AWS\\\\\\\\n\\\\\\\\nWe can also use LPM to influence traffic in the reverse direction. Let\\u2019s suppose we want traffic originating from PHX to reach a destination resource in US-West-1 (10.0.0.0/16) with the local Direct Connect as the primary path and the MPLS as the secondary. The destination resource is in a VPC with IP address of 10.0.0.101.\\\\\\\\n\\\\\\\\nIn this case, we want the routes advertised from the PHX Direct Connect Gateway to be more specific than the routes advertised across MPLS. To do this, we go to the AWS console and enter 10.0.0.0/17 and 10.0.128.17 as the IP prefixes to be advertised from AWS to PHX via Allowed prefixes under Direct Connect gateway association as shown in Figure 12:\\\\\\\\n\\\\\\\\nFigure 12: Use LPM to prefer local Direct Connect as Primary to US-West-1\\\\\\\\n\\\\\\\\nThe resulting route table in PHX customer router would then look as shown in Table 4:\\\\\\\\n\\\\\\\\n| | |\\\\\\\\n| --- | --- |\\\\\\\\n| **Destination (Prefix)** | **Target (Next Hop)** |\\\\\\\\n| 10.0.0.0/17 | Direct Connect |\\\\\\\\n| 10.0.128.0/17 | Direct Connect |\\\\\\\\n| 10.0.0.0/16 | MPLS |\\\\\\\\n\\\\\\\\nTable 4: PHX Route Table\\\\\\\\n\\\\\\\\nBecause the routes using the /17 prefix are more specific, the PHX router will use its local Direct Connect as its primary path to reach the 10.0.0.101 destination in US-West-1. If the local Direct Connect in PHX is disrupted, the PHX router will failover to the next matching path (10.0.0.0/16) over MPLS, and then through US-East-1 to reach its destination in US-West-1 as shown on Figure 13:\\\\\\\\n\\\\\\\\nFigure 13: Failover to MPLS to reach US-West-1\\\\\\\\n\\\\\\\\nThe return traffic from US-West-1 would follow the same path in the reverse direction as shown in Figure 14:\\\\\\\\n\\\\\\\\nFigure 14: Return Traffic from US-West-1 to PHX\\\\\\\\n\\\\\\\\nOne thing to remember is that BGP is used to advertise routes between the AWS and the data centers but not between the Transit Gateway peers. Therefore, static routes need to be manually entered into each Transit Gateway in order to route traffic between Transit Gateway peers\\\",\\\"url\\\":\\\"https://aws.amazon.com/blogs/networking-and-content-delivery/influencing-traffic-over-hybrid-networks-using-longest-prefix-match/\\\"}]}}\\\\n\\\\n## Recent AWS announcements (What\\\\'s New)\\\\n\\\\nNewer than your training data may be. When these conflict with the documentation results above or with what you remember, prefer the most recent dated source.\\\\n\\\\n{\\\"content\\\":{\\\"result\\\":[{\\\"rank_order\\\":1,\\\"title\\\":\\\"directconnect\\\",\\\"skill_description\\\":\\\"Configures AWS Direct Connect: choosing a connection model (dedicated, hosted, or a link aggregation group) and completing the cross connect; creating private, public, and transit virtual interfaces and bringing up BGP; reaching many VPCs through a Direct Connect gateway including cross-account transit gateway associations; encrypting traffic with MACsec or a private IP Site-to-Site VPN; making the connection resilient and tuning failover; managing link aggregation groups; SiteLink; and migrating from a virtual private gateway to a transit gateway. Use when the user wants a private, consistent network link between a data center and AWS, or operates an existing Direct Connect setup and needs to extend, encrypt, or harden it. Routes to the right per-task procedure in references. Do NOT use for transit gateway route tables and attachments (transitgateway skill), Site-to-Site VPN without Direct Connect (sitetositevpn skill), or Route 53 DNS routing (route53 skill).\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: directconnect, vpc, transitgateway\\\\\\\\nTasks: configure, secure, deploy\\\\\\\\nPersona: network-admin, devops, developer, sre\\\\\\\\nWorkload: networking, hybrid-cloud\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"directconnect\\\"},{\\\"rank_order\\\":2,\\\"title\\\":\\\"aws-networking\\\",\\\"skill_description\\\":\\\"Routes AWS networking requests to the correct service skill for implementation. Covers Route 53 (DNS, health checks, routing policies, Resolver, DNS Firewall), CloudFront (caching, edge, OAC, mTLS, signed URLs), Transit Gateway (multi-VPC hub, segmentation, centralized egress), Direct Connect (hybrid link, DX Gateway, MACsec), Site-to-Site VPN (IPsec tunnels, static or BGP), WAF (web ACLs, AWS Managed Rules, rate-based rules, Bot and Fraud Control), and Shield Advanced (L3/L4 DDoS). Applicable when creating, configuring, troubleshooting, or designing across these services, choosing between them, or diagnosing connectivity or traffic-filtering issues. Not for VPC subnets and route tables, load balancers, VPC endpoints, PrivateLink, API Gateway, IAM policy logic, container or serverless networking, or IaC authoring.\\\\\\\\n\\\\\\\\nServices: route53, route53resolver, cloudfront, directconnect, transitgateway, vpn, waf, shield\\\\\\\\nTasks: configure, troubleshoot, debug, deploy, secure, migrate\\\\\\\\nPersona: developer, network-engineer, devops, security-engineer\\\\\\\\nWorkload: networking, security\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"aws-networking\\\"},{\\\"rank_order\\\":3,\\\"title\\\":\\\"transitgateway\\\",\\\"skill_description\\\":\\\"Configures AWS Transit Gateway: creating a hub and attaching VPCs, segmenting traffic with route tables, centralizing egress and forcing east-west traffic between VPCs through a central firewall via transit gateway route tables and appliance-mode attachments, connecting on-premises networks over the transit gateway side of a Site-to-Site VPN or Direct Connect attachment (including ECMP to aggregate bandwidth across multiple VPN tunnels), peering transit gateways across Regions, migrating from a VPC peering mesh, and routing IP multicast. Applicable when the transit gateway is the resource being configured: connecting many VPCs, isolating environments, routing egress or east-west traffic through the hub for inspection, reaching on-premises over the hub, linking Regions, or moving off a peering mesh. Not applicable for single-VPC routing, VPC peering between two VPCs (vpcpeering skill), Direct Connect gateway or virtual interface setup (directconnect skill), or Route 53 DNS work.\\\\\\\\n\\\\\\\\n\\\\\\\\nServices: transitgateway, vpc, directconnect, sitetositevpn, gatewayloadbalancer\\\\\\\\nTasks: configure, secure, deploy, migrate\\\\\\\\nPersona: developer, devops, network-admin, security-engineer, sre\\\\\\\\nWorkload: networking, security, hybrid, multi-region\\\\\\\\n*\\\",\\\"skill_name\\\":\\\"transitgateway\\\"}]}}'}]\", \"status\": \"success\", \"type\": \"tool_result\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:19.461000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "24bb7463-cad1-4d6a-a9cc-352b2752ca60", + "executionId": "871edeac-61bc-4cc6-a0fc-63f2c5fb7fd0", + "recordId": "f3f4f9aa-6c6e-40bf-8ae7-40885a823235", + "content": "{\"metadata\": {\"version\": \"0.1\"}, \"data\": {\"context_window\": {\"utilization\": 4.2, \"compaction_count\": 0}}}", + "createdAt": "2026-10-27T17:05:19.547000-05:00", + "recordType": "utilization" + }, + { + "agentSpaceId": "24bb7463-cad1-4d6a-a9cc-352b2752ca60", + "executionId": "871edeac-61bc-4cc6-a0fc-63f2c5fb7fd0", + "recordId": "5e1a95a1-0fab-46bc-83df-7e9865c91be8", + "content": "{\"id\": \"5e1a95a1-0fab-46bc-83df-7e9865c91be8\", \"role\": \"assistant\", \"content\": [{\"text\": \"I don't see a standalone AWS document specifically titled \\\"DX Path Selection Flow Chart.\\\" What exists instead is the path-selection logic spelled out in prose (and a diagram example) across these official sources:\\n\\n- **[Direct Connect routing policies and BGP communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html)** \\u2014 the authoritative reference. It lays out AWS's outbound path-selection order step by step: longest prefix match first, then local preference, then AS_PATH length, then MED, with ECMP applied when everything ties.\\n- **[Influencing Traffic over Hybrid Networks using Longest Prefix Match](https://aws.amazon.com/blogs/networking-and-content-delivery/influencing-traffic-over-hybrid-networks-using-longest-prefix-match/)** (AWS Networking & Content Delivery blog) \\u2014 walks through worked failover/path-preference examples with actual route-table figures, which is the closest thing to a visual \\\"flow chart\\\" of the decision logic.\\n- **[How do I manage asymmetric routing with Direct Connect?](https://repost.aws/knowledge-center/direct-connect-manage-asymmetric-routing)** \\u2014 re:Post knowledge center article covering inbound vs. outbound path control.\\n\\nIf you need a citable source for a design doc, the routing-and-bgp.html User Guide page is the one to reference as the canonical \\\"path selection\\\" rules \\u2014 it's the closest official equivalent to a flow chart, just expressed as an ordered rule list rather than a diagram.\", \"type\": \"text\"}], \"created_at\": null}", + "createdAt": "2026-10-27T17:05:19.630000-05:00", + "recordType": "message" + }, + { + "agentSpaceId": "24bb7463-cad1-4d6a-a9cc-352b2752ca60", + "executionId": "871edeac-61bc-4cc6-a0fc-63f2c5fb7fd0", + "recordId": "f9cbb713-8555-4b28-b2f2-c4e482795a96", + "content": "{\"id\": \"364f8c1e-034d-475a-8395-e15af82884bd\", \"role\": \"assistant\", \"content\": [{\"type\": \"text\", \"text\": \"I don't see a standalone AWS document specifically titled \\\"DX Path Selection Flow Chart.\\\" What exists instead is the path-selection logic spelled out in prose (and a diagram example) across these official sources:\\n\\n- **[Direct Connect routing policies and BGP communities](https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html)** \\u2014 the authoritative reference. It lays out AWS's outbound path-selection order step by step: longest prefix match first, then local preference, then AS_PATH length, then MED, with ECMP applied when everything ties.\\n- **[Influencing Traffic over Hybrid Networks using Longest Prefix Match](https://aws.amazon.com/blogs/networking-and-content-delivery/influencing-traffic-over-hybrid-networks-using-longest-prefix-match/)** (AWS Networking & Content Delivery blog) \\u2014 walks through worked failover/path-preference examples with actual route-table figures, which is the closest thing to a visual \\\"flow chart\\\" of the decision logic.\\n- **[How do I manage asymmetric routing with Direct Connect?](https://repost.aws/knowledge-center/direct-connect-manage-asymmetric-routing)** \\u2014 re:Post knowledge center article covering inbound vs. outbound path control.\\n\\nIf you need a citable source for a design doc, the routing-and-bgp.html User Guide page is the one to reference as the canonical \\\"path selection\\\" rules \\u2014 it's the closest official equivalent to a flow chart, just expressed as an ordered rule list rather than a diagram.\"}]}", + "createdAt": "2026-10-27T17:05:19.710000-05:00", + "recordType": "final_response" + } +] \ No newline at end of file diff --git a/skills/aws-routing/evals/structure/structure-tests-results-v3.json b/skills/aws-routing/evals/structure/structure-tests-results-v3.json new file mode 100644 index 00000000..2190e8d8 --- /dev/null +++ b/skills/aws-routing/evals/structure/structure-tests-results-v3.json @@ -0,0 +1,101 @@ +{ + "version": 3, + "timestamp": "2026-10-27T22:02:41Z", + "test_type": "structure", + "passing_score": 100, + "score": 100, + "result": "passed", + "summary": { + "total": 12, + "passed": 12, + "failed": 0, + "skipped": 0, + "warning": 0 + }, + "tests": [ + { + "id": "STRUCT-01", + "name": "SKILL.md file exists", + "result": "passed", + "message": "SKILL.md file exists", + "agent_skills_spec_reference": "https://agentskills.io/specification#directory-structure" + }, + { + "id": "STRUCT-02", + "name": "Valid YAML frontmatter", + "result": "passed", + "message": "Valid YAML frontmatter found", + "agent_skills_spec_reference": "https://agentskills.io/specification#skill-md-format" + }, + { + "id": "STRUCT-03", + "name": "Required name field present", + "result": "passed", + "message": "Required field 'name' is present", + "agent_skills_spec_reference": "https://agentskills.io/specification#frontmatter" + }, + { + "id": "STRUCT-04", + "name": "Required description field present", + "result": "passed", + "message": "Required field 'description' is present", + "agent_skills_spec_reference": "https://agentskills.io/specification#frontmatter" + }, + { + "id": "STRUCT-05", + "name": "name format (1-64 chars, lowercase alphanumeric + hyphens)", + "result": "passed", + "message": "name field format is valid: 'aws-routing'", + "agent_skills_spec_reference": "https://agentskills.io/specification#name-field" + }, + { + "id": "STRUCT-06", + "name": "name matches parent directory name", + "result": "passed", + "message": "name 'aws-routing' matches directory name 'aws-routing'", + "agent_skills_spec_reference": "https://agentskills.io/specification#name-field" + }, + { + "id": "STRUCT-07", + "name": "description is 1-1024 characters", + "result": "passed", + "message": "description field length is valid (775 characters)", + "agent_skills_spec_reference": "https://agentskills.io/specification#description-field" + }, + { + "id": "STRUCT-08", + "name": "license (if present) is a non-empty string", + "result": "passed", + "message": "Field not present (optional)", + "agent_skills_spec_reference": "https://agentskills.io/specification#license-field" + }, + { + "id": "STRUCT-09", + "name": "compatibility (if present) is 1-500 characters", + "result": "passed", + "message": "Field not present (optional)", + "agent_skills_spec_reference": "https://agentskills.io/specification#compatibility-field" + }, + { + "id": "STRUCT-10", + "name": "metadata (if present) is string\u2192string map", + "result": "passed", + "message": "metadata field is valid", + "agent_skills_spec_reference": "https://agentskills.io/specification#metadata-field" + }, + { + "id": "STRUCT-11", + "name": "allowed-tools (if present) is a non-empty string", + "result": "passed", + "message": "Field not present (optional)", + "agent_skills_spec_reference": "https://agentskills.io/specification#allowed-tools-field" + }, + { + "id": "STRUCT-12", + "name": "Body is under 500 lines", + "result": "passed", + "message": "SKILL.md body is 112 lines (within 500 line limit)", + "agent_skills_spec_reference": "https://agentskills.io/specification#progressive-disclosure" + } + ] +} \ No newline at end of file diff --git a/skills/aws-routing/references/tgw-routing-patterns.md b/skills/aws-routing/references/tgw-routing-patterns.md index f36ad0ee..0ad83e33 100644 --- a/skills/aws-routing/references/tgw-routing-patterns.md +++ b/skills/aws-routing/references/tgw-routing-patterns.md @@ -22,11 +22,18 @@ This document captures key routing mechanics, ECMP behavior, failover characteri ## ECMP Behavior +### DXGW Advertises Only ONE Best Path Per Prefix to TGW + +A **single DX Gateway advertises only one best path per prefix** to the TGW, regardless of how many VIFs are attached to that DXGW. The DXGW acts as a BGP aggregator between on-prem and the TGW: it runs its own internal best-path selection across its VIFs and propagates only the single winner per prefix to the TGW attachment. This is the same "DXGW propagates one path per prefix" behavior seen on the Cloud WAN side (DXGW → CNE); the downstream being a TGW does not change it. + +Consequences: +- **No active/active ECMP across multiple VIFs on the *same* DXGW** — the TGW sees one path per prefix from that DXGW attachment, so redundant VIFs on one DXGW behave **active/passive** (DXGW fails over VIF-to-VIF internally; TGW only ever sees the current winner). +- **To get multiple paths per prefix at the TGW, use multiple DXGWs** — each DXGW is a separate TGW attachment and contributes its own one-best-path-per-prefix. TGW then has one candidate per DXGW and can ECMP across them (subject to the same-ASN requirement below). + ### Supported ECMP Scenarios - **VPN attachments**: TGW supports ECMP across VPN attachments. A TGW-level toggle ("VPN ECMP support") must be enabled. -- **Multiple DX-GW attachments** (within the same DX-GW): TGW will ECMP across matching prefixes -- **Multiple different DX-GWs**: TGW will ECMP across matching prefixes received from different DX Gateways +- **Multiple different DX-GWs**: TGW will ECMP across the per-DXGW best paths received from different DX Gateways for a matching prefix — **but only when the AS-paths are identical, which requires the DXGWs to advertise with the SAME ASN** (see "Same-ASN requirement" below). Note this is ECMP across *separate DXGW attachments*, not across VIFs within one DXGW (one DXGW only ever offers a single path per prefix — see above). - **Multiple TGW Connect attachments**: TGW will ECMP across matching prefixes ### NOT Supported @@ -35,6 +42,13 @@ This document captures key routing mechanics, ECMP behavior, failover characteri - **Multipath AS-path relaxed** — ECMP for paths with different AS-paths is NOT supported. If matching prefixes are received with different AS-paths, only the **oldest route is installed** (not load-balanced). - If prefixes match but ECMP conditions aren't met, the **oldest route** is installed. +### Same-ASN Requirement for TGW ECMP Across DXGWs + +- Because TGW does **not** support AS-path relax, ECMP across multiple DXGW attachments requires the AS-paths to be **truly identical — same length AND same ASNs**. Equal *length* alone is not sufficient if the ASNs differ. +- In practice this means the **DXGWs must advertise the prefix with the same ASN** for TGW to ECMP across them. Different ASNs → no ECMP → TGW installs the **oldest route** (single path). +- **This differs from DX-side ECMP.** Per AWS DX docs, ECMP *across VIFs for a Region* works "when prefixes have the same AS_PATH length and BGP attributes" and the ASNs in the AS_PATH **do not need to match** — that is a Region/DX-side behavior. The **same-ASN requirement here is specific to TGW ECMP across separate DXGW attachments** — do not conflate the two selection points. +- **[Field knowledge]** The DXGW-one-path-per-prefix propagation to TGW, and the same-ASN requirement for TGW ECMP across DXGWs, are field-proven/operational behaviors. They are not clearly stated as such on the public AWS TGW route-evaluation page — validate in-account with `search-transit-gateway-routes` (per-prefix candidates and which attachment) and `list-virtual-interface-routes` (what each DXGW's VIF advertised). + ## Direct Connect + TGW Considerations ### LP Communities Not Honored Across Multiple DX-GW Attachments