Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions skills/aws-routing/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Changelog

## [1.2.5] - 2026-10-27
### Added
- `references/tgw-routing-patterns.md`: new "DXGW Advertises Only ONE Best Path Per Prefix to TGW" subsection. A single DXGW propagates one winner per prefix to the TGW, so redundant VIFs on one DXGW are active/passive at the TGW; multiple paths per prefix at the TGW require multiple DXGW attachments. Marked `[Field knowledge]`.
### Changed
- `references/tgw-routing-patterns.md`: removed the "multiple DX-GW attachments (within the same DX-GW)" ECMP line, which contradicted the one-path-per-prefix behavior. Clarified that TGW ECMP across separate DXGWs needs identical AS-paths (same ASNs, not just equal length) because TGW does not support AS-path relax, and contrasted this with DX-side ECMP, where AS_PATH ASNs do not need to match.
- `SKILL.md`: `description` rewritten to open with "Use this skill when..." and focus on user intent, keeping the same scope and symptom triggers (BP-01).
- `SKILL.md`: reference files are now markdown links, each with a condition for when to load it; the routing-analysis steps are a checklist with a final self-check step. No technical guidance changed. Driven by skill evaluation tool best-practices findings (BP-04, BP-05, BP-16, BP-17).
- `evals/evals.json`: converted to the skill evaluation tool's functional schema (`skill_name`, `task_type`, `should_trigger`), keeping the nine existing evals unchanged and adding two negative evals.
## [1.2.4] - 2026-10-21
- Agent types now use the API values in `metadata.aws-devops-agent-skills.agent-types` (`CHAT`, `INCIDENT_RCA`, `INCIDENT_TRIAGE`), the field that sets which agents load the skill.

Expand Down
54 changes: 34 additions & 20 deletions skills/aws-routing/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
name: aws-routing
description: 'Analyze and troubleshoot AWS routing and BGP path selection - how AWS chooses a network path, why traffic takes an unexpected route, and how to steer it. Use for AWS Cloud WAN route evaluation and CNE path selection; Direct Connect Gateway (DXGW) path selection and local vs remote region preference; Transit Gateway route tables, ECMP, and peering; VPC route tables and longest-prefix-match; BGP traffic engineering with local-preference communities (7224:7100/7200/7300), AS-path prepending, and MED; DX + VPN redundancy and active/active vs active/passive design; asymmetric routing and failover. Activate on symptoms like "traffic is leaving the region", "not using my local Direct Connect", "routing to the wrong VIF/location", "path is asymmetric", or "how do I make AWS prefer this path". Read-only: analysis, recommendations, and describe/get/list validation commands grounded in public AWS docs.'
description: 'Use this skill when the user wants to know which path AWS will pick for their traffic, why traffic took an unexpected route, or how to make AWS prefer a path. Covers Cloud WAN CNE route evaluation, Direct Connect Gateway local vs remote region preference, Transit Gateway route tables, ECMP and peering, VPC longest-prefix-match, BGP traffic engineering (LP communities 7224:7100/7200/7300, AS-path prepending, MED), DX + VPN redundancy and failover, and asymmetric routing. Use it even when the user only describes a symptom, such as "traffic is leaving the region", "not using my local Direct Connect", "routing to the wrong VIF/location", or "path is asymmetric". Read-only: analysis, recommendations, and describe/get/list validation commands grounded in public AWS docs.'
metadata:

Check warning on line 4 in skills/aws-routing/SKILL.md

View workflow job for this annotation

GitHub Actions / validate-skills

Skill publishing rules

"metadata.summary" is missing. It will be required once every skill has one: one line of at most 200 characters, shown as the skill's card text in catalogs. See the "Skill Publishing Rules" section of CONTRIBUTING.md.
version: "1.2.4"
version: "1.2.5"
author: midakin
aws-devops-agent-skills.agent-types: "CHAT, INCIDENT_RCA"
---
Expand Down Expand Up @@ -55,16 +55,22 @@

## How to analyze a routing problem

1. **Establish the path.** Identify source, destination, and every routing construct in between
(VPC route table → TGW/Cloud WAN CNE → DXGW → VIF → on-prem, or the relevant subset).
2. **Name the route table.** Be explicit about which table makes each decision (VPC RT, TGW RT,
CNE RT, DXGW internal selection).
3. **Apply the evaluation order for that construct** (see below).
4. **Identify the deciding attribute** (longest prefix, local preference, AS-path, MED, source type).
5. **Flag non-determinism** (e.g., ECMP or "deterministically random" tiebreakers) and recommend a
deterministic alternative.
6. **Cite the AWS documentation** that supports the behavior.
7. **Provide read-only validation commands** the user can run to confirm.
Work through this checklist in order; each step depends on the one before it.

- [ ] **1. Establish the path.** Identify source, destination, and every routing construct in
between (VPC route table → TGW/Cloud WAN CNE → DXGW → VIF → on-prem, or the relevant subset).
- [ ] **2. Name the route table.** Be explicit about which table makes each decision (VPC RT, TGW RT,
CNE RT, DXGW internal selection).
- [ ] **3. Apply the evaluation order for that construct** (see below).
- [ ] **4. Identify the deciding attribute** (longest prefix, local preference, AS-path, MED, source
type).
- [ ] **5. Flag non-determinism** (e.g., ECMP or "deterministically random" tiebreakers) and
recommend a deterministic alternative.
- [ ] **6. Cite the AWS documentation** that supports the behavior.
- [ ] **7. Provide read-only validation commands** the user can run to confirm.
- [ ] **8. Self-check before answering.** Re-read the answer against steps 1-7: every route table is
named, the evaluation order matches the construct, each cited URL was actually retrieved, and every
command is read-only. Fix any gap before presenting.

## Key evaluation rules

Expand All @@ -86,14 +92,22 @@

## Reference material

Detailed mechanics, patterns, and caveats are in `references/`:
- `cloudwan-dx-routing-patterns.md` - Cloud WAN route evaluation, DXGW path selection, community
traffic-engineering patterns, regional inspection, multi-region egress
- `dx-routing-patterns.md` - DX BGP mechanics, BFD, MED, ASN ranges, VIF considerations, CloudHub,
when VIFs can talk, active/active vs active/passive
- `tgw-routing-patterns.md` - TGW route evaluation, ECMP limits, DX/VPN caveats, failover, peering
- `vpn-dx-redundancy-patterns.md` - VPN + DX redundancy, route preference, ECMP limits, asymmetry
- `networking-strategic-questions.md` - discovery questions per service
Detailed mechanics, patterns, and caveats are in `references/`. Load only the file that matches the
construct deciding the path:
- [Cloud WAN + DX routing patterns](references/cloudwan-dx-routing-patterns.md) - load when the path
crosses a Cloud WAN CNE: CNE route evaluation, DXGW path selection into Cloud WAN, community
traffic-engineering patterns, regional inspection, multi-region egress.
- [Direct Connect routing patterns](references/dx-routing-patterns.md) - load when the question is
about a DX VIF or DXGW without Cloud WAN: BGP mechanics, BFD, MED, ASN ranges, VIF considerations,
CloudHub, when VIFs can talk, active/active vs active/passive.
- [Transit Gateway routing patterns](references/tgw-routing-patterns.md) - load when a TGW route
table makes the decision: TGW route evaluation, ECMP limits, DXGW-to-TGW behavior, DX/VPN caveats,
failover, peering.
- [VPN + DX redundancy patterns](references/vpn-dx-redundancy-patterns.md) - load when VPN backs up
or runs alongside DX: route preference between them, ECMP limits, asymmetric return paths.
- [Networking strategic questions](references/networking-strategic-questions.md) - load when key
facts are missing (regions, attachments, ASNs, communities) and you need discovery questions to ask
before analyzing.

## Output expectations

Expand Down
307 changes: 307 additions & 0 deletions skills/aws-routing/evals/best-practices/v4/benchmark.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,307 @@
{
"timestamp": "2026-10-27T22:03:07Z",
"version": "v4",
"model": "us.anthropic.claude-sonnet-5",
"iterations": 3,
"avg_confidence": "high",
"consistency": {
"consistency_score": 100,
"reports": [
{
"iteration": 1,
"result": "passed",
"score": 100,
"total": 17,
"passed": 14,
"failed": 0,
"skipped": 3,
"warning": 0,
"errored": 0,
"low_confidence": 0,
"avg_confidence": "high"
},
{
"iteration": 2,
"result": "passed",
"score": 100,
"total": 17,
"passed": 14,
"failed": 0,
"skipped": 3,
"warning": 0,
"errored": 0,
"low_confidence": 0,
"avg_confidence": "high"
},
{
"iteration": 3,
"result": "passed",
"score": 100,
"total": 17,
"passed": 14,
"failed": 0,
"skipped": 3,
"warning": 0,
"errored": 0,
"low_confidence": 0,
"avg_confidence": "high"
}
],
"per_test": [
{
"id": "BP-01",
"name": "Effective description",
"results": [
"passed",
"passed",
"passed"
],
"confidences": [
"high",
"high",
"high"
],
"avg_confidence": "high",
"consistent": true
},
{
"id": "BP-02",
"name": "Verbosity and conciseness",
"results": [
"passed",
"passed",
"passed"
],
"confidences": [
"high",
"high",
"high"
],
"avg_confidence": "high",
"consistent": true
},
{
"id": "BP-03",
"name": "Detailed reference materials must not be in body",
"results": [
"passed",
"passed",
"passed"
],
"confidences": [
"medium",
"high",
"medium"
],
"avg_confidence": "medium",
"consistent": true
},
{
"id": "BP-04",
"name": "Reference files must be linked",
"results": [
"passed",
"passed",
"passed"
],
"confidences": [
"high",
"high",
"high"
],
"avg_confidence": "high",
"consistent": true
},
{
"id": "BP-05",
"name": "Reference links must specify when to load",
"results": [
"passed",
"passed",
"passed"
],
"confidences": [
"high",
"high",
"high"
],
"avg_confidence": "high",
"consistent": true
},
{
"id": "BP-06",
"name": "Reference path format",
"results": [
"passed",
"passed",
"passed"
],
"confidences": [
"high",
"high",
"high"
],
"avg_confidence": "high",
"consistent": true
},
{
"id": "BP-07",
"name": "Prescription vs description",
"results": [
"passed",
"passed",
"passed"
],
"confidences": [
"medium",
"medium",
"medium"
],
"avg_confidence": "medium",
"consistent": true
},
{
"id": "BP-08",
"name": "Defaults, not menus",
"results": [
"passed",
"passed",
"passed"
],
"confidences": [
"medium",
"medium",
"medium"
],
"avg_confidence": "medium",
"consistent": true
},
{
"id": "BP-09",
"name": "Favor procedures over declarations",
"results": [
"passed",
"passed",
"passed"
],
"confidences": [
"high",
"high",
"high"
],
"avg_confidence": "high",
"consistent": true
},
{
"id": "BP-10",
"name": "Gotchas section",
"results": [
"passed",
"passed",
"passed"
],
"confidences": [
"medium",
"medium",
"medium"
],
"avg_confidence": "medium",
"consistent": true
},
{
"id": "BP-11",
"name": "Templates and outputs missing",
"results": [
"passed",
"passed",
"passed"
],
"confidences": [
"medium",
"medium",
"medium"
],
"avg_confidence": "medium",
"consistent": true
},
{
"id": "BP-12",
"name": "Templates and outputs must not be in body",
"results": [
"passed",
"passed",
"passed"
],
"confidences": [
"high",
"high",
"high"
],
"avg_confidence": "high",
"consistent": true
},
{
"id": "BP-13",
"name": "Asset files must be linked",
"results": [
"skipped",
"skipped",
"skipped"
],
"consistent": true
},
{
"id": "BP-14",
"name": "Asset links must specify when to load",
"results": [
"skipped",
"skipped",
"skipped"
],
"consistent": true
},
{
"id": "BP-15",
"name": "Asset path format",
"results": [
"skipped",
"skipped",
"skipped"
],
"consistent": true
},
{
"id": "BP-16",
"name": "Step-by-step guidance",
"results": [
"passed",
"passed",
"passed"
],
"confidences": [
"high",
"high",
"high"
],
"avg_confidence": "high",
"consistent": true
},
{
"id": "BP-17",
"name": "Validation loops",
"results": [
"passed",
"passed",
"passed"
],
"confidences": [
"high",
"high",
"high"
],
"avg_confidence": "high",
"consistent": true
}
]
}
}
Loading
Loading