Skip to content

Fix three coverage report misclassifications - #209

Merged
ale210 merged 1 commit into
masterfrom
coverage-script-classification-fixes
Aug 31, 2026
Merged

Fix three coverage report misclassifications#209
ale210 merged 1 commit into
masterfrom
coverage-script-classification-fixes

Conversation

@ale210

@ale210 ale210 commented Aug 31, 2026

Copy link
Copy Markdown
Member

Three resource types were being classified wrongly by the coverage report. All three were found by reconciling a full run against the open tickets on #203 — the resources turned up as "unmanaged with no ticket" when none of them can be managed at all.

1. SSM sessions were counted as unmanaged resources

The tagging API supplement has no type filter, so it added whatever resourcegroupstaggingapi get-resources returned. That includes SSM sessions: every ecs execute-command leaves one in session history, and each landed in the report as an untagged — therefore unmanaged — resource.

This is worse than it first looks. Sessions age out of history on their own, so it is not a slow leak; it means the denominator moves depending on whether anyone happened to shell into a container in the days before the run. Two runs on 2026-08-31 disagreed for exactly this reason, which matters because the coverage ratio is the metric #203 is tracked against.

Fixed with a skip list, $script:TaggingApiSkipPatterns, so another such type can be added later without restructuring the collector.

2. IAM Identity Center reserved roles were counted as unmanaged

AWSReservedSSO_AdministratorAccess_… sits under /aws-reserved/. Identity Center re-provisions these from permission sets and discards tags written to them, so they cannot hold managed-by — the same situation as the /aws-service-role/ case the script already handled one branch above.

3. Lambda layers were not collected at all

Add-LambdaResources called lambda list-functions only, and the tagging API does not return layers, so they were invisible. This was found the hard way: the layer sqlalchemy:2 was part of the multi-tenant-db stack and had to be read out of the function's config, because nothing in the report knew it existed.

The taggable unit is the layer, not the layer version. lambda list-tags accepts arn:...:layer:<name> and rejects arn:...:layer:<name>:<version> with a ValidationException, so versions are counted through their layer — the same shape as ECS task definition revisions being counted through their family.

Effect

Verified against 035866691871 before and after. Unmanaged 90 → 81, of which 6 were the separately-deleted multi-tenant-db stack, so this change accounts for 3: two sessions dropped and the SSO role reclassified. Coverage 71.1% → 71.9%.

Untaggable gains one row with a new reason, IAM Identity Center reserved role. No resource moved into or out of terraform-incubator or terraform-devops-security, which is the property that matters — this changes only what is counted, never what is reported as managed.

Testing

  • Full run against 035866691871 across both regions, before and after, diffed by ARN.
  • Asserted after the run: zero session/ecs-execute-command rows, the SSO role present as untaggable with the new note, and the managed buckets unchanged at 191 / 16.
  • Test-TaggingApiSkip checked against sessions in both regions plus three ARNs that must not match, including ssm:...:parameter/rds_credentials.
  • Parse-checked with PSParser::Tokenize.

The layer collector currently runs against an empty set — the account has zero Lambda layers now that sqlalchemy is gone, so the loop is exercised but adds no rows. The ARN shape and the tagging behaviour were confirmed against the live API before it was deleted.

Not included

Four resources also came out of that reconciliation as unmanageable-but-counted — OrganizationAccountAccessRole, the IAM Access Analyzer service-role pair, and the default.postgres13 / default.postgres15 parameter groups. These are all genuinely taggable, so marking them untaggable in the script would be a lie. They want a hand-applied managed-by=exempt tag instead, which is a change to AWS rather than to this script, and in the case of OrganizationAccountAccessRole a decision first — it is a standing cross-account AdministratorAccess path, so "exempt" may be the wrong answer.

Both blind-spot lists are updated, the one in README.md and the one the script prints after every run.

Skip SSM sessions in the tagging API supplement, treat IAM Identity
Center reserved roles as untaggable, and collect Lambda layers.
@ale210
ale210 merged commit 9fc3bff into master Aug 31, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant