ci: Handle concurrent deletions in CI

[ystros]

This pipeline exists in the ubuntu-jammy, ubuntu-noble, and ubuntu-resolute
pipelines. They all run at the same time. The cleanup of 3 year old images is
looking at ALL distributions in each pipeline. This caused failures where one
pipeline would delete an AMI and then the other 2 pipelines would try to delete
that AMI again and fail.

We cannot limit the cleanup to the current pipeline's OS, because we want to
clean up old AMIs even after a stemcell line is EOL and its pipeline has been
deleted. So this commit makes the cleanup script tolerant to the AMI having
been deleted.

This also removes an unused snapshot_id cleanup mechanism.

[coderabbitai]

Walkthrough

The cleanup task removes the snapshot_id parameter and no longer filters AMIs using it. The shell script adds a helper that treats selected missing AWS EC2 resources as successful deletions while propagating other errors. Image deregistration and snapshot deletion now use this helper, including snapshots that previously could be skipped.

Suggested reviewers: aramprice

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check	⚠️ Warning	The description explains the change, but it omits required template sections like Merge Forward guidance and AI Review Feedback.	Add the template sections, including Merge Forward branch guidance and AI Review Feedback status for each AI comment.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly summarizes the main change: CI cleanup now handles concurrent deletions.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch update-cleanup-scripts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Alphasite]

i think its functionally fine, but have a few minor nits. My main concern is around snapshot_id, but i figure you did your due diligence

[Alphasite]

nit: echo should probably be printf

[Alphasite]

gemini seems to think there is 1 more condition which may be worth a looksie at:

InvalidSnapshot.InUse Race Condition
When multiple pipelines are sweeping AMIs, they usually follow two steps:

1. Deregister the AMI.
2. Delete the underlying EBS Snapshots.

Because AWS is eventually consistent, if Pipeline A deregisters the AMI, AWS takes a few moments to "unlock" the underlying snapshot. If Pipeline B hits the snapshot deletion step during this window, AWS will throw InvalidSnapshot.InUse (not NotFound). If this function doesn't tolerate InUse in a concurrent environment, Pipeline B will fail completely.

[Alphasite]

nit: should this be named something like: run_tolerate_missing_errors or something that doesn't specifically mention delete since its also used for deregister.

[Copilot]

Pull request overview

This PR updates the Light AWS AMI cleanup task to tolerate expected “already deleted” errors caused by concurrent pipeline runs cleaning up the same shared AWS account, preventing CI failures when multiple pipelines attempt to delete the same AMI/snapshot.

Changes:

• Add a delete_tolerating_missing wrapper to treat “not found/unavailable” AWS CLI delete errors as non-fatal.
• Use the wrapper for aws ec2 deregister-image and aws ec2 delete-snapshot operations.
• Remove the unused snapshot_id parameter and its associated cleanup/query logic.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
ci/tasks/light-aws/cleanup-ami.yml	Removes the unused snapshot_id task param to match current script behavior.
ci/tasks/light-aws/cleanup-ami.sh	Adds collision-tolerant deletion wrapper and applies it to AMI/snapshot deletion; removes obsolete snapshot-id-based logic.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.