Skip to content

cloudwatch-log-alarm-lambda-remediation-cdk: Log-based alarm with auto-remediation#3228

Open
NithinChandranR-AWS wants to merge 1 commit into
aws-samples:mainfrom
NithinChandranR-AWS:NithinChandranR-AWS-feature-cloudwatch-log-alarm-lambda-remediation-cdk
Open

cloudwatch-log-alarm-lambda-remediation-cdk: Log-based alarm with auto-remediation#3228
NithinChandranR-AWS wants to merge 1 commit into
aws-samples:mainfrom
NithinChandranR-AWS:NithinChandranR-AWS-feature-cloudwatch-log-alarm-lambda-remediation-cdk

Conversation

@NithinChandranR-AWS

@NithinChandranR-AWS NithinChandranR-AWS commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Description

Monitor application logs with Amazon CloudWatch Logs Insights queries and auto-remediate via AWS Lambda and AWS Systems Manager when error thresholds are breached.

Key Innovation

Uses the new AWS::CloudWatch::LogAlarm CloudFormation resource (GA July 2026) which eliminates the traditional 3-resource workaround (MetricFilter + Metric + Standard Alarm). A single LogAlarm resource runs a Logs Insights query on schedule and evaluates results against a threshold directly.

Architecture

Amazon CloudWatch Logs → Amazon CloudWatch Log Alarm (Logs Insights query, 5-min schedule) → Amazon SNS → AWS Lambda (remediation) → AWS Systems Manager RunCommand

How it works

  1. Application logs flow to Amazon CloudWatch Logs
  2. Log Alarm runs filter @message like /ERROR/ | stats count(*) every 5 minutes
  3. When error count ≥ 5, alarm publishes to Amazon SNS
  4. AWS Lambda classifies the alarm and sends AWS Systems Manager RunCommand to restart the service on tagged Amazon EC2 instances

Deployed and Tested

Stack deploys in us-east-1 (~50s). Full notification chain verified:

  • LogAlarm evaluates Logs Insights query on schedule
  • Alarm state transition triggers Amazon SNS notification
  • AWS Lambda invoked with alarm payload, correctly parses state and reason
  • AWS Systems Manager RunCommand targets instances tagged AutoRemediate=true

…ediation

Deploy self-healing architecture using AWS::CloudWatch::LogAlarm (new
CFN resource, July 2026). Logs Insights query counts ERROR messages
every 5 minutes, triggers alarm when threshold breached, Amazon SNS
fans out to AWS Lambda which runs remediation via AWS Systems Manager
RunCommand on tagged Amazon EC2 instances.

Eliminates the 3-resource MetricFilter workaround — single LogAlarm
resource replaces MetricFilter + Metric + Standard Alarm. 5 services
composed: Amazon CloudWatch Logs, Log Alarm, Amazon SNS, AWS Lambda,
AWS Systems Manager.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants