Skip to content

dsql-cdc-eventbridge-fanout-cdk: DSQL CDC to EventBridge fan-out#3227

Open
NithinChandranR-AWS wants to merge 2 commits into
aws-samples:mainfrom
NithinChandranR-AWS:NithinChandranR-AWS-feature-dsql-cdc-eventbridge-fanout-cdk
Open

dsql-cdc-eventbridge-fanout-cdk: DSQL CDC to EventBridge fan-out#3227
NithinChandranR-AWS wants to merge 2 commits into
aws-samples:mainfrom
NithinChandranR-AWS:NithinChandranR-AWS-feature-dsql-cdc-eventbridge-fanout-cdk

Conversation

@NithinChandranR-AWS

@NithinChandranR-AWS NithinChandranR-AWS commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Description

Stream real-time database changes from Amazon Aurora DSQL via CDC to Amazon EventBridge for content-based routing to multiple consumers.

Architecture

Amazon Aurora DSQL (CDC) → Amazon Kinesis Data Streams → AWS Lambda (CDC processor) → Amazon EventBridge (custom bus) → 3 consumers:

  • Amazon SQS: Audit trail (ALL change events)
  • AWS Step Functions: Data validation workflow (INSERT events)
  • Amazon SNS: Real-time alerting (DELETE events)

What makes this different

This pattern is fundamentally different from:

  • apigw-lambda-dsql (existing, basic CRUD) — that treats DSQL as a target. This uses DSQL as an event source.
  • sample-aurora-dsql-cdc-demo (separate repo) — that builds a RAG/chatbot with CDC → embeddings → S3 Vectors. This demonstrates event-driven microservices fan-out with content-based routing.

7 AWS services composed. Removing any one breaks the architecture.

Implementation Notes

  • Custom Resource Lambda manages DSQL CDC stream lifecycle (no CFN resource type yet)
  • CDC stream uses boto3 create_stream API (GA July 8, 2026)
  • Bundled boto3 1.43.46+ since Lambda runtime doesn't ship it yet
  • CDC format is Debezium-style: op='c'/'u'/'d' with before/after payload

Deployed and Tested

Stack deploys in us-east-1. All 3 fan-out targets verified:

  • Amazon SQS: 4 messages received (INSERT + UPDATE + DELETE + initial)
  • AWS Step Functions: 1 execution SUCCEEDED (INSERT validation)
  • Amazon SNS: DELETE event routed to alerting topic
  • AWS Lambda CDC processor: 3 records processed, 3 events published, 0 failures

Event-driven microservices pattern using Amazon Aurora DSQL CDC (GA July
2026) as a change event source. CDC events flow through Amazon Kinesis
to AWS Lambda, which classifies operations and publishes typed events to
Amazon EventBridge. Three independent consumers demonstrate content-based
routing: Amazon SQS (audit), AWS Step Functions (validation), Amazon SNS
(alerting). 7 AWS services, 29 CFN resources.

Fundamentally different from existing apigw-lambda-dsql (basic CRUD) and
sample-aurora-dsql-cdc-demo (RAG/semantic search). This pattern shows
DSQL as an event source for microservices fan-out via EventBridge.
…e boto3

DSQL CDC uses Debezium-style format: op='c'/'u'/'d' with before/after
fields. Map op codes to human-readable types for EventBridge routing.
Bundle boto3>=1.43.36 via CDK Docker bundling (Lambda runtime ships
older version without dsql.create_stream support).

Deployed and tested end-to-end:
- 3 records processed, 3 events published, 0 failures
- Amazon SQS audit: receives all 3 event types (INSERT, UPDATE, DELETE)
- AWS Step Functions: 1 INSERT validation SUCCEEDED
- Amazon SNS: DELETE event routed to alerting topic
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants