Skip to content

Feature request: Parser support for EventBridge WITH_METADATA subscriber payloads #5761

Description

@svozza

Use case

Problem statement

Add Parser support for Amazon EventBridge Custom Event Bus subscriber deliveries using the WITH_METADATA transformer, where Data contains a standard EventBridge event.

The current AWS guide calls this Custom Event Bus, distinguishes it from Custom Event Bus - Classic, and uses EventBridgeV2 / eventsv2 for the SDK / CLI. This request concerns the enhanced subscriber delivery described by that guide.

Lambda consumers need to validate the outer delivery structure, inspect delivery metadata, and validate application data inside Data.detail without maintaining a separate copy of Powertools' EventBridge schema.

Current behavior

In @aws-lambda-powertools/parser 2.35.0:

  • EventBridgeSchema validates a single classic EventBridge event.
  • EventBridgeEnvelope validates that event and extracts its detail.
  • Neither accepts a subscriber batch of { Data, Metadata, SystemMetadata } records or validates its metadata. Applications must define the wrapper themselves.

Illustrative FIFO input that should be supported (using generic application values):

[
  {
    "Data": {
      "version": "0",
      "id": "envelope-event-id",
      "detail-type": "OrderCreated",
      "source": "com.example.orders",
      "account": "123456789012",
      "time": "2026-09-30T12:00:00Z",
      "region": "eu-west-1",
      "resources": [],
      "detail": {
        "orderId": "order-123"
      }
    },
    "Metadata": {},
    "SystemMetadata": {
      "ContentType": "application/eventbridge+json",
      "DeduplicationId": "order-created-123",
      "EventGroupId": "customer-123",
      "aws:DeliveryType": "LIVE",
      "aws:EventId": "bus-event-id",
      "aws:Source": "com.example.orders",
      "aws:DetailType": "OrderCreated",
      "aws:IngestionTime": "2026-09-30T12:00:00Z",
      "aws:SequenceNumber": "900719925474099301"
    }
  }
]

Solution/User Experience

Service contract

The official documentation establishes the following:

  • WITH_METADATA delivers Data, Metadata, and SystemMetadata. With PutEvents, Data is the standard EventBridge event. With PutRawEvents, it can instead contain arbitrary JSON or encoded binary data.
  • Lambda receives an array when BatchConfiguration is omitted or MaxBatchSize is greater than 1, including when an actual batch contains only one event. Explicitly setting MaxBatchSize: 1 produces a single object.
  • The SystemMetadata documentation explicitly states: "Every value is a string" and "only aws:DeliveryType is always present." This is the basis for the proposed required/optional fields; finite live samples cannot establish universal requiredness.
  • Subscriber types are UNORDERED and FIFO. EventGroupId supplies grouping for FIFO ordering. DeduplicationId is a publishing concern and also applies to ungrouped events. The documentation describes aws:SequenceNumber as the event's position on the bus, not a field exclusive to FIFO subscribers.
  • Metadata is a string-to-string map supplied through PutRawEvents. The guide describes it as absent or empty for PutEvents; the live PutEvents captures below all contained Metadata: {}. Accepting omission is a compatibility choice based on the documentation, not an observed omission in these captures.

Consequently, do not require grouping, deduplication, or sequence metadata on every record, or infer subscriber type from the presence of those fields. Do not infer that an omitted publishing input will be absent from delivered metadata: the service generated grouping and deduplication values in the live tests below.

Live Lambda validation

Validated on 2026-09-30 in eu-west-1, using AWS CLI 2.37.7, with a temporary Custom Event Bus and a real Lambda capture function. Nine subscribers targeted separate Lambda aliases using WITH_METADATA and LambdaParameters.InvocationType: REQUEST_RESPONSE. Events were delivered by EventBridge; the capture function was not invoked directly with fabricated payloads.

Published 16 PutEvents events and 2 PutRawEvents JSON events. Captured 38 Lambda invocations containing 66 records: 50 live records and 16 replay records. Counts include delivery to multiple subscribers, not 66 distinct published events.

Subscriber configuration Tested types Observed Lambda payload
BatchConfiguration omitted UNORDERED, FIFO Arrays, including a one-element array and a six-element array. DescribeSubscriber resolved the configuration to size 10 / window 0.
MaxBatchSize: 1, window 0 UNORDERED, FIFO A single wrapper object on every invocation.
MaxBatchSize: 3, window 5 seconds UNORDERED, FIFO Arrays containing one or three records.
StartingPosition: POINT_IN_TIME, PointType: HORIZON, size 3 UNORDERED, FIFO Arrays containing two or three records, all with aws:DeliveryType: "REPLAY".
Raw JSON, batching omitted UNORDERED One-element arrays; Data was the application JSON, with no standard EventBridge event structure.

Additional checks and qualifications:

  • Every one of the 64 delivered PutEvents records passed the repository's existing EventBridgeSchema when validating Data. The outer invocation payloads failed that schema, confirming the missing-wrapper support. Raw JSON Data failed it as expected.
  • All 66 delivered sequence numbers were strings larger than Number.MAX_SAFE_INTEGER and matched the corresponding PutEvents/PutRawEvents response exactly; one observed value was "10000000000000003000".
  • UNORDERED events published with no SystemMetadata still arrived with service-generated EventGroupId and DeduplicationId. A deduplication ID without an explicit group, and an explicit group without a deduplication ID, were also accepted and delivered. Their presence does not identify a FIFO subscriber.
  • Data.id differed from SystemMetadata["aws:EventId"] in all 64 standard-event captures. The latter matched the publishing response's EventId. The schema must not require these two identifiers to match.
  • Every standard-event capture contained Metadata: {}. A raw JSON event without producer metadata also contained {}; one with Metadata: { "tenant": "example", "purpose": "parser-validation" } retained that exact map.
  • Every captured system metadata value was a string. aws:Source and aws:DetailType were present for the standard events and absent from the raw JSON events. aws:SchemaId and aws:RegistryType were not exercised.

The samples verify these configurations, not every service configuration or region. In particular, they do not prove that frequently observed fields are universally required: all captures included event IDs, ingestion times, sequence numbers, grouping IDs, and deduplication IDs. Their proposed optionality follows the explicit documentation quoted above. The nonempty-array rule and envelope return type below are Parser design proposals, not service guarantees inferred from the samples. All temporary AWS resources were removed after collecting the evidence.

Proposed API

Recommend the transformer-specific names below, subject to maintainer naming review:

Export Behavior
EventBridgeWithMetadataSystemMetadataSchema Validates the documented delivery metadata.
EventBridgeWithMetadataRecordSchema Validates one wrapper, reusing EventBridgeSchema for Data, requiring SystemMetadata, and accepting optional Metadata: Record<string, string>.
EventBridgeWithMetadataSchema Validates a nonempty array of those records, following existing Parser batch schema conventions.
EventBridgeWithMetadataEnvelope Validates a batch and each Data.detail against a caller-supplied Zod schema, returning the parsed detail values as T[] in input order.

Expose inferred types EventBridgeWithMetadataSystemMetadata, EventBridgeWithMetadataRecord, and EventBridgeWithMetadataEvent through /types; the last represents the batch.

System metadata validation should follow the documented wire contract:

Field Validation
aws:DeliveryType Required enum: LIVE or REPLAY.
ContentType Optional string; documented as application/eventbridge+json for PutEvents.
EventGroupId, DeduplicationId Optional strings.
aws:EventId, aws:IngestionTime, aws:SequenceNumber Optional strings.
aws:Source, aws:DetailType, aws:SchemaId, aws:RegistryType Optional strings, also listed in the service contract.

Keep aws:SequenceNumber as a string throughout parsing and type inference. Reject numeric input rather than coercing it. Do not add timestamp, identifier-equality, or sequence-format constraints beyond the documented contract; the existing EventBridgeSchema continues to validate Data.time.

The initial scope is standard EventBridge events inside Data, such as those produced by PutEvents. These APIs should not claim to validate every possible PutRawEvents payload. The optional metadata map models the wrapper without weakening Data to unknown.

Proposed usage:

import { parse } from '@aws-lambda-powertools/parser';
import {
  EventBridgeWithMetadataEnvelope,
} from '@aws-lambda-powertools/parser/envelopes';
import {
  EventBridgeSchema,
  EventBridgeWithMetadataRecordSchema,
} from '@aws-lambda-powertools/parser/schemas';
import { z } from 'zod';

const detailSchema = z.object({ orderId: z.string() });

export const handler = async (event: unknown) => {
  const details = parse(event, EventBridgeWithMetadataEnvelope, detailSchema);
  // Inferred as { orderId: string }[].
  return details;
};

// Extend the record schema when the handler needs delivery metadata too.
const recordSchema = EventBridgeWithMetadataRecordSchema.extend({
  Data: EventBridgeSchema.extend({ detail: detailSchema }),
});
const batchSchema = recordSchema.array().nonempty();

Use recordSchema directly for the single-object delivery produced by MaxBatchSize: 1. Keep the batch schema and envelope array-based so their return types remain predictable.

The envelope should follow the existing ArrayEnvelope contract:

  • parse<T>(event, detailSchema): T[].
  • safeParse<T>(event, detailSchema): ParsedResult<unknown, T[]>.
  • Validate the entire wrapper and metadata before returning details.
  • Throw ParseError with the underlying validation error as its cause; safeParse returns success: false, the error, and the original input.
  • Preserve validation paths such as [1, "Data", "detail", "orderId"] and [0, "SystemMetadata", "aws:DeliveryType"]. An invalid record fails the batch; the envelope does not silently discard records or implement partial batch responses.

Expose schemas through /schemas and /schemas/eventbridge-with-metadata, the envelope through /envelopes and /envelopes/eventbridge-with-metadata, and inferred types through /types, for both ESM and CJS. Follow the existing barrel, wildcard export, and typesVersions conventions; add export mappings only where needed.

Acceptance criteria

  • Parse valid one-event and multi-event WITH_METADATA arrays and preserve their order; reject empty arrays consistently with the proposed batch contract.
  • Reuse EventBridgeSchema for every Data value, including its existing optional replay-name support. Accept distinct Data.id and SystemMetadata["aws:EventId"].
  • Preserve "900719925474099301" and the observed "10000000000000003000" exactly as strings, infer sequence numbers as string | undefined, and reject numbers without coercion.
  • Support both LIVE and REPLAY; reject missing or unsupported delivery types.
  • Accept captured UNORDERED and FIFO deliveries with grouping/deduplication/sequence fields present, including service-generated values. Add synthetic omission tests based on documented optionality, including metadata containing only aws:DeliveryType; do not present those omissions as observed service output or treat these fields as proof of FIFO.
  • Reject malformed Data, missing/null/non-object SystemMetadata, and non-string values for documented optional metadata fields when present. Accept absent or empty Metadata, validate supplied map values as strings, and retain that map when using the full schema.
  • Validate a caller-supplied detail schema with the envelope, including transformations, parse/safeParse, failure paths, and whole-batch failure behavior. Demonstrate extending the record schema to retain metadata.
  • Export schemas, inferred types, and the envelope from the documented public ESM/CJS entry points; verify package builds and consumer imports.
  • Add unit tests covering the above with the project's required 100% source coverage, type tests for array/output inference and metadata optionality, and documentation in docs/features/parser.md backed by compiling examples under examples/snippets/parser/. Cover manual parsing, middleware/decorator use, and the MaxBatchSize: 1 object exception.
  • Confirm final naming and behavior during maintainer review against existing EventBridgeSchema, record schemas such as SqsRecordSchema, ArrayEnvelope, and schema-extension conventions. Existing classic EventBridge APIs retain their behavior.

Official references

Reviewed on 2026-09-30:

Alternative solutions

  • Application-owned Zod wrapper around EventBridgeSchema: works today but duplicates delivery validation, envelope handling, tests, and maintenance across applications.
  • Extend EventBridgeSchema/EventBridgeEnvelope to accept both formats: mixes object and batch contracts and complicates existing inferred return types. Separate exports preserve the current API.
  • EventBridgeV2SubscriberSchema / EventBridgeV2SubscriberEnvelope: consistent with AWS's documented SDK naming and existing Powertools V2 names, but a subscriber can also deliver RAW, JSONATA, and arbitrary PutRawEvents data. WithMetadata identifies this proposal's transformer shape more directly.
  • EnhancedEventBridgeSubscriberSchema: understandable, but the current guide uses “Custom Event Bus.” The transformer name is a more precise descriptor.
  • Generic WITH_METADATA schema accepting arbitrary Data: useful as a potential separate capability, but would not itself validate the standard EventBridge structure requested here.
  • Return metadata from the envelope: handlers needing metadata can extend the record schema, consistent with Parser's documented distinction between envelopes and full-event validation.

Acknowledgment

Future readers

Please react with 👍 and your use case to help us understand customer demand.

Activity

  1. added
    triageThis item has not been triaged by a maintainer, please wait
    on Sep 30, 2026
  2. added theissue type on Sep 30, 2026
  3. removed
    triageThis item has not been triaged by a maintainer, please wait
    on Oct 2, 2026
  4. proton0210 commented on Oct 4, 2026

    @proton0210
    Contributor

    Happy to take this on—I've opened #5776. Please assign it to me.

  5. powertools-for-aws-oss-automation commented on Oct 4, 2026

    @powertools-for-aws-oss-automation

    Warning

    This issue is now closed. Please be mindful that future comments are hard for our team to see.
    If you need more assistance, please either reopen the issue, or open a new issue referencing this one.
    If you wish to keep having a conversation with other community members under this issue feel free to do so.

  6. added
    pending-releaseThis item has been merged and will be released soon
    and removed
    pending-releaseThis item has been merged and will be released soon
    on Oct 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions