From a5c76b63c8fcbc03c774532cd191189a59caa8d2 Mon Sep 17 00:00:00 2001 From: Doug Perkes Date: Fri, 17 Jul 2026 09:22:19 -0500 Subject: [PATCH] New serverless pattern - sam-dotnet-responsestreaming-bedrock --- .../.gitignore | 16 ++ .../README.md | 183 ++++++++++++++++++ .../events/generate-story.json | 31 +++ .../sam-dotnet-responsestreaming-bedrock.json | 64 ++++++ .../src/Program.cs | 105 ++++++++++ .../src/StoryStreaming.csproj | 19 ++ .../template.yaml | 90 +++++++++ 7 files changed, 508 insertions(+) create mode 100644 sam-dotnet-responsestreaming-bedrock/.gitignore create mode 100644 sam-dotnet-responsestreaming-bedrock/README.md create mode 100644 sam-dotnet-responsestreaming-bedrock/events/generate-story.json create mode 100644 sam-dotnet-responsestreaming-bedrock/sam-dotnet-responsestreaming-bedrock.json create mode 100644 sam-dotnet-responsestreaming-bedrock/src/Program.cs create mode 100644 sam-dotnet-responsestreaming-bedrock/src/StoryStreaming.csproj create mode 100644 sam-dotnet-responsestreaming-bedrock/template.yaml diff --git a/sam-dotnet-responsestreaming-bedrock/.gitignore b/sam-dotnet-responsestreaming-bedrock/.gitignore new file mode 100644 index 000000000..eb9697107 --- /dev/null +++ b/sam-dotnet-responsestreaming-bedrock/.gitignore @@ -0,0 +1,16 @@ +## .NET +bin/ +obj/ +*.user +*.suo +*.userosscache +*.sln.docstates + +## SAM +.aws-sam/ +samconfig.toml + +## IDE +.idea/ +.vs/ +*.swp diff --git a/sam-dotnet-responsestreaming-bedrock/README.md b/sam-dotnet-responsestreaming-bedrock/README.md new file mode 100644 index 000000000..aa541dadc --- /dev/null +++ b/sam-dotnet-responsestreaming-bedrock/README.md @@ -0,0 +1,183 @@ +# AWS Lambda Response Streaming with Amazon Bedrock — Story Generator (.NET) + +This sample demonstrates **Lambda Response Streaming through Amazon API Gateway** combined with **Amazon Bedrock** (Claude Sonnet) for .NET. A Lambda function invokes Claude Sonnet via the Bedrock `ConverseStream` API and streams the generated story token-by-token back to the client through API Gateway response streaming. + +## Architecture + +``` +┌──────────────┐ ┌──────────────────┐ ┌────────────────────┐ ┌─────────────┐ +│ Client │ │ API Gateway │ │ Lambda Function │ │ Amazon │ +│ (curl) │──GET─▶│ REST API │──────▶│ (Response │──────▶│ Bedrock │ +│ │◀─stream│ ResponseTransfer │◀stream│ Streaming) │◀stream│ (Claude) │ +│ │ │ Mode: STREAM │ │ │ │ │ +└──────────────┘ └──────────────────┘ └────────────────────┘ └─────────────┘ +``` + +## What It Demonstrates + +- **End-to-end streaming** — Bedrock streams tokens → Lambda pipes them through → API Gateway streams to client. The user sees the story appear word-by-word. +- **API Gateway response streaming** — Uses `ResponseTransferMode: STREAM` with the `/response-streaming-invocations` Lambda endpoint. +- **Bedrock ConverseStream API** — Uses the AWS SDK's `ConverseStreamAsync` to get streaming responses from Claude Sonnet. +- **Customizable story generation** — Users can specify genre, setting, and theme via query parameters. +- **Minimal latency** — First tokens appear within seconds, no waiting for the full generation to complete. + +## Project Structure + +``` +├── template.yaml # SAM template (API Gateway REST API + Lambda + Bedrock IAM) +├── events/ +│ └── generate-story.json # Sample API Gateway proxy event +└── src/ + ├── Program.cs # Streaming handler (Bedrock → Lambda → API Gateway → Client) + └── StoryStreaming.csproj +``` + +--- + +## Prerequisites + +- [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10.0) +- [AWS SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html) +- AWS account with credentials configured (`aws configure`) +- **Amazon Bedrock model access** — Enable access to Claude Sonnet in the [Bedrock console](https://console.aws.amazon.com/bedrock/home#/modelaccess) + +--- + +## Build & Deploy + +### Build + +```bash +sam build +``` + +### Deploy + +```bash +sam deploy --guided +``` + +Follow the prompts to configure stack name, region, and confirm IAM role creation. On subsequent deploys: + +```bash +sam deploy +``` + +--- + +## Testing + +### 1. Get the API endpoint + +```bash +API_URL=$(aws cloudformation describe-stacks \ + --stack-name sam-dotnet-responsestreaming-bedrock \ + --query "Stacks[0].Outputs[?OutputKey=='ApiEndpoint'].OutputValue" \ + --output text) +``` + +### 2. Generate a story with defaults + +```bash +curl --no-buffer "$API_URL" +``` + +You'll see the story appear word-by-word in real-time as Claude generates it. + +### 3. Customize the story + +```bash +# Science fiction story +curl --no-buffer "$API_URL?genre=science+fiction&setting=a+space+station+orbiting+Jupiter&theme=finding+home" + +# Mystery story +curl --no-buffer "$API_URL?genre=mystery&setting=a+foggy+Victorian+London+street&theme=trust+and+betrayal" + +# Horror story +curl --no-buffer "$API_URL?genre=horror&setting=an+abandoned+hospital&theme=facing+your+fears" +``` + +### Parameters + +| Parameter | Default | Description | +|-----------|---------|-------------| +| `genre` | fantasy | The genre of the story (e.g., science fiction, mystery, horror, romance) | +| `setting` | a mysterious ancient library | Where the story takes place | +| `theme` | the power of curiosity | The central theme or message | + +--- + +## How It Works + +### API Gateway → Lambda (Response Streaming) + +The API Gateway integration uses `/response-streaming-invocations` to invoke Lambda with streaming: + +```yaml +Integration: + Type: AWS_PROXY + IntegrationHttpMethod: POST + ResponseTransferMode: STREAM + Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2021-11-15/functions/${StoryFunction.Arn}/response-streaming-invocations +``` + +### Lambda → Bedrock (ConverseStream) + +The Lambda function calls Bedrock's streaming API and pipes each token to the response stream: + +```csharp +// Set up Lambda response streaming +using var responseStream = LambdaResponseStreamFactory.CreateHttpStream(prelude); +using var writer = new StreamWriter(responseStream); + +// Call Bedrock with streaming +var response = await bedrockClient.ConverseStreamAsync(converseRequest); + +// Pipe each token from Bedrock → API Gateway → Client +foreach (var item in response.Stream.AsEnumerable()) +{ + if (item is ContentBlockDeltaEvent deltaEvent && deltaEvent.Delta?.Text is not null) + { + await writer.WriteAsync(deltaEvent.Delta.Text); + await writer.FlushAsync(); // Sends immediately to the client + } +} +``` + +Key points: +- `ConverseStreamAsync` returns an event stream that yields `ContentBlockDeltaEvent` items as Claude generates tokens. +- Each `FlushAsync()` pushes the token(s) through API Gateway to the client immediately. +- The model used is `us.anthropic.claude-sonnet-5` (cross-region inference profile for higher availability). +- `Temperature: 0.8` and `TopP: 0.9` produce creative, varied stories. + +--- + +## Cleanup + +```bash +sam delete +``` + +--- + +## Useful Commands + +| Command | Description | +|---------|-------------| +| `sam build` | Build the Lambda function | +| `sam deploy --guided` | Deploy with interactive prompts | +| `sam deploy` | Deploy with saved config | +| `sam logs --tail` | Tail CloudWatch logs | +| `sam delete` | Tear down the stack | +| `dotnet build src/` | Build locally without SAM | + +--- + +## References + +- [Building responsive APIs with API Gateway response streaming](https://aws.amazon.com/blogs/compute/building-responsive-apis-with-amazon-api-gateway-response-streaming/) +- [Serverless strategies for streaming LLM responses](https://aws.amazon.com/blogs/compute/serverless-strategies-for-streaming-llm-responses/) +- [Lambda Response Streaming (.NET SDK PR)](https://github.com/aws/aws-lambda-dotnet/pull/2288) +- [Amazon Bedrock ConverseStream API](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html) +- [Claude Sonnet on Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-anthropic-claude-sonnet-5.html) +- [AWS SAM Developer Guide](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/) diff --git a/sam-dotnet-responsestreaming-bedrock/events/generate-story.json b/sam-dotnet-responsestreaming-bedrock/events/generate-story.json new file mode 100644 index 000000000..fb500291c --- /dev/null +++ b/sam-dotnet-responsestreaming-bedrock/events/generate-story.json @@ -0,0 +1,31 @@ +{ + "resource": "/story", + "path": "/story", + "httpMethod": "GET", + "headers": { + "Accept": "*/*", + "Host": "abc123.execute-api.us-east-1.amazonaws.com" + }, + "queryStringParameters": { + "genre": "science fiction", + "setting": "a space station orbiting Jupiter", + "theme": "finding home in unexpected places" + }, + "pathParameters": null, + "stageVariables": null, + "requestContext": { + "resourceId": "abc123", + "resourcePath": "/story", + "httpMethod": "GET", + "requestId": "test-request-id", + "accountId": "123456789012", + "stage": "prod", + "identity": { + "sourceIp": "127.0.0.1", + "userAgent": "curl/8.0" + }, + "apiId": "abc123" + }, + "body": null, + "isBase64Encoded": false +} diff --git a/sam-dotnet-responsestreaming-bedrock/sam-dotnet-responsestreaming-bedrock.json b/sam-dotnet-responsestreaming-bedrock/sam-dotnet-responsestreaming-bedrock.json new file mode 100644 index 000000000..e329ed399 --- /dev/null +++ b/sam-dotnet-responsestreaming-bedrock/sam-dotnet-responsestreaming-bedrock.json @@ -0,0 +1,64 @@ +{ + "title": "Lambda Response Streaming with Amazon Bedrock — Story Generator (.NET)", + "description": "Stream LLM-generated story text token-by-token from Amazon Bedrock through Lambda response streaming and API Gateway to the client.", + "language": ".NET", + "level": "200", + "framework": "SAM", + "introBox": { + "headline": "How it works", + "text": [ + "This pattern deploys a Lambda function that invokes Amazon Bedrock (Claude Sonnet) via the ConverseStream API and pipes generated tokens directly through API Gateway response streaming to the client.", + "The user specifies a genre, setting, and theme via query parameters, and the story appears word-by-word in the terminal. First tokens arrive within seconds — no waiting for the full generation to complete.", + "This demonstrates end-to-end streaming from Bedrock through Lambda to the client using the .NET response streaming SDK (LambdaResponseStreamFactory.CreateHttpStream)." + ] + }, + "gitHub": { + "template": { + "repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/sam-dotnet-responsestreaming-bedrock", + "templateURL": "serverless-patterns/sam-dotnet-responsestreaming-bedrock", + "projectFolder": "sam-dotnet-responsestreaming-bedrock", + "templateFile": "template.yaml" + } + }, + "resources": { + "bullets": [ + { + "text": "Lambda Response Streaming Documentation", + "link": "https://docs.aws.amazon.com/lambda/latest/dg/response-streaming.html" + }, + { + "text": "Amazon Bedrock ConverseStream API", + "link": "https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-call.html" + }, + { + "text": "Amazon Bedrock", + "link": "https://aws.amazon.com/bedrock/" + } + ] + }, + "deploy": { + "text": [ + "sam build", + "sam deploy --guided" + ] + }, + "testing": { + "text": [ + "See the GitHub repo for detailed testing instructions." + ] + }, + "cleanup": { + "text": [ + "Delete the stack: sam delete." + ] + }, + "authors": [ + { + "name": "Doug Perkes", + "image": "", + "bio": "Senior Solutions Architect at AWS, focused on .NET and serverless.", + "linkedin": "dougperkes", + "twitter": "" + } + ] +} diff --git a/sam-dotnet-responsestreaming-bedrock/src/Program.cs b/sam-dotnet-responsestreaming-bedrock/src/Program.cs new file mode 100644 index 000000000..502a67b06 --- /dev/null +++ b/sam-dotnet-responsestreaming-bedrock/src/Program.cs @@ -0,0 +1,105 @@ +using System.Net; +using Amazon.BedrockRuntime; +using Amazon.BedrockRuntime.Model; +using Amazon.Lambda.APIGatewayEvents; +using Amazon.Lambda.Core; +using Amazon.Lambda.Core.ResponseStreaming; +using Amazon.Lambda.RuntimeSupport; +using Amazon.Lambda.Serialization.SystemTextJson; + +#pragma warning disable CA2252 // Opt in to preview features (response streaming) + +var bedrockClient = new AmazonBedrockRuntimeClient(); + +// The function handler that will be called for each Lambda event. +// Invokes Claude Sonnet via Bedrock ConverseStream and pipes the response +// through API Gateway response streaming to the client. +var handler = async (APIGatewayProxyRequest request, ILambdaContext context) => +{ + // Parse optional query parameters for story customization + var genre = "fantasy"; + var setting = "a mysterious ancient library"; + var theme = "the power of curiosity"; + + if (request.QueryStringParameters is not null) + { + if (request.QueryStringParameters.TryGetValue("genre", out var g) && !string.IsNullOrWhiteSpace(g)) + genre = g; + if (request.QueryStringParameters.TryGetValue("setting", out var s) && !string.IsNullOrWhiteSpace(s)) + setting = s; + if (request.QueryStringParameters.TryGetValue("theme", out var t) && !string.IsNullOrWhiteSpace(t)) + theme = t; + } + + // Build the prompt + var prompt = $""" + Write a short story (about 500-800 words) in the {genre} genre. + Setting: {setting} + Theme: {theme} + + Write in a vivid, engaging style. Start the story immediately without any preamble. + """; + + // Set up the HTTP response stream + var prelude = new HttpResponseStreamPrelude + { + StatusCode = HttpStatusCode.OK, + Headers = + { + { "Content-Type", "text/plain; charset=utf-8" }, + { "Cache-Control", "no-cache" }, + { "X-Content-Type-Options", "nosniff" } + } + }; + + using var responseStream = LambdaResponseStreamFactory.CreateHttpStream(prelude); + using var writer = new StreamWriter(responseStream) { AutoFlush = false }; + + // Call Bedrock ConverseStream with Claude Sonnet + var converseRequest = new ConverseStreamRequest + { + ModelId = "us.anthropic.claude-sonnet-5", + Messages = + [ + new Message + { + Role = ConversationRole.User, + Content = [new ContentBlock { Text = prompt }] + } + ], + InferenceConfig = new InferenceConfiguration + { + MaxTokens = 2048 + } + }; + + try + { + var response = await bedrockClient.ConverseStreamAsync(converseRequest); + + // Stream each token as it arrives from Bedrock to the client + foreach (var item in response.Stream.AsEnumerable()) + { + if (item is ContentBlockDeltaEvent deltaEvent && deltaEvent.Delta?.Text is not null) + { + await writer.WriteAsync(deltaEvent.Delta.Text); + await writer.FlushAsync(); + } + } + + // End with a newline + await writer.WriteLineAsync(); + await writer.FlushAsync(); + } + catch (Exception ex) + { + context.Logger.LogError($"Error invoking Bedrock: {ex.Message}"); + await writer.WriteLineAsync($"\n\n[Error generating story: {ex.Message}]"); + await writer.FlushAsync(); + } +}; + +// Build and run the Lambda runtime +await LambdaBootstrapBuilder.Create(handler, new DefaultLambdaJsonSerializer()) + .Build() + .RunAsync(); diff --git a/sam-dotnet-responsestreaming-bedrock/src/StoryStreaming.csproj b/sam-dotnet-responsestreaming-bedrock/src/StoryStreaming.csproj new file mode 100644 index 000000000..7fe7f95ff --- /dev/null +++ b/sam-dotnet-responsestreaming-bedrock/src/StoryStreaming.csproj @@ -0,0 +1,19 @@ + + + net10.0 + enable + enable + Exe + StoryStreaming + Lambda + true + true + + + + + + + + + diff --git a/sam-dotnet-responsestreaming-bedrock/template.yaml b/sam-dotnet-responsestreaming-bedrock/template.yaml new file mode 100644 index 000000000..7d8ff791b --- /dev/null +++ b/sam-dotnet-responsestreaming-bedrock/template.yaml @@ -0,0 +1,90 @@ +AWSTemplateFormatVersion: '2010-09-09' +Transform: AWS::Serverless-2016-10-31 +Description: > + Story generator using Amazon Bedrock (Claude Sonnet) with Lambda Response Streaming + through API Gateway. Streams a generated story back to the client token-by-token. + +Globals: + Function: + Timeout: 120 + MemorySize: 512 + Runtime: dotnet10 + Architectures: + - x86_64 + +Resources: + # ───────────────────────────────────────────────────────────────────── + # REST API with response streaming + # ───────────────────────────────────────────────────────────────────── + StoryApi: + Type: AWS::ApiGateway::RestApi + Properties: + Name: story-streaming-api + Description: REST API with response streaming for Bedrock story generation + + StoryResource: + Type: AWS::ApiGateway::Resource + Properties: + RestApiId: !Ref StoryApi + ParentId: !GetAtt StoryApi.RootResourceId + PathPart: story + + StoryMethod: + Type: AWS::ApiGateway::Method + Properties: + RestApiId: !Ref StoryApi + ResourceId: !Ref StoryResource + HttpMethod: GET + AuthorizationType: NONE + Integration: + Type: AWS_PROXY + IntegrationHttpMethod: POST + ResponseTransferMode: STREAM + TimeoutInMillis: 120000 + Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2021-11-15/functions/${StoryFunction.Arn}/response-streaming-invocations + + ApiDeployment: + Type: AWS::ApiGateway::Deployment + DependsOn: StoryMethod + Properties: + RestApiId: !Ref StoryApi + + ApiStage: + Type: AWS::ApiGateway::Stage + Properties: + RestApiId: !Ref StoryApi + DeploymentId: !Ref ApiDeployment + StageName: prod + + # ───────────────────────────────────────────────────────────────────── + # Lambda Function — story generator with Bedrock streaming + # ───────────────────────────────────────────────────────────────────── + StoryFunction: + Type: AWS::Serverless::Function + Properties: + Handler: StoryStreaming + CodeUri: ./src/ + Description: Streams a Bedrock-generated story via API Gateway response streaming + Policies: + - Statement: + - Effect: Allow + Action: + - bedrock:InvokeModelWithResponseStream + Resource: "*" + + # Permission for API Gateway to invoke the function via response streaming + StoryFunctionPermission: + Type: AWS::Lambda::Permission + Properties: + FunctionName: !GetAtt StoryFunction.Arn + Action: lambda:InvokeFunction + Principal: apigateway.amazonaws.com + SourceArn: !Sub arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${StoryApi}/* + +Outputs: + ApiEndpoint: + Description: API Gateway endpoint URL for the story streaming endpoint + Value: !Sub "https://${StoryApi}.execute-api.${AWS::Region}.amazonaws.com/prod/story" + StoryFunctionArn: + Description: Story streaming Lambda function ARN + Value: !GetAtt StoryFunction.Arn