Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions sam-dotnet-responsestreaming-bedrock/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
## .NET
bin/
obj/
*.user
*.suo
*.userosscache
*.sln.docstates

## SAM
.aws-sam/
samconfig.toml

## IDE
.idea/
.vs/
*.swp
183 changes: 183 additions & 0 deletions sam-dotnet-responsestreaming-bedrock/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
# AWS Lambda Response Streaming with Amazon Bedrock — Story Generator (.NET)

This sample demonstrates **Lambda Response Streaming through Amazon API Gateway** combined with **Amazon Bedrock** (Claude Sonnet) for .NET. A Lambda function invokes Claude Sonnet via the Bedrock `ConverseStream` API and streams the generated story token-by-token back to the client through API Gateway response streaming.

## Architecture

```
┌──────────────┐ ┌──────────────────┐ ┌────────────────────┐ ┌─────────────┐
│ Client │ │ API Gateway │ │ Lambda Function │ │ Amazon │
│ (curl) │──GET─▶│ REST API │──────▶│ (Response │──────▶│ Bedrock │
│ │◀─stream│ ResponseTransfer │◀stream│ Streaming) │◀stream│ (Claude) │
│ │ │ Mode: STREAM │ │ │ │ │
└──────────────┘ └──────────────────┘ └────────────────────┘ └─────────────┘
```

## What It Demonstrates

- **End-to-end streaming** — Bedrock streams tokens → Lambda pipes them through → API Gateway streams to client. The user sees the story appear word-by-word.
- **API Gateway response streaming** — Uses `ResponseTransferMode: STREAM` with the `/response-streaming-invocations` Lambda endpoint.
- **Bedrock ConverseStream API** — Uses the AWS SDK's `ConverseStreamAsync` to get streaming responses from Claude Sonnet.
- **Customizable story generation** — Users can specify genre, setting, and theme via query parameters.
- **Minimal latency** — First tokens appear within seconds, no waiting for the full generation to complete.

## Project Structure

```
├── template.yaml # SAM template (API Gateway REST API + Lambda + Bedrock IAM)
├── events/
│ └── generate-story.json # Sample API Gateway proxy event
└── src/
├── Program.cs # Streaming handler (Bedrock → Lambda → API Gateway → Client)
└── StoryStreaming.csproj
```

---

## Prerequisites

- [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10.0)
- [AWS SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html)
- AWS account with credentials configured (`aws configure`)
- **Amazon Bedrock model access** — Enable access to Claude Sonnet in the [Bedrock console](https://console.aws.amazon.com/bedrock/home#/modelaccess)

---

## Build & Deploy

### Build

```bash
sam build
```

### Deploy

```bash
sam deploy --guided
```

Follow the prompts to configure stack name, region, and confirm IAM role creation. On subsequent deploys:

```bash
sam deploy
```

---

## Testing

### 1. Get the API endpoint

```bash
API_URL=$(aws cloudformation describe-stacks \
--stack-name sam-dotnet-responsestreaming-bedrock \
--query "Stacks[0].Outputs[?OutputKey=='ApiEndpoint'].OutputValue" \
--output text)
```

### 2. Generate a story with defaults

```bash
curl --no-buffer "$API_URL"
```

You'll see the story appear word-by-word in real-time as Claude generates it.

### 3. Customize the story

```bash
# Science fiction story
curl --no-buffer "$API_URL?genre=science+fiction&setting=a+space+station+orbiting+Jupiter&theme=finding+home"

# Mystery story
curl --no-buffer "$API_URL?genre=mystery&setting=a+foggy+Victorian+London+street&theme=trust+and+betrayal"

# Horror story
curl --no-buffer "$API_URL?genre=horror&setting=an+abandoned+hospital&theme=facing+your+fears"
```

### Parameters

| Parameter | Default | Description |
|-----------|---------|-------------|
| `genre` | fantasy | The genre of the story (e.g., science fiction, mystery, horror, romance) |
| `setting` | a mysterious ancient library | Where the story takes place |
| `theme` | the power of curiosity | The central theme or message |

---

## How It Works

### API Gateway → Lambda (Response Streaming)

The API Gateway integration uses `/response-streaming-invocations` to invoke Lambda with streaming:

```yaml
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
ResponseTransferMode: STREAM
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2021-11-15/functions/${StoryFunction.Arn}/response-streaming-invocations
```

### Lambda → Bedrock (ConverseStream)

The Lambda function calls Bedrock's streaming API and pipes each token to the response stream:

```csharp
// Set up Lambda response streaming
using var responseStream = LambdaResponseStreamFactory.CreateHttpStream(prelude);
using var writer = new StreamWriter(responseStream);

// Call Bedrock with streaming
var response = await bedrockClient.ConverseStreamAsync(converseRequest);

// Pipe each token from Bedrock → API Gateway → Client
foreach (var item in response.Stream.AsEnumerable())
{
if (item is ContentBlockDeltaEvent deltaEvent && deltaEvent.Delta?.Text is not null)
{
await writer.WriteAsync(deltaEvent.Delta.Text);
await writer.FlushAsync(); // Sends immediately to the client
}
}
```

Key points:
- `ConverseStreamAsync` returns an event stream that yields `ContentBlockDeltaEvent` items as Claude generates tokens.
- Each `FlushAsync()` pushes the token(s) through API Gateway to the client immediately.
- The model used is `us.anthropic.claude-sonnet-5` (cross-region inference profile for higher availability).
- `Temperature: 0.8` and `TopP: 0.9` produce creative, varied stories.

---

## Cleanup

```bash
sam delete
```

---

## Useful Commands

| Command | Description |
|---------|-------------|
| `sam build` | Build the Lambda function |
| `sam deploy --guided` | Deploy with interactive prompts |
| `sam deploy` | Deploy with saved config |
| `sam logs --tail` | Tail CloudWatch logs |
| `sam delete` | Tear down the stack |
| `dotnet build src/` | Build locally without SAM |

---

## References

- [Building responsive APIs with API Gateway response streaming](https://aws.amazon.com/blogs/compute/building-responsive-apis-with-amazon-api-gateway-response-streaming/)
- [Serverless strategies for streaming LLM responses](https://aws.amazon.com/blogs/compute/serverless-strategies-for-streaming-llm-responses/)
- [Lambda Response Streaming (.NET SDK PR)](https://github.com/aws/aws-lambda-dotnet/pull/2288)
- [Amazon Bedrock ConverseStream API](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html)
- [Claude Sonnet on Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-anthropic-claude-sonnet-5.html)
- [AWS SAM Developer Guide](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/)
31 changes: 31 additions & 0 deletions sam-dotnet-responsestreaming-bedrock/events/generate-story.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
{
"resource": "/story",
"path": "/story",
"httpMethod": "GET",
"headers": {
"Accept": "*/*",
"Host": "abc123.execute-api.us-east-1.amazonaws.com"
},
"queryStringParameters": {
"genre": "science fiction",
"setting": "a space station orbiting Jupiter",
"theme": "finding home in unexpected places"
},
"pathParameters": null,
"stageVariables": null,
"requestContext": {
"resourceId": "abc123",
"resourcePath": "/story",
"httpMethod": "GET",
"requestId": "test-request-id",
"accountId": "123456789012",
"stage": "prod",
"identity": {
"sourceIp": "127.0.0.1",
"userAgent": "curl/8.0"
},
"apiId": "abc123"
},
"body": null,
"isBase64Encoded": false
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
{
"title": "Lambda Response Streaming with Amazon Bedrock — Story Generator (.NET)",
"description": "Stream LLM-generated story text token-by-token from Amazon Bedrock through Lambda response streaming and API Gateway to the client.",
"language": ".NET",
"level": "200",
"framework": "SAM",
"introBox": {
"headline": "How it works",
"text": [
"This pattern deploys a Lambda function that invokes Amazon Bedrock (Claude Sonnet) via the ConverseStream API and pipes generated tokens directly through API Gateway response streaming to the client.",
"The user specifies a genre, setting, and theme via query parameters, and the story appears word-by-word in the terminal. First tokens arrive within seconds — no waiting for the full generation to complete.",
"This demonstrates end-to-end streaming from Bedrock through Lambda to the client using the .NET response streaming SDK (LambdaResponseStreamFactory.CreateHttpStream)."
]
},
"gitHub": {
"template": {
"repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/sam-dotnet-responsestreaming-bedrock",
"templateURL": "serverless-patterns/sam-dotnet-responsestreaming-bedrock",
"projectFolder": "sam-dotnet-responsestreaming-bedrock",
"templateFile": "template.yaml"
}
},
"resources": {
"bullets": [
{
"text": "Lambda Response Streaming Documentation",
"link": "https://docs.aws.amazon.com/lambda/latest/dg/response-streaming.html"
},
{
"text": "Amazon Bedrock ConverseStream API",
"link": "https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-call.html"
},
{
"text": "Amazon Bedrock",
"link": "https://aws.amazon.com/bedrock/"
}
]
},
"deploy": {
"text": [
"sam build",
"sam deploy --guided"
]
},
"testing": {
"text": [
"See the GitHub repo for detailed testing instructions."
]
},
"cleanup": {
"text": [
"Delete the stack: <code>sam delete</code>."
]
},
"authors": [
{
"name": "Doug Perkes",
"image": "",
"bio": "Senior Solutions Architect at AWS, focused on .NET and serverless.",
"linkedin": "dougperkes",
"twitter": ""
}
]
}
Loading