Skip to content

feat(eval): ondemand simulate — replay a dataset, evaluate synchronously #2071

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jariy17 wants to merge 6 commits into from
Open

feat(eval): ondemand simulate — replay a dataset, evaluate synchronously #2071

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
b511a1f
feat(eval): ondemand simulate handler (batch pattern: ingestion-wait-…
Aug 27, 2026
592e671
test(eval): ondemand simulate edge tests + register under ondemand
Aug 27, 2026
225898d
test(eval): ondemand simulate fixture golden + clock seam; strip hand…
Aug 27, 2026
7f61f1f
fix(eval): ondemand simulate — use withUserCancellation; forward turn…
Aug 28, 2026
34dbc77
fix(eval): ondemand simulate — expectedResponse is trace-level (corre…
Aug 28, 2026
2c20882
test(eval): dedupe golden session ids via uniquePerRecording helper (…
Aug 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion src/core/eval.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,7 @@ export class EvalClient implements CoreEvalClient {
// logger for batch-evaluation result-log diagnostics
private readonly logger: Logger = noopLogger,
private readonly newSessionId: () => string = randomUUID,
private readonly now: () => number = () => Date.now(),
) {}

async createEvaluator(
Expand Down Expand Up @@ -582,7 +583,7 @@ export class EvalClient implements CoreEvalClient {
const logGroupName = runtimeLogGroup(runtimeId, qualifier);
const serviceName = runtimeServiceName(runtimeName, qualifier);

const endMs = input.window ? +input.window.endTime : Date.now();
const endMs = input.window ? +input.window.endTime : this.now();

@notgitika notgitika Aug 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why did we make this change just curious

@jariy17 jariy17 Aug 28, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the Golden tests, it requires the inputs to be deterministic. However, Date.now() is not deterministic. Therefore, I needed to inject a now function that could mocked by our unit tests. Look at our unit tests for this.

const startMs = input.window ? +input.window.startTime : endMs - SEVEN_DAYS_MS;
const startSec = Math.floor(startMs / 1000);
const endSec = Math.floor(endMs / 1000);
Expand Down
2 changes: 2 additions & 0 deletions src/core/index.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ type CoreClientConfig = {
logger: Logger;
fetch?: CoreFetch;
newSessionId?: () => string;
now?: () => number;
};

// CoreClient is the single entry point to the Bedrock AgentCore APIs. It owns the
Expand Down Expand Up @@ -87,6 +88,7 @@ export class CoreClient implements AwsClients {
fetch,
this.logger.child({ module: "eval" }),
config.newSessionId,
config.now,
);

this.projectManager = new FsProjectManager({
Expand Down
26 changes: 26 additions & 0 deletions src/handlers/eval/ondemand/__fixtures__/EvaluateCommand.1aea66f982338a76.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
{
"evaluationResults": [
{
"evaluatorArn": "arn:aws:bedrock-agentcore:::evaluator/Builtin.Correctness",
"evaluatorId": "Builtin.Correctness",
"evaluatorName": "Builtin.Correctness",
"context": {
"spanContext": {
"sessionId": "00000000-0000-4000-8000-4a4a72800001",
"traceId": "6a91fe944cf38d6b2d8c3c6a66e63ad4"
}
},
"explanation": "The agent responded with '4' which matches the expected response of '4'. The trailing newline is irrelevant to the correctness of the answer.",
"value": 1,
"label": "Correct",
"tokenUsage": {
"inputTokens": 484,
"outputTokens": 53,
"totalTokens": 537
},
"ignoredReferenceInputFields": [
"assertions"
]
}
]
}
41 changes: 41 additions & 0 deletions src/handlers/eval/ondemand/__fixtures__/GetEvaluatorCommand.5ee939ef1fda1ca7.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
{
"evaluatorArn": "arn:aws:bedrock-agentcore:::evaluator/Builtin.Correctness",
"evaluatorId": "Builtin.Correctness",
"evaluatorName": "Builtin.Correctness",
"evaluatorConfig": {
"llmAsAJudge": {
"ratingScale": {
"numerical": [
{
"value": {
"string": "0.0",
"type": "bigDecimal"
},
"label": "Incorrect"
},
{
"value": 0.5,
"label": "Partially correct"
},
{
"value": {
"string": "1.0",
"type": "bigDecimal"
},
"label": "Correct"
}
]
}
}
},
"level": "TRACE",
"status": "ACTIVE",
"createdAt": {
"$date": "2024-10-22T00:00:00.000Z"
},
"updatedAt": {
"$date": "2024-10-22T00:00:00.000Z"
},
"description": "Response Quality Metric. Evaluates whether the information in the agent's response is factually accurate",
"lockedForModification": true
}
146 changes: 146 additions & 0 deletions src/handlers/eval/ondemand/__fixtures__/GetQueryResultsCommand.548f1c221addde3b.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
{
"queryLanguage": "CWLI",
"results": [
[
{
"field": "@message",
"value": "{\"resource\":{\"attributes\":{\"telemetry.sdk.language\":\"python\",\"telemetry.sdk.name\":\"opentelemetry\",\"telemetry.sdk.version\":\"1.40.0\",\"service.name\":\"asdf_MyAgent.DEFAULT\",\"aws.log.group.names\":\"/aws/bedrock-agentcore/runtimes/asdf_MyAgent-3s5axvBC6Q-DEFAULT\",\"aws.log.stream.names\":\"otel-rt-logs\",\"deployment.environment.name\":\"bedrock-agentcore:default\",\"cloud.resource_id\":\"arn:aws:bedrock-agentcore:us-west-2:685197708687:runtime/asdf_MyAgent-3s5axvBC6Q/runtime-endpoint/DEFAULT:DEFAULT\",\"cloud.platform\":\"aws_bedrock_agentcore\",\"cloud.provider\":\"aws\",\"cloud.region\":\"us-west-2\",\"telemetry.auto.version\":\"0.17.0-aws\",\"aws.local.service\":\"asdf_MyAgent.DEFAULT\",\"aws.service.type\":\"gen_ai_agent\"}},\"scope\":{\"name\":\"strands.telemetry.tracer\"},\"timeUnixNano\":1787952795780790373,\"observedTimeUnixNano\":1787952796735286012,\"severityNumber\":9,\"severityText\":\"\",\"body\":{\"input\":{\"messages\":[{\"role\":\"system\",\"content\":{\"content\":\"[{\\\"text\\\": \\\"\\\\n You are a helpful assistant. Use tools when appropriate.\\\\n \\\"}]\"}},{\"role\":\"user\",\"content\":{\"content\":\"[{\\\"text\\\": \\\"What is 2+2? Reply with just the number.\\\"}]\"}}]},\"output\":{\"messages\":[{\"role\":\"assistant\",\"content\":{\"finish_reason\":\"tool_use\",\"message\":\"[{\\\"toolUse\\\": {\\\"toolUseId\\\": \\\"tooluse_3jtUttUiLMVTz1Qqd4hYGz\\\", \\\"name\\\": \\\"add_numbers\\\", \\\"input\\\": {\\\"a\\\": 2, \\\"b\\\": 2}}}]\"}}]}},\"attributes\":{\"session.id\":\"00000000-0000-4000-8000-4a4a72800001\",\"event.name\":\"strands.telemetry.tracer\"},\"flags\":1,\"traceId\":\"6a91fe944cf38d6b2d8c3c6a66e63ad4\",\"spanId\":\"7b1eeecf45c6e146\"}"
},
{
"field": "sessionId",
"value": "00000000-0000-4000-8000-4a4a72800001"
},
{
"field": "traceId",
"value": "6a91fe944cf38d6b2d8c3c6a66e63ad4"
},
{
"field": "spanId",
"value": "7b1eeecf45c6e146"
},
{
"field": "@ptr",
"value": "Cs0BCo8BCkw2ODUxOTc3MDg2ODc6L2F3cy9iZWRyb2NrLWFnZW50Y29yZS9ydW50aW1lcy9hc2RmX015QWdlbnQtM3M1YXh2QkM2US1ERUZBVUxUEAAaJGY2ZGY3NDIyLWVlZTQtNDhkNS05NDg3LTE0NTZkYjdkZGE1YiIOCKCUkduCNBCHpMP7hDRA0cHR4dszSAASNxoYAgajvKv7AAAAABTZwGgABqkf6XAAAAKyIAEolYKq0oQ0MJyuqtKENDgqQJe/AkiwkQFQgGYgAhAcGAE="
}
],
[
{
"field": "@message",
"value": "{\"resource\":{\"attributes\":{\"telemetry.sdk.language\":\"python\",\"telemetry.sdk.name\":\"opentelemetry\",\"telemetry.sdk.version\":\"1.40.0\",\"service.name\":\"asdf_MyAgent.DEFAULT\",\"aws.log.group.names\":\"/aws/bedrock-agentcore/runtimes/asdf_MyAgent-3s5axvBC6Q-DEFAULT\",\"aws.log.stream.names\":\"otel-rt-logs\",\"deployment.environment.name\":\"bedrock-agentcore:default\",\"cloud.resource_id\":\"arn:aws:bedrock-agentcore:us-west-2:685197708687:runtime/asdf_MyAgent-3s5axvBC6Q/runtime-endpoint/DEFAULT:DEFAULT\",\"cloud.platform\":\"aws_bedrock_agentcore\",\"cloud.provider\":\"aws\",\"cloud.region\":\"us-west-2\",\"telemetry.auto.version\":\"0.17.0-aws\",\"aws.local.service\":\"asdf_MyAgent.DEFAULT\",\"aws.service.type\":\"gen_ai_agent\"}},\"scope\":{\"name\":\"strands.telemetry.tracer\"},\"timeUnixNano\":1787952795781997134,\"observedTimeUnixNano\":1787952796735435934,\"severityNumber\":9,\"severityText\":\"\",\"body\":{\"input\":{\"messages\":[{\"role\":\"tool\",\"content\":{\"role\":\"tool\",\"content\":\"{\\\"a\\\": 2, \\\"b\\\": 2}\",\"id\":\"tooluse_3jtUttUiLMVTz1Qqd4hYGz\"}}]},\"output\":{\"messages\":[{\"role\":\"assistant\",\"content\":{\"message\":\"[{\\\"text\\\": \\\"4\\\"}]\",\"id\":\"tooluse_3jtUttUiLMVTz1Qqd4hYGz\"}}]}},\"attributes\":{\"session.id\":\"00000000-0000-4000-8000-4a4a72800001\",\"event.name\":\"strands.telemetry.tracer\"},\"flags\":1,\"traceId\":\"6a91fe944cf38d6b2d8c3c6a66e63ad4\",\"spanId\":\"4ad640cbd3587ff3\"}"
},
{
"field": "sessionId",
"value": "00000000-0000-4000-8000-4a4a72800001"
},
{
"field": "traceId",
"value": "6a91fe944cf38d6b2d8c3c6a66e63ad4"
},
{
"field": "spanId",
"value": "4ad640cbd3587ff3"
},
{
"field": "@ptr",
"value": "Cs0BCo8BCkw2ODUxOTc3MDg2ODc6L2F3cy9iZWRyb2NrLWFnZW50Y29yZS9ydW50aW1lcy9hc2RmX015QWdlbnQtM3M1YXh2QkM2US1ERUZBVUxUEAAaJGY2ZGY3NDIyLWVlZTQtNDhkNS05NDg3LTE0NTZkYjdkZGE1YiIOCKCUkduCNBCHpMP7hDRA0cHR4dszSAASNxoYAgajvKv7AAAAABTZwGgABqkf6XAAAAKyIAEolYKq0oQ0MJyuqtKENDgqQJe/AkiwkQFQgGYgAhAdGAE="
}
],
[
{
"field": "@message",
"value": "{\"resource\":{\"attributes\":{\"telemetry.sdk.language\":\"python\",\"telemetry.sdk.name\":\"opentelemetry\",\"telemetry.sdk.version\":\"1.40.0\",\"service.name\":\"asdf_MyAgent.DEFAULT\",\"aws.log.group.names\":\"/aws/bedrock-agentcore/runtimes/asdf_MyAgent-3s5axvBC6Q-DEFAULT\",\"aws.log.stream.names\":\"otel-rt-logs\",\"deployment.environment.name\":\"bedrock-agentcore:default\",\"cloud.resource_id\":\"arn:aws:bedrock-agentcore:us-west-2:685197708687:runtime/asdf_MyAgent-3s5axvBC6Q/runtime-endpoint/DEFAULT:DEFAULT\",\"cloud.platform\":\"aws_bedrock_agentcore\",\"cloud.provider\":\"aws\",\"cloud.region\":\"us-west-2\",\"telemetry.auto.version\":\"0.17.0-aws\",\"aws.local.service\":\"asdf_MyAgent.DEFAULT\",\"aws.service.type\":\"gen_ai_agent\"}},\"scope\":{\"name\":\"strands.telemetry.tracer\"},\"timeUnixNano\":1787952795782144605,\"observedTimeUnixNano\":1787952796735532818,\"severityNumber\":9,\"severityText\":\"\",\"body\":{\"input\":{\"messages\":[{\"role\":\"user\",\"content\":{\"content\":\"[{\\\"text\\\": \\\"What is 2+2? Reply with just the number.\\\"}]\"}}]},\"output\":{\"messages\":[{\"role\":\"assistant\",\"content\":{\"message\":\"[{\\\"toolUse\\\": {\\\"toolUseId\\\": \\\"tooluse_3jtUttUiLMVTz1Qqd4hYGz\\\", \\\"name\\\": \\\"add_numbers\\\", \\\"input\\\": {\\\"a\\\": 2, \\\"b\\\": 2}}}]\",\"tool.result\":\"[{\\\"toolResult\\\": {\\\"toolUseId\\\": \\\"tooluse_3jtUttUiLMVTz1Qqd4hYGz\\\", \\\"status\\\": \\\"success\\\", \\\"content\\\": [{\\\"text\\\": \\\"4\\\"}]}}]\"}},{\"role\":\"assistant\",\"content\":\"[{\\\"toolResult\\\": {\\\"toolUseId\\\": \\\"tooluse_3jtUttUiLMVTz1Qqd4hYGz\\\", \\\"status\\\": \\\"success\\\", \\\"content\\\": [{\\\"text\\\": \\\"4\\\"}]}}]\"}]}},\"attributes\":{\"session.id\":\"00000000-0000-4000-8000-4a4a72800001\",\"event.name\":\"strands.telemetry.tracer\"},\"flags\":1,\"traceId\":\"6a91fe944cf38d6b2d8c3c6a66e63ad4\",\"spanId\":\"a37ebb08225f5f32\"}"
},
{
"field": "sessionId",
"value": "00000000-0000-4000-8000-4a4a72800001"
},
{
"field": "traceId",
"value": "6a91fe944cf38d6b2d8c3c6a66e63ad4"
},
{
"field": "spanId",
"value": "a37ebb08225f5f32"
},
{
"field": "@ptr",
"value": "Cs0BCo8BCkw2ODUxOTc3MDg2ODc6L2F3cy9iZWRyb2NrLWFnZW50Y29yZS9ydW50aW1lcy9hc2RmX015QWdlbnQtM3M1YXh2QkM2US1ERUZBVUxUEAAaJGY2ZGY3NDIyLWVlZTQtNDhkNS05NDg3LTE0NTZkYjdkZGE1YiIOCKCUkduCNBCHpMP7hDRA0cHR4dszSAASNxoYAgajvKv7AAAAABTZwGgABqkf6XAAAAKyIAEolYKq0oQ0MJyuqtKENDgqQJe/AkiwkQFQgGYgAhAeGAE="
}
],
[
{
"field": "@message",
"value": "{\"resource\":{\"attributes\":{\"telemetry.sdk.language\":\"python\",\"telemetry.sdk.name\":\"opentelemetry\",\"telemetry.sdk.version\":\"1.40.0\",\"service.name\":\"asdf_MyAgent.DEFAULT\",\"aws.log.group.names\":\"/aws/bedrock-agentcore/runtimes/asdf_MyAgent-3s5axvBC6Q-DEFAULT\",\"aws.log.stream.names\":\"otel-rt-logs\",\"deployment.environment.name\":\"bedrock-agentcore:default\",\"cloud.resource_id\":\"arn:aws:bedrock-agentcore:us-west-2:685197708687:runtime/asdf_MyAgent-3s5axvBC6Q/runtime-endpoint/DEFAULT:DEFAULT\",\"cloud.platform\":\"aws_bedrock_agentcore\",\"cloud.provider\":\"aws\",\"cloud.region\":\"us-west-2\",\"telemetry.auto.version\":\"0.17.0-aws\",\"aws.local.service\":\"asdf_MyAgent.DEFAULT\",\"aws.service.type\":\"gen_ai_agent\"}},\"scope\":{\"name\":\"strands.telemetry.tracer\"},\"timeUnixNano\":1787952797468097953,\"observedTimeUnixNano\":1787952801835110321,\"severityNumber\":9,\"severityText\":\"\",\"body\":{\"input\":{\"messages\":[{\"role\":\"system\",\"content\":{\"content\":\"[{\\\"text\\\": \\\"\\\\n You are a helpful assistant. Use tools when appropriate.\\\\n \\\"}]\"}},{\"role\":\"user\",\"content\":{\"content\":\"[{\\\"text\\\": \\\"What is 2+2? Reply with just the number.\\\"}]\"}},{\"role\":\"tool\",\"content\":{\"content\":\"[{\\\"toolResult\\\": {\\\"toolUseId\\\": \\\"tooluse_3jtUttUiLMVTz1Qqd4hYGz\\\", \\\"status\\\": \\\"success\\\", \\\"content\\\": [{\\\"text\\\": \\\"4\\\"}]}}]\"}}]},\"output\":{\"messages\":[{\"role\":\"assistant\",\"content\":{\"content\":\"[{\\\"toolUse\\\": {\\\"toolUseId\\\": \\\"tooluse_3jtUttUiLMVTz1Qqd4hYGz\\\", \\\"name\\\": \\\"add_numbers\\\", \\\"input\\\": {\\\"a\\\": 2, \\\"b\\\": 2}}}]\"}},{\"role\":\"assistant\",\"content\":{\"finish_reason\":\"end_turn\",\"message\":\"[{\\\"text\\\": \\\"4\\\"}]\"}}]}},\"attributes\":{\"session.id\":\"00000000-0000-4000-8000-4a4a72800001\",\"event.name\":\"strands.telemetry.tracer\"},\"flags\":1,\"traceId\":\"6a91fe944cf38d6b2d8c3c6a66e63ad4\",\"spanId\":\"ad52d36b18057ff1\"}"
},
{
"field": "sessionId",
"value": "00000000-0000-4000-8000-4a4a72800001"
},
{
"field": "traceId",
"value": "6a91fe944cf38d6b2d8c3c6a66e63ad4"
},
{
"field": "spanId",
"value": "ad52d36b18057ff1"
},
{
"field": "@ptr",
"value": "Cs0BCo8BCkw2ODUxOTc3MDg2ODc6L2F3cy9iZWRyb2NrLWFnZW50Y29yZS9ydW50aW1lcy9hc2RmX015QWdlbnQtM3M1YXh2QkM2US1ERUZBVUxUEAAaJGY2ZGY3NDIyLWVlZTQtNDhkNS05NDg3LTE0NTZkYjdkZGE1YiIOCKCUkduCNBCHpMP7hDRA0cHR4dszSAASNxoYAgajvKv7AAAAABTZwGgABqkf6XAAAAKyIAEolYKq0oQ0MJyuqtKENDgqQJe/AkiwkQFQgGYgAhAnGAE="
}
],
[
{
"field": "@message",
"value": "{\"resource\":{\"attributes\":{\"telemetry.sdk.language\":\"python\",\"telemetry.sdk.name\":\"opentelemetry\",\"telemetry.sdk.version\":\"1.40.0\",\"service.name\":\"asdf_MyAgent.DEFAULT\",\"aws.log.group.names\":\"/aws/bedrock-agentcore/runtimes/asdf_MyAgent-3s5axvBC6Q-DEFAULT\",\"aws.log.stream.names\":\"otel-rt-logs\",\"deployment.environment.name\":\"bedrock-agentcore:default\",\"cloud.resource_id\":\"arn:aws:bedrock-agentcore:us-west-2:685197708687:runtime/asdf_MyAgent-3s5axvBC6Q/runtime-endpoint/DEFAULT:DEFAULT\",\"cloud.platform\":\"aws_bedrock_agentcore\",\"cloud.provider\":\"aws\",\"cloud.region\":\"us-west-2\",\"telemetry.auto.version\":\"0.17.0-aws\",\"aws.local.service\":\"asdf_MyAgent.DEFAULT\",\"aws.service.type\":\"gen_ai_agent\"}},\"scope\":{\"name\":\"strands.telemetry.tracer\"},\"timeUnixNano\":1787952797468400669,\"observedTimeUnixNano\":1787952801835250906,\"severityNumber\":9,\"severityText\":\"\",\"body\":{\"input\":{\"messages\":[{\"role\":\"user\",\"content\":{\"content\":\"[{\\\"text\\\": \\\"What is 2+2? Reply with just the number.\\\"}]\"}},{\"role\":\"tool\",\"content\":{\"content\":\"[{\\\"toolResult\\\": {\\\"toolUseId\\\": \\\"tooluse_3jtUttUiLMVTz1Qqd4hYGz\\\", \\\"status\\\": \\\"success\\\", \\\"content\\\": [{\\\"text\\\": \\\"4\\\"}]}}]\"}}]},\"output\":{\"messages\":[{\"role\":\"assistant\",\"content\":{\"content\":\"[{\\\"toolUse\\\": {\\\"toolUseId\\\": \\\"tooluse_3jtUttUiLMVTz1Qqd4hYGz\\\", \\\"name\\\": \\\"add_numbers\\\", \\\"input\\\": {\\\"a\\\": 2, \\\"b\\\": 2}}}]\"}}]}},\"attributes\":{\"session.id\":\"00000000-0000-4000-8000-4a4a72800001\",\"event.name\":\"strands.telemetry.tracer\"},\"flags\":1,\"traceId\":\"6a91fe944cf38d6b2d8c3c6a66e63ad4\",\"spanId\":\"d7b0a909896809ba\"}"
},
{
"field": "sessionId",
"value": "00000000-0000-4000-8000-4a4a72800001"
},
{
"field": "traceId",
"value": "6a91fe944cf38d6b2d8c3c6a66e63ad4"
},
{
"field": "spanId",
"value": "d7b0a909896809ba"
},
{
"field": "@ptr",
"value": "Cs0BCo8BCkw2ODUxOTc3MDg2ODc6L2F3cy9iZWRyb2NrLWFnZW50Y29yZS9ydW50aW1lcy9hc2RmX015QWdlbnQtM3M1YXh2QkM2US1ERUZBVUxUEAAaJGY2ZGY3NDIyLWVlZTQtNDhkNS05NDg3LTE0NTZkYjdkZGE1YiIOCKCUkduCNBCHpMP7hDRA0cHR4dszSAASNxoYAgajvKv7AAAAABTZwGgABqkf6XAAAAKyIAEolYKq0oQ0MJyuqtKENDgqQJe/AkiwkQFQgGYgAhAoGAE="
}
],
[
{
"field": "@message",
"value": "{\"resource\":{\"attributes\":{\"telemetry.sdk.language\":\"python\",\"telemetry.sdk.name\":\"opentelemetry\",\"telemetry.sdk.version\":\"1.40.0\",\"service.name\":\"asdf_MyAgent.DEFAULT\",\"aws.log.group.names\":\"/aws/bedrock-agentcore/runtimes/asdf_MyAgent-3s5axvBC6Q-DEFAULT\",\"aws.log.stream.names\":\"otel-rt-logs\",\"deployment.environment.name\":\"bedrock-agentcore:default\",\"cloud.resource_id\":\"arn:aws:bedrock-agentcore:us-west-2:685197708687:runtime/asdf_MyAgent-3s5axvBC6Q/runtime-endpoint/DEFAULT:DEFAULT\",\"cloud.platform\":\"aws_bedrock_agentcore\",\"cloud.provider\":\"aws\",\"cloud.region\":\"us-west-2\",\"telemetry.auto.version\":\"0.17.0-aws\",\"aws.local.service\":\"asdf_MyAgent.DEFAULT\",\"aws.service.type\":\"gen_ai_agent\"}},\"scope\":{\"name\":\"strands.telemetry.tracer\"},\"timeUnixNano\":1787952797468576817,\"observedTimeUnixNano\":1787952801835352043,\"severityNumber\":9,\"severityText\":\"\",\"body\":{\"input\":{\"messages\":[{\"role\":\"system\",\"content\":\"\\n You are a helpful assistant. Use tools when appropriate.\\n \"},{\"role\":\"user\",\"content\":{\"content\":\"[{\\\"text\\\": \\\"What is 2+2? Reply with just the number.\\\"}]\"}}]},\"output\":{\"messages\":[{\"role\":\"assistant\",\"content\":{\"message\":\"4\\n\",\"finish_reason\":\"end_turn\"}}]}},\"attributes\":{\"session.id\":\"00000000-0000-4000-8000-4a4a72800001\",\"event.name\":\"strands.telemetry.tracer\"},\"flags\":1,\"traceId\":\"6a91fe944cf38d6b2d8c3c6a66e63ad4\",\"spanId\":\"5f3fc94f1decac15\"}"
},
{
"field": "sessionId",
"value": "00000000-0000-4000-8000-4a4a72800001"
},
{
"field": "traceId",
"value": "6a91fe944cf38d6b2d8c3c6a66e63ad4"
},
{
"field": "spanId",
"value": "5f3fc94f1decac15"
},
{
"field": "@ptr",
"value": "Cs0BCo8BCkw2ODUxOTc3MDg2ODc6L2F3cy9iZWRyb2NrLWFnZW50Y29yZS9ydW50aW1lcy9hc2RmX015QWdlbnQtM3M1YXh2QkM2US1ERUZBVUxUEAAaJGY2ZGY3NDIyLWVlZTQtNDhkNS05NDg3LTE0NTZkYjdkZGE1YiIOCKCUkduCNBCHpMP7hDRA0cHR4dszSAASNxoYAgajvKv7AAAAABTZwGgABqkf6XAAAAKyIAEolYKq0oQ0MJyuqtKENDgqQJe/AkiwkQFQgGYgAhApGAE="
}
]
],
"statistics": {
"recordsMatched": 6,
"recordsScanned": 139,
"estimatedRecordsSkipped": 107,
"bytesScanned": 126707,
"estimatedBytesSkipped": 113696,
"logGroupsScanned": 1
},
"status": "Complete"
}
Loading
Loading