Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 45 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

A yoke is the wooden frame that makes oxen and horses pull in the direction you want. Yoke does the same for models: it is the runtime layer wrapped around a model so it can do real work reliably. It turns AgentScope's agent capabilities into a ready-to-use HTTP service, so you can focus on your business logic instead of glue code:

- One endpoint for multi-turn conversations: `POST /api/chat`, plus `POST /api/chat/stream` that streams the reasoning process over SSE.
- One endpoint for multi-turn conversations: `POST /api/chat`, plus `POST /api/chat/stream` that streams the reasoning process over SSE using the AG-UI protocol.
- A built-in `get_weather` tool queries current weather through the open-source [Open-Meteo](https://open-meteo.com) API (no key needed; its free tier is for non-commercial use only).
- A temporary agent is built for every request and closed afterwards; conversation history lives in a shared state store and is isolated per session.
- Models are plugged in through the official AgentScope starters: OpenAI-compatible APIs (works with DashScope, DeepSeek and others), Anthropic, Gemini and Ollama. Switching is a dependency and configuration change.
Expand All @@ -44,47 +44,65 @@ curl -X POST localhost:8080/api/chat \
# {"sessionId":"…","reply":"…"}
```

Send the returned `sessionId` with later requests to continue the same conversation; conversations with different `sessionId`s are isolated from each other. Run the tests with `./gradlew build`.
Send the returned `sessionId` with later requests to continue the same conversation; conversations with different `sessionId`s are isolated from each other. A `sessionId` you send yourself must follow the same rule as `threadId` below (letters, digits and `_.-`, 1 to 64 characters, not all dots), otherwise the request returns 400; a blank one is treated as missing. Run the tests with `./gradlew build`.

### Streaming
### Streaming (AG-UI)

`POST /api/chat/stream` takes the same request body and pushes events (`text/event-stream`) while the agent is reasoning. Use `curl -N` to turn off buffering:
`POST /api/chat/stream` speaks the [AG-UI](https://docs.ag-ui.com/concepts/events) protocol: the request body is a standard `RunAgentInput`, and the response is a `text/event-stream` of AG-UI events pushed while the agent is reasoning. Use `curl -N` to turn off buffering:

```bash
curl -N -X POST localhost:8080/api/chat/stream \
-H 'Content-Type: application/json' \
-d '{"sessionId": "demo", "message": "What is the weather in Beijing?"}'
-d '{"threadId": "demo", "runId": "run-1", "messages": [{"id": "msg-1", "role": "user", "content": "What is the weather in Beijing?"}]}'
```

Illustrative output (from a stub model):
Request rules (any violation returns 400 before the stream starts):

- `Content-Type` must be `application/json`, otherwise 415. This also stops a cross-site page from starting a run with a `text/plain` request that skips the CORS preflight
- `threadId` is the session ID and is required: letters, digits and `_.-`, 1 to 64 characters, not all dots. It shares the same history as `sessionId` of `POST /api/chat`
- `runId` is required and must not be blank (`RunAgentInput` requires it)
- `messages` must not be empty, every message needs an `id` and a `role`, and the last one must be a `user` message with non-blank plain text (multimodal content blocks are not supported)
- The server keeps the history itself and only uses the **last** `user` message. Earlier messages, `tools`, `context`, `state`, `forwardedProps` and `resume` are ignored, so a client cannot add tools to the agent or rewrite the history. Only the text of that message is kept: its `id`, `toolCalls` and `toolCallId` never reach the history, and the message ID is generated on the server. The adapter is also configured with `ToolMergeMode.AGENT_ONLY`, so only server-registered tools are ever used

Illustrative output (values are made up, field names come from real runs against a stub model, long fields are elided). Only `data:` lines are sent; the event type is the `type` field in the JSON, there is no SSE `event:` name:

```
event:textDelta
data:{"text":"Let me check."}
data:{"type":"RUN_STARTED","threadId":"demo","runId":"…","input":{…}}

data:{"type":"TOOL_CALL_START","threadId":"demo","runId":"…","toolCallId":"call_1","toolCallName":"get_weather"}

data:{"type":"TOOL_CALL_ARGS","threadId":"demo","runId":"…","toolCallId":"call_1","delta":"{\"city\":\"Beijing\"}"}

event:toolCall
data:{"id":"call_1","name":"get_weather","arguments":"{\"city\":\"Beijing\"}"}
data:{"type":"TOOL_CALL_END","threadId":"demo","runId":"…","toolCallId":"call_1"}

event:toolResult
data:{"id":"call_1","name":"get_weather","state":"success","output":"\"Sunny, 25 C\""}
data:{"type":"TOOL_CALL_RESULT","threadId":"demo","runId":"…","toolCallId":"call_1","content":"\"Sunny, 25 C\"","role":"tool","messageId":"…"}

event:textDelta
data:{"text":"It is sunny, 25 C."}
data:{"type":"TEXT_MESSAGE_START","threadId":"demo","runId":"…","messageId":"…","role":"assistant"}

event:done
data:{"sessionId":"demo","text":"It is sunny, 25 C.","usage":{"inputTokens":83,"outputTokens":27,"totalTokens":110}}
data:{"type":"TEXT_MESSAGE_CONTENT","threadId":"demo","runId":"…","messageId":"…","delta":"It is sunny, 25 C."}

data:{"type":"TEXT_MESSAGE_END","threadId":"demo","runId":"…","messageId":"…"}

data:{"type":"CUSTOM","threadId":"demo","runId":"…","name":"token_usage","value":{"delta":{…},"cumulative":{"inputTokens":83,"outputTokens":27,…},"modelCall":{…}}}

data:{"type":"RUN_FINISHED","threadId":"demo","runId":"…"}
```

The event names form a closed set:
| Event `type` | Meaning |
|---|---|
| `RUN_STARTED` | Always the first event, carries `threadId` and `runId` |
| `TEXT_MESSAGE_START` / `TEXT_MESSAGE_CONTENT` / `TEXT_MESSAGE_END` | Generated text; concatenate the `delta` of the `CONTENT` events in order to get the reply |
| `TOOL_CALL_START` / `TOOL_CALL_ARGS` / `TOOL_CALL_END` | A tool the model decided to call; `TOOL_CALL_ARGS` carries fragments of the arguments JSON and only appears when the model streams the arguments in pieces |
| `TOOL_CALL_RESULT` | The result of a tool, matched to the call by `toolCallId` |
| `CUSTOM` with `name` = `token_usage` | Token usage after each model call, with `delta` and `cumulative` |
| `CUSTOM` with `name` = `hint` | A hint the framework adds during reasoning, `value` has `source` and `text` |
| `RUN_FINISHED` | The normal end |
| `RUN_ERROR` | Reasoning failed; the HTTP status is still 200 and the stream ends normally instead of dropping the connection. The message is the raw exception message and is not redacted, so do not show it to end users directly |

- `textDelta`: a chunk of generated text; concatenate them in order to get the reply
- `toolCall`: a tool the model decided to call, with its name and arguments already assembled
- `toolResult`: the result of a tool, matched to its `toolCall` by `id`
- `hint`: a hint the framework adds during reasoning
- `done`: the normal end, with the full reply, the `sessionId` (generated by the server if you did not send one) and the accumulated token usage
- `error`: reasoning failed; the HTTP status is still 200 and the stream ends normally instead of dropping the connection. The message is the raw exception message and is not redacted, so do not show it to end users directly
Every stream starts with `RUN_STARTED`, even when the agent cannot be created, and ends with either `RUN_FINISHED` or `RUN_ERROR`, never both. When the agent fails while running, the AG-UI adapter maps the exception to `code` (`TIMEOUT_ERROR`, `INTERRUPTED_ERROR`, `INVALID_INPUT_ERROR` or `INTERNAL_ERROR`); failures outside the adapter, such as the agent not being created, are always `INTERNAL_ERROR`. There is no `done` event and the final text is not repeated: the client concatenates `TEXT_MESSAGE_CONTENT`. The temporary agent is closed when the stream completes, fails or the client disconnects. The adapter's `runTimeout` setting is not read by AgentScope 2.0.4, so it does not limit the run time.

Every stream ends with either `done` or `error`. A blank `message` returns 400 before the stream starts. The temporary agent is closed when the stream completes, fails or the client disconnects.
> **Breaking change**
> The earlier private events (`textDelta`, `toolCall`, `toolResult`, `hint`, `done`, `error`) and the `{"sessionId", "message"}` request body of the streaming endpoint are gone. The streaming endpoint now only accepts `Content-Type: application/json`; requests without a `Content-Type`, or with `text/plain` or `application/*+json`, get 415. `POST /api/chat` now validates `sessionId` with the same rule as `threadId` (non-blank values like `user:42` used to be accepted and now return 400); a blank `sessionId` still starts a new session.

> **Note**
> The default in-memory state store only suits a single process, and history is lost when the service restarts. For multi-instance deployments, switch to a Redis or JDBC implementation. Concurrent requests for the same session are not serialized, so avoid sending them at the same time.
Expand Down Expand Up @@ -116,10 +134,10 @@ To use another model, replace `agentscope-openai-spring-boot-starter` in `build.
src/main/java/com/yoke/
├── YokeApplication.java Application entry point
├── api/ HTTP contract (ChatApi: paths, validation and docs live here)
├── codec/ Conversion between domain types and framework types (MessageCodec, EventCodec)
├── config/ Shared agent configuration (builder, state store)
├── codec/ Conversion between domain types and framework types (MessageCodec, AguiStreamCodec, HintEventConverter)
├── config/ Shared agent configuration (builder, state store, AG-UI adapter)
├── controller/ Contract implementation (ChatController)
└── model/ Request, response and stream event objects (ChatRequest, ChatResponse, ChatEvent)
└── model/ Request and response objects (ChatRequest, ChatResponse)
src/main/resources/ application.yml and the per-profile configuration
docs/brand/ Logo
```
Expand Down
73 changes: 44 additions & 29 deletions README_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

Yoke(轭)是套在牛马身上、让它们听使唤拉车的木架;Yoke 则是套在模型外面、让它能稳定干活的那层运行框架。它把 AgentScope 的 Agent 能力包装成一个开箱即用的 HTTP 服务,让你专注于业务,而不是胶水代码:

- 一个接口完成多轮对话:`POST /api/chat`;另有 `POST /api/chat/stream`,通过 SSE 边推理边推送。
- 一个接口完成多轮对话:`POST /api/chat`;另有 `POST /api/chat/stream`,按 AG-UI 协议通过 SSE 边推理边推送。
- 内置 `get_weather` 工具,通过开源的 [Open-Meteo](https://open-meteo.com) 接口查询当前天气(无需密钥,免费版仅限非商业使用)。
- 每次请求新建临时 Agent,用完即关闭;对话历史保存在共享的状态存储里,按会话隔离。
- 通过 AgentScope 官方 starter 接入各种模型:OpenAI 兼容协议(可对接 DashScope、DeepSeek 等)、Anthropic、Gemini、Ollama,换依赖和配置即可。
Expand All @@ -44,50 +44,65 @@ curl -X POST localhost:8080/api/chat \
# {"sessionId":"…","reply":"…"}
```

后续请求带上返回的 `sessionId`,即可延续同一段对话;不同 `sessionId` 的对话历史互相隔离。运行测试:`./gradlew build`。
后续请求带上返回的 `sessionId`,即可延续同一段对话;不同 `sessionId` 的对话历史互相隔离。自己传的 `sessionId` 要符合下面 `threadId` 的同一条规则(字母、数字、`_.-`,1~64 位,不能全是点),否则返回 400;空白视同未传。运行测试:`./gradlew build`。

### 流式调用
### 流式调用(AG-UI)

`POST /api/chat/stream` 的请求体与上面相同,Agent 推理的同时就把事件推送出来(`text/event-stream`)。用 `curl -N` 关闭缓冲:
`POST /api/chat/stream` 使用 [AG-UI](https://docs.ag-ui.com/concepts/events) 协议:请求体是标准的 `RunAgentInput`,响应是 `text/event-stream`,Agent 推理的同时就把 AG-UI 事件推送出来。用 `curl -N` 关闭缓冲:

```bash
curl -N -X POST localhost:8080/api/chat/stream \
-H 'Content-Type: application/json' \
-d '{"sessionId": "demo", "message": "北京天气怎么样"}'
-d '{"threadId": "demo", "runId": "run-1", "messages": [{"id": "msg-1", "role": "user", "content": "北京天气怎么样"}]}'
```

示意输出(来自桩模型):
请求规则(任何一条不满足,都会在建立流之前返回 400):

- `Content-Type` 必须是 `application/json`,否则返回 415。这也挡住了跨站页面用不触发 CORS 预检的 `text/plain` 请求发起对话
- `threadId` 就是会话 ID,必填:字母、数字、`_.-`,1~64 位,不能全是点。它与 `POST /api/chat` 的 `sessionId` 共用同一份历史
- `runId` 必填且不能为空白(`RunAgentInput` 要求非空)
- `messages` 不能为空,每条消息都要有 `id` 和 `role`,最后一条必须是 `user` 消息且有非空白的纯文本(不支持多模态内容块)
- 对话历史由服务端保存,只采用**最后一条** `user` 消息;前面的消息、`tools`、`context`、`state`、`forwardedProps`、`resume` 都会被忽略,客户端不能借此给 Agent 加工具或改写历史。这条消息也只取文本:它的 `id`、`toolCalls`、`toolCallId` 不会进入历史,消息 ID 由服务端生成。适配器还配置了 `ToolMergeMode.AGENT_ONLY`,只会用服务端注册的工具

示意输出(取值是编的,字段名来自桩模型的实际运行,较长的字段做了省略)。只发 `data:` 行,事件类型是 JSON 里的 `type` 字段,没有 SSE 的 `event:` 名:

```
event:textDelta
data:{"text":"我来查一下天气。"}
data:{"type":"RUN_STARTED","threadId":"demo","runId":"…","input":{…}}

data:{"type":"TOOL_CALL_START","threadId":"demo","runId":"…","toolCallId":"call_1","toolCallName":"get_weather"}

data:{"type":"TOOL_CALL_ARGS","threadId":"demo","runId":"…","toolCallId":"call_1","delta":"{\"city\":\"北京\"}"}

event:toolCall
data:{"id":"call_1","name":"get_weather","arguments":"{\"city\":\"北京\"}"}
data:{"type":"TOOL_CALL_END","threadId":"demo","runId":"…","toolCallId":"call_1"}

event:toolResult
data:{"id":"call_1","name":"get_weather","state":"success","output":"\"晴,25 度(北京)\""}
data:{"type":"TOOL_CALL_RESULT","threadId":"demo","runId":"…","toolCallId":"call_1","content":"\"晴,25 度(北京)\"","role":"tool","messageId":"…"}

event:textDelta
data:{"text":"北京今天"}
data:{"type":"TEXT_MESSAGE_START","threadId":"demo","runId":"…","messageId":"…","role":"assistant"}

event:textDelta
data:{"text":"晴,25 度。"}
data:{"type":"TEXT_MESSAGE_CONTENT","threadId":"demo","runId":"…","messageId":"…","delta":"北京今天晴,25 度。"}

event:done
data:{"sessionId":"demo","text":"北京今天晴,25 度。","usage":{"inputTokens":83,"outputTokens":27,"totalTokens":110}}
data:{"type":"TEXT_MESSAGE_END","threadId":"demo","runId":"…","messageId":"…"}

data:{"type":"CUSTOM","threadId":"demo","runId":"…","name":"token_usage","value":{"delta":{…},"cumulative":{"inputTokens":83,"outputTokens":27,…},"modelCall":{…}}}

data:{"type":"RUN_FINISHED","threadId":"demo","runId":"…"}
```

事件名是一个封闭集合:
| 事件 `type` | 含义 |
|---|---|
| `RUN_STARTED` | 总是第一个事件,带 `threadId` 和 `runId` |
| `TEXT_MESSAGE_START` / `TEXT_MESSAGE_CONTENT` / `TEXT_MESSAGE_END` | 模型生成的文本;按顺序拼接 `CONTENT` 事件的 `delta` 即得到回复 |
| `TOOL_CALL_START` / `TOOL_CALL_ARGS` / `TOOL_CALL_END` | 模型决定调用的工具;`TOOL_CALL_ARGS` 的 `delta` 是入参 JSON 的片段,只有模型分片返回入参时才会出现 |
| `TOOL_CALL_RESULT` | 工具的执行结果,通过 `toolCallId` 与调用对应 |
| `CUSTOM`,`name` 为 `token_usage` | 每次模型调用结束后的 token 用量,含 `delta` 和 `cumulative` |
| `CUSTOM`,`name` 为 `hint` | 框架在推理过程中追加的提示,`value` 里有 `source` 和 `text` |
| `RUN_FINISHED` | 正常结束 |
| `RUN_ERROR` | 推理出错;HTTP 状态码仍是 200,流正常结束而不是断开连接。message 是原样的异常信息,未做脱敏,不要直接展示给终端用户 |

- `textDelta`:模型生成的一段文本增量,按顺序拼接即得到回复
- `toolCall`:模型决定调用的工具,名称和入参已经攒齐
- `toolResult`:工具的执行结果,通过 `id` 与 `toolCall` 对应
- `hint`:框架在推理过程中追加的提示
- `done`:正常结束,带完整回复、`sessionId`(没传时由服务端生成)和累计 token 用量
- `error`:推理出错;HTTP 状态码仍是 200,流正常结束而不是断开连接。message 是原样的异常信息,未做脱敏,不要直接展示给终端用户
每次流都以 `RUN_STARTED` 开头(创建 Agent 失败时也一样),以 `RUN_FINISHED` 或 `RUN_ERROR` 结尾,二者不会同时出现。Agent 运行中出错时,`RUN_ERROR` 的 `code` 由 AG-UI 适配器按异常类型映射(`TIMEOUT_ERROR`、`INTERRUPTED_ERROR`、`INVALID_INPUT_ERROR` 或 `INTERNAL_ERROR`);适配器之外的失败(例如创建 Agent 失败)一律是 `INTERNAL_ERROR`。没有 `done` 事件,也不重复最终全文:由客户端拼接 `TEXT_MESSAGE_CONTENT`。流正常结束、出错或客户端断开时,临时 Agent 都会被关闭。适配器的 `runTimeout` 配置在 AgentScope 2.0.4 里没有被读取,所以它不能限制运行时长。

每次流都以 `done` 或 `error` 结尾。`message` 为空白时在建立流之前就返回 400。流正常结束、出错或客户端断开时,临时 Agent 都会被关闭。
> **不兼容变更**
> 流式接口原先的私有事件(`textDelta`、`toolCall`、`toolResult`、`hint`、`done`、`error`)和 `{"sessionId", "message"}` 请求体已经移除。流式接口现在只接受 `Content-Type: application/json`,不带 `Content-Type`、或者用 `text/plain`、`application/*+json` 的请求都会返回 415。`POST /api/chat` 的 `sessionId` 现在按 `threadId` 的同一条规则校验(以前 `user:42` 这类非空白值能通过,现在返回 400);空白的 `sessionId` 仍然新建会话。

> **注意**
> 默认的内存状态存储只适合单进程,服务重启后历史会丢失;多副本部署时需要换成 Redis 或 JDBC 实现。同一会话的并发请求不会被串行化,请避免同时发送。
Expand Down Expand Up @@ -119,10 +134,10 @@ data:{"sessionId":"demo","text":"北京今天晴,25 度。","usage":{"inputTok
src/main/java/com/yoke/
├── YokeApplication.java 应用入口
├── api/ HTTP 接口契约(ChatApi,路径、校验和文档都在这里)
├── codec/ 领域类型与框架类型之间的转换(MessageCodec、EventCodec)
├── config/ Agent 共享配置(Builder、状态存储)
├── codec/ 领域类型与框架类型之间的转换(MessageCodec、AguiStreamCodec、HintEventConverter)
├── config/ Agent 共享配置(Builder、状态存储、AG-UI 适配器)
├── controller/ 接口实现(ChatController)
└── model/ 请求、响应与流式事件对象(ChatRequest、ChatResponse、ChatEvent)
└── model/ 请求与响应对象(ChatRequest、ChatResponse)
src/main/resources/ application.yml 及各环境配置
docs/brand/ logo
```
Expand Down
Loading
Loading