diff --git a/docs/design/brick_runtime_interface_v1.md b/docs/design/brick_runtime_interface_v1.md index ede9b13..a321624 100644 --- a/docs/design/brick_runtime_interface_v1.md +++ b/docs/design/brick_runtime_interface_v1.md @@ -124,8 +124,10 @@ Keep the input tensors valid and unmodified until `is_generation_complete()` ret Repeated calls for the same generation must provide the same input values, but their storage addresses may change between calls. -After `is_generation_complete()` returns `true`, the user must call `reset_states()` before starting -a new session. +When `is_generation_complete()` returns `true`, the current generation has finished and the +application can provide new input to `run()`. For bricks that support continuing a session across +generations, keep the same brick instance and preserve its state between generations. Call +`reset_states()` before starting an independent session on that instance. `api()` returns the exact `info.json` `api` value implemented by the brick, such as `tts.v1`. The value identifies the public input/output tensor contract and is available immediately after diff --git a/docs/howto/developing_apps_with_compute_flow.md b/docs/howto/developing_apps_with_compute_flow.md index 2f65098..005f266 100644 --- a/docs/howto/developing_apps_with_compute_flow.md +++ b/docs/howto/developing_apps_with_compute_flow.md @@ -115,7 +115,21 @@ application will automatically support both current and future variants without ### Keep Inputs Valid During Generation -Keep the input tensors valid and unmodified until `is_generation_complete()` returns `true`. +Keep the input tensors valid and unmodified until `is_generation_complete()` returns `true`. Each +call to `run()` for that generation must supply the same input values. Once generation completes, +you can supply new input. + +### Continue a conversation across turns + +With the `lm_text_generic` brick using the llama.cpp runner, each turn has its own generation: + +1. Pass `What is a KV cache?` as `text_prompt` and call `run()` with that prompt until + `is_generation_complete()` returns `true`. +2. On the same brick instance, pass only the follow-up `How much memory does it use?` as + `text_prompt`. Call `run()` with that prompt until the second response completes. The runner + reuses its KV cache from the previous turn. Passing the full transcript would append the history + again. +3. Call `reset_states()` before starting an independent conversation. ### Follow the Brick Interface Specification