Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions docs/design/brick_runtime_interface_v1.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,8 +124,10 @@ Keep the input tensors valid and unmodified until `is_generation_complete()` ret
Repeated calls for the same generation must provide the same input values, but their storage
addresses may change between calls.

After `is_generation_complete()` returns `true`, the user must call `reset_states()` before starting
a new session.
When `is_generation_complete()` returns `true`, the current generation has finished and the
application can provide new input to `run()`. For bricks that support continuing a session across
generations, keep the same brick instance and preserve its state between generations. Call
`reset_states()` before starting an independent session on that instance.

`api()` returns the exact `info.json` `api` value implemented by the brick, such as `tts.v1`. The
value identifies the public input/output tensor contract and is available immediately after
Expand Down
16 changes: 15 additions & 1 deletion docs/howto/developing_apps_with_compute_flow.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,21 @@ application will automatically support both current and future variants without

### Keep Inputs Valid During Generation

Keep the input tensors valid and unmodified until `is_generation_complete()` returns `true`.
Keep the input tensors valid and unmodified until `is_generation_complete()` returns `true`. Each
call to `run()` for that generation must supply the same input values. Once generation completes,
you can supply new input.

### Continue a conversation across turns

With the `lm_text_generic` brick using the llama.cpp runner, each turn has its own generation:

1. Pass `What is a KV cache?` as `text_prompt` and call `run()` with that prompt until
`is_generation_complete()` returns `true`.
2. On the same brick instance, pass only the follow-up `How much memory does it use?` as
`text_prompt`. Call `run()` with that prompt until the second response completes. The runner
reuses its KV cache from the previous turn. Passing the full transcript would append the history
again.
3. Call `reset_states()` before starting an independent conversation.

### Follow the Brick Interface Specification

Expand Down
Loading