As we integrate the CogStack Model Gateway with more systems a need arises for requests to be grouped together. More specifically, we wanna be able to track multiple requests as a single task, being able to reason about e2e latency, progress, and outcome.
Grouping can be implemented based on an optional group ID at the Gateway entrypoint. Based on our current needs and with future-proofing in mind, that ID can consist of 2 parts, which would allow for more fine-grained insights down the line, by denoting both the overall task a request is part of, as well as its subpart which could be useful for tracking dependency groups.
As we integrate the CogStack Model Gateway with more systems a need arises for requests to be grouped together. More specifically, we wanna be able to track multiple requests as a single task, being able to reason about e2e latency, progress, and outcome.
Grouping can be implemented based on an optional group ID at the Gateway entrypoint. Based on our current needs and with future-proofing in mind, that ID can consist of 2 parts, which would allow for more fine-grained insights down the line, by denoting both the overall task a request is part of, as well as its subpart which could be useful for tracking dependency groups.