Add top-level README and publish H100 benchmark reports - #3
Open
abhay1999 wants to merge 2 commits into
Open
Conversation
Adds a real top-level README explaining what this repo is, its layout, and a quick-start command, replacing the placeholder stub. Also brings over the published benchmark reports from agentgateway/agentgateway#2526 - the real 16x H100 GPU campaign comparing agentgateway standalone and agentgateway-on-Kubernetes against a plain Kubernetes Service, with campaign manifest, provenance, metrics, and charts for both comparisons. Signed-off-by: abhay1999 <abhaychaurasiya19@gmail.com>
danehans
suggested changes
Sep 1, 2026
Comment on lines
+78
to
+79
| - [k8s service (RR) native results](../../../../../results/llm-d-benchmark/optimized-baseline-v0230-gateway-refresh-20260817/runs/service) | ||
| - [agentgateway on Kubernetes native results](../../../../../results/llm-d-benchmark/optimized-baseline-v0230-gateway-refresh-20260817/runs/agentgateway-gateway) |
Contributor
There was a problem hiding this comment.
These two links are broken.
Contributor
Author
There was a problem hiding this comment.
Removed, they pointed into the gitignored results/ tree.
Comment on lines
+78
to
+79
| - [k8s service (RR) native results](../../../../../results/llm-d-benchmark/optimized-baseline-v0230-gateway-refresh-20260817/runs/service) | ||
| - [agentgateway standalone native results](../../../../../results/llm-d-benchmark/optimized-baseline-v0230-gateway-refresh-20260817/runs/agentgateway-standalone) |
Contributor
Author
There was a problem hiding this comment.
Removed, same issue.
Comment on lines
+23
to
+26
| ```bash | ||
| make kind-create | ||
| BENCHMARK_TREATMENT=service BENCHMARK_CAMPAIGN_ID=local-sim make benchmark | ||
| ``` |
| BENCHMARK_TREATMENT=service BENCHMARK_CAMPAIGN_ID=local-sim make benchmark | ||
| ``` | ||
|
|
||
| See [`inference/README.md`](inference/README.md) for the full set of treatments, |
Contributor
There was a problem hiding this comment.
Remove the following from inference/README.md:
PR #1 intentionally migrates tooling only. Curated reports from the original agentgateway repository change are not part of this migration and will be published separately with their supporting evidence.
- Removed the "native results" links in both comparison READMEs. They pointed into the gitignored results/ tree, which was never committed, per review. - Added a note in the root README pointing out that agentgateway- standalone/-gateway currently need more resources than a laptop Kind cluster provides, linking agentgateway#2. - Removed the now-stale "reports will be published separately" line from inference/README.md, since this PR publishes them. Signed-off-by: abhay1999 <abhaychaurasiya19@gmail.com>
danehans
approved these changes
Sep 3, 2026
danehans
left a comment
danehans
left a comment
Contributor
There was a problem hiding this comment.
Thanks for resolving my review feedback. @howardjohn I plan on creating an agentgateway/website PR to publish these results to to the inference routing docs, PTAL.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
Two tasks for this repo while we wait on management's decision on funding the full benchmarking campaign:
Top-level README - the repo had just a one-line stub pointing at
inference/README.md. Replaced it with a real explanation of what the repo is, its layout, a quick-start command, and a summary of the current published results.Reports from agentgateway/agentgateway#2526 - brought over the actual published campaign (
optimized-baseline-v0230-gateway-refresh-20260817, 16x H100 GPUs, both comparisons: service-vs-agentgateway-standalone and service-vs-agentgateway-gateway) with its campaign manifest, provenance, metrics, and charts. Verified this is the exact same file set as in #2526 by diffing againstgh pr view 2526 --json files.Also updated
inference/reports/README.md's placeholder text ("reports will be added separately") since reports are now actually here.Next up
Once this PR merges, next is an
agentgateway/websitePR adding benchmark report subpages under/docs/standalone/latest/inference/and/docs/kubernetes/latest/inference/, followed by #2 (making router/EPP resource requests configurable for local Kind testing).