From af5572df234136e6fd6756845043790c32897269 Mon Sep 17 00:00:00 2001 From: lgunreddi Date: Thu, 9 Jul 2026 16:06:22 -0400 Subject: [PATCH 1/7] Create worker-affinity.mdx --- serverless/load-balancing/worker-affinity.mdx | 65 +++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 serverless/load-balancing/worker-affinity.mdx diff --git a/serverless/load-balancing/worker-affinity.mdx b/serverless/load-balancing/worker-affinity.mdx new file mode 100644 index 000000000..0f8030a49 --- /dev/null +++ b/serverless/load-balancing/worker-affinity.mdx @@ -0,0 +1,65 @@ +--- +title: "Worker affinity" +sidebarTitle: "Worker affinity" +description: "Pin follow-up requests to a specific worker using the X-Runpod-Worker-Id header." +--- + +Worker affinity lets clients route follow-up requests to the same worker that served an earlier request. This is useful for stateful workloads where a worker holds session state in memory and re-routing to a different worker would require reloading it. + +## How it works + +Every response from the load balancer includes an `X-Runpod-Worker-Id` header that identifies the worker that handled the request: + +``` +X-Runpod-Worker-Id: +``` + +To pin a subsequent request to that worker, send the header back with one of the following values: + +| Header value | Behavior | +|---|---| +| *(omitted or empty)* | Normal routing — least-load selection, no preference | +| `` | **Soft affinity**: prefer the specified worker; fall back to least-load if the worker is unavailable or at capacity | +| `strict ` | **Strict affinity**: only use the specified worker; wait if the worker is at capacity, return `404` if the worker is gone | +| `strict-resume ` | **Strict-resume affinity**: same as strict, but also resumes the worker pod if it has been scaled down | + +## Affinity modes + +### Soft affinity + +``` +X-Runpod-Worker-Id: wrk_abc123 +``` + +The load balancer attempts to route to the specified worker. If the worker is unavailable, at capacity, or not found, the request falls through to normal least-load selection. This mode never fails a request due to affinity; it is best-effort. + +### Strict affinity + +``` +X-Runpod-Worker-Id: strict wrk_abc123 +``` + +The load balancer only routes to the specified worker. If the worker is at capacity, the request waits. If the worker is gone and not found in the active worker set, the load balancer returns a `404` with reason `affinity_worker_gone`. Use this mode when falling back to a different worker would produce incorrect results. + +### Strict-resume affinity + +``` +X-Runpod-Worker-Id: strict-resume wrk_abc123 +``` + +Same as strict, but if the worker pod has been scaled down, the load balancer calls `ResumePodById` to bring it back up before routing the request. The worker is scoped to the endpoint and cross-endpoint access is not possible. + + +`strict-resume` effectively prevents scale-down for the pinned worker as long as requests keep arriving. This keeps the worker running and accruing cost. Use this mode intentionally and ensure your client stops sending the affinity header when the session ends. + + +## Worker ID on responses + +The load balancer always sets `X-Runpod-Worker-Id` on the response to reflect the worker that actually served the request. Use the value from the response header for pinning rather than tracking it separately. + +## Error reference + +| Status | Reason | Description | +|--------|--------|-------------| +| `404` | `affinity_worker_gone` | Strict mode: the requested worker is not in the active worker set | +| `503` | `affinity_resume_failed` | Strict-resume mode: the worker pod could not be resumed | From abb40396d0712431cf7256647e8ce361da9281c8 Mon Sep 17 00:00:00 2001 From: lgunreddi Date: Thu, 9 Jul 2026 16:07:46 -0400 Subject: [PATCH 2/7] Update docs.json --- docs.json | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs.json b/docs.json index 474ced9ad..8e4c5a7cb 100644 --- a/docs.json +++ b/docs.json @@ -150,7 +150,8 @@ "pages": [ "serverless/load-balancing/overview", "serverless/load-balancing/build-a-worker", - "serverless/load-balancing/vllm-worker" + "serverless/load-balancing/vllm-worker", + "serverless/load-balancing/worker-affinity" ] }, { From 3a44c9e69ee109e69f857219e83da4839e59e7e8 Mon Sep 17 00:00:00 2001 From: lgunreddi Date: Fri, 17 Jul 2026 09:59:14 -0400 Subject: [PATCH 3/7] Update worker-affinity.mdx --- serverless/load-balancing/worker-affinity.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/serverless/load-balancing/worker-affinity.mdx b/serverless/load-balancing/worker-affinity.mdx index 0f8030a49..e55c77dfc 100644 --- a/serverless/load-balancing/worker-affinity.mdx +++ b/serverless/load-balancing/worker-affinity.mdx @@ -50,7 +50,7 @@ X-Runpod-Worker-Id: strict-resume wrk_abc123 Same as strict, but if the worker pod has been scaled down, the load balancer calls `ResumePodById` to bring it back up before routing the request. The worker is scoped to the endpoint and cross-endpoint access is not possible. -`strict-resume` effectively prevents scale-down for the pinned worker as long as requests keep arriving. This keeps the worker running and accruing cost. Use this mode intentionally and ensure your client stops sending the affinity header when the session ends. +`strict-resume` keeps a worker running as long as requests keep coming in. Workers scale down after being idle (no requests) for a set period. If your  client sends a request before that idle period expires, the timer resets and  the worker stays up. This means a worker receiving regular traffic, with or  without strict-resume, will never scale down.  When your session is done, send your next request without the  `X-Runpod-Worker-Id` header so the worker is no longer pinned and can go idle. ## Worker ID on responses From 591e60f86afb638ac972469ddc55e96f11a1c44e Mon Sep 17 00:00:00 2001 From: lgunreddi Date: Fri, 17 Jul 2026 10:09:58 -0400 Subject: [PATCH 4/7] Update worker-affinity.mdx --- serverless/load-balancing/worker-affinity.mdx | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/serverless/load-balancing/worker-affinity.mdx b/serverless/load-balancing/worker-affinity.mdx index e55c77dfc..120c134d9 100644 --- a/serverless/load-balancing/worker-affinity.mdx +++ b/serverless/load-balancing/worker-affinity.mdx @@ -49,9 +49,11 @@ X-Runpod-Worker-Id: strict-resume wrk_abc123 Same as strict, but if the worker pod has been scaled down, the load balancer calls `ResumePodById` to bring it back up before routing the request. The worker is scoped to the endpoint and cross-endpoint access is not possible. - -`strict-resume` keeps a worker running as long as requests keep coming in. Workers scale down after being idle (no requests) for a set period. If your  client sends a request before that idle period expires, the timer resets and  the worker stays up. This means a worker receiving regular traffic, with or  without strict-resume, will never scale down.  When your session is done, send your next request without the  `X-Runpod-Worker-Id` header so the worker is no longer pinned and can go idle. - +`strict-resume` keeps a worker running as long as requests keep coming in. Workers scale down after being idle (no requests) for a set period. If your  client sends a request before that idle period expires, the timer resets and  the worker stays up. This means a worker receiving regular traffic, with or  without strict-resume, will never scale down.  + + + When your session is done, send your next request without the  `X-Runpod-Worker-Id` header so the worker is no longer pinned and can go idle. + ## Worker ID on responses From 0fdfa59f2b111a5021c95c5194178888390a5f8c Mon Sep 17 00:00:00 2001 From: lgunreddi Date: Fri, 17 Jul 2026 14:57:40 -0400 Subject: [PATCH 5/7] Update worker-affinity.mdx --- serverless/load-balancing/worker-affinity.mdx | 36 +++++++++---------- 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/serverless/load-balancing/worker-affinity.mdx b/serverless/load-balancing/worker-affinity.mdx index 120c134d9..a86eb6464 100644 --- a/serverless/load-balancing/worker-affinity.mdx +++ b/serverless/load-balancing/worker-affinity.mdx @@ -16,43 +16,43 @@ X-Runpod-Worker-Id: To pin a subsequent request to that worker, send the header back with one of the following values: -| Header value | Behavior | -|---|---| -| *(omitted or empty)* | Normal routing — least-load selection, no preference | -| `` | **Soft affinity**: prefer the specified worker; fall back to least-load if the worker is unavailable or at capacity | -| `strict ` | **Strict affinity**: only use the specified worker; wait if the worker is at capacity, return `404` if the worker is gone | -| `strict-resume ` | **Strict-resume affinity**: same as strict, but also resumes the worker pod if it has been scaled down | +| Header value | Behavior | +| --------------------------- | ------------------------------------------------------------------------------------------------------------------------- | +| *(omitted or empty)* | Normal routing — normal worker selection, no preference | +| `` | **Soft affinity**: prefer the specified worker; fall back to normal worker selection if the worker is unavailable or at capacity | +| `strict ` | **Strict affinity**: only use the specified worker; wait if the worker is at capacity, return `404` if the worker is gone | +| `strict-resume ` | **Strict-resume affinity**: same as strict, but also resumes the worker pod if it has been scaled down | ## Affinity modes ### Soft affinity ``` -X-Runpod-Worker-Id: wrk_abc123 +X-Runpod-Worker-Id: pod-abc123 ``` -The load balancer attempts to route to the specified worker. If the worker is unavailable, at capacity, or not found, the request falls through to normal least-load selection. This mode never fails a request due to affinity; it is best-effort. +The load balancer attempts to route to the specified worker. If the worker is unavailable, at capacity, or not found, the request falls through to normal worker selection. This mode never fails a request due to affinity; it is best-effort. ### Strict affinity ``` -X-Runpod-Worker-Id: strict wrk_abc123 +X-Runpod-Worker-Id: strict pod-abc123 ``` -The load balancer only routes to the specified worker. If the worker is at capacity, the request waits. If the worker is gone and not found in the active worker set, the load balancer returns a `404` with reason `affinity_worker_gone`. Use this mode when falling back to a different worker would produce incorrect results. +The load balancer only routes to the specified worker. If the worker is at capacity, the request waits up to ~5 minutes. If the worker does not free up within that time, the request returns `400` with reason `timed out waiting for worker`. If the worker is gone and not found in the active worker set, the load balancer returns a `404` with reason `affinity_worker_gone`. Use this mode when falling back to a different worker would produce incorrect results. ### Strict-resume affinity ``` -X-Runpod-Worker-Id: strict-resume wrk_abc123 +X-Runpod-Worker-Id: strict-resume pod-abc123 ``` -Same as strict, but if the worker pod has been scaled down, the load balancer calls `ResumePodById` to bring it back up before routing the request. The worker is scoped to the endpoint and cross-endpoint access is not possible. +Same as strict, but if the worker pod has been scaled down, the load balancer calls `ResumePodById` to bring it back up and retries while the request waits. If the resume cannot complete, it keeps retrying rather than failing immediately. The request only ends in `400` on timeout, or `404` if the pod is truly gone (terminated or redeployed under a new ID). Cross-endpoint access is not possible. -`strict-resume` keeps a worker running as long as requests keep coming in. Workers scale down after being idle (no requests) for a set period. If your  client sends a request before that idle period expires, the timer resets and  the worker stays up. This means a worker receiving regular traffic, with or  without strict-resume, will never scale down.  +`strict-resume` keeps a worker running as long as requests keep coming in. Workers scale down after being idle (no requests) for a set period. If your client sends a request before that idle period expires, the timer resets and the worker stays up. This means a worker receiving regular traffic, with or without strict-resume, will never scale down. - When your session is done, send your next request without the  `X-Runpod-Worker-Id` header so the worker is no longer pinned and can go idle. + When your session is done, send your next request without the `X-Runpod-Worker-Id` header so the worker is no longer pinned and can go idle. ## Worker ID on responses @@ -61,7 +61,7 @@ The load balancer always sets `X-Runpod-Worker-Id` on the response to reflect th ## Error reference -| Status | Reason | Description | -|--------|--------|-------------| -| `404` | `affinity_worker_gone` | Strict mode: the requested worker is not in the active worker set | -| `503` | `affinity_resume_failed` | Strict-resume mode: the worker pod could not be resumed | +| Status | Reason | Description | +| ------ | ---------------------- | ---------------------------------------------------------------------------------------------------- | +| `404` | `affinity_worker_gone` | Strict or strict-resume: the requested worker is not in the active worker set | +| `400` | `worker_timeout` | Strict or strict-resume: the pinned worker did not become available within the request timeout (~5 minutes) | From a7c52b28b104b6d98d9d188513e83b70b7ce17f7 Mon Sep 17 00:00:00 2001 From: lgunreddi Date: Fri, 17 Jul 2026 15:00:38 -0400 Subject: [PATCH 6/7] Update release-notes.mdx --- release-notes.mdx | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/release-notes.mdx b/release-notes.mdx index 89f4e5767..7aba0e321 100644 --- a/release-notes.mdx +++ b/release-notes.mdx @@ -17,6 +17,10 @@ rss: true

New Release [Scale Instant Clusters - BETA](/instant-clusters/scale-clusters)

You can now add pods to a running Instant Cluster to increase GPU capacity without recreating it. New pods join the cluster's private network automatically, matching the existing GPU type, template, and network storage. Currently in beta for eligible accounts. +**July 14, 2026** + +

New Release [Worker affinity for Serverless load balancer endpoints](/serverless/load-balancing/worker-affinity)

You can now pin follow-up requests to the same worker using the `X-Runpod-Worker-Id` response header. Send the header back on subsequent requests to choose from three modes: soft (prefer the worker, fall back to normal selection if unavailable), strict (route only to that worker, wait if at capacity), or strict-resume (same as strict, but automatically resumes a scaled-down worker). Useful for stateful workloads where session state is held in worker memory and re-routing would require reloading it. + **July 1, 2026**

Improvement [Updated Serverless endpoint creation flow](https://docs.runpod.io/serverless/endpoints/overview)

The endpoint creation flow now offers six deployment paths — Hello World, Hugging Face LLM, Docker, GitHub, Flash, and Hub (which replaces the previous "Ready-to-Deploy Repos" option). Each path walks you through the right setup for your use case. From 8a831e0bde21994d2f40213c1eeff758b9201cf1 Mon Sep 17 00:00:00 2001 From: lgunreddi Date: Fri, 17 Jul 2026 15:05:46 -0400 Subject: [PATCH 7/7] Update worker-affinity.mdx --- serverless/load-balancing/worker-affinity.mdx | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/serverless/load-balancing/worker-affinity.mdx b/serverless/load-balancing/worker-affinity.mdx index a86eb6464..45c7984ed 100644 --- a/serverless/load-balancing/worker-affinity.mdx +++ b/serverless/load-balancing/worker-affinity.mdx @@ -31,7 +31,9 @@ To pin a subsequent request to that worker, send the header back with one of the X-Runpod-Worker-Id: pod-abc123 ``` -The load balancer attempts to route to the specified worker. If the worker is unavailable, at capacity, or not found, the request falls through to normal worker selection. This mode never fails a request due to affinity; it is best-effort. +- Routes to the specified worker if available. +- Falls back to normal worker selection if the worker is unavailable, at capacity, or not found. +- Never fails a request due to affinity — this mode is best-effort. ### Strict affinity @@ -39,7 +41,11 @@ The load balancer attempts to route to the specified worker. If the worker is un X-Runpod-Worker-Id: strict pod-abc123 ``` -The load balancer only routes to the specified worker. If the worker is at capacity, the request waits up to ~5 minutes. If the worker does not free up within that time, the request returns `400` with reason `timed out waiting for worker`. If the worker is gone and not found in the active worker set, the load balancer returns a `404` with reason `affinity_worker_gone`. Use this mode when falling back to a different worker would produce incorrect results. +- Routes only to the specified worker. +- If the worker is at capacity, the request waits up to ~5 minutes. +- If the worker does not free up in time, returns `400` with reason `timed out waiting for worker`. +- If the worker is gone, returns `404` with reason `affinity_worker_gone`. +- Use this mode when falling back to a different worker would produce incorrect results. ### Strict-resume affinity @@ -47,7 +53,10 @@ The load balancer only routes to the specified worker. If the worker is at capac X-Runpod-Worker-Id: strict-resume pod-abc123 ``` -Same as strict, but if the worker pod has been scaled down, the load balancer calls `ResumePodById` to bring it back up and retries while the request waits. If the resume cannot complete, it keeps retrying rather than failing immediately. The request only ends in `400` on timeout, or `404` if the pod is truly gone (terminated or redeployed under a new ID). Cross-endpoint access is not possible. +- Same as strict, but also resumes a scaled-down worker before routing. +- If the resume cannot complete immediately, it keeps retrying while the request waits. +- Returns `400` on timeout, or `404` if the pod is truly gone (terminated or redeployed under a new ID). +- Cross-endpoint access is not possible. `strict-resume` keeps a worker running as long as requests keep coming in. Workers scale down after being idle (no requests) for a set period. If your client sends a request before that idle period expires, the timer resets and the worker stays up. This means a worker receiving regular traffic, with or without strict-resume, will never scale down.