Deploying to gh-pages from @ dstackai/dstack@fb8282e 🚀

Author: peterschmidt85

assets/images/social/blog/presets.png

@@ -5057,7 +5057,7 @@ <h1 id="changelog">Changelog<a class="headerlink" href="#changelog" title="Perma
   <div class="md-post__content md-typeset">
     <h2 id="presets-a-toolkit-for-agent-based-inference-optimization"><a class="toclink" href="../presets/">Presets: a toolkit for agent-based inference optimization</a></h2>
 <p>Optimizing model inference is agent work now. Every inference provider does it inside its own process, on its own serving stack, with its own harness around the optimization agent. Despite the progress in open-source serving frameworks, what gets published is a benchmark, often without the workload, the concurrency, and the hardware behind it. The optimized deployment itself stays tied to the stack that produced it.</p>
-<p>Today we're introducing a preview of presets: an open-source toolkit that streamlines inference optimization with agents, and a portable preset format that can be deployed to any cloud, Kubernetes cluster, or bare-metal fleet.</p>
+<p>Today we're introducing a preview of presets: an open-source toolkit that streamlines inference optimization with agents, and a portable preset format.</p>
 <p><img src="https://dstack.ai/static-assets/static-assets/images/dstack-presets.png" width="650" /></p>
 
 

blog/changelog/index.html

@@ -5057,7 +5057,7 @@ <h1 id="blog">Blog<a class="headerlink" href="#blog" title="Permanent link">&par
   <div class="md-post__content md-typeset">
     <h2 id="presets-a-toolkit-for-agent-based-inference-optimization"><a class="toclink" href="presets/">Presets: a toolkit for agent-based inference optimization</a></h2>
 <p>Optimizing model inference is agent work now. Every inference provider does it inside its own process, on its own serving stack, with its own harness around the optimization agent. Despite the progress in open-source serving frameworks, what gets published is a benchmark, often without the workload, the concurrency, and the hardware behind it. The optimized deployment itself stays tied to the stack that produced it.</p>
-<p>Today we're introducing a preview of presets: an open-source toolkit that streamlines inference optimization with agents, and a portable preset format that can be deployed to any cloud, Kubernetes cluster, or bare-metal fleet.</p>
+<p>Today we're introducing a preview of presets: an open-source toolkit that streamlines inference optimization with agents, and a portable preset format.</p>
 <p><img src="https://dstack.ai/static-assets/static-assets/images/dstack-presets.png" width="650" /></p>
 
 

blog/index.html

@@ -1,7 +1,7 @@
 ---
 title: "Presets: a toolkit for agent-based inference optimization"
 date: 2026-08-20
-description: "A preview of presets: a toolkit that streamlines agent-based inference optimization, and a portable format that deploys the result to any cloud or datacenter."
+description: "A preview of presets: an open-source toolkit that streamlines inference optimization with agents, and a portable preset format."
 slug: presets
 image: https://dstack.ai/static-assets/static-assets/images/dstack-presets.png
 categories:
@@ -12,15 +12,13 @@ categories:
 
 Optimizing model inference is agent work now. Every inference provider does it inside its own process, on its own serving stack, with its own harness around the optimization agent. Despite the progress in open-source serving frameworks, what gets published is a benchmark, often without the workload, the concurrency, and the hardware behind it. The optimized deployment itself stays tied to the stack that produced it.
 
-Today we're introducing a preview of presets: an open-source toolkit that streamlines inference optimization with agents, and a portable preset format that can be deployed to any cloud, Kubernetes cluster, or bare-metal fleet.
+Today we're introducing a preview of presets: an open-source toolkit that streamlines inference optimization with agents, and a portable preset format.
 
 <img src="https://dstack.ai/static-assets/static-assets/images/dstack-presets.png" width="650" />
 
 <!-- more -->
 
-Despite the progress in open-source serving frameworks, there is still no way to get optimization and deployment that delivers portable performance. That work happens inside each inference provider's own proprietary stack, with its own harness around the agent. This has to change.
-
-Deploying Kimi K3 should be as simple as deploying a Docker image, on any silicon and in any datacenter. Taking a model someone has already optimized and running it on your own hardware should not mean rebuilding the stack or repeating the optimization, whichever silicon you have.
+Deploying Kimi K3 to any cloud, Kubernetes cluster, or bare-metal fleet should be as simple as deploying a Docker image. Taking a model someone has already optimized and running it on your own hardware should not mean rebuilding the stack or repeating the optimization, whichever silicon you have.
 
 ## Toolkit and portable format
 

blog/posts/presets.md

@@ -6,7 +6,7 @@
       <meta charset="utf-8">
       <meta name="viewport" content="width=device-width,initial-scale=1">
 
-        <meta name="description" content="A preview of presets: a toolkit that streamlines agent-based inference optimization, and a portable format that deploys the result to any cloud or datacenter.">
+        <meta name="description" content="A preview of presets: an open-source toolkit that streamlines inference optimization with agents, and a portable preset format.">
 
 
         <meta name="author" content="dstack GmbH">
@@ -140,7 +140,7 @@
 
 <meta property="og:type" content="website" />
 <meta property="og:title" content="Presets: a toolkit for agent-based inference optimization - dstack" />
-<meta property="og:description" content="A preview of presets: a toolkit that streamlines agent-based inference optimization, and a portable format that deploys the result to any cloud or datacenter." />
+<meta property="og:description" content="A preview of presets: an open-source toolkit that streamlines inference optimization with agents, and a portable preset format." />
 <meta property="og:image" content="
   https://dstack.ai/static-assets/static-assets/images/dstack-presets.png" />
 <meta property="og:image:type" content="image/png" />
@@ -149,7 +149,7 @@
 <meta property="og:url" content="https://dstack.ai/blog/presets/" />
 <meta property="twitter:card" content="summary_large_image" />
 <meta property="twitter.title" content="Presets: a toolkit for agent-based inference optimization - dstack" />
-<meta property="twitter:description" content="A preview of presets: a toolkit that streamlines agent-based inference optimization, and a portable format that deploys the result to any cloud or datacenter." />
+<meta property="twitter:description" content="A preview of presets: an open-source toolkit that streamlines inference optimization with agents, and a portable preset format." />
 <meta property="twitter:image" content="
   https://dstack.ai/static-assets/static-assets/images/dstack-presets.png" />
 </head>
@@ -4917,12 +4917,11 @@ <h2>Need help?</h2>
 
 <h1 id="presets-a-toolkit-for-agent-based-inference-optimization">Presets: a toolkit for agent-based inference optimization<a class="headerlink" href="#presets-a-toolkit-for-agent-based-inference-optimization" title="Permanent link">&para;</a></h1>
 <p>Optimizing model inference is agent work now. Every inference provider does it inside its own process, on its own serving stack, with its own harness around the optimization agent. Despite the progress in open-source serving frameworks, what gets published is a benchmark, often without the workload, the concurrency, and the hardware behind it. The optimized deployment itself stays tied to the stack that produced it.</p>
-<p>Today we're introducing a preview of presets: an open-source toolkit that streamlines inference optimization with agents, and a portable preset format that can be deployed to any cloud, Kubernetes cluster, or bare-metal fleet.</p>
+<p>Today we're introducing a preview of presets: an open-source toolkit that streamlines inference optimization with agents, and a portable preset format.</p>
 <p><img src="https://dstack.ai/static-assets/static-assets/images/dstack-presets.png" width="650" /></p>
 <!-- more -->
 
-<p>Despite the progress in open-source serving frameworks, there is still no way to get optimization and deployment that delivers portable performance. That work happens inside each inference provider's own proprietary stack, with its own harness around the agent. This has to change.</p>
-<p>Deploying Kimi K3 should be as simple as deploying a Docker image, on any silicon and in any datacenter. Taking a model someone has already optimized and running it on your own hardware should not mean rebuilding the stack or repeating the optimization, whichever silicon you have.</p>
+<p>Deploying Kimi K3 to any cloud, Kubernetes cluster, or bare-metal fleet should be as simple as deploying a Docker image. Taking a model someone has already optimized and running it on your own hardware should not mean rebuilding the stack or repeating the optimization, whichever silicon you have.</p>
 <h2 id="toolkit-and-portable-format">Toolkit and portable format<a class="headerlink" href="#toolkit-and-portable-format" title="Permanent link">&para;</a></h2>
 <p>Think of Docker images. Docker gives you a toolkit for building an image, and a format that deploys to any datacenter or cloud reproducibly.</p>
 <p><code>dstack</code> introduces a similar concept applied to building optimized inference and its deployment. Presets offer two things: a toolkit that streamlines the optimization itself using agents, and a portable format that deploys the final preset to any cloud, Kubernetes cluster, or bare-metal fleet.</p>

blog/presets/index.html

@@ -6009,7 +6009,8 @@ <h3 id="idle-duration">Idle duration<a class="headerlink" href="#idle-duration"
 the shorter of the fleet's and run's <code>idle_duration</code> is used.
 If the run reuses an existing fleet instance, only the fleet's
 <a href="../../reference/dstack.yml/fleet/#idle_duration"><code>idle_duration</code></a> applies.</p>
-<p>If an instance remains <code>idle</code>, it is automatically terminated after <code>idle_duration</code>.</p>
+<p>If an instance remains <code>idle</code>, it is automatically terminated after <code>idle_duration</code>,
+unless the fleet is already at <code>nodes.min</code>.</p>
 <blockquote>
 <p>Not applied for container-based backends (Kubernetes, Vast.ai, Runpod).</p>
 </blockquote>

docs/concepts/dev-environments/index.html

@@ -318,20 +318,22 @@ Backend fleets allow you to set `backends` to specify which backends are allowed
 
 ### Idle duration
 
-By default, instances of a backend fleet stay `idle` for 3 days and can be reused within that time.
-If an instance is not reused within this period, it is automatically terminated.
+Instances of a backend fleet that aren't running any jobs are `idle` and can be reused by new runs.
+`dstack` terminates idle instances above [`nodes.min`](#nodes) after
+[`idle_duration`](../reference/dstack.yml/fleet.md#idle_duration), which defaults to 3 days.
 
-To change the default idle duration, set
-[`idle_duration`](../reference/dstack.yml/fleet.md#idle_duration) in the fleet configuration (e.g., `0s`, `1m`, or `off` for
-unlimited).
+For `idle_duration` to have an effect, `nodes` must be a range. Set `min` to `0` and `target` to
+the number of instances to pre-provision:
 
 <div editor-title="fleet.dstack.yml">
 
 ```yaml
 type: fleet
 name: my-fleet
 
-nodes: 2
+nodes:
+  min: 0
+  target: 2
 
 # Terminate instances idle for more than 1 hour
 idle_duration: 1h
@@ -342,6 +344,16 @@ resources:
 
 </div>
 
+`dstack apply` provisions `target` instances, and every instance that stays idle for
+`idle_duration` is terminated until the fleet is down to `min`. Set `idle_duration` to `0s` to
+terminate instances as soon as they become idle, or to `off` to never terminate them.
+
+!!! info "Fixed number of nodes"
+    Instances are never terminated on idle duration if the fleet is already at `nodes.min`, since
+    `dstack` would immediately provision new ones to maintain `min`. A fleet with a fixed size,
+    e.g. `nodes: 2`, has `min` equal to `2`, so it keeps its instances regardless of
+    `idle_duration`.
+
 ### Blocks
 
 By default, a job uses the entire instance—e.g., all 8 GPUs. To allow multiple jobs on the same instance, set the `blocks` property to divide the instance. Each job can then use one or more blocks, up to the full instance.

docs/concepts/fleets.md

@@ -5599,17 +5599,19 @@ <h3 id="spot-policy">Spot policy<a class="headerlink" href="#spot-policy" title=
 <h3 id="backends_1">Backends<a class="headerlink" href="#backends_1" title="Permanent link">&para;</a></h3>
 <p>Backend fleets allow you to set <code>backends</code> to specify which backends are allowed to be used.</p>
 <h3 id="idle-duration">Idle duration<a class="headerlink" href="#idle-duration" title="Permanent link">&para;</a></h3>
-<p>By default, instances of a backend fleet stay <code>idle</code> for 3 days and can be reused within that time.
-If an instance is not reused within this period, it is automatically terminated.</p>
-<p>To change the default idle duration, set
-<a href="../../reference/dstack.yml/fleet/#idle_duration"><code>idle_duration</code></a> in the fleet configuration (e.g., <code>0s</code>, <code>1m</code>, or <code>off</code> for
-unlimited).</p>
+<p>Instances of a backend fleet that aren't running any jobs are <code>idle</code> and can be reused by new runs.
+<code>dstack</code> terminates idle instances above <a href="#nodes"><code>nodes.min</code></a> after
+<a href="../../reference/dstack.yml/fleet/#idle_duration"><code>idle_duration</code></a>, which defaults to 3 days.</p>
+<p>For <code>idle_duration</code> to have an effect, <code>nodes</code> must be a range. Set <code>min</code> to <code>0</code> and <code>target</code> to
+the number of instances to pre-provision:</p>
 <div editor-title="fleet.dstack.yml">
 
 <div class="highlight"><pre><span></span><code><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">fleet</span>
 <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">my-fleet</span>
 
-<span class="nt">nodes</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">2</span>
+<span class="nt">nodes</span><span class="p">:</span>
+<span class="w">  </span><span class="nt">min</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">0</span>
+<span class="w">  </span><span class="nt">target</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">2</span>
 
 <span class="c1"># Terminate instances idle for more than 1 hour</span>
 <span class="nt">idle_duration</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">1h</span>
@@ -5620,6 +5622,16 @@ <h3 id="idle-duration">Idle duration<a class="headerlink" href="#idle-duration"
 
 </div>
 
+<p><code>dstack apply</code> provisions <code>target</code> instances, and every instance that stays idle for
+<code>idle_duration</code> is terminated until the fleet is down to <code>min</code>. Set <code>idle_duration</code> to <code>0s</code> to
+terminate instances as soon as they become idle, or to <code>off</code> to never terminate them.</p>
+<div class="admonition info">
+<p class="admonition-title">Fixed number of nodes</p>
+<p>Instances are never terminated on idle duration if the fleet is already at <code>nodes.min</code>, since
+<code>dstack</code> would immediately provision new ones to maintain <code>min</code>. A fleet with a fixed size,
+e.g. <code>nodes: 2</code>, has <code>min</code> equal to <code>2</code>, so it keeps its instances regardless of
+<code>idle_duration</code>.</p>
+</div>
 <h3 id="blocks">Blocks<a class="headerlink" href="#blocks" title="Permanent link">&para;</a></h3>
 <p>By default, a job uses the entire instance—e.g., all 8 GPUs. To allow multiple jobs on the same instance, set the <code>blocks</code> property to divide the instance. Each job can then use one or more blocks, up to the full instance.</p>
 <div class="tabbed-set tabbed-alternate" data-tabs="5:2"><input checked="checked" id="backend-fleets_2" name="__tabbed_5" type="radio" /><input id="ssh-fleets_2" name="__tabbed_5" type="radio" /><div class="tabbed-labels"><label for="backend-fleets_2">Backend fleets</label><label for="ssh-fleets_2">SSH fleets</label></div>

docs/concepts/fleets/index.html

@@ -6946,7 +6946,8 @@ <h3 id="idle-duration">Idle duration<a class="headerlink" href="#idle-duration"
 the shorter of the fleet's and run's <code>idle_duration</code> is used.
 If the run reuses an existing fleet instance, only the fleet's
 <a href="../../reference/dstack.yml/fleet/#idle_duration"><code>idle_duration</code></a> applies.</p>
-<p>If an instance remains <code>idle</code>, it is automatically terminated after <code>idle_duration</code>.</p>
+<p>If an instance remains <code>idle</code>, it is automatically terminated after <code>idle_duration</code>,
+unless the fleet is already at <code>nodes.min</code>.</p>
 <blockquote>
 <p>Not applied for container-based backends (Kubernetes, Vast.ai, Runpod).</p>
 </blockquote>

docs/concepts/services/index.html

@@ -36,6 +36,7 @@ the shorter of the fleet's and run's `idle_duration` is used.
 If the run reuses an existing fleet instance, only the fleet's
 [`idle_duration`](../reference/dstack.yml/fleet.md#idle_duration) applies.
 
-If an instance remains `idle`, it is automatically terminated after `idle_duration`.
+If an instance remains `idle`, it is automatically terminated after `idle_duration`,
+unless the fleet is already at `nodes.min`.
 
 > Not applied for container-based backends (Kubernetes, Vast.ai, Runpod).