feat(workflows): share model fallbacks across LLM blocks

Author: icecrasher321

apps/docs/content/docs/workflows/blocks/agent.mdx

@@ -87,7 +87,7 @@ Some settings live under advanced, or appear only for models that support them:
 - **Reasoning effort / Thinking level.** For models with extended reasoning, how much the model thinks before answering. Higher is more thorough but slower and costs more tokens.
 - **Prompt caching.** For Anthropic Claude models, reuses the system prompt and tool definitions between runs instead of re-reading them every time. Cached input costs a tenth of the normal rate, but writing the cache costs 1.25x, so leave it off for one-off runs and turn it on when the same agent runs repeatedly. The cache covers a prefix only if it reaches 1,024 tokens (2,048 on Haiku) — below that Anthropic ignores it and nothing changes. Entries expire after five minutes of no use.
 - **API key.** Your key for the chosen provider. Hidden on hosted Sim, which supplies one.
-- **Fallback models.** An ordered list of up to five models to try when the request to the selected model fails, whether the provider is overloaded, rate-limited, or down. Sim tries the 2nd choice, then the 3rd, and so on, once each, and `<agent.model>` reports the model that answered. Hosted models need no setup. A model that needs its own key takes it from a workspace environment variable you pick on the row; a model on the same provider as the selected model reuses the block's key. Azure, Bedrock, and Vertex models can only be fallbacks for a selected model of the same family, since they use that model's credentials. The Auto model cannot be a fallback. A fallback runs with the selected model's settings where its provider accepts them: temperature and max output tokens are clamped to the fallback's limits, and when the fallback has a reasoning effort, thinking level, or verbosity setting that the selected model's value does not fit, the row shows that field so you can pick a value for it; leave it empty and the provider's default applies.
+- **Fallback models.** An ordered list of up to five models to try when the request to the selected model fails, whether the provider is overloaded, rate-limited, or down. Sim tries the 2nd choice, then the 3rd, and so on, once each, and `<agent.model>` reports the model that answered. On hosted Sim, hosted models use your workspace's BYOK or platform credentials; local and self-hosted installations may still require a key. A model that needs its own key takes it from a workspace environment variable you pick on the row; a model on the same provider as the selected model reuses the block's key. A stored row key stops applying when its key field is hidden. Providers that require family-specific credentials, such as Vertex, can only be fallbacks for a selected model of the same family. The Auto model cannot be a fallback. A fallback runs with the selected model's settings where its provider accepts them: temperature and max output tokens are clamped to the fallback's limits, and when the fallback has a reasoning effort, thinking level, or verbosity setting that the selected model's value does not fit, the row shows that field so you can pick a value for it; leave it empty and the provider's default applies.
 - **Retry on fail.** Retries the selected model after a failure, up to a maximum number of tries with a wait between them. When its tries run out, the fallback models are tried in order, once each, with no wait before the first of them. A fallback is never retried. A failure that happens after the model already called a tool runs that conversation again on the next try or the next model, so keep fallbacks and retry off for agents whose tools must not repeat.
 
 OpenAI and Gemini cache automatically at no extra cost and need no setting; their discount is already reflected in what you are charged.

apps/docs/content/docs/workflows/blocks/evaluator.mdx

@@ -32,6 +32,12 @@ The content to score. Usually an earlier output like `<agent.content>`. Structur
 
 The model that does the scoring, defaulting to `claude-sonnet-4-6`. Stronger reasoning models give more consistent scores. Type or pick any supported model. **Temperature** and a **System Prompt** are available under advanced, and on hosted Sim the API key is supplied for you.
 
+### Fallback models
+
+Under **Additional fields**, add up to five models to try in order when a model request fails. With **Retry on fail** enabled, the selected model exhausts its tries first; each fallback is then tried once. Every attempt uses the same content, metrics, and response schema. `<evaluator.model>` reports the model that answered.
+
+Hosted models on hosted Sim use workspace BYOK or platform credentials. On local or self-hosted installations, a fallback on another provider may need a secret selected on its row; same-provider fallbacks reuse the selected model's key. The picker shows supported tuning fields when the selected model's settings cannot be inherited. Auto cannot be a fallback.
+
 ## Outputs
 
 The Evaluator returns a number for each metric, read by the metric's lowercase name:

apps/docs/content/docs/workflows/blocks/router.mdx

@@ -35,6 +35,12 @@ Each route is a **title** and a **description** of when to choose it ("Route her
 
 The model that makes the decision, defaulting to `claude-sonnet-4-6`. Stronger reasoning models route more accurately; a faster, cheaper model is fine when the routes are clearly distinct. Type or pick any supported model, or a local one through Ollama or VLLM. On hosted Sim the API key is supplied for you.
 
+### Fallback models
+
+Under **Additional fields**, add up to five models to try in order when a model request fails. With **Retry on fail** enabled, the selected model exhausts its tries first; each fallback is then tried once. Every attempt uses the same context and route definitions. `<router.model>` reports the model that answered. This also works for existing legacy Router blocks.
+
+Hosted models on hosted Sim use workspace BYOK or platform credentials. On local or self-hosted installations, a fallback on another provider may need a secret selected on its row; same-provider fallbacks reuse the selected model's key. Auto cannot be a fallback. A completed `NO_MATCH` decision still takes the error path; it does not trigger another model request.
+
 ## Outputs
 
 | Output | What it is |

apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/components/model-fallback-list/model-fallback-list.test.tsx

@@ -1,8 +1,16 @@
 /**
- * @vitest-environment node
+ * @vitest-environment jsdom
  */
+import { act } from 'react'
+import { createRoot } from 'react-dom/client'
 import { renderToStaticMarkup } from 'react-dom/server'
-import { beforeEach, describe, expect, it, vi } from 'vitest'
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
+import {
+  getDeploymentShape,
+  resetDeploymentShape,
+  resolveDeploymentShape,
+  seedDeploymentShape,
+} from '@/lib/core/config/deployment-shape'
 
 const { subBlockValues, mockSetValue } = vi.hoisted(() => ({
   subBlockValues: {
@@ -22,7 +30,7 @@ vi.mock('next/navigation', () => ({
 }))
 
 vi.mock('@sim/emcn', () => ({
-  Button: ({
+  Chip: ({
     children,
     disabled,
     'aria-label': ariaLabel,
@@ -35,7 +43,7 @@ vi.mock('@sim/emcn', () => ({
       {children}
     </button>
   ),
-  Combobox: ({
+  ChipCombobox: ({
     options,
     value,
     placeholder,
@@ -52,6 +60,21 @@ vi.mock('@sim/emcn', () => ({
       ))}
     </div>
   ),
+  ChipDropdown: ({
+    options,
+    value,
+    placeholder,
+  }: {
+    options: Array<{ value: string; label: string }>
+    value?: string
+    placeholder?: string
+  }) => (
+    <div data-combobox={placeholder} data-value={value}>
+      {options.map((option) => (
+        <span key={option.value}>{option.label}</span>
+      ))}
+    </div>
+  ),
   Label: ({ children }: { children?: React.ReactNode }) => <span>{children}</span>,
   Tooltip: {
     Root: ({ children }: { children?: React.ReactNode }) => <>{children}</>,
@@ -102,6 +125,8 @@ vi.mock('@/stores/providers/store', () => ({
 }))
 
 vi.mock('@/blocks/utils', () => ({
+  shouldRequireApiKeyForModel: (model: string) =>
+    model.startsWith('openrouter/') || (model.startsWith('gpt') && !getDeploymentShape().hosted),
   getModelOptions: () => [
     { id: 'claude-sonnet-5', label: 'claude-sonnet-5' },
     { id: 'gpt-5', label: 'gpt-5' },
@@ -117,7 +142,6 @@ vi.mock('@/lib/workflows/blocks/fallback-models', async (importOriginal) => {
     ...actual,
     isViableFallbackModel: (model: string, primary: string) =>
       model !== 'sim-auto' && model !== primary,
-    fallbackRowNeedsApiKey: (model: string) => model.startsWith('openrouter/'),
     getFallbackTuningKnobsToShow: (model: string) => (model === 'gpt-5' ? ['reasoningEffort'] : []),
     getTuningOptionsForModel: (model: string, knob: string) =>
       model === 'gpt-5' && knob === 'reasoningEffort' ? ['auto', 'low', 'high'] : null,
@@ -126,6 +150,16 @@ vi.mock('@/lib/workflows/blocks/fallback-models', async (importOriginal) => {
 
 import { ModelFallbackList } from '@/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/components/model-fallback-list/model-fallback-list'
 
+beforeEach(() => {
+  vi.stubGlobal('IS_REACT_ACT_ENVIRONMENT', true)
+  seedDeploymentShape({ ...resolveDeploymentShape(), hosted: true })
+})
+
+afterEach(() => {
+  resetDeploymentShape()
+  vi.unstubAllGlobals()
+})
+
 function render(extra: Partial<React.ComponentProps<typeof ModelFallbackList>> = {}) {
   return renderToStaticMarkup(
     <ModelFallbackList blockId='block-1' subBlockId='fallbackModels' disabled={false} {...extra} />
@@ -186,6 +220,27 @@ describe('ModelFallbackList', () => {
     expect(html).toContain('Create Secret')
   })
 
+  it('updates key visibility when hosted context arrives after mount, without rewriting the rows', async () => {
+    seedDeploymentShape({ ...resolveDeploymentShape(), hosted: false })
+    subBlockValues.fallbackModels = [{ id: 'r1', model: 'gpt-5' }]
+    const container = document.createElement('div')
+    const root = createRoot(container)
+    try {
+      await act(async () => {
+        root.render(<ModelFallbackList blockId='block-1' subBlockId='fallbackModels' />)
+      })
+      expect(container.querySelector('[data-combobox="Select a secret"]')).not.toBeNull()
+
+      await act(async () => {
+        seedDeploymentShape({ ...resolveDeploymentShape(), hosted: true })
+      })
+      expect(container.querySelector('[data-combobox="Select a secret"]')).toBeNull()
+      expect(mockSetValue).not.toHaveBeenCalled()
+    } finally {
+      await act(async () => root.unmount())
+    }
+  })
+
   it('shows a tuning field only for the knobs the helper says need one', () => {
     subBlockValues.fallbackModels = [
       { id: 'r1', model: 'gpt-5', reasoningEffort: 'low' },

apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/components/model-fallback-list/model-fallback-list.tsx

@@ -1,10 +1,11 @@
 'use client'
 
 import { memo, useCallback, useEffect, useMemo, useRef } from 'react'
-import { Button, Combobox, type ComboboxOption, Label, Tooltip } from '@sim/emcn'
+import { Chip, ChipCombobox, ChipDropdown, type ComboboxOption, Label, Tooltip } from '@sim/emcn'
 import { ChevronDown, ChevronUp, Plus, Trash } from '@sim/emcn/icons'
 import { generateShortId } from '@sim/utils/id'
 import { useParams } from 'next/navigation'
+import { useDeploymentShape } from '@/lib/core/config/deployment-shape'
 import { writePendingCredentialCreateRequest } from '@/lib/credentials/client-state'
 import {
   addFallbackRow,
@@ -96,6 +97,8 @@ const FallbackRow = memo(function FallbackRow({
   onMove,
   onRemove,
 }: FallbackRowProps) {
+  /** Credential visibility follows the server-resolved shape, including late hydration. */
+  useDeploymentShape()
   const modelOptions = useMemo(
     (): ComboboxOption[] =>
       viableOptions.map((option) => ({
@@ -105,104 +108,90 @@ const FallbackRow = memo(function FallbackRow({
     [viableOptions, takenModels, row.model]
   )
 
-  const { needsApiKey, tuningFields } = useMemo(() => {
-    if (!row.model) return { needsApiKey: false, tuningFields: [] }
-    return {
-      needsApiKey: fallbackRowNeedsApiKey(row.model, primaryModel),
-      tuningFields: getFallbackTuningKnobsToShow(row.model, primaryModel, primaryTuning).map(
-        (knob) => ({
-          knob,
-          options: (getTuningOptionsForModel(row.model, knob) ?? []).map(
-            (value): ComboboxOption => ({ label: value, value })
-          ),
-        })
-      ),
-    }
-  }, [row.model, primaryModel, primaryTuning])
+  const needsApiKey = fallbackRowNeedsApiKey(row.model, primaryModel)
+  const tuningFields = getFallbackTuningKnobsToShow(row.model, primaryModel, primaryTuning).map(
+    (knob) => ({
+      knob,
+      options: (getTuningOptionsForModel(row.model, knob) ?? []).map((value) => ({
+        label: value,
+        value,
+      })),
+    })
+  )
 
   /** Only a reference is ever shown; anything else that reached the store reads as unset. */
   const apiKeyValue = isWholeEnvVarReference(row.apiKey) ? row.apiKey : ''
 
   return (
     <div
       data-fallback-row-id={row.id}
-      className='overflow-visible rounded-sm border border-[var(--border-1)]'
+      className='space-y-3 rounded-lg border border-[var(--border)] p-3'
     >
-      <div className='flex items-center justify-between rounded-t-[4px] border-[var(--border-1)] border-b bg-[var(--surface-4)] px-2.5 py-[5px]'>
-        <span className='text-[var(--text-tertiary)] text-sm'>{ordinalChoiceLabel(index)}</span>
-        <div className='flex items-center gap-2'>
+      <div className='flex items-center justify-between'>
+        <span className='text-[var(--text-muted)] text-small'>{ordinalChoiceLabel(index)}</span>
+        <div className='flex items-center'>
           {canMove && (
             <>
               <Tooltip.Root>
                 <Tooltip.Trigger asChild>
-                  <Button
-                    variant='ghost'
+                  <Chip
+                    leftIcon={ChevronUp}
                     onClick={() => onMove(row.id, -1)}
                     disabled={readOnly || index === 0}
-                    className='h-auto p-0'
                     aria-label='Move up'
-                  >
-                    <ChevronUp className='size-[14px]' />
-                  </Button>
+                  />
                 </Tooltip.Trigger>
                 <Tooltip.Content>Move up</Tooltip.Content>
               </Tooltip.Root>
               <Tooltip.Root>
                 <Tooltip.Trigger asChild>
-                  <Button
-                    variant='ghost'
+                  <Chip
+                    leftIcon={ChevronDown}
                     onClick={() => onMove(row.id, 1)}
                     disabled={readOnly || isLast}
-                    className='h-auto p-0'
                     aria-label='Move down'
-                  >
-                    <ChevronDown className='size-[14px]' />
-                  </Button>
+                  />
                 </Tooltip.Trigger>
                 <Tooltip.Content>Move down</Tooltip.Content>
               </Tooltip.Root>
             </>
           )}
           <Tooltip.Root>
             <Tooltip.Trigger asChild>
-              <Button
-                variant='ghost'
+              <Chip
+                leftIcon={Trash}
                 onClick={() => onRemove(row.id)}
                 disabled={readOnly}
-                className='h-auto p-0 text-[var(--text-error)] hover-hover:text-[var(--text-error)]'
                 aria-label='Remove fallback model'
-              >
-                <Trash className='size-[14px]' />
-              </Button>
+              />
             </Tooltip.Trigger>
             <Tooltip.Content>Remove</Tooltip.Content>
           </Tooltip.Root>
         </div>
       </div>
 
-      <div className='flex flex-col gap-2 px-2.5 pt-1.5 pb-2.5'>
-        <div className='flex flex-col gap-1.5'>
-          <Label>Model</Label>
-          <Combobox
-            options={modelOptions}
-            value={row.model}
-            onChange={(model) => onChangeModel(row.id, model)}
-            placeholder='Select a model'
-            disabled={readOnly}
-            searchable
-            searchPlaceholder='Search models...'
-            maxHeight={240}
-            emptyMessage='No models available'
-          />
-        </div>
+      <div className='flex flex-col gap-3'>
+        <ChipCombobox
+          options={modelOptions}
+          value={row.model}
+          onChange={(model) => onChangeModel(row.id, model)}
+          placeholder='Select a model'
+          aria-label={`${ordinalChoiceLabel(index)} model`}
+          disabled={readOnly}
+          searchable
+          searchPlaceholder='Search models...'
+          maxHeight={240}
+          emptyMessage='No models available'
+        />
         {needsApiKey && (
           <div className='flex flex-col gap-1.5'>
             <Label>API key</Label>
-            <Combobox
+            <ChipCombobox
               options={envVarOptions}
               value={apiKeyValue}
               onChange={(apiKey) => onChangeApiKey(row.id, apiKey)}
               placeholder='Select a secret'
+              aria-label={`${ordinalChoiceLabel(index)} API key`}
               disabled={readOnly}
               searchable
               searchPlaceholder='Search secrets...'
@@ -214,12 +203,14 @@ const FallbackRow = memo(function FallbackRow({
         {tuningFields.map(({ knob, options }) => (
           <div key={knob} className='flex flex-col gap-1.5'>
             <Label>{FALLBACK_TUNING_LABELS[knob]}</Label>
-            <Combobox
+            <ChipDropdown
               options={options}
               value={row[knob] ?? options[0]?.value ?? ''}
               onChange={(value) => onChangeTuning(row.id, knob, value)}
               placeholder={`Select ${FALLBACK_TUNING_LABELS[knob].toLowerCase()}`}
+              aria-label={`${ordinalChoiceLabel(index)} ${FALLBACK_TUNING_LABELS[knob].toLowerCase()}`}
               disabled={readOnly}
+              className='w-full'
             />
           </div>
         ))}
@@ -250,6 +241,7 @@ export function ModelFallbackList({
   const workspaceId = typeof params?.workspaceId === 'string' ? params.workspaceId : ''
   const { navigateToSettings } = useSettingsNavigation()
   const { isModelUsable } = usePermissionConfig()
+  const deploymentShape = useDeploymentShape()
   const providers = useProvidersStore((state) => state.providers)
   const [storeValue, setStoreValue] = useSubBlockValue<FallbackModelEntry[]>(blockId, subBlockId)
   const [primaryModelValue] = useSubBlockValue<string>(blockId, 'model')
@@ -311,7 +303,7 @@ export function ModelFallbackList({
           value: option.id,
           ...(option.icon ? { icon: option.icon } : {}),
         })),
-    [primaryModel, isModelUsable, providers]
+    [primaryModel, isModelUsable, providers, deploymentShape]
   )
 
   const takenModels = useMemo(() => new Set(rows.map((row) => row.model).filter(Boolean)), [rows])
@@ -417,15 +409,15 @@ export function ModelFallbackList({
         />
       ))}
       {!readOnly && (
-        <Button
-          variant='ghost'
+        <Chip
+          variant='outline'
+          leftIcon={Plus}
+          fullWidth
           onClick={handleAdd}
           disabled={rows.length >= MAX_FALLBACK_MODELS}
-          className='h-7 w-full justify-start gap-1.5 border border-[var(--border-1)] border-dashed text-[var(--text-muted)] text-small'
         >
-          <Plus className='size-[14px]' />
           Add fallback model
-        </Button>
+        </Chip>
       )}
     </div>
   )