Skip to content

fix(workflows): skip dynamically bound selectors in id validation - #7799

Open
mzxchandra wants to merge 5 commits into
stagingfrom
fix/selector-lint-reference-guard
Open

fix(workflows): skip dynamically bound selectors in id validation#7799
mzxchandra wants to merge 5 commits into
stagingfrom
fix/selector-lint-reference-guard

Conversation

@mzxchandra

Copy link
Copy Markdown
Contributor

Summary

Tier-2 selector validation (collectSelectorFieldsvalidateSelectorIds) is a static id-existence check against the workspace. It cannot evaluate a value whose id only arrives at execution time, so a <block.output> or {{ENV_VAR}} binding written into a selector field was reported as a resource that does not exist — on every graph write.

Two commits:

fix(workflows): split multi-select values without tearing references
A reference token may legitimately contain a comma (<start.pick(a,b)>), and its fragments read as plain literals once split, so the existing .split(',') turned one dynamic value into several bogus ids. Adds splitOutsideReferences, which treats only commas outside every reference token as separators.

fix(workflows): skip dynamically bound selectors in id validation
collectSelectorFields now skips dynamically bound values via the existing containsReference, and splits with splitOutsideReferences. Filtering is per entry, not on the whole string: a multi-select can mix literal ids with dynamic ones, and testing <a.b>,kb_real,<c.d> as a whole would drop kb_real along with the references.

Behaviour is unchanged for literal ids — one that does not resolve is still reported.

Performance

Token spans are marked once into a lookup rather than rescanned per comma. A per-comma tokens.some() is O(commas × tokens):

value size per-comma scan marked once
30KB 42ms 3ms
120KB 425ms 5ms
240KB 2512ms 12ms

This runs synchronously on the graph-write paths, which admit bodies up to MAX_IMPORT_BODY_BYTES (10MB), so the quadratic form was worth avoiding. A regression test asserts the 240KB case stays under 1s.

Test Coverage

lib/workflows/sanitization/references.ts  (NEW splitOutsideReferences)
├── splits on separator commas ............................. ★★★
├── trims entries, drops empties ........................... ★★★
├── comma inside <workflow.reference> is not a separator .... ★★★
├── comma inside {{ENV,VAR}} is not a separator ............. ★★★
├── stays linear on a 240KB value .......................... ★★★
└── nested env-var inside a reference (known limitation) .... ★★★

lib/workflows/editing/validation.ts  (collectSelectorFields)
├── scalar reference / env-var / partial template -> skip ... ★★★
├── scalar literal -> still validated ...................... ★★★
├── multi-select: some references -> filter, keep the rest .. ★★★
├── multi-select: all references -> skip, no empty DB call .. ★★★
├── multi-select opening AND closing with a reference ....... ★★★
├── native array value (second entry into the filter) ....... ★★★
├── non-string entries ..................................... ★★★
├── unbalanced delimiters are not references -> validated ... ★★★
├── runs after the canonical active-member check ........... ★★
├── consumer: collectUnresolvedReferences ................... ★★★
├── consumer: validateWorkflowSelectorIds ................... ★★★
└── against the real block registry (no fixture drift) ...... ★★

Legend: ★★★ behaviour + edge + error · ★★ happy path

245 tests pass across lib/workflows/editing/ and lib/workflows/sanitization/. Full apps/sim suite green.

Known limitations (characterized by tests, not fixed here)

  • A reference that nests an env-var placeholder still splits. findWorkflowReferenceTokens collects {{...}} first and suppresses any workflow token overlapping one, so the outer <...> span is never recorded and its commas are unprotected: <start.body.pick({{A}},b)>['<start.body.pick({{A}}', 'b)>']. This is byte-identical to the .split(',') it replaces — not a regression — and the root cause is the overlap rule in the shared @sim/utils/workflow-references tokenizer, so fixing it belongs there.
  • A near-miss reference still tears. Anything failing isLikelyWorkflowReferenceSegment (<a.b+c,d>, unbalanced <a.b,kb_x) gets no span and splits. Loud rather than silent.
  • Follow-up worth considering: when every selector in a graph is reference-bound, the lint now emits no findings and no note, which reads the same as "checked, all clean". A "N dynamic bindings not statically checked" note would fix that, but buildWorkflowLintReport iterates its collectors as a homogeneous pair, so surfacing a skip count means changing both signatures and that loop. Left out of this PR deliberately.

Review

Reviewed against the repo checklist plus testing, maintainability, and security passes, and two independent adversarial passes.

  • Security: no findings. The lint is advisory (lint-report.ts: "Findings never block a write"), and credentials are re-authorized at execution time by authorizeCredentialUseForAuth independently of anything the editor recorded, so a more lenient editor check cannot grant access.
  • Fixed during review: the first draft reimplemented containsReference with a substring test (which wrongly matched value <limit && value>max), used an unanchored whole-value check that let <start.a>,kb_real,<start.b> skip kb_real, and scanned tokens per comma.

Test plan

  • bun run test in apps/sim — full suite green
  • bun run type-check clean
  • biome check clean on all changed files
  • bun run check:api-validation passes
  • End-to-end against a local dev server: the three reference forms drop from one unresolved-reference finding each to zero; a literal id that does not resolve still reports one

A `<block.path>` or `{{ENV_VAR}}` token may legitimately contain a comma
(`<start.pick(a,b)>`), and its fragments read as plain literals once split, so a
naive `.split(',')` turns one dynamically bound value into several bogus ids.

Adds `splitOutsideReferences`, which treats only commas outside every reference
token as separators. Token spans are marked once into a lookup rather than
rescanned per comma: a per-comma `tokens.some()` is O(commas x tokens) and took
~2.5s on a 240KB value of repeated `{{A}},`, which is reachable on the 10MB
graph-write paths.

Known limitation, unchanged from the `.split(',')` this replaces and covered by a
characterization test: the tokenizer suppresses a workflow span that overlaps an
environment token, so `<start.body.pick({{A}},b)>` still splits.
Tier-2 selector validation is a static id-existence check against the workspace,
so it cannot evaluate a value whose id only arrives at execution time. A
`<block.output>` or `{{ENV_VAR}}` binding written into a selector field was
therefore reported as a resource that does not exist, on every graph write.

`collectSelectorFields` now skips those values via the existing
`containsReference`, and splits multi-select values with
`splitOutsideReferences` so a reference containing a comma is not torn into
fragments that each get validated as an id.

Filtering is per entry rather than on the whole string: a multi-select can mix
literal ids with dynamic ones, and testing `<a.b>,kb_real,<c.d>` as a whole would
drop `kb_real` along with the references.

Verified end to end against a local dev server: the three reference forms drop
from one unresolved-reference finding each to zero, while a literal id that does
not resolve still reports one.
@vercel

vercel Bot commented Sep 13, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Sep 13, 2026 2:03am UTC

Request Review

@greptile-apps

greptile-apps Bot commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge; the previous findings are resolved and no new actionable defects remain.

Summary

  • Adds reference-aware comma splitting that protects workflow and environment-reference regions, including overlapping nested placeholders.
  • Filters dynamic selector entries individually while continuing to validate literal entries in mixed selections.
  • Replaces the oversized-list fallback with linear splitting while retaining a per-entry classification limit.
  • Adds unit and integration coverage for reference forms, mixed lists, oversized inputs, malformed delimiters, and the real block registry.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Selector field value] --> B{Comma-separated string?}
    B -- Yes --> C[Split only on commas outside reference regions]
    B -- No --> D[Keep scalar or native array]
    C --> E[Classify each entry]
    D --> E
    E --> F{Oversized or dynamically bound?}
    F -- Yes --> G[Skip static ID validation]
    F -- No --> H[Validate literal selector ID]
    H --> I[Report unresolved literal resources]
Loading

Reviews (4) · Last reviewed commit: "fix(workflows): split references without..."

Comment thread apps/sim/lib/workflows/sanitization/references.ts Outdated
Comment thread apps/sim/lib/workflows/editing/selector-reference-guard.test.ts Outdated
@mzxchandra

Copy link
Copy Markdown
Contributor Author

@cubic review

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

@cubic review

@mzxchandra I have started the AI code review. It will take a few minutes to complete.

Review round 1.

`findWorkflowReferenceTokens` is contractually non-overlapping, so for
`<a.pick({{B}},c)>` it reports only the inner `{{B}}` and discards the outer
candidate. That is correct for a tokenizer and wrong for a splitter, which needs
the union of protected regions rather than a disjoint set, so the comma was
unprotected and `c)>` was validated as a literal id.

Adds a candidate pass built from the tokenizer's own exported predicates, leaving
the shared package's non-overlapping contract untouched. It runs only when an
environment token is present, since overlap with one is the only reason a
workflow candidate is dropped.

Also caps the value length before tokenizing. Reference detection parses the
whole string and the tokenizer is superlinear in candidate count (795ms for a
240KB value of repeated `<a.b>,`), which this change newly puts on a write path
that admits megabytes. Past the cap the field is skipped rather than parsed; the
lint is advisory, so declining to check is the safe direction.

Adds `as const` to the test context object per the repo's TypeScript conventions.
@mzxchandra

Copy link
Copy Markdown
Contributor Author

@greptile

@mzxchandra

Copy link
Copy Markdown
Contributor Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@mzxchandra I have started the AI code review. It will take a few minutes to complete.

Comment thread apps/sim/lib/workflows/editing/validation.ts Outdated
…tor value

Review round 2.

The length cap skipped the whole field, so an oversized list of plain literal ids
lost validation it previously had. Literals never needed tokenization, so the cap
was broader than the cost it was there to bound.

It now gives up only the reference-aware split: past the cap the value is split
plainly, and its entries are classified and validated as usual, since they are
short enough that the tokenizer's per-candidate cost does not apply (1MB of
literal ids across 30000 entries measures ~9ms). Only an individual entry past
the cap is skipped, where there is no cheap way to tell a literal from a dynamic
binding.
@mzxchandra

Copy link
Copy Markdown
Contributor Author

@greptile

Comment thread apps/sim/lib/workflows/editing/validation.ts
Review round 3.

An oversized value fell back to plain splitting, which tore a comma-bearing
reference into fragments that were then validated as literal ids. The fallback
existed to avoid `findWorkflowReferenceTokens`, which is superlinear in candidate
count.

That pass was never needed here. It returns contractually NON-overlapping tokens,
and the O(tokens^2) overlap check is the cost of producing that partition. A
splitter only needs to know whether an index sits inside SOME reference, so
scanning environment placeholders and `<...>` candidates independently gives the
union directly - cheaper and more accurate, since nothing is suppressed.

Splitting is now linear and the size fallback is gone, so a comma-bearing
reference survives at any length:

  240KB of `<a.b>,`          682ms -> 15ms
  240KB of `<a.p({{X}},y)>,` 177ms -> 12ms
  1MB of literal ids                   1ms

The length cap now applies to a single ENTRY rather than the whole field, which
is all it was ever needed for: classifying one entry tokenizes it, and there is
no cheap way to tell a literal from a dynamic binding past that size.

Exports `ENV_REFERENCE_PATTERN` from `@sim/utils` rather than duplicating the
pattern, so the two stay in step.
@mzxchandra

Copy link
Copy Markdown
Contributor Author

@greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant