Skip to content

feat(evals): add legend-state evals to async-state - #386

Open
mikolajadamowicz wants to merge 2 commits into
callstackincubator:mainfrom
mikolajadamowicz:mikolajadamowicz/legend-state
Open

mikolajadamowicz wants to merge 2 commits into
callstackincubator:mainfrom
mikolajadamowicz:mikolajadamowicz/legend-state

Conversation

@mikolajadamowicz

Copy link
Copy Markdown

Summary

Adds Legend State (@legendapp/state v3) as a new baseline library in the async-state category, with 5 focused evals covering its fine-grained observable model. Each eval ships a prompt, atomic requirements, a baseline app/App.tsx, and a reference implementation.

async-state grows from 13 → 18 evals; the suite total goes from 134 → 139.

Evals Added

# Eval Focus
15 rn-legend-state-observable-async-action-lifecycle observable store, explicit idle/loading/success/error action lifecycle, tracked reads via useValue/use$/observer
16 rn-legend-state-computed-derived-state-batch Computed functions inside the observable, batch for multi-field writes, Memo/Computed fine-grained rendering
17 rn-legend-state-persist-asyncstorage-hydration-gate syncObservable + AsyncStorage persist plugin, hydration gating on syncState(...).isPersistLoaded
18 rn-legend-state-reactive-two-way-input $TextInput two-way binding via $value, computed dirty flag, Show-driven save bar
19 rn-legend-state-async-observable-lazy-load Async observable activating lazily on first tracked read, loading/error derived from syncState instead of useEffect + useState

Category Documentation

evals/async-state/README.md gains a Legend State baseline entry, v3 doc references (including the v2 → v3 migration notes for persistObservable → syncObservable and useSelector → useValue), and six new traceability rules:

  • D19 — read observables through tracked contexts, not untracked .get() in render
  • D20 — model derived values as computed functions in the observable
  • D21 — batch multi-field writes to avoid intermediate renders and persistence churn
  • D22 — gate persisted UI on isPersistLoaded
  • D23 — two-way bind inputs with reactive $ components instead of `useState
  • D24 — model lazily-loaded server data as async observables with syncState

Policies Enforced

  • Leak policy — prompts describe the scenario and name the library, but never the graded target identifiers (batch, Memo, syncState, isPersistLoaded, $value, etc. are requirement-side only where applicable)
  • Guard policy — no baseline-passable requirements; each baseline app/App.useState/useEffect` version the prompt asks to replace
  • Deterministic data — network evals use the pinned dummyjson.com endpoint patterns and documented response shapes; evals 16 and 18 are fully local

Other Changes

  • testbench/package.json: adds @legendapp/state@^3.0.0-beta.48
  • README.md and paper/benchmark-methodology-whitepaper.tex: updated eval counts and the async-state category scope to include Legend State

- Adds observable async lifecycle, computed/batch, persist/hydration, and
  two-way binding evals covering @legendapp/state v3 patterns
- Updates async-state README, whitepaper eval counts, and adds the
  @legendapp/state dependency

feat(evals): add 4 legend-state evals to async-state

- Adds observable async lifecycle, computed/batch, persist/hydration, and
  two-way binding evals covering @legendapp/state v3 patterns
- Updates async-state README, whitepaper eval counts, and adds the
  @legendapp/state dependency
- Adds a new eval (lazy-loaded product screen) covering async observable
  data fetching with syncState-derived loading/error status, plus
  requirements.yaml grading it.
- Updates existing Legend State evals' prompts to specify v3 and removes
  unused React default imports flagged by the linter.
- Bumps eval counts (138→139, async-state 17→18) in README and the
  benchmark methodology paper to stay in sync.
@artus9033
artus9033 self-requested a review August 19, 2026 11:09

@artus9033 artus9033 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants