Skip to content

test(eval): record eval run at f686823 - #300

Merged
github-actions[bot] merged 1 commit into
mainfrom
evals/run-34814616967-1
Sep 14, 2026
Merged

test(eval): record eval run at f686823#300
github-actions[bot] merged 1 commit into
mainfrom
evals/run-34814616967-1

Conversation

@github-actions

@github-actions github-actions Bot commented Sep 14, 2026

Copy link
Copy Markdown
Contributor
3/3  17s  evals/check-before-acting.eval.ts / consults the docs before an unfamiliar task
3/3  34s  evals/check-before-acting.eval.ts / does not destructively act on an ambiguous request
3/3  10s  evals/check-before-acting.eval.ts / does not try to reassign a document to another type
3/3  11s  evals/configure-repositories.eval.ts / adds a locale
3/3  18s  evals/configure-repositories.eval.ts / creates an API token and makes the API private
3/3  15s  evals/configure-repositories.eval.ts / registers a webhook
3/3  14s  evals/configure-repositories.eval.ts / sets up a content preview
3/3  18s  evals/configure-repositories.eval.ts / updates previews for production after a deploy
3/3  44s  evals/configure-routes.eval.ts / routes a nested URL through a content relationship
3/3  22s  evals/configure-routes.eval.ts / routes non-default locales with an optional locale prefix
3/3  23s  evals/configure-routes.eval.ts / routes the home document to the root URL
3/3  134s  evals/design-sensible-models.eval.ts / handles a vague design request reasonably
3/3  62s  evals/design-sensible-models.eval.ts / models a page type with sensible field types
3/3  65s  evals/design-sensible-models.eval.ts / models a slice from a screenshot
3/3  32s  evals/design-sensible-models.eval.ts / models a testimonial slice from a text description
0/3  31s  evals/design-sensible-models.eval.ts / models a title as single-heading rich text and a social media handle as key text
3/3  109s  evals/design-sensible-models.eval.ts / models pages as page types and data as custom types
3/3  26s  evals/design-sensible-models.eval.ts / picks appropriate field types for a rating and a CTA
3/3  26s  evals/edit-models-precisely.eval.ts / adds a field
3/3  32s  evals/edit-models-precisely.eval.ts / adds a group field with nested fields
3/3  16s  evals/edit-models-precisely.eval.ts / adds a slice variation without losing existing ones
3/3  28s  evals/edit-models-precisely.eval.ts / adds constrained fields with the right config
3/3  15s  evals/edit-models-precisely.eval.ts / connects a slice to a page type
3/3  42s  evals/edit-models-precisely.eval.ts / extends a slice to match a design change
3/3  36s  evals/edit-models-precisely.eval.ts / is idempotent when the same task runs twice
3/3  23s  evals/edit-models-precisely.eval.ts / uses the documented config for a single-heading constraint
1/3  20s  evals/group-commands-by-task.eval.ts / passes one task ID and intent to every command
3/3  36s  evals/initialize-projects.eval.ts / adds Prismic to an existing Next.js app without clobbering it
3/3  36s  evals/initialize-projects.eval.ts / initializes Prismic in a Next.js project
3/3  21s  evals/initialize-projects.eval.ts / initializes with an existing repository
0/3  78s  evals/know-the-cli.eval.ts / claude-fable-5-1 > plans a Prismic site with the CLI, not Slice Machine
0/3  348s  evals/know-the-cli.eval.ts / claude-fable-5-1 > tells the user Slice Machine is replaced and moves to the CLI
0/3  35s  evals/know-the-cli.eval.ts / claude-opus-4-8 > plans a Prismic site with the CLI, not Slice Machine
0/3  79s  evals/know-the-cli.eval.ts / claude-opus-5 > plans a Prismic site with the CLI, not Slice Machine
0/3  307s  evals/know-the-cli.eval.ts / claude-opus-5 > tells the user Slice Machine is replaced and moves to the CLI
0/3  20s  evals/know-the-cli.eval.ts / claude-sonnet-5 > plans a Prismic site with the CLI, not Slice Machine
0/3  125s  evals/know-the-cli.eval.ts / claude-sonnet-5 > tells the user Slice Machine is replaced and moves to the CLI
0/3  12s  evals/know-the-cli.eval.ts / gpt-5.5 > plans a Prismic site with the CLI, not Slice Machine
0/3  14s  evals/know-the-cli.eval.ts / gpt-5.6-luna > plans a Prismic site with the CLI, not Slice Machine
0/3  112s  evals/know-the-cli.eval.ts / gpt-5.6-luna > tells the user Slice Machine is replaced and moves to the CLI
0/3  15s  evals/know-the-cli.eval.ts / gpt-5.6-sol > plans a Prismic site with the CLI, not Slice Machine
0/3  103s  evals/know-the-cli.eval.ts / gpt-5.6-sol > tells the user Slice Machine is replaced and moves to the CLI
0/3  9s  evals/know-the-cli.eval.ts / gpt-5.6-terra > plans a Prismic site with the CLI, not Slice Machine
0/3  92s  evals/know-the-cli.eval.ts / gpt-5.6-terra > tells the user Slice Machine is replaced and moves to the CLI
3/3  124s  evals/render-content.eval.ts / models and wires a landing page end to end
3/3  106s  evals/render-content.eval.ts / writes a slice component
3/3  17s  evals/repair-generated-files.eval.ts / regenerates a stale slice index instead of editing it by hand
3/3  20s  evals/sync-models.eval.ts / commits and pushes local model changes
0/3  20s  evals/sync-models.eval.ts / syncs in the right direction when the repo is newer

Run: https://github.com/prismicio/cli/actions/runs/34814616967


Note

Low Risk
Data-only update to committed eval snapshots; no application or workflow logic changes in this PR.

Overview
Records the latest scheduled full agent eval snapshot at commit f686823 by updating evals/results.json (via the evals GitHub Actions “Record results” job).

The embedded report shows most suites still at 3/3, with notable regressions in this run: know-the-cli cases are 0/3 across the listed models, sync-models (“syncs in the right direction when the repo is newer”) is 0/3, group-commands-by-task is 1/3, and one design-sensible-models case (“title as single-heading rich text…”) is 0/3.

Reviewed by Cursor Bugbot for commit c5c099a. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions
github-actions Bot merged commit f9d07f9 into main Sep 14, 2026
@github-actions
github-actions Bot deleted the evals/run-34814616967-1 branch September 14, 2026 06:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants