Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion apps/docs/demo/sequences/new-pet.seq.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ export default sequence({
}),
},
})
// #endregion flow
.step('Add the pet', addPet) // keeps the id the store gave it
.step('Fetch it back', (ctx) =>
getPet.with({
Expand All @@ -27,4 +28,3 @@ export default sequence({
},
}),
)
// #endregion flow
1 change: 1 addition & 0 deletions apps/docs/public/llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,7 @@ export default sequence({
- `request.with({ ... })` changes a request for one step (for example its `assert`, or the values it sends).
- Also: `.input(name, ctx => controls)` asks a person for values, `.each(name, items, make)` repeats a request per item, `.check(name, fn)` asserts on the flow's state, `.use(otherFlow)` includes another flow.
- Controls are `c.number`, `c.text`, `c.select`, `c.multiselect`, `c.boolean`, `c.date`, `c.object`, `c.array`. A flow also runs unattended in CI and as a monitor, so every input and every `.input()` control needs a default that makes sense with nobody to ask.
- Several values for one thing (travelers, line items): `c.object({ name: c.text(...), gender: c.select([...]) }, { label: 'Traveler 1' })` is a group, and `c.array(item, { min: 1, max: 10, label: 'Travelers' })` is a list whose length a person sets. The form shows groups with the same fields as one table with a row for each, so declare them with one helper (`const traveler = (label: string) => c.object({ ... }, { label })`). A list with no value starts at `min` items, each made from its own defaults. Read them as `{{traveler1.name}}` or `{{travelers[0].name}}`, or `ctx.inputs.travelers` (`.each(name, ctx => ctx.inputs.travelers, ...)` makes a request for each). A flat key with a dot (`'traveler1.name'`) is not a path: use a group.
- For generated data: `import { faker } from '@specbook/core/faker'` and use it in a default, for example `c.text({ default: () => faker.person.fullName() })`. It is typed, and `specbook init` adds `@faker-js/faker` to `package.json` (`npm install` fetches it). A `{{faker.person.firstName}}` placeholder (`faker.<module>.<method>`, no arguments) also works in a field.
- Each step is also a request you can run alone.

Expand Down
2 changes: 1 addition & 1 deletion apps/docs/src/content/docs/reference/cli.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ description: Every command and what it does.
| `--timeout ms`, `--global-timeout ms` | Limit each request (default: the config's `timeoutMs`, else 30000; `0` waits), and the whole run. |
| `--safe` | Refuses requests that may change data. Use it against production. |
| `--reporter list\|json\|junit\|github\|ndjson[:path]` | Chooses reporters. Repeat the flag to use several. |
| `--input k=v`, `--var k=v` | Sets a flow input, or a variable. Repeat either. A variable can override an address. |
| `--input k=v`, `--var k=v` | Sets a flow input, or a variable. Repeat either. A variable can override an address. The value is read as JSON when it is, so a list is `--input 'travelers=[{"name":"Ada"}]'`. |
| `--secrets-file <path>` | Reads secrets from a dotenv file. |
| `--trace on-failure\|always\|off` | When to keep a step's request and response. The default is `on-failure`. |
| `--quiet` | Prints only failures and the summary. |
Expand Down
2 changes: 1 addition & 1 deletion apps/docs/src/content/docs/reference/embed.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ A plain element gets a Play button beside it. A button or link that carries the
| `data-specbook-try="service/Operation"` | Opens a request. A bare `Operation` works when the build has one service. A request file such as `requests/petstore/find-pets.req.ts` opens that request, with its checks. |
| `data-specbook-flow="id"` | Opens a flow by its id. |
| `data-specbook-example="name"` | Starts from a named example. |
| `data-specbook-inputs='{"quantity":2}'` | Sets a flow's inputs, over their defaults. |
| `data-specbook-inputs='{"quantity":2}'` | Sets a flow's inputs, over their defaults. A group or a list is given as an object or an array, and what it leaves out keeps its default. |
| `data-specbook-env="demo"` | Chooses the environment. Set it on an element or on any ancestor, so a page can set it once. |
| `data-specbook-tab="docs"` | Opens on the Docs tab instead of Try. |
| `data-specbook-auto-run` | Runs as soon as the panel opens. Never against production. |
Expand Down
12 changes: 12 additions & 0 deletions apps/docs/src/pages/test.astro
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,18 @@ import { capture } from '../lib/captures'
</p>
</div>
<Snippet file="demo/sequences/new-pet.seq.ts" region="flow" />
<div class="prose">
<p>
Inputs can repeat, too. Declare a traveler as a <code>c.object</code> and a booking's travelers as a{' '}
<code>c.array</code> with a minimum and a maximum. QA sets how many there are and gets a row for each,
and CI can pass the whole list with <code>--input travelers=[…]</code>.
</p>
<p>
<a href="https://demo.specbook.dev/#/seq/ui-demo" target="_blank" rel="noopener"
>Try it on the demo site ↗</a
>
</p>
</div>

<h2 class="title">Give QA a portal</h2>
<div class="prose">
Expand Down
46 changes: 46 additions & 0 deletions apps/example/sequences/ui-demo.seq.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
import { c, sequence } from '@specbook/core'
import { faker } from '@specbook/core/faker'

// A flow with no request: it exists to show what QA gets from the inputs. Change how many travelers, press the die to make
// another name for one traveler, or Generate all for a new set, then Run to see the checks.
export default sequence({
id: 'ui-demo',
title: 'Dynamic inputs',
group: 'UI',
description:
'A demo of generated inputs, with no request to send. A booking starts with two travelers (1 to 10): set how many, make a first and last name for each with the dice or Generate all, choose a gender, say which one is the primary traveler, and give a date of birth if you want to. Run only checks what was entered.',
tags: ['demo'],
inputs: {
travelers: c.array(
c.object({
firstName: c.text({ default: () => faker.person.firstName() }),
lastName: c.text({ default: () => faker.person.lastName() }),
gender: c.select(['female', 'male', 'other'] as const),
role: c.select(['primary', 'companion'] as const, { default: 'companion' }),
dateOfBirth: c.date({ description: 'Optional.' }),
}),
{
min: 1,
max: 10,
label: 'Travelers',
// the first is the primary traveler; each one is made from the defaults above, with its own generated names
default: () => [{ role: 'primary' }, {}],
},
),
},
})
.check('Every traveler has a first and last name', (ctx, { assert }) => {
for (const t of ctx.inputs.travelers)
assert.ok(t.firstName.trim() !== '' && t.lastName.trim() !== '', 'a traveler has a name')
})
.check('Exactly one traveler is primary', (ctx, { assert }) => {
const primary = ctx.inputs.travelers.filter((t) => t.role === 'primary').length
assert.is(primary, 1, `${primary} travelers are primary`)
})
.check('A date of birth, when given, is a date', (ctx, { assert }) => {
for (const t of ctx.inputs.travelers)
assert.ok(
!t.dateOfBirth || !Number.isNaN(Date.parse(t.dateOfBirth)),
'a date of birth is a date',
)
})
1 change: 1 addition & 0 deletions packages/cli/agent-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,7 @@ export default sequence({
- `request.with({ ... })` changes a request for one step (for example its `assert`, or the values it sends).
- Also: `.input(name, ctx => controls)` asks a person for values, `.each(name, items, make)` repeats a request per item, `.check(name, fn)` asserts on the flow's state, `.use(otherFlow)` includes another flow.
- Controls are `c.number`, `c.text`, `c.select`, `c.multiselect`, `c.boolean`, `c.date`, `c.object`, `c.array`. A flow also runs unattended in CI and as a monitor, so every input and every `.input()` control needs a default that makes sense with nobody to ask.
- Several values for one thing (travelers, line items): `c.object({ name: c.text(...), gender: c.select([...]) }, { label: 'Traveler 1' })` is a group, and `c.array(item, { min: 1, max: 10, label: 'Travelers' })` is a list whose length a person sets. The form shows groups with the same fields as one table with a row for each, so declare them with one helper (`const traveler = (label: string) => c.object({ ... }, { label })`). A list with no value starts at `min` items, each made from its own defaults. Read them as `{{traveler1.name}}` or `{{travelers[0].name}}`, or `ctx.inputs.travelers` (`.each(name, ctx => ctx.inputs.travelers, ...)` makes a request for each). A flat key with a dot (`'traveler1.name'`) is not a path: use a group.
- For generated data: `import { faker } from '@specbook/core/faker'` and use it in a default, for example `c.text({ default: () => faker.person.fullName() })`. It is typed, and `specbook init` adds `@faker-js/faker` to `package.json` (`npm install` fetches it). A `{{faker.person.firstName}}` placeholder (`faker.<module>.<method>`, no arguments) also works in a field.
- Each step is also a request you can run alone.

Expand Down
92 changes: 72 additions & 20 deletions packages/core/src/controls.ts
Original file line number Diff line number Diff line change
Expand Up @@ -81,13 +81,25 @@ export const c = {
kind: 'multiselect',
opts: { ...o, options },
}),
object: <S extends Controls>(shape: S): Control<InferControls<S>> => ({
/** A group of fields; its `label` heads the row when objects of the same shape are shown as one table. */
object: <S extends Controls>(
shape: S,
o: { label?: string; description?: string } = {},
): Control<InferControls<S>> => ({
kind: 'object',
opts: { shape },
opts: { ...o, shape },
}),
/** A list of `min` to `max` items, each made from the item's own defaults when nothing is given. */
array: <T>(
item: Control<T>,
o: { default?: T[]; min?: number; max?: number } = {},
o: {
/** the starting list, or a function that makes it; an item may name only some fields, the rest start from their own defaults */
default?: Default<Partial<T>[]>
min?: number
max?: number
label?: string
description?: string
} = {},
): Control<T[]> => ({ kind: 'array', opts: { ...o, item } }),
}

Expand Down Expand Up @@ -141,17 +153,20 @@ export function resolveControl(name: string, ctl: Control, given: unknown): unkn
const shape = ctl.opts.shape as Controls
const g = (given ?? {}) as Record<string, unknown>
return Object.fromEntries(
Object.entries(shape).map(([k, v]) => [k, resolveControl(k, v, g[k])]),
Object.entries(shape).map(([k, v]) => [k, resolveControl(`${name}.${k}`, v, g[k])]),
)
}
case 'array': {
const v = (given ?? ctl.opts.default ?? []) as unknown[]
const { min, max } = ctl.opts
// nothing given: the list starts at its minimum, each item made from its own defaults
const v = (given ??
startingValue(ctl.opts.default) ??
Array.from({ length: min ?? 0 })) as unknown[]
if (min !== undefined && v.length < min)
throw new ControlError(`"${name}" needs at least ${min} items`)
if (max !== undefined && v.length > max)
throw new ControlError(`"${name}" allows at most ${max} items`)
return v.map((item) => resolveControl(name, ctl.opts.item, item))
return v.map((item, i) => resolveControl(`${name}[${i}]`, ctl.opts.item, item))
}
case 'boolean':
return given ?? startingValue(ctl.opts.default) ?? false
Expand All @@ -171,22 +186,59 @@ export function resolveControls(
)
}

/** Serializable description of one control, for a UI to render an input step. */
function describeControl(name: string, ctl: Control): Record<string, unknown> {
const text = (k: string) => (typeof ctl.opts[k] === 'string' ? { [k]: ctl.opts[k] } : {})
const meta = { ...text('label'), ...text('description') }
if (ctl.kind === 'select' || ctl.kind === 'multiselect') {
const labelFn = ctl.opts.label as ((o: unknown) => string) | undefined
const options = (ctl.opts.options as unknown[]).map((o) => ({
label: labelFn ? labelFn(o) : String(optionValue(ctl, o)),
value: optionValue(ctl, o),
}))
let initial: unknown
try {
initial = resolveControl(name, ctl, undefined)
} catch {
/* no options: nothing to start from */
}
return {
kind: ctl.kind,
options,
...(initial !== undefined ? { default: initial } : {}),
...meta,
}
}
if (ctl.kind === 'object')
return {
kind: 'object',
shape: Object.fromEntries(
Object.entries(ctl.opts.shape as Controls).map(([k, v]) => [k, describeControl(k, v)]),
),
...meta,
}
if (ctl.kind === 'array')
return {
kind: 'array',
item: describeControl(name, ctl.opts.item),
default: startingValue(ctl.opts.default),
min: ctl.opts.min,
max: ctl.opts.max,
...meta,
}
return {
kind: ctl.kind,
default: startingValue(ctl.opts.default),
generated: isGenerated(ctl),
...(ctl.opts.min !== undefined ? { min: ctl.opts.min } : {}),
...(ctl.opts.max !== undefined ? { max: ctl.opts.max } : {}),
...meta,
}
}

/** Serializable description for a UI to render an input step. */
export function describeControls(controls: Controls) {
return Object.fromEntries(
Object.entries(controls).map(([k, ctl]) => {
if (ctl.kind === 'select' || ctl.kind === 'multiselect') {
const labelFn = ctl.opts.label as ((o: unknown) => string) | undefined
const options = (ctl.opts.options as unknown[]).map((o) => ({
label: labelFn ? labelFn(o) : String(optionValue(ctl, o)),
value: optionValue(ctl, o),
}))
return [k, { kind: ctl.kind, options }]
}
return [
k,
{ kind: ctl.kind, default: startingValue(ctl.opts.default), generated: isGenerated(ctl) },
]
}),
Object.entries(controls).map(([k, ctl]) => [k, describeControl(k, ctl)]),
)
}
52 changes: 51 additions & 1 deletion packages/core/test/controls.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,56 @@ describe('controls', () => {
const d = describeControls({
f: c.select([{ id: 1, n: 'A' }], { label: (o) => o.n, value: (o) => o.id }),
})
expect(d.f).toEqual({ kind: 'select', options: [{ label: 'A', value: 1 }] })
expect(d.f).toEqual({ kind: 'select', options: [{ label: 'A', value: 1 }], default: 1 })
})
it('a list with no value given starts at its minimum, each item from its own defaults', () => {
let n = 0
const t = c.array(c.object({ name: c.text({ default: () => `P${++n}` }) }), { min: 2, max: 3 })
expect(resolveControls({ t })).toEqual({ t: [{ name: 'P1' }, { name: 'P2' }] })
expect(resolveControls({ t: c.array(c.text(), {}) })).toEqual({ t: [] })
})
it('names the path of a value that is wrong', () => {
const t = c.array(c.object({ gender: c.select(['F', 'M'] as const) }), { min: 1 })
expect(() => resolveControls({ t }, { t: [{ gender: 'F' }, { gender: 'Z' }] })).toThrow(
/t\[1\]\.gender/,
)
expect(() =>
resolveControls({ g: c.object({ x: c.select(['a'] as const) }) }, { g: { x: 'b' } }),
).toThrow(/g\.x/)
})
it('describes groups and lists with their shape, labels and limits', () => {
const d = describeControls({
who: c.object({ name: c.text({ label: 'Name', default: 'x' }) }, { label: 'Traveler 1' }),
all: c.array(c.object({ n: c.number({ min: 1 }) }), { min: 1, max: 4, label: 'Travelers' }),
})
expect(d.who).toMatchObject({
kind: 'object',
label: 'Traveler 1',
shape: { name: { kind: 'text', label: 'Name', default: 'x' } },
})
expect(d.all).toMatchObject({
kind: 'array',
min: 1,
max: 4,
label: 'Travelers',
item: { kind: 'object', shape: { n: { kind: 'number', min: 1 } } },
})
})
it('a list can start from a function that names only some fields of each item', () => {
let n = 0
const t = c.array(
c.object({
name: c.text({ default: () => `P${++n}` }),
role: c.text({ default: 'companion' }),
}),
{ min: 1, default: () => [{ role: 'primary' }, {}] },
)
expect(resolveControls({ t })).toEqual({
t: [
{ name: 'P1', role: 'primary' },
{ name: 'P2', role: 'companion' },
],
})
expect(describeControls({ t }).t).toMatchObject({ default: [{ role: 'primary' }, {}] })
})
})
13 changes: 13 additions & 0 deletions packages/core/test/runner.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -315,6 +315,7 @@ describe('runSequence', () => {
{ label: 'X1', value: 'a' },
{ label: 'X2', value: 'b' },
],
default: 'a',
},
})
expect(calls.at(-1)?.url).toBe('/book/b')
Expand Down Expand Up @@ -907,3 +908,15 @@ describe('OAuth client credentials', () => {
expect(r.steps[0]?.error).toContain('has no usable token address')
})
})

describe('a flow whose list input has a minimum', () => {
it('runs on its defaults: the list starts at its minimum', async () => {
const seq = sequence({
id: 'lists',
title: 'Lists',
inputs: { people: c.array(c.object({ name: c.text({ default: 'Ada' }) }), { min: 2 }) },
})
const r = await runSequence(seq, {} as never)
expect(r.status).not.toBe('error')
})
})
Loading
Loading