Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/quiet-json-schema-help.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'@shopify/cli-kit': minor
'@shopify/cli': minor
---

Show JSON Schema in command help and use `--json-schema` to print result, error, and event schemas.
1,169 changes: 1,053 additions & 116 deletions docs-shopify.dev/generated/generated_docs_data_v2.json

Large diffs are not rendered by default.

16 changes: 10 additions & 6 deletions docs/cli/json-output.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ commands; remove each entry when converted, and never add new finite commands to
## Define the result beside the domain service

Keep the schema beside the service that produces the result. One Zod schema supplies runtime validation, the inferred
TypeScript type, JSON encoding, and the type shown in command help.
TypeScript type, JSON encoding, and the JSON Schema shown in command help.

```ts
import {defineJsonOutputSchema, type InferJsonOutputSchema} from '@shopify/cli-kit/node/json-output-schema'
Expand All @@ -34,8 +34,8 @@ export const widgetListJsonOutputSchema = defineJsonOutputSchema({
export type WidgetListResult = InferJsonOutputSchema<typeof widgetListJsonOutputSchema>
```

Add nested object schemas to `definitions` so generated help gives them stable names. Use `.passthrough()` only when
the public result deliberately permits additional keys.
Optionally add nested object schemas to `definitions` to give them stable names and references in JSON Schema.
Use `.passthrough()` only when the public result deliberately permits additional keys.

## Connect the command and encoder

Expand Down Expand Up @@ -75,8 +75,8 @@ Presenters continue to own terminal text, output channels, files, and exit behav
on terminal rendering (including React/Ink), Oclif, filesystem output, or CLI errors.

Events are separate from finite results. Progress events can drive spinners or status messages while the command is
running, but they aren't fields in the final JSON result. Errors continue through the standard CLI error path and
stderr; don't encode failures as successful result shapes merely to support `--json`.
running, but they aren't fields in the final JSON result. Errors continue through the standard CLI error path;
don't encode failures as successful result shapes merely to support `--json`.

## Preserve compatibility

Expand Down Expand Up @@ -115,5 +115,9 @@ Tests should verify:
- errors and exit behavior; and
- prompt behavior independently from `--json` and `--no-input`.

Command help includes the generated TypeScript contract automatically through `jsonOutputSchema`. Run the manifest,
Command help includes the result's JSON Schema automatically through `jsonOutputSchema`. `--json-schema` prints one
JSON Schema (draft-07) accepting a result, a fatal error document, or a side event. The `Result`, `Error`, and `Event`
definitions describe these separately; results and fatal errors go to stdout, and side events go to stderr.

Both outputs come from the same Zod definitions used to validate and encode results. Run the manifest,
README, and code-documentation refresh commands required by CI after changing command metadata.
2 changes: 1 addition & 1 deletion packages/app/src/cli/commands/organization/list.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import {authAliasFlag, globalFlags, jsonFlag} from '@shopify/cli-kit/node/cli'
import BaseCommand from '@shopify/cli-kit/node/base-command'

export default class OrganizationList extends BaseCommand {
static baseFlags = authAliasFlag
static baseFlags = {...BaseCommand.baseFlags, ...authAliasFlag}

static summary = 'List Shopify organizations you have access to.'

Expand Down
2 changes: 1 addition & 1 deletion packages/app/src/cli/utilities/app-command.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ interface AppCommandOutput {
}

export default abstract class AppCommand extends BaseCommand {
static baseFlags = authAliasFlag
static baseFlags = {...BaseCommand.baseFlags, ...authAliasFlag}

environmentsFilename(): string {
return configurationFileNames.appEnvironments
Expand Down
3 changes: 2 additions & 1 deletion packages/cli-kit/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,8 @@
"strip-ansi": "7.2.0",
"supports-hyperlinks": "3.2.0",
"which": "4.0.0",
"zod": "3.25.76"
"zod": "3.25.76",
"zod-to-json-schema": "3.25.2"
Comment thread
dmerand marked this conversation as resolved.
},
"devDependencies": {
"@types/archiver": "5.3.2",
Expand Down
158 changes: 148 additions & 10 deletions packages/cli-kit/src/public/node/base-command.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ import {unstyled} from './output.js'
import {defineJsonOutputSchema} from './json-output-schema.js'
import {zod} from './schema.js'
import {afterEach, beforeEach, describe, expect, test, vi} from 'vitest'
import {Flags} from '@oclif/core'
import {Flags, type Config} from '@oclif/core'
import {Ajv} from 'ajv'

let originalStdinIsTTY: boolean | undefined
let originalStdoutIsTTY: boolean | undefined
Expand Down Expand Up @@ -285,6 +286,20 @@ describe('command events', () => {
})

describe('command descriptions', () => {
test('preserves schema patterns that resemble Markdown links', () => {
class CommandWithPattern extends Command {
static get jsonOutputSchema() {
return defineJsonOutputSchema({name: 'Result', schema: zod.string().regex(/[a-z](value)/)})
}

public async run(): Promise<void> {}
}

const description = CommandWithPattern.descriptionForHelp()!
const schema = JSON.parse(description.match(/```json\n([\s\S]+)\n```/)![1]!)
expect(schema.pattern).toBe('[a-z](value)')
})

test('includes a JSON output schema without mutating the Markdown description', () => {
class CommandWithJsonOutput extends Command {
static get jsonOutputSchema() {
Expand All @@ -301,15 +316,22 @@ describe('command descriptions', () => {
public async run(): Promise<void> {}
}

expect(CommandWithJsonOutput.description).toBe(`Returns a value. "Learn more" (https://shopify.dev).

With \`--json\`, the command returns \`CommandResult\`:

\`\`\`ts
interface CommandResult {
value: string
}
\`\`\``)
expect(CommandWithJsonOutput.description).toContain('Returns a value. "Learn more" (https://shopify.dev).')
expect(CommandWithJsonOutput.description).toContain(
'Use `--json-schema` to print the result, error, and event schemas.',
)
const helpSchema = JSON.parse(CommandWithJsonOutput.description!.match(/```json\n([\s\S]+)\n```/)![1]!)
expect(helpSchema).toEqual({
$schema: 'http://json-schema.org/draft-07/schema#',
title: 'CommandResult',
type: 'object',
properties: {value: {type: 'string'}},
required: ['value'],
additionalProperties: false,
})
const validate = new Ajv().compile(helpSchema)
expect(validate({value: 'ready'})).toBe(true)
expect(validate({value: 1})).toBe(false)
expect(CommandWithJsonOutput.descriptionWithMarkdown).toBe('Returns a value. [Learn more](https://shopify.dev).')

CommandWithJsonOutput.descriptionForHelp()
Expand All @@ -318,6 +340,122 @@ interface CommandResult {
})
})

describe('JSON output schema flag', () => {
class CommandWithJsonOutput extends Command {
static get jsonOutputSchema() {
return defineJsonOutputSchema({
name: 'CommandResult',
schema: zod.object({value: zod.string()}),
})
}

public async run(): Promise<void> {}
}

test.each([
{argv: ['--json-schema'], environment: ''},
{argv: ['--json-schema', '--', 'forwarded'], environment: ''},
{argv: [], environment: '1'},
])('prints only the schema and exits: %j', async ({argv, environment}) => {
vi.stubEnv('SHOPIFY_FLAG_JSON_SCHEMA', environment)
const outputMock = mockAndCaptureOutput()
const command = new CommandWithJsonOutput(argv, {} as Config)
const exit = vi.spyOn(process, 'exit').mockImplementation(() => {
throw new Error('process.exit')
})

try {
await expect(
(
command as unknown as {
exitWithJsonSchemaWhenRequested(): Promise<void>
}
).exitWithJsonSchemaWhenRequested(),
).rejects.toThrow('process.exit')
expect(exit).toHaveBeenCalledWith(0)
const schema = JSON.parse(outputMock.output())
expect(schema).toMatchObject({
$schema: 'http://json-schema.org/draft-07/schema#',
title: 'CommandOutput',
anyOf: [{$ref: '#/definitions/Result'}, {$ref: '#/definitions/Error'}, {$ref: '#/definitions/Event'}],
})
const validate = new Ajv({validateFormats: false}).compile(schema)
expect(validate({value: 'ready'})).toBe(true)
expect(validate({error: {type: 'abort', message: 'Failed'}})).toBe(true)
expect(
validate({type: 'diagnostic', timestamp: '2026-08-26T12:00:00.000Z', level: 'info', message: 'Ready'}),
).toBe(true)
expect(
validate({type: 'progress', timestamp: '2026-08-26T12:00:00.000Z', status: 'started', operation: 'upload'}),
).toBe(true)
expect(validate({value: 1})).toBe(false)
expect(validate({error: {type: 'external', message: 'Missing command and args'}})).toBe(false)
expect(validate({type: 'progress', timestamp: '2026-08-26T12:00:00.000Z', status: 'started'})).toBe(false)
} finally {
exit.mockRestore()
}
})

test('ignores the schema flag after the passthrough boundary', async () => {
const outputMock = mockAndCaptureOutput()
const command = new CommandWithJsonOutput(['--', '--json-schema'], {} as Config)

await expect(
(
command as unknown as {
exitWithJsonSchemaWhenRequested(): Promise<void>
}
).exitWithJsonSchemaWhenRequested(),
).resolves.toBeUndefined()
expect(outputMock.output()).toBe('')
})

test('waits for stdout to flush before exiting', async () => {
mockAndCaptureOutput()
const command = new CommandWithJsonOutput(['--json-schema'], {} as Config)
let finishWrite: (() => void) | undefined
const write = vi
.spyOn(process.stdout, 'write')
.mockImplementationOnce((_chunk, callback: BufferEncoding | (() => void) | undefined) => {
if (typeof callback === 'function') finishWrite = callback
return true
})
const exit = vi.spyOn(process, 'exit').mockImplementation(() => {
throw new Error('process.exit')
})

try {
const result = (
command as unknown as {
exitWithJsonSchemaWhenRequested(): Promise<void>
}
).exitWithJsonSchemaWhenRequested()
const assertion = expect(result).rejects.toThrow('process.exit')

expect(write).toHaveBeenCalledWith('', expect.any(Function))
expect(exit).not.toHaveBeenCalled()
finishWrite?.()
await assertion
expect(exit).toHaveBeenCalledWith(0)
} finally {
write.mockRestore()
exit.mockRestore()
}
})

test('throws an error when the command has no schema', async () => {
const command = new MockCommand(['--json-schema'], {} as Config)

await expect(
(
command as unknown as {
exitWithJsonSchemaWhenRequested(): Promise<void>
}
).exitWithJsonSchemaWhenRequested(),
).rejects.toThrow('This command does not define a JSON output schema.')
})
})

describe('applying environments', async () => {
const runTestInTmpDir = (testName: string, testFunc: (tmpDir: string) => Promise<void>) => {
test(testName, async () => {
Expand Down
58 changes: 46 additions & 12 deletions packages/cli-kit/src/public/node/base-command.ts
Original file line number Diff line number Diff line change
@@ -1,18 +1,20 @@
import {isDevelopment} from './context/local.js'
import {addPublicMetadata} from './metadata.js'
import {AbortError} from './error.js'
import {runWithCommandEventsForCommand} from './command-events.js'
import {outputContent, outputResult, outputToken} from './output.js'
import {commandEventOutputSchema, runWithCommandEventsForCommand} from './command-events.js'
import {jsonErrorOutputSchema} from './error/schema.js'
import {flushStdout, outputContent, outputResult, outputToken} from './output.js'
import {setCurrentSessionAlias} from './session.js'
import {terminalSupportsPrompting} from './system.js'
import {hashString} from './crypto.js'
import {isTruthy} from './context/utilities.js'
import {setCurrentCommandId} from './global-context.js'
import {defineJsonOutputSchema, type JsonOutputSchema} from './json-output-schema.js'
import {zod} from './schema.js'
import {JsonMap} from '../../private/common/json.js'
import {underscore} from '../common/string.js'
import {Command, Config, Errors} from '@oclif/core'
import {Command, Config, Errors, Flags} from '@oclif/core'
import {OutputFlags, Input, ParserOutput, FlagInput, OutputArgs} from '@oclif/core/parser'
import type {JsonOutputSchema} from './json-output-schema.js'

// eslint-disable-next-line @typescript-eslint/no-explicit-any
export type ArgOutput = OutputArgs<any>
Expand All @@ -33,7 +35,13 @@ interface EnvironmentFlags {
}

abstract class BaseCommand extends Command {
static baseFlags: FlagInput<{}> = {}
static baseFlags: FlagInput<{}> = {
'json-schema': Flags.boolean({
description: "Print the command's JSON schemas.",
env: 'SHOPIFY_FLAG_JSON_SCHEMA',
}),
}

static descriptionWithMarkdown?: string

public static get jsonOutputSchema(): JsonOutputSchema | undefined {
Expand All @@ -50,10 +58,8 @@ abstract class BaseCommand extends Command {

// Include the JSON result schema and convert Markdown links to plain text for command help.
public static descriptionForHelp(): string | undefined {
return appendJsonOutputSchema(this.descriptionWithMarkdown ?? '', this.jsonOutputSchema).replace(
/(\[)(.*?)(])(\()(.*?)(\))/gm,
'"$2" ($5)',
)
const description = (this.descriptionWithMarkdown ?? '').replace(/(\[)(.*?)(])(\()(.*?)(\))/gm, '"$2" ($5)')
return appendJsonOutputSchema(description, this.jsonOutputSchema)
}

/** @deprecated Use descriptionForHelp instead. */
Expand Down Expand Up @@ -81,6 +87,7 @@ abstract class BaseCommand extends Command {
}

protected async init(): Promise<unknown> {
await this.exitWithJsonSchemaWhenRequested()
this.exitWithTimestampWhenEnvVariablePresent()
setCurrentCommandId(this.id ?? '')
if (!isDevelopment()) {
Expand Down Expand Up @@ -129,6 +136,31 @@ abstract class BaseCommand extends Command {
}
}

protected async exitWithJsonSchemaWhenRequested(): Promise<void> {
const passthroughIndex = this.argv.indexOf('--')
const commandArguments = passthroughIndex === -1 ? this.argv : this.argv.slice(0, passthroughIndex)
if (!commandArguments.includes('--json-schema') && !isTruthy(process.env.SHOPIFY_FLAG_JSON_SCHEMA)) return

const command = this.constructor as typeof BaseCommand
const outputSchema = command.jsonOutputSchema
if (!outputSchema) {
throw new AbortError('This command does not define a JSON output schema.')
}

const commandSchema = defineJsonOutputSchema({
name: 'CommandOutput',
schema: zod.union([outputSchema.schema, jsonErrorOutputSchema.schema, commandEventOutputSchema.schema]),
definitions: {
Result: outputSchema.schema,
Error: jsonErrorOutputSchema.schema,
Event: commandEventOutputSchema.schema,
},
})
outputResult(JSON.stringify(commandSchema.jsonSchema, null, 2))
await flushStdout()
process.exit(0)
Comment thread
dmerand marked this conversation as resolved.
}

protected async parse<
TFlags extends FlagOutput & {path?: string; verbose?: boolean; 'auth-alias'?: string},
TGlobalFlags extends FlagOutput,
Expand Down Expand Up @@ -413,10 +445,12 @@ function commandSupportsFlag(flags: FlagInput | undefined, flagName: string): bo
function appendJsonOutputSchema(description: string, outputSchema: JsonOutputSchema | undefined): string {
if (!outputSchema) return description

const jsonOutputDescription = `With \`--json\`, the command returns \`${outputSchema.name}\`:
const jsonOutputDescription = `Output from \`--json\` conforms to the \`${outputSchema.name}\` schema.

Use \`--json-schema\` to print the result, error, and event schemas.

\`\`\`ts
${outputSchema.typescript}
\`\`\`json
${JSON.stringify(outputSchema.jsonSchema, null, 2)}
\`\`\``

return [description, jsonOutputDescription].filter(Boolean).join('\n\n')
Expand Down
Loading
Loading