Skip to content

fix(workflow): validate declared input and output schemas at publish and run boundaries (issue #274) - #359

Open
konlue wants to merge 4 commits into
agentic-ai-java:mainfrom
konlue:feat/issue274-workflow-schema-contract
Open

konlue wants to merge 4 commits into
agentic-ai-java:mainfrom
konlue:feat/issue274-workflow-schema-contract

Conversation

@konlue

@konlue konlue commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

Describe what this PR does / why we need it

Issue #274 reported that WorkflowDefinition exposes inputSchema/outputSchema, but the validator only rejected blank strings and the runner never read either field — an illegal schema could be published, a run missing a required input still executed its tools, and the outcome came back COMPLETED. This PR makes the declared schemas real contracts at both the publish and the run boundary, using the same JSON Schema capability the agent layer already ships (networknt, draft 2020-12, version managed by the root pom), so no competing validation stack is introduced and no public API changes.

What is delivered

Publish and instantiation

WorkflowDefinitionValidator now parses every non-blank declared schema with the networknt registry (draft 2020-12) and rejects malformed ones with an error naming the field and the dialect. Blank schemas keep their existing rejection; definitions without schemas keep their existing behavior untouched.

Input boundary, before any side effect

On a fresh run, WorkflowAgent validates the request inputs against the declared input schema before the first step executes — the issue's reproduction (missing required input, tool executed, COMPLETED) now fails with zero tool executions. The failure is a WorkflowValidationException carrying one structured issue per violating location, reusing AgentResult.OutputValidationIssue(errorCode, fieldPath, message) and the existing SCHEMA_MALFORMED/SCHEMA_VIOLATION error codes.

Output boundary, no COMPLETED for a violating output

Before the COMPLETED outcome is delivered — on the single completion path shared by fresh runs and resumes — the output snapshot is validated against the declared output schema. A violation terminates the stream with the same structured exception, which matches the existing contract that errors terminate the stream exceptionally; AgentOutcome itself is unchanged.

Documented boundaries

Resumes do not re-validate inputs: the continuation owns the state and a resume request often carries no inputs, so re-validating would make suspended workflows impossible to finish — output acceptance still applies on resume. Subflows share the parent run's inputs, so a nested definition's input schema is a publish-time contract for its own callers and is not applied again inside the parent run. Both boundaries are covered by tests.

Two pre-existing bugs on the resume path, fixed

The workflow module had no suspend/resume tests, and the issue's acceptance criteria ask for that coverage — writing the tests surfaced two latent bugs. First, ExecutionResult.continueWith wrapped a suspension in a result whose data was null while the suspended-outcome construction reads that data, so any suspension NPE'd immediately; the wrapper now keeps the suspension-point snapshot. Second, resuming invoked the wrapped agent continuation twice (once inside continueWith, once again in the composed step lambda of executeSteps/Loop) — a real agent rejects a second call carrying the same resume token. The composed step now receives the resumed result instead of re-resuming, and the resume tests assert that the agent is invoked exactly once per resume.

Testing

New WorkflowSchemaContractTest with 10 cases: malformed input and output schemas rejected at publish, blank-schema regression, unchanged behavior without schemas, missing required input rejected with zero tool executions, type mismatch reporting the field path, valid input and output completing, output violation failing instead of completing, resume completing without input re-validation, resume output violation still failing, and subflow schema validation at publish without gating parent runs. Run against the unmodified code the suite fails 5F+2E (red, including the issue's reproduction); with the fix the whole argi-workflow-core module is 48/48 green, checkstyle clean.

Fixes #274

@yuluo-yx

yuluo-yx commented Nov 8, 2026

Copy link
Copy Markdown
Member

WorkflowDefinitionValidator.validateSchema() only loads the schema; it does not validate it against the meta-schema. I reproduced successful publication of {"required":"question"}, [], and 123; the invalid required is silently ignored at runtime. Please reject these at publish/build time and add regression coverage. Also fix the codespell failure on objec in WorkflowSchemaContractTest.java:61.

@konlue
konlue force-pushed the feat/issue274-workflow-schema-contract branch from f19c3d8 to 89b150f Compare November 9, 2026 07:27
@konlue

konlue commented Nov 9, 2026

Copy link
Copy Markdown
Contributor Author

Meta-schema validation at publish and build time

Fixed. validateSchema now validates the declared document against the bundled draft 2020-12 meta-schema before compiling it, so a document that is not an object or boolean, or that violates a keyword, is rejected where the definition is validated — which covers both WorkflowRegistry.publish and WorkflowAgent.builder. The three cases you reproduced are now rejected with the offending field name in the message: {"required":"question"}, [] and 123. Plain object schemas, empty objects and boolean schemas still publish.

Regression coverage

WorkflowSchemaContractTest gained a parameterized case that publishes each invalid document as both inputSchema and outputSchema, plus a positive case for object, empty-object and boolean schemas. The codespell failure is gone: the intentionally truncated literal no longer contains the misspelled token.

Verification

Rebased onto the current main (the branch conflicted with the subflow-recursion change in the same validator, both sides are kept). argi-workflow-core module tests pass (62/62, including the 16 contract tests), checkstyle and format validation report no violations, and the red→green transition was confirmed on the old validator — all five invalid documents published successfully before the change.

@yuluo-yx

Copy link
Copy Markdown
Member

WorkflowAgent.continueWith() drops the remaining steps when the resumed child suspends again. Reproduced on this head: child suspends twice, then completes; the workflow returns COMPLETED with the following step never executed. Preserve the continuation across repeated suspensions and add a regression test.

konlue and others added 4 commits November 10, 2026 23:40
…and run boundaries (issue agentic-ai-java#274)

- WorkflowDefinitionValidator: parse non-blank inputSchema/outputSchema with the
  networknt draft 2020-12 registry; blank rejection and no-schema behavior unchanged
- WorkflowAgent: compile declared schemas at construction; validate request inputs
  before the first step (no tool or agent side effects); validate the output snapshot
  on the shared completion path before delivering COMPLETED; failures carry
  AgentResult.OutputValidationIssue and terminate the stream with
  WorkflowValidationException
- ExecutionResult.continueWith: keep the suspension-point snapshot so a suspended
  outcome can be built (pre-existing NPE, untested path)
- boundaries: resumes do not re-validate inputs; subflow input schemas are a
  publish-time contract, not an extra gate inside the parent run
- tests: WorkflowSchemaContractTest, 10 cases (red 5F+2E before, green after)
Self-review of the agentic-ai-java#274 branch found that resuming a suspended workflow
invoked the wrapped agent continuation twice: continueWith already resumed
the continuation, and the composed step lambdas of executeSteps and Loop
resumed it again. A real agent would reject the second call with the same
resume token. The composed step now receives the resumed result through a
ContinuedStep callback, and the resume tests assert a single invocation.
Also fixes the indentation of the runSafely/run block (format:apply).
…s (issue agentic-ai-java#274 review)

Resuming a suspended workflow step that suspends again returned the new
suspension with a bare continuation: the composed step that continues the
remaining steps was dropped, so the run completed without ever executing
what followed. executeSteps and Loop now recompose their continue-after
step onto every repeated suspension (continueAfter / continueLoopAfter),
and two regression tests drive a child that suspends twice — the following
step and the remaining loop iterations both run after the second resume.
@konlue
konlue force-pushed the feat/issue274-workflow-schema-contract branch from 89b150f to 95e2586 Compare November 10, 2026 15:42
@konlue

konlue commented Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

@yuluo-yx Fixed and pushed (95e2586, branch rebased onto the current main first).

Root cause: the composition introduced with the single-resume fix only handled one suspension. executeSteps wrapped the first suspension in a continue-with step, but when the wrapped continuation suspended again, that step returned the new suspension with a bare continuation — the "then continue the remaining steps" work was dropped, and the run completed after the child finally succeeded without ever executing what followed. Your reproduction is exact: two suspensions, then COMPLETED with the next step skipped. The Loop composition had the same hole for both a repeated suspension of the wrapped continuation and a suspension raised inside the remaining iterations.

Fix: the continue-after step is now recomposed onto every repeated suspension instead of being dropped — continueAfter for the step sequence and continueLoopAfter for loops (remaining iterations survive both a repeated suspension of the wrapped continuation and a suspension raised inside the later iterations). The single-resume contract is unchanged: the wrapped continuation still runs exactly once per resume.

Regression coverage: two new tests in WorkflowSchemaContractTest — a child that suspends twice and then completes, asserting the following tool step still executes and the agent is invoked exactly twice across resumes; and a loop body suspending twice, asserting the remaining iterations still run to maxIterations (both red before the fix: the run ended COMPLETED after invocation 3 with the following work skipped). argi-workflow-core is 64/64 green after the rebase, checkstyle and format clean, and the downstream framework regression shows zero failures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG][P2] Workflow 定义的输入输出 Schema 应参与发布校验与运行验收

2 participants