You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I generated this issue with an agent that used .github/agents/ado-aw.agent.md.
I reviewed the generated issue and confirm it is being filed directly in githubnext/ado-aw.
Problem summary
There is no route by which a value computed at runtime can reach any work-item field. CreateWorkItemParams is #[serde(deny_unknown_fields)] over {title, description, tags}, and UpdateWorkItemParams exposes only a fixed set (title, body, state, area_path, iteration_path, assignee, tags). The only mechanism that writes arbitrary fields is custom-fields on CreateWorkItemConfig — a compile-time constant map in front matter.
#1924 already named this limitation in passing: "custom-fields does not help: it is a compile-time constant map." That issue was resolved for the specific case of the description field, but the general gap remains.
Concrete impact. Our daily SFF failure-analysis agent runs a skill that computes the release version at analysis time and emits it on each create proposal (Microsoft.VSTS.Build.FoundIn, One_custom.Version). Under dry_run the skill does not write to ADO — it returns proposals the agent replays as create-work-item safe outputs — so those fields are dropped in transit. Result: 67 of 190 auto-filed bugs had a blank Version and were therefore missing from every triage view that filters on it. We backfilled 33 by hand.
The workaround is to hardcode the value in front matter:
That works, but it makes a compile-time constant stand in for a runtime value: the version must now be duplicated in two places and bumped by hand every release, and nothing detects it going stale. It also cannot express anything genuinely per-item — severity, repro rate, or a found-in-build that differs per failure bucket all have a single value per pipeline or none at all.
The failure mode is silent, which is the worst part.deny_unknown_fields means the extra data is discarded with no error surfaced to the agent; the safe-output executor reports success and the pipeline goes green. This is the same shape as #1924 — "created successfully, the executor reports success, and the only way to notice is for a human to open the bug." Nobody noticed for weeks.
Worth noting the same silent drop applies to per-item area_path on create-work-item, and that one cost us more. Every auto-filed bug landed in the project root area, our dedup query scoped candidates by area path and so could never see a bug it had previously filed, and the agent re-filed the same failures every day until someone deduplicated 8 bugs by hand after a single run.
There is consequently no field an agent can populate to carry a computed value, on either tool — the attempt is rejected at the tool boundary rather than failing later.
Setting the value via front-matter custom-fields does reach the work item (confirmed by decoding the compiled lock's toolConfigs blob), but only as a constant fixed at compile time.
Expected: an agent can attach arbitrary work-item fields to the item it is creating or updating, so that values determined during the run reach ADO.
Actual: only title, description and tags survive; everything else must be known when the workflow is compiled.
Proposed next step
Add an optional field map to the agent-facing payloads, merged over the operator's custom-fields (agent value winning, or config winning — either is fine, as long as it is documented):
/// Additional work item fields, keyed by full reference name/// (e.g. "Microsoft.VSTS.Build.FoundIn").#[serde(default)]pub fields: std::collections::HashMap<AdoWorkItemFieldRef,String>,
on both CreateWorkItemParams and UpdateWorkItemParams. AdoWorkItemFieldRef already exists and is already used for custom-fields keys, so the validation and sanitization path is in place.
Our preference is for this to be unrestricted, mirroring how custom-fields behaves today — the operator has already opted into create-work-item, and the agent already controls the title, description and tags of the very same work item, so field writes on an item it is authorized to create seem within the existing boundary.
That said, we recognize this widens what a prompt-injected agent can reach, and #369 shows that is territory you have already audited. If you conclude it needs gating, we would be unblocked by whatever mechanism you consider appropriate — the requirement is only that a value computed during the run can reach a field, not that it be ungoverned.
Related: #1924 (same silent-success failure mode, and the origin of the "compile-time constant map" observation) and #1899 (agent-supplied assigned_to, resolved by giving the agent a way to express per-item intent it previously could not).
Two smaller notes while filing:
The drop being silent is arguably a defect independent of this request. If deny_unknown_fields is intentional, having the executor surface a warning when a proposal carries data it discards would have saved us the entire investigation.
Submission requirements
.github/agents/ado-aw.agent.md.githubnext/ado-aw.Problem summary
There is no route by which a value computed at runtime can reach any work-item field.
CreateWorkItemParamsis#[serde(deny_unknown_fields)]over{title, description, tags}, andUpdateWorkItemParamsexposes only a fixed set (title,body,state,area_path,iteration_path,assignee,tags). The only mechanism that writes arbitrary fields iscustom-fieldsonCreateWorkItemConfig— a compile-time constant map in front matter.#1924 already named this limitation in passing: "
custom-fieldsdoes not help: it is a compile-time constant map." That issue was resolved for the specific case of the description field, but the general gap remains.Concrete impact. Our daily SFF failure-analysis agent runs a skill that computes the release version at analysis time and emits it on each create proposal (
Microsoft.VSTS.Build.FoundIn,One_custom.Version). Underdry_runthe skill does not write to ADO — it returns proposals the agent replays ascreate-work-itemsafe outputs — so those fields are dropped in transit. Result: 67 of 190 auto-filed bugs had a blank Version and were therefore missing from every triage view that filters on it. We backfilled 33 by hand.The workaround is to hardcode the value in front matter:
That works, but it makes a compile-time constant stand in for a runtime value: the version must now be duplicated in two places and bumped by hand every release, and nothing detects it going stale. It also cannot express anything genuinely per-item — severity, repro rate, or a found-in-build that differs per failure bucket all have a single value per pipeline or none at all.
The failure mode is silent, which is the worst part.
deny_unknown_fieldsmeans the extra data is discarded with no error surfaced to the agent; the safe-output executor reports success and the pipeline goes green. This is the same shape as #1924 — "created successfully, the executor reports success, and the only way to notice is for a human to open the bug." Nobody noticed for weeks.Worth noting the same silent drop applies to per-item
area_pathoncreate-work-item, and that one cost us more. Every auto-filed bug landed in the project root area, our dedup query scoped candidates by area path and so could never see a bug it had previously filed, and the agent re-filed the same failures every day until someone deduplicated 8 bugs by hand after a single run.Reproduction details
Environment: ado-aw 0.50.0 (Windows), Azure DevOps Server, Bug work item type.
Configure a
create-work-itemsafe output in agent front matter.Ask the safe-outputs MCP server for its tool definitions:
Observed — the agent-facing schema exposes only:
Repeating with
--enabled-tools update-work-item:There is consequently no field an agent can populate to carry a computed value, on either tool — the attempt is rejected at the tool boundary rather than failing later.
Setting the value via front-matter
custom-fieldsdoes reach the work item (confirmed by decoding the compiled lock'stoolConfigsblob), but only as a constant fixed at compile time.Expected: an agent can attach arbitrary work-item fields to the item it is creating or updating, so that values determined during the run reach ADO.
Actual: only
title,descriptionandtagssurvive; everything else must be known when the workflow is compiled.Proposed next step
Add an optional field map to the agent-facing payloads, merged over the operator's
custom-fields(agent value winning, or config winning — either is fine, as long as it is documented):on both
CreateWorkItemParamsandUpdateWorkItemParams.AdoWorkItemFieldRefalready exists and is already used forcustom-fieldskeys, so the validation and sanitization path is in place.Our preference is for this to be unrestricted, mirroring how
custom-fieldsbehaves today — the operator has already opted intocreate-work-item, and the agent already controls the title, description and tags of the very same work item, so field writes on an item it is authorized to create seem within the existing boundary.That said, we recognize this widens what a prompt-injected agent can reach, and #369 shows that is territory you have already audited. If you conclude it needs gating, we would be unblocked by whatever mechanism you consider appropriate — the requirement is only that a value computed during the run can reach a field, not that it be ungoverned.
Related: #1924 (same silent-success failure mode, and the origin of the "compile-time constant map" observation) and #1899 (agent-supplied
assigned_to, resolved by giving the agent a way to express per-item intent it previously could not).Two smaller notes while filing:
deny_unknown_fieldsis intentional, having the executor surface a warning when a proposal carries data it discards would have saved us the entire investigation.description-fieldfrom [agent-issue]:create-work-itemhardcodesSystem.Description, so Bug work items are filed blank #1924 is merged onmainbut not in v0.50.0, the current latest release (closed 8/20, released 8/14). Not a request, just confirming our read — we are waiting on 0.51.0 for it.