fix(processor): honour json struct tags when unmarshalling YAML - #247
fix(processor): honour json struct tags when unmarshalling YAML#247Git-PrinceNagar wants to merge 2 commits into
Conversation
YAML-authored assets were decoded straight into Azure SDK types that only declare `json` struct tags. gopkg.in/yaml.v3 does not promote anonymously embedded struct fields or honour json tags, so documents such as custom role definitions parsed into empty structs and failed later with a confusing "no name provided" validation error. Route YAML through JSON so both formats converge on the same decoder, and normalize map[any]any keys to strings so non-string YAML keys remain marshalable. Fixes Azure/Azure-Landing-Zones#4225 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #247 +/- ##
==========================================
- Coverage 47.87% 47.75% -0.13%
==========================================
Files 54 54
Lines 4913 4938 +25
==========================================
+ Hits 2352 2358 +6
- Misses 2273 2294 +21
+ Partials 288 286 -2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Add tests for the two error branches introduced in unmarshalYAML: an invalid YAML document, and a document that is valid YAML but has no JSON representation (NaN), which fails at json.Marshal. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
| Code under test | golangci-lint |
goconst findings |
|---|---|---|
origin/main @ 5f18137, unmodified |
v2.11.4 | 0 |
origin/main @ 5f18137, unmodified |
v2.12.2 | 77 |
| this branch | v2.11.4 | 0 |
| this branch | v2.12.2 | 77 |
#246 already pins the version to v2.11.4; once it merges this branch can be rebased. I have left the workflow untouched here to keep the change set limited to the fix.
|
Hi, the library was designed to support YAML for the non-azure resources - e.g. archetypes, etc. If support for yaml for azure resources is desired then it should be extended to all resource types. However this was a deliberate decision to not support this initially. |
Overview/Summary
Custom role definitions authored in YAML are not loaded by
alzlib. All fields aresilently discarded during unmarshalling, and the failure surfaces later as a validation
error that does not indicate the underlying cause:
An equivalent role definition authored in JSON loads correctly, so YAML support is
non-functional in practice despite being documented in
role-definitions.md:Root cause
assets.RoleDefinitionanonymously embedsarmauthorization.RoleDefinition, whichdeclares only
json:"..."struct tags and implements a customjson.Unmarshaler. Itcarries no
yamltags.encoding/jsonpromotes the fields of an anonymously embedded struct automatically;gopkg.in/yaml.v3does not, unless the field is taggedyaml:",inline", and it neitherhonours
jsontags nor invokesjson.Unmarshaler. As a result,yaml.Unmarshalcould notmap any field of the document onto the destination type. The
propertiesblock was neverapplied and every field retained its zero value. Validation subsequently observed an empty
roleNameand returnedErrNoNameProvided— accurate, but reported far from the point offailure.
Fixes Azure/Azure-Landing-Zones#4225
This PR fixes/adds/changes/removes
unmarshalYAMLto route YAML through the JSON decoder. YAML is decodedinto an untyped
any, re-marshalled to JSON, and passed to the existingunmarshalJSONimplementation. Both formats now converge on a single, already-tested decode path, which
prevents them from diverging again.
normalizeYAMLToJSON.yaml.v3decodes a mapping intomap[string]anyonlywhen every key is a string; otherwise it yields
map[any]any, whichencoding/jsoncannot marshal. This helper recursively converts such keys to strings via
fmt.Sprint,mirroring JSON semantics, where object keys are always strings.
the stage at which they occurred.
internal/processor/unmarshaler_test.go, covering the reportedrole definition failure, YAML/JSON parity, non-string keys, and regression cover for
policy definitions, policy set definitions and policy assignments.
Breaking Changes
unmarshalYAMLis shared by all asset processors, so the blast radius of this change wastreated as the primary risk and assessed accordingly.
UnmarshalYAMLimplementations are no longer invoked. Three internal types(
libArchetype,libArchetypeOverride,libArchitecture) implementyaml.Unmarshalerand become unreachable now that YAML is routed through JSON. Their behaviour is preserved
by the JSON path, as evidenced by the existing test suite passing unchanged. They have
been left in place to keep this change set minimal, and can be removed in a follow-up if
maintainers prefer.
parse identically. The change is additive in effect: YAML documents that previously
produced empty structs are now populated correctly.
Testing Evidence
Build, tests, vet and lint
go build ./...go vet ./...go test ./...(all packages)golangci-lint runLint output was compared against a baseline captured on the unmodified tree to confirm that
this change introduces no new findings. The findings are pre-existing and repo-wide:
go-lint.ymlsetsversion: latest, andgolangci-lintv2.12 extendedgoconstto_test.gofiles, so they also appear on unmodifiedmain.End-to-end verification against an ALZ Accelerator deployment
Because the defect manifests during library initialisation rather than in isolation, it was
also verified at deployment level. Three runs were performed against the same Accelerator
configuration, varying only the role definition file format and the library build:
Plan: 1213 to addno name provided,Failed to initialize AlzLibPlan: 1213 to addRuns 1 and 3 produce an identical Terraform plan, establishing that the change restores
parity between the two formats rather than suppressing the error.
As part of this Pull Request I have
mainbranchgo vet,golangci-lint, and an end-to-end Accelerator deployment