Skip to content

feat(wrapper-generator): emit -All and nextLink following for list cmdlets - #3746

Merged
Joywambui-maina merged 28 commits into
powershell-v3from
feat/wrapper-pagination-generator
Aug 27, 2026
Merged

feat(wrapper-generator): emit -All and nextLink following for list cmdlets#3746
Joywambui-maina merged 28 commits into
powershell-v3from
feat/wrapper-pagination-generator

Conversation

@Joywambui-maina

Copy link
Copy Markdown
Contributor

Changes proposed:

-Emit an -All switch on every list-shaped cmdlet that follows each non-empty @odata.nextLink to exhaustion, streaming each page to the pipeline before the next request goes out
-Declare the switch on the public dispatcher too, so it reaches the worker — the binder rejects an undeclared parameter before forwarding ever happens
-Cap the total with a bound -Top at whole-page granularity, matching the published ListCmdlet's shipped semantics
-Warn once when a first response still carries a nextLink and -All wasn't supplied — no extra request. This is deliberately stronger than the published SDK, which truncates silently; approved in the design spec
-Continue through kiota's WithUrl, re-applying headers only, so ConsistencyLevel and caller -Headers survive page 2 onward while the link carries its own query state
-Let a pipeline stop (Select-Object -First N, Ctrl+C) pass through the shared catch instead of being re-branded a Graph failure
Record the decisions and their evidence in tools/WrapperGenerator/docs/pagination.md

Joywambui-maina and others added 25 commits August 5, 2026 13:43
Graph declares Edm.Int32/Int64 as "number" with the real type in the
format; mapping by type alone emitted double? against Kiota's int? and
did not compile. An explicit format now decides the CLR type, mirroring
Kiota's own mapping.
…racle audit

Auditing every v1.0 GET command in MgCommandMetadata.json against the
singularizer surfaced four words where the rules disagree with shipped
cmdlet names: Cookies -> "Cooky" (ships as ...HostCookie), Skus kept
as-is (ships as Get-MgSubscribedSku), Dns -> "Dn" (ships as
Get-MgDomainVerificationDnsRecord), Ios -> "Io" (ships as
Get-MgDeviceAppManagementIosManagedAppProtection). Adds two irregulars
and two invariants, each with a pinned test, and refreshes the README
test count.

82 tests passing. Full-inventory match after fix: 796 of 870 noun
segments; the remaining 74 are action/function segments and AutoRest
hand renames, tracked separately.
…nt, start edge-case catalog

Only 2 of 30 shipped whois-family commands truncate "Whois" to "Whoi"; per
review decision the generator emits the corrected ...HostWhois (no alias for
the old name), and the parity gate reports it as [CORRECTED] instead of
failing. "Statistics" joins the invariants, found via the DEVX Humanizer
exception list. edge-cases/naming-edge-cases.md starts the per-class
catalog of naming defects. 88 tests passing.
Build-WrapperModule.ps1 turns one OpenAPI doc into an importable module
(kiota client + wrappers + csproj + dll + PSD1 manifest), reading the
Kiota-compatible docs by default with a hard kiota timeout and per-module
doc fallback. Test-WrapperModule.ps1 imports each build in a fresh pwsh
and verifies exports, worker pairing, and the sessionless NoGraphSession
path. All 35 cmdlet-producing v1.0 modules build and pass.
Compiling all v1.0 modules against freshly generated kiota clients
surfaced eight alignment defects, each fixed and pinned by a test:
dispatchers re-wrapped worker errors (NoGraphSession was lost); body
properties colliding with path ids (published convention: -DeviceId1);
bare model types colliding with namespaces and BCL types (now fully
qualified, mirroring kiota's move-inside and reserved-name renames at
root and in sub-namespaces); collection responses resolved from their
own $ref; underscore members (riskEventTypes_v2 -> RiskEventTypesV2);
$select/$expand emitted only where declared; re-fetch only where a GET
exists; media/content endpoints skipped like $value.
…ant, catalog kiota edge cases

The full-module parity sweep found two shipped-name issues: AutoRest
truncated /places/{id}/checkIns at the preposition (8 commands ship as
*-MgPlaceCheck while Get-MgPlaceCheckInCount keeps "In") - corrected per
policy with gate rows and pinned tests; and "Rights" needs to be an
inflection invariant (Get-MgPrivacySubjectRightsRequest, 42 cmdlets) -
Compliance now matches 23 of 23. New edge-cases/kiota-alignment file
catalogs the compile-found defect classes; README refreshed. 103 tests.
A second operation resolving to an already-written cmdlet file now fails
generation with the full collision list instead of silently overwriting it,
which is the silent-drop failure mode AutoRest had.

OData cast list/item pairs (owners/graph.user) now merge like plain pairs,
and the sweep's collisions land as cited NamingOverrides entries: termStore
and agreement-file stitches, default-singleton renames (SubSite,
DefaultDrive, DefaultCalendarEvent), and nested navs the SDK never shipped.
Remaining families are tracked on #3704.
… data

Derive-CollisionResolutions.ps1 replays the checked-in collision inventory
(212 lines, 365 contested routes) against MgCommandMetadata and emits
exact-match resolution data: 191 suppressions (routes the published SDK
prunes) and 64 renames (published nouns), each entry carrying its oracle
evidence. The files embed into the generator and apply only when
UseCollisionData is set; -Validate fails on drift, and a new xunit test
runs it on every `dotnet test` so staleness fails the suite instead of
depending on someone remembering to run the script by hand. Derivation
itself fails on any unclassified or ambiguous route. Only 2 cross-path
variant merges exist in all of v1.0 (GroupPhoto, ShareListItem) - deferred
with the singleton side kept, cataloged in crosspath-merge-edge-cases.md.

Full 39-module v1.0 generation now produces zero collisions; 20 published
commands that lost filename races are recovered; exact-name matches rise
5,042 -> 5,098.

Also: cmdlets emit into a per-module namespace derived from the client
namespace instead of the leftover MgPoC placeholder; Build-WrapperModule's
generated csproj references Authentication by a relative path instead of
an absolute one; its -Configuration parameter now actually reaches the
wrapper generator's own build, not just the final module build; and a
pre-existing nullable warning in the list/item pairing check is fixed.
121 tests pass.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
fix(wrapper-generator): fail loudly on cmdlet file collisions
Request bodies bound only top-level primitives, so 4,466 property
occurrences across the v1.0 specs had no parameter. Every shape the
classifier reaches now binds: referenced models and enums, formatted
strings, schema-less UntypedNode values (converted on assignment, nulls
dropped to match the published SDK's AddIf), and the numeric INF/NaN
union. The invented -Password pair is replaced by the published
-PasswordProfile. New gates verify it - omission oracle, coverage sweep,
inventory diff, runtime conversions: 0 unbound across all 38 specs, 35
modules build and import, 148 tests. The pre-existing naming-parity gap
is tracked separately.
Both lines carried the same singularizer-era fixes as different commits, so
the five conflicts resolve to identical rules plus each side's real
additions: powershell-v3's docs/edge-cases layout and lean comments win;
the packaging line's Rights invariant, ResolveParameterNameCollisions, and
PlaceCheckIn corrections survive. README test count set to the measured
121; its no-compile-test claim was already false here and is corrected.
- Emit media/content downloads (78 routes), completing the shape list in #3709
- Stamp each cmdlet with a [GraphRoute] attribute so the parity gate reads the
  operation's route from the compiled assembly rather than reconstructing it from
  generated C#, removing the cast and parameterized-function exclusions that left
  1,669 cmdlets unverified
- Order Count before the cast suffix on /$count routes (126 cmdlets)
- Drop the unusable -OutFile parameter from content writes returning an entity
- Collapse the three parallel OData segment tables into one
- Refresh generator docs against measured figures

38/38 modules generate and build, 184/184 tests.
…e}/wrapper

All 38 modules' Kiota clients and generated cmdlets are committed - 11,719
cmdlet files, 9,051 public names - so a clean checkout builds with only the
.NET SDK. The API version nests under wrapper/ because AutoRest clears
src/{Module}/{version} on regeneration; both projects target netstandard2.0
so one dll path serves Core and Desktop. DirectoryObjects no longer
double-declares publicKeyInfrastructure. Gates and docs re-measured against
this corpus: parity 9,548 of 10,385 joinable, omission oracle 0 failures.
…cation binding

Manifests now declare RequiredModules on Microsoft.Graph.Authentication (minimum
read from its csproj) and packages no longer embed the Authentication assemblies:
a module-local copy at a different version splits the GraphSession static under
Windows PowerShell's loader, reporting NoGraphSession while connected.
ModuleVersion now equals the package version - the hard-coded placeholder made
every versioned-folder install refuse to import. Proven by marker-client probes;
package gate passes.
…e base class

Microsoft.Graph.Wrapper.Runtime hosts what every generated cmdlet duplicated:
GraphClientCmdlet owns -AccessToken/-Headers and transport acquisition - the
session path reuses one request adapter keyed to the session HttpClient's
identity (rebuilt on reconnect), the token path shares one HttpClient with
per-request auth headers instead of leaking a connection pool per call.
GraphRouteAttribute, UntypedValue and the bearer provider move here from the
per-module Shared.g.cs. 22 pinned tests; emitted code unchanged until the
emitter adopts the base class in the follow-up.
Every generated cmdlet now derives from GraphClientCmdlet: transport, the
-AccessToken/-Headers surface and Graph error translation come from
Microsoft.Graph.Wrapper.Runtime instead of being repeated per file, and the
per-module Shared.g.cs is gone. The corpus shrinks ~417k lines (30%) with
zero behavioral change: operation inventory unchanged (11,719 = 11,719),
parity identical (9,548 of 10,385), omission oracle 0 failures, 38/38 build
from a clean index snapshot. Also fixes the parity gate picking the runtime
dll for modules sorting after R, and repo-root discovery in git worktrees.
…d Authentication module

The first live-tenant run of a wrapper cmdlet crashed casting kiota's
ObservabilityOptions: module bins carried the whole Authentication dependency
closure (~60 dlls), loading a second identity of each assembly into the default
load context beside the one the installed Microsoft.Graph.Authentication serves
from its isolated context. Bins now carry only the three wrapper assemblies
plus the two kiota dlls Authentication does not ship - Std.UriTemplate
(preloaded via RequiredAssemblies: its requester lives in the isolated context
and cannot probe the module folder) and Serialization.Multipart. PruneModuleBin
removes package assets by package id so kept versions stay derived from the
restore graph; DisableTransitiveProjectReferences and Private="false" stop the
Authentication project copies. Cmdlet discovery and the parity gate now import
Authentication before loading wrapper assemblies - nine cmdlets typed with
kiota's Date/Time structs force those loads at class load, and both probes
silently degraded without the resolver. Proven live against Graph: session,
dispatcher, adapter reuse and 16-way parallel all pass; tools/Test-WrapperLive.ps1
is that gate, kept in the repo because offline gates are structurally blind to
resolution faults that only manifest on a real request. Inventory unchanged
(11,719 = 11,719), parity identical (9,548 of 10,385), oracle 0 failures,
package gate PASS.
The #3712 evidence sweep found four defects between "packages exist" and
"testers can install them": the nuspec declared no dependencies, so a clean
machine got the wrapper without Authentication and import failed
(Install-Module resolves from NuGet metadata, not the manifest); packages
versioned as stable 2.39.0, colliding number-for-number with the real SDK's
release train; the module GUID was random every build, breaking Update-Module
identity across handout iterations; and both checkouts held stale pre-fix
nupkgs under shipping ids. Packages now declare Microsoft.Graph.Authentication
as an open-floor dependency (matching the manifest minimum and the use-latest
ruling), always carry a prerelease label (-Prerelease, default
wrapperpreview01), and derive a stable RFC 4122 name-based GUID from the
module name - no lookup table, identical across builds. Stale artifacts
deleted. Proven three rounds each: 38 of 38 packages carry the dependency,
label and distinct stable GUIDs; Save-Module from a folder repository
auto-resolves Authentication, and the saved layout imports and completes a
live Graph call in a host that can see nothing else; package gate PASS. Also
corrects the Std.UriTemplate comment - the AssemblyRef lives in
Microsoft.Kiota.Abstractions, not the HTTP library - and the live gate's
committed name (tools/Test-WrapperLive.ps1).
…dlets

List cmdlets emitted only the first page and dropped @odata.nextLink, so
scripts silently received partial data. EmitListGet - the single template
behind every list-shaped cmdlet - now emits an -All switch that follows each
non-empty nextLink to exhaustion, streaming every page to the pipeline before
the next request is issued, and the public dispatcher declares the switch so
it reaches the worker. A bound -Top caps the total under -All at whole-page
granularity, matching the published ListCmdlet's shipped semantics. Without
-All a surviving nextLink writes one short warning and costs no extra request
- deliberately stronger than the published SDK, which truncates silently;
approved in the design spec. Continuations go through kiota's WithUrl and
re-apply headers only: the link already carries the query state, and a
raw-URL builder ignores query bindings. A pipeline stop passes through the
shared catch instead of being re-branded a Graph failure. Decisions and
evidence in tools/WrapperGenerator/docs/pagination.md. Proven: 189 generator
tests including five pagination pins, and tools/Test-WrapperPaging.ps1 drives
a real compiled cmdlet and public dispatcher across a stubbed two-page
collection - ten assertions, three rounds, all pass.

The regenerated corpus follows in a separate pull request.
Comment thread tools/WrapperGenerator/CmdletEmitter.cs Outdated
Checking Stopping between pages cannot interrupt a request that is already
running, so a slow or large page ignored Ctrl+C until it completed. The paging
loop now hands StoppingToken to GetAsync.

Cmdlet.PipelineStopToken would be the direct route and IS declared by
PowerShellStandard.Library, but it is absent from the Windows PowerShell 5.1
runtime these netstandard2.0 modules also target - it would compile, pass on
PowerShell 7 and throw on 5.1. StopProcessing is virtual on every supported
edition, so the token is raised there instead.

The emitted catch also had to stop swallowing cancellation, or a stop would
surface as GraphRequestFailed rather than ending the pipeline.
…e catch

The Ctrl+C change altered the emitted catch filter without updating the two
emitter tests that pin its text. 189 of 189 pass again.
@Joywambui-maina
Joywambui-maina force-pushed the feat/wrapper-pagination-generator branch 2 times, most recently from ca3f753 to 40f7b22 Compare August 27, 2026 18:55
Base automatically changed from feat/wrapper-packaging to powershell-v3 August 27, 2026 20:57
@Joywambui-maina
Joywambui-maina dismissed Peter Ombwa (peombwa)’s stale review August 27, 2026 20:57

The base branch was changed.

Post-squash update merge. Content is identical to the verified rebase of this
branch's two commits onto powershell-v3 (patch-identity checked).
@Joywambui-maina
Joywambui-maina merged commit d3b05ec into powershell-v3 Aug 27, 2026
4 checks passed
@Joywambui-maina
Joywambui-maina deleted the feat/wrapper-pagination-generator branch August 27, 2026 21:41
Joywambui-maina added a commit that referenced this pull request Aug 28, 2026
Post-squash update merge; content equals the verified rebase plus the
restoration of #3740 content dropped by the earlier #3746 update merge.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants