Type Python projection outputs as non-null by default - #190
Open
leileizhang (lei9444) wants to merge 12 commits into
Open
leileizhang (lei9444) wants to merge 12 commits into
leileizhang (lei9444) wants to merge 12 commits into
Conversation
Output annotations (method and property results, out tuples, async results, returned collection elements and collection-class items) were rendered by four overlapping recursive helpers that each decided `| None` from the type alone, with no notion of position, member facts or target file. Add python/nullability.rs with the AnnotationSurface (runtime .py vs .pyi stub), OutputPosition and OutputSite model and output_admits_none, the only place that decides whether an output admits None. Replace py_return_type_safe, py_output_type, py_return_type, py_async_return_type, py_array_return_type and py_collection_return_type with one recursive py_output_annotation that separates spelling from nullability, and thread the surface through the .py and .pyi call sites. Callback parameters keep the legacy py_return_type_safe wrapper. No generated output changes: a 40-namespace Windows.winmd corpus (26,940 .py and 27,022 .pyi files) is byte-identical before and after. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
WinRT metadata has no nullability and most APIs raise instead of returning null, so the .pyi stubs now type received values as non-null by default, like the JavaScript declarations: method and property results, async results, out values, and returned collections with their elements, keys and values. The stub arm of output_admits_none keeps `| None` for IReference<T> values everywhere, for the Return/OutParam/AsyncResult/activation results of members whose CLR name is Try followed by an uppercase letter, for Object values, and for delegate-typed values outside callback parameters. Runtime .py annotations stay pessimistic, and inputs, struct fields, implementation protocols and callback parameters are unchanged. On a 40-namespace Windows.winmd corpus every .py file is byte-identical and 40,947 stub annotations lose `| None`, all at output positions. A natural consumer script goes from 30 pyright / 29 mypy errors to 0. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Remove the `if x is None: raise` guards in five stock-Windows samples where the values are now typed non-null, and keep the `Try*` guard in the OCR sample. With the previous stubs the unguarded samples produce 27 mypy --strict and 25 pyright errors; with the new stubs both report none, and every sample still runs. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Mixed-language test coverageWorkflow status: ✅ Passed
|
Review of the stub policy found two gaps. Some Windows SDK members are documented to return null, such as Accelerometer.GetDefault and DispatcherQueue.GetForCurrentThread. scripts/extract-null-results.py derives their doc comment IDs from MicrosoftDocs/winrt-api at a pinned commit. It collects returns and property-value sections that say the result can be null, and remarks that say the member itself returns null, then applies reviewed overrides. api-docs/windows-null-results.txt holds only the api-ids (933 members). Metadata parsing records the fact on each method, using the interface definition and the class chain that the documentation lists members under. The stub policy keeps `| None` on those members' results. Anyone can store null in a mutable IVector, IMap or observable collection, so its elements, item positions and element-reading members (GetAt, Lookup, GetMany) keep `| None`. Read-only views, iterators and arrays stay non-null. The element rule is a separate branch of the policy. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Inherited MutableSequence and MutableMapping mutators (append, extend, update, setdefault) take the element type of the collection base, so they accept None only while mutable collection bases keep `| None`. Lock that in with a strict mypy consumer of an observable vector and a JSON object. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The pinned Windows API docs require callers to check several sensor GetCurrentReading results for null, phrased as "must first check that the value is not null". Teach the extractor to recognize that return-value instruction without treating generic negations, input checks, or null holders as nullable results. Regenerate the table and add focused extractor and generated-stub regressions across all sensor variants. A real native IVector containing a null slot exposes None through its vector, view, and iterator wrappers, so reference collection reads are conservative for every collection interface; map keys and value types stay non-null. Align mutable collection writes with those annotations. A shared generated helper converts None reference elements and map values to a null DynWinRTValue, while None map keys and value-type elements raise TypeError. Cover vector create/append/insert/index/slice/extend and map update/setdefault/index writes with native read-back tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Collection item wrapping can recurse into IReference<T> inside iterable, vector, map, or array inputs. Detect those nested shapes when deciding whether to emit _dynwinrt_box_reference, with scalar nested collections as a negative control. The generated-module regression checks the nested calls and the runtime-backed collection test imports and invokes the emitted helper. Run that focused runtime test as a separate e2e-runtime step after installing the matching Python wheel. Keep the existing implementation selector unchanged so other focused probes can compose without conflicts. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
IMapView.Split succeeds while returning two null interface pointers in the runtime. Record that exact two-slot output contract in MethodMeta and keep only those two generated Python results optional. Map key nullability now follows the declared ABI type. Object, interface, runtime-class, delegate, and parameterized-interface keys accept a real null DynWinRTValue and read back None through mappings and key-value pairs. String, Guid, scalar, enum, struct, and unsupported async key shapes remain non-null and fail before type-specific conversion. Cover generated Object and IStringable maps, String/Guid controls, ABC contains/equality/hash behavior, key-value-pair iteration, view lookup, and native Split returning (None, None). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The runtime array converters preserve null slots as None for runtime classes, interfaces, Object, delegates, and parameterized references. Apply that same reference-element policy to ordinary method/out/async array results while keeping the array object itself and String/Guid/value/byte elements non-null. Add generated source/stub and strict mypy/pyright regressions, plus matching wheel native methods returning Resource[], IItem[], and Int32[] controls. Route the live PropertyValue inspectable-array E2E through generated IPropertyValue.get_inspectable_array rather than manual ABI decoding, and run the focused native test in e2e-runtime. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Exercise generated IPropertyValue projection and implementation paths for value and nullable object arrays so the reference-array E2E contributes meaningful coverage to the generated module it introduced. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Allocate the collection-item runtime helper through PythonSupportSymbol so metadata declarations keep their names while generated imports and every collection conversion share a collision-safe alias. Add custom-WinMD packaged and standalone runtime regressions for vector/map None writes, factory rendering, strict typing, and an old-output TypeError negative control. Run the native selector against the matching wheel in CI. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
.pyistubs typed almost every reference output as optional, forcing unnecessary guards.Key changes
| NoneforIReference<T>,Try*,Object, delegates, and Windows SDK members documented as nullable.GetCurrentReadingdocs.Nonecollection inputs to null WinRT values, and rejects invalid keys/elements withTypeError.IMapView.Splitout values as optional without widening other map-view results.Notes
.pyreturn annotations remain pessimistic; collection input signatures now reflect safeNonewrites.Microsoft.*; pyright strict may flag now-redundantis Nonechecks.