Python semantic layer: semantic_layer() and measure collection - #249
Draft
jat255 wants to merge 10 commits into
Draft
Python semantic layer: semantic_layer() and measure collection#249jat255 wants to merge 10 commits into
jat255 wants to merge 10 commits into
Conversation
_collect()'s list/tuple branch merged harvested source with sources.update(), so within a nested list the last definition of a Python name won; semantic_layer() itself uses setdefault, so the first wins. Use the same rule in both places so source_text is independent of how measures are nested.
jat255
force-pushed
the
jat255/wwmt-layer-collection
branch
from
September 2, 2026 03:14
433b1ce to
fa3ebf0
Compare
…es on import failure
…against name collisions
…port machinery access
…emantic_layer() calls
jat255
force-pushed
the
jat255/wwmt-layer-collection
branch
from
September 2, 2026 04:02
fa3ebf0 to
f5c7fd7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Third of four stacked PRs building the Python semantic layer (M3). Based on #248.
semantic_layer()collects measures from inline decorated functions, bare records, lists, module objects,.pyfile paths, and directories of them. Duplicate names, non-measure items, and missing paths all fail at construction.The layer keeps two mappings, keyed differently on purpose.
measuresis keyed by measure name andsource_textby Python function name. The two differ whenever@measure(name=...)renames a measure, so one map would lose the other. Both are wrapped inMappingProxyType, because a plaindicton a frozen dataclass is still mutable in place. Only text is kept: the worker session reads measure definitions and never receives a callable, which is the security propertypkg-r/R/measures.Rstates in its header.The R package has no equivalent of loading files by path, because
read_measures()sources every measure file into one shared environment and helpers resolve for free. Python modules do not work that way, so a measure file has to import its helpers. Making that true took more than the docstring line originally planned.While loading a file by path, its parent directory goes on
sys.pathand comes off again in afinally, and only if this code put it there. It is appended rather than inserted at the front. Both forms let a sibling import resolve, but prepending also lets ajson.pysitting beside the measures shadow the standard library for everything imported afterwards, which is the usual script footgun and hard to diagnose.Appending removes that hazard and leaves a smaller one, so loading a directory first checks every
.pyfile in it and refuses when a name would collide. The check consultsimportlib.util.find_spec()as well assys.modules, because the case most likely to bite is a package that is installed but not yet imported: without the spec lookup, aduckdb.pynext to the measures passes every other test and then the sibling'simport duckdbsilently resolves to the installed package, leaving the author's helper unreachable with no error.sys.modulesis still consulted alongside it, for a helper another measure directory already loaded under its bare name, which has no discoverable spec.The check, the
sys.pathchange, and module execution are serialized under a reentrant lock. Reentrant because the lock is held while user code runs, and a measure file that constructs another layer during its own import would otherwise deadlock its own thread.Rejected: giving loaded files a synthetic package context so relative imports work. It does not help absolute imports, and relative imports inside a directory that is not a package read strangely.
Not in this PR: injection resolution and the public exports, which are the PR above it.