Skip to content

Add nbversion, which finds the cells that differ between 3.14 and 3.15 - #72

Merged
tamnd merged 1 commit into
mainfrom
nbversion-tool
Aug 29, 2026
Merged

Add nbversion, which finds the cells that differ between 3.14 and 3.15#72
tamnd merged 1 commit into
mainfrom
nbversion-tool

Conversation

@tamnd

@tamnd tamnd commented Aug 29, 2026

Copy link
Copy Markdown
Owner

The lessons are written against the pinned 3.15. Every reader who clicks a Colab badge is on 3.14, and so is every widget that runs in the browser, because Pyodide has not shipped 3.15 yet. Most cells do not notice. Some do, and those are the dangerous ones: the cell still runs and still prints something that looks right.

The one that started this is LOAD_COMMON_CONSTANT. On 3.15 a function that falls off the end loads None with that instruction and co_consts does not contain None at all. On 3.14 it is a LOAD_CONST and None is in the table. T01 says "look, co_consts is (6,)" and a reader in Colab sees (6, None), with nothing on the page to tell them which of the two of them is wrong.

This is the first half of #2: the mechanism, not the annotations.

What is here

nbversion record runs every lesson on the interpreter it is invoked with and writes one small JSON file per notebook, mapping cell id to normalised output. It is not an executed notebook, because the diff of two executed notebooks is mostly metadata and unreadable. Cells are keyed by id rather than position, so inserting a cell does not report every cell after it as changed.

nbversion compare reads two of those directories and gives one of four verdicts per cell.

verdict what it means fails
declared the cell differs and the notebook says so no
undeclared the cell differs and nothing says so yes
stale the cell carries a note and the two interpreters now agree yes
missing the two runs saw different sets of cells yes

Both directions are checked on purpose. stale is the one that is tempting to leave out, and a note that has stopped being true is worse than no note at all: a reader who checks one against their own interpreter, finds it wrong, and decides the notes are decoration has been misled by the thing that was meant to help them.

Declaring a difference is one keyword in the lesson's build.py:

lesson.code(source, differs="On 3.14 the last instruction is LOAD_CONST rather than LOAD_COMMON_CONSTANT, and None is in co_consts.")

That writes the sentence into the cell's own metadata, under a cpython_internals namespace so it cannot collide with Jupyter or Colab, and adds a markdown cell underneath so a reader sees it without opening the metadata. The note lives on the cell rather than in a list somewhere else in the repository, because a list drifts: delete the cell and the entry stays behind. quiet=True skips the visible half, for the lessons where one paragraph near the top already explains a difference that a dozen cells then show.

The normaliser

This is the part the whole tool rests on, so it is deliberately timid. Every substitution throws away a real difference, and the differences worth finding are exactly the ones a careless normaliser sweeps up. The rule is that a pattern is normalised only when it varies between two runs of the same interpreter, which makes it noise rather than a version difference. That is addresses, absolute paths, temporary file names and durations, and nothing else. Opcode names, sizes, byte counts and jump offsets all survive, because those are the point. There are tests asserting each of those is left alone.

Errors keep the exception type and the message and lose the traceback, because a lesson that raises on purpose cares about which exception it got, not about how many frames were on the stack.

What it found

Running it locally right now, against the twelve committed lessons:

12 notebook(s) compared: 64 undeclared

Twelve of those are the build banner, which prints the version and is supposed to differ. The rest are real, and several are worth a paragraph of prose rather than a one line note:

  • LOAD_COMMON_CONSTANT instead of LOAD_CONST for a trailing None, which also changes co_consts and the byte count of every code object in T01, T05, T06 and T10.
  • 3.15 gives RESUME and GET_ITER a cache entry, so every offset in T06 and T07 is 2 to 4 higher than on 3.14, and the jump arithmetic worked through in T06 comes out different.
  • The small integer cache is -5 to 1024 on 3.15 and -5 to 256 on 3.14, which is the central observation of two cells in T08.
  • sys.monitoring has 17 events on 3.15 and 12 on 3.14.

None of that is annotated in this PR. Adding the notes changes every lesson and renumbers cells, so it is its own change, along with the CI job that makes the check gating.

Policy

CONTRIBUTING.md gets a section saying what to do when a cell differs, including the case the issue asks about: when the differing cell is the lesson's central observation, a note is not enough. Either the lesson gets a short section explaining both versions, because the difference is itself worth teaching, or the example changes to one that behaves the same on both. LOAD_COMMON_CONSTANT is interesting and gets explained. A line number inside asyncio is not, and the cell should stop printing it.

Checking

just versions   # record on both interpreters, then compare
just test       # 77 new tests, none of which need two interpreters

just versions is not part of just check, because it executes every notebook twice and takes minutes. CI will get it for free in the next change, since the notebooks job already runs on both versions and only has to keep its recording.

The lessons are written against the pinned 3.15. Every reader who clicks a Colab badge is on 3.14, and so is every widget that runs in the browser, because Pyodide has not shipped 3.15 yet. Most cells do not notice. Some do, and those are the dangerous ones: the cell still runs and still prints something that looks right.

`nbversion record` runs the lessons on one interpreter and writes a small JSON file per notebook, cell id to normalised output. `nbversion compare` reads two of those and gives one of four verdicts per cell: declared, undeclared, stale or missing. Undeclared and stale both fail, because a note that has stopped being true is worse than no note.

Declaring a difference is one keyword on the builder, `lesson.code(source, differs="...")`, which writes the sentence into the cell's metadata and adds a markdown note underneath so a reader on Colab sees it. `quiet=True` skips the visible half, for the lessons where one paragraph up top covers a difference a dozen cells then show.

The normaliser is deliberately timid. Every substitution throws away a real difference, so a pattern is only normalised when it varies between two runs of the same interpreter: addresses, absolute paths, temporary names and durations. Opcode names, sizes, byte counts and offsets all survive, because those are the point.

No lessons are annotated here and there is no CI job yet. Running it locally today reports 64 undeclared cells across the twelve lessons, which is the next change.
@tamnd
tamnd merged commit f5b44ae into main Aug 29, 2026
8 checks passed
tamnd added a commit that referenced this pull request Aug 29, 2026
…73)

* Declare every lesson cell whose output depends on the Python version

Second half of #2. #72 built the tool and left the lessons unannotated,
because annotating them touches all twelve.

`nbversion compare` now reports 64 declared cells and nothing else, so the
`versions` job in CI can gate on it. The job hangs off the notebooks matrix:
each leg records what its interpreter printed and uploads it, and one job
downloads both and compares.

Four differences account for most of it, so their wording lives in
nbbuild.notes and every lesson that hits one says the same sentence. On 3.15
the implicit `return None` is a LOAD_COMMON_CONSTANT and None is not in
co_consts. RESUME and GET_ITER carry an inline cache in 3.15 and not in 3.14,
which moves every offset by two to four. The shared range of small integers
stops at 1024 rather than 256. And several cells count files in the reader's
own installation, which is a build choice rather than a version.

Notes are quiet where a paragraph nearby already explains the difference, and
visible where the cell is the only one that hits it. T05, T06, T07 and Z02
gained that paragraph. Thirteen cells carry a visible note. Twelve of the
quiet ones are the banner, which prints the caveat itself.

The three interesting judgement calls. T08's central observation is the small
integer cache, so the lesson measures it rather than asserting a number, and
both answers are correct on the interpreter that gives them. T07's stack chain
prints line numbers from asyncio and Jupyter, which belong to whatever is
running the notebook, so the note says to read the bottom of the list. And
`sys.monitoring` has 17 local events on 3.15 and 12 on 3.14, which the lesson
already discussed in prose before this check existed.

* Split the note in two: differs is checkable, varies is not

CI found two stale notes that are not stale. On a runner both interpreters
come from the same builder, so a cell that prints the build's configure flags
prints the same thing twice and the check concluded the note had stopped
being true. On a laptop with a framework Python and a uv-managed one it
differs, and the note is right there. Neither machine is wrong, and comparing
two recordings cannot tell you which.

So there are two keys now. `differs=` is a claim about the language: this
prints one thing on 3.14 and another on 3.15. Two recordings can check it, and
it fails when it stops being true. `varies=` is a claim about the reader's
machine: the flags it was built with, how many files are in its standard
library, how deep the C stack goes. It reads identically to a reader and the
comparison reports it without judging it, because the judgement would depend
on which two machines happened to make the recordings.

Ten cells moved to `varies=`. Locally the comparison now says 53 declared and
11 noted with nothing failing, and on a runner where the two builds agree the
same eleven still pass.

`Lesson.code` refuses a cell that is both, since guessing which one the author
meant would put the wrong thing in the metadata.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant