Add bpc and BP-AST, the first blueprint with generated sections

[tamnd]

Closes #17.

What this is

BP-AST specifies the node vocabulary: 19 types, 113 concrete node kinds, 198 fields, the C representation of all of it, the arena the tree lives in, and the validation pass that rejects trees the grammar allows. Sections 1, 2, 5 and 8 of it are not typed by anybody. They are compiled out of Parser/Python.asdl by a new tool, tools/bpc.

The reason is that those sections are transcription rather than specification. A table of every field of every node is right on the day it is typed and wrong the first time upstream adds one, and nobody finds out, because a reader who trusted the table has no reason to check it. Where upstream ships the material in a form a program can read, the specification should be generated from it. Parser/Python.asdl is the clearest case of that in CPython, and it is the same file CPython generates its own node structs, its C constructors and its Python classes from.

How it works

blueprints/sources/BP-AST.md holds the prose with a one line directive where each generated block belongs:

<!-- bpc: nodes -->

uv run bpc build swaps each directive for its block and writes blueprints/BP-AST.md. Both files are committed. The output is what people read and what bpcheck and refcheck lint, and it keeps the boundary visible:

<!-- bpc:begin nodes -->
...generated...
<!-- bpc:end nodes -->

That is what makes "no hand written content in a generated section" something anybody can check rather than something everybody has to remember. just build-blueprints rebuilds, just blueprints now runs bpc check as well as bpcheck lint and fails if what is committed has drifted. It reports rather than repairs, because a checker that silently fixes what it finds checks nothing, and the diff is the thing somebody is supposed to read before the pin moves.

Section 1 is the one exception to "the whole section is generated". Its scope prose is hand written and its table is generated, because where this blueprint stops and BP-PARSER starts is not in the ASDL and never will be. The document says that in the section itself rather than leaving it for a reader to work out.

Where the line numbers come from

bpc imports CPython's Parser/asdl.py from the pinned checkout rather than parsing the grammar itself. A second ASDL parser would be a second opinion about what the grammar means, and the reason to generate this material at all is that there should be one.

What asdl.py does not give back is where anything was written: it parses to a tree of Module, Type, Constructor and Field with no line numbers anywhere. So model.py runs asdl.py's own tokenizer a second time, which does carry them, and walks the two in step. A definition is found by looking for a type name followed by =. That is not a detail: a plain forward scan for the first arg token lands on line 116, where arg is the type of three fields of arguments, rather than line 119 where it is declared, and every citation for the type would have pointed at another type's field list. There is a test named after exactly that.

The result is 145 generated citations, each pointing at a single line with the cited name on it. If upstream moves a definition, just citations fails instead of quietly pointing at whatever moved into that slot.

Two things the generator nearly lost

expr?*. Two fields in the grammar carry both quantifiers: Dict.keys and arguments.kw_defaults. asdl.py sets Field.seq and Field.opt from the last quantifier only, so both arrive looking like plain sequences, and the ? survives in Field.quantifiers and nowhere else. A port that reads seq and opt types them as lists of expressions and then crashes on the first {**d}, because a None key is how dictionary unpacking is written and there is no node for it. Field.marks keeps the full list, the tables say "sequence of optional" for those two rows, and section 6.7 explains what the gaps mean in each.

The attributes. They do not behave the way section 5 originally said. Attributes are never required by the constructor, so a node can always be built without them. The two declared int?, end_lineno and end_col_offset, default to None like any other optional. The two declared int, lineno and col_offset, have no value at all, and reading one raises AttributeError. Nothing complains until compile sees the tree, which is where a missing line number becomes TypeError: required field "lineno" missing from stmt. That is where INV-AST-008 is actually enforced, and it is not where a reader expects, so the document now says so and two tests hold it up.

Same for the field defaults, which were checked against the running interpreter rather than assumed: required raises TypeError naming the field, optional is None, a sequence is a new empty list per node, and expr_context is the Load singleton.

The conformance tests

tools/bpc/tests/test_bpc_conformance.py reads the pinned grammar and compares it against the ast module of the interpreter running the tests. Every type and every constructor is a class, _fields and _attributes are the grammar's names in the grammar's order, and leaving a field out does what section 5 says.

Section 8 of the blueprint names those tests by their function names, and there is a test in test_bpc_render.py that reads the generated section 8 and asserts every function it names exists in the file it says it is in. A conformance section pointing at a test nobody can run is the failure this whole arrangement exists to avoid.

They skip when the running interpreter's version does not match the pinned tag, on the same reasoning refcheck already uses: a difference between v3.15.0rc1 and whatever else is installed is a fact about the two versions, not a failure of the document.

Also in here

tools/bpc/README.md explains the directives, the line number walk and why check does not repair. blueprints/README.md gains the BP-AST row the index rule requires plus a section on what generating a blueprint section is and is not worth doing for. The root README.md gains a bpc row and a BP-AST row.

CI runs bpc check in the citations job, which is the one with a CPython checkout, and the tree independent bpc tests in the blueprints job.

Checks

just check is green. 1188 passed and 3 skipped, up from 1115 passed and 2 skipped on main, so 73 new tests. 535 citations resolve in 4 roots, up from 352. 3 blueprints lint clean and 1 is up to date against its source.

Not in this PR

BP-AST is Status: partial, which is honest. Section 3 gives three algorithms and there are 113 constructors; the one shown is the shape they all share. Sections 6 and 9 are written from reading the source rather than from having ported it. Both get revisited when BP-PARSER and BP-CODEGEN land and the boundaries get tested by something other than prose.