From 4de8e9047681cefb607617a42eb611b1b49023c4 Mon Sep 17 00:00:00 2001 From: Pascal Garber Date: Fri, 4 Sep 2026 01:01:14 +0200 Subject: [PATCH 1/8] Learn: fix entities Pango cannot render MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three tutorial paragraphs are currently blank in the GNOME app. `–` and `×` are HTML entities; the tutorial's `` is Pango markup, and GMarkup resolves only `& < > " '` plus numeric character references. Measured on Pango 1.57: `pango_parse_markup` fails with "entity name is not known", and a `Gtk.Label` whose markup fails to parse renders as an empty string — so the reader loses the whole paragraph, with no error logged anywhere. Write the characters themselves, which parse fine and are what the committed catalogs already carry as the msgid: the emitter has been out of step with the POT since the entities appeared, and nothing noticed because the POT is committed and only regenerated by hand. Claude-Session: https://claude.ai/code/session_01URLYj1wgGxVNRbBpNT8xAV --- packages/learn/tutorial.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/packages/learn/tutorial.mdx b/packages/learn/tutorial.mdx index 71305abb..4b3d8286 100644 --- a/packages/learn/tutorial.mdx +++ b/packages/learn/tutorial.mdx @@ -648,10 +648,10 @@ with `00000011` will be `00000010`. The effect of this is to mask out the least significant two bits of the accumulator, setting the others to zero. This converts a number in the range of -0–255 to a number in the range of 0–3. +0–255 to a number in the range of 0–3. After this, the value `2` is added to the accumulator, to create a final random -number in the range 2–5. +number in the range 2–5. The result of this subroutine is to load a random byte into `$00`, and a random number between 2 and 5 into `$01`. Because the least significant byte comes @@ -765,7 +765,7 @@ The next bit updates the head of the snake depending on the direction. This is probably the most complicated part of the code, and it's all reliant on how memory locations map to the screen, so let's look at that in more detail. -You can think of the screen as four horizontal strips of 32 × 8 pixels. +You can think of the screen as four horizontal strips of 32 × 8 pixels. These strips map to `$0200-$02ff`, `$0300-$03ff`, `$0400-$04ff` and `$0500-$05ff`. The first rows of pixels are `$0200-$021f`, `$0220-$023f`, `$0240-$025f`, etc. From a8a07423ff0e1eb9895a7dbc30881350518643ae Mon Sep 17 00:00:00 2001 From: Pascal Garber Date: Fri, 4 Sep 2026 01:01:21 +0200 Subject: [PATCH 2/8] Learn: strip trailing newline in NS code blocks `endsWith("\\n")` tests for a backslash followed by an `n`, not for a newline, so the NativeScript target never trimmed the trailing newline that the GTK and HTML targets both strip. Every block of code in the Android tutorial therefore ended with a blank line, and the same literal differed between the three artifacts generated from one MDX source. Claude-Session: https://claude.ai/code/session_01URLYj1wgGxVNRbBpNT8xAV --- .../learn/tsx/components/nativescript/ns-code.compontent.tsx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/learn/tsx/components/nativescript/ns-code.compontent.tsx b/packages/learn/tsx/components/nativescript/ns-code.compontent.tsx index bc0cffcf..ce2454ac 100644 --- a/packages/learn/tsx/components/nativescript/ns-code.compontent.tsx +++ b/packages/learn/tsx/components/nativescript/ns-code.compontent.tsx @@ -248,7 +248,7 @@ export class NsCode extends Component { codeContent = renderSSR(codeContent); } - if (codeContent.endsWith("\\n")) { + if (codeContent.endsWith("\n")) { codeContent = codeContent.slice(0, -1); } From 04772599f651649c7f56aef58164aaaf55341628 Mon Sep 17 00:00:00 2001 From: Pascal Garber Date: Fri, 4 Sep 2026 01:01:34 +0200 Subject: [PATCH 3/8] Learn: check the generated tutorial artifacts MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit One MDX source is rendered into three unrelated artifacts — `dist/*.ui` for app-gnome, `dist/*.ns.xml` for app-android, `dist/*.html` for app-web — and nothing tested any of them. Only app-gnome fails loudly on a broken one, and it fails at runtime, in whatever paragraph the reader happens to open. `check.js` asserts structural properties rather than bytes: a snapshot of a 136 KB generated file is a diff nobody reads and everybody regenerates, so it stops being evidence the first time someone reformats the emitter. It checks that all six artifacts were written; that the `.ui` and `.ns.xml` are well-formed XML with the expected root and only elements and object classes their consumer can resolve (`Gtk.Builder` and NativeScript's `Builder.load` both refuse the whole document otherwise); that every translatable label carries its TRANSLATORS comment, which is a translator's only context given `noLocation`; that every label is markup Pango accepts; and that the same code literals reach all three targets, each of which encodes them differently. Written against the two defects it found on main, both fixed in the preceding commits: the `–`/`×` entities that blank three paragraphs, and the trailing newline the NativeScript target kept. 19 self-test cases run before any artifact is read, so the rules cannot rot into a check that passes over output nobody has looked at. `check` rebuilds `dist/` first, deliberately: validating whatever artifact happens to be on disk is the same defect that emptied the catalogs on 2026-09-03. Named `check` to match `@learn6502/translations`, so `gjsify foreach -t check` and the CI type-check job pick it up. Claude-Session: https://claude.ai/code/session_01URLYj1wgGxVNRbBpNT8xAV --- .github/workflows/ci.yml | 9 + AGENTS.md | 18 ++ packages/learn/README.md | 39 ++- packages/learn/check.js | 459 ++++++++++++++++++++++++++++++++++++ packages/learn/package.json | 3 + 5 files changed, 524 insertions(+), 4 deletions(-) create mode 100644 packages/learn/check.js diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 395fc8f4..ad33dac5 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -102,6 +102,15 @@ jobs: gjsify workspace @learn6502/examples build gjsify workspace @learn6502/learn build + # The tutorial is rendered into three unrelated artifacts and only + # app-gnome fails loudly on a broken one — at runtime, in whatever + # paragraph the reader opens. This validates all three. It rebuilds + # `packages/learn/dist` itself rather than trusting the step above: + # checking whatever artifact happens to be on disk is the defect that + # emptied the catalogs on 2026-09-03. + - name: Validate generated tutorial artifacts + run: gjsify workspace @learn6502/learn check + - name: Type check (gjsify tsc under GJS) run: | gjsify workspace @learn6502/core check diff --git a/AGENTS.md b/AGENTS.md index f2591f6e..de2ce983 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -317,6 +317,24 @@ deployed to GitHub Pages); dev = Vite + HMR (`vite.app.config.ts`). The classic skilldrick Jekyll tutorial was removed in the app-web rewrite. Focus: browser compatibility, responsive design, leverage `core` + `common-ui`. +## Learn — tutorial content + +Applies to `packages/learn/` (`tutorial.mdx`, `quick-help.mdx`, `tsx/`). + +One MDX source, three artifacts, three consumers: `dist/*.ui` (app-gnome, via `Gtk.Builder`), +`dist/*.ns.xml` (app-android) and `dist/*.html` (app-web) — plus `packages/translations`, which +extracts the tutorial's translatable strings from `dist/*.ui`. + +**HTML entities are not markup here.** `Gtk.Label` renders an empty string when its markup fails +to parse, and Pango knows only `& < > " '` plus numeric references. Write the +character itself — `–`, `×`, a real non-breaking space — never `–`, `×` or ` `: +those cost the reader the entire paragraph, silently. + +**Validate:** `gjsify workspace @learn6502/learn check` (rebuilds `dist/`, then checks XML +well-formedness, the element vocabulary, TRANSLATORS comments, label markup and code literals +across all three targets — also run in CI). Details in +[packages/learn/README.md](packages/learn/README.md). + ## Translations Applies to all `.po` files in `packages/translations/`. diff --git a/packages/learn/README.md b/packages/learn/README.md index d7520e03..05e30dda 100644 --- a/packages/learn/README.md +++ b/packages/learn/README.md @@ -8,11 +8,18 @@ The main tutorial content is stored in `tutorial.mdx`, which is based on Nick Mo ## Transformation Capabilities -The package includes tools to transform the MDX content into different formats: +Each MDX document is rendered into one artifact per platform, and each artifact has exactly one +consumer: -- **GNOME Application**: The content is transformed into GNOME Blueprint UI files (`.ui`) for use in the native GNOME application -- **Web Version**: Planned support for generating web-compatible content -- **Android App**: Potential future support for Android application content +| Artifact | Consumer | Loaded by | +| ----------------------- | ----------- | ----------------------------------------------------- | +| `dist/*.ui` | app-gnome | `Gtk.Builder`, as a `MdxView` template | +| `dist/*.ns.xml` | app-android | NativeScript `Builder.load` (copied to `app/mdx/`) | +| `dist/*.html` | app-web | imported by the web tutorial view | + +`packages/translations` is a fourth consumer: its `xgettext` run extracts the tutorial's +translatable strings from `dist/*.ui`, which is why `@learn6502/translations`'s build builds this +package first. ## Development @@ -32,6 +39,30 @@ gjsify run build This will generate the necessary output files in the `dist/` directory. +### Checking + +```bash +gjsify workspace @learn6502/learn check +``` + +`check.js` rebuilds `dist/` and then validates the generated artifacts structurally — no byte +snapshot, so it survives reformatting the emitter but still fails on the changes that cost a +reader something. CI runs it on every pull request. It asserts that: + +- all six artifacts were written; +- the `.ui` and `.ns.xml` are well-formed XML with the expected root, and contain only elements + and object classes their consumer can resolve — `Gtk.Builder` and NativeScript's `Builder.load` + both refuse the whole document otherwise; +- every translatable label carries its `TRANSLATORS:` comment, which is the only context a + translator gets (the catalogs are generated with `noLocation`); +- every label is markup Pango accepts. A `Gtk.Label` whose markup fails to parse renders as an + empty string, so an HTML-only entity such as `–` costs the reader the whole paragraph — + write the character itself (`–`, `×`) in the MDX; +- the same code literals reach all three targets. They are what the reader retypes into the + editor, and each target encodes them differently (``, an escaped `w:SourceView`, ``). + +It says nothing about whether the tutorial is *correct*; that still needs a reader. + ## License - The tutorial content is licensed under the [Creative Commons Attribution 4.0 International License](https://creativecommons.org/licenses/by/4.0/) diff --git a/packages/learn/check.js b/packages/learn/check.js new file mode 100644 index 00000000..64819a34 --- /dev/null +++ b/packages/learn/check.js @@ -0,0 +1,459 @@ +/** + * Structural validation for the generated tutorial artifacts. + * + * One MDX source is rendered into three unrelated targets — `dist/*.ui` for + * app-gnome, `dist/*.ns.xml` for app-android, `dist/*.html` for app-web — and + * nothing downstream reads all three. A renderer change that breaks only one of + * them therefore reaches `main` looking green: the GNOME app is the only + * consumer that fails loudly, and it fails at runtime, in whatever paragraph + * the reader happens to open. + * + * The rules below are structural on purpose. A byte snapshot of a 136 KB + * generated file is a diff nobody reads and everybody regenerates, so it stops + * being evidence the first time someone reformats the emitter. What is asserted + * here is what the three consumers actually require, and every rule names the + * damage it prevents. + * + * Every markup verdict was measured rather than inferred: `Pango.parse_markup` + * on GTK 4 / Pango 1.57 rejects `–` and `×` as unknown entities, + * and a `Gtk.Label` whose markup fails to parse renders as an empty string — a + * whole tutorial paragraph gone, with no error anywhere. + * + * Run via `gjsify workspace @learn6502/learn check`, which rebuilds `dist/` + * first: a check that reads whatever artifact happened to be lying around is + * the same defect it exists to catch. + */ + +import { existsSync, readFileSync } from "node:fs"; + +// The package directory, which is where `gjsify workspace` runs a script from. +// Deliberately not derived from `import.meta.url`: the `--app gjs` target runs +// a bundle written to `dist/`, so a script-relative path would resolve one +// directory too deep and the run would report success having read nothing. +const DIST = "dist"; + +/** The MDX documents rendered by `tsx/index.tsx`, each into all three targets. */ +const DOCUMENTS = ["tutorial", "quick-help"]; + +/** + * The element vocabulary each target may contain. + * + * These are allow-lists, not descriptions: a new element in a generated file is + * a claim that the consumer can render it, and that claim only becomes true + * once someone has taught the consumer about it. app-gnome loads the `.ui` + * through `Gtk.Builder`, which fails the whole view on an object class it + * cannot resolve, and NativeScript's `Builder.load` does the same for the + * `.ns.xml`. So growing this list is a deliberate act, paired with the widget + * that handles the new element. + */ +const VOCABULARY = { + ui: { + root: "interface", + elements: new Set(["interface", "requires", "template", "child", "object", "property", "style", "class"]), + // `SourceView` is app-gnome's own widget (`packages/app-gnome/src/widgets`), + // not a GTK class — it resolves because the app registers its GType before + // the builder runs. + objectClasses: new Set(["GtkBox", "GtkLabel", "SourceView"]), + }, + ns: { + root: "StackLayout", + elements: new Set(["StackLayout", "HtmlView", "w:SourceView"]), + }, +}; + +/** + * Tags a `Gtk.Label` may contain, and the attributes each one accepts. + * + * This is Pango's vocabulary rather than HTML's, and deliberately narrower than + * what Pango would tolerate: it is the same list `packages/translations/check.js` + * enforces on the other side of the pipeline, so a fragment that survives + * extraction into the catalogs is one translators are allowed to keep. + */ +const LABEL_TAGS = new Set(["a", "b", "big", "i", "s", "small", "sub", "sup", "tt", "u"]); +const LABEL_TAG_ATTRIBUTES = { a: { allowed: new Set(["href", "title"]), required: ["href"] } }; + +/** + * The entity references both XML and GMarkup resolve, plus numeric ones. + * + * Neither parser knows the HTML entity set. `–`, `×` and ` ` + * are the ones a Markdown author reaches for out of habit, and each of them + * costs the reader the entire paragraph it appears in. + */ +const ENTITY = /^&(?:amp|lt|gt|quot|apos|#\d+|#x[0-9a-fA-F]+);/; + +/** Attribute inside a start tag, with the value in either quote style. */ +const ATTRIBUTE = /([a-zA-Z_:][\w.:-]*)\s*=\s*(?:"([^"]*)"|'([^']*)')/g; + +const TRANSLATABLE_LABEL = /]*)>([\s\S]*?)<\/property>/g; +const BLOCK_CODE_UI = /]*>\s*([\s\S]*?)<\/property>/g; +const INLINE_CODE_LABEL = /([\s\S]*?)<\/tt>/g; +const INLINE_CODE_HTML = /]*>([\s\S]*?)<\/code>/g; +const BLOCK_CODE_HTML = /]*?\scode="([^"]*)"/g; +const HTML_ATTRIBUTE = /\bhtml="([^"]*)"/g; +const SOURCE_VIEW_CODE = /]*?\scode="([^"]*)"/g; + +/** Resolves the five predefined entities and numeric character references. */ +function unescapeXml(text) { + return text.replace(/&(amp|lt|gt|quot|apos|#\d+|#x[0-9a-fA-F]+);/g, (_, reference) => { + const named = { amp: "&", lt: "<", gt: ">", quot: '"', apos: "'" }[reference]; + if (named) return named; + const code = reference[1] === "x" ? parseInt(reference.slice(2), 16) : Number(reference.slice(1)); + return String.fromCodePoint(code); + }); +} + +/** + * Index of the `>` that closes the tag starting at `start`, ignoring any `>` + * inside a quoted attribute value. + */ +function findTagEnd(source, start) { + let quote = null; + for (let index = start + 1; index < source.length; index++) { + const character = source[index]; + if (quote) { + if (character === quote) quote = null; + } else if (character === '"' || character === "'") { + quote = character; + } else if (character === ">") { + return index; + } + } + return -1; +} + +/** + * Reads the attributes of a start tag, reporting anything the attribute grammar + * does not account for. + * + * An unquoted or unterminated value is the shape that silently swallows the + * rest of the document, so what is left over after removing every recognised + * attribute has to be whitespace and nothing else. + */ +function parseAttributes(attributeList) { + const attributes = new Map(); + let remainder = attributeList; + for (const match of attributeList.matchAll(ATTRIBUTE)) { + attributes.set(match[1], match[2] ?? match[3]); + remainder = remainder.replace(match[0], ""); + } + return { attributes, malformed: remainder.trim().length > 0 }; +} + +/** + * Walks `source` as markup, reporting every shape the target parser rejects and + * returning the elements it found. + * + * Hand-written because the toolchain ships no XML parser that runs under both + * Node and GJS, and because one scan has to feed the well-formedness rule, the + * vocabulary rule and the attribute rules at once: a tag the balance check + * accepts but the vocabulary check never sees is a gap wide enough to walk an + * unknown widget through. + * + * `fragment` allows text and several elements at the top level, which is what a + * `Gtk.Label` markup string is; a document must have exactly one root element. + * `tagAttributes` of `"any"` skips attribute validation — XML places no + * restriction on attribute names, while Pango accepts them only on ``. + */ +function scanMarkup(source, { tags = null, tagAttributes = "any", fragment = false } = {}) { + const errors = []; + const elements = []; + const stack = []; + let roots = 0; + let index = 0; + + const scanText = (text) => { + for (let at = text.indexOf("&"); at !== -1; at = text.indexOf("&", at + 1)) { + if (!ENTITY.test(text.slice(at))) errors.push(`unknown entity or unescaped &: ${clip(text.slice(at, at + 16))}`); + } + }; + + while (index < source.length) { + const start = source.indexOf("<", index); + if (start === -1) { + scanText(source.slice(index)); + break; + } + scanText(source.slice(index, start)); + + // The XML declaration and comments carry nothing the rules below need, but + // they have their own terminators and would otherwise scan as a malformed + // start tag. + if (source.startsWith("" : "-->"; + const end = source.indexOf(terminator, start); + if (end === -1) { + errors.push(`unterminated ${terminator === "?>" ? "declaration" : "comment"}`); + break; + } + index = end + terminator.length; + continue; + } + + const end = findTagEnd(source, start); + if (end === -1) { + errors.push(`unterminated tag: ${clip(source.slice(start, start + 40))}`); + break; + } + const token = source.slice(start, end + 1); + index = end + 1; + + const close = /^<\/([a-zA-Z_:][\w.:-]*)\s*>$/.exec(token); + if (close) { + if (stack.pop() !== close[1]) errors.push(`unbalanced `); + continue; + } + + const open = /^<([a-zA-Z_:][\w.:-]*)([\s\S]*?)(\/?)>$/.exec(token); + if (!open) { + errors.push(`malformed tag: ${clip(token)}`); + continue; + } + + const [, name, attributeList, selfClosing] = open; + const { attributes, malformed } = parseAttributes(attributeList); + if (malformed) errors.push(`unquoted or malformed attribute in <${name}>: ${clip(attributeList.trim())}`); + if (tags && !tags.has(name)) errors.push(`unsupported element: <${name}>`); + + if (tagAttributes !== "any") { + const rules = tagAttributes[name]; + for (const attribute of attributes.keys()) { + if (!rules?.allowed.has(attribute)) errors.push(`<${name}> does not support the ${attribute} attribute`); + } + for (const attribute of rules?.required ?? []) { + if (!attributes.has(attribute)) errors.push(`<${name}> is missing the ${attribute} attribute`); + } + } + + elements.push({ name, attributes }); + if (stack.length === 0) roots++; + if (!selfClosing) stack.push(name); + } + + for (const name of stack) errors.push(`unclosed <${name}>`); + if (!fragment && roots !== 1) errors.push(`expected exactly one root element, found ${roots}`); + return { errors, elements }; +} + +function clip(text) { + return text.length > 70 ? `${text.slice(0, 70)}…` : text; +} + +function count(values) { + const counts = new Map(); + for (const value of values) counts.set(value, (counts.get(value) ?? 0) + 1); + return counts; +} + +/** Members of `expected` that `actual` does not cover, with multiplicity. */ +function missing(expected, actual) { + const lost = []; + for (const [value, times] of expected) { + const short = times - (actual.get(value) ?? 0); + for (let index = 0; index < short; index++) lost.push(value); + } + return lost; +} + +function read(name) { + return readFileSync(`${DIST}/${name}`, "utf8"); +} + +/** The translatable labels of one `.ui`, with their xgettext comment. */ +function translatableLabels(ui) { + return [...ui.matchAll(TRANSLATABLE_LABEL)].map(([, attributes, body]) => ({ + comment: /comments="([^"]*)"/.exec(attributes)?.[1] ?? null, + markup: unescapeXml(body), + })); +} + +/** + * The label markup must be markup Pango accepts. + * + * `Gtk.Label` with `use-markup` renders an empty string when parsing fails, so + * an entity Pango does not know costs the reader the whole paragraph and + * nothing reports it. Measured against Pango 1.57: `–` and `×` both + * fail with "entity name is not known", while the literal characters `–` and + * `×` parse — which is what the MDX has to contain. + */ +function labelMarkupErrors(markup) { + return scanMarkup(markup, { tags: LABEL_TAGS, tagAttributes: LABEL_TAG_ATTRIBUTES, fragment: true }).errors; +} + +/** Inline and block code literals as they reach each of the three targets. */ +function codeLiterals(document) { + const ui = read(`${document}.ui`); + const ns = read(`${document}.ns.xml`); + const html = read(`${document}.html`); + + return { + ui: { + inline: translatableLabels(ui).flatMap((label) => + [...label.markup.matchAll(INLINE_CODE_LABEL)].map((match) => unescapeXml(match[1])) + ), + block: [...ui.matchAll(BLOCK_CODE_UI)].map((match) => unescapeXml(match[1])), + }, + ns: { + // Inline code is escaped inside the `html` attribute of an `HtmlView`; + // block code is a `w:SourceView` element of its own. + inline: [...ns.matchAll(HTML_ATTRIBUTE)].flatMap((match) => + [...unescapeXml(match[1]).matchAll(SOURCE_VIEW_CODE)].map((code) => unescapeXml(code[1])) + ), + block: [...ns.matchAll(SOURCE_VIEW_CODE)].map((match) => unescapeXml(match[1])), + }, + html: { + inline: [...html.matchAll(INLINE_CODE_HTML)].map((match) => unescapeXml(match[1])), + block: [...html.matchAll(BLOCK_CODE_HTML)].map((match) => unescapeXml(match[1])), + }, + }; +} + +/** + * Fragments every rule must classify correctly, checked before any artifact is + * read. + * + * A structural check that has quietly stopped biting is worse than no check: it + * reports success over output nobody has looked at. The rejected shapes are the + * ones the respective parser refuses; the accepted ones are output the emitter + * legitimately produces today. + */ +const SELF_TEST = [ + ["label", "Use LDA and Step", "accept"], + ["label", 'See the 6502.', "accept"], + ["label", "A & B, 32 × 8 pixels, range 0–255", "accept"], + ["label", "numeric character reference: ×", "accept"], + ["label", "range of 0–255", "reject"], + ["label", "32 × 8 pixels", "reject"], + ["label", "A   B", "reject"], + ["label", "A & B", "reject"], + ["label", "bolditalic", "reject"], + ["label", "no such tag", "reject"], + ["label", "no href", "reject"], + ["label", 'LDA', "reject"], + ["label", "unclosed", "reject"], + ["document", '', "accept"], + ["document", "", "accept"], + ["document", "", "reject"], + ["document", "", "reject"], + ["document", "", "reject"], + ["document", "a & b", "reject"], +]; + +/** Self-test results that came out the wrong way round. */ +function selfTestFailures() { + const wrong = []; + for (const [kind, source, expected] of SELF_TEST) { + const errors = kind === "label" ? labelMarkupErrors(source) : scanMarkup(source).errors; + const verdict = errors.length ? "reject" : "accept"; + if (verdict !== expected) + wrong.push(`expected to ${expected}: ${clip(source)} (${errors.join("; ") || "no problem reported"})`); + } + return wrong; +} + +function check() { + const selfTest = selfTestFailures(); + if (selfTest.length) { + console.error("The structural rules no longer classify their own test cases:"); + for (const failure of selfTest) console.error(` ${failure}`); + return 1; + } + + // Rule 1 — the emitter wrote every artifact its three consumers import. + // A missing file is the one failure that would let every rule below pass by + // reading nothing, so it is checked first and aborts the run. + const artifacts = DOCUMENTS.flatMap((document) => [`${document}.ui`, `${document}.ns.xml`, `${document}.html`]); + const absent = artifacts.filter((name) => !existsSync(`${DIST}/${name}`)); + if (absent.length) { + console.error(`${DIST}/ is missing ${absent.length} artifact(s): ${absent.join(", ")}`); + console.error("Run `gjsify workspace @learn6502/learn build` first."); + return 1; + } + + const failures = []; + const report = (where, problems) => { + for (const problem of problems) failures.push({ where, problem }); + }; + + for (const document of DOCUMENTS) { + const ui = read(`${document}.ui`); + const ns = read(`${document}.ns.xml`); + + // Rule 2 — the `.ui` and `.ns.xml` are well-formed XML with the expected + // root, and contain only elements their consumer can resolve. `Gtk.Builder` + // and NativeScript's `Builder.load` both refuse the whole document on a + // parse error, so this is the difference between one broken paragraph and a + // blank screen. + for (const [name, source, vocabulary] of [ + [`${document}.ui`, ui, VOCABULARY.ui], + [`${document}.ns.xml`, ns, VOCABULARY.ns], + ]) { + const scan = scanMarkup(source, { tags: vocabulary.elements }); + report(name, scan.errors); + if (scan.elements[0]?.name !== vocabulary.root) + report(name, [`root element is <${scan.elements[0]?.name}>, expected <${vocabulary.root}>`]); + + // Rule 3 — every object class the `.ui` names is one app-gnome can + // resolve. `Gtk.Builder` fails on an unknown GType, and it is not the + // emitter that finds out: the app is. + for (const element of scan.elements) { + const objectClass = element.name === "object" ? element.attributes.get("class") : null; + if (objectClass && !vocabulary.objectClasses?.has(objectClass)) + report(name, [`unknown object class: ${objectClass}`]); + } + } + + const labels = translatableLabels(ui); + if (!labels.length) report(`${document}.ui`, ["no translatable labels — the renderer produced no text"]); + + for (const label of labels) { + // Rule 4 — every translatable string carries its translator comment. + // xgettext copies `comments=` into the POT as the `#.` line, which is the + // only context a translator gets: the catalogs are generated with + // `noLocation`, so without it the string arrives with no provenance at + // all. + if (!label.comment?.startsWith("TRANSLATORS:")) + report(`${document}.ui`, [`translatable label without a TRANSLATORS comment: ${clip(label.markup)}`]); + + // Rule 5 — the label markup parses as Pango markup. + for (const error of labelMarkupErrors(label.markup)) + report(`${document}.ui`, [`unrenderable label markup — ${error}: ${clip(label.markup)}`]); + } + + // Rule 6 — the same code literals reach all three targets. + // They are what the reader retypes into the editor, and each target encodes + // them differently (`` on GTK, an escaped `w:SourceView` on + // NativeScript, `` on the web), so a renderer change that drops or + // mangles one of them shows up nowhere else. + const literals = codeLiterals(document); + for (const kind of ["inline", "block"]) { + const reference = count(literals.ui[kind]); + for (const target of ["ns", "html"]) { + const actual = count(literals[target][kind]); + report( + `${document} (${kind} code)`, + missing(reference, actual).map((literal) => `missing from the ${target} target: ${clip(literal)}`) + ); + report( + `${document} (${kind} code)`, + missing(actual, reference).map((literal) => `only in the ${target} target: ${clip(literal)}`) + ); + } + } + } + + for (const { where, problem } of failures) console.error(`${where}: ${problem}`); + + if (failures.length) { + console.error(`\n${failures.length} problem(s) in the generated artifacts.`); + return 1; + } + + console.log( + `All generated artifacts passed structural validation (${DOCUMENTS.length} documents × 3 targets, ${SELF_TEST.length} self-test cases).` + ); + return 0; +} + +const status = check(); +// `gjsify run` keeps the GJS main loop alive, so signal the result explicitly. +if (status !== 0) throw new Error("learn artifact check failed"); diff --git a/packages/learn/package.json b/packages/learn/package.json index bd47bad3..8a450e76 100644 --- a/packages/learn/package.json +++ b/packages/learn/package.json @@ -10,6 +10,9 @@ "build:js": "gjsify run dist/index.js", "build": "gjsify run build:mdx && gjsify run build:js && gjsify run build:copy", "build:copy": "cp -r dist/tutorial.ns.xml ../app-android/app/mdx/tutorial.xml && cp -r dist/quick-help.ns.xml ../app-android/app/mdx/quick-help.xml", + "check:gen": "gjsify build check.js --app gjs --outfile dist/check.gen.js", + "check:run": "gjsify run dist/check.gen.js", + "check": "gjsify run build && gjsify run check:gen && gjsify run check:run", "clear": "rm -rf dist" }, "author": "Nick Morgan, Pascal Garber", From bc5e75a134d0c3ebd839c8b4c770bab0e2fbe16c Mon Sep 17 00:00:00 2001 From: Pascal Garber Date: Fri, 4 Sep 2026 01:01:49 +0200 Subject: [PATCH 4/8] Translations: build learn before extracting MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `build.js` lists `../learn/dist/**/*.ui` among its xgettext sources, so the extraction reads a build artifact of a sibling package. `@learn6502/translations` declares `@learn6502/learn` as a dependency, but building one package does not build what it depends on, so the artifact was whatever the previous run left behind — and with `packages/learn/dist` absent the glob simply matched nothing. On 2026-09-03 `gjsify workspace @learn6502/translations build` therefore emptied the catalogs: the POT went from 457 msgids to 238, every tutorial paragraph turned into an obsolete `#~` entry in all sixteen `.po` files, and the build exited 0. `check` then reported "All translations passed structural validation", because a string that is gone is a string no rule can look at. Two changes, because the ordering and the loud failure are different jobs: - `build:learn` runs `gjsify workspace @learn6502/learn build` as the first step of `build`. The order now lives in the script that performs the extraction, which is what a developer and an agent both reach for. `-t` was the alternative and is not enough on its own: it is a flag at the call site, so nothing forces anyone to pass it, and it would also rebuild `@learn6502/core` and `@learn6502/examples` (~4.9 s) whose output the extraction never reads — `learn` is this package's only workspace dependency, and it builds from committed source in 3.3 s. - `build.js` asserts the artifacts exist before extracting, so running the extraction on its own fails with the command to run instead of silently rewriting sixteen catalogs. gjsify #1521 makes an unmatched source glob loud at the core; this keeps the failure attributable on the pinned toolchain, and covers the case of a partial `dist/`. Verified by reproduction: with `packages/learn/dist` removed, the POT collapsed to 238 msgids before the change and stays at 457 after it. Claude-Session: https://claude.ai/code/session_01URLYj1wgGxVNRbBpNT8xAV --- packages/translations/README.md | 6 ++++++ packages/translations/build.js | 34 +++++++++++++++++++++++++++++- packages/translations/package.json | 3 ++- 3 files changed, 41 insertions(+), 2 deletions(-) diff --git a/packages/translations/README.md b/packages/translations/README.md index 5da4790c..67631f34 100644 --- a/packages/translations/README.md +++ b/packages/translations/README.md @@ -29,6 +29,12 @@ Support for every new language is welcome. - **UI Elements & Messages**: Managed through `.po` files - **Tutorial Content**: Located in [packages/learn/tutorial.mdx](https://github.com/JumpLink/Learn6502/blob/main/packages/learn/tutorial.mdx) +The tutorial's strings — 219 of the 457 in the catalogs — do not reach `xgettext` from the MDX +directly. They are extracted from `packages/learn/dist/*.ui`, which `@learn6502/learn` generates, +so `build` below builds that package first. Do not extract without it: with `packages/learn/dist` +missing or stale, extraction quietly produces a POT without the tutorial and rewrites all sixteen +catalogs to match. The build refuses to run in that state rather than emptying them. + ## Translation Guidelines 1. **Do not translate**: diff --git a/packages/translations/build.js b/packages/translations/build.js index dd10a17d..1b97c31e 100644 --- a/packages/translations/build.js +++ b/packages/translations/build.js @@ -1,9 +1,41 @@ import { gettextPlugin, xgettextPlugin, po2jsonPlugin } from "@gjsify/vite-plugin-gettext"; -import { readFileSync } from "node:fs"; +import { existsSync, readFileSync } from "node:fs"; const APPLICATION_ID = "eu.jumplink.Learn6502"; const VERSION = JSON.parse(readFileSync("./package.json", "utf8")).version; +/** + * Extraction sources that are another package's build output, not source. + * + * `../learn/dist/*.ui` holds the tutorial rendered from `tutorial.mdx`, and it + * carries 219 of the 457 strings in the POT — every paragraph of the tutorial. + * Because it is a build artifact, `xgettext` reads whatever the last run of + * `@learn6502/learn` happened to leave behind: with the directory missing the + * glob simply matches nothing, extraction succeeds, and the POT plus all 16 + * catalogs are rewritten without the tutorial. That happened on 2026-09-03 — + * the build exited 0, and `check` passed over the gutted catalogs, because a + * string that is gone is a string no rule can look at. + * + * The ordering fix is the `build:learn` step in `package.json`, which builds + * this package's only workspace dependency before extraction runs. The + * assertion below is the second half: it makes the same mistake loud for anyone + * who runs the extraction on its own, instead of silently emptying the + * catalogs. + */ +const GENERATED_SOURCES = { + "@learn6502/learn": ["../learn/dist/tutorial.ui", "../learn/dist/quick-help.ui"], +}; + +for (const [workspace, artifacts] of Object.entries(GENERATED_SOURCES)) { + const absent = artifacts.filter((path) => !existsSync(path)); + if (absent.length) + throw new Error( + `Cannot extract translatable strings: ${absent.join(", ")} ${absent.length === 1 ? "is" : "are"} missing. ` + + `Run \`gjsify workspace ${workspace} build\` first, or use \`gjsify workspace @learn6502/translations build\`, ` + + `which does it for you.` + ); +} + // Extract translatable strings from source files to create a POT template const xgettext = xgettextPlugin({ sources: [ diff --git a/packages/translations/package.json b/packages/translations/package.json index 14689a5d..57e769cc 100644 --- a/packages/translations/package.json +++ b/packages/translations/package.json @@ -11,9 +11,10 @@ "po" ], "scripts": { + "build:learn": "gjsify workspace @learn6502/learn build", "build:gen": "gjsify build build.js --app gjs --outfile dist/build.gen.js", "build:run": "gjsify run dist/build.gen.js", - "build": "gjsify run build:gen && gjsify run build:run", + "build": "gjsify run build:learn && gjsify run build:gen && gjsify run build:run", "check:gen": "gjsify build check.js --app gjs --outfile dist/check.gen.js", "check:run": "gjsify run dist/check.gen.js", "check": "gjsify run check:gen && gjsify run check:run", From a524c9426794862ee9dabc4663aa3ff0c5219167 Mon Sep 17 00:00:00 2001 From: Pascal Garber Date: Fri, 4 Sep 2026 01:40:12 +0200 Subject: [PATCH 5/8] build: propagate failures through gjsify chains MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A package script whose whole body is a single `gjsify …` command exits 0 no matter what that command did. Measured on the pinned CLI 0.16.3 (`runScript` in `@gjsify/cli/dist/cli.gjs.mjs`): such a body is tokenized and dispatched in-process, then the wrapper exits with `process.exitCode ?? 0` — and `gjsify build` / `gjsify run` report failure without setting it. Any shell operator in the body takes the spawn path instead and the status survives. `gjsify tsc` is unaffected; it spawns `gjs`. Every leaf of the learn/translations pipeline had that shape, so: - `gjsify workspace @learn6502/learn check` exited 0 while printing 303 problems — the gate added for exactly this failure class could not fail CI; - `gjsify workspace @learn6502/translations check` exited 0 over a catalog with `` and `–` in a msgstr, so the CI catalog gate has never been able to fail either; - a broken `@learn6502/learn` build exited 0 and the translations build went on to extract from whatever `dist/` held — the ordering step guarded nothing; - `learn`'s own build ran `dist/index.js` from the *previous* bundle after the current one failed to compile, and copied that output on. Collapsing the `:gen`/`:run` pairs into chains restores the status at every level: a broken learn build now stops the translations build with exit 1 and the POT untouched at 457 msgids (measured). The fix belongs upstream in gjsify; until it lands, splitting these chains back into single-command steps disarms the gates again, which is why AGENTS.md now says so. Claude-Session: https://claude.ai/code/session_01URLYj1wgGxVNRbBpNT8xAV --- AGENTS.md | 14 ++++++++++++++ packages/examples/package.json | 4 +--- packages/learn/package.json | 8 ++------ packages/translations/package.json | 9 ++------- 4 files changed, 19 insertions(+), 16 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index de2ce983..0ca68641 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -22,6 +22,20 @@ Principles: maximize code reuse via `common-ui`/`6502` packages; keep platform c | vite-plugin-gettext | `packages/vite-plugin-gettext/` | Vite plugin for gettext localization | TypeScript | | vite-plugin-blueprint | `packages/vite-plugin-blueprint/` | Vite plugin for Blueprint `.blp` files | TypeScript | +## Package scripts + +A `build` or `check` script must be a **chain**, never a lone `gjsify …` command. Measured on the +pinned CLI (0.16.3, `runScript` in `@gjsify/cli/dist/cli.gjs.mjs`): when a script body tokenizes +as a single plain `gjsify …` command the CLI dispatches it in-process and then exits with +`process.exitCode ?? 0` — `gjsify build` and `gjsify run` report failure without setting it, so +the script exits 0 while its work failed, and every caller above it sees green. Any shell operator +in the body disables that path. `gjsify tsc` is unaffected (it spawns `gjs`). + +This is why `learn`, `translations` and `examples` have no `:gen`/`:run` sub-scripts. Splitting a +chain back into single-command steps disarms the gate silently: it was how +`gjsify workspace @learn6502/translations check` reported "All translations passed" over catalogs +it had just emptied. Fix belongs upstream in gjsify; until then, keep the chains. + ## TypeScript Applies to all `.ts`/`.tsx` files. diff --git a/packages/examples/package.json b/packages/examples/package.json index a7dd97e9..094b3e0c 100644 --- a/packages/examples/package.json +++ b/packages/examples/package.json @@ -12,9 +12,7 @@ "./examples": "./examples.ts" }, "scripts": { - "build:gen": "gjsify build build.ts --app gjs --outfile dist/build.js", - "build:run": "gjsify run dist/build.js", - "build": "gjsify run build:gen && gjsify run build:run" + "build": "gjsify build build.ts --app gjs --outfile dist/build.js && gjsify run dist/build.js" }, "devDependencies": { "@types/node": "^25.9.1", diff --git a/packages/learn/package.json b/packages/learn/package.json index 8a450e76..de55a9fa 100644 --- a/packages/learn/package.json +++ b/packages/learn/package.json @@ -6,13 +6,9 @@ "type": "module", "main": "index.js", "scripts": { - "build:mdx": "gjsify build tsx/index.tsx --app gjs --globals node --outfile dist/index.js", - "build:js": "gjsify run dist/index.js", - "build": "gjsify run build:mdx && gjsify run build:js && gjsify run build:copy", + "build": "gjsify build tsx/index.tsx --app gjs --globals node --outfile dist/index.js && gjsify run dist/index.js && gjsify run build:copy", "build:copy": "cp -r dist/tutorial.ns.xml ../app-android/app/mdx/tutorial.xml && cp -r dist/quick-help.ns.xml ../app-android/app/mdx/quick-help.xml", - "check:gen": "gjsify build check.js --app gjs --outfile dist/check.gen.js", - "check:run": "gjsify run dist/check.gen.js", - "check": "gjsify run build && gjsify run check:gen && gjsify run check:run", + "check": "gjsify run build && gjsify build check.js --app gjs --outfile dist/check.gen.js && gjsify run dist/check.gen.js", "clear": "rm -rf dist" }, "author": "Nick Morgan, Pascal Garber", diff --git a/packages/translations/package.json b/packages/translations/package.json index 57e769cc..49426e96 100644 --- a/packages/translations/package.json +++ b/packages/translations/package.json @@ -11,13 +11,8 @@ "po" ], "scripts": { - "build:learn": "gjsify workspace @learn6502/learn build", - "build:gen": "gjsify build build.js --app gjs --outfile dist/build.gen.js", - "build:run": "gjsify run dist/build.gen.js", - "build": "gjsify run build:learn && gjsify run build:gen && gjsify run build:run", - "check:gen": "gjsify build check.js --app gjs --outfile dist/check.gen.js", - "check:run": "gjsify run dist/check.gen.js", - "check": "gjsify run check:gen && gjsify run check:run", + "build": "gjsify workspace @learn6502/learn build && gjsify build build.js --app gjs --outfile dist/build.gen.js && gjsify run dist/build.gen.js", + "check": "gjsify build check.js --app gjs --outfile dist/check.gen.js && gjsify run dist/check.gen.js", "clear": "rm -rf dist" }, "devDependencies": { From d7f3cd7bfaa5b811a02166bb67697edd49e03c6d Mon Sep 17 00:00:00 2001 From: Pascal Garber Date: Fri, 4 Sep 2026 01:40:33 +0200 Subject: [PATCH 6/8] Learn: fail the build when a write fails MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The three emitters caught every write error, logged it and returned. The build then exited 0 with the previous run's artifact still on disk, and everything downstream — the artifact check, and `xgettext` reading `dist/*.ui` — passed over output that no longer matched the source. There is no recovery to make here, so there is nothing for a catch to do. Let the rejection out: `gjsify run dist/index.js` now exits non-zero and, with the build script chained, so does the build. Claude-Session: https://claude.ai/code/session_01URLYj1wgGxVNRbBpNT8xAV --- packages/learn/tsx/index.tsx | 30 +++++++++++------------------- 1 file changed, 11 insertions(+), 19 deletions(-) diff --git a/packages/learn/tsx/index.tsx b/packages/learn/tsx/index.tsx index 8a5dc621..67c384fe 100644 --- a/packages/learn/tsx/index.tsx +++ b/packages/learn/tsx/index.tsx @@ -7,33 +7,25 @@ import { components as NsComponents, generateNativeScriptXml, NsRoot } from "./c import { writeFile } from "node:fs/promises"; import { withSourceFileContext } from "./utils.ts"; +// A failed write is not recoverable here and must not be swallowed: the build +// would exit 0 having left the previous run's artifact in place, and everything +// downstream — the check, and `xgettext` reading `dist/*.ui` — would then pass +// over output that no longer matches the source. + async function generateGtkUiXml(fileName: string, component: string) { const output = `` + component; - - try { - await writeFile(`dist/${fileName}.ui`, output, "utf-8"); - console.log(`Output saved to ${fileName}.ui`); - } catch (error) { - console.error("Error saving file:", error); - } + await writeFile(`dist/${fileName}.ui`, output, "utf-8"); + console.log(`Output saved to ${fileName}.ui`); } async function saveNativeScriptXml(fileName: string, component: string) { - try { - await writeFile(`dist/${fileName}.ns.xml`, component, "utf-8"); - console.log(`Output saved to ${fileName}.ns.xml`); - } catch (error) { - console.error("Error saving NativeScript XML file:", error); - } + await writeFile(`dist/${fileName}.ns.xml`, component, "utf-8"); + console.log(`Output saved to ${fileName}.ns.xml`); } async function generateHtml(fileName: string, component: string) { - try { - await writeFile(`dist/${fileName}.html`, component, "utf-8"); - console.log(`Output saved to ${fileName}.html`); - } catch (error) { - console.error("Error saving file:", error); - } + await writeFile(`dist/${fileName}.html`, component, "utf-8"); + console.log(`Output saved to ${fileName}.html`); } // Generate GTK UI files From a0edb89cad3d19ea43b592ba4b87ef42b6082a84 Mon Sep 17 00:00:00 2001 From: Pascal Garber Date: Fri, 4 Sep 2026 01:40:33 +0200 Subject: [PATCH 7/8] Learn: catch dropped paragraphs and stale dist MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three gaps found by mutating the emitter and re-running the check: Dropping every seventh plain-text paragraph from the `.ui` only — the file `xgettext` builds the catalogs from, so precisely the loss this package's check exists to prevent — passed. Only code literals were compared across targets; prose was not. The `.html` is rendered from the same MDX in the same run and matches the `.ui` paragraph for paragraph (146/146 and 80/80 today), so it can serve as the witness. It now does. An emitter that stops writing an artifact passed, because `check` rebuilt into a `dist/` that still held the previous run's file. On a fresh CI checkout it would have failed; locally, where the developer looks before pushing, it did not. `check` now clears `dist/` first. Reordering `code` and `language` inside a `SourceView` — meaningless in XML, ignored by `Gtk.Builder` — turned the check red with 22 messages blaming the *other* two targets, because the block-code extractor required `` to come first. It no longer does, and an extraction self-test pins that: an extractor that quietly matches less than it should does not report anything, it shrinks the set the rule compares against. A `SourceView` carrying no code at all is now reported rather than cancelling out on both sides of the comparison. Claude-Session: https://claude.ai/code/session_01URLYj1wgGxVNRbBpNT8xAV --- packages/learn/README.md | 17 ++++- packages/learn/check.js | 132 +++++++++++++++++++++++++++++++++--- packages/learn/package.json | 2 +- 3 files changed, 137 insertions(+), 14 deletions(-) diff --git a/packages/learn/README.md b/packages/learn/README.md index 05e30dda..6e657973 100644 --- a/packages/learn/README.md +++ b/packages/learn/README.md @@ -45,9 +45,11 @@ This will generate the necessary output files in the `dist/` directory. gjsify workspace @learn6502/learn check ``` -`check.js` rebuilds `dist/` and then validates the generated artifacts structurally — no byte -snapshot, so it survives reformatting the emitter but still fails on the changes that cost a -reader something. CI runs it on every pull request. It asserts that: +`check.js` clears and rebuilds `dist/`, then validates the generated artifacts structurally — no +byte snapshot, so it survives reformatting the emitter but still fails on the changes that cost a +reader something. The clear is part of the rule, not tidiness: with the previous run's output +still on disk, an emitter that stops writing a target is read as one that wrote the same thing +again. CI runs it on every pull request. It asserts that: - all six artifacts were written; - the `.ui` and `.ns.xml` are well-formed XML with the expected root, and contain only elements @@ -58,9 +60,18 @@ reader something. CI runs it on every pull request. It asserts that: - every label is markup Pango accepts. A `Gtk.Label` whose markup fails to parse renders as an empty string, so an HTML-only entity such as `–` costs the reader the whole paragraph — write the character itself (`–`, `×`) in the MDX; +- the `.ui` carries every paragraph the `.html` does. The `.ui` is what `xgettext` builds the + catalogs from, so prose that stops reaching it stops existing for every translator, and nothing + downstream can miss what is no longer there — the `.html`, rendered from the same MDX in the + same run, is the witness; - the same code literals reach all three targets. They are what the reader retypes into the editor, and each target encodes them differently (``, an escaped `w:SourceView`, ``). +The rules run after their own self-tests: the markup rules against fragments each parser accepts +or rejects, and the extractors against shapes the emitter is allowed to produce. An extractor that +quietly matches less than it should is the worse failure of the two, because it shrinks the set a +rule compares against instead of reporting anything. + It says nothing about whether the tutorial is *correct*; that still needs a reader. ## License diff --git a/packages/learn/check.js b/packages/learn/check.js index 64819a34..fd339ce6 100644 --- a/packages/learn/check.js +++ b/packages/learn/check.js @@ -19,9 +19,11 @@ * and a `Gtk.Label` whose markup fails to parse renders as an empty string — a * whole tutorial paragraph gone, with no error anywhere. * - * Run via `gjsify workspace @learn6502/learn check`, which rebuilds `dist/` - * first: a check that reads whatever artifact happened to be lying around is - * the same defect it exists to catch. + * Run via `gjsify workspace @learn6502/learn check`, which clears and rebuilds + * `dist/` first: a check that reads whatever artifact happened to be lying + * around is the same defect it exists to catch, and a leftover from the + * previous run makes an emitter that stopped writing look like one that wrote + * the same thing again. */ import { existsSync, readFileSync } from "node:fs"; @@ -85,10 +87,18 @@ const ENTITY = /^&(?:amp|lt|gt|quot|apos|#\d+|#x[0-9a-fA-F]+);/; const ATTRIBUTE = /([a-zA-Z_:][\w.:-]*)\s*=\s*(?:"([^"]*)"|'([^']*)')/g; const TRANSLATABLE_LABEL = /]*)>([\s\S]*?)<\/property>/g; -const BLOCK_CODE_UI = /]*>\s*([\s\S]*?)<\/property>/g; +// The whole `SourceView` element, not `` pinned to the +// front of it: property order carries no meaning in XML and `Gtk.Builder` +// ignores it, so an extractor that depends on it turns a legitimate emitter +// refactor into a red build — and, worse, reports the loss against the *other* +// targets, which are the ones that did not change. +const BLOCK_CODE_UI_OBJECT = /]*>([\s\S]*?)<\/object>/g; +const CODE_PROPERTY = /([\s\S]*?)<\/property>/; const INLINE_CODE_LABEL = /([\s\S]*?)<\/tt>/g; const INLINE_CODE_HTML = /]*>([\s\S]*?)<\/code>/g; const BLOCK_CODE_HTML = /]*?\scode="([^"]*)"/g; +/** The elements `html.components.tsx` renders prose into, each a `.ui` label. */ +const HTML_TEXT_BLOCK = /<(p|h1|h2|h3|h4|li)\b[^>]*>([\s\S]*?)<\/\1>/g; const HTML_ATTRIBUTE = /\bhtml="([^"]*)"/g; const SOURCE_VIEW_CODE = /]*?\scode="([^"]*)"/g; @@ -279,6 +289,40 @@ function labelMarkupErrors(markup) { return scanMarkup(markup, { tags: LABEL_TAGS, tagAttributes: LABEL_TAG_ATTRIBUTES, fragment: true }).errors; } +/** Readable text of a markup fragment, normalised for comparison across targets. */ +function plainText(markup) { + return unescapeXml(markup.replace(/<[^>]*>/g, "")) + .replace(/\s+/g, " ") + .trim(); +} + +/** + * Block-code literals of a `.ui`, with the number of `SourceView` objects they + * came from. + * + * The two numbers are reported separately because an editor widget with no code + * in it is a defect the comparison against the other targets cannot see: it + * removes the literal from both sides at once. + */ +function uiBlockCode(ui) { + const objects = [...ui.matchAll(BLOCK_CODE_UI_OBJECT)]; + const codes = objects + .map((object) => CODE_PROPERTY.exec(object[1])) + .filter((code) => code !== null) + .map((code) => unescapeXml(code[1])); + return { objects: objects.length, codes }; +} + +/** The prose of a `.ui`, one entry per translatable label. */ +function uiProse(ui) { + return translatableLabels(ui).map((label) => plainText(label.markup)); +} + +/** The prose of an `.html`, one entry per rendered text block. */ +function htmlProse(html) { + return [...html.matchAll(HTML_TEXT_BLOCK)].map((match) => plainText(match[2])).filter(Boolean); +} + /** Inline and block code literals as they reach each of the three targets. */ function codeLiterals(document) { const ui = read(`${document}.ui`); @@ -290,7 +334,7 @@ function codeLiterals(document) { inline: translatableLabels(ui).flatMap((label) => [...label.markup.matchAll(INLINE_CODE_LABEL)].map((match) => unescapeXml(match[1])) ), - block: [...ui.matchAll(BLOCK_CODE_UI)].map((match) => unescapeXml(match[1])), + block: uiBlockCode(ui).codes, }, ns: { // Inline code is escaped inside the `html` attribute of an `HtmlView`; @@ -338,6 +382,42 @@ const SELF_TEST = [ ["document", "a & b", "reject"], ]; +/** + * Extractors run over shapes the emitter is allowed to produce. + * + * The markup rules have SELF_TEST; the extractors had nothing, and an extractor + * that quietly matches less than it should is the worse failure: it does not + * report anything, it shrinks the reference set the comparison is made against. + * Property order is the concrete case — XML gives it no meaning, so the emitter + * may reorder freely and the extraction has to survive it. + */ +const EXTRACTION_TEST = [ + [ + "block code is found whatever order the object's properties are in", + () => + [ + 'LDA #$016502', + '6502LDA #$01', + ] + .map((properties) => uiBlockCode(`${properties}`)) + .every((found) => found.objects === 1 && found.codes.join() === "LDA #$01"), + ], + [ + "an editor with no code is counted but yields no literal", + () => { + const found = uiBlockCode('6502'); + return found.objects === 1 && found.codes.length === 0; + }, + ], + ["a label's prose survives its markup", () => plainText("Use LDA\n and Step") === "Use LDA and Step"], + [ + "prose is found under every element the html target renders text into", + () => + htmlProse('

Registers

The A register

  • one
').join("|") === + "Registers|The A register|one", + ], +]; + /** Self-test results that came out the wrong way round. */ function selfTestFailures() { const wrong = []; @@ -351,7 +431,10 @@ function selfTestFailures() { } function check() { - const selfTest = selfTestFailures(); + const selfTest = [ + ...selfTestFailures(), + ...EXTRACTION_TEST.filter(([, holds]) => !holds()).map(([what]) => `extraction no longer holds: ${what}`), + ]; if (selfTest.length) { console.error("The structural rules no longer classify their own test cases:"); for (const failure of selfTest) console.error(` ${failure}`); @@ -405,8 +488,26 @@ function check() { const labels = translatableLabels(ui); if (!labels.length) report(`${document}.ui`, ["no translatable labels — the renderer produced no text"]); + // Rule 4 — the `.ui` carries every paragraph the `.html` does. + // The `.ui` is what `xgettext` extracts the catalogs from, so prose that + // stops reaching it is a string that stops existing for every translator, + // and neither the catalogs nor any other rule here can miss what is no + // longer there. The `.html` is the independent witness: it is rendered from + // the same MDX in the same run, so the two have to agree paragraph for + // paragraph. (The `.ns.xml` is not usable for this — it groups list items + // into one `HtmlView`, so it has no per-paragraph structure to compare.) + const prose = { ui: count(uiProse(ui)), html: count(htmlProse(read(`${document}.html`))) }; + report( + document, + missing(prose.html, prose.ui).map((text) => `paragraph rendered to html but not to the .ui: ${clip(text)}`) + ); + report( + document, + missing(prose.ui, prose.html).map((text) => `paragraph rendered to the .ui but not to html: ${clip(text)}`) + ); + for (const label of labels) { - // Rule 4 — every translatable string carries its translator comment. + // Rule 5 — every translatable string carries its translator comment. // xgettext copies `comments=` into the POT as the `#.` line, which is the // only context a translator gets: the catalogs are generated with // `noLocation`, so without it the string arrives with no provenance at @@ -414,17 +515,27 @@ function check() { if (!label.comment?.startsWith("TRANSLATORS:")) report(`${document}.ui`, [`translatable label without a TRANSLATORS comment: ${clip(label.markup)}`]); - // Rule 5 — the label markup parses as Pango markup. + // Rule 6 — the label markup parses as Pango markup. for (const error of labelMarkupErrors(label.markup)) report(`${document}.ui`, [`unrenderable label markup — ${error}: ${clip(label.markup)}`]); } - // Rule 6 — the same code literals reach all three targets. + // Rule 7 — the same code literals reach all three targets. // They are what the reader retypes into the editor, and each target encodes // them differently (`` on GTK, an escaped `w:SourceView` on // NativeScript, `` on the web), so a renderer change that drops or // mangles one of them shows up nowhere else. const literals = codeLiterals(document); + + // An editor widget with no code in it is invisible to the comparison below, + // because it removes the literal from the `.ui` reference set and from the + // target at the same time. + const editors = uiBlockCode(ui); + if (editors.objects !== editors.codes.length) + report(`${document}.ui`, [ + `${editors.objects - editors.codes.length} SourceView object(s) without a code property`, + ]); + for (const kind of ["inline", "block"]) { const reference = count(literals.ui[kind]); for (const target of ["ns", "html"]) { @@ -449,7 +560,8 @@ function check() { } console.log( - `All generated artifacts passed structural validation (${DOCUMENTS.length} documents × 3 targets, ${SELF_TEST.length} self-test cases).` + `All generated artifacts passed structural validation (${DOCUMENTS.length} documents × 3 targets, ` + + `${SELF_TEST.length + EXTRACTION_TEST.length} self-test cases).` ); return 0; } diff --git a/packages/learn/package.json b/packages/learn/package.json index de55a9fa..44ebaca7 100644 --- a/packages/learn/package.json +++ b/packages/learn/package.json @@ -8,7 +8,7 @@ "scripts": { "build": "gjsify build tsx/index.tsx --app gjs --globals node --outfile dist/index.js && gjsify run dist/index.js && gjsify run build:copy", "build:copy": "cp -r dist/tutorial.ns.xml ../app-android/app/mdx/tutorial.xml && cp -r dist/quick-help.ns.xml ../app-android/app/mdx/quick-help.xml", - "check": "gjsify run build && gjsify build check.js --app gjs --outfile dist/check.gen.js && gjsify run dist/check.gen.js", + "check": "gjsify run clear && gjsify run build && gjsify build check.js --app gjs --outfile dist/check.gen.js && gjsify run dist/check.gen.js", "clear": "rm -rf dist" }, "author": "Nick Morgan, Pascal Garber", From cf6ec1076c4518140d02c9bb8d0a868b0df4ca1a Mon Sep 17 00:00:00 2001 From: Pascal Garber Date: Fri, 4 Sep 2026 01:40:34 +0200 Subject: [PATCH 8/8] Translations: assert the extraction has input MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The guard checked that `../learn/dist/*.ui` exists. Existence is not the property that matters: with both files present but truncated to zero bytes, extraction reproduced the 2026-09-03 collapse exactly — POT from 457 msgids to 238, 259 strings obsoleted in `de.po`, exit 0. That is the state a half-finished build leaves behind. Assert content instead: each file must carry `translatable="yes"`, the attribute `xgettext` keys on. Freshness is still not asserted, and an mtime would not assert it either — it says when a file was written, which a CI cache restores. What the build order guarantees is that the files were just produced by a build that succeeded; this assertion covers the case where someone runs the extraction on its own. Claude-Session: https://claude.ai/code/session_01URLYj1wgGxVNRbBpNT8xAV --- packages/translations/README.md | 7 ++++--- packages/translations/build.js | 33 +++++++++++++++++++++++++-------- 2 files changed, 29 insertions(+), 11 deletions(-) diff --git a/packages/translations/README.md b/packages/translations/README.md index 67631f34..bbf3b8ee 100644 --- a/packages/translations/README.md +++ b/packages/translations/README.md @@ -31,9 +31,10 @@ Support for every new language is welcome. The tutorial's strings — 219 of the 457 in the catalogs — do not reach `xgettext` from the MDX directly. They are extracted from `packages/learn/dist/*.ui`, which `@learn6502/learn` generates, -so `build` below builds that package first. Do not extract without it: with `packages/learn/dist` -missing or stale, extraction quietly produces a POT without the tutorial and rewrites all sixteen -catalogs to match. The build refuses to run in that state rather than emptying them. +so `build` below builds that package first, and stops if that build fails. Do not extract without +it: with those files missing — or present but empty, which is what a half-finished build leaves — +extraction quietly produces a POT without the tutorial and rewrites all sixteen catalogs to match. +`build.js` refuses to extract unless the files are there *and* carry translatable strings. ## Translation Guidelines diff --git a/packages/translations/build.js b/packages/translations/build.js index 1b97c31e..1e510cea 100644 --- a/packages/translations/build.js +++ b/packages/translations/build.js @@ -16,21 +16,38 @@ const VERSION = JSON.parse(readFileSync("./package.json", "utf8")).version; * the build exited 0, and `check` passed over the gutted catalogs, because a * string that is gone is a string no rule can look at. * - * The ordering fix is the `build:learn` step in `package.json`, which builds - * this package's only workspace dependency before extraction runs. The - * assertion below is the second half: it makes the same mistake loud for anyone - * who runs the extraction on its own, instead of silently emptying the - * catalogs. + * The ordering fix is the first step of this package's `build` in + * `package.json`, which builds its only workspace dependency before extraction + * runs. The assertion below is the second half: it makes the same mistake loud + * for anyone who runs the extraction on its own, instead of silently emptying + * the catalogs. + * + * The assertion is on content, not on the file being there. A `.ui` that exists + * but carries no `translatable="yes"` property reproduces the 2026-09-03 + * collapse exactly — measured: with both files truncated to zero bytes the POT + * goes from 457 msgids to 238 and 259 strings go obsolete in `de.po`, and an + * existence check passes over it. Freshness is not what is being asserted here + * (an mtime would only say when a file was written, which a CI cache restores + * anyway); what is asserted is that the extraction has something to extract. */ const GENERATED_SOURCES = { "@learn6502/learn": ["../learn/dist/tutorial.ui", "../learn/dist/quick-help.ui"], }; +/** The attribute `xgettext` keys on when it reads a GtkBuilder file. */ +const TRANSLATABLE = 'translatable="yes"'; + for (const [workspace, artifacts] of Object.entries(GENERATED_SOURCES)) { - const absent = artifacts.filter((path) => !existsSync(path)); - if (absent.length) + const unusable = artifacts + .map((path) => { + if (!existsSync(path)) return `${path} is missing`; + if (!readFileSync(path, "utf8").includes(TRANSLATABLE)) return `${path} carries no translatable strings`; + return null; + }) + .filter((problem) => problem !== null); + if (unusable.length) throw new Error( - `Cannot extract translatable strings: ${absent.join(", ")} ${absent.length === 1 ? "is" : "are"} missing. ` + + `Cannot extract translatable strings: ${unusable.join(", ")}. ` + `Run \`gjsify workspace ${workspace} build\` first, or use \`gjsify workspace @learn6502/translations build\`, ` + `which does it for you.` );