Repository navigation
ci: the nightly starts again, the playground's two red samples, the ladder deploy's answer, the rig's reference build, name_lookup's sort - #4382
Conversation
The nightly workflow has died at startup every night since the commit_messages job landed in extended_checks.yml: a called workflow's job may not ask for a permission its caller withholds, and nightly.yml granted contents and actions only. Every pull_request run of nightly.yml died the same way. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Each record store is posted to the ladder service on dasweb-1, and any non-200 read as "service down?". A stale service refusing newer stores with HTTP 400 (a tune stamp past its old 1024-byte cap) kept the live board three weeks old while the workflow stayed green. One helper now posts each document and prints the whole answer as a workflow warning. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
std::stable_sort allocates through get_temporary_buffer, deprecated in libstdc++ 12 and fatal under -Werror with clang-19 - the sites PR 3901 removed from the compiler. The perfect-hash builder's bucket ordering had brought one back. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
setup_asr_rig's translation reference never materialized on a Debian 12 Zen 4 box: gcc 12 reads its bundled GEMM kernels as maybe-uninitialized under -march=native, and the build's -Werror rides a cmake warnings list. Seeding that list from the cache puts the specific -Wno-error ahead of it on a Linux box; the define would break an MSVC configure, and the Mac's clang needs no relief. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The runtime's fio binding now references pread and pwrite, so a sample
whose link keeps that object imports wasi fd_pread and fd_pwrite, and
WebAssembly.instantiate refused the module ("function import requires a
callable") - the nightly playground's Linq sample in wasm mode. The
shim answers both with BADF like its preopen probes: no sample opens a
file.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
There was a problem hiding this comment.
🔵 Needs a closer look
Several hunks hinge on behavior that can't be confirmed from the repo alone—the marian flag against an external CMake submodule and the wasm fd_pread/fd_pwrite shim plus deploy helper only verifiable against the live production deploy and nightly.
No open issues 🎉
What changed in this PR
This is a maintenance PR that unbreaks the nightly CI cron and repairs several smaller breaks across the playground, pages deploy, the ASR benchmark rig, and a -Werror build failure. The core unblock is a GitHub Actions permissions fix: the nightly's extended_checks caller job must grant every permission any job in the called reusable workflow declares (pull-requests: read), or GitHub rejects the whole run at startup — validation happens before the job's if gate skips commit_messages.
Changes:
- CI: grant
pull-requests: readon the nightlyextended_checkscaller; refactor the pages-deploy ladder import into a helper that surfaces the service's full response (e.g. an HTTP 400 schema refusal) as a workflow warning instead of a misleading "service down?", while keeping a refused connection from aborting the step underset -euo pipefail. - Build/toolchain: migrate
name_lookup.cppfromdas::stable_sortto the repo'sdas_stable_sort(avoids a deprecation-fatalstd::stable_sortunder clang-19 + libstdc++ 12-Werror); seed the marian reference build's warnings with-Wno-error=maybe-uninitializedon Linux (gcc 12 AVX-512 false positive on Zen 4). - Playground: stub WASI
fd_pread/fd_pwritewith BADF so the fio-importing wasm modules instantiate, and loosen the Exp-loop sample's assertion tolerance (eps10 → 32) to the correct float32-sum headroom.
| File | Description |
|---|---|
.github/workflows/nightly.yml |
Adds pull-requests: read so the called extended_checks workflow passes GitHub's startup permission check. |
.github/workflows/pages.yml |
Routes ladder store/annotation imports through one ladder_import helper that prints the full non-200 response as a warning and tolerates refused connections under set -euo pipefail. |
src/simulate/name_lookup.cpp |
Switches bucket ordering to das_stable_sort (pointer range) to avoid the deprecated std::stable_sort; adds the das_qsort_r.h include. |
modules/dasLLAMA/performance/setup_asr_rig.das |
Seeds the marian cmake warnings list with -Wno-error=maybe-uninitialized on non-Apple/Windows hosts to survive the gcc false positive. |
web/examples/ui/src/main.js |
Adds fd_pread/fd_pwrite WASI shim stubs returning BADF, consistent with the existing preopen probes. |
web/examples/ui/samples/examples/exp.das |
Raises the Exp-loop verification tolerance from 10 to 32 to match the exact float32 sequential sum. |
Note: a private human-deferral signal was recorded because a few hunks (the marian external-CMake flag and the wasm shim/deploy helper) can only be fully validated against the external submodule and the live production deploy, which is outside this repo's reach.
Review effort: Balanced
The sample sums exp(1/k) over a million terms in float32 and asserted the sum within 10 of 1e6. The exact sum is 15.47 over, and the float32 sequential sum lands exactly 10 over (numpy agrees), so the assert sat on its own boundary and failed in both engines of the nightly playground once the terms changed by an ulp. The guard is against a broken exp, not rounding: it allows 32. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ubuntu-latest became Ubuntu 26.04 (resolute) on Oct 25, and the two linux lanes that fetch llvm-22-dev named the noble repo by hand, so their apt install no longer resolved on the runner - the nightly's daspkg_index and extended_checks linux jobs. apt.llvm.org carries one repo per codename (noble and resolute both answer), so the lanes read the codename from lsb_release. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ee8506b to
412273e
Compare
There was a problem hiding this comment.
🔵 Needs a closer look
The change spans multiple arms—deploy helper, compiler rig flags, a wasm shim, and apt codename—that hinge on live production services and external toolchains with no in-repo tests, leaving correctness dependent on the author's manual runtime validation that can't be reproduced here.
No open issues 🎉
Review effort: Balanced
|
|
|
|
|
|
ubuntu-latest became Ubuntu 26.04 on Oct 25; there the kisak-mesa PPA serves a lavapipe it marks preliminary, and it segfaults inside libvulkan_lvp.so on dasVulkan's test_compute_features. The lane was green on 24.04 with the PPA's noble build, so it stays on that image until the 26.04 lavapipe settles. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The ExternalProject applied patches/glfw-3.5.1-ime.patch with git apply
under GIT_CEILING_DIRECTORIES, meant to keep git from discovering this
checkout above the extracted source. msys2's git takes no notice of a
Windows-style ceiling, finds the checkout, and in repository mode skips
every file of the patch ("Skipped patch 'include/GLFW/glfw3.h'") while
exiting 0 - so the mingw nightly compiled dasGlfw and dasImgui against
an unpatched header and failed on every preedit symbol. GIT_DIR naming
a path that does not exist turns discovery off everywhere: msys2 git
2.54, Git for Windows 2.55, Debian git 2.39 and Apple git 2.54 all
apply the 16 header hunks that way.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The whitelist kernel's uint lane cast int_bits_to_float(ia[i]) to uint, and ia holds negative ints whose bit patterns are NaNs. CUDA defines that cast as 0; C leaves it undefined, so the CPU reference run of the kernel is what UBSan's nightly cell reports (SimNode_Cast<uint, float>: "-nan is outside the range of representable values"). The lane keeps its float-to-uint cast but masks the sign and the top exponent bit first - a finite float below 2.0, the same value on both tiers. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The nightly's whole-folder dasLLAMA lint on linux - the only lint that walks files a PR did not touch - reports LINT019 on test_kernel_coverage.das: the nolint:STYLE039 on the Russian prompt literals suppresses nothing there, while the darwin flavor, whose Metal half compiles those literals, fires STYLE039 on them. That is the case the LINT019 message names: a rule that fires only in one flavor's compile keeps its marker and adds LINT019 to the list, as this file's other markers do. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
8395ee5 to
e64fdde
Compare
The nightly's darwin doc-verify (the only lane that compiles the authored pages' das blocks) reds two pages from today's merges: the lint page's STYLE048 example reads m, Mode, n and r that no block declares, and the translate tutorial's loop reads lines that only its companion program declares. Each block now carries the `given` markers for what its prose assumes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
5070190 to
1e37bf4
Compare
There was a problem hiding this comment.
🔵 Needs a closer look
The change bundles numerous CI/build-infrastructure fixes whose correctness hinges on external behaviors that can't be verified here, compounded by operationally risky pages-deploy and nightly-permission edits.
No open issues 🎉
Review effort: Balanced
There was a problem hiding this comment.
🔵 Needs a closer look
The change spans broad production CI/deploy surface with several author-flagged untested arms that only validate at runtime after deploy, exceeding what can be confidently verified in review.
No open issues 🎉
Review effort: Balanced
Why. The nightly workflow has failed at startup every night since Oct 24: its
extended_checkscaller withholdspull-requests: read, which the calledcommit_messagesjob asks for, so GitHub refuses the run before any job starts. The playground nightly, dispatched by hand today, fails two samples on production: one wasm module no longer instantiates (afd_preadimport the runner's shim lacks) and the Exp loop sample's guard sits on its own float32 boundary. Its first run on this PR then showed the next break: ubuntu-latest became Ubuntu 26.04 on Oct 25, and the two linux lanes that name apt.llvm.org's noble repo by hand no longer install llvm-22-dev there, and the vulkan lane's lavapipe from the PPA's preliminary 26.04 build segfaults. Two more master regressions surfaced on the same run: the mingw lane compiles dasGlfw against an unpatched GLFW header because msys2's git applies nothing of the IME patch and exits 0, and the ubsan cell dies in the CUDA oracle's CPU reference casting a NaN to uint. Three smaller breaks ride along: the pages deploy reported a ladder service rejecting stores (HTTP 400) as "service down?" for three weeks, the rig's translation reference build died on a gcc 12 AVX-512 false positive on zen4, and name_lookup.cpp called std::stable_sort, fatal under -Werror with clang-19 on libstdc++ 12.What changes.
extended_checkscaller job grantspull-requests: read.git applywithGIT_DIRnaming a path that does not exist, so no git discovers this checkout above the extracted source (msys2 git 2.54, Git for Windows 2.55, Debian 2.39 and Apple 2.54 all apply the 16 header hunks that way).nolint:STYLE039on the Russian prompt literals also carriesLINT019: the rule fires only on the darwin flavor, whose Metal half compiles them, and the nightly's whole-folder linux lint (the only lint that walks files a PR did not touch) read the marker as stale.das-doc: givenmarkers for names their blocks assume (the lint page's STYLE048 example, the translate tutorial'slines); the nightly's darwin doc-verify is the only lane that compiles those blocks.fd_preadandfd_pwritewith BADF, like its preopen probes.set -eo pipefail.-Wno-error=maybe-uninitializedon a Linux box.das_stable_sort.Observable behavior.
daspkg_index / index_sweepandextended_checks (linux, all):llvm-22-dev : Depends: libllvm22 (= 1:22.1.8...) but 1:22.1.2-1ubuntu1 is to be installed-> the install resolves.vulkan / integration_linux:test_compute_features.das - crashed (exit status 139)inside libvulkan_lvp.so -> the suite runs on the 24.04 lavapipe.build_windows_mingw:use of undeclared identifier 'glfwSetPreeditCallback'-> compiles against the patched header.simulate_nodes.h:2167: -nan is outside the range of representable values of type 'unsigned int'intest_cuda_oracle.das-> the suite runs clean under UBSan.extended_checks (linux, all):LINT019: stale nolint - STYLE039 suppresses nothingon test_kernel_coverage.das -> clean on linux, and darwin's STYLE039 stays suppressed.extended_checks (darwin15, all)doc-verify:[red] lint.rst,[red] dasLLAMA_17_translate.rst-> both green (checked here withutils/internal/doc-verify).WebAssembly.instantiate(): Import #22 "wasi_snapshot_preview1" "fd_pread"-> runs.assert failed at main.das:20-> prints its timings.ladder import of m5.json failed (service down?)->::warning::ladder import of m5.json FAILED: {"error" : "models[0].runs[2].tune: longer than 1024"} HTTP 400.setup_asr_rig --refson Debian 12 / Zen 4:marian-decoder - did not materialize-> built.Where to look. The twelve hunks are independent; the permissions line in nightly.yml is the one that unblocks the cron, and the shim stub is the one a visitor sees.
#nightly
Validation, claims, ledger
Validation
GIT_CEILING_DIRECTORIESform applies 0 of 16 header hunks and exits 0 (-vprintsSkipped patchfor every file); theGIT_DIRform applies 16 with msys2 git, Git for Windows, Debian git on zen4 and Apple git on the M5._cuda_common.das:305, the math whitelist's CPU run), and the fixed test runs clean under a CI-flag ubsan build there, with the unfixed file as the control.#nightlyabove): astartup_failureconclusion would mean the fix is wrong."exp loop", 0.034475000, 10and no assert; wasm (queued, built by the production buildd, run) printed"exp loop", 0.005999000, 10and no assert. The unchanged source asserted in the same session first. Natively the sample runs clean in the interpreter and under JIT; numpy's float32 sequential sum of the same terms is 1,000,010, the value the sample computes.playground-e2elane runs the suite on this PR without WASM; the wasm-mode evidence for the shim is the nightly'splaygroundjob against production after this lands (the Linq sample's wasm import list exists only in the deployed build), which I will dispatch right after the deploy. The shim is copied verbatim into_site/playground/by the deploy.tests-cpp/small/test_name_lookup.cppcase controls the sort (comparator flipped: 1 case, 3499 assertions fail; restored: 8/8 cases pass).setup_asr_rig --checkreturned 0. The script line itself runs only on a fresh rig.set -euo pipefailwith a stubbed ssh; a bash probe confirmed the command substitution would otherwise exit the step (codex round finding, fixed).set -e), fixed; the two playground commits landed after the round and were not re-reviewed (a stub pair and one constant).Claims - stated, not tested
-Wno-errorin its/WXlist, and the Mac's clang needs none; neither arm has a test.Not done
utils/internal/dasllama-ladder/README.md); no workflow redeploys it after a schema change.