Skip to content

G1: the current stripe #2

Open
Open

Description

@tamnd

This tracks G1, the current stripe. This is the milestone rucc cares about most: it says which of today's kernels GCC 16 builds and boots on the tier 1 platforms, and it proves that rucc-kernel can read gcc-kernel cells as its reference. The full text is in docs/spec/12-milestones.md.

Harness

  • gk-boot, the QEMU rig with serial framing and boot budgets
  • gk-init for 2.6 and later, built per platform by the era GCC
  • Config fragments for eras M13 and M14
  • Boots to L6, KUnit to L7, and splat detection for L8
  • The three run rule for cells that reach L5, with flaky cells marked
  • The i386 platform and its toolchain bundles
  • The current stripe of 09.6, dense, on x86_64, i386 and arm64, with defconfig and tinyconfig
  • The GCC 16 column, dense over releases 5.0 to 7.x
  • The warning census of 11.5 on the Current set
  • The config differential across columns of 11.3 on the Current set
  • gk publish with the markdown heat maps

Open questions

  • 3: the measured cost of a modern cell, and whether tier 2 is cut back
  • 8: KVM against TCG on the era diagonal
  • 10: the measured cost of allmodconfig on the Current set

Elsewhere

  • rucc-kernel runs rk config-diff with a gcc-kernel GCC 16 cell directory as one side, unmodified

Exit criterion

  • The current stripe is complete, and every cell has a verdict or a reason for n/a
  • rk config-diff in rucc-kernel reads a gcc-kernel cell directory and produces a table, which proves the data contract of 10.4
  • reports/new-gcc-16.2.md exists, written as if 16.2 were new, as a rehearsal of 10.8

Activity

  1. added this to the G1: the current stripe milestone on Sep 30, 2026
  2. tamnd commented on Oct 1, 2026

    @tamnd
    OwnerAuthor

    #28 adds gk-init for 2.6 and later, plus gk init, which builds it and writes the initramfs.

    It's a static program with no libc, using raw syscalls on x86_64, i386 and arm64. The smoke suite has 18 checks, each framed with GK- marker lines. The archive is written by gk with fixed owners, times and inodes, and server2 and server3 produce identical digests for all three platforms.

    One deviation from the checklist wording: it isn't built by the era GCC. Each platform pins a GCC column and a host in platforms.toml, which is currently GCC 8.5 in the trixie host, with i386 using the x86_64 bundle and -m32. The 2.6 era columns aren't forged yet, and since every column of a row boots the same archive, a single pinned build is enough. Spec 07.4 now says this. I'll move the pin once older bundles exist.

    Next is the QEMU container and gk-boot.

  3. tamnd commented on Oct 1, 2026

    @tamnd
    OwnerAuthor

    #29 adds gk boot and the gk-boot QEMU container, so the rig now exists. It boots an image with the platform's initramfs under TCG with 2 CPUs and 1 GB, and writes boot.log and boot.json. It stops on power off, when the budget runs out, or after 5 quiet seconds following the end marker.

    As a first test I booted server2's Ubuntu 6.8 kernel, which passed 18 of 18 smoke checks in 15 seconds. That run also caught an init bug that put the wrong suite name on the end line, and it's fixed in the same PR.

    I'm leaving the rig item unticked until cells actually boot through it. That is the next PR: L5 and L6 in gk cell, plus the qemu and initramfs coordinates. A 7.2.8 x86_64 image is building on server2 to test it.

  4. tamnd commented on Oct 1, 2026

    @tamnd
    OwnerAuthor

    #30 is merged, and with #29 the QEMU rig is done, so that box is ticked.

    Cells now boot. gk cell 7.2.8 gcc-16.2.0 --platform x86_64 built in 31 minutes on server2 and reached L6 with 18 of 18 smoke checks. The gk-boot container digest and the initramfs digest are part of a booted cell's identity, and a cell run with --no-boot keeps its old identity.

    The three run rule also went in with #30: a cell that boots at all boots three times, takes the lowest rung, and is marked flaky when the runs disagree. I'll tick that box once the arm64 cell running on server2 has gone through it end to end.

    A finding from the first 7.2.8 boot: the M11 to M14 fragments build every KUnit suite in, and they all run before init. Under TCG the full set took 328 seconds on 7.2.8 x86_64, more than the 300 second boot budget. Smoke boots now turn KUnit off on the command line, and KUnit gets its own boot and budget in the next PR.

    Something to plan for: L7 grades KUnit against the era GCC's cell, and for 7.2.8 that is gcc-14.2.0, which we have not forged. gccs.toml only has the newest point release of each series. Until 12.2.0 and 14.2.0 are forged, cells will record that they had nothing to grade against and stay at L6.

  5. tamnd commented on Oct 1, 2026

    @tamnd
    OwnerAuthor

    #31 is merged: KUnit boots and L7 grading.

    A cell whose config has KUnit now boots a second time after each smoke boot that passed. That boot runs every suite, with its own budget of 1200 seconds, and kunit.json grades the suites against the era GCC's cell: only suites the era cell passed in all three runs count, and a run reaches L7 when it passed all of them.

    On the 7.2.8 x86_64 GCC 16.2 image, the KUnit boot read 176 suites in 283 seconds, and 6 failed, all of them the generic page table iommu suites. Whether that is GCC 16 or the kernel will be clear once the era GCC cell has run. gcc-14.2.0 and gcc-12.2.0 are forging on server2 now for exactly that.

    The "Boots to L6, KUnit to L7, and splat detection for L8" box stays open until L8 is in. One thing L8 has to deal with: that KUnit boot logged 19 splats from tests that warn on purpose, so splats on a KUnit boot have to be compared against the reference rather than counted outright.

  6. tamnd commented on Oct 1, 2026

    @tamnd
    OwnerAuthor

    #32 is merged, which puts L8 in place: objtool warnings from the build logs and splats from the console. KUnit splats only count when the era GCC's cell did not show the same one. They are matched by a key without the CPU, task and offsets, because the 7.2.8 KUnit boot logs 19 warnings from suites that warn on purpose.

    All the rungs are now there in code. I'll tick the ladder box once a cell has climbed past L6 for real, which needs the gcc-14.2.0 era cell. That GCC is forging on server2 now.

    Released v0.1.1 with everything from #28 to #32: https://github.com/tamnd/gcc-kernel/releases/tag/v0.1.1

  7. tamnd commented on Oct 1, 2026

    @tamnd
    OwnerAuthor

    #34 is merged: gk publish now writes reports/matrix-<platform>.md heat maps, one letter per cell, with a * on flaky cells and the newest cell winning when a square has been run more than once. That ticks the gk publish box. The per-GCC and per-kernel pages come later with gk explain.

    On the dev store it gives one row so far, 7.2.8 with R under GCC 16.2 on x86_64. The real tables come with the current stripe.

  8. tamnd commented on Oct 1, 2026

    @tamnd
    OwnerAuthor

    #35 adds gk config-diff. It compares the config of the newest cells for two GCC columns, or every pair of neighbouring columns in the store when none are named, and prints a markdown table per pair. I'm leaving the config differential item open until it has been run over the Current set, which needs more than one column per kernel in the store first.

    Still running on server2: the arm64 7.2.8 cell (three boots end to end), and forge4 building gcc-14.2.0, the era GCC that 7.2.8's KUnit and splat grading is measured against.

  9. 18 remaining items

  10. tamnd commented on Oct 3, 2026

    @tamnd
    OwnerAuthor

    Released v0.1.7 (#71), with the KUnit skip in the configuration, the buster and bullseye hosts, and the two new reports. Both stripe queues are still going, with the 7.2.8 rows on x86_64 and i386 nearly through their columns.

  11. tamnd commented on Oct 4, 2026

    @tamnd
    OwnerAuthor

    #105 keeps the reports from going blank when kernel.org moves a Current line. The pins moved to 7.2.9, 6.18.55 and 6.12.112 in #85. Until those points have cells, the warning census and the configuration differential read 7.2.8, 6.18.54 and 6.12.111, which G1 has already run, and they switch over once the new points run. The stripe's progress bars still count only the pinned points, so they read 0 until the new rows run.

    server3 is still on the dense 6.12.111 x86_64 row. 11.5.0 works and 12.5.0 builds, with builds of about 42 minutes each under a load near 100.

  12. tamnd commented on Oct 4, 2026

    @tamnd
    OwnerAuthor

    #109 fixes the progress bars of the README. The stripe counted every upstream column, so the G3 columns from 2.5.8 to 6.3.0 were added to every Current row, and the bars also lost the cells of the Current kernels that kernel.org had moved to a new point. The stripe is now every upstream column from 8.1, as in spec 09.6, and a moved pin counts the cells of its point before until the new one has run. After the publish of #110 the stripe stands at 56 of 546 crossings.

  13. tamnd commented on Oct 5, 2026

    @tamnd
    OwnerAuthor

    #122 makes a search run a stored cell again when its build went over the budget. The dense arm64 row of 6.18.54 on server2 lost gcc-12.2.0 and gcc-12.5.0 to the 90 minute budget under a load near 20, while gcc-11.5.0 next to them reaches L6. The queue picks those two up again on its next pass instead of leaving them in the matrix as failures.

    server3 is on the tinyconfig x86_64 rows now, at 6.1.188, about five minutes a cell.

  14. tamnd commented on Oct 5, 2026

    @tamnd
    OwnerAuthor

    #128 publishes 1586 cells from both machines, up from 95. That covers the current stripe cells for GCC 16.2 so far (6.1 to 6.18 on x86_64 run at L6), the frontier searches of the 2.6 history and the museum. reports/eras.md now has rows from M0 on.

  15. tamnd commented on Oct 6, 2026

    @tamnd
    OwnerAuthor

    The tinyconfig cells of the current stripe on x86_64 turned out to be 32 bit kernels, as ARCH=x86 with allnoconfig turns 64BIT off. #136 builds the row with ARCH=x86_64. The 84 tinyconfig cells on x86_64 run again on server3, so the stripe item stays open until they are back.

  16. tamnd commented on Oct 7, 2026

    @tamnd
    OwnerAuthor

    A finding on the current stripe from #173: 5.15.221 and 5.15.222 on x86_64 reach L7 with every GCC from 8 to 14 and then fail L8 on objtool: fred_sysvec_reboot()+0x48: unreachable instruction. The fred_ entry wrappers came to 5.15 in 5.15.168, so the stable backport is at fault and not GCC. It is the stable-objtool-fred-reboot signature, with no fix yet.

  17. tamnd commented on Oct 7, 2026

    @tamnd
    OwnerAuthor

    Another finding on the current stripe, from #175: 6.12.111 fails L7 with GCC 15.3, 16.1 and 16.2 on i386 and x86_64 because the overflow KUnit suite's DEFINE_FLEX_test expects __builtin_dynamic_object_size to ignore __counted_by, which GCC 15 is the first to support. 04e403e6627d fixed the test in 6.15-rc1, and it is not on 6.12.y. This is gcc15-counted-by-kunit.

  18. tamnd commented on Oct 7, 2026

    @tamnd
    OwnerAuthor

    #179: the 6.12.111 arm64 row is all rig failures. Seven cells found no kernel tree because it was removed from the cache while they started, and the rest went over budget or hit a full disk. Those seven are now tree-gone (host-tool), and rig_failed covers them, so a search reruns them. The row will be rerun once server3's queue drains. Unclassified is 296.

  19. tamnd commented on Oct 7, 2026

    @tamnd
    OwnerAuthor

    #187 names the RCU stall that every GCC hits on 5.10.270 x86_64 at L8. It is the rtc lib KUnit test walking 160,000 years under TCG, which upstream split in 95c46336ab47 (v6.10-rc1) and 5.10.y never took. The cells get the emulator kind, and unclassified goes from 272 to 264.

  20. tamnd commented on Oct 7, 2026

    @tamnd
    OwnerAuthor

    #188 names the NULL dereference that every GCC hits on 6.1.188 and 6.1.189 in KUnit. It comes from the 6.1.y backport of the longest-symbol test, so the kernel is at fault for every compiler. The PR adds a kernel-bug kind to spec 08.3 for this kind of failure. Unclassified goes from 264 to 263, and the 6.1.188 row will grade once its gcc-12.2.0 reference cell is rerun on server3.

  21. tamnd commented on Oct 7, 2026

    @tamnd
    OwnerAuthor

    #196 excuses the cpus smoke check when the kernel is built without SMP. The eight 7.2.9 tinyconfig cells on x86_64 that stopped at L5 only failed that check, because tinyconfig leaves SMP off and QEMU gives two CPUs. They will run again on server3.

  22. tamnd commented on Oct 7, 2026

    @tamnd
    OwnerAuthor

    #200: the first i386 tinyconfig+gk cell (7.2.9, GCC 16.2.0) booted, but its clock check failed. tinyconfig switches off COMPAT_32BIT_TIME, and gk-init's 32-bit clock_gettime and nanosleep need it. configs/tiny.i386 and configs/tiny.arm now turn it back on, and the x86_64 and arm64 tiny cells keep their digests. The cell will be run again once the current queue gets to it.

  23. tamnd commented on Oct 9, 2026

    @tamnd
    OwnerAuthor

    #229 publishes 2765 cells, with 796 new ones from server3. The current stripe is at 210 of 546 cells (38%, up from 171), with x86_64 defconfig at 84 of 91. The GCC 16 column is still at 63 of 346. Its arm64 defconfig cells take about ten hours each on server3, and two of them (7.2.9 and 6.18.55) were lost to a full disk and are queued again, along with the 6.12.112 x86_64 tinyconfig row.

    v0.1.19 is out with the bisection and forge changes since v0.1.18.

  24. tamnd commented on Oct 9, 2026

    @tamnd
    OwnerAuthor

    #247 keeps cells that a full disk stopped out of the matrix and the release reports. Those cells had made reports/new-gcc-16.2.md show GCC 16.2 failing on 7.2.9 i386 and on 6.12.112 and 7.2.9 arm64, when the build machine had simply run out of space.

    Those GCC 16.2 cells are queued again on server3, along with 6.18.55, 6.1.189 and 5.15.222 on arm64. One arm64 defconfig build takes about eleven hours on the shared machine, so the arm64 half of the stripe will fill in slowly. server3 has eight cores, and I have stopped starting new parallel chains. The remaining cells go through one queue that starts a cell only when fewer than five of ours are running.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions