flasher: bake test overlay directly into rootfs partition

[Manigurr]

Problem

LAVA jobs for rb3gen2-core-kit (e.g. job 253181 on the kodiak_testing
pipeline) consistently fail at:

-sh: /lava-253181/environment: No such file or directory
<LAVA_SIGNAL_RETRUNCODE 1>
'. /lava-253181/environment' failed with return code 1!

templates/boot/flasher.jinja2 downloads the combined _build.tar
and the device-side flasher dds it directly onto storage as a raw
disk image — no separate rootfs.img is ever extracted or produced
anywhere in this flow. The postprocess step nonetheless writes:

OVERLAY=overlay.tar.gz
OVERLAY_PATH=/
ROOTFS_IMAGE=rootfs.img

into flash.settings, pointing at a rootfs.img file that never
exists. The device-side flasher has nothing to apply the overlay to,
so the LAVA test overlay (which contains the environment file every
lava-test-shell action sources) silently never makes it onto the
booted device.

Fix

In the postprocess docker step:

1. Locate the GPT partition named rootfs inside the downloaded raw
   image via sfdisk -d.
2. dd just that partition out to its own file.
3. Bake the test overlay (LAVA overlay + Test_binaries.tar.gz) into
   it with e2mkdir/e2cp (no mount/loop devices, no extra container
   privileges needed).
4. Verify /lava-<job>/environment is present inside the partition
   before continuing — fail fast with a clear message otherwise.
5. dd the patched partition back into place at the same offset.

The now-dead overlay: deploy image and OVERLAY= / OVERLAY_PATH=
/ ROOTFS_IMAGE= settings are removed, since the overlay ships
pre-baked inside the single flashed image.

Testing

Validated end-to-end locally against a synthetic GPT image (boot +
rootfs partitions, matching the real device's PARTLABEL=rootfs
layout seen in the job's boot log) using the exact
ghcr.io/mwasilew/docker-mkbootimage:master postprocess container:

• Rendered the Jinja2 template and confirmed it parses as valid YAML.
• Ran the extracted postprocess steps unmodified inside the real
  container against the synthetic image + a fake LAVA overlay +
  Test_binaries.tar.gz.
• Independently re-extracted the resulting rootfs partition afterward
  and confirmed via e2ls/e2cp that /lava-<job>/environment and
  the Test_binaries payload are both present and intact.

One assumption worth a sanity check against a real
rb3gen2-core-kit_build.tar before merging: that its GPT partition is
named exactly rootfs (matches root=PARTLABEL=rootfs and the
EXT4-fs (sda2) mount seen in job 253181's boot log).

[Manigurr]

Update: the original version of this PR wrongly assumed flash_image was a raw GPT-partitioned disk image, and tried to locate a rootfs partition via sfdisk. Tested against the real rb3gen2-core-kit_build.tar (job 253372), this failed immediately:

sfdisk: rb3gen2-core-kit_build.tar: does not contain a recognized partition table

Downloaded and inspected the real artifact - it's an actual POSIX tar archive containing Qualcomm Firehose/QDL flashing files (rawprogram5.xml, gpt_backup0.bin, xbl.elf, patch2.xml) plus a top-level rootfs.img:

$ tar -tvf rb3gen2-core-kit_build.tar
drwxrwxr-x ubuntu/ubuntu     0 ./
-rw-r--r-- ubuntu/ubuntu  2407 ./rawprogram5.xml
-rw-r--r-- ubuntu/ubuntu 20480 ./gpt_backup0.bin
-rw-r--r-- ubuntu/ubuntu 840000 ./xbl.elf
-rw-r--r-- ubuntu/ubuntu  6263 ./patch2.xml
-rw-r--r-- ubuntu/ubuntu 2579496960 ./rootfs.img

Pushed a corrected version: extract the tar, bake the overlay directly into flash_content/rootfs.img with e2cp/e2mkdir (no partition offset math at all), verify /lava-<job>/environment, then re-pack the tar with the baked rootfs.img swapped in - every other member preserved untouched. Also dropped the postprocess timeout from 60 to 45 minutes (LAVA rejected 60 with "Action timeout is larger than job timeout" against the 50-minute job timeout).

Re-validated end-to-end: built a synthetic tar from the real small members (downloaded above) + a local ext4 rootfs.img, ran the exact rendered postprocess steps in the real ghcr.io/mwasilew/docker-mkbootimage:master container, and confirmed afterward that the repacked tar still contains every original member untouched with the overlay now present in rootfs.img.

[Manigurr]

Second correction: in-place byte-offset patch instead of full re-tar

Real job 253378 ran with the previous fix (commit 9b15cad) and the device actually booted cleanly end-to-end — SBL/XBL/UEFI/Linux boot, login prompt reached — but hit the original symptom again: -sh: /lava-253378/environment: No such file or directory.

The flash step's own log gave it away:

Calling: ['flash-universal.sh', '-s', 'settings/flash.settings']
>> 835+1 records in
>> 835+1 records out
>> 3506257920 bytes (3.5 GB, 3.3 GiB) copied, 1.01281 s, 3.5 GB/s

A single dd of the entire 3.3GB combined tar at a physically-impossible 3.5 GB/s — this is handing the raw tar bytes directly to the board's EDL/Sahara/Firehose loader, which almost certainly locates rootfs.img (and the other images) inside the tar via byte-exact offsets described in the embedded rawprogram*.xml/patch*.xml. The previous fix fully extracted the tar and rebuilt it from scratch with tar -cf — which doesn't guarantee identical member order, header format, or padding versus the original. That's enough to desync the firehose loader's offset parsing, so our modified rootfs.img likely never actually got written to the device; it just booted whatever rootfs was already on that test rig.

Fix (commit 6903943)

Stop re-tarring entirely. Patch rootfs.img in place at its exact byte offset inside the original tar:

1. Use Python's tarfile module to get rootfs.img's offset_data and size inside the combined tar (no full extraction needed).
2. dd just that byte range out (iflag=skip_bytes,count_bytes).
3. Bake the overlay into it with e2cp/e2mkdir as before — ext4 images are fixed-size, so this never changes the file's size.
4. dd the baked bytes back into the original tar at the same offset with conv=notrunc,oflag=seek_bytes — no truncation, no resize.

Every other byte of the tar (headers, member order, padding, trailer) stays byte-for-byte identical to the original artifact.

Local verification

Built a synthetic combined tar matching the real one's member layout (rawprogram5.xml, gpt_backup0.bin, xbl.elf, patch2.xml, rootfs.img last) and ran the exact postprocess logic through the real ghcr.io/mwasilew/docker-mkbootimage:master container:

• Confirmed the region of the tar before and after rootfs.img's data span is byte-for-byte identical (cmp) to the pre-patch original.
• Confirmed the rootfs.img data region did change (overlay baked in).
• Confirmed a plain tar -xf extraction of the patched tar yields a rootfs.img whose e2ls shows the overlay (/lava-999999/environment, /Lava_binaries/dummy_tool) baked in.

Not yet verified against real hardware — that'll show in the next LAVA run.