Skip to content

docker: fix the cache directory, keep the code read-only and install only the vCenter bindings - #573

Open
semx wants to merge 2 commits into
developmentfrom
chore/slim-image
Open

semx wants to merge 2 commits into
developmentfrom
chore/slim-image

Conversation

@semx

@semx semx commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

Two changes to the image, both checked by running the image against NetBox, not only by building it.

Caching was off in every container. /app belongs to root, so the service user cannot create the default cache directory /app/cache, and every run logs Error writing to cache directory: /app/cache / NetBox caching DISABLED and fetches all objects from NetBox again. The code itself was copied with the service user as owner, so the process could rewrite it. Now the code is copied as root and read-only for the service user, and /app/cache is created owned by the service user and group 0 with mode 0770 (group 0 covers platforms that run images with an arbitrary uid).

Smaller, pinned SDK. vcf-sdk is a meta package that pulls every VMware Cloud Foundation binding (NSX, SDDC Manager, Operations, Fleet LCM, Installer, vSAN data protection). netbox-sync only uses create_vsphere_client and the tagging client from vmware-vcenter, which is now installed pinned to the version vcf-sdk resolves to today (9.1.1.0), so rebuilding a tag gives the same SDK. All 552 Python files under vmware/ and com/ are byte-identical to the current image; the 155 that are gone belong to SDDC Manager, the VCF installer and snapservice. Also drops the apt-get update line that installed nothing.

current development image this PR
size, uncompressed 339 MB 248 MB
size, gzipped 58 MB 52 MB
NetBox cache disabled every run 26 cache files after a run

Functional check, both images the same way, under a rootless Docker daemon (Docker 29.1.3 on Ubuntu 26.04, docker info reports name=rootless): NetBox 4.4.5 in containers, two vcsim instances replaying the captured vCenters from tests/fixtures/vcsim (vchvr 6.7 with 28 VMs, vc001 8.0.3 with 34 VMs) as two sources in one config, netbox-sync run with --read-only --cap-drop ALL --security-opt no-new-privileges, settings mounted read-only, a volume on /app/cache, two passes each:

current image this PR
pass 1 exit 0, 0 errors, 1142 created, 253 updated exit 0, 0 errors, 1142 created, 253 updated
pass 2 exit 0, 0 errors, 0 created, 1 updated exit 0, 0 errors, 0 created, 1 updated
NetBox after each pass 5 devices, 30 interfaces, 62 VMs, 96 VM interfaces, 2 clusters, 126 MAC addresses the same
warnings and errors identical line for line, apart from the two cache warnings
cache NetBox caching DISABLED on every run enabled, 26 cache files

Rootless mode needs nothing from the image beyond what this keeps: it runs as a non-root user, opens no ports and writes only to /app/cache. --help works and the process runs as uid 1000 in both. vCenter tag sync cannot be exercised against vcsim (it does not implement the vAPI endpoint the SDK talks to), which is why the SDK files are compared byte for byte instead.

This is independent of the release; fine to merge before or after v1.9.0.

vcf-sdk is a meta package that pulls every VMware Cloud Foundation binding
(NSX, SDDC Manager, Operations, Fleet LCM, Installer, vSAN data protection).
netbox-sync only needs create_vsphere_client and the tagging client, which
come from vmware-vcenter and the vapi runtime and common client it depends
on. Install vmware-vcenter pinned to the version vcf-sdk resolves to today,
so a rebuild of the same tag gets the same SDK, and drop the apt-get line
that installed nothing.

Built from the same development commit: 339 MB -> 248 MB uncompressed,
58 MB -> 52 MB gzipped. All 552 Python files under vmware/ and com/ in the
new image are byte-identical to the current one; the 155 files that are no
longer installed belong to SDDC Manager, the VCF installer and snapservice.
pyvmomi, vmware-vapi-runtime, vmware-vapi-common-client and vmware-vcenter
stay at 9.1.1.0, --help works and the process still runs as uid 1000.
The application files were copied with the service user as owner, so the
running process could rewrite its own code, while /app itself stayed owned
by root, so the default cache directory (/app/cache) could not be created
and every container run logged "NetBox caching DISABLED" and fetched all
objects from NetBox again.

Copy the code as root, read-only for the service user, and create /app/cache
owned by the service user and group 0 with mode 0770, which also covers
platforms that run the image with an arbitrary uid in group 0.

Checked with --read-only --cap-drop ALL --security-opt no-new-privileges and
a volume on /app/cache: the cache is writable, the code is not, --help works,
and an arbitrary uid in group 0 can write the cache. Before this change the
same checks failed on the cache and succeeded on writing the code.
@semx
semx requested a review from bb-Ricardo as a code owner September 30, 2026 09:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant