periphery: ignore host ZFS ARC when /proc/meminfo comes from lxcfs - #1635
Open
phedoreanu wants to merge 1 commit into
Open
phedoreanu wants to merge 1 commit into
phedoreanu wants to merge 1 commit into
Conversation
Inside an LXC container lxcfs serves a cgroup-scoped /proc/meminfo, but /proc/spl/kstat/zfs/arcstats is not virtualized and still reports the host's ARC. Subtracting that ARC from the container's used memory saturates to zero on any container smaller than the host's ARC, so Komodo shows 0% RAM for every LXC guest on a ZFS Proxmox host. Detect the lxcfs case by checking /proc/self/mountinfo for a FUSE filesystem mounted on /proc/meminfo and skip the ARC adjustment there. Bare-metal and Docker-on-host setups keep the ARC handling from moghtech#1489.
Author
|
Verified on a live host. Proxmox 9 (kernel 7.0.14-17-pve, lxcfs 7.0.0-pve1, rpool on ZFS with a 6.28 GiB ARC), unprivileged Ubuntu LXC with a 1.5 GiB limit, Periphery as a systemd binary (not in Docker). Consecutive stats records for that server in Core's
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Since #1489 (shipped in 2.3.0), Periphery subtracts the ZFS ARC size from used memory. Inside an LXC container on a ZFS Proxmox host that goes wrong: lxcfs serves a cgroup-scoped
/proc/meminfo, but/proc/spl/kstat/zfs/arcstatsis not virtualized and still shows the host's ARC. The subtraction saturates to zero, so every LXC guest smaller than the host's ARC reports0.00 GBused.This is the mechanism described in #1139 (comment). On my fleet, four LXC containers (1 to 2 GiB each) on a host with a 6.3 GiB ARC have reported
mem_used_gb: 0in every stats record since they picked up 2.3.x:Fix
Check
/proc/self/mountinfofor a FUSE filesystem mounted on/proc/meminfo. When that is the case the meminfo numbers are container-scoped and the ARC cannot be part of them, so the ARC is treated as zero (both forusedand for the reportedmem_zfs_arc_gb).I preferred this over the
arc >= totalclamp suggested in the issue because the clamp still subtracts the full host ARC from any container whose limit is larger than the ARC (for example an 8 GiB container on a host with a 6 GiB ARC), which underreports instead of zeroing. Bare-metal hosts and Periphery running in Docker directly on a ZFS host see the real/proc/meminfoand keep the ARC handling from #1489 unchanged.The second scope mismatch from that comment (Periphery in Docker inside an LXC seeing its own cgroup) is a separate problem and not addressed here.
Testing
parse_zfs_arc_size.cargo test -p komodo_periphery stats::memandcargo fmt --all -- --checkpass.Refs #1139, #125.