Skip to content

Memory leaks with sysmon 1.3.7 #211

Description

Describe the bug
I'm receiving reports from a sysadmin of Sysmon 1.3.7 leaking memory in the region of 5MB an hour on a number of Ubuntu 22.04 machines.

Following the previous issue #170 I've ran Sysmon from the main branch at commit 8283661 with valgrind for a short period and it does appear to indicate that there are more memory leaks.

To Reproduce

  • Install sysmon
  • Observe RSS usage increase over time
  • Wait for OOM killer or Restart service

Sysmon version
1.3.7

Distro/kernel version
Ubuntu 22.04.01 - 6.8.0-1030-gcp

Sysmon configuration
The Sysmon configuration when the issue occurred.

Logs
Valgrind

==35150== Memcheck, a memory error detector
==35150== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==35150== Using Valgrind-3.22.0 and LibVEX; rerun with -h for copyright info
==35150== Command: /opt/sysmon/sysmon -i /opt/sysmon/config.xml -service
==35150== Parent PID: 35057
==35150==
==35150== Syscall param bpf(attr->expected_attach_type) points to uninitialised byte(s)
==35150==    at 0x4F6D25D: syscall (syscall.S:38)
==35150==    by 0x487A876: sys_bpf (bpf.c:75)
==35150==    by 0x487A876: sys_bpf_fd (bpf.c:83)
==35150==    by 0x487A876: sys_bpf_prog_load (bpf.c:92)
==35150==    by 0x487F930: probe_kern_prog_name (libbpf.c:4527)
==35150==    by 0x488300E: kernel_supports (libbpf.c:4910)
==35150==    by 0x488300E: kernel_supports (libbpf.c:4898)
==35150==    by 0x48852AC: bpf_object__create_map (libbpf.c:5034)
==35150==    by 0x48903B5: bpf_object__create_maps (libbpf.c:5296)
==35150==    by 0x4891692: bpf_object_load (libbpf.c:7738)
==35150==    by 0x4891692: bpf_object__load (libbpf.c:7787)
==35150==    by 0x48749A5: ebpfStart (telemetryLoader.c:1366)
==35150==    by 0x48749A5: ebpfStart (telemetryLoader.c:1303)
==35150==    by 0x48750ED: telemetryStart (telemetryLoader.c:1556)
==35150==    by 0x17E808: main (sysmonforlinux.c:1681)
==35150==  Address 0x1ffeffe9b4 is on thread 1's stack
==35150==  in frame #2, created by probe_kern_prog_name (libbpf.c:4510)
==35150==
==35150== Syscall param bpf(attr->prog_ifindex) points to uninitialised byte(s)
==35150==    at 0x4F6D25D: syscall (syscall.S:38)
==35150==    by 0x487A876: sys_bpf (bpf.c:75)
==35150==    by 0x487A876: sys_bpf_fd (bpf.c:83)
==35150==    by 0x487A876: sys_bpf_prog_load (bpf.c:92)
==35150==    by 0x487F930: probe_kern_prog_name (libbpf.c:4527)
==35150==    by 0x488300E: kernel_supports (libbpf.c:4910)
==35150==    by 0x488300E: kernel_supports (libbpf.c:4898)
==35150==    by 0x48852AC: bpf_object__create_map (libbpf.c:5034)
==35150==    by 0x48903B5: bpf_object__create_maps (libbpf.c:5296)
==35150==    by 0x4891692: bpf_object_load (libbpf.c:7738)
==35150==    by 0x4891692: bpf_object__load (libbpf.c:7787)
==35150==    by 0x48749A5: ebpfStart (telemetryLoader.c:1366)
==35150==    by 0x48749A5: ebpfStart (telemetryLoader.c:1303)
==35150==    by 0x48750ED: telemetryStart (telemetryLoader.c:1556)
==35150==    by 0x17E808: main (sysmonforlinux.c:1681)
==35150==  Address 0x1ffeffe9b0 is on thread 1's stack
==35150==  in frame #2, created by probe_kern_prog_name (libbpf.c:4510)
==35150==
==35150== Syscall param bpf(attr->value) points to uninitialised byte(s)
==35150==    at 0x4F6D25D: syscall (syscall.S:38)
==35150==    by 0x487B6D1: sys_bpf (bpf.c:75)
==35150==    by 0x487B6D1: bpf_map_update_elem (bpf.c:394)
==35150==    by 0x4874A76: ebpfStart (telemetryLoader.c:1405)
==35150==    by 0x4874A76: ebpfStart (telemetryLoader.c:1303)
==35150==    by 0x48750ED: telemetryStart (telemetryLoader.c:1556)
==35150==    by 0x17E808: main (sysmonforlinux.c:1681)
==35150==  Address 0x1ffeffee84 is on thread 1's stack
==35150==  in frame #2, created by ebpfStart (telemetryLoader.c:1314)
==35150==
==35150== Syscall param bpf(attr->value) points to uninitialised byte(s)
==35150==    at 0x4F6D25D: syscall (syscall.S:38)
==35150==    by 0x487B6D1: sys_bpf (bpf.c:75)
==35150==    by 0x487B6D1: bpf_map_update_elem (bpf.c:394)
==35150==    by 0x4874AA5: ebpfStart (telemetryLoader.c:1412)
==35150==    by 0x4874AA5: ebpfStart (telemetryLoader.c:1303)
==35150==    by 0x48750ED: telemetryStart (telemetryLoader.c:1556)
==35150==    by 0x17E808: main (sysmonforlinux.c:1681)
==35150==  Address 0x1ffeffee84 is on thread 1's stack
==35150==  in frame #2, created by ebpfStart (telemetryLoader.c:1314)
==35150==
--35150-- WARNING: unhandled eBPF command 28
--35150-- WARNING: unhandled eBPF command 28
--35150-- WARNING: unhandled eBPF command 28
==35150==
==35150== HEAP SUMMARY:
==35150==     in use at exit: 230,146 bytes in 501 blocks
==35150==   total heap usage: 630,473 allocs, 629,972 frees, 846,234,225 bytes allocated
==35150==
==35150== 0 bytes in 1 blocks are definitely lost in loss record 1 of 268
==35150==    at 0x4846828: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==35150==    by 0x1807B8: setConfigFromStoredArgv (sysmonforlinux.c:1070)
==35150==    by 0x17DEDE: main (sysmonforlinux.c:1442)
==35150==
==35150== 189,142 bytes in 192 blocks are definitely lost in loss record 268 of 268
==35150==    at 0x4846828: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==35150==    by 0x1955B4: ProcessCache::ProcessAdd(GUID, SYSMON_EVENT_HEADER*) (eventsCommon.cpp:366)
==35150==    by 0x195C41: GenerateUniquePGUID(GUID*, SYSMON_EVENT_HEADER*, bool) (eventsCommon.cpp:499)
==35150==    by 0x19738A: EventResolveField (eventsCommon.cpp:1888)
==35150==    by 0x197619: EventProcess(SYSMON_EVENT_TYPE_FMT*, SYSMON_DATA_DESCRIPTOR*, SYSMON_EVENT_HEADER*, unsigned long*) (eventsCommon.cpp:2463)
==35150==    by 0x198773: DispatchEvent (eventsCommon.cpp:2922)
==35150==    by 0x17FE2F: processProcessCreate (sysmonforlinux.c:623)
==35150==    by 0x4881B40: perf_buffer__process_record (libbpf.c:11925)
==35150==    by 0x4881C93: perf_event_read_simple.constprop.0 (libbpf.c:11553)
==35150==    by 0x488FAC1: perf_buffer__process_records (libbpf.c:11947)
==35150==    by 0x488FAC1: perf_buffer__poll (libbpf.c:11972)
==35150==    by 0x4875199: telemetryStart (telemetryLoader.c:1572)
==35150==    by 0x17E808: main (sysmonforlinux.c:1681)
==35150==
==35150== LEAK SUMMARY:
==35150==    definitely lost: 189,142 bytes in 193 blocks
==35150==    indirectly lost: 0 bytes in 0 blocks
==35150==      possibly lost: 0 bytes in 0 blocks
==35150==    still reachable: 38,988 bytes in 287 blocks
==35150==         suppressed: 0 bytes in 0 blocks
==35150== Reachable blocks (those to which a pointer was found) are not shown.
==35150== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==35150==
==35150== Use --track-origins=yes to see where uninitialised values come from
==35150== For lists of detected and suppressed errors, rerun with: -s
==35150== ERROR SUMMARY: 7 errors from 7 contexts (suppressed: 0 from 0)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions