[BUG] Memory leak during web uploads (Stream method): Go live heap grows unbounded, container gets OOM-killed

[alexzandermatveev]

Please confirm the following

• I have read and agree to AGPL-3.0 Section 15 .
  The program is provided "as is" without any warranties; you bear all risks of using it.

• I have read and agree to AGPL-3.0 Section 16 .
  The copyright holders and distributors are not liable for any damages resulting from the use or inability to use the program.

• I confirm my description is clear, polite, helps developers quickly locate the issue, and complies with community rules.

• I have read the OpenList documentation.

• I confirm there are no duplicate issues or discussions.

• I confirm this is an OpenList issue, not caused by other reasons (such as network, dependencies, or operation).

• I believe this issue must be handled by OpenList and not by a third party.

• I confirm this issue is not fixed in the latest version.

• I have not read these checkboxes and therefore I just ticked them all, Please close this issue.

OpenList Version (required)

4.2.6

Storage Driver Used (required)

Teldrive

Bug Description (required)

During a multi-file upload from the web UI the Go LIVE heap grows monotonically and is never
freed while uploads are in flight. GC cannot reclaim the memory (objects stay referenced),
runs in a death spiral near GOMEMLIMIT, and the process eventually exceeds its container
memory limit and is killed by the kernel (cgroup OOM). Consequences: all sessions invalidated
(server-side token cache is in-memory), reverse proxy returns 502, upload tasks are dropped.

Key facts:

• Grows with the NUMBER of parallel upload streams, not with file size: ~15 files x ~200 KB
  (about 3 MB of data total) pushed the live heap from ~437 MB to 557+ MB.
• Persists with chunk_size=2 MiB and upload_concurrency=1, so it is not driver chunk buffering.
• Memory is released instantly (RSS drops to ~68 MB) the moment the upload batch finishes —
  the memory is referenced only while uploads are in flight.
• GOMEMLIMIT=380MiB cannot help: GC runs continuously but the heap is live, not garbage.
• The built-in backpressure (min_free_memory / auto_memory_limit) reads host-wide free memory
  (/proc/meminfo), which is wrong when OpenList itself runs in a container with its own limit —
  it never throttles even though the container is about to be killed.

Expected behavior: upload pipeline should apply backpressure based on its own cgroup memory
usage (e.g. /sys/fs/cgroup/memory.current) and/or stop retaining references to completed
upload state. Ideally: pause accepting new chunks/streams when the process approaches its
memory limit instead of being OOM-killed.

Logs (required)

GODEBUG=gctrace=1 during a Stream upload (~15 files, ~200 KB each, ~3 MB total):

gc  92 @107.4s  437->437->437 MB, 437 MB goal
gc 130 @109.6s  488->488->488 MB
gc 160 @111.4s  532->536->536 MB
gc 171 @112.1s  554->557->554 MB

Hundreds of GC cycles within seconds (death spiral), live heap (last number) keeps climbing.

Kernel side (cgroup limit 320 MiB at the time):

[12:52:28] oom-kill:constraint=...,task=openlist,uid=1001
[12:52:28] Out of memory: Killed process 67767 (openlist) total-vm:3923960kB, anon-rss:731308kB
[13:32:02] Memory cgroup out of memory: Killed process 74546 (openlist) anon-rss:301604kB
[13:40:20] Memory cgroup out of memory: Killed process 75528 (openlist) anon-rss:321912kB

After the batch finishes, RSS instantly drops to ~68 MB — nothing was freed by GC while uploading.

Configuration File Content (required)

config.json (sensitive fields masked):

{
  "force": false,
  "site_url": "",
  "cdn": "",
  "jwt_secret": "<masked>",
  "token_expires_in": 48,
  "database": { "type": "sqlite3", "db_file": "data/data.db", "table_prefix": "x_" },
  "scheme": { "address": "0.0.0.0", "http_port": 5244 },
  "temp_dir": "data/temp",
  "log": { "enable": true, "name": "data/log/log.log" },
  "auto_memory_limit": 200,
  "min_free_memory": 120,
  "max_concurrency": 64,
  "s3": { "enable": false },
  "ftp": { "enable": false },
  "sftp": { "enable": false },
  "mcp": { "enable": false },
  "last_launched_version": "v4.2.6"
}

Teldrive storage driver addition (sensitive fields masked):

{
"root_folder_path": "/",
"url": "http://openlistfix:8080",
"cookie": "access_token=",
"use_share_link": false,
"chunk_size": 2,
"random_chunk_name": false,
"upload_concurrency": 1
}

AI Generated Content

• I used AI tools to generate this content
• I did not use AI tools to generate this content

AI model used

GLM 5.3 Flash

[jyxjjj]

shirou/gopsutil#1758