Repository navigation
Freeze the objects allocated before forking the gunicorn workers - #7584
Merged
Merged
Conversation
marco-c
force-pushed
the
web-gc-freeze
branch
from
November 19, 2026 20:39
17422a7 to
b9ed792
Compare
With --preload, the workers share the master's memory until they write to it, and a full garbage collection writes to every object it traverses: when the preloaded heap is large, this unshares it in every process. With the v0.0.655 production image and traffic, each of the four gunicorn processes got its own copy of ~170 MB of preloaded memory within a minute or two (~420 MB -> ~890 MB for the container), which gc.freeze() in the pre_fork hook prevents (~405 MB). As recommended by the gc.freeze() documentation, collections are also disabled while the app is preloaded, so that they don't leave holes in the shared pages that the workers would fill. They are enabled again after freezing, in the master too, as it keeps running. Now that the web processes don't import the ML and AI libraries anymore, there is little left to unshare and the freeze makes no measurable difference: this is insurance against heavy imports creeping back into them.
marco-c
force-pushed
the
web-gc-freeze
branch
from
November 19, 2026 20:50
b9ed792 to
77f11bf
Compare
marco-c
marked this pull request as ready for review
November 19, 2026 21:43
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With
--preload, the workers share the master's memory until they write to it, and a full garbage collection writes to every object it traverses. When the preloaded heap is large, this unshares it in every process. Running the v0.0.655 production image locally with the production traffic mix, each of the four gunicorn processes got its own copy of ~170 MB of preloaded memory within a minute or two (~420 MB → ~890 MB). That matches the step-up of the production pods in #7586.gc.freeze()in gunicorn'spre_forkhook keeps collections from traversing the objects allocated before forking. With the same v0.0.655 image and traffic, memory stays at ~405 MB.As recommended by the
gc.freeze()documentation, collections are also disabled while the app is preloaded, so they don't leave holes in the shared pages for the workers to fill. They're re-enabled right after freezing, in the master too, as it keeps running with background threads.#7583 and #7585 remove most of that heap from the web processes. With them, preload alone stays flat (~122 MB under the same traffic) and the freeze makes no measurable difference. This is insurance against the problem returning if heavy imports creep back into the web. Dropping
--preloadinstead would double memory (~246 MB, as each worker keeps a full private copy) and make worker restarts re-import the app instead of forking.