fix: defer boot_agent until after eventlet monkey_patch to prevent recursion error - #903
CagriYonca wants to merge 1 commit into
Conversation
pvital
left a comment
There was a problem hiding this comment.
Great work @CagriYonca.
I have a few considerations.
c0dbd67 to
38966d8
Compare
f9bc0fd to
eea8860
Compare
|
Update after comments: Prevent double boot_agent() execution: Handled module-level unconditional call with a return_early flag and added an _eventlet_booted guard per process. Fix Gunicorn 26+ crash on startup: Replaced eager wrap_function_wrapper("gunicorn.workers.geventlet", ...) with wrapt.when_imported(...) to prevent ModuleNotFoundError when the removed eventlet worker is not present. Remove fragile /proc inspection: Replaced /proc/{ppid}/cmdline checking (which failed under Docker PID 1, console scripts, and python -m gunicorn) by hooking directly into EventletWorker.patch(), running exclusively post-fork in workers. Make deferred boot opt-in: Restricted deferred initialization behind INSTANA_EVENTLET_DEFERRED_BOOT=true so non-eventlet environments (Celery, gthread, standalone scripts) don't silently lose tracing. Fix SQLAlchemy test collection: Moved module-level create_engine(...) into a fixture and explicitly pinned the dialect to postgresql+psycopg2:// to avoid Pytest collection failures on Python 3.11+ / SQLAlchemy 2.1+. Replace find_spec("gunicorn") with runtime sys.modules check: Both hooks are now always registered. The eventlet.monkey_patch wrapper skips boot when gunicorn.workers.geventlet is in sys.modules, preventing arbiter-side boot under --preload while covering gunicorn ≥26 and non-gunicorn scenarios where the old branch left no boot path. |
…cursion error Signed-off-by: Cagri Yonca <cagri@ibm.com>
eea8860 to
75b7d4f
Compare
Problem & Root Cause
When running under Gunicorn with the
eventletworker class, Instana's early auto-instrumentation eagerly imports third-party libraries in the master process beforeeventlet.monkey_patchexecutes.Modules such as
urllib3andpymongocapture module-level references to the unpatchedssl.SSLContext. When Eventlet subsequently patches thesslmodule,ssl.SSLContextis replaced with its green subclass. Initializing SSL connections later causessuperlookups inside property setters to resolve the global green subclass instead of the original class, resulting in an infinite recursion and aRecursionError.Solution
This PR defers
boot_agentexecution wheneventletis present in the environment:eventlet.monkey_patchusingwrapt.boot_agentand all eager instrumentation imports until aftereventlet.monkey_patchcompletes in the forked worker process.Changes
src/instana/__init__.py: Added_defer_boot_until_eventlet_patchto defer agent boot until worker post-patch.tests/frameworks/test_eventlet_autotrace.py: Added unit tests covering deferred boot registration, worker execution, and arbiter suppression.