[SPARK-58021][CONNECT] Reap orphaned local pool members

[ericm-db]

What changes were proposed in this pull request?

This is layer 5 of the nine-PR local Connect pool stack:

#57684 -> #57685 -> #57907 -> #57686 -> #58247 -> #57687 -> #58248 -> #57102 -> #57688

Until #57686 merges, GitHub shows its cumulative diff. The review unit introduced here is commit
68833332d27.

This layer adds janitor and orphan recovery on top of normal retirement:

• validated pending-launch records;
• cleanup of dead, timed-out, malformed, and partially published launches;
• retirement of dead, unreachable, idle, and orphaned claimed servers;
• attendant PID-reuse checks and complete launch-process-group cleanup;
• recovery of server PIDs from malformed records and daemon PID files; and
• stale conf-file and unreferenced member-directory garbage collection.

Acquisition, forceful purge, SparkSession integration, and warmup remain in later PRs.

Why are the changes needed?

Pool members outlive individual Python call frames and can be orphaned by crashes or SIGKILL.
Separating recovery from normal release makes the adversarial state matrix and its idempotency
contract reviewable without acquisition and user-facing orchestration.

Does this PR introduce any user-facing change?

No. The pool is not wired into SparkSession in this layer.

How was this patch tested?

Added thirteen focused tests at this layer, bringing the suite to 46 tests. They cover pending
launch failures, PID reuse, launch process groups, stale conf state, unusable and idle members,
malformed records, dead clients, and member-directory garbage collection.

python/run-tests --testnames pyspark.sql.tests.connect.test_connect_local_server_pool

At the stack tip, the equivalent direct unittest invocation passed all 56 pool tests, including
the two real-server E2E tests.

The rebuilt commit passed Python AST parsing, git diff --check, and changed-file ASCII and
100-column checks.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Fable 5) and OpenAI Codex (GPT-5)