[SPARK-58021][CONNECT] Add forceful local pool purge

[ericm-db]

What changes were proposed in this pull request?

This is layer 7 of the nine-PR local Connect pool stack:

#57684 -> #57685 -> #57907 -> #57686 -> #58247 -> #57687 -> #58248 -> #57102 -> #57688

Until lower layers merge, GitHub shows their cumulative diff. The review unit introduced here is
commit 196221a8d5a.

This layer adds the forceful escape hatch for returning the pool to a clean slate:

• scans every state kind directly, including malformed and duplicate claimed records;
• verifies attendants and servers before signalling reused PIDs;
• SIGKILLs recoverable attendants, launch groups, and server processes;
• removes all state files and member directories; and
• exposes python -m pyspark.sql.connect.local_server_pool --purge.

SparkSession integration and JIT warmup remain in later PRs.

Why are the changes needed?

Normal retirement deliberately preserves retryable state. Operators also need a bounded,
destructive recovery path when the pool itself is corrupt or wedged. Keeping purge separate makes
its stronger signalling and deletion semantics explicit and independently reviewable.

Does this PR introduce any user-facing change?

Yes. It adds python -m pyspark.sql.connect.local_server_pool --purge, which force-stops every
local pool process it can verify and empties the pool directory. SparkSession still does not select
the pool until #57102.

How was this patch tested?

Added one focused corruption-and-process-lifecycle test at this layer, bringing the suite to 51
tests. It covers ready, pending, half-started, duplicate-claimed, retiring, and malformed members.

python/run-tests --testnames pyspark.sql.tests.connect.test_connect_local_server_pool

At the stack tip, the equivalent direct unittest invocation passed all 56 pool tests, including
the two real-server E2E tests.

The rebuilt commit passed Python AST parsing, git diff --check, and changed-file ASCII and
100-column checks.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Fable 5) and OpenAI Codex (GPT-5)