Harden upload and logging boundaries and make extraction workers recoverable

.github/workflows/ci.yml

@@ -16,7 +16,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        check: [ruff, type, tests, bandit]
+        check: [ruff, type, tests, bandit, dependencies]
     name: "${{ matrix.check }}"
     steps:
       # Common setup
@@ -39,7 +39,7 @@ jobs:
         working-directory: efile_app
         run: |
           uv python install 3.12
-          uv sync --group dev
+          uv sync --locked --group dev
 
       # No per-check installs needed; everything is managed by uv
 
@@ -86,6 +86,11 @@ jobs:
         working-directory: efile_app
         run: uv run bandit -r efile --exclude efile/tests
 
+      - name: Audit resolved dependencies
+        if: matrix.check == 'dependencies'
+        working-directory: efile_app
+        run: uv run pip-audit --local --skip-editable
+
   javascript:
     name: javascript
     runs-on: ubuntu-latest

docs/developer-notes/security-hardening.md

@@ -0,0 +1,34 @@
+# Security and extraction hardening
+
+This work keeps YAML authoring and the existing per-process configuration cache. It addresses the highest-priority findings from the review of `f3b1d17`.
+
+## Changes
+
+- Remove the public legacy S3 upload, mock upload, bucket diagnostic, and session-debug routes in every environment. Normal uploads continue through the authenticated draft workflow.
+- Remove submission headers, payloads, response bodies, and filing-history session tokens from application logs. Default application logging to INFO, including production.
+- Upgrade Django to 5.2.17 and update the resolved packages flagged by the dependency audit. CI installs the locked dependencies and runs `pip-audit`, including the PDF-processing dependencies. The editable application and the two unpublished GitHub court-directory packages are outside this audit's coverage.
+- Restrict jurisdiction lookup to the canonical names of installed, non-symlink YAML files. Unknown names and path aliases fail before file access or cache insertion. Restart processes when adding or removing jurisdiction files. Generic public pages use no jurisdiction configuration until a state is selected.
+- Give each extraction attempt a unique claim token and renewable lease. Requeued or reclaimed work invalidates earlier results, failures, and heartbeats. Recover expired final attempts to a terminal failure and apply bounded exponential retry backoff.
+- Run each extraction in a separate Linux process with a wall-clock alarm and address-space limit. The supervisor renews the lease every 30 seconds while the child runs, terminates obsolete or timed-out children, and records interrupted attempts without persisting exception contents.
+- Recheck the AI preference and claim before analysis and before the evidence and classification stages. Results from an obsolete attempt cannot update the draft. A preference change cannot retract data already sent or an upstream request already in flight.
+- Disable persistent database connections in ASGI deployment settings, configure WhiteNoise through `STORAGES`, and remove the bucket-listing request from ordinary S3 client initialization. S3 calls now have explicit connection/read timeouts and bounded retries.
+
+## Deployment
+
+Stop old extraction workers before applying migration `0028_extraction_claim_leases` and starting the new web and worker image. Old worker code does not honor claim tokens; mixing worker versions defeats the protection. Existing pending jobs remain eligible. Interrupted processing jobs without a lease become recoverable after the existing 15-minute stale interval.
+
+`DOCUMENT_EXTRACTION_TIMEOUT_SECONDS` defaults to 600. `DOCUMENT_EXTRACTION_MEMORY_MB` defaults to 768 and limits virtual address space, not just resident memory. Numerical libraries use one thread in each child. Leave memory for the supervisor and operating system when setting container limits. Validate these bounds with representative documents before increasing replica counts; they are safety bounds, not measured capacity recommendations. The deployed worker must support Linux `resource` limits and `SIGALRM`.
+
+Each worker supervisor processes one job at a time. Size workers using measured queue wait, processing duration, peak memory, database connections, and upstream quotas. No deployment replica count or connection pool has been changed here.
+
+Rebuild the image so `uv sync --frozen` installs the new lockfile and `collectstatic` produces hashed and compressed assets. Verify production response headers after deployment. Review retained logs and rotate or revoke credentials confirmed to have been recorded; source changes cannot remove historical disclosures.
+
+## Follow-up work
+
+- Aggregate upload limits and storage quotas, actual file-content validation, durable upload staging, and orphan cleanup.
+- Restore CSRF checks on browser session mutation and submission endpoints, with matching JavaScript changes and CSRF-enforcing tests.
+- Stop unnecessary session writes on polling; test expiry behavior before changing the global session setting.
+- Measure web workflows and extraction recovery under realistic load, including PostgreSQL contention, upstream rate limits, and difficult PDFs. Unit interleaving tests do not establish production throughput.
+- Validate and version compiled YAML configurations; add derived-response caching only where profiling shows a benefit.
+
+The submission claim and ambiguous-outcome safeguards are unchanged. Extraction retries must not be reused as an automatic court-submission retry mechanism.

efile_app/efile/api/case_type_config.py

@@ -1,6 +1,6 @@
 from django.http import JsonResponse
 
-from ..utils.config_loader import JurisdictionConfigLoader, config_loader
+from ..utils.config_loader import InvalidJurisdiction, JurisdictionConfigLoader, config_loader
 
 
 def get_case_type_config(request):
@@ -11,7 +11,9 @@ def get_case_type_config(request):
         jurisdiction = request.GET.get("jurisdiction") or request.session.get("jurisdiction")
 
         # Use the new jurisdiction-aware configuration loader
-        config_data = config_loader.load_jurisdiction_config(jurisdiction)
+        config_data = (
+            config_loader.base_config if jurisdiction is None else config_loader.load_jurisdiction_config(jurisdiction)
+        )
 
         # Process case types to ensure proper inheritance from base_case_types
         processed_case_types = {}
@@ -49,5 +51,7 @@ def get_case_type_config(request):
 
         return JsonResponse({"success": True, "config": response_data})
 
+    except InvalidJurisdiction as e:
+        return JsonResponse({"success": False, "error": str(e)}, status=400)
     except Exception as e:
         return JsonResponse({"success": False, "error": f"Configuration loading error: {str(e)}"}, status=500)

efile_app/efile/api/filing_views.py

@@ -31,7 +31,6 @@ def get_tyler_token(request, jurisdiction=None):
 
     # Fallback to session
     auth_tokens = request.session.get("auth_tokens", {})
-    logger.debug(f"Auth tokens in session: {auth_tokens}")
 
     # Try different Tyler token key formats
     tyler_token = (
@@ -228,8 +227,7 @@ def payment_fees(request):
                     }
                 )
             else:
-                # Debug, not info: fee responses echo party names and case details.
-                logger.debug("EFSP fee response body: %s", response.text[:2000])
+                logger.warning("EFSP fee request failed status=%s", response.status_code)
                 error_message = describe_efsp_error(response)
 
                 return JsonResponse(

efile_app/efile/api/suffolk_api_views.py

@@ -332,7 +332,6 @@ def get_party_types_from_suffolk_api(request):
                 return JsonResponse({"success": False, "error": "No party types returned from Suffolk API"}, status=400)
         else:
             logger.warning(f"Suffolk API request failed with status: {response.status_code}")
-            logger.warning(f"Response: {response.text}")
             return JsonResponse(
                 {"success": False, "error": f"Suffolk API returned status {response.status_code}"},
                 status=response.status_code,

efile_app/efile/api/urls.py

@@ -28,20 +28,12 @@
 )
 from .filing_views import get_filings, payment_fees
 from .payment_views import delete_payment_account, waiver_account
-from .s3_upload import (
-    mock_s3_upload,
-    simple_s3_upload,
-    test_s3_connection,
-)
 from .suffolk_api_views import get_party_types_from_suffolk_api, lookup_case
 
 app_name = "api"
 
 urlpatterns = [
     path("get-party-types/", get_party_types_from_suffolk_api, name="get_party_types"),
-    path("simple-s3-upload/", simple_s3_upload, name="simple_s3_upload"),
-    path("mock-s3-upload/", mock_s3_upload, name="mock_s3_upload"),
-    path("test-s3-connection/", test_s3_connection, name="test_s3_connection"),
     # Dropdown API endpoints
     path("dropdowns/case-categories/", get_case_categories, name="case_categories"),
     path("dropdowns/case-types/", get_case_types, name="case_types"),

efile_app/efile/context_processors.py

@@ -2,7 +2,7 @@
 Context processors for jurisdiction-aware templates
 """
 
-from .utils.config_loader import config_loader
+from .utils.config_loader import InvalidJurisdiction, config_loader
 
 
 def jurisdiction_context(request):
@@ -19,7 +19,14 @@ def jurisdiction_context(request):
         if len(segments) >= 3 and segments[1] == "jurisdiction":
             current_jurisdiction = segments[2].lower()
 
-    config = config_loader.load_jurisdiction_config(current_jurisdiction)
+    # Generic pages have no selected state. Do not pass absence through the
+    # strict request-to-configuration boundary as a jurisdiction identifier.
+    try:
+        config = config_loader.load_jurisdiction_config(current_jurisdiction) if current_jurisdiction else {}
+    except InvalidJurisdiction:
+        # Error templates also run context processors. Invalid request input
+        # must not prevent Django from rendering the original error response.
+        config = {}
 
     return {
         "jurisdiction": current_jurisdiction,

efile_app/efile/management/commands/process_document_extractions.py

@@ -1,17 +1,42 @@
 import logging
+import multiprocessing
 import time
 
+from django.conf import settings
 from django.core.management.base import BaseCommand
-
-from efile.services.document_extractions import (
-    claim_next_extraction,
-    process_document_extraction,
-    record_extraction_failure,
-)
+from django.db import close_old_connections
 
 logger = logging.getLogger(__name__)
 
 
+def _run_claim(job_id, claim_token, memory_mb, timeout_seconds):
+    """Isolate PDF parsing and model clients from the queue supervisor."""
+    import os
+    import resource
+    import signal
+
+    import django
+
+    # One job per child: numerical libraries must not reserve a thread pool
+    # proportional to the host CPU count before PDF processing even starts.
+    os.environ["OPENBLAS_NUM_THREADS"] = "1"
+    os.environ["OMP_NUM_THREADS"] = "1"
+    # The deployed worker is Linux. Enforce bounds before loading the PDF;
+    # the alarm also bounds an orphaned child if its supervisor is killed.
+    memory_bytes = memory_mb * 1024 * 1024
+    resource.setrlimit(resource.RLIMIT_AS, (memory_bytes, memory_bytes))
+    signal.alarm(timeout_seconds)
+    django.setup()
+
+    from efile.services.document_extractions import process_document_extraction, record_extraction_failure
+
+    try:
+        process_document_extraction(job_id, claim_token)
+    except Exception as error:
+        logger.error("Document extraction job %s failed (%s)", job_id, type(error).__name__)
+        record_extraction_failure(job_id, claim_token, error)
+
+
 class Command(BaseCommand):
     help = "Process queued lead-document extraction jobs"
 
@@ -20,19 +45,48 @@ def add_arguments(self, parser):
         parser.add_argument("--poll-interval", type=float, default=2.0)
 
     def handle(self, *args, **options):
+        from efile.services.document_extractions import (
+            claim_next_extraction,
+            record_extraction_failure,
+            renew_extraction_lease,
+        )
+
         while True:
+            close_old_connections()
             job = claim_next_extraction()
             if job is None:
                 if options["once"]:
                     return
                 time.sleep(max(0.1, options["poll_interval"]))
                 continue
 
+            timeout = max(1, settings.DOCUMENT_EXTRACTION_TIMEOUT_SECONDS)
+            child = multiprocessing.get_context("spawn").Process(
+                target=_run_claim,
+                args=(job.pk, job.claim_token, settings.DOCUMENT_EXTRACTION_MEMORY_MB, timeout),
+            )
+            deadline = time.monotonic() + timeout
             try:
-                process_document_extraction(job.pk)
-            except Exception as error:
-                logger.exception("Document extraction job %s failed", job.pk)
-                record_extraction_failure(job.pk, error)
+                child.start()
+                while child.is_alive():
+                    child.join(timeout=min(30, max(0, deadline - time.monotonic())))
+                    if not child.is_alive():
+                        break
+                    close_old_connections()
+                    if time.monotonic() >= deadline or not renew_extraction_lease(job.pk, job.claim_token):
+                        child.terminate()
+                        break
+            finally:
+                if child.pid is not None:
+                    child.join(timeout=5)
+                    if child.is_alive():
+                        child.kill()
+                        child.join(timeout=5)
+                    child.close()
+                # No-op for completed, failed, or superseded claims. Handles
+                # OOM, a signal, and other exits without a Python exception.
+                close_old_connections()
+                record_extraction_failure(job.pk, job.claim_token, "Worker exited")
 
             if options["once"]:
                 return

efile_app/efile/migrations/0028_extraction_claim_leases.py

@@ -0,0 +1,29 @@
+# Generated by Django 5.2.17 on 2026-09-30 15:16
+
+import django.utils.timezone
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('efile', '0027_filingdraft_disclaimer_acceptance'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='documentextraction',
+            name='available_at',
+            field=models.DateTimeField(default=django.utils.timezone.now),
+        ),
+        migrations.AddField(
+            model_name='documentextraction',
+            name='claim_token',
+            field=models.UUIDField(blank=True, editable=False, null=True),
+        ),
+        migrations.AddField(
+            model_name='documentextraction',
+            name='lease_expires_at',
+            field=models.DateTimeField(blank=True, null=True),
+        ),
+    ]

efile_app/efile/models.py

@@ -433,6 +433,9 @@ class Status(models.TextChoices):
     )
     status = models.CharField(max_length=20, choices=Status.choices, default=Status.PENDING, db_index=True)
     attempts = models.PositiveSmallIntegerField(default=0)
+    claim_token = models.UUIDField(null=True, blank=True, editable=False)
+    lease_expires_at = models.DateTimeField(null=True, blank=True)
+    available_at = models.DateTimeField(default=timezone.now)
     total_pages = models.PositiveIntegerField(blank=True, null=True)
     pages_analyzed = models.PositiveIntegerField(blank=True, null=True)
     # Structured direct evidence is separate from the flattened review copy so