Fix upload contention, copy filing codes from the EFSP database, and add guided code search

.github/workflows/accessibility.yml

@@ -59,17 +59,18 @@ jobs:
           curl --retry 30 --retry-connrefused --retry-delay 1 --fail http://127.0.0.1:8000/choose-jurisdiction
           npm run test:a11y
 
-      - name: Run Confirm case editing checks
+      - name: Run confirm case and filing search accessibility checks
         working-directory: efile_app
         env:
           A11Y_STORAGE_STATE: playwright/.auth/a11y.json
         # Same server and seeded session as the Axe checks; the court lists
-        # are mocked in the browser.
-        run: npm run test:confirm-case
+        # are mocked in the browser. Includes desktop/mobile Axe audits and
+        # keyboard checks for contextual filters, claim amounts, and path details.
+        run: npm run test:confirm-case -- --output=test-results/confirm-case
 
       - name: Run party role keyboard checks
         working-directory: efile_app
-        run: npx playwright test --config=playwright.parties.config.js
+        run: npx playwright test --config=playwright.parties.config.js --output=test-results/parties
 
       - name: Upload accessibility results
         if: always()

README.md

@@ -8,8 +8,29 @@ A minimal Django app for form submission and review. The Django project lives un
   - Python 3.10+
   - uv
 
-
-### Using uv (recommended)
+### Single-command local startup (`run_all.sh`)
+
+To quickly run the entire local development stack without building the full Docker container:
+
+```bash
+./run_all.sh
+```
+
+This script:
+
+- Verifies Astral `uv` and synchronizes the local virtual environment (`efile_app/.venv`)
+- Automatically initializes `efile_app/.env` from `.env.example` if needed
+- Starts LocalStack (S3 mock) in Docker if Docker is running
+- Applies Django database migrations
+- Runs the Django development server (`http://127.0.0.1:8000`) and background extraction worker together
+- Starts the filing code index worker, which copies changed courts' codes from the EFSP codes database once a day and indexes them
+- Cleanly stops all child processes upon pressing `Ctrl+C`
+
+Filing code search becomes available after the first successful sync, which needs `EFSP_CODES_DATABASE_URL` in `efile_app/.env` (the test EFSP's codes database, read only). `--legacy-code-crawl` explicitly enables the older full crawler instead.
+
+Run `./run_all.sh --help` to see additional options (e.g., custom ports, running without Docker, or `--no-code-index` to skip the court code index worker).
+
+### Using uv manually
 
 - __0) Install uv__ (one-time)
   - macOS (Homebrew):

compose.yml

@@ -87,6 +87,17 @@ services:
     volumes:
       *litefile-volumes
 
+  code_index_worker:
+    image: litefile:web
+    command: uv run python manage.py refresh_filing_code_index --daily --retry-interval 900 --verbosity 2 --cache-dir /data/filing-code-cache
+    depends_on:
+      web:
+        condition: service_healthy
+    environment:
+      <<: *litefile-environment
+    volumes:
+      *litefile-volumes
+
 volumes:
   localstack:
     driver: local

docs/docs/admin/configuration.md

@@ -20,6 +20,10 @@ LITEFile follows [Twelve-Factor App](https://12factor.net/) principles, configur
 | `DATABASE_URL` | Yes (Prod) | `sqlite:///db.sqlite3` | Database connection string (e.g., `postgres://user:pass@host:5432/dbname`). |
 | `DJANGO_ALLOWED_HOSTS` | Yes (Prod) | `localhost,127.0.0.1` | Comma-separated list of allowed hostnames/domains. |
 | `EFSP_URL` | No | `https://efile-test.suffolklitlab.org` | Base URL for the EFSP REST API endpoint. |
+| `FILING_CODE_SYNC_MODE` | No | `database` | Where filing codes come from. `database` copies them from the EFSP codes database (needs `EFSP_CODES_DATABASE_URL`); `bulk` uses the proxy's HTTP catalog export; `legacy` the older full crawler. |
+| `EFSP_CODES_DATABASE_URL` | For `database` sync | — | PostgreSQL URL of the EFSP proxy's codes database, for example `postgresql://user:password@host:5432/postgres?sslmode=require`. Read only; prefer a read-only role. Use the test EFSP's database outside production. |
+| `FILING_CODE_SYNC_TIME` | No | `04:30` | Daily time (`HH:MM`) the code index worker copies and indexes codes. Set it after the EFSP's own Tyler update (production 02:13, test 19:35 Eastern). |
+| `FILING_CODE_SYNC_TIMEZONE` | No | `America/New_York` | Time zone for `FILING_CODE_SYNC_TIME`. |
 | `SUFFOLK_EFILE_API_KEY` | No | `""` | API authentication key for the EFSP proxy service. |
 | `OPENAI_API_KEY` | Optional | `None` | API key for OpenAI or compatible LLM provider used for document extraction. |
 | `OPENAI_BASE_URL` | Optional | `https://api.openai.com/v1/` | Base URL for OpenAI-compatible endpoint (or local LLM gateway). |

docs/docs/admin/deployment.md

@@ -61,14 +61,17 @@ primary_region = 'lax'
 
 [env]
   DJANGO_SETTINGS_MODULE = "efile.settings_staging"
+  FILING_CODE_SYNC_MODE = "legacy"
 
 [deploy]
   # Run database migrations before each release
   release_command = "uv run python manage.py migrate --noinput --fake-initial"
+  release_command_timeout = "30m"
 
 [processes]
   app = "uv run gunicorn efile.asgi:application -k uvicorn.workers.UvicornWorker --bind 0.0.0.0:8000 --workers 2 --timeout 60"
   extraction_worker = "uv run python manage.py process_document_extractions"
+  code_index_worker = "uv run python manage.py refresh_filing_code_index --interval 86400"
 
 [http_service]
   internal_port = 8000
@@ -90,6 +93,107 @@ Document analysis runs outside the web request in the `extraction_worker` proces
 
 By default, LITEFile sends only the first 20 PDF pages for analysis. Set `DOCUMENT_EXTRACTION_MAX_PAGES` to a positive integer to change that cap. DOCX text is limited to the first 100,000 characters with `DOCUMENT_EXTRACTION_MAX_TEXT_CHARS`; Word files have no reliable page boundaries. Review identifies when either limit omitted part of a document. `DOCUMENT_EXTRACTION_MAX_ATTEMPTS` controls how many times a failed job is tried before the filer is sent to manual review.
 
+### Filing code search index
+
+Filing codes come straight from the EFSP proxy's codes database. The proxy
+updates its codes from Tyler once a day (production at 02:13, test at 19:35
+Eastern). Each day at `FILING_CODE_SYNC_TIME` in `FILING_CODE_SYNC_TIMEZONE`, the
+`code_index_worker` copies every court whose codes changed into LITEFile's own
+database, then rebuilds the search index for those courts. Set the time well
+after the proxy's update: Fly staging, which uses the test EFSP, runs at 21:30
+Eastern. Production should run at about 04:30. Keep one Machine running in the
+`code_index_worker` process group.
+
+LITEFile connects with `EFSP_CODES_DATABASE_URL` and only ever reads. The court
+list and every court's tables are read in one `READ ONLY`, `REPEATABLE READ`
+transaction, so the copy is a consistent snapshot and the database refuses any
+write. Supabase's connection pooler ignores connection-level settings, which is
+why read-only is set per transaction. The queries match the proxy's own
+filing-catalog export: courts with all three installed code lists, non-criminal
+case categories, and filings that aren't court-use only.
+
+LITEFile connects as its own role, `litefile_codes_reader`, never with the
+proxy's credentials. The proxy keeps user data in the same database, so the role
+can read only the five code tables. It is also read only at the server, which
+holds even through Supabase's pooler. The test EFSP's database already has it.
+For another environment's database, run this once as its `postgres` role,
+with a new random password:
+
+```sql
+CREATE ROLE litefile_codes_reader LOGIN PASSWORD '<random>'
+  NOSUPERUSER NOCREATEDB NOCREATEROLE NOINHERIT NOREPLICATION NOBYPASSRLS
+  CONNECTION LIMIT 5;
+ALTER ROLE litefile_codes_reader SET default_transaction_read_only = on;
+ALTER ROLE litefile_codes_reader SET statement_timeout = '10min';
+GRANT CONNECT ON DATABASE postgres TO litefile_codes_reader;
+GRANT USAGE ON SCHEMA public TO litefile_codes_reader;
+GRANT SELECT ON public.location, public.installedversion, public.casecategory,
+  public.casetype, public.filing TO litefile_codes_reader;
+```
+
+Through Supabase's pooler, the user name is `litefile_codes_reader.<project ref>`:
+`postgresql://litefile_codes_reader.<project ref>:<password>@<pooler host>:5432/postgres?sslmode=require`.
+The proxy's nightly update reloads these tables' rows without dropping them, so
+the grants persist. If a proxy schema migration ever recreates one of them,
+grant `SELECT` on it again. Postgres lets every role create session-private
+temporary tables (a `PUBLIC` default that can't be revoked from one role). Those
+can't touch shared data, and the role's read-only default refuses them unless a
+client turns read-only off, which LITEFile never does.
+
+A court whose revision is unchanged is not read again. A court that suddenly
+has no filing types usually means the proxy's update is half finished, so the
+copy stops and keeps the old one. Failed states are retried every 15 minutes
+(`--retry-interval 900`), then the schedule returns to daily. On startup, the
+worker syncs immediately if any jurisdiction has no current index. That is the
+case on a fresh deployment, or after a search-rules change.
+
+#### Resync and rebuild from the staff tools
+
+Superusers see a **Filing codes** page in the staff tools (at the private `LITEFILE_STAFF_PATH`). It shows
+each jurisdiction's copy and index, recent runs, and two buttons, for one
+jurisdiction or all of them:
+
+- **Resync codes** copies changed courts from the EFSP now, then updates their
+  part of the index.
+- **Rebuild code search indexes** rebuilds the whole index from the copy already
+  in LITEFile, without contacting the EFSP. Use it after changing search rules
+  in `filing_code_search.yaml`.
+
+The buttons queue a job, which the code index worker starts within a minute.
+Each request is recorded in the staff audit log.
+
+#### From the command line
+
+Run these from `efile_app/` in the target environment:
+
+```bash
+uv run python manage.py migrate --noinput
+uv run python manage.py refresh_filing_code_index                          # resync every state now
+uv run python manage.py refresh_filing_code_index --jurisdiction vermont   # one state
+uv run python manage.py refresh_filing_code_index --rebuild                # rebuild from the local copy
+uv run python manage.py refresh_filing_code_index --dry-run --jurisdiction vermont  # report changes only
+uv run python manage.py refresh_filing_code_index --daily                  # what the worker runs
+```
+
+On the test EFSP, Vermont (22 courts, about 1.1 million search paths) copies and
+indexes in about two minutes. A run with nothing changed takes seconds.
+
+`FILING_CODE_SYNC_MODE=bulk` (the proxy's HTTP `filing_catalog` export) and
+`legacy` (the full list-API crawler, `--legacy-crawl`) remain available as
+fallbacks. Neither needs database access. `--cache-dir` applies only to the
+legacy crawler.
+
+Local Docker Compose and `./run_all.sh` start the same daily worker. They read
+`EFSP_CODES_DATABASE_URL` and `FILING_CODE_SYNC_TIME` from `efile_app/.env`.
+
+Before the first successful sync, the modal explains that search is unavailable
+and the usual court lists remain available. The dialog does not trigger indexing,
+but retries an unavailable search automatically while it stays open.
+Results older than two days carry an out-of-date notice. Every selected path is
+checked against the live court lists before it updates the form. A changed
+`EFSP_URL` or thesaurus revision requires a fresh index, so test-server codes and
+incompatible search terms cannot leak into another environment.
+
 ### Setting Fly.io production secrets:
 ```bash
 fly secrets set \
@@ -101,6 +205,7 @@ fly secrets set \
   AWS_S3_REGION_NAME="us-east-1" \
   OPENAI_API_KEY="sk-..." \
   GOTENBERG_URL="https://..." \
+  EFSP_CODES_DATABASE_URL="postgresql://<read-only role>:...@<host>:5432/postgres?sslmode=require" \
   GOTENBERG_USERNAME="..." \
   GOTENBERG_PASSWORD="..."
 ```