perf: Postgres storage tuning for app_attest hot tables

[FieldMarshallObvious]

Description

This PR reduces bloat on the three high-churn tables in the app_attest database by tuning autovacuum, hot updates, and fillfactor on those tables. Utilizing a hand-written Alembic migration. The changes are storage-param-only ALTER TABLE ... SET (...) statements. Updating catalog settings, run fast, and do not rewrite the table. Please note the migration can be reverted to the previous state without impacting existing behavior.

Tables:

• challenges: lower autovacuum_vacuum_scale_factor 0.2 -> 0.05 and raise autovacuum_vacuum_cost_limit to 2000, so autovacuum reclaims the dead rows from the per-request insert+delete churn sooner and faster.
• public_keys: set fillfactor=85 and scale_factor=0.05 so the per-request counter/updated_at updates stay hot, avoiding primary-key index writes and index bloat.
• mlpa_user_capacity: set fillfactor=70 and raise the autovacuum threshold to 1000 (scale_factor=0) so the single hot row's updates stay hot on one page and we stop over-vacuuming a one-row table

QA Path

Validated on a local database. First confirmed no change in existing behavior: the app_attest unit/integration tests pass, and the attest→assert flow is unchanged.

The migration itself was verified up and down: reloptions on the three tables are NULL before, show the tuned values after upgrade, and return to NULL after downgrade.

Behavior of each table was then confirmed using the pageinspect and pgstattuple extensions via local scripts, toggling the migration (downgrade 919c4d382c42 / upgrade head)between a "before" and "after" run.

Challenges

Seeds 900 rows, then updates 150 rows to create 150 dead tuples. Effective trigger = 50 + scale_factor × reltuples:

	trigger (reltuples=900)	150 dead →
BEFORE (0.2)	230	autovacuum did not fire
AFTER (0.05)	95	autovacuum fired

Additionally simulated performance by inserting 3000 rows in challenges, then each round insert 30 new challenges and delete 30 olds ones. Resulting in 450 dead tuples per minute. Before tuning dead rows climb to ~870 and autovacuum fires every ~2 cycles. After the tuning the number of rows climb to ~450 and autovacuum rows about every cycle.

Public Keys

150 keys with 5000 counter-bump updates. Hot-update ratio measured from pg_stat_user_tables, and hot chains confirmed via pageinspect:

	fillfactor	HOT ratio
BEFORE	100	~3.8%
AFTER	85	~21.6%

MLPA User Capacity

500 updates against the single id=1 row; ~99% HOT, confined to one page.

	autovacuum threshold	500 dead →
BEFORE	~50	autovacuum fired
AFTER	1000	autovacuum did not fire

Related Tickets & Documents

AIPLAT-1141

[FieldMarshallObvious]

/build-pr

[randy-concepcion]

Nice work, this looks great!

Nothing blocking here, but the only thing I'm curious about was the hot ratio for public keys being ~21.6% which is a pretty wide gap from the ticket's "~100%". Any thoughts on what accounts for that? Or is that something we'd expect to improve as the existing rows get updated over time?

Still a solid improvement from 3.8%! 🙌

[FieldMarshallObvious]

Nice work, this looks great!

Nothing blocking here, but the only thing I'm curious about was the hot ratio for public keys being ~21.6% which is a pretty wide gap from the ticket's "~100%". Any thoughts on what accounts for that? Or is that something we'd expect to improve as the existing rows get updated over time?

Still a solid improvement from 3.8%! 🙌

We would expect the HOT ratio for public keys to reach ~100% in production, the reason it's so low is because of my test. There is 150 rows which have 5000 updates, which exhausts the headroom alot faster than the HOT-pruning reclaims it.