Narrowing a question by industry made it fail: rank how narrow a side is, instead of asking whether it is - #58
Merged
Conversation
Adding an industry filter to block A turned a working question into a failing
one. "What facilities are within 5 km upstream from PFOS samples in York
County" answers in 3s with block A empty; add NAICS 221320 to it and both
projections time out in query planning at 30s.
`sideIsConstrained` was a boolean: region, filter or IRI pin, any of them means
constrained. An industry filter set it to true, the reorder stopped firing, and
the query went back to seeding from block A, which now meant every sewage
treatment facility in the country rather than the 118 sample points in one
county. The comment two lines above it already said "narrowed is not small"; the
predicate did not act on its own warning.
It now returns a rank, and the body leads with the higher one:
3 an IRI pin. The executor chose this slice.
2 a region. Bounded by geography, and the graph is indexed for it.
1 an entity filter only. Selective in kind, unbounded in extent.
0 nothing.
Ties keep the anchor-first order, so this only changes cases where both sides
are constrained at different levels. Everything where one side was constrained
and the other was not keeps the plan it had, which the snapshot confirms: of 375
shapes, 2 move, both variants with a region on one side and a bare filter on the
other.
Measured on the live endpoint, all with block A carrying no region:
1 code, unbounded 429 planning timeout -> 35 facilities, 16s
1 code, 5 km 429 planning timeout -> 22 facilities, 8s
8 codes, 5 km 429 planning timeout -> 88 facilities, 3s
8 codes, unbounded 429 planning timeout -> 192 facilities, 15s
21 codes still fails. That is a different cost, the `|| EXISTS { ?ic
fio:subcodeOf ?sel }` clause evaluated per selected code and duplicated inside
the bounded aggregate, and it is not addressed here.
One shipped dashboard question changes plan. `samples-downstream-waste-indiana`
has samples scoped to Indiana and NAICS 5622 with no region, so its anchor is
nationwide and its target is one state. Both orders were run: identical answers,
136 samples and 415 facilities either way, reordered no slower at 13s and 2s
against 16s and 3s. `check-query-joins` asserted that no prebuilt may reorder,
which described the old predicate rather than a rule; it now expects this one to
and says why.
30 new assertions pin the ranking in both directions, and a mutation reverting
it to the boolean fails them.
QUERY-MATRIX.md Part 11: what the boolean predicate cost, the rank that replaced it, the measured before and after with block A unscoped, and the two things it does not fix. DEBUGGING.md gains a second 2026-09-20 entry, with the one-variable isolation table showing that block A had to be small rather than merely filtered, and the note that a long industry selection fails for a different reason. W38 entry 17. Written to be read next to entry 15, since this is the same class of bug one level up and was found by doing the obvious next thing with the map entry 15 fixed.
…r it
Yesterday's entries said industry selections beyond about ten codes fail, and
blamed the `FILTER(?ic = ?sel || EXISTS { ?ic fio:subcodeOf ?sel })` clause.
Both halves were wrong, and both were written from a measurement taken before
the narrowness ranking landed.
21 codes answers fine at 5 km: 101 facilities in 13s, 84 samples in 23s. It
fails only unbounded, at 430 MB. Code count is a cost on the closure, and a
distance bound already caps it, so a long selection is a reason to set a
distance rather than to shorten the list.
codes 5 km unbounded
1 22 facilities, 8s 35 facilities, 16s
8 88 facilities, 3s 192 facilities, 15s
21 101 facilities, 13s 500, out of memory at 430 MB
The clause is cleared as well, by trying the fix rather than reasoning about it.
`fio:subcodeOf` is materialised transitively (325110 is a direct subcode of 32,
325 and 3251), so `?ic fio:subcodeOf? ?sel` is an exact rewrite of the FILTER.
Measured on the 21-code bounded query: 13s with the FILTER, query-planning
timeout with the path. The FILTER is the fast form. Recorded as a rejected
design so nobody optimises it again.
|
🚅 Deployed to the explorer-app-pr-58 environment in sawgraph-explorer
|
railway-app
Bot
temporarily deployed
to
sawgraph-explorer / explorer-app-pr-58
September 20, 2026 06:40
Destroyed
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #55, found by doing the next obvious thing with the map #55 had just repaired.
#55 made "what facilities are within 5 km upstream from PFOS samples in York County" answer in 3s. It returns 418 facilities across 261 distinct industries, including 21 dental offices, 12 hospitals and 8 schools, because block A carries no industry filter and "facility" means anything with a NAICS code. The obvious next step is to filter block A to PFAS source categories.
That returned nothing: a 30s timeout in query planning, on both projections. Adding a filter to a working question made it fail.
The cause, one level above #55
sideIsConstrainedanswered yes for a region, an entity filter or an IRI pin alike. An industry filter therefore made the anchor count as constrained, the reorder from #55 stopped firing, and the query seeded from block A again. Only now block A meant every facility in that industry nationwide.The comment two lines above the predicate already read "narrowed is not small: a county of wells is narrowed and still enormous". The predicate did not act on its own warning.
Isolated by varying one thing at a time, all at 5 km, York County, detections only:
Block A had to be small, not merely filtered, and only a region made it small.
The fix
sideIsConstrainedbecomessideNarrowness, returning a rank, and the body leads with the higher:Ties keep the anchor-first order, so this only changes questions where both sides are constrained at different levels. Everything where one side was constrained and the other was not keeps the plan it had. That containment is measured, not argued: of the 375 shapes in the query snapshot, 2 move, both variants with a region on one side and a bare filter on the other.
Measured on the live endpoint, block A carrying no region throughout:
The filtered map is the useful one: 101 candidates instead of 418, dentists and schools gone, with sewage treatment, dry cleaners, boatyards, airports and petroleum terminals at the top.
One dashboard question changes plan
samples-downstream-waste-indianascopes samples to Indiana and filters block C to NAICS 5622 with no region, so its anchor is nationwide and its target is one state. Both orders were run before accepting the change: identical answers, 136 samples and 415 facilities either way, reordered no slower (13s and 2s against 16s and 3s).check-query-joinshad asserted that no prebuilt may reorder. That described the old boolean rather than a rule, so it now expects this one to, and records the measurement as the reason.Guardrails
30 new assertions in
check-query-joins(307 to 337) pin the ranking in both directions: a bare filter must not outrank a region, a region must outrank a bare filter, and for bounded queries the aggregate must seed from the side the body leads with. Mutation-tested: reverting the rank to the old boolean fails them.All seven checks pass, including the 375-shape snapshot from #55, which is what bounds the blast radius to 2 shapes.
A correction to #55's docs, included here
#55 shipped three documents saying industry selections beyond about ten codes fail, and blaming
FILTER(?ic = ?sel || EXISTS { ?ic fio:subcodeOf ?sel }). Both halves are wrong, and both came from a measurement taken before this ranking landed.fio:subcodeOfis materialised transitively (325110is a direct subcode of32,325and3251), so?ic fio:subcodeOf? ?selis an exact rewrite. Measured: 13s with the FILTER, query-planning timeout with the path. Recorded as a rejected design.Review notes
Three commits: the fix and its assertions, the docs, and the correction above. The snapshot file moves because 2 shapes changed plan; the rest of its 1,400 lines are untouched.
Worth knowing: every correction in this PR came from re-running a measurement rather than from reasoning, and reasoning would have gone the wrong way each time.