Improve SQL generator by grouping "exists" statements better - #471
Merged
Merged
Conversation
When an attribute is attached directly to some servertypes and inherited by others through a related_via attribute, _real_condition_sql() built a single EXISTS over the value table whose WHERE OR'ed the relation paths together. Postgres can turn a correlated EXISTS with one path into a hash semi join, but not one whose correlation to "server" is an OR of alternatives: it fell back to a nested loop over every (server, sub) pair and evaluated the inherited path as a sub plan for each of them. Emit one EXISTS per path instead and OR them outside, each guarded by its servertype test written first. The guard order matters: Postgres reorders AND clauses by cost at the top level only, not inside the branches of an OR, and otherwise evaluates left to right, so the cheap test now short-circuits the EXISTS for servertypes that do not use that path. Boolean-equivalent to the old form, since the guards do not depend on the inner row and factor out of the existential.
An attribute inherited through a related_via attribute was filtered with an EXISTS correlated to the outer server row. Postgres evaluated it once per candidate server, re-finding the same matching rows every time and then testing each (server, match) pair one by one. Render each inherited path as "server.server_id IN (subquery)" with nothing in the subquery referencing the outer server. Postgres then evaluates it once, hashes the ids and probes per row, and for a supernet path drives the containment join from the matching networks' prefixes, which is what the GiST index exists for. Cost becomes (candidates + matches) rather than their product. The directly attached path keeps its correlated EXISTS, which Postgres already turns into a semi join (or an anti join under NOT), so the SQL for attributes that are not inherited anywhere is unchanged. The selected column is a NOT NULL foreign key in every branch, so NOT IN keeps set semantics: no NULL can make the test unknown. _supernet_af_sql() factors the address-family joins out of _supernet_exists_sql(), which still serves direct supernet filters and ContainedOnlyBy and emits exactly what it did before.
Contributor
Author
|
This code has been written with assistance of an LLM. I tested and verified and reviewed them myself. The improvements especially for bigger queries are quite impressive I saw queries dropping from 40 to 5 seconds runtime. |
brainexe
approved these changes
Sep 23, 2026
brainexe
left a comment
brainexe
left a comment
Member
There was a problem hiding this comment.
code looks valid + did some local tests and it still produced the expected results
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains improvements to the sql_generator module to better group exists for related attributes allowing the
Postgres query planner to filter rows more efficient.