Skip to content

fix for add from pool bug (long loading time) mantis 47724 - #11821

Open
mglaubitz wants to merge 1 commit into
ILIAS-eLearning:release_10from
mglaubitz:release_10-47724
Open

fix for add from pool bug (long loading time) mantis 47724#11821
mglaubitz wants to merge 1 commit into
ILIAS-eLearning:release_10from
mglaubitz:release_10-47724

Conversation

@mglaubitz

Copy link
Copy Markdown

we let one of our local AI models (GLM 5.2) analyse the problem and create a bug fix. this reduces loading time on ourt exam-test server from 3.6 minutes to 6.2 seconds (!)

… index

The 'Add from pool' question browser (ilObjTestGUI -> ilTestQuestionBrowserTableGUI
-> ilAssQuestionList::load) built one large query with 7 correlated EXISTS
subqueries (feedback x4, hints, taxonomies) as SELECT fields. These were
evaluated for every candidate row BEFORE ORDER BY/LIMIT, so on large
instances (bug report: ~199k rows) the query ran 30-40 min and blocked
other queries.

Fix: when a Range is set and neither a HAVING filter nor an ORDER BY on a
computed column (feedback/hints/taxonomies) is active, load in two phases:
  Phase A: SELECT question_id + required JOINs/filters + GROUP BY +
           ORDER BY + LIMIT (no EXISTS subqueries) -> small paginated id set
  Phase B: full SELECT incl. feedback/hints/taxonomies EXISTS, restricted
           to the paginated ids via IN (...) -> flags computed only for the
           ~800 visible rows instead of the full candidate set.

Fallback to the original single-phase query for: no range, HAVING filter
(feedback/hints = true|false), ORDER BY feedback/hints/taxonomies.

buildOrderQueryExpression now qualifies columns for phase A (qpl_questions.*
not selected -> ambiguous title) and backticks qualified names per segment
(`qpl_questions`.`title`).

Adds composite index i6 (obj_fi, original_id, title) on qpl_questions
via ilTestQuestionPool10DBUpdateSteps::step_3() to support the phase A
filter (obj_fi IN ... AND original_id IS NULL) + default title ordering.

Adds ilAssQuestionListTwoPhaseTest (15 tests) covering the two-phase path,
all fallback conditions, column qualification and SQL shape.

Verified with EXPLAIN against the local docker DB: phase A has no
DEPENDENT SUBQUERY (3 simple JOINs only), phase B's subqueries are
evaluated over rows=2 (paginated set) instead of the full candidate set.
@kergomard

kergomard commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Hi @mglaubitz

Thank you for the PR.

First one question:

  • Did a developer look at this PR before it was posted? Who was it, so I can interact with them and so I know who takes responsibility for this code? Please be so kind and only have developers create PR changing code, we need a vis-a-vis to interact with.

What happens if you only add the first of the two changes (the index) provided in this PR: This index seems to be added to improve the existing query (and potentially the new one) as the corresponding fields seem to not show up elsewhere in your changes. It should already improve the situation somewhat. What was the improvement if only this change was applied (see)?

Questions:

  • Why does the existence of a range disallow the usage of a two phase query? I can see why it would not be necessary, but no reason to disallow it.
  • Structurally this to me seems wrong: It creates a completely separate execution path if a set of conditions is met. I think the correct path would be to create a "vanilla" path when there are no filters or limits. This path can (and probably already should contain two steps). Then the first step is enriched with the data that is needed for a correct filtering and ordering. The first query should then not only retrieve the ids, but all data that can easily be retrieved. The second step should only add the data that is hard to retrieve and it should only do so, if retrieving it was not already necessary to complete the first step.

Change Requests:

  • Please remove all comments. They just clutter the PR.
  • Please never have two nested function calls on one line.
  • Please go through the code and fix unnecessary complexity (e.g. here) and inconsistencies (e.g. that variables are once in curly braces and once not in the previous link, where the use of variables should be removed anyway, but just as an example). All variables in strings should always be in curly braces.
  • Please move all table names used to class constants. This is more relevant now, as you are qualifying fields in a separate function "qualifyOrderField()". Please qualify everything not only the order fields. This will lead to a more readable query.

This is just the first round, as far as I can see. I.e. function naming seems to be somewhat lacking, but the correct names will only become clear once we really have a final structure. Once this is done the code should have become more readable and logical, so that we can figure out, what else needs to be done.

Best,
@kergomard

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement php Pull requests that update Php code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants