Repository navigation
chore(deps): update dependencies within their current major versions - #98
Merged
Merged
Conversation
Since DHIS2 2.42 the legacy app paths the scrape instructions point to (dhis-web-data-visualizer, dhis-web-maps, api/apps/line-listing, dhis-web-event-visualizer, dhis-web-event-reports) are redirected to /apps/<name>, where the global shell loads the app in an iframe. The scraper only searches the top-level document, so every item timed out (E2201) and no dashboard could be converted on 2.42+. Every goto step now opens <app>/index.html with redirect=false, the bypass the global shell itself uses for its iframe (GlobalShellFilter in dhis2-core). On 2.41 there is no shell and the parameter is ignored. The admin login clicked the login button and returned immediately, while the login app's POST /api/auth/login was still in flight. Work that arrived in that window (right after a service start, a worker replacement or a re-login) failed impersonation with a 401 (E2301). The worker now posts to /api/auth/login itself and only continues once loginStatus is SUCCESS. This endpoint exists since 2.41.0 and no longer depends on the login app's markup. Verified with a local Docker stack per version (2.41.10, 2.42.6, 2.43.2 and master) using the Sierra Leone e2e dump. AI Assisted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Run scrape steps through a promise chain instead of awaiting inside a loop, mark fields that are only set in the constructor as readonly, return the empty result directly from the async convert method, and use node: imports and Number.parseInt. No behaviour change. AI Assisted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dashboard details are fetched as the service account with the recipient's locale as a query parameter. Since DHIS2 2.42 the locale parameter only takes effect together with translate=true (UserContextInterceptor, introduced with DHIS2-18061 / DHIS2-18044), while 2.41 translated by default. Without it the dashboard title and item headings were rendered in the service account's language instead of the recipient's. Verified on 2.42.6 (nb recipient now gets the Norwegian dashboard and item names) and on 2.41.10 (unchanged). AI Assisted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ctor as readonly Resolves the SonarCloud finding on the file touched by the translate fix. No behaviour change. AI Assisted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he login Address review feedback: go back to the sequential for...of loop for the scrape steps (the reduce-based version only existed to satisfy SonarCloud S9382, the steps must run in order) and explain why the login first opens /api/ping on the DHIS2 origin. AI Assisted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Go back to the reduce-based sequential promise chain for the scrape steps. SonarCloud S9382 flags awaits inside a for...of loop, but the steps must run strictly in order, which a promise chain expresses without the per-iteration awaits. Agreed in review as the preferred shape over accepting the findings. AI Assisted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Address review feedback: the global shell filter skips the redirect for both directory-root and index.html paths when redirect=false is in the query string, so rewriting the path to index.html is unnecessary. That leaves two lines, which now live inline in the goto step, and removes the toStandaloneAppUrl helper and its tests. AI Assisted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Puppeteer's page.click dispatches a mouse event at the element's centre coordinates and leaves the mouse there. On the next conversion a worker could load a pivot table with its Period chip under that stale mouse position, so the chip's tooltip opened on top of the "HTML (.html)" menu item. The click then landed on the tooltip, no download tab opened and waitForTarget timed out after 30s (E2201). Calling element.click() in the page clicks the intended element regardless of overlays and never moves the mouse. AI Assisted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* chore: run the e2e suite across DHIS2 versions and fix setup decay The nightly e2e run has failed since January 2026: analytics tables were only generated for the last year, and the dump's data ends in 2025, so no analytics table was created and the generation script still reported success. The dump also cannot be upgraded past 2.41, the DHIS2 health check relies on curl which the 2.44 image no longer ships, and two cluster integration tests count a worker message that can arrive after the response. - Generate analytics from ANALYTICS_FIRST_YEAR (default 2025) onwards and fail when DHIS2 reports an error - Replace the DHIS2 container health check with a wait script in the post-install container - Run the nightly suite for core-dev 2.41, 2.42, 2.43 and master, building the service image from the checked out code - Wait for trailing worker messages in the single and parallel request integration tests, as the worker exit test already does - Document the corrected dump in DB.md (limited service account, fixed periods, receivers group, null org unit cleanup) AI Assisted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore: resolve SonarCloud findings in the nightly workflow and setup scripts Pin the third-party actions in the nightly workflow to full commit SHAs and use [[ ]] for the retry checks in the setup scripts. AI Assisted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore: add a script that creates the e2e database dump Address review feedback that producing a new dump was manual, finicky work. scripts/create-e2e-db-dump.sh starts DHIS2 on the stock Sierra Leone database and applies every change the e2e suite relies on: the service account, test users and their user groups, the "One of everything" dashboard, Norwegian translations and locale, fixed periods in DATA_YEAR, email and push analytics settings, no HTTP caching, and the push analytics jobs. It then dumps the database without analytics tables. DB.md documents the script and what it changes. The new dump is published on the 1.0.4 release as dump-2.41-2025.sql.gz and .env.e2e points at it. ANALYTICS_FIRST_YEAR moves into .env.e2e so it stays in step with the dump's data year. The fixtures are regenerated from this dump for dhis2/core:2.41 and for core-dev 2.41, 2.42, 2.43 and master. AI Assisted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore: take the push analytics URL template from the e2e environment Resolve SonarCloud S5332 (clear-text protocol) in the dump script: the URL template is now built in docker/e2e-db/docker-compose.yml from HOST and PORT in .env.e2e instead of being a literal in the script. The resulting system setting is unchanged. AI Assisted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
A worker kept the previously loaded app document when it switched the impersonated user or when a conversion failed, because subsequent goto steps only change the hash. A stuck app then failed every later item on that worker. Navigating to about:blank first forces a full page load; the codebase had no existing helper for forcing a document reload. AI Assisted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When several workers open Data Visualizer for the same new user at the same time, each one tries to create the user's analytics userDataStore keys. All but one get a 409, DV throws an uncaught error and the item times out. A single retry starts from a full page load, by which time the keys exist, so the app loads normally. The codebase has no shared retry helper, so the retry stays local to the scraper. AI Assisted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rker fix: recover the scraper when Data Visualizer fails to load for a new user
fix: click scrape step targets through the DOM
The login path now throws an AuthenticationError carrying the DHIS2 loginStatus, but establishNonExpiringAdminSession caught it and rethrew a generic message, so the actual cause never reached the logs. Log the caught error first. Carried over from #90, whose form-login change no longer applies now that login goes through /api/auth/login. AI Assisted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
All integration test files bound the mock cluster to port 1337, and node --test runs files in parallel, so three of the four binds failed. supertest 7.1 masked this by silently re-listening the failed server on a random port; supertest 7.3 records the late EADDRINUSE and fails the request with it. Listening on port 0 gives every file its own port. AI Assisted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings every dependency to its latest release within the current major, plus the dev-only majors nock 15 and globals 17, and refreshes the lockfile with npm update and npm audit fix (84 -> 48 audit findings). Supersedes dependabot PRs #69, #76, #82, #83, #84, #85, #87, #88, #89. Puppeteer now types the pageerror payload as Error | unknown, so logPageEvents reads the message only from an Error instance. Kept out as larger changes: release-it 19+ (#71), puppeteer 25, typescript 7, eslint 10, commitlint 21, lint-staged 17, dotenv 18 and @types/node 26. AI Assisted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
karolinelien
marked this pull request as ready for review
October 19, 2026 15:45
HendrikThePendric
added
the
e2e
HendrikThePendric
approved these changes
Oct 22, 2026
HendrikThePendric
changed the base branch from
fix/global-shell-standalone-app-url
to
master
October 22, 2026 08:17
HendrikThePendric
approved these changes
Oct 22, 2026
HendrikThePendric
enabled auto-merge (squash)
October 22, 2026 08:35
… npm 10 The lockfile was generated with npm 11, which leaves out the optional conventional-commits-parser@^6.4.0 peer of the git-client nested under @release-it/conventional-changelog. npm 10.9.1 in the alpine:3.20 Docker image then fails npm ci with "Missing: conventional-commits-parser@6.4.0 from lock file", breaking the e2e image build. Regenerated with npm 10.9.1; npm ci passes on both npm 10.9.1 and npm 11. AI Assisted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #93 so the multi-version e2e setup applies; retarget to
masteronce that merges.Why
What
npm auditdrops from 84 to 48 findings; the rest need majors kept out of this PR.@types/node26.pageerrorpayload asError | unknown;logPageEventshandles that.Version changes
Production image:
npm prune --productiondrops all devDependencies, so only the first four rows reach the running service.pageerrorpayload) is handled. Residual risk: the image drives Alpine 3.20's system Chromium, not Puppeteer's bundled Chrome, so the e2e run must pass before mergeRequestHandler, whose requests the integration tests exercise through nockAppScrapereslint.config.js; never loaded by the service. Lint passesstart:*scripts onlystart:devonlyTransitive packages were refreshed by
npm updateandnpm audit fixwithin the ranges their parents allow.How verified
npm run lintandnpm run buildpass.npm run test:unit: 21/21, twice in anode:ltscontainer and once on Windows.AI Assisted