Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,8 @@ The WebView root component is assigned to `globalThis.webViewComponent` (not exp

[src/services/projectStorage.ts](src/services/projectStorage.ts) — owns all `papi.storage` reads and writes for interlinearizer projects. Two serialization queues prevent interleaved read-modify-write races. Tests must call `resetQueuesForTesting()` between tests because module state is not cleared by `resetMocks`.

Drafts are loaded and saved through [AnalysisRepository](src/types/analysis-repository.ts), the seam a later storage backend implements. [DraftRepository](src/services/draftRepository.ts) implements it over the draft envelope and its per-book shards, and rewrites only the books that changed. It imports nothing from PAPI and reaches storage only through a `KeyValueStorage`. That lets the perf harness run it in Node, straight over the storage files ([perf/storage.ts](perf/storage.ts)).

### Components

Prefer Platform.Bible's `platform-bible-react` components (`Button`, `Input`, `Textarea`, `Label`, `Switch`, `RadioGroup`, `Popover`, `Tooltip`, etc.) over hand-rolled raw HTML (`<button>`, `<input>`, `<textarea>`, `<label>`, custom dropdowns) wherever an equivalent exists — this keeps the UI visually and behaviorally consistent with the host app and gives us theming and accessibility for free. Reserve raw elements for cases where the platform component can't preserve behavior the extension depends on (e.g. `field-sizing: content` gloss inputs); when you do, add a comment explaining why. When a raw element is replaced by a platform component in tests, extend `__mocks__/platform-bible-react.tsx` to stub the newly-used component.
Expand Down
21 changes: 8 additions & 13 deletions e2e-tests/global-setup-perf.ts
Original file line number Diff line number Diff line change
@@ -1,26 +1,21 @@
import type { FullConfig } from '@playwright/test';
import { execSync } from 'child_process';
import fs from 'fs';
import path from 'path';
import { execSync } from 'child_process';
import { DEV_STORAGE_DIR } from '../perf/paths';
import globalSetupCdp, { CDP_PORT } from './global-setup-cdp';
import { isPortInUse, WEBSOCKET_PORT } from './global-setup';
import globalTeardownCdp from './global-teardown-cdp';
import { launchPackagedApp, PACKAGED_APP_PATH, stopPackagedApp } from './perf-app';

/** The development host's storage for this extension's projects and drafts. */
const STORAGE_DIR = path.resolve(
__dirname,
'../../paranext-core/dev-appdata/extensions/interlinearizer/user-data',
);

/** Where the developer's storage waits out a run. */
const STORAGE_BACKUP_DIR = `${STORAGE_DIR}.perf-backup`;
const STORAGE_BACKUP_DIR = `${DEV_STORAGE_DIR}.perf-backup`;

/** Puts back storage a run set aside, discarding what the run stored. */
function restoreStorage(): void {
if (!fs.existsSync(STORAGE_BACKUP_DIR)) return;
fs.rmSync(STORAGE_DIR, { recursive: true, force: true });
fs.renameSync(STORAGE_BACKUP_DIR, STORAGE_DIR);
fs.rmSync(DEV_STORAGE_DIR, { recursive: true, force: true });
fs.renameSync(STORAGE_BACKUP_DIR, DEV_STORAGE_DIR);
}

/**
Expand All @@ -46,10 +41,10 @@ export default async function globalSetupPerf(config: FullConfig): Promise<() =>
}
// A crashed run leaves its backup in place; put it back before setting storage aside again.
restoreStorage();
fs.mkdirSync(STORAGE_DIR, { recursive: true });
fs.renameSync(STORAGE_DIR, STORAGE_BACKUP_DIR);
fs.mkdirSync(DEV_STORAGE_DIR, { recursive: true });
fs.renameSync(DEV_STORAGE_DIR, STORAGE_BACKUP_DIR);
try {
fs.mkdirSync(STORAGE_DIR);
fs.mkdirSync(DEV_STORAGE_DIR);
await globalSetupCdp(config);
} catch (error) {
restoreStorage();
Expand Down
11 changes: 8 additions & 3 deletions e2e-tests/perf-app.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@ import { spawn } from 'child_process';
import fs from 'fs';
import os from 'os';
import path from 'path';
import { CACHE_DIR } from '../perf/paths';
import { CDP_PORT, waitForRendererSettled } from './global-setup-cdp';
import { isPortInUse, waitForPort, WEBSOCKET_PORT } from './global-setup';
import { CACHE_DIR, DEV_STORAGE_DIR } from '../perf/paths';
import { E2E_SETTINGS_OVERRIDES } from './fixtures/helpers';
import { isPortInUse, waitForPort, WEBSOCKET_PORT } from './global-setup';
import { CDP_PORT, waitForRendererSettled } from './global-setup-cdp';
import { killProcessTree, waitForPidExit } from './process-utils';

/** Path to a packaged app's executable; set, it is what the harness launches. */
Expand All @@ -15,6 +15,11 @@ export const PACKAGED_APP_PATH = process.env.PERF_APP_PATH;
/** Home the packaged app runs under, apart from the developer's profile and kept across restarts. */
export const PERF_HOME_DIR = path.join(CACHE_DIR, 'app-home');

/** This extension's projects and drafts in the app a run drives. */
export const STORAGE_DIR = PACKAGED_APP_PATH
? path.join(PERF_HOME_DIR, '.platform.bible', 'extensions', 'interlinearizer', 'user-data')
: DEV_STORAGE_DIR;

/** Display backend the app runs on under Linux, overridable with `PERF_OZONE_PLATFORM`. */
export const OZONE_PLATFORM =
process.platform === 'linux' ? (process.env.PERF_OZONE_PLATFORM ?? 'x11') : undefined;
Expand Down
90 changes: 58 additions & 32 deletions e2e-tests/tests/perf/app-perf.spec.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
import type { Browser, Frame, Page } from '@playwright/test';
import type { DraftProject } from 'interlinearizer';
import path from 'path';
import { readDraftJson, readManifest } from '../../../perf/dataset/store';
import { TIERS, type TierSpec } from '../../../perf/dataset/tiers';
import { provenance, writeResults } from '../../../perf/results';
import { addSavedProject, readDraft, writeDraft } from '../../../perf/storage';
import { readCaptureMeta } from '../../../perf/usj-cache';
import { expect, test } from '../../fixtures/cdp.fixture';
import { waitForAppAndInterlinearizerReady } from '../../fixtures/helpers';
Expand All @@ -11,6 +13,7 @@ import {
OZONE_PLATFORM,
PACKAGED_APP_PATH,
stopPackagedApp,
STORAGE_DIR,
} from '../../perf-app';
import type { HostHeap } from './host-inspector';
import {
Expand Down Expand Up @@ -57,11 +60,36 @@ function tokenGlossInputs(frame: Frame) {
return frame.locator('input[aria-label^="Gloss for "]:not([aria-label^="Gloss for morpheme"])');
}

/**
* The view holding an edit or Save whose autosave has yet to be seen landing. Closing a view sends
* its pending autosave, which could then land after the next tier's seed and overwrite it.
*/
let unsavedFrame: Frame | undefined;

/** Glosses the `index`th visible word and commits it, as a reader moving on with Tab does. */
async function editGloss(frame: Frame, index: number, value: string): Promise<void> {
const input = tokenGlossInputs(frame).nth(index);
await input.fill(value);
await input.press('Tab');
unsavedFrame = frame;
}

/** Waits for the view's autosave to land, and stops counting the view as unsaved. */
async function waitForAutosave(frame: Frame): Promise<Measure[]> {
const measures = await waitForMeasure(frame, 'autosave');
if (unsavedFrame === frame) unsavedFrame = undefined;
return measures;
}

/** Waits for any autosave an edit or Save started to land. */
async function settleAutosave(): Promise<void> {
if (unsavedFrame) await waitForAutosave(unsavedFrame);
}

/** Closes the open view once any autosave it owes has landed, so none lands after a later seed. */
async function closeSettledView(page: Page): Promise<void> {
await settleAutosave();
await closeView(page);
}

/** Picks `item` from the Project menu, starting over when a re-render closes the menu mid-click. */
Expand All @@ -87,13 +115,17 @@ async function enableMarks(page: Page): Promise<void> {
}

/**
* Restarts the packaged app and points `app` at the new instance, with no view open.
* Restarts the packaged app and points `app` at the new instance, with no view open. Runs
* `whileStopped` between the old instance exiting and the new one launching, outside the startup
* time.
*
* @returns Milliseconds until the new instance was ready.
*/
async function restart(app: App): Promise<number> {
async function restart(app: App, whileStopped?: () => Promise<void>): Promise<number> {
await settleAutosave();
await app.browser?.close();
await stopPackagedApp();
await whileStopped?.();
const startupMs = await launchPackagedApp();
Object.assign(app, await connect());
// A view the layout restored would load before a timed open could.
Expand All @@ -102,31 +134,22 @@ async function restart(app: App): Promise<number> {
return startupMs;
}

/** Stores the tier's draft and saved projects, adding each project's id to `created` as it lands. */
/**
* Stores the tier's draft and saved projects straight into the app's extension storage, adding each
* project's id to `created` as it lands. A running app sees the draft from its next load of it, so
* no view may be open and no autosave in flight.
*/
async function seedTier(
page: Page,
projectId: string,
tier: TierSpec,
draftJson: string,
created: string[],
): Promise<void> {
await sendCommand(page, 'interlinearizer.saveDraft', projectId, draftJson);
const analysisJson = JSON.stringify(JSON.parse(draftJson).analysis);
const draft: DraftProject = { ...JSON.parse(draftJson), sourceProjectId: projectId };
await writeDraft(STORAGE_DIR, draft);
for (let i = 0; i < tier.savedProjects; i += 1) {
// eslint-disable-next-line no-await-in-loop
const projectJson = await sendCommand(
page,
'interlinearizer.createProject',
projectId,
['en'],
undefined,
`Perf ${tier.name} ${i + 1}`,
);
if (typeof projectJson !== 'string') throw new Error('createProject returned no project');
const { id } = JSON.parse(projectJson);
created.push(id);
// eslint-disable-next-line no-await-in-loop
await sendCommand(page, 'interlinearizer.saveAnalysis', id, analysisJson, 'null');
created.push(await addSavedProject(STORAGE_DIR, draft, `Perf ${tier.name} ${i + 1}`));
}
}

Expand Down Expand Up @@ -172,7 +195,7 @@ async function measureOpen(
.forEach(({ duration }) => open.host.addValue('page-fetch', duration));
const tokenizes = hostEntries.filter(({ name }) => name === 'getTokenizedBook:tokenize').length;
if (tokenizes !== 1) console.warn(`Opening ${book} tokenized it ${tokenizes} times, not once`);
await closeView(page);
await closeSettledView(page);
}

/** The bytes a measure's detail records, when it records any. */
Expand Down Expand Up @@ -207,15 +230,17 @@ async function measureTier(
return;
}

await closeView(app.page);
await seedTier(app.page, projectId, tier, draftJson, createdProjects);
const seed = () => seedTier(projectId, tier, draftJson, createdProjects);
await closeSettledView(app.page);
// A packaged app restarts before its first open below, so it is seeded while stopped.
if (!PACKAGED_APP_PATH) await seed();

// The first open after each app start.
if (PACKAGED_APP_PATH) {
const cold = scenarioSamples();
for (let run = 0; run < RUNS; run += 1) {
// eslint-disable-next-line no-await-in-loop
cold.webView.addValue('app-startup', await restart(app));
cold.webView.addValue('app-startup', await restart(app, run === 0 ? seed : undefined));
// eslint-disable-next-line no-await-in-loop
await measureOpen(app.page, projectId, tier.viewBook, cold);
}
Expand Down Expand Up @@ -303,7 +328,7 @@ async function measureTier(
// eslint-disable-next-line no-await-in-loop
await waitForMeasure(frame, 'dispatch-render');
// eslint-disable-next-line no-await-in-loop
const measures = await waitForMeasure(frame, 'autosave');
const measures = await waitForAutosave(frame);
edits.webView.add(measures, ['dispatch-render', 'autosave-serialize', 'autosave']);
// eslint-disable-next-line no-await-in-loop
edits.addHostCall(HOST_CALLS.saveDraft, await takeHostEntries(page), measures);
Expand All @@ -317,7 +342,7 @@ async function measureTier(
// eslint-disable-next-line no-await-in-loop
await editGloss(frame, i % 20, `undo-${i}`);
}
await waitForMeasure(frame, 'autosave');
await waitForAutosave(frame);
await clearMeasures(frame);
await recordMemory('withFullUndoHistory');

Expand Down Expand Up @@ -367,11 +392,12 @@ async function measureTier(
await takeHostEntries(page);
// eslint-disable-next-line no-await-in-loop
await openProjectMenuItem(frame, /^Save(?! As)/);
unsavedFrame = frame;
// eslint-disable-next-line no-await-in-loop
const measures = await waitForMeasure(frame, 'save');
// A save then persists the draft as clean, and that re-render would close the next run's menu.
// eslint-disable-next-line no-await-in-loop
await waitForMeasure(frame, 'autosave');
await waitForAutosave(frame);
saves.webView.add(measures, ['save']);
const bytes = detailBytes(measures, 'save');
if (bytes !== undefined) saves.webView.addValue('save-bytes', bytes);
Expand All @@ -386,7 +412,7 @@ async function measureTier(
const concordance = new Samples();
for (let run = 0; run < RUNS; run += 1) {
// eslint-disable-next-line no-await-in-loop
await closeView(page);
await closeSettledView(page);
// eslint-disable-next-line no-await-in-loop
const { frame: view } = await openView(page, projectId, tier.viewBook);
// eslint-disable-next-line no-await-in-loop
Expand All @@ -412,9 +438,9 @@ test('measure the app', async ({ mainPage }) => {

const requested = (process.env.PERF_TIERS ?? '').split(',').filter(Boolean);
const tiers = TIERS.filter((t) => requested.length === 0 || requested.includes(t.name));
await closeView(mainPage);
await closeSettledView(mainPage);
await deleteLeftoverProjects(mainPage, projectId);
const originalDraft = await sendCommand(mainPage, 'interlinearizer.getDraft', projectId);
const originalDraft = await readDraft(STORAGE_DIR, projectId);
await enableMarks(mainPage);

const createdProjects: string[] = [];
Expand All @@ -430,7 +456,7 @@ test('measure the app', async ({ mainPage }) => {
result.error = e instanceof Error ? e.message : String(e);
} finally {
// Each tier's picker lists only the projects that tier seeded.
await closeView(app.page);
await closeSettledView(app.page);
await Promise.all(
createdProjects
.splice(0)
Expand All @@ -440,10 +466,10 @@ test('measure the app', async ({ mainPage }) => {
console.log(`${tier.name}: ${JSON.stringify(result, undefined, 2)}`);
}, Promise.resolve());
} finally {
await closeView(app.page);
await closeSettledView(app.page);
await app.page.evaluate((key) => localStorage.removeItem(key), PERF_MARKS_KEY);
await setHostPerfMarks(app.page, false);
await sendCommand(app.page, 'interlinearizer.saveDraft', projectId, originalDraft);
await writeDraft(STORAGE_DIR, originalDraft);
}

const file = writeResults('app', {
Expand Down
3 changes: 3 additions & 0 deletions perf/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ A dataset is a tier name, or a `draft:<sourceProjectId>` or `project:<id>` key i
- Global setup builds the extension for production (`E2E_BUILD_SCRIPT=build:production`).
- The run gets empty extension storage, so listing projects reads only what the run seeds. Your own storage waits in `user-data.perf-backup` beside it and comes back on teardown.

The run writes each tier's draft and saved projects straight into the app's extension storage from Node. It goes through the extension's draft repository, so seeding needs no command that carries a whole draft. A dev core keeps running meanwhile, so the run seeds only once the view has closed and its last autosave has landed.

Quit Platform.Bible first. A perf run refuses to start while any instance holds its ports, since that app may own the storage a run sets aside.

The timings are `ilz:`-prefixed performance entries. The WebView records them only while the `interlinearizer.perfMarks` localStorage key is `true`, and the extension host only after the `interlinearizer.setPerfMarks` command turns them on; the spec does both. `interlinearizer.readPerf` takes the host's entries, with its process id.
Expand All @@ -48,6 +50,7 @@ Set `PERF_APP_PATH` to the packaged executable (`release/build/linux-unpacked/pl
- Setup builds the extension for production.
- The app runs with its home folder (`HOME`, and `USERPROFILE` on Windows) set to `perf/.cache/app-home`, so it installs WEB into its own profile and never touches yours.
- Teardown asks it to quit.
- Each tier is seeded while the app is stopped, inside the restart before its first open.
- `perf:app` also times the first open after each app start, restarting the app before each run. A dev core gets the warm scenarios only.

On Linux the app runs on X11 unless `PERF_OZONE_PLATFORM` names another backend, such as `wayland` when Xwayland is unavailable; results record it.
Expand Down
6 changes: 6 additions & 0 deletions perf/paths.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,12 @@ export const DATASETS_DIR = path.join(CACHE_DIR, 'datasets');
/** Committed result files. */
export const RESULTS_DIR = path.join(PERF_DIR, 'results');

/** The development app's storage for this extension's projects and drafts. */
export const DEV_STORAGE_DIR = path.resolve(
PERF_DIR,
'../../paranext-core/dev-appdata/extensions/interlinearizer/user-data',
);

/** Short name of the platform's bundled sample project, the text every tier is drawn from. */
export const SOURCE_PROJECT_NAME = 'WEB';

Expand Down
Loading
Loading