ci: native parity tests against the real demo app on macos-26 by rdlabo · Pull Request #180 · rdlabo-dev/ionic-theme-ios27

rdlabo · 2026-09-16T10:33:42Z

Summary

NativeGlassPoCTests (XCTest) drives the actual Capacitor demo app: with -parity, ParityMode.swift mounts UIKit reference controls (Save button, 3-item segmented control, 4-item tab bar) and the app component reports DOM frames of the real /main/index/native-ui-shell page + shell ion-tab-bar over a WKScriptMessageHandler, so web↔native geometry is compared live in a single run — no JSON fixtures, no separate fixture app.
Audit target is the shared native-ui-shell page (same as the iOS 27 branch's focused test): back button, Save, segment, menu/actions buttons, and the shell tab bar. The harness resizes each reference to the measured web width so item divisions compare like for like.
Geometry assertions are hard gates where meaningful (segment item widths match exactly at 65pt each); known real gaps (segment height 48 vs 32pt, tab-pill item anchoring/division 83.25 vs 74pt) are wrapped in XCTExpectFailure so they stay visible without red CI.
Light and dark runs attach screenshots + the full metrics payload as artifacts.
scripts/verify-ios26-parity.sh builds the demo, cap syncs, installs on the simulator, generates the test project with XcodeGen, and runs xcodebuild test. DerivedData is kept outside the uploaded artifacts dir.
Workflow: manual workflow_dispatch + weekly (Mon 03:00 JST) on macos-26 / iOS 26.5. ci-simulator.sh is fetched from the pinned ionic-theme-utils sha (ec42938).
demo/ios/App regenerated on the Capacitor 8 SPM layout (CocoaPods files removed).

Test plan

sh scripts/verify-ios26-parity.sh locally: 2 tests, 0 failures (expected-failure annotations only)
GitHub Actions run green: https://github.com/rdlabo-dev/ionic-theme-ios27/actions/runs/35116127182
Review parity-report JSON + screenshots in the uploaded artifact

Generated with Devin

NativeGlassPoCTests drives the actual Capacitor demo app: a parity-mode native overlay mounts UIKit references (buttons, toggle, segmented control, slider, tab bar) while the native-glass-poc page reports DOM frames over the WebKit bridge, so geometry is compared live in one run. - AppDelegate/ParityMode mount references and merge web metrics into a JSON report label; XCTest asserts frames with tolerances. - Known gaps (button height, segment height, tab pill x) are wrapped in XCTExpectFailure so real differences stay visible without red CI. - Workflow runs the parity script on macos-26 (iOS 26.5), manual + weekly. - Simulator setup script is fetched from the pinned ionic-theme-utils sha. - Regenerate demo/ios/App on the Capacitor 8 SPM layout (Pods removed). Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

The report label mounts empty; waitForExistence returned before web metrics arrived, so CI parsed an empty string. Wait on a non-empty label predicate instead. DerivedData (multi-GB index caches) also lived inside VERIFY_ARTIFACTS_DIR and was uploaded wholesale; move it to a separate scratch dir. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

devin-ai-integration

Devin Review found 3 potential issues.

devin-ai-integration · 2026-09-16T10:36:56Z

+  {
+    path: 'native-glass-poc',
+    loadComponent: () => import('./native-glass-poc/native-glass-poc.page').then((m) => m.NativeGlassPocPage),
+  },


🔍 テスト用ページが通常のデモ配信にも含まれる

native-glass-poc は通常ルートとして常時公開され、ネイティブハーネスがないブラウザでも直接アクセスできます。テスト専用コードを本番デモへ含める方針か確認してください。

Was this helpful? React with 👍 or 👎 to provide feedback.

devin-ai-integration · 2026-09-16T10:36:57Z

+  selector: 'app-native-glass-poc',
+  templateUrl: './native-glass-poc.page.html',
+  styleUrls: ['./native-glass-poc.page.scss'],
+  standalone: true,


🔍 Angular 規約に反する明示的な standalone 指定

リポジトリ規約は暗黙的 standalone を求めていますが、この新規コンポーネントは standalone: true を明記しています。動作には影響しませんが、削除を検討してください。

Was this helpful? React with 👍 or 👎 to provide feedback.

devin-ai-integration · 2026-09-16T10:36:58Z

+    const timer = setInterval(() => {
+      if ((window as any).webkit?.messageHandlers?.parity) {
+        clearInterval(timer);
+        void this.#router.navigateByUrl('/native-glass-poc');
+      }
+    }, 200);
+    setTimeout(() => clearInterval(timer), 10_000);


🔍 通常起動でも10秒間ポーリングが動作する

ルートコンポーネントはブラウザを含む全起動で、200ミリ秒ごとにネイティブハンドラーを探索します。テストモードだけで開始できる仕組みを検討してください。

Was this helpful? React with 👍 or 👎 to provide feedback.

github-actions · 2026-09-16T10:38:27Z

Playwright test results

175 passed

Details

175 tests across 14 suites
1 minute, 38 seconds
5f1d11b
This detailed result covers Ionic 9 only. Ionic 8 runs against the same screenshots in a separate matrix job; check the workflow run for both results. To update the screenshots, comment with /update-screenshots.

github-actions · 2026-09-16T10:41:15Z

📊 Ionic 9 Playwright Test Report

View the detailed Ionic 9 report: https://rdlabo-dev.github.io/ionic-theme-ios27/pr-180/

Ionic 8 runs against the same screenshots in a separate matrix job. View both results in the workflow run.

github-actions · 2026-09-16T10:42:00Z

📊 Ionic 9 Playwright Test Report

View the detailed Ionic 9 report: https://rdlabo-dev.github.io/ionic-theme-ios27/pr-180/

Ionic 8 runs against the same screenshots in a separate matrix job. View both results in the workflow run.

Replace the standalone native-glass-poc page with the same /main/index/native-ui-shell audit page used on the iOS 27 branch so both themes are measured against one real demo surface. Web metrics now live in the app component and the parity harness resizes its UIKit references to the measured web widths, so segment division is asserted strictly while known tab-pill anchoring differences stay non-strict. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

github-actions · 2026-09-16T15:40:45Z

📊 Ionic 9 Playwright Test Report

View the detailed Ionic 9 report: https://rdlabo-dev.github.io/ionic-theme-ios27/pr-180/

Ionic 8 runs against the same screenshots in a separate matrix job. View both results in the workflow run.

github-actions · 2026-09-16T15:59:32Z

📊 Ionic 9 Playwright Test Report

View the detailed Ionic 9 report: https://rdlabo-dev.github.io/ionic-theme-ios27/pr-180/

Ionic 8 runs against the same screenshots in a separate matrix job. View both results in the workflow run.

rdlabo

The live UIKit/Web geometry comparison is an interesting experiment, but this PR is no longer a practical merge candidate. It is based on the old ios26 branch and combines the parity test with a large demo iOS project regeneration, an application-only bridge, and several expected failures.

That creates substantial production-demo and maintenance surface for checks that overlap with the existing native shell suites and do not currently act as strict regression gates. I’m closing this PR rather than rebasing the whole harness. If a specific geometry invariant still needs coverage, it should be added as a focused assertion to the current native fixture or shell suite from main.

rdlabo and others added 2 commits September 16, 2026 18:56

devin-ai-integration Bot reviewed Sep 16, 2026

View reviewed changes

github-actions Bot added a commit that referenced this pull request Sep 16, 2026

Update test report for PR #180 b7e2c8f

793f43a

rdlabo mentioned this pull request Sep 16, 2026

Native parity gaps: toolbar route missing UIKit tabs (iOS 27), search retirement, tab positions (iOS 26.5) #181

Closed

github-actions Bot added a commit that referenced this pull request Sep 16, 2026

Update test report for PR #180 3ca6184

1ca637c

github-actions Bot added a commit that referenced this pull request Sep 16, 2026

Update test report for PR #180 3ca6184

a7ab330

rdlabo commented Sep 21, 2026

View reviewed changes

rdlabo closed this Sep 21, 2026

Navigation Menu

Uh oh!

ci: native parity tests against the real demo app on macos-26 - #180

ci: native parity tests against the real demo app on macos-26#180
rdlabo wants to merge 3 commits into
ios26from
ci/native-e2e-ios26

rdlabo commented Sep 16, 2026 •

edited

Loading

Uh oh!

devin-ai-integration Bot left a comment

Uh oh!

devin-ai-integration Bot Sep 16, 2026

Uh oh!

devin-ai-integration Bot Sep 16, 2026

Uh oh!

devin-ai-integration Bot Sep 16, 2026

Uh oh!

github-actions Bot commented Sep 16, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 16, 2026

Uh oh!

github-actions Bot commented Sep 16, 2026

Uh oh!

github-actions Bot commented Sep 16, 2026

Uh oh!

github-actions Bot commented Sep 16, 2026

Uh oh!

rdlabo left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

rdlabo commented Sep 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Sep 16, 2026

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Sep 16, 2026

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Sep 16, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Sep 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Playwright test results

Details

Uh oh!

github-actions Bot commented Sep 16, 2026

Uh oh!

github-actions Bot commented Sep 16, 2026

Uh oh!

github-actions Bot commented Sep 16, 2026

Uh oh!

github-actions Bot commented Sep 16, 2026

Uh oh!

rdlabo left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

rdlabo commented Sep 16, 2026 •

edited

Loading

github-actions Bot commented Sep 16, 2026 •

edited

Loading