Repository navigation
perf(wda): stop a simulator's app with simctl before WebDriverAgent's terminate - #271
Conversation
… terminate WebDriverAgent's terminate is XCTest's, which took 1.1-2.1 s to stop the app on a simulator; `simctl terminate` takes 0.2 s. A flow pays it at every launchApp of a running app (clearState's launch stops it before the reinstall) and at stopApp and killApp. With a session on a simulator, the driver now stops the app with simctl, then sends WebDriverAgent's terminate, which returns at once for an app that is not running and keeps the session's bookkeeping: an app of the session stopped behind its back reads as crashed at the next request. launchApp's stop before a launch goes through the same path. A real device stops it through WebDriverAgent only.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
Thanks, Luka. We measured this on our simulator before merging, and it made terminating slower:
|
|
Thanks, you're right, and I could reproduce it. With the runner starting its own WebDriverAgent (xcodebuild, Xcode 27.2, an iOS 27.2 simulator), WebDriverAgent's terminate after simctl's took 8-12 s in 13 of 25 stops of our app with this change, and in none of 20 without it; a flow of five launchApp/stopApp pairs ran 50-60 s against 29 s. My numbers came from a WebDriverAgent started with |
On a simulator the WDA driver stops an app through WebDriverAgent (
/wda/apps/terminate), which is XCTest's terminate: on an iPhone 18 Pro simulator (iOS 27.2) it took 1.1-2.1 s for our app, wherexcrun simctl terminatetakes 0.19 s. A flow pays it at everylaunchAppof a running app (aclearStatelaunch stops it before the reinstall, a plain one before relaunching), and atstopAppandkillApp. DeviceLab's iOS driver and Maestro stop a simulator's app with simctl.The change:
terminateApp, when there is a session and the device is a simulator, runssimctl terminatefirst, then sends WebDriverAgent's terminate as before. The second call is still needed: when the session's app is stopped behind WebDriverAgent's back, its next request fails with "The application under test … is not running, possibly crashed". After simctl it returns in 20-30 ms (the app is not running) and clears that state.launchApp's stop before a launch goes throughterminateApptoo. A real device is unchanged (WebDriverAgent only), as is a simulator without a session (simctl only, as before). simctl's own error (the app was not running) is logged at debug level and ignored.Measured on that simulator, three rounds each: simctl then WebDriverAgent 0.18 + 0.02 s against WebDriverAgent alone 1.1-2.1 s, the app state "not running" after and the next request answering normally; five flows that start with
launchApp: clearStatein one runner process: each 0.3-0.8 s faster, 71.6 s against 74.5 s for the five (medians), every flow passing with both.Tests:
TestTerminateAppOnASimulatorUsesSimctlFirst(with the package'sexecCommandseam: a simulator runs simctl, then WebDriverAgent's terminate; a device only the latter; fails without the change).go test -race ./pkg/driver/wda/ ./pkg/executor/ ./pkg/flow/passes; golangci-lint v2.13.2: 0 issues.