Repository navigation
feat(hardware-button): press phone volume and power buttons on Android and iOS #3848
Description
Purpose
Agents can't press a phone's volume or power button. Today they fall back to raw adb shell input keyevent on Android and have no route at all on iOS. PR #3817 was going to add a hardware-key command, but it is being cut down to TV-only tv-remote keys, so phones are left out. This issue adds a small, honest phone vocabulary for those buttons. It is a separate decision from the TV remote keys.
iOS can press some hardware buttons; our implementation just has no route for them yet. XCUIDevice.Button has volumeUp and volumeDown, and Appium XCUITest mobile: pressButton accepts volumeup and volumedown on real devices. iOS does not expose a public power or lock button press.
API
agent-device hardware-button volume-up
agent-device hardware-button volume-down
agent-device hardware-button powerawait client.command.hardwareButton({ button: 'volume-down' });
// => { action: 'hardware-button', button: 'volume-down', message: 'Pressed hardware button volume-down' }- One positional argument,
<volume-up|volume-down|power>. No aliases. An unknown name returnsINVALID_ARGSbefore any device call. - No hold verb or duration flag in the MVP (see "Long press" below).
- No
--settleand no post-action observation. The command must not activate or re-foreground the session app. This usesaction-button's trait set, notpress's. - The result reports that the press was dispatched. It does not report a resulting volume level or screen state.
Ownership: extend the system-button family, not a new dispatcher
packages/contracts/src/system-button-runtime.ts already owns no-argument device presses (home, appSwitcher, actionButton): one fact cell per button, a family unsupported default, and bindSystemButton. This change adds volumeUp, volumeDown and power to SYSTEM_BUTTONS and SYSTEM_BUTTON_LABELS, and adds optional Interactor members volumeUp?(), volumeDown?(), power?(). The positional picks the cell, the way keyboard <status|dismiss|enter> picks a keyboard-family cell.
- Every owner that already calls
systemButtonRuntimeOperationFactsgets the new cells refused through its existingunsupporteddenial. Only the owners in the matrix below name a cell as available. - No
SystemControlunion, no key table shared withtv-remote, noandroid-keycode:Nescape hatch, and nocase 'hardware-button'dispatcher that bypasses the family binder. home,app-switcherandaction-buttonkeep their existing commands.hardware-buttondoes not accepthome,app-switcheroraction, so each button has exactly one public route. Foldingaction-buttonintohardware-buttonwould be a separate migration decision.- On TV targets (Android TV, tvOS),
hardware-buttonrefuses with a hint that points totv-remote. After feat(tv-remote): add hardware and media buttons for TV targets #3817,tv-remoteowns TV volume and power keys.
Support matrix (MVP)
Every known platform, device-kind, and provider refusal happens at fact admission, before any adb, runner, or provider dispatch, and uses an existing RuntimeOperationUnavailability reason. A physical device hardware-presence check may run in the runner, but it must refuse before pressing an absent button.
| Owner / backend | Device kind | volume-up / volume-down | power | Mechanism / refusal |
|---|---|---|---|---|
| Local Android (ADB) | emulator, physical phone/tablet | available | available | input keyevent 24 / 25 / 26 (KEYCODE_VOLUME_UP/DOWN, KEYCODE_POWER), same route as home (keyevent 3) |
| Local Android (ADB) | Android TV target | refused | refused | unsupported-platform-leaf, hint: use tv-remote |
| Limrun Android | cloud emulator | available | available | same input keyevent route Limrun Android home/app-switcher use |
| Appium Android (UiAutomator2) | phone/tablet session only | available when the provider declares the capability | same | mobile: pressKey { keycode }, the call keyboard Enter already makes. Undeclared capability: owner-capability-missing with the provider note |
| Local Apple runner | physical iPhone/iPad | available | refused | XCUIDevice.shared.press(.volumeUp/.volumeDown), guarded by hasHardwareButton(_:) (iOS 16+). No public XCUIDevice lock/power button: unsupported-platform-leaf |
| Local Apple runner | iOS/iPadOS Simulator | refused | refused | unsupported-device-kind. The SDK header marks XCUIDeviceButtonVolumeUp/Down XCUI_SIMULATOR_UNAVAILABLE, so the runner code must be inside #if !targetEnvironment(simulator) |
| Local Apple runner | tvOS, macOS, watchOS, visionOS | refused | refused | unsupported-platform-leaf (tvOS hint: tv-remote) |
| Appium iOS (XCUITest) | real device | available when declared | refused | mobile: pressButton { name: 'volumeup' | 'volumedown' } (real devices only, per driver docs). mobile: lock sets lock state; it is not a button press, so it is not mapped to power |
| Appium iOS (XCUITest) | simulator | refused | refused | unsupported-device-kind. If the provider does not report the kind, the cell refuses rather than guessing |
| Limrun iOS | cloud simulator | refused in MVP | refused in MVP | SDK performActions types list buttonDown/buttonUp with volumeUp, volumeDown, lock and side, but there is no live evidence. See Future |
| Web, Linux, macOS desktop, HarmonyOS, Vega | — | refused | refused | each owner's existing family denial |
Within an admitted Apple leaf, if hasHardwareButton returns false at press time, the runner returns UNSUPPORTED_OPERATION with a hardware hint. This is the same pattern as actionButton in RunnerTests+CommandExecution.swift. It is never error-text matching.
Press events, not volume settings
hardware-button volume-down sends one button event. What the OS does with it (which stream changes, whether a HUD shows, whether the foreground app intercepts it) is device behavior, and the command does not claim any of it. Setting an absolute volume (Android cmd media_session volume --set, iOS: no public API) is a different feature. If it is wanted, it belongs in the settings catalog (packages/contracts/src/settings-catalog.ts, which has no volume entry today), not here.
power toggles the screen on Android. After a press that turns the screen off, later commands may hit a locked screen. The help text says so. It does not add an automatic wake.
Long press
Long press is not in the MVP. The backends do not agree:
- Android ADB:
input keyevent --longpress <code>, already used bytv-remote longpress. Appium Android:mobile: pressKey { isLongPress: true }. Both use a fixed system long-press duration, not a caller-chosen duration. - Local Apple and Appium iOS:
XCUIDevice.press(_:)has noforDuration:overload. Appium'spressButtonduration applies only to tvOS. - Limrun iOS:
buttonDown+wait+buttonUpcould hold for a duration, but this is unverified.
If long press is added later, represent it as a press shape, without a separate runtime-operation fact cell (ADR 0019 §2). Unsupported shapes must refuse before the button event is sent. Its API and validation are a separate decision.
Out of scope / future (each its own decision)
| Candidate | Evidence | Why not MVP |
|---|---|---|
sleep, wake |
Android KEYCODE_SLEEP 223 / KEYCODE_WAKEUP 224 (API 20), idempotent |
Android-only. Cheap to add as two more family cells once someone needs a lock/wake flow |
camera |
Android KEYCODE_CAMERA 27. Apple XCUIDevice.Button.camera (iOS, Xcode 16.3+). Appium camera (Xcode 16+, iOS 16+, real devices only) |
#2699 left Camera Control out on purpose (press only, no slide/light-press). Needs that decision, plus a physical iPhone 16 for evidence |
mute |
Android KEYCODE_VOLUME_MUTE 164 |
Android-only. iPhones have no mute key |
iOS power / lock |
No public XCUIDevice button. Appium mobile: lock / unlock / isLocked is lock state. Limrun lock/side buttons (types only) |
A lock-state command would be a different contract from a press |
| Limrun iOS volume | SDK types only | Admit once a live Limrun run shows a volume change |
| Siri | XCUISiriService.activate(voiceRecognitionText:) is a text query API, not a button |
Different feature. No promise here |
| Silent / ring switch | No XCUITest, Appium or Limrun API found | Unsupported. No promise |
| Absolute volume | — | settings catalog decision, see above |
Completion conditions
agent-device hardware-button volume-downon a local Android emulator returns{ action: 'hardware-button', button: 'volume-down', message }, andadb shell cmd media_session volume --stream 3 --getreads one step lower than before.volume-upraises it back.agent-device hardware-button poweron a local Android emulator changesadb shell dumpsys powermWakefulnessfromAwaketoAsleep. A second press restoresAwake.- On a physical iPhone over the local runner,
hardware-button volume-upsucceeds. The evidence is the system volume HUD in a screenshot taken right after, or a changedAVAudioSession.outputVolumeread in a fixture app. The session app is not activated or re-foregrounded. hardware-button volume-upon an iOS Simulator, andhardware-button poweron any Apple leaf, refuse at admission with a typed reason and hint. No runner request is issued, and a test asserts this. The runner still builds for the Simulator destination.- Appium Android: a unit test asserts
mobile: pressKeywith{ keycode: 24 | 25 | 26 }. Appium iOS: a unit test assertsmobile: pressButtonwith{ name: 'volumeup' | 'volumedown' }, and thatpoweris refused. A live run on any Appium Android session, and on a real-device Appium iOS session if available, is posted on the PR. If none is possible, the PR says so. hardware-button volume-upon Android TV and tvOS refuses with a hint totv-remote. On web, Linux, macOS, HarmonyOS, Vega and Limrun iOS, it refuses at admission.hardware-button sidewaysreturnsINVALID_ARGSbefore any adb, runner or provider call.- A provider scenario under
test/integration/provider-scenarios/covers the Android ADB argv and the response shape (integration-progressstays complete). The MCP tool andclient.command.hardwareButtonexist, and the MCP output schema is derived fromCommandResultMap. - CLI help,
website/docs/docs/commands.md, and every refusal hint name the same backends the facts admit. pnpm check:affected --runpasses, including the registry, owner-files and generic-route completeness gates.
Dependencies
- Blocked by: feat(tv-remote): add hardware and media buttons for TV targets #3817 landing in its TV-only form (it removes
hardware-keyand folds TV volume/power intotv-remote). Rebase onto it so there is exactly one route per target class. - If the command registry has no room for a descriptor, make a statement-neutral registry split under ADR 0027. Do not raise its cap to bypass the gate. The TV-only update to feat(tv-remote): add hardware and media buttons for TV targets #3817 removes its descriptor and its proposed cap increase.
- Device verification needs a physical iPhone for condition 3 (see
docs/agents/device-verification.md). The Android conditions run on any emulator.
Remaining uncertainties
- Whether a volume
XCUIDevice.presson a physical iPhone shows the HUD or changesoutputVolumewhile an XCUITest session is attached. Not live-verified. Appium documents it as working on real devices. - Whether a cloud Appium iOS session reports real-device vs simulator in a way the fact can read before dispatch. If not, the iOS cell refuses unless the provider declares real devices.
Sources
- Apple,
XCUIDevice.Button(home, volumeUp, volumeDown, action, camera; Xcode 16.3): https://developer.apple.com/documentation/xcuiautomation/xcuidevice/button - Apple,
hasHardwareButton(_:)(iOS 16+): https://developer.apple.com/documentation/xcuiautomation/xcuidevice/hashardwarebutton(_:) - Xcode 27.1 SDK header
XCUIAutomation.framework/Headers/XCUIDeviceButton.h:XCUIDeviceButtonVolumeUp/DownareXCUI_SIMULATOR_UNAVAILABLE.XCUIDevice.hhas onlypressButton:(no duration) and no lock button. - Appium XCUITest
mobile: pressButton,performIoHidEvent,lock: https://github.com/appium/appium-xcuitest-driver/blob/master/docs/reference/execute-methods.md#mobile-pressbutton - Appium UiAutomator2
mobile: pressKey(isLongPress),lock: https://github.com/appium/appium-uiautomator2-driver#mobile-presskey - Android
KeyEventconstants: https://developer.android.com/reference/android/view/KeyEvent @limrun/api0.64.1ios-client.d.tsPerformActionbuttonDown/buttonUpbutton names.- Precedent: feat(action-button): press the iPhone Action Button #2699 / feat(action-button): press the iPhone Action Button #2702 (action-button), refactor(system-button): declare home, app-switcher and action-button as one button family #2715 (system-button family), PR feat(tv-remote): add hardware and media buttons for TV targets #3817 (Android emulator volume transcript, as reported).
Activity
thymikee commentedon Oct 20, 2026
Decision: Dependency on #3817: rescoped to coordination; the phone API is implemented on current main.
Verified against
origin/mainat421dd7e8e7:hardware-keycommand.git grep "'hardware-key'"over the command surface (packages/command-registry,src/commands) finds nothing on main.tv-remoteis already TV-only: it owns the DPAD/select/menu/home/back keys for TV targets and their runtimes, and no phone owner admits the cell.So the phone
hardware-buttoncommand can land without #3817. The two surfaces stay disjoint by construction:hardware-buttonaccepts onlyvolume-up | volume-down | powerand refuses Android TV and tvOS sessions at fact admission with a hint namingtv-remote.tv-remotekeeps TV keys on TV targets and stays untouched by this work.After #3817 merges, exactly one route per target class still holds: phone buttons via
hardware-button, TV keys viatv-remote. The PR is based on current main and carries none of #3817's ancestor diff, so #3817 stays TV-only and needs no rebase from this change.feat(hardware-button): press phone volume and power keys once (#3848)
feat(hardware-button): press phone volume and power keys once (#3848)
feat(hardware-button): press phone volume and power keys once (#3859)
feat(hardware-button): add Android mute, sleep, wake and media keys (#…