Skip to content

feat(hardware-button): press phone volume and power buttons on Android and iOS #3848

Closed

Description

@thymikee

Purpose

Agents can't press a phone's volume or power button. Today they fall back to raw adb shell input keyevent on Android and have no route at all on iOS. PR #3817 was going to add a hardware-key command, but it is being cut down to TV-only tv-remote keys, so phones are left out. This issue adds a small, honest phone vocabulary for those buttons. It is a separate decision from the TV remote keys.

iOS can press some hardware buttons; our implementation just has no route for them yet. XCUIDevice.Button has volumeUp and volumeDown, and Appium XCUITest mobile: pressButton accepts volumeup and volumedown on real devices. iOS does not expose a public power or lock button press.

API

agent-device hardware-button volume-up
agent-device hardware-button volume-down
agent-device hardware-button power
await client.command.hardwareButton({ button: 'volume-down' });
// => { action: 'hardware-button', button: 'volume-down', message: 'Pressed hardware button volume-down' }
  • One positional argument, <volume-up|volume-down|power>. No aliases. An unknown name returns INVALID_ARGS before any device call.
  • No hold verb or duration flag in the MVP (see "Long press" below).
  • No --settle and no post-action observation. The command must not activate or re-foreground the session app. This uses action-button's trait set, not press's.
  • The result reports that the press was dispatched. It does not report a resulting volume level or screen state.

Ownership: extend the system-button family, not a new dispatcher

packages/contracts/src/system-button-runtime.ts already owns no-argument device presses (home, appSwitcher, actionButton): one fact cell per button, a family unsupported default, and bindSystemButton. This change adds volumeUp, volumeDown and power to SYSTEM_BUTTONS and SYSTEM_BUTTON_LABELS, and adds optional Interactor members volumeUp?(), volumeDown?(), power?(). The positional picks the cell, the way keyboard <status|dismiss|enter> picks a keyboard-family cell.

  • Every owner that already calls systemButtonRuntimeOperationFacts gets the new cells refused through its existing unsupported denial. Only the owners in the matrix below name a cell as available.
  • No SystemControl union, no key table shared with tv-remote, no android-keycode:N escape hatch, and no case 'hardware-button' dispatcher that bypasses the family binder.
  • home, app-switcher and action-button keep their existing commands. hardware-button does not accept home, app-switcher or action, so each button has exactly one public route. Folding action-button into hardware-button would be a separate migration decision.
  • On TV targets (Android TV, tvOS), hardware-button refuses with a hint that points to tv-remote. After feat(tv-remote): add hardware and media buttons for TV targets #3817, tv-remote owns TV volume and power keys.

Support matrix (MVP)

Every known platform, device-kind, and provider refusal happens at fact admission, before any adb, runner, or provider dispatch, and uses an existing RuntimeOperationUnavailability reason. A physical device hardware-presence check may run in the runner, but it must refuse before pressing an absent button.

Owner / backend Device kind volume-up / volume-down power Mechanism / refusal
Local Android (ADB) emulator, physical phone/tablet available available input keyevent 24 / 25 / 26 (KEYCODE_VOLUME_UP/DOWN, KEYCODE_POWER), same route as home (keyevent 3)
Local Android (ADB) Android TV target refused refused unsupported-platform-leaf, hint: use tv-remote
Limrun Android cloud emulator available available same input keyevent route Limrun Android home/app-switcher use
Appium Android (UiAutomator2) phone/tablet session only available when the provider declares the capability same mobile: pressKey { keycode }, the call keyboard Enter already makes. Undeclared capability: owner-capability-missing with the provider note
Local Apple runner physical iPhone/iPad available refused XCUIDevice.shared.press(.volumeUp/.volumeDown), guarded by hasHardwareButton(_:) (iOS 16+). No public XCUIDevice lock/power button: unsupported-platform-leaf
Local Apple runner iOS/iPadOS Simulator refused refused unsupported-device-kind. The SDK header marks XCUIDeviceButtonVolumeUp/Down XCUI_SIMULATOR_UNAVAILABLE, so the runner code must be inside #if !targetEnvironment(simulator)
Local Apple runner tvOS, macOS, watchOS, visionOS refused refused unsupported-platform-leaf (tvOS hint: tv-remote)
Appium iOS (XCUITest) real device available when declared refused mobile: pressButton { name: 'volumeup' | 'volumedown' } (real devices only, per driver docs). mobile: lock sets lock state; it is not a button press, so it is not mapped to power
Appium iOS (XCUITest) simulator refused refused unsupported-device-kind. If the provider does not report the kind, the cell refuses rather than guessing
Limrun iOS cloud simulator refused in MVP refused in MVP SDK performActions types list buttonDown/buttonUp with volumeUp, volumeDown, lock and side, but there is no live evidence. See Future
Web, Linux, macOS desktop, HarmonyOS, Vega — refused refused each owner's existing family denial

Within an admitted Apple leaf, if hasHardwareButton returns false at press time, the runner returns UNSUPPORTED_OPERATION with a hardware hint. This is the same pattern as actionButton in RunnerTests+CommandExecution.swift. It is never error-text matching.

Press events, not volume settings

hardware-button volume-down sends one button event. What the OS does with it (which stream changes, whether a HUD shows, whether the foreground app intercepts it) is device behavior, and the command does not claim any of it. Setting an absolute volume (Android cmd media_session volume --set, iOS: no public API) is a different feature. If it is wanted, it belongs in the settings catalog (packages/contracts/src/settings-catalog.ts, which has no volume entry today), not here.

power toggles the screen on Android. After a press that turns the screen off, later commands may hit a locked screen. The help text says so. It does not add an automatic wake.

Long press

Long press is not in the MVP. The backends do not agree:

  • Android ADB: input keyevent --longpress <code>, already used by tv-remote longpress. Appium Android: mobile: pressKey { isLongPress: true }. Both use a fixed system long-press duration, not a caller-chosen duration.
  • Local Apple and Appium iOS: XCUIDevice.press(_:) has no forDuration: overload. Appium's pressButton duration applies only to tvOS.
  • Limrun iOS: buttonDown + wait + buttonUp could hold for a duration, but this is unverified.

If long press is added later, represent it as a press shape, without a separate runtime-operation fact cell (ADR 0019 §2). Unsupported shapes must refuse before the button event is sent. Its API and validation are a separate decision.

Out of scope / future (each its own decision)

Candidate Evidence Why not MVP
sleep, wake Android KEYCODE_SLEEP 223 / KEYCODE_WAKEUP 224 (API 20), idempotent Android-only. Cheap to add as two more family cells once someone needs a lock/wake flow
camera Android KEYCODE_CAMERA 27. Apple XCUIDevice.Button.camera (iOS, Xcode 16.3+). Appium camera (Xcode 16+, iOS 16+, real devices only) #2699 left Camera Control out on purpose (press only, no slide/light-press). Needs that decision, plus a physical iPhone 16 for evidence
mute Android KEYCODE_VOLUME_MUTE 164 Android-only. iPhones have no mute key
iOS power / lock No public XCUIDevice button. Appium mobile: lock / unlock / isLocked is lock state. Limrun lock/side buttons (types only) A lock-state command would be a different contract from a press
Limrun iOS volume SDK types only Admit once a live Limrun run shows a volume change
Siri XCUISiriService.activate(voiceRecognitionText:) is a text query API, not a button Different feature. No promise here
Silent / ring switch No XCUITest, Appium or Limrun API found Unsupported. No promise
Absolute volume — settings catalog decision, see above

Completion conditions

  1. agent-device hardware-button volume-down on a local Android emulator returns { action: 'hardware-button', button: 'volume-down', message }, and adb shell cmd media_session volume --stream 3 --get reads one step lower than before. volume-up raises it back.
  2. agent-device hardware-button power on a local Android emulator changes adb shell dumpsys power mWakefulness from Awake to Asleep. A second press restores Awake.
  3. On a physical iPhone over the local runner, hardware-button volume-up succeeds. The evidence is the system volume HUD in a screenshot taken right after, or a changed AVAudioSession.outputVolume read in a fixture app. The session app is not activated or re-foregrounded.
  4. hardware-button volume-up on an iOS Simulator, and hardware-button power on any Apple leaf, refuse at admission with a typed reason and hint. No runner request is issued, and a test asserts this. The runner still builds for the Simulator destination.
  5. Appium Android: a unit test asserts mobile: pressKey with { keycode: 24 | 25 | 26 }. Appium iOS: a unit test asserts mobile: pressButton with { name: 'volumeup' | 'volumedown' }, and that power is refused. A live run on any Appium Android session, and on a real-device Appium iOS session if available, is posted on the PR. If none is possible, the PR says so.
  6. hardware-button volume-up on Android TV and tvOS refuses with a hint to tv-remote. On web, Linux, macOS, HarmonyOS, Vega and Limrun iOS, it refuses at admission.
  7. hardware-button sideways returns INVALID_ARGS before any adb, runner or provider call.
  8. A provider scenario under test/integration/provider-scenarios/ covers the Android ADB argv and the response shape (integration-progress stays complete). The MCP tool and client.command.hardwareButton exist, and the MCP output schema is derived from CommandResultMap.
  9. CLI help, website/docs/docs/commands.md, and every refusal hint name the same backends the facts admit.
  10. pnpm check:affected --run passes, including the registry, owner-files and generic-route completeness gates.

Dependencies

Remaining uncertainties

  • Whether a volume XCUIDevice.press on a physical iPhone shows the HUD or changes outputVolume while an XCUITest session is attached. Not live-verified. Appium documents it as working on real devices.
  • Whether a cloud Appium iOS session reports real-device vs simulator in a way the fact can read before dispatch. If not, the iOS cell refuses unless the provider declares real devices.

Sources

Activity

  1. thymikee commented on Oct 20, 2026

    @thymikee
    MemberAuthor

    Decision: Dependency on #3817: rescoped to coordination; the phone API is implemented on current main.

    Verified against origin/main at 421dd7e8e7:

    • main never had the unmerged hardware-key command. git grep "'hardware-key'" over the command surface (packages/command-registry, src/commands) finds nothing on main.
    • main's tv-remote is already TV-only: it owns the DPAD/select/menu/home/back keys for TV targets and their runtimes, and no phone owner admits the cell.

    So the phone hardware-button command can land without #3817. The two surfaces stay disjoint by construction:

    • hardware-button accepts only volume-up | volume-down | power and refuses Android TV and tvOS sessions at fact admission with a hint naming tv-remote.
    • tv-remote keeps TV keys on TV targets and stays untouched by this work.

    After #3817 merges, exactly one route per target class still holds: phone buttons via hardware-button, TV keys via tv-remote. The PR is based on current main and carries none of #3817's ancestor diff, so #3817 stays TV-only and needs no rebase from this change.

  2. added
    enhancementNew feature or request
    and removed on Oct 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions