display(iOS): fix touch gestures and add drag to scroll mode - #7893
Merged
Merged
Conversation
The long press recognizer required the tap recognizers to fail first. A tap only fails once the finger has been held for well over a second, so a normal long press was delivered as a tap when the finger lifted: the "Right Click" action produced a left click (or nothing in touch mode) and "Click & Hold" never pressed the button. Reverse the dependency so the tap waits for the long press, which fails as soon as a short touch ends and so adds no delay to taps. When the long press action is disabled the recognizer is not allowed to begin, otherwise holding before lifting would no longer click. Reworked from #7697. Assisted-by: Claude:claude-fable-5-1
Every pan and the pinch required the three finger keyboard swipes to fail first, and a swipe recognizer only fails once the touch has moved far enough to be judged or after about half a second. Moving the cursor with one finger, scrolling with two or pinching therefore did nothing for a quarter to half a second, and a short or slow two finger scroll was mostly lost. A swipe can only be confused with a pan that uses the same number of touches, so only the three finger pan still waits for the keyboard swipes. The pinch needs a guard instead: before iOS 18 it begins with two of three fingers and would take the touches from the swipe, so it is not allowed to begin while three fingers are down. That also stops three fingers from zooming the display on those versions. The two finger pan still waits for the two finger swipe when the two are bound to different actions. When both scroll the mouse wheel there is nothing to tell apart: the pan scrolls right away and a swipe adds its wheel tick on top, where before any steady two finger drag was taken for a swipe and scrolled a single tick. Because the two now scroll together, the swipe follows the "Invert Scroll" setting like the pan. Reworked from #7697. Assisted-by: Claude:claude-fable-5-1
A touch that starts at the top or bottom edge of the screen is held back by the system in case it turns into a system gesture. A drag from the top edge of the VM display pulled down Notification Center and the guest never saw the touch at all. Ask the system to defer its edge gestures while the VM display is up, next to where the home indicator is already hidden, so the first swipe goes to the guest and only a second one is taken by the system. This uses the SwiftUI modifier, which needs iOS 16. Overriding the view controller preference instead needs the hosting controller patched to forward it, and was ignored on iOS 17. On iOS 18 and later a swipe up from the bottom edge still goes home, because the system ignores the deferral there while the home indicator is hidden. Touches at the bottom edge do reach the guest immediately. Reworked from #7697. Assisted-by: Claude:claude-fable-5-1
Touch gestures could only produce a left drag and a right click, so a guest that needs the middle button (paste or pan in many Linux and CAD programs) or a right button drag could not be used without a mouse. Add "Middle Click" to the long press and two finger tap actions, and "Right Click & Hold" and "Middle Click & Hold" to the two and three finger pan actions. These are new values for existing settings and the defaults are unchanged. Reworked from #7697. Assisted-by: Claude:claude-fable-5-1
In the touch modes the left button goes down the moment a finger lands, because the touch itself is the drag. Every other gesture starts with a finger landing too, so a two finger tap, scroll or pinch first clicked and dragged whatever was under the first finger until the gesture was recognized, and a long press held the left button for half a second before its right click. Add "Touch mode (drag to scroll)" as a third touch mode that behaves like a touch screen and never presses a button on touch down. A tap is a left click, a long press runs the long press action, and dragging one finger scrolls the mouse wheel so that the content follows the finger. Since a touch no longer drags, a long press that moves turns into a left button drag so windows and selections can still be moved with one finger. A ring is shown when the long press is recognized because nothing in the guest reacts until the finger lifts. A finger never lands on the same spot twice, and a guest only counts two clicks on the same spot as a double click, so a second tap that follows quickly and close by clicks where the first one did. The existing touch modes are unchanged. Reworked from #7697. Assisted-by: Claude:claude-fable-5-1
A gesture that holds a mouse button only released it when the gesture ended. When the system takes over the touch instead, for example a long press drag that starts at the bottom edge and turns into the swipe to go home, the gesture is cancelled and the guest was left with the button held down until the next click. Reworked from #7697. Assisted-by: Claude:claude-fable-5-1
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a rewrite of #7697 by @ifilipis. That PR identified real problems with touch input on iOS, but it arrived as one 900-line commit that also changed defaults and switched existing gestures off for current users. This branch keeps the fixes, splits them into one commit per issue, and leaves the existing touch modes and all defaults as they are.
What changes
defersSystemGestures(on:)(iOS 16+) next to the existingpersistentSystemOverlays(.hidden). On iOS 18 and later a swipe up from the bottom edge still goes home, because the system ignores the deferral while the home indicator is hidden.Claims in #7697
main.UISwipeGestureRecognizer; pan and swipe run together when both scroll. The PR's velocity based detection was left out.UIViewController; the patched getter was ignored on iOS 17.Platform/iOS.Resolves #4738
Testing
Testing: Tested by a human on iPhone 16 Pro, iOS 26.7. The author acknowledges that this change has been tested and/or reviewed by a human in accordance with UTM's AI contribution guidelines.
The device test covered 22 steps over four settings configurations: all defaults in Drag cursor mode, the new mode with the new actions, the old touch mode with pan and swipe bound to different actions, and long press disabled. It included the cases that cannot be simulated: three finger swipes and two finger gestures with the fingers landing at different times. One step was skipped (Apple Pencil with palm rejection, and an external pointer, while the new mode is selected). The first device pass found that double tap did not work in the new mode; that is fixed in change 5 and was retested.
Automated testing, in addition: every change was run before and after on iPhone simulators with iOS 17.0, 18.0, 26.0 and 27.0, against a Windows XP guest. A UI test bundle injected synthetic multi finger touches, and a local logging shim (not part of this PR) recorded the mouse events sent to the guest and the recognizer state changes. iOS 18, 26 and 27 agree on every check. iOS 17 differs, which is where the pinch guard in change 2 and the SwiftUI modifier in change 3 came from. Synthetic touches land all fingers at the same instant, so staggered landing was only tested on the device.
Not tested: iOS 15 and 16 (on iOS 15 change 3 has no effect, the modifier needs iOS 16), iPad, and visionOS (the modifier is unavailable there and compiled out; the visionOS target was not built).
🤖 Generated with Claude Code