Two-thumb: use the native decoder's second pointer track (research + pointer-id normalisation) - #136
Closed
AsafMah wants to merge 2 commits into
Closed
Two-thumb: use the native decoder's second pointer track (research + pointer-id normalisation)#136AsafMah wants to merge 2 commits into
AsafMah wants to merge 2 commits into
Conversation
…ck harness (#135) Research outcome for "make two-thumb typing work WITH the native gesture decoder". Findings in docs/TWO_THUMB_TEMPORAL_ALIGNMENT.md. The AOSP decoder models TWO per-pointer tracks (MAX_POINTER_COUNT_G == 2) and can spell a word by alternating between them. We have never used the second one: InputPointers.appendAll hardcodes pointer id 0, so every merged multi-part trail lands in track 0 as one long glide with a synthetic connector. Measured against the real AOSP ProximityInfoState on the host (new harness): - track membership is decided purely by pointer id; shifting or overlapping timestamps moves zero points between tracks, which falsifies "temporally shift the strokes so the library sees them as simultaneous" as a mechanism; - deliberate overlap drives speed rates negative (impossible from real input), i.e. it actively corrupts the features fed to the closed weighting policy; - global time monotonicity IS required: refreshSpeedRates walks raw indices across the pointer boundary, so per-stroke clocks that restart go negative. Bug fixed: if no point carries id 0, Suggest::initializeSearch early-returns and the gesture produces zero suggestions. Reachable in ordinary two-thumb use (thumb A down, thumb B down, thumb A lifts, thumb B swipes on with id 1). Any id >= 2 reaches no track at all. PointerIdNormalizer renumbers raw MotionEvent pointer ids in first-seen order at the BatchInputArbiter seam; it is the identity mapping for normal input and only repairs the broken cases. - PointerIdNormalizer + 7 JVM unit tests - two_pointer_track_test.cpp: tunable pointer-id / time-policy / tap-promotion knobs and a printed sweep table; runs in CI with the existing native suite - native host suite 83/83; JVM suite unchanged vs baseline (only the 4 documented Windows-only ParserTest failures) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
AsafMah
added a commit
that referenced
this pull request
Aug 20, 2026
Section 5 was still headed "THE OPEN TASK - publish signed v0.2.0", which is no longer true: v0.2.0 was published on 2026-08-20 with all four signed APKs and is marked latest. The runner outage that blocked it resolved on its own; Release run 31128748928 succeeded and produced the draft. Reframes section 5 from a blocker into the verified release procedure, since the recipe itself is still what the next release should follow, and keeps the outage signature so it is recognised rather than re-debugged. Also refreshes the TL;DR table (v0.2.0 published, current dev head, the four open PRs including #134, #136 and #137) and replaces section 12's "publish v0.2.0" item with the work that is actually outstanding: device verification of #134 and #137, re-pointing LeanType-check-upstream-main to v4.1.2 to re-check the two guarded upstream defects, reporting the emoji accelerated-delete bug upstream, and deciding the fate of the unfinished worktrees whose commits exist on no remote. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: a88bd77f-d993-44c5-9efc-c7124f0d825e
…135) Acts on the research in docs/TWO_THUMB_TEMPORAL_ALIGNMENT.md: the native decoder keeps two per-pointer tracks and we have only ever used one, because InputPointers.appendAll hardcoded pointer id 0 for the whole merged trail. StrokeAligner is the merge seam WordComposer.setBatchInputPointers now delegates to. Two modes: CONNECTOR (default) everything on track 0 — the historical behaviour, reproduced byte-for-byte including the 25/60 ms base re-timing. DUAL_POINTER prior fragments on track 0, the in-flight stroke on track 1, so the decoder's own two-pointer search runs and there is no synthetic connector to hallucinate letters across. The class enforces the four constraints the research established: track 0 is always non-empty (an empty track 0 makes Suggest::initializeSearch return zero suggestions), only ids 0 and 1 are emitted, timestamps stay globally monotonic, and a given raw index never changes track mid-gesture. IdealPrefixTrailBuilder is ported from the stranded #99/B7b branch and is now reachable at runtime instead of via a side-by-side build type. It traces the composing prefix through key centres and promotes a one-letter prefix to a four-point micro-stroke — the "turn taps into small swipes" half of the original hypothesis, which the research found sound. It iterates code points and returns null when any letter is not on the keyboard, so a hole in the synthetic path can never replace the raw trail. Four prefs on Two-Thumb Typing → Recognition, all defaulting to today's behaviour; the mode picker is hidden when no native gesture lib is present because the Java fallback ignores pointer ids entirely. Review fixes folded in: - CONNECTOR pins the timing knobs to 25/60 so tuning them in dual mode and switching back cannot silently change "one joined trail". - a current stroke that is ITSELF multi-pointer keeps its own ids instead of being flattened onto track 1, preserving genuine simultaneous two-thumb structure (new InputPointers.appendAllPreservingIds). - IdealPrefixTrailBuilder no longer silently drops unmappable letters. Tests: 31 new (StrokeAligner 19, IdealPrefixTrailBuilder 10, settings 2), including a proof that DUAL_POINTER changes pointer ids only — never geometry or timing — so the Java fallback engine cannot regress. Full JVM suite 356 tests with only the 4 documented Windows-only ParserTest failures; native host suite 83/83; assembleOfflineliteDebugNoMinify builds. Not yet verified on device: whether two tracks SCORE better than one merged trail is decided by the closed weighting policy and needs the A/B in §6a. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This was referenced Aug 20, 2026
Owner
Author
|
Superseded by #141, which contains every commit from this branch plus the upstream v4.1.2 merge and the side-by-side Both APKs are built and installed on the device. Closing in favour of #141. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Research + implementation for "make two-thumb typing work with the native gesture decoder, via temporal alignment". Full write-up:
docs/TWO_THUMB_TEMPORAL_ALIGNMENT.md. Fixes #135.Verdict on the hypothesis
Directionally right, mechanically wrong. The goal is achievable and natively supported; the knob is
pointerIds[], not the clock. Time still matters, but only to keep the concatenated array globally monotonic.The measurement
A new harness drives the real AOSP
ProximityInfoStateon the host — the same preprocessing compiled intolibjni_latinimegoogle.so— so this is measured, not inferred:InputPointers.appendAllhardcoded pointer id 0, so both fragments landed in track 0 as one glide with a connector jump.minSpeedis impossible from real input (speed = length / duration) ⇒ garbage features. Overlap is the worst configuration tested.What ships
Everything defaults to today's behaviour — nothing changes until a knob moves.
1. Bug fix (on by default). If no point carries id 0,
Suggest::initializeSearchearly-returns ⇒ zero suggestions. Reachable in ordinary two-thumb use: thumb A down (id 0), thumb B down (id 1), thumb A lifts, thumb B swipes on still carrying id 1. Ids ≥ 2 reach no track at all.PointerIdNormalizerrenumbers raw MotionEvent ids in first-seen order at theBatchInputArbiterseam — identity mapping for normal input, repairs only the broken cases.2.
StrokeAligner— the multi-part merge seam, extracted fromWordComposer.setBatchInputPointers.CONNECTOR(default) reproduces the old behaviour byte-for-byte;DUAL_POINTERputs prior fragments on track 0 and the in-flight stroke on track 1. It enforces all four constraints the research established (non-empty track 0, ids ∈ {0,1}, monotonic time, stable ids across incremental recognition).3.
IdealPrefixTrailBuilder— ported from the stranded #99/B7b branch, now runtime-toggleable instead of needing a side-by-side build type. Traces the prefix through key centres; a one-letter prefix becomes a 4-point micro-stroke.Prior art found in-repo
Epic #97 (B7 fake-track synthesis) is open; #98 and #99 are built but stranded on
b7a-prefix-aware-stripping, never merged todev. #100 is this hypothesis, specified months ago and never built. All of them treat the problem as a single track — "fake-track" is ironic, since the decoder has a real second track sitting unused. #97 also classifies #29 (per-thumb attribution) as a "different problem"; that separation looks wrong — it's the same lever.Review fixes folded in
An adversarial cross-model review found three real issues, all fixed:
CONNECTORnow pins the timing knobs to 25/60, so tuning them in dual mode and switching back can't silently change "one joined trail".IdealPrefixTrailBuilderiterates code points and returns null on any unmappable letter, rather than silently producing a holed path.Validation
native-tests.yml.ParserTestfailures — verified against a stashed baseline, by failing test name.assembleOfflineliteDebugNoMinifybuilds (validates Compose settings + resources).DUAL_POINTERchanges pointer ids only — never geometry or timing — so the Java fallback engine (which ignores ids) provably cannot regress. The mode picker is hidden when no native lib is present.InputLogicTestflakes were seen during development; both were non-reproducible (3/3 clean on this branch and on baseline, different test each time, in looper/message-timing paths).Not verified
On-device. Whether two tracks score better than one merged trail is decided by the closed weighting policy — no host test can produce a decoded word, since this tree has no gesture suggest policy. §6a of the doc has the A/B to run before considering any default change. The pointer-id fix also touches the live gesture path and deserves a real two-thumb check.