Skip to content

Two-thumb: use the native decoder's second pointer track (research + pointer-id normalisation) - #136

Closed
AsafMah wants to merge 2 commits into
devfrom
asafmah-two-thumb-temporal-alignment
Closed

Two-thumb: use the native decoder's second pointer track (research + pointer-id normalisation)#136
AsafMah wants to merge 2 commits into
devfrom
asafmah-two-thumb-temporal-alignment

Conversation

@AsafMah

@AsafMah AsafMah commented Aug 20, 2026

Copy link
Copy Markdown
Owner

Research + implementation for "make two-thumb typing work with the native gesture decoder, via temporal alignment". Full write-up: docs/TWO_THUMB_TEMPORAL_ALIGNMENT.md. Fixes #135.

Verdict on the hypothesis

Claim Verdict
The AOSP library can handle two simultaneous strokes Confirmed — it models exactly two pointer tracks and can spell a word by alternating between them
Temporally shift strokes so it accepts them as simultaneous Falsified — membership is decided purely by pointer id; shifting/overlapping moves zero points, and overlap actively corrupts the decoder's speed features
Promote taps into micro-swipes ✅ Sound — and already built on a stranded branch (#99); now ported and shipped opt-in
There's a better lever than the connector hack Pointer-id attribution — necessary to reach the second track

Directionally right, mechanically wrong. The goal is achievable and natively supported; the knob is pointerIds[], not the clock. Time still matters, but only to keep the concatenated array globally monotonic.

The measurement

A new harness drives the real AOSP ProximityInfoState on the host — the same preprocessing compiled into libjni_latinimegoogle.so — so this is measured, not inferred:

config                         | t0.used t0.n [raw range] minSpeed | t1.used t1.n [raw range] minSpeed
today  (all id 0, monotonic)   |   1     28 [  0.. 33]    0.491 |   0      0 [ -1.. -1]    0.000
split  (0/1, monotonic)        |   1     13 [  0.. 12]    0.884 |   1     15 [ 13.. 33]    0.521
split  (0/1, overlap 50%)      |   1     13 [  0.. 12]   -1.037 |   1     15 [ 13.. 33]  -1.247
all id 1                       |   0      0 [ -1.. -1]    0.000 |   1     28 [  0.. 33]    0.491
all id 2                       |   0      0 [ -1.. -1]    0.000 |   0      0 [ -1.. -1]    0.000
  • Today the second track is dead. InputPointers.appendAll hardcoded pointer id 0, so both fragments landed in track 0 as one glide with a connector jump.
  • Splitting ids partitions cleanly, with better speed rates than the merged trail.
  • Negative minSpeed is impossible from real input (speed = length / duration) ⇒ garbage features. Overlap is the worst configuration tested.

What ships

Everything defaults to today's behaviour — nothing changes until a knob moves.

1. Bug fix (on by default). If no point carries id 0, Suggest::initializeSearch early-returns ⇒ zero suggestions. Reachable in ordinary two-thumb use: thumb A down (id 0), thumb B down (id 1), thumb A lifts, thumb B swipes on still carrying id 1. Ids ≥ 2 reach no track at all. PointerIdNormalizer renumbers raw MotionEvent ids in first-seen order at the BatchInputArbiter seam — identity mapping for normal input, repairs only the broken cases.

2. StrokeAligner — the multi-part merge seam, extracted from WordComposer.setBatchInputPointers. CONNECTOR (default) reproduces the old behaviour byte-for-byte; DUAL_POINTER puts prior fragments on track 0 and the in-flight stroke on track 1. It enforces all four constraints the research established (non-empty track 0, ids ∈ {0,1}, monotonic time, stable ids across incremental recognition).

3. IdealPrefixTrailBuilder — ported from the stranded #99/B7b branch, now runtime-toggleable instead of needing a side-by-side build type. Traces the prefix through key centres; a one-letter prefix becomes a 4-point micro-stroke.

Two-Thumb Typing → Recognition Default
Joining word parts (one joined trail ↔ two separate thumb tracks) one joined trail
Redraw earlier word parts cleanly off
Word-part trail speed / Pause before the new part 25 ms / 60 ms

Prior art found in-repo

Epic #97 (B7 fake-track synthesis) is open; #98 and #99 are built but stranded on b7a-prefix-aware-stripping, never merged to dev. #100 is this hypothesis, specified months ago and never built. All of them treat the problem as a single track — "fake-track" is ironic, since the decoder has a real second track sitting unused. #97 also classifies #29 (per-thumb attribution) as a "different problem"; that separation looks wrong — it's the same lever.

Review fixes folded in

An adversarial cross-model review found three real issues, all fixed:

  • CONNECTOR now pins the timing knobs to 25/60, so tuning them in dual mode and switching back can't silently change "one joined trail".
  • A current stroke that is itself multi-pointer keeps its own ids instead of being flattened onto track 1 — preserving genuine simultaneous two-thumb structure.
  • IdealPrefixTrailBuilder iterates code points and returns null on any unmappable letter, rather than silently producing a holed path.

Validation

  • Native host suite 83/83 (+6 new); runs in CI via the existing native-tests.yml.
  • JVM suite 356 tests, only the 4 documented Windows-only ParserTest failures — verified against a stashed baseline, by failing test name.
  • assembleOfflineliteDebugNoMinify builds (validates Compose settings + resources).
  • A test proves DUAL_POINTER changes pointer ids only — never geometry or timing — so the Java fallback engine (which ignores ids) provably cannot regress. The mode picker is hidden when no native lib is present.
  • Two InputLogicTest flakes were seen during development; both were non-reproducible (3/3 clean on this branch and on baseline, different test each time, in looper/message-timing paths).

Not verified

On-device. Whether two tracks score better than one merged trail is decided by the closed weighting policy — no host test can produce a decoded word, since this tree has no gesture suggest policy. §6a of the doc has the A/B to run before considering any default change. The pointer-id fix also touches the live gesture path and deserves a real two-thumb check.

…ck harness (#135)

Research outcome for "make two-thumb typing work WITH the native gesture
decoder". Findings in docs/TWO_THUMB_TEMPORAL_ALIGNMENT.md.

The AOSP decoder models TWO per-pointer tracks (MAX_POINTER_COUNT_G == 2) and
can spell a word by alternating between them. We have never used the second
one: InputPointers.appendAll hardcodes pointer id 0, so every merged multi-part
trail lands in track 0 as one long glide with a synthetic connector.

Measured against the real AOSP ProximityInfoState on the host (new harness):
- track membership is decided purely by pointer id; shifting or overlapping
  timestamps moves zero points between tracks, which falsifies "temporally
  shift the strokes so the library sees them as simultaneous" as a mechanism;
- deliberate overlap drives speed rates negative (impossible from real input),
  i.e. it actively corrupts the features fed to the closed weighting policy;
- global time monotonicity IS required: refreshSpeedRates walks raw indices
  across the pointer boundary, so per-stroke clocks that restart go negative.

Bug fixed: if no point carries id 0, Suggest::initializeSearch early-returns
and the gesture produces zero suggestions. Reachable in ordinary two-thumb use
(thumb A down, thumb B down, thumb A lifts, thumb B swipes on with id 1). Any
id >= 2 reaches no track at all. PointerIdNormalizer renumbers raw MotionEvent
pointer ids in first-seen order at the BatchInputArbiter seam; it is the
identity mapping for normal input and only repairs the broken cases.

- PointerIdNormalizer + 7 JVM unit tests
- two_pointer_track_test.cpp: tunable pointer-id / time-policy / tap-promotion
  knobs and a printed sweep table; runs in CI with the existing native suite
- native host suite 83/83; JVM suite unchanged vs baseline (only the 4
  documented Windows-only ParserTest failures)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
AsafMah added a commit that referenced this pull request Aug 20, 2026
Section 5 was still headed "THE OPEN TASK - publish signed v0.2.0", which
is no longer true: v0.2.0 was published on 2026-08-20 with all four
signed APKs and is marked latest. The runner outage that blocked it
resolved on its own; Release run 31128748928 succeeded and produced the
draft.

Reframes section 5 from a blocker into the verified release procedure,
since the recipe itself is still what the next release should follow, and
keeps the outage signature so it is recognised rather than re-debugged.

Also refreshes the TL;DR table (v0.2.0 published, current dev head, the
four open PRs including #134, #136 and #137) and replaces section 12's
"publish v0.2.0" item with the work that is actually outstanding: device
verification of #134 and #137, re-pointing LeanType-check-upstream-main
to v4.1.2 to re-check the two guarded upstream defects, reporting the
emoji accelerated-delete bug upstream, and deciding the fate of the
unfinished worktrees whose commits exist on no remote.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a88bd77f-d993-44c5-9efc-c7124f0d825e
…135)

Acts on the research in docs/TWO_THUMB_TEMPORAL_ALIGNMENT.md: the native
decoder keeps two per-pointer tracks and we have only ever used one, because
InputPointers.appendAll hardcoded pointer id 0 for the whole merged trail.

StrokeAligner is the merge seam WordComposer.setBatchInputPointers now
delegates to. Two modes:

  CONNECTOR    (default) everything on track 0 — the historical behaviour,
               reproduced byte-for-byte including the 25/60 ms base re-timing.
  DUAL_POINTER prior fragments on track 0, the in-flight stroke on track 1, so
               the decoder's own two-pointer search runs and there is no
               synthetic connector to hallucinate letters across.

The class enforces the four constraints the research established: track 0 is
always non-empty (an empty track 0 makes Suggest::initializeSearch return zero
suggestions), only ids 0 and 1 are emitted, timestamps stay globally monotonic,
and a given raw index never changes track mid-gesture.

IdealPrefixTrailBuilder is ported from the stranded #99/B7b branch and is now
reachable at runtime instead of via a side-by-side build type. It traces the
composing prefix through key centres and promotes a one-letter prefix to a
four-point micro-stroke — the "turn taps into small swipes" half of the
original hypothesis, which the research found sound. It iterates code points
and returns null when any letter is not on the keyboard, so a hole in the
synthetic path can never replace the raw trail.

Four prefs on Two-Thumb Typing → Recognition, all defaulting to today's
behaviour; the mode picker is hidden when no native gesture lib is present
because the Java fallback ignores pointer ids entirely.

Review fixes folded in:
- CONNECTOR pins the timing knobs to 25/60 so tuning them in dual mode and
  switching back cannot silently change "one joined trail".
- a current stroke that is ITSELF multi-pointer keeps its own ids instead of
  being flattened onto track 1, preserving genuine simultaneous two-thumb
  structure (new InputPointers.appendAllPreservingIds).
- IdealPrefixTrailBuilder no longer silently drops unmappable letters.

Tests: 31 new (StrokeAligner 19, IdealPrefixTrailBuilder 10, settings 2),
including a proof that DUAL_POINTER changes pointer ids only — never geometry
or timing — so the Java fallback engine cannot regress. Full JVM suite 356
tests with only the 4 documented Windows-only ParserTest failures; native host
suite 83/83; assembleOfflineliteDebugNoMinify builds.

Not yet verified on device: whether two tracks SCORE better than one merged
trail is decided by the closed weighting policy and needs the A/B in §6a.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@AsafMah

AsafMah commented Aug 20, 2026

Copy link
Copy Markdown
Owner Author

Superseded by #141, which contains every commit from this branch plus the upstream v4.1.2 merge and the side-by-side experimental build type (com.asafmah.leantypedual.exp, shown as LeanTypeDual EXP in the input-method picker) so it can be A/B'd against a working daily driver.

Both APKs are built and installed on the device. Closing in favour of #141.

@AsafMah AsafMah closed this Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants