Compose Keyboard Swipe Neural Encoder
Swipe typing on the phone, no inference library
- Year
- 2026
- Kind
- Models
- Status
- In use
89.1% top-1, up from 73.5%
A 634K-parameter swipe-typing model for my Android keyboard, trained on 896K real swipes and run in plain Kotlin with no inference library. This site's home page runs the same weights in your browser.
The keyboard's old decoder matched swipe shapes, which throws away timing, so words like put and putt looked identical. This temporal convolutional network reads the timed trace, emits letters through CTC, and reads the keyboard through a cosine basis, so the same weights fit any layout. A beam search over a trie of the word list turns its output into words. The forward pass is hand-written Kotlin, and this site's home page runs the same weights in the browser.
The problem
Compose Keyboard, my open-source Android keyboard, first decoded swipes by shape: resample the path by distance and match it against each word's ideal path. That throws away timing, so put and putt trace the same shape, and it reaches 73.5% top-1. This model also reads timing, and runs on the phone with no inference library. It follows the FUTO Swipe paper (Miller and Kostarevas, 2026); I reimplemented and trained it, refit its decoder and ported it to Kotlin.
How it works
- 01
Resample by time, not distance
Training uses about 896,000 real swipes from the FUTO Swipe Dataset. Each trace goes through a 60 Hz pass, then down to 64 points evenly spaced in time. Time spacing keeps the dwell on a doubled letter, the only signal that tells putt from put.
- 02
Eight features into a temporal convolution
A 7-tap Savitzky-Golay filter turns the points into 8 channels, position, velocity, acceleration, speed and curvature, without amplifying touch noise. Five ConvNeXt-style blocks follow, with depthwise convolutions dilated by 1, 2, 3, 5 and 8 steps, so together they see the whole gesture. That is 634K parameters in all.
- 03
Read the keyboard through a cosine basis
The network never scores a key directly. At each of 32 output steps it emits an 8x8 spatial pattern as 64 cosine coefficients, plus a gate for how strongly a letter is meant there. Sampling the pattern at each key's centre gives letter scores, so a new layout or size needs a new basis matrix, not a new model.
- 04
Train with CTC under co-augmentation
The loss is CTC (connectionist temporal classification), which aligns the 32 steps to a shorter word through a blank symbol. Trace and keyboard are rotated, flipped, scaled and sheared together, so a model that works upside down cannot have memorised QWERTY. Training took 82 minutes on one GPU.
- 05
Decode against a lexicon trie
A CTC beam search walks a trie of the word list, so only real words survive. A prefix may repeat its last letter only from paths ending in a blank, which the network emits when the finger lingers: that is how putt beats put. The scoring constants were fit on the dev split by Bayesian optimization.
- 06
Run it by hand on the phone
SwipeNet.kt transcribes the PyTorch model into Kotlin line by line: about 35 million multiply-adds per gesture over 2.5 MB of weights, with BatchNorm folded into the convolutions and a parity test against PyTorch. This site's home page runs the same weights through a TypeScript port in a Web Worker.
Results
- 89.07%
- top-1 on 47,552 FUTO test swipes with the app's 150K-word dictionary (92.00% with every test word in it; geometric decoder: 73.50%)
- 92.09%
- top-1 with the test vocabulary added to the lexicon, which isolates spatial decoding
- 97.69%
- top-3 in that same setting
Decisions
Single-threaded on purpose
Threading was tried and reverted: on big.LITTLE phones, work landing on efficiency cores made the join 1.5 to 5x slower. The effort went into the pointwise projections, about 90% of the arithmetic, which accumulate 8 outputs per pass to fill ARM's issue slots.
The cheap decoder draws the preview
The network costs about 75 ms per gesture on a mid-range phone, the geometric decoder about 0.1 ms. So the live preview, refreshed every 55 ms, uses the geometric decoder, and the committed word comes from the network.
Beam width 50, not 100
In the paper's sweep, accuracy saturates by a beam of 100. The app uses 50: half the search for about half a point of top-1.
What I learned
- Training and the app must preprocess identically, and a mismatch fails silently: no crash, just worse words.
- The paper's scoring constants were fit to a 162K-word lexicon and didn't transfer to this app's, so they were refit.