Type, Then Correct: Intelligent Text Correction Techniques for Mobile Text Entry Using Neural Networks
Project Overview
We can rethink the text correction process on mobile phones. Instead of moving the cursor and delete the errors, how about we type the correction first? For this project, we present three novel text correction techniques to improve the correction process: Drag-n-Drop, Drag-n-Throw, and Magic Key. All of the techniques skip error-deletion and cursor-positioning procedures, and instead allow the user to type the correction first, and then apply that correction to a previously committed error. Specifically, Drag-n-Drop allows a user to drag a correction and drop it on the error position. Drag-n-Throw lets a user drag a correction from the keyboard suggestion list and “throw” it to the approximate area of the error text, with a neural network determining the most likely error in that area. Magic Key allows a user to type a correction and tap a designated key to highlight possible error candidates, which are also determined by a neural network. The user can navigate among these candidates by directionally dragging from atop the key, and can apply the correction by simply tapping the key.
2018.10 - 2019.4
ACE Lab, University of Washington
Mingrui Zhang, He Wen, Jacob O. Wobbrock
Designer, Mobile & Algorithm Developer
Video
Algorithm
To support Drag-n-Throw and Magic Key, we designed a Recurrent Neural Network (RNN) for error detection. When the raw text and the correction is fed into the network, it will output the corrected version of the text. The network utilizes attention mechanism and a seq-2-seq structure. When a correction action is triggered, the mobile will send the current text in the text editing area to the server, and the server then splits the text into small segments, and do error detection on each segment. The server then returns the best candidate to the mobile application.
Publication
Mingrui Ray Zhang, He Wen, and Jacob O. Wobbrock. 2019. Type, Then Correct: Intelligent Text Correction Techniques for Mobile Text Entry Using Neural Networks. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (UIST ’19). Association for Computing Machinery, New York, NY, USA, 843–855. [link]
