- My Ph.D. Annual Review Presentation Slides 1st year 2nd year
- My Thesis Proposal Presentation on Youtube /on Bilibili /Slides
Intelligent Text Entry Application
The ACM CHI Conference on Human Factors in Computing Systems (CHI), 2021 Video My presentation
PhraseFlow is a phrase-level input keyboard that is able to correct previous text based on the subsequently input sequences. For example, if the user types "I love in Seattle", the keyboard will correct "love" to "live" after it sees "in Seattle". The project evaluated the usability of PhraseFlow and attempted to design a pracitcally usable phrase-level keyboard.
The ACM Symposium on User Interface Software and Technology (UIST), 2020 Video Wenzhe's Presentation
JustCorrect is a post-correction technique for smartphones, sharing the same genre with Type, Then Correct. JustCorrect utilizes the word embeddings and language models to detect the error and display correction options automatically.
Proceedings of Graphics Interface (GI), 2020 Video My presentation
We proposed a set of on-keyboard text editing gestures for the mobile keyboard, similar to the desktop keyboard shotcuts, including ring/letter/swipe gestures to facilitate fast editing tasks such as cursor-moving/copy/paste/cut/undo. Gedit provides one- and two-handed operation modes, and is also compatible with the gesture typing input.
The ACM Symposium on User Interface Software and Technology (UIST), 2019 My Presentation Video Project Page
Instead of normal touch+cursor based correction process, why cannot we rethink of the correction interaction? In this paper, we present three novel interactions that allow the user to type the correction first, then apply it to the error place. Furthermore, we applied deep learning technology to enable automatic error detection for the interaction. Our correction RNN model
The ACM Symposium on User Interface Software and Technology (UIST), 2015 Video
A novel air-typing method, Leapmotion tracking fingers, improved Bayes prediction model with application developed. Users reached the speed of 29.2 WPM on average.
Text Entry Evaluation
The ACM Symposium on User Interface Software and Technology (UIST), 2019 Video
In this work, we present a new underlying model that supersedes the input stream model for general-purpose method-independent character-level text entry evaluation. Specifically, we present an approach that replaces the input stream with transcription sequences, or “T-sequences” for short. In brief, T-sequences are snapshots of the entire transcribed string after each text-changing action is taken by the user. Every pair of successive snapshots are then analyzed to compute character-level text entry metrics. TextTest++ platform
The ACM CHI Conference on Human Factors in Computing Systems (CHI), 2019 My Presentation Related Blog Post
We define the text entry Throughput as a performance metric combining the speed and accuracy. Throughput is derived from the transmission ratio in the information theory. Unlike other metrics, throughput is less affected by speed-accuracy tradeoffs, thus it enables cross-device, cross-publication comparison. Throughput calculation library
Communication with Emojis 🧐
The ACM CHI Conference on Human Factors in Computing Systems (CHI), 2021 My presentation Related Blog Post (in Chinese)
A speech-based emoji input system, designed for blind and low vision users. Use natural language style emoji query and context sensitive emoji suggestions based on the spoken content. Voicemoji speeds up the emoji entry process by 91% than the iOS keyboard.
iConference, 2021 My presentation
We compared how the lexical based and semantic based emoji suggestion mechanisms affected the online chatting experience through an in-lab study and a field deployment. The results showed that the suggestion system of emojis did not influence the chatting experience, and users enjoy using both suggestion systems for different reasons.
Voice User Interface
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2020
Through a one-month deployment study, we investigated how families learnt new functionalities of smart speakers, include 1) which features families are aware of and engage with, and 2) how families explore, discover, and learn to use the Echo Dot. Drawing from diffusion of innovation theory, we describe how a home-based voice interface might be positioned as a near-peer to the user and help them discover new functionalities.
The ACM CHI Conference on Human Factors in Computing Systems (CHI), 2019 Erin's Presentation Related Blog Post
We investigated different types of communication breakdowns and the repairing strategies between the conversation of family members and Alexa. Our findings indicates that improving technology’s ability to identify the communication partners and to provide specific clarification responses will ultimately improve the conversational interaction experience.
The ACM CHI Conference on Human Factors in Computing Systems (CHI), 2021
Revamp is a online shopping system aimed to provide simplified experience for Blind and Low Vision (BLV) users. It extracts the user review from the product page using linguistic rules, and generate QA interfaces based on the review data, which provides the visual appearance information of the product.
IEEE Pacific Visualization Symposium (PacificVis), Notes, 2020
We provide an intuitive visualization tool for natural language processing tasks where attention is mapped between documents with imbalanced sizes. We extend the flow map visualization to enhance the readability of the attention-augmented documents. Our project page
The ACM CHI Conference on Human Factors in Computing Systems (CHI), 2019 Best Paper Award
We present Anchored Audio Smapling (AAS) method for collecting remote data of qualitative audio samples during field development with young children. The anchor event triggers the recording, and a sliding window surrounding this anchor captures both antecedent and ensuing recording. Our AAS Library for Android