Some of my research materials:

Intelligent Text Entry Application

  • PhraseFlow: Designs and Empirical Studies of Phrase-Level Input Mingrui Zhang, Shumin Zhai
    The ACM CHI Conference on Human Factors in Computing Systems (CHI), 2021 Video My presentation
    PhraseFlow is a phrase-level input keyboard that is able to correct previous text based on the subsequently input sequences. For example, if the user types "I love in Seattle", the keyboard will correct "love" to "live" after it sees "in Seattle". The project evaluated the usability of PhraseFlow and attempted to design a pracitcally usable phrase-level keyboard.
  • JustCorrect: Intelligent Post Hoc Text Correction Techniques on Smartphones Wenzhe Cui, Suwen Zhu, Mingrui Zhang, H. Andrew Schwartz, Jacob O. Wobbrock, Xiaojun Bi
    The ACM Symposium on User Interface Software and Technology (UIST), 2020 Video Wenzhe's Presentation
    JustCorrect is a post-correction technique for smartphones, sharing the same genre with Type, Then Correct. JustCorrect utilizes the word embeddings and language models to detect the error and display correction options automatically.
  • Gedit: Keyboard gestures for mobile text editing Mingrui Zhang, Jacob O. Wobbrock
    Proceedings of Graphics Interface (GI), 2020 Video My presentation
    We proposed a set of on-keyboard text editing gestures for the mobile keyboard, similar to the desktop keyboard shotcuts, including ring/letter/swipe gestures to facilitate fast editing tasks such as cursor-moving/copy/paste/cut/undo. Gedit provides one- and two-handed operation modes, and is also compatible with the gesture typing input.
  • Type, Then Correct: Intelligent Text Correction Techniques for Mobile Text Entry Using Neural Networks Mingrui Zhang, He Wen, Jacob O. Wobbrock
    The ACM Symposium on User Interface Software and Technology (UIST), 2019 My Presentation Video Project Page
    Instead of normal touch+cursor based correction process, why cannot we rethink of the correction interaction? In this paper, we present three novel interactions that allow the user to type the correction first, then apply it to the error place. Furthermore, we applied deep learning technology to enable automatic error detection for the interaction. Our correction RNN model
  • ATK: Enabling Ten-Finger Freehand Typing in Air Based on 3D Hand Tracking Data Xin Yi, Chun Yu, Mingrui Zhang, Sida Gao
    The ACM Symposium on User Interface Software and Technology (UIST), 2015 Video
    A novel air-typing method, Leapmotion tracking fingers, improved Bayes prediction model with application developed. Users reached the speed of 29.2 WPM on average.
  • Text Entry Evaluation

  • Beyond the Input Stream: Making Text Entry Evaluations More Flexible with Transcription Sequences Mingrui Zhang, Jacob O. Wobbrock
    The ACM Symposium on User Interface Software and Technology (UIST), 2019 Video
    In this work, we present a new underlying model that supersedes the input stream model for general-purpose method-independent character-level text entry evaluation. Specifically, we present an approach that replaces the input stream with transcription sequences, or “T-sequences” for short. In brief, T-sequences are snapshots of the entire transcribed string after each text-changing action is taken by the user. Every pair of successive snapshots are then analyzed to compute character-level text entry metrics. TextTest++ platform
  • Text Entry Throughput: Towards Unifying Speed and Accuracy in a Single Performance Metric Mingrui Zhang, Shumin Zhai, Jacob O. Wobbrock
    The ACM CHI Conference on Human Factors in Computing Systems (CHI), 2019 My Presentation Related Blog Post
    We define the text entry Throughput as a performance metric combining the speed and accuracy. Throughput is derived from the transmission ratio in the information theory. Unlike other metrics, throughput is less affected by speed-accuracy tradeoffs, thus it enables cross-device, cross-publication comparison. Throughput calculation library
  • Communication with Emojis 🧐

  • Voicemoji: Emoji entry using voice for visually impaired people Mingrui Zhang, Ruolin Wang, Xuhai Xu, Qisheng Li, Ather Sharif, Jacob O. Wobbrock
    The ACM CHI Conference on Human Factors in Computing Systems (CHI), 2021 My presentation Related Blog Post (in Chinese)
    A speech-based emoji input system, designed for blind and low vision users. Use natural language style emoji query and context sensitive emoji suggestions based on the spoken content. Voicemoji speeds up the emoji entry process by 91% than the iOS keyboard.
  • A comparative study of lexical and semantic emoji suggestion systems Mingrui Zhang, Alex Mariakakis, Jacob Burke, Jacob O. Wobbrock
    iConference, 2021 My presentation
    We compared how the lexical based and semantic based emoji suggestion mechanisms affected the online chatting experience through an in-lab study and a field deployment. The results showed that the suggestion system of emojis did not influence the chatting experience, and users enjoy using both suggestion systems for different reasons.
  • Communication with Smart Assistants 🤖

  • Can Conversational Agents Change the Way Children Talk to People? Alexis Hiniker, Amelia Wang, Jonathan Tran, Mingrui Zhang, Jenny Redesky, Kiley Sobel, Sungsoo Ray Hong
    Proceedings of the 20th annual ACM conference on interaction design and children (IDC), 2021 Related UW News
    Does hanging out with Alexa or Siri affect the language routine children use to communicate with their fellow humans? In this work, we built two conversational agents, and let them teach the kids a word "bungo" to make the the agenets speak quickly. Although the kids used the word with the agents, they were aware of the social context when facing similar situation with other people.
  • Assumptions Checked: How Families Learn About and Use the Echo Dot Erin Beneteau, Yini Guan, Olivia K. Richards, Mingrui Zhang, Julie A. Kientz, Jason Yip, Alexis Hiniker
    Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2020
    Through a one-month deployment study, we investigated how families learnt new functionalities of smart speakers, include 1) which features families are aware of and engage with, and 2) how families explore, discover, and learn to use the Echo Dot. Drawing from diffusion of innovation theory, we describe how a home-based voice interface might be positioned as a near-peer to the user and help them discover new functionalities.
  • Communication Breakdowns Between Families and Alexa Erin Beneteau, Olivia K. Richards, Mingrui Zhang, Julie A. Kientz, Jason Yip, Alexis Hiniker
    The ACM CHI Conference on Human Factors in Computing Systems (CHI), 2019 Erin's Presentation Related Blog Post
    We investigated different types of communication breakdowns and the repairing strategies between the conversation of family members and Alexa. Our findings indicates that improving technology’s ability to identify the communication partners and to provide specific clarification responses will ultimately improve the conversational interaction experience.
  • MISC

  • Revamp: Enhancing Accessible Information Seeking Experience of Online Shopping for Blind or Low Vision Users Ruolin Wang, Zixuan Chen, Mingrui Zhang, Zhaoheng Li, Zhixu Liu, Zihang Dang, Chun Yu, Xiang "Anthony" Chen
    The ACM CHI Conference on Human Factors in Computing Systems (CHI), 2021
    Revamp is a online shopping system aimed to provide simplified experience for Blind and Low Vision (BLV) users. It extracts the user review from the product page using linguistic rules, and generate QA interfaces based on the review data, which provides the visual appearance information of the product.
  • InteractiveAttention Model Explorer for NLP Tasks with Unbalanced Data Sizes Zhihang Dong, Tongshuang Wu, Sicheng Song, Mingrui Zhang
    IEEE Pacific Visualization Symposium (PacificVis), Notes, 2020
    We provide an intuitive visualization tool for natural language processing tasks where attention is mapped between documents with imbalanced sizes. We extend the flow map visualization to enhance the readability of the attention-augmented documents. Our project page
  • Anchored Audio Sampling: A Seamless Method for Exploring Children’s Thoughts During Deployment Studies Alexis Hiniker, Jon E. Froehlich, Mingrui Zhang, Erin Beneteau
    The ACM CHI Conference on Human Factors in Computing Systems (CHI), 2019 Best Paper Award
    We present Anchored Audio Smapling (AAS) method for collecting remote data of qualitative audio samples during field development with young children. The anchor event triggers the recording, and a sliding window surrounding this anchor captures both antecedent and ensuing recording. Our AAS Library for Android

  • © Mingrui Zhang. All rights reserved.