Messaging Beyond Texts with Real-time Image Suggestions



Abstract


While people primarily communicate with text in mobile chat applications, they are increasingly using visual elements such as images, emojis, and memes. Using such visual elements could help users communicate clearly and make chatting experience enjoyable. However, finding and inserting contextually appropriate images during the chat can be both tedious and distracting. We introduce MilliCat, a real-time image suggestion system that recommends images that match the chat content within a mobile chat application (i.e., autocomplete with images) MilliCat combines natural language processing (e.g., keyword extraction, dependency parsing) and mobile computing (e.g., resource and energy-efficiency) techniques to autonomously make image suggestions when users might want to use images. Through multiple user studies, we investigated the effectiveness of our design choices, the frequency and motivation of image usage by the participants, and the impact of MilliCat on mobile chat experiences. Our results indicate that MilliCat’s real-time image suggestion enables users to quickly and conveniently select and display images on mobile chat by significantly reducing the latency in the image selection process (3.19× improvement) and consequently more frequent image usage (1.8×) than existing olutions. Our study participants reported that they used images more often with MilliCat as the images helped them convey information more effectively, emphasize their opinion, express emotions, and have fun chatting experience.


Video




Publications


Messaging Beyond Texts with Real-time Image Suggestions
Joongyum Kim, Taesik Gong, Kyungsik Han, Juho Kim, JeongGil Ko, and Sung-Ju Lee
ACM International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) 2020.
PDF

A Picture is Worth a Thousand Words: Improving Mobile Messaging with Real-time Autonomous Image Suggestion
Joon-Gyum Kim, Chia-Wei Wu, Alvin Chiang, JeongGil Ko, and Sung-Ju Lee
Proceedings of ACM HotMobile 2016, St. Augustine, FL, February 2016.
PDF


People


Joongyum Kim

KAIST

Taesik Gong

KAIST

Kyungsik Han

Ajou University

Juho Kim

KAIST

JeongGil Ko

Yonsei University

Sung-Ju Lee

KAIST