ACM Conference on Human Factors in Computing Systems (CHI)
May 1, 2024
Multiple CHCI members, including Eugenia Rho, Chris North, Sang Won Lee, Yan Chen, Kurt Luther, Yaxing Yao, and Denis Gracanin, have contributed research, workshop papers, and demos to CHI this year. CHCI affiliates are listed below in bold.
The ACM CHI Conference on Human Factors in Computing Systems is a premier international Human-Computer Interaction (HCI) conference. CHI – pronounced ‘kai’ – annually brings together researchers and practitioners from all over the world and from diverse cultures, backgrounds, and positionalities who aim to improve the world with interactive digital technologies.
This year, the conference embraces the theme of “Surfing the World,” which focuses on pushing forth the wave of cutting-edge technology and riding the tide of new developments in human-computer interaction. CHI will take place in Honolulu, Hawaiʻi, USA, from May 11-16, 2024, while supporting remote attendance.
Papers
Xiaohan Ding, Buse Carik, Uma Sushmitha Gunturi, Valerie Reyna, and Eugenia Rho
Leveraging Prompt-Based Large Language Models: Predicting Pandemic Health Decisions and Outcomes Through Social Media Language
Abstract: We introduce a multi-step reasoning framework using prompt-based LLMs to examine the relationship between social media language patterns and trends in national health outcomes. Grounded in fuzzy-trace theory, which emphasizes the importance of “gists” of causal coherence in effective health communication, we introduce Role-Based Incremental Coaching (RBIC), a prompt-based LLM framework, to identify gists at scale. Using RBIC, we systematically extract gists from subreddit discussions opposing COVID-19 health measures (Study 1). We then track how these gists evolve across key events (Study 2) and assess their influence on online engagement (Study 3). Finally, we investigate how the volume of gists is associated with national health trends like vaccine uptake and hospitalizations (Study 4). Our work is the first to empirically link social media linguistic patterns to real-world public health trends, highlighting the potential of prompt-based LLMs in identifying critical online discussion patterns that can form the basis of public health communication strategies.
Sungwon In, Eric Kroks, Kirsten Whitley, Chris North, and Yalong Yang (former VT/CS/CHCI)
Evaluating Navigation and Comparison Performance of Computational Notebooks on Desktop and in Virtual Reality
Abstract: The computational notebook serves as a versatile tool for data analysis. However, its conventional user interface falls short of keeping pace with the ever-growing data-related tasks, signaling the need for novel approaches. With the rapid development of interaction techniques and computing environments, there is a growing interest in integrating emerging technologies in data-driven workflows. Virtual reality, in particular, has demonstrated its potential in interactive data visualizations. In this work, we aimed to experiment with adapting computational notebooks into VR and verify the potential benefits VR can bring. We focus on the navigation and comparison aspects as they are primitive components in analysts' workflow. We have designed and implemented a Branching & Merging functionality to improve comparison further. We tested computational notebooks on the desktop and VR, both with and without the added Branching & Merging capability. We found VR significantly facilitated navigation compared to desktop, and the ability to create branches enhanced comparison.
Donghan Hu, Sang Won Lee
Title: Exploring the Effectiveness of Time-Lapse Screen Recordings for Self-Reflection in Work Contexts
Abstract: Effective self-tracking in working contexts empowers individuals to explore and reflect on past activities. Recordings of computer activities contain rich metadata that can offer valuable insight into users' previous tasks and endeavors. However, presenting a simple summary of time usage may not effectively engage users with data because it is not contextualized, and users may not understand what to do with the data. This work explores time-lapse videos as a visual-temporal medium to facilitate self-reflection among workers in productivity contexts. To explore this space, we conducted a four-week study (n=15) to investigate how a computer screen's history of states can help workers recall previous undertakings and gain comprehensive insights via self-reflection. Our results support that watching time-lapse videos can enhance self-reflection more effectively than traditional self-tracking tools by providing contextual clues about users' past activities. The experience with both traditional tools and time-lapse videos resulted in increased productivity. Additionally, time-lapse videos assist users in cultivating a positive understanding of their work. We discuss how multimodal cues, such as time-lapse videos, can complement personal informatics tools.
Kenan Kamel A Alghythee, Adel Hrncic, Karthik Singh, Sumanth Kunisetty, Yaxing Yao, Nikita Soni
Title: Towards Understanding Family Privacy and Security Literacy Conversations at Home: Design Implications for Privacy Literacy Interfaces
Abstract: Policymakers and researchers have emphasized the crucial role of parent-child conversations in shaping children's digital privacy and security literacy. Despite this emphasis, little is known about the current nature of these parent-child conversations, including their content, structure, and children's engagement during these conversations. This paper presents the findings of an interview study involving 13 parents of children ages under 13 reflecting on their privacy literacy practices at home. Through qualitative thematic analysis, we identify five categories of parent-child privacy and security conversations and examine parents' perceptions of their children's engagement during these discussions. Our findings show that although parents used different conversation approaches, rule-based conversations were one of the most common approaches taken by our participants, with example-based conversations perceived to be effective by parents. We propose important design implications for developing effective privacy educational technologies for families to support parent-child conversations.
Chaoran Chen, Weijun Li, Wenxin Song, Yanfang Ye, Yaxing Yao, Toby Jia-Jun Li
Title: An Empathy-Based Sandbox Approach to Bridge the Privacy Gap among Attitudes, Goals, Knowledge, and Behaviors
Abstract: Managing privacy to reach privacy goals is challenging, as evidenced by the privacy attitude-behavior gap. Mitigating this discrepancy requires solutions that account for both system opaqueness and users' hesitations in testing different privacy settings due to fears of unintended data exposure. We introduce an empathy-based approach that allows users to experience how privacy attributes may alter system outcomes in a risk-free sandbox environment from the perspective of artificially generated personas. To generate realistic personas, we introduce a novel pipeline that augments the outputs of large language models (e.g., GPT-4) using few-shot learning, contextualization, and chain of thoughts. Our empirical studies demonstrated the adequate quality of generated personas and highlighted the changes in privacy-related applications (e.g., online advertising) caused by different personas. Furthermore, users demonstrated cognitive and emotional empathy towards the personas when interacting with our sandbox. We offered design implications for downstream applications in improving user privacy literacy.
Lanjing Liu, Chao Zhang, Zhicong Lu
Title: Wrist-bound Guanxi, Jiazu, and Kuolie: Unpacking Chinese Adolescent Smartwatch-Mediated Socialization
Abstract: Adolescent peer relationships, essential for their development, are increasingly mediated by digital technologies. As this trend continues, wearable devices, especially smartwatches tailored for adolescents, are reshaping their socialization. In China, smartwatches like XTC have gained wide popularity, introducing unique features such as "Bump-to-Connect" and exclusive social platforms. Nonetheless, how these devices influence adolescents' peer experience remains unknown. Addressing this, we interviewed 18 Chinese adolescents (age: 11 -- 16), discovering a smartwatch-mediated social ecosystem. Our findings highlight the ice-breaking role of smartwatches in friendship initiation and their use for secret messaging with local peers. Within the online smartwatch community, peer status is determined by likes and visibility, leading to diverse pursuit activities (i.e., chu guanxi, jiazu, kuolie) and negative social dynamics. We discuss the core affordances of smartwatches and Chinese cultural factors that influence adolescent social behavior and offer implications for designing future wearables that responsibly and safely support adolescent socialization.
Workshops Organized by CHCI affiliates
Workshop on Human-Notebook Interactions
Organizers: Jesse Harden, Rebecca Faust, Katherine E. Isaacs, Nurit Kirshenbaum, Chris North, April Wang, John Wenskovitch, and Jian Zhao.
Abstract - Computational notebooks have become an ubiquitous medium for much data science work. At their best, computational notebooks actualize Knuth's concept of literate programming with their ability to weave code, text, and outputs, such as visuals, into a computational narrative. These notebooks support incremental and iterative analysis, explanation of an analyst's thoughts and processes, and sharing of code, text, and visuals in one document.
However, notebook systems are in their infancy in terms of user interface and interaction design. Recent research describes several pain points users of computational notebooks have, including effectively managing more complex, non-linear data science and analysis workflows, messiness, version control, debugging issues, making the most of larger, widescreen displays, and more.
In this workshop, we seek to explore, catalogue, and innovate with respect to issues of user interface and interaction design of computational notebooks.
Organizers: Vikram Mohanty (VT CS/CHCI alum), among others.
Abstract: Global CO₂ emissions are on the rise, driven by transportation, agriculture, and energy production activities. While corporations and policymakers play a crucial role in reducing greenhouse gas emissions, our workshop pivots the focus towards individuals and communities as essential agents of change. Individual consumers and private households, responsible for a significant portion of global emissions, often underestimate their impact on climate change and potential for mitigation. This workshop aims to address these gaps by encouraging carbon reduction, promoting carbon literacy and empowering local communities to take meaningful action.
Workshop Papers
Matthew Wilchek, Linhan Wang, Feras A. Batarseh, and Kurt Luther
KHAIT: K-9 Handler Artificial Intelligence Teaming for Collaborative Sensemaking
Abstract: Following natural and manmade disasters, urban search and rescue (USAR) involving canines (i.e., K-9) face time delays and communication hurdles. This paper proposes a method to improve USAR sensemaking by incorporating object detection Artificial Intelligence (AI) and Augmented Reality (AR), aiming to close the communication gap between USAR animals and their handlers. SAR dogs are equipped with an AI-powered camera and edge computing hardware, while their handlers wear HoloLens-2 AR headsets. This setup allows for advanced object detection from the dogs' perspective, with captured data transmitted to handlers' headsets as hologram indicators to enhance survivor localization. This multidisciplinary strategy, merging human-in-the-loop (HITL), canine, and AI capabilities, aims to transform SAR missions, ensuring faster and safer rescues. At the CHI sensemaking workshop, we hope to solicit feedback regarding our proposed design and evaluation that will have broad relevance to attendees interested in human-AI and animal-computer interaction, AR, and/or disaster response.
Poorvesh Dongre, Majid Behravan, Kunal Gupta, Mark Billinghurst, and Denis Gracanin
Title: Integrating Physiological Data with Large Language Models for Empathic Human-AI Interaction. PhysioCHI24 Workshop
Abstract: This paper explores enhancing empathy in Large Language Models (LLMs) by integrating them with physiological data that can be used to interpret users' mental and emotional states. We propose a multi-faceted approach: (1) constructing evolving user profiles using real-time physiological data, (2) developing deep learning models that use physiological data for recognizing mental and emotional states, and (3) integrating the predictive models with LLMs for empathic interaction. We showcase the application of this approach in an Empathic LLM (EmLLM) chatbot for stress monitoring and control. We also discuss the results of a pilot study that evaluates this EmLLM chatbot based on its ability to accurately predict user stress, provide human-like responses, and assess the therapeutic alliance with the user.
Daniel Vargas-Diaz, Junghoon Chung, Donghan Hu, Sol Lim, Sang Won Lee
Title: Developing Context-Aware Sit-Stand Desks for Promoting Healthy and Productive Behaviors. Workshop on Office Wellbeing by Design.
Abstract: To mitigate the risk of chronic diseases caused by prolonged sitting, sit-stand desks are promoted as an effective intervention to foster healthy behaviors among knowledge workers by allowing periodic posture switching between sitting and standing. However, conventional systems let users manually switch the mode, and some research visited automated notification systems with pre-set time intervals. While this regular notification can promote healthy behaviors, such notification can act as external interruptions that hinder individuals' working productivity. Notably, knowledge workers are known to be reluctant to change their physical postures when concentrating. To address these issues, we propose considering work context based on their screen activities to encourage computer users to alternate their postures when it can minimize disruption, promoting healthy and productive behaviors. To that end, we are in the process of building a context-aware sit-stand desk that can promote healthy and productive behaviors. To that end, we have completed two modules: an application that monitors users' computer's ongoing activities and a control module that can measure the height of sit-stand desks for data collection and also allows their computer to control the desk height. The collected data includes computer activities, measured desk height, and their willingness to switch to standing modes and will be used to build an LSTM prediction model to suggest optimal time points for posture changes, accompanied by appropriate desk height. In this work, we acknowledge previous relevant research, outline ongoing deployment efforts, and present our plan to validate the effectiveness of our approach via user studies.
Daniel Vargas-Diaz, Jisun Kim, Sulakna Karunaratna, Maegan Reinhardt, Caroline Hornburg, Koeun Choi, Sang Won Lee
Title: TaleMate: Exploring the use of Voice Agents for Parent-Child Joint Reading Experiences. Workshop on Child-centered AI Design
Abstract: Joint reading is a key activity for early learners, with caregiver-child interactions such as questioning and feedback playing an essential role in children's cognitive and linguistic development. However, for some parents, actively engaging children in storytelling can be challenging. To address this, we introduce TaleMate—a platform designed to enhance shared reading by leveraging conversational agents that have been shown to support children's engagement and learning. It features eight unique voice agents, each with a distinct tone and style, capable of portraying specific characters in a story. TaleMate enables a dynamic, participatory reading experience where parents and children can choose which characters they wish to embody. Feedback and suggestions from parents and children have informed the design of TaleMate, highlighting its effectiveness and potential to cater to user-specific needs. Moreover, the system navigates the challenges posed by digital reading tools, such as decreased parent-child interaction, and builds upon the benefits of traditional and digital reading techniques. TaleMate offers an innovative approach to fostering early reading habits, bridging the gap between traditional joint reading practices and the digital reading landscape.
Andrew Jelson, Sang Won Lee
Title: An empirical study to understand how students use ChatGPT for writing essays and how it affects their ownership, In2Writing Workshop
Abstract: As large language models (LLMs) become more powerful and ubiquitous, systems like ChatGPT are increasingly used by students to help them with writing tasks. To better understand how these tools are used, we investigate how students might use an LLM for essay writing, for example, to study the queries asked to ChatGPT and the responses that ChatGPT gives. To that end, we plan to conduct a user study that will record the user writing process and present them with the opportunity to use ChatGPT as an AI assistant. This study's findings will help us understand how these tools are used and how practitioners --- such as educators and essay readers --- should consider writing education and evaluation based on essay writing.
Adnan Abbas, Sang Won Lee
Title: PITCH: Productivity and Mental Well-being Coaching through Daily Conversational Interaction
Abstract: Efficient task planning is essential for productivity and mental well-being, yet individuals often struggle to create realistic plans and reflect upon their productivity. Leveraging the advancement in artificial intelligence (AI), conversational agents have emerged as a promising tool for enhancing productivity. Our work focuses on externalizing plans through conversation, aiming to solidify intentions and foster focused action, thereby positively impacting their productivity and mental well-being. We share our plan of designing a conversational agent to offer insightful questions and reflective prompts for increasing plan adherence by leveraging the social interactivity of natural conversations. Previous studies have shown the effectiveness of such agents, but many interventions remain static, leading to decreased user engagement over time. To address this limitation, we propose a novel rotation and context-aware prompting strategy, providing users with varied interventions daily. Our system, PITCH, utilizes large language models (LLMs) to facilitate externalization and reflection on daily plans. Through this study, we investigate the impact of externalizing tasks with conversational agents on productivity and mental well-being, and the effectiveness of a rotation strategy in maintaining user engagement.
Xinyi Zhang, Jake Frohich, Pamela J. Wisniewski, Jin-Hee Cho, Lifu Huang, Sang Won Lee
Title: Generating A Crowdsourced Conversation Dataset to Combat Cybergrooming
Abstract: Cybergrooming emerges as a growing threat to adolescent safety and mental health. One way to combat cybergrooming is to leverage predictive artificial intelligence (AI) to detect predatory behaviors in social media. However, these methods can encounter challenges like false positives and negative implications such as privacy concerns. Another complementary strategy involves using generative artificial intelligence to empower adolescents by educating them about predatory behaviors. To this end, we envision developing state-of-the-art conversational agents to simulate the conversations between adolescents and predators for educational purposes. Yet, one key challenge is the lack of a dataset to train such conversational agents. In this position paper, we present our motivation for empowering adolescents to cope with cybergrooming. We propose to develop large-scale, authentic datasets through an online survey targeting adolescents and parents. We discuss some initial background behind our motivation and proposed design of the survey, such as situating the participants in artificial cybergrooming scenarios, then allowing participants to respond to the survey to obtain their authentic responses.
Ashley Zhang, Xiaohang Tang, Yan Chen, Steve Oney
Title: Making Code Understandable at Scale for Sensemaking in Introductory Programming Education. Workshop on Sensemaking
Abstract: The increasing need of programming skills has driven the exponential growth of CS learners recently. In large programming courses, instructors use programming exercises to give students hands-on opportunities to practice. However, it is challenging to make sense of students' code at scale, because of the complex collection of information and the wide variation among submissions.
If using students' code submission data in meaningful ways, instructors can get insights into students' thought process, misconceptions, and common trends in class. With limited instruction resources, it is impossible to provide tailored feedback to every individual student. Tools have been designed to facilitate instructor's sensemaking of complex collection of students' code submissions, through combining techniques from Human-Computer Interaction, Visualization, and Natural Language Processing.
This paper will introduce two systems we designed for instructors to make sense of students' code at scale, discuss gaps in instructor's sensemaking process, and explore future design opportunities to support gaps in sensemaking.
Remote Doctoral Consortium
Poorvesh Dongre
Title: Physiology-Driven Empathic Large Language Models (EmLLMs) for Mental Health Support
Abstract: Wearable devices show promise in monitoring and managing mental health, but gaps exist in accurately predicting users' mental states and cognitively engaging with users to provide mental health support with wearable data. In this proposal, I present the concept of physiology-driven Empathic Large Language Models (EmLLMs) for mental health support. EmLLMs monitor users and their surrounding environment using wearable devices to predict their mental and emotional states and interact with them based on these states. I present the application of this approach for monitoring and managing excess stress in the workplace. To improve the accuracy of stress prediction, I developed a novel Science-Guided Machine Learning (SGML) model that automatically extracts features from raw wearable data. To engage with users cognitively, I developed an (EmLLM) chatbot that provides psychotherapy based on predicted user stress. I present the SGML model's preliminary findings and results from a pilot user study that evaluates the EmLLM chatbot.
Late-Breaking Work
Lanjing Liu, Lan Gao, Nikita Soni, Yaxing Yao
Title: Integrating Family Privacy Education and Informal Learning Spaces: Characteristics, Challenges and Design Opportunities
Abstract: Children face increasing privacy risks and the need to navigate complex choices, while privacy education is not sufficient due to limited education scope and family involvement. We advocate for informal learning spaces (ILS) as a pioneering channel for family-based privacy education, given their established role in holistic technology and digital literacy education, which specifically targets family groups. In this paper, we conducted an interview study with five families to understand revealing current approaches to privacy education and engagement with ILS for family-based learning. Our findings highlight the transformative potential of ILS in family privacy education, considering existing practices and challenges. We discuss the reasons for family-based privacy education in ILS and identify potential design opportunities. Additionally, we outline our future work, which involves expanding participant involvement and conducting co-design activities with family groups to create design prototypes.