CHCI Contributions to UIST and CSCW

Paper at UIST

Scrapbook: Screenshot-Based Bookmarks for Effective Digital Resource Curation across Applications.

Donghan Hu  and Sang Won Lee.

Modern knowledge workers typically need to use multiple resources, such as documents, web pages, and applications, at the same time. This complexity in their computing environments forces workers to restore various resources in the course of their work. However, conventional curation methods like bookmarks, recent document histories, and file systems place limitations on effective retrieval. Such features typically work only for resources of one type within one application, ignoring the interdependency between resources needed for a single task. In addition, text-based handles do not provide rich cues for users to recognize their associated resources. Hence, the need to locate and reopen relevant resources can significantly hinder knowledge workers’ productivity. 

To address these issues, we designed and developed Scrapbook, a novel application for digital resource curation across applications that uses screenshot-based bookmarks. Scrapbook extracts and stores all the metadata (URL, file location, and application name) of windows visible in a captured screenshot to facilitate restoring them later. A week-long field study indicated that screenshot-based bookmarks helped participants curate digital resources. Additionally, participants reported that multimodal—visual and textual—data helped them recall past computer activities and reconstruct working contexts efficiently.

Papers at CSCW

Intimate Narratives: An Assets-Based Approach To Develop Holistic Perspectives of Student Mothers’ Lives and Their Use of Technology in Parenting.

(DOI: https://dl.acm.org/doi/10.1145/3555635)

Neelma Bhatti (VT/CS/CHCI alum), Amarachi Blessing Mbakwe, Sandra Nnadi, Geetha Saarunya, Aakash Gautam (VT/CS/CHCI alum), D. Scott McCrickard, Aisling Kelliher

This paper details our collaborative approach in capturing a holistic understanding of parental technology use through an assets-based framework. We steer the focus away from the design of technology as the central force of technological innovation, and instead support participants to reflect and describe intimate details that highlight specific use-contexts of technology in their lives. We leverage a group of foreign graduate student mothers' self-described unique strengths to gain an in-depth account of their lived experiences with technology. 

As research participants and co-authors, our collaborators elicit intimate narratives about meaningful events in their lives, bringing social and cultural aspects of their lived experience to the forefront, and thus providing broader context of their use of technology. We detail and reflect upon our approach of promoting user agency by creating an affinity group, fostering a safe and intimate space for research engagement, and describe the implications of using our adapted research methodology in intimate settings. We conclude by highlighting the various ways in which technology facilitates foreign student parenting, as well as the ways in which it serves as a temporary band-aid solution, prompting consideration of larger social issues.

The Long Way Home: News Values in Stories Told by Appalachian Trail Thru-Hikers on Social Media.

Lindah Kotut (VT/CS/CHCI Alum), Michael Horning , D. Scott McCrickard

Technology use by long-distance hikers provides a fascinating glimpse of the future of HCI research, as it intersects with technology isolation and (non)use. We can understand the use of technology, explore opportunities for rural and outdoor computing, and work with underrepresented communities in these spaces to amplify their stories. In this paper, we focus on the thru-hikers on the Appalachian Trail and the stories they share on Instagram. We apply narrative analysis to understand the story content, and also leverage a News Value framework to describe how they craft their stories. The findings highlight opportunities to understand the different storytelling categories that thru-hikers employ, and the platforms they choose for the telling. We discuss the usefulness of understanding technology and storytelling needs for each type of storyteller, and explore the implications for designing technology and tools to amplify stories from underserved users and to support the telling of compelling stories.

Posters at UIST

TaskScape: Fostering Holistic View on To-do List With Tracking Plan and Emotion.

Marx Wang and Sang Won Lee

Despite advancements with intelligence and connectivity in the workspace, productivity tools, such as to-do list applications, still measure workers’ performance by a binary state—completed, not yet completed, and the number of tasks completed. Such quantitative measurements can often overlook human values and individual well-being. While concepts such as positive computing and digital well-being are on the rise in the HCI community, few systems have been proposed to effectively integrate holistic considerations for mental and emotional well-being into productivity tools. In this work, we depart from the classic task list management tool and explore the construction of well-being-centered to-do list software. 

We propose a task management system–TaskScape—, which allows users to have awareness of the following two aspects: (1) how they plan and complete tasks and (2) how they feel towards their work. With the proposed system, we will investigate if having a holistic view of their tasks can facilitate reflection about what they work on, how they stick to their plans, and how their tasks portfolio supports their emotional well-being, nudging users to reflect upon their work, planning performance, and their emotional values towards their work. In this poster, we share the design, development, and ongoing validation progress of TaskScape, which is aimed at nudging workers to holistically view work productivity, thus reminding users that work is not just work but life.

iThem: Programming Internet of Things beyond Trigger-action Pattern.

Marx Wang, Daniel Manesh, Ruipu Hu, Sang Won Lee

With emerging technologies bringing Internet of Things (IoT) devices into domestic environments, trigger-action programming such as IFTTT with its simple if-this-then-that pattern provides an effective way for end-users to connect fragmented intelligent services and program their own smart home/work space automation. While the simplicity of trigger-action programming can be effective for non-programmers with its straightforward concepts and graphical user interface, it does not allow the algorithmic expressivity that a programming language has. For instance, the simple if-this-then-that structure cannot cover complex algorithms that arise from real world scenarios involving multiple conditions or keeping track of a sequence of conditions (e.g., incrementing counters, triggering one action if two conditions are both true).

In this exploratory work, we take an alternative approach by creating a programmable channel between application programming interfaces (APIs), which allows programmers to preserve states and to use them to write complex algorithms. We propose iThem, which stands for intelligence of them—internet of things, that allows programmers to author any complex algorithms that can connect different IoT services and fully unleash the freedom of a general programming language. In this poster, we share the design, development, and ongoing validation progress of iThem, which piggybacks on existing programmable IoT system IFTTT, and which allows for a programmable channel that connects triggers and actions in IFTTT with versatility.

Echofluid: An interface for remote choreography learning and co-creation using machine learning techniques. 

Marx Wang, Zach Duer, Scotty Hardwig, Sam Lally, Alayna Ricard,  Myounghoon Jeon. 

(This work was supported by a 2020 VT ICAT Rapid Response Grant, PI: Zach Duer, co-PIs: Scotty Hardwig, Myounghoon Jeon) 

Born from physical activities, dance carries beyond mere body movement. Choreographers interact with audiences’ perceptions through the kinaesthetics, creativity, and expressivity of whole-body performance, inviting them to construct experience, emotion, culture, and meaning together. Computational choreography support can bring endless possibilities into this one of the most experiential and creative artistic forms.

While various interactive and motion technologies have been developed and adopted to support creative choreographic processes,  little work has been done in exploring incorporating machine learning in a choreographic system, and few remote dance teaching systems in particular have been suggested. In this exploratory work, we proposed Echofluid-a novel AI-based choreographic learning and support system that allows student dancers to compose their own AI models for learning, evaluation, exploration, and creation. In this poster, we present the design, development and ongoing validation process of Echofluid, and discuss the possibilities of applying machine learning in collaborative art and dance as well as the opportunities of augmenting interactive experiences between the performers and audiences with emerging technologies.

Demo at UIST

Demonstration of Geppetteau: Enabling haptic perceptions of virtual fluids in various vessel profiles using a string-driven haptic interface.

Shahabedin Sagheb, Frank Wencheng Liu, Alex Vuong, Shiling Dai, Ryan Wirjadi, Yueming Bao, Robert LiKamWa

Liquids sloshing around in vessels produce unique unmistakable tactile sensations of handling fluids in daily life, laboratory environments, and industrial contexts. Providing nuanced congruent tactile sensations would enrich interactions of handling fluids in virtual reality (VR). To this end, we introduce Geppetteau, a novel string-driven weight-shifting mechanism capable of providing a continuous spectrum of perceivable tactile sensations of handling virtual liquids in VR vessels. Geppetteau's weight-shifting actuation system can be housed in 3D-printable shells, adapting to varying vessel shapes and sizes. A variety of different fluid behaviors can be felt using our haptic interface. In this work, Geppetteau assumes the shape of conical, spherical, cylindrical, and cuboid flasks, widening the range of augmentable shapes beyond the state-of-the-art of existing mechanical systems.