Virginia Tech® home

Significant CHCI contributions and awards at IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

Significant CHCI contributions and awards at IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

There were significant CHCI contributions and awards at the IEEE International Symposium on Mixed and Augmented Reality (ISMAR), held virtually from Bari, Italy from October 4th through 8th, 2021. IEEE ISMAR is the premier symposium for Augmented Reality (AR) and Mixed Reality (MR). ISMAR explores the advances in commercial and research activities related to AR, MR, and Virtual Reality (VR) by continuing the expansion of its scope over the past several years.The symposium is organized and supported by the IEEE Computer Society, IEEE VGTC and ACM SIGGRAPH.

General Chair:

Joe Gabbard (ISE), head of the VT ISE COGENT Lab (with Co-chairs Antonello Uva and Michele Fiorentino from Politecnico di Bari, Italy, and Veronica Teichrieb from Universidade Federal de Pernambuco, Brazil.) was the General Chair of the 20th International Symposium on Mixed & Augmented Reality (ISMAR).  This year's ISMAR conference hosted nearly 550 researchers and practitioners and facilitated sharing and discussion of state-of-the-art scientific contributions in augmented and virtual reality.  Gabbard started planning ISMAR 2021 in January 2020 -- a process that took over 22 months to complete.

Keynote Speaker:

Doug Bowman (CS), Director of CHCI gave the closing keynote address at the conference, with a talk entitled User Experience Considerations for Everyday Augmented Reality. The abstract is below.

Future AR glasses will be much like today’s smartphones, as our everyday information access and productivity devices. Technical challenges in optics, power, and tracking remain, but are solvable. However, technical achievements alone are insufficient to ensure that everyday AR systems will be productive, usable, useful, and satisfying. We must also design effective methods for interacting with and managing AR content, and we must understand the effects of always-on AR, on both individuals and communities. In this talk, I present a vision of future everyday AR use cases, and discuss recent user experience (UX) research aimed at enabling this vision. I discuss UX design recommendations for making information access through AR more convenient and usable than today’s smartphones and smartwatches, while at the same time not distracting users from what’s going on in the real world around them. I conclude with a call to action for researchers and designers to carefully consider how everyday AR can benefit society and how to avoid potential pitfalls.


Doug Bowman was also honored with the ISMAR Career Impact award, which recognizes a scholar in mixed and augmented reality whose work has had a profound impact on the field over a significant period of time.

ISMAR Conference Committee Chairs

  • Nayara de Oliveira Faria (PhD student, ISE): Workshop on Perceptual and Cognitive Issues in XR (PERCxR), Co-organizer; Diversity, Inclusion, and Accessibility Chair
  • Wallace Lages (Assistant Professor, SOVA): “Pitch Your Lab” Session Chair
  • Sang Won Lee (Assistant Professor, CS): Online Experiences Chair
  • Alex Krasner (MS student, CS): Conference Communication and Planning Channel Organizer
  • Lee Lisle (PhD student, CS): Website Co-chair
  • Cassidy Nelson (PhD candidate, ISE): Mentor/Mentee Program Chair, Student Volunteer Chair

Paper Presentations:

Wallace Lages (Assistant Professor, SOVA), Anthony Steed, University College London, Tuukka M. Takala, Waseda University, Dan Archer, University College London, and Robert W. Lindeman, University of Canterbury

Title: Directions for 3D User Interface Research from Consumer VR Games

Abstract: With the continuing development of affordable immersive virtual reality (VR) systems, there is now a growing market for consumer content. The current form of consumer systems is not dissimilar to the lab-based VR systems of the past 30 years: the primary input mechanism is a head-tracked display and one or two tracked hands with buttons and joysticks on hand-held controllers. Over those 30 years, a very diverse academic literature has emerged that covers design and ergonomics of 3D user interfaces (3DUIs). However, the growing consumer market has engaged a very broad range of creatives that have built a very diverse set of designs. Sometimes these designs adopt findings from the academic literature, but other times they experiment with completely novel or counter-intuitive mechanisms. In this paper and its online adjunct, we report on novel 3DUI design patterns that are interesting from both design and research perspectives: they are highly novel, potentially broadly re-usable and/or suggest interesting avenues for evaluation. The supplemental material, which is a living document, is a crowd-sourced repository of interesting patterns. This paper is a curated snapshot of those patterns that were considered to be the most fruitful for further elaboration.

Multi-screen non-diegetic menu screen in Beat Saber

Workshop Presentation:

Jon Flittner (PhD candidate, ISE)

Title:Adapting Image Analysis Measures of Visual Clutter to Multiple Plane Augmented Reality User Interfaces

Abstract: This study focuses on understanding and quantifying the effects of clutter in multiple plane Augmented Reality User Interfaces (AR UI’s). It determines if image analysis measures of visual clutter can be expanded from single plane AR UI’s to multiple plane AR UI’s as well as examining the effect that multiple plane AR UI’s and clutter has on user performance. Factors that may affect user performance that are analyzed include the number of AR UI planes, plane arrays, and clutter inherent to the target. Image analysis measures of clutter were specifically chosen as they can be applied to complex and naturalistic images that may be found in AR UI applications, and in single plane AR UI’s. The end goal of this research is to develop an algorithm capable of predicting user performance for a given AR UI. In this experiment, twelve participants performed a visual search task of locating a target object in an array of objects where objects were located in planes of different distances. Participants completed this task under two different clutter levels (low and high) against seven different AR UI plane arrays (2m, 4m, 6m, 2m4m, 2m6m, 4m6m, 2m4m6m) where the target was found in every available AR UI plane for each combination, with repetition. Task performance was measured through response time. Results show significant differences in response time between clutter levels, target clutter, and plane array.


Alex Krasner (MS student, CS)

Title: MusiKeys: Investigating Auditory-Physical Feedback Replacement Technique for Mid-air Typing

Abstract: Augmented reality headsets have great potential to transform the modern workplace as the technology improves. However, a major obstacle in bringing AR headsets into workplaces is the need for a precise, virtual, mid-air typing solution. Transitioning from physical to virtual keyboards is difficult due to loss of many physical affordances, such as the ability to tell between touching and pressing a key. We present our system, MusiKeys, as an investigation into the effects of presenting a user with auditory tones and effects as replacements for every kind of feedback that could ordinarily be perceived through touching a keyboard.

Doctoral Consortium:

Leo Pavanatto (PhD student, CS)

Title: Designing Augmented Reality Virtual Displays for Productivity Work

Abstract: We must consider alternative displays for supporting productivity work in the context of an increasingly work-from-home world. Augmented reality virtual monitors can fulfill these needs by equipping users with large screen real estate, while maintaining portability, cost-effectiveness, and not occupying physical space. However, there are open questions regarding how to design virtual monitors. In my dissertation, I plan to investigate the design of virtual monitors to enhance productivity everywhere. This work comprises a group of design and user study contributions. I conducted a user study to understand the feasibility of virtual monitors and their tradeoffs when compared against physical monitors. I further propose investigating the design of static properties and dynamic behaviors that cannot be achieved through physical monitors.