Collaboration in Mixed Reality

Details are updated when publicly available. Contact me for more information.

( Ongoing )

Mixed Reality (MR) is especially suited for guidance during physical tasks where individuals work together to perform actions in the real world. In my dissertation, I explore how MR's unique affordances can help us go "beyond being there" and actively support collaborators’ needs to make it easier to provide and receive guidance. I do so using a (human)communication-first approach where I evaluate interaction techniques that prioritize easing the task of communicating over realism.

MR collaboration
MR-mediated remote collaboration over a physical task - surgery.

The dissertation spans a few projects:

Project MaLi > Uncovering the effects of spatial information and automated guidance to task objects.

This project focuses on referencing - the ability to refer to an object in a way that is understood by others. Being crucial for communication during physical tasks, we evaluated two ways of easing the process of referencing in a 2x2 mixed factorial experiment. We also dive into how aspects of referencing can be offloaded to reduce collaborators' need for spatial information. This work is pulished at CHI 2021 (Read the paper, Watch the talk).

Method#1 - Explicit representation of relevant spatial information.
spatial information.
Method#2 - Guidance to referenced objects.
guidance.

Project UnMapped > Exploiting expert's knowledge in collaborative XR. Details to come.

Project Framework > A communication-focused conceptual framework for XR collaboration. Details to come.

Other Projects > My work on HoloCPR and ARTEMIS are also part of this dissertation.


Publications:

Janet G Johnson, Danilo Gasques, Tommy Sharkey, Evan Schmitz, and Nadir Weibel. Do You Really Need to Know Where “That” Is? Enhancing Support for Referencing in Collaborative Mixed Reality Environments. CHI '21, Yokohama, Japan. [Video, Slides]

Janet G. Johnson, Danilo Gasques Rodrigues, Madhuri Gubbala, and Nadir Weibel. HoloCPR: Designing and Evaluating a Mixed Reality Interface for Time-Critical Emergencies. PervasiveHealth '18, New York, NY USA. [Slides]


Collaborators:

Tommy Sharkey, Evan Schmitz, Danilo Gasques, Cynthia Butarbutar, Danica Xiong, Ren Sy, Nadir Weibel, Mark Billinghurst