CHS: Small: Collaborative Research: 3D Audio Augmentation for Limited Field of View Augmented Reality Systems for Medical Training

Medical task simulators can provide a safe, affordable, and repeatable environment in which practitioners can rehearse procedures without impacting patient safety. Augmented reality (AR) is therefore the ideal display technology in this field, allowing the user to directly interact and practice skills within a natural environment. But current AR displays typically have a narrow field of view (FOV), which makes it difficult for users to immediately attend to an object outside of their periphery. This research will employ 3D audio to overcome that challenge. As a benefit to other AR and medical researchers, the system itself and other materials created for the present work (design, implementation, and tutorials) will be openly available on the project's website for all to use, modify, and contribute, and an open-source community will be created to link users and researchers who have similar interests as the present work. Undergraduate, women, and minority students will be engaged in all project activities through the Distributed Research Experiences for Undergraduates (DREU) program of the NSF-funded iAAMCS (Institute for African-American Mentoring in Computing Sciences), as well as the PI's courses on 3D sound design. Results will also be disseminated at relevant conference venues. The present work will advance research relating to the use of 3D audio for cueing in narrow FOV contexts, thereby improving interaction in large AR environments, and will create generalizable best-practices that lead to successful experiences when using AR devices.

This research will develop the tools, methods, and infrastructure to evaluate how 3D sound can be used to enhance AR. This area of research is increasingly important, because as virtual reality (VR) and AR systems become more commonplace, it is imperative to design tools to overcome device limitations. The practical application of the proposed work will be realized through the evaluation of an AR-based prostate biopsy training procedure that will capture and reconstruct a full surgical procedure, with at least 3 dynamic participants in very close proximity. The project will pursue three main themes: (1) Infrastructure Development - The PIs and team will develop an extensible open-source software system that allows users to test their desired sound mappings; (2) Sound Mapping Quantification - Although sound has been used to convey spatial information in numerous contexts, the appropriate strategy for mapping periphery information to spatial sound attributes has not yet been determined, so the PIs propose a series of user studies to determine the mappings that most successfully help a user to attend to a target outside of their FOV; (3) Practical Application - To assess how well the proposed solution mitigates the challenge of a narrow FOV, a user study will be conducted to determine how the addition of 3D audio affects participants learning to perform the steps of a prostate biopsy procedure.