CHS: Medium: Data-Mediated Communication with Proximal Robots for Emergency Response

Robots may augment emergency response teams by collecting information in environments that may be dangerous or inaccessible for human responders, such as in wildfire fighting, search and rescue, or hurricane response. For example, robots might collect critical visual, mapping, and environmental data to inform responders of conditions ahead that could improve their awareness of the operational environment. These data would assist in planning and re-planning courses of action and enhance in-the-field decision making. However, response teams currently have little ability to directly access robot-collected information in the field, despite its value for rapidly responding to local conditions, because current systems typically route the data through a central command post. This project's goal is to design systems that support more direct access and analysis for first responders while not imposing additional distractions or operational risks through using faulty data. Through collaboration with several local response groups, the project team will develop better understandings of responders' needs and concerns around robot-collected data, algorithms and visualizations that meet those needs using augmented reality technologies, and systems that integrate well with responders' actual work practices. The project will also develop a series of demonstrations, outreach activities, and technology challenges based on the project goals aimed at increasing public interest in science, including among high school students and underrepresented groups in computer science. Overall, this research will develop fundamental knowledge in robotics and visualization, leading to new methods and tools that enable responders to take advantage of robot-collected data while in the field. In particular, this project will explore how see-through augmented reality head-mounted displays (ARHMDs) might offer an intuitive and powerful medium for in situ analysis of robot-collected data through developing an ARHMD system that allows emergency responders to interact with robot-collected information in the contexts of where, when, and how that data was obtained. The team will conduct empirical studies to guide the design of system components that allow responders to actively analyze available data through interactive visualization, passively view digital traces and "data drops" left by robots as they collect information about the environment, and query specific information such as camera feeds on-demand. The team will also develop novel algorithms for 3D scene reconstruction and simultaneous location and mapping that will be useful for a broad variety of applications. Overall, the project will contribute empirical knowledge of how different factors of ARHMD visualizations influence data interpretation, novel algorithms for estimating, correcting, and sharing maps between intermittently-networked agents in the field, and information regarding how data from collocated robots can mediate human-robot interactions, particularly within the context of emergency response.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.