EAGER: Mixed Reality for Improved Science Reading Comprehension

This EArly grant for Exploratory Research (EAGER) is a partnership between East Carolina University (ECU), Spark Plug Games, Cumberland County Schools, and Teresa C Berrien Elementary School that will develop and perform preliminary testing of mixed reality (MR) to improve understanding of academic language in elementary science text. Understanding science vocabulary is crucial for elementary school students to learn and enjoy science but many students fail to fully comprehend fundamental science vocabulary needed for scientific literacy. Mixed reality (MR) is an extension of augmented reality (AR) in that the virtual objects and experiences overlay or are superimposed on the physical world. In MR, the user can interact with the virtual objects as well as the physical world. This project plans technology that will activate virtual content in science texts at the appropriate time to assist the learner in reading and understanding new vocabulary. The MR technology will make the science content of a science reading module come alive with 3D images, moving diagrams, and sound. The goal is to improve students' comprehension of science text by using mixed reality (MR) to develop engaging personalized learning systems. The project is an exploratory study to develop and conduct preliminary tests of a prototype system meant to demonstrate its usefulness and effectiveness. It will be determined what is needed to design and develop MR learning environments, specifically focused on improving upper elementary grade students' science reading comprehension. The project will lead the way for development of quality MR learning experiences and eventually for affordable accessible products suitable for a broader implementation in elementary schools, libraries, and homes across the world.

Utilizing both qualitative and quantitative data in an iterative approach to design and development, this EAGER grant is poised to create and test a working prototype built on the Unity Game Engine and deployed through Microsoft HoloLens glasses. A reading module will be chosen that closely aligns with the content in the North Carolina Standard Course of Study for 5th grade science and conveys challenging abstract concepts with academic language and difficult vocabulary. The technology will activate content in science texts at the appropriate time to assist the learner in reading and understanding new vocabulary. The appropriate time will be operationally defined as when the reader's eye track over a predetermined science vocabulary term (e.g., Potential Energy). This part of the effort will require the creation of a software integration layer by Sparkplug Games to stitch together several proven but discrete technologies into one system. The head mounted display will monitor the student's direction and duration of gaze to inform the system of what content is currently important. The research will take place in two phases. In the first phase, hardware and software will be integrated to create the MR enhanced science module. This phase will include collecting student data (read and think aloud) and curriculum specialist/teacher interviews to inform the enhancement of the text (e.g. which concepts should be enhanced, how much explanation is helpful, how does the student activate and deactivate further explanations). Measures will include 'Think Aloud' and researcher observations to determine which vocabulary and concepts are most difficult for students to process and interviews with teachers and district personnel to align the materials to North Carolina and other standards. The second phase will test and iteratively improve the prototype MR experience with 5th grade students (N = 20-30) to ensure that the technology works smoothly and provides an accessible and engaging experience for students reading below, at, and above grade level. Measures will include 'Think Aloud' use of MR enhanced text, observation to compare reading with and without MR enhanced text, student and teacher interviews, and a pre- and post- Oral Reading Fluency test analyzed using non-parametric statistical measures. Diverse participants will be selected to test the prototype with individuals reflecting the spectrum of reading skills, gender, racial, and ethnic diversity of the 5th graders. Fifth grade teachers and other school and district staff such as instructional coaches and administrators will also participate -- their perspectives will be solicited during the design process in order to create an MR experience that aligns to the content they are expected to teach and is workable in terms of classroom implementation. This project is expected to have a broad societal impact in three ways. First, it will develop a framework for using data to develop MR materials that bridge science learning and interest into effective elementary literacy practices and supplement traditional science instructional tools (e.g. kits), which have been difficult to implement effectively in some environments. Second, this project aims to develop classroom tools for teachers to use emerging technology in exciting ways, increasing engagement and improving learning outcomes for diverse students. Third, this project aims to develop skills that are fundamental to success in future STEM learning.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.