ASPECT: Advancing Science Performance with Emerging Computer Technologies

Advancing Science Performance with Emerging Computer Technologies (ASPECT) combines Unity (a cross-platform game engine and integrated development environment) with cutting-edge haptic technology to provide upper elementary students with a new way of accessing core science content, reaching beyond what is typically done in today's classrooms. Haptic feedback may engage embodied knowledge that would otherwise lie untapped. This affordance becomes important when one considers the invisible aspects that undergird many school science concepts (e.g. buoyancy, magnetism, and intermolecular forces.) Haptic interfaces provide learners access to invisible forces (often difficult or impossible to create in real-world scenarios) and may help fill gaps in an individual's chain of reasoning about abstract science content. The core research question that undergirds this exploratory project is: How does the addition of haptic feedback influence users' understandings of core, often invisible, science content?

The work is conducted by a cross disciplinary team from North Carolina State University, the Renaissance Computing Institute a local school system. The projecct includes experts in education, computer science, and art and design. The project uses an informant design approach; actively engaging children and local expert STEM teachers in the development and testing of simulations to help students learn about buoyancy, magnetism, and molecular forces emphasizing crosscutting concepts like cause and effect, systems, and energy. The simulations support student scientific inquiry with in-simulation cognitive tools including the just-in-time presentation of ancillary background information and a virtual science notebook planning tool.

Evaluation and testing includes focus groups with children and STEM teacher informants using low-tech versions of the simulations to elicit feedback on artwork, character features, storyboarded instructional sequences, and potential measures of performance and learning. Usability testing generates data on task performance (including success rate, completion time, and workload) user behavior, and user preference and feeds the iterative development process. Preliminary estimates of the impact are being made through a series of small-scale classroom-based pilot tests near the end of the design cycle for each simulation. These pilot studies employ a randomized pre-test-post-test control group research design with convenience samples of 40-60 grade 3-5 students each year. Participants are split into four groups (based on the haptic and visual rendering of the underlying forces being taught): none (just basic rendering of the objects in the simulation with no visual or haptic forces), visual rendering (including visualization of the forces involved), haptic rendering of forces (with no visualization of forces), and visual plus haptic rendering of forces. A mixed-methods approach is used to garner both quantitative and qualitative data regarding subjects' conceptions of the target content. Measures include open-ended questions, drawing tasks, concept mapping, objective close-ended questions, and retrospective probing.

A main thrust ASPECT is to lay the groundwork for a more inclusive cognitive model of how children integrate and use visual and haptic information in multisensory learning environments. The immediate product of ASPECT's exploratory work is the proof-of-concept that haptics can be successfully integrated with the Unity platform to build simulations that enhance and deepen upper elementary students' science learning. The study will also contribute to the development of design guidelines for the haptic-augmentation of science simulations that can be used by other researchers.