EAGER: Baseline Library of Undergraduate Metrics on Education and Research (BLUMER)

The scale and outcomes of undergraduate research in the US are poorly understood, in part because they are difficult to track and no standardized metrics exist for measuring them. While existing research has utilized surveys of graduates over time to obtain data on this topic, this method is both expensive and necessarily incomplete. Educational institutions and policymakers therefore rely on incomplete information when making policies that impact or promote undergraduate research activities. Given evidence that undergraduate research experiences impact students' willingness to undertake graduate study and careers in STEM, as well as continuing challenges to diversity and equity in these fields, more complete data is needed to inform both educational research and effective policymaking. This project addresses the problem by developing standards for measuring and understanding undergraduate research pathways using untapped data sources, including scientific publications by undergraduates and records maintained by the schools at which they study. The initial research dataset of undergraduate research products and outcomes focuses on undergraduate activities in multiple settings: research-focused universities, historically black colleges and universities (HBCUs), and STEM-focused companies. In addition to broadening the information base on undergraduate research available to educational institutions, policymakers, and researchers, this project advances new methods in educational data collection, data management, and dissemination. Preserving the privacy and security of educational records is extremely important and represents a technical challenge for researchers trying to collect and disseminate such data at scale. This project draws inspiration from previous innovations in health data systems that both preserve privacy and allow for responsible research on large-scale real-world data sets.

Project aims will be accomplished through the collaborative development and refinement of a data collection protocol, implementation of the data collection protocol across partner institutions, and initial development of a secure, compliant, and usable system for storing and working with data. A systematic literature review on methods for evaluating undergraduate research activities, outputs, and subsequent student outcomes will be conducted at the start of the project. Next, interviews will be conducted with appropriate staff at partner institutions to understand and evaluate their existing practices for tracking undergraduate research. Based on the information gathered through these steps, metrics to answer important questions about undergraduate research outcomes along with a corresponding data collection protocol will be jointly developed with partner institutions. An initial dataset will be collected using the developed protocol, focusing on undergraduate research in the fields of computer science and engineering. The policies, procedures, and research methods found to be most suitable to this work will be published to enable more effective and responsible research of undergraduate activity by the broader research community. Finally, requirements for a system to store and disseminate the data will be developed with the goal of providing a platform for future data collection and research at a national scale, across all undergraduate majors.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.