CICI: SSC: Integrity Introspection for Scientific Workflows (IRIS)

Scientists use computer systems to analyze and store their scientific data, sometimes in a complex process across multiple machines in different geographical locations. It has been observed that sometimes during this complex process, scientific data is unintentionally modified or accidentally tampered with, with errors going undetected and corrupt data becoming part of the scientific record. The IRIS project tackles the problem of detecting and diagnosing these unintentional data errors that might occur during the scientific processing workflow. The approach is to collect data relevant to the correctness and integrity of the scientific data from various parts of the computing and network system involved in the processing, and to analyze the collected data using machine learning techniques to uncover errors in the scientific data processing. The solutions are integrated into Pegasus, a popular 'workflow management system' - a software used to describe the complex process in a user-friendly way and that handles the details of processing for the scientists. The research methods will be validated on national computing resources with exemplar scientific applications from gravitational-wave physics, earthquake science, and bioinformatics. These solutions will allow scientists, and our society, to be more confident of scientific findings based on collected data.

Data-driven science workflows often suffer from unintentional data integrity errors when executing on distributed national cyberinfrastructure (CI). However, today, there is a lack of tools that can collect and analyze integrity-relevant data from workflows and thus, many of these errors go undetected jeopardizing the validity of scientific results. The goal of the IRIS project is to automatically detect, diagnose, and pinpoint the source of unintentional integrity anomalies in scientific workflows executing on distributed CI. The approach is to develop an appropriate threat model and incorporate it in an integrity analysis framework that collects workflow and infrastructure data and uses machine learning (ML) algorithms to perform the needed analysis. The framework is powered by novel ML-based methods developed through experimentation in a controlled testbed and validated in and made broadly available on NSF production CI. The solutions will be integrated into the Pegasus workflow management system, which is used by a wide variety of scientific domains. An important part of the project is the engagement with science application partners in gravitational-wave physics, earthquake science, and bioinformatics to deploy the analysis framework for their workflows, and to iteratively fine tune the threat models, ML model training, and ML model validation in a feedback loop.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.