In situ Indexing and Query Processing of AMR Data

In situ Indexing and Query Processing of AMR Data

One of the most significant advances for large-scale scientific simulations has been the advent of Adaptive Mesh Refinement, or AMR. This technique allows a simulation to dynamically shift its “computational focus” (with additional resolution) to regions that are “unusual” or “interesting,” without devoting the same level of intensity to more “usual” areas, saving considerable memory, computation, and storage resources, while achieving at least as much fidelity as a comparable non-AMR simulation. Yet, this greatest strength of AMR—selective refinement of salient areas of simulation—is also becoming its biggest weakness: the complex structure of AMR data renders ineffective many existing technologies that support analysis. For instance, current indexing and query processing methods, which enable real-time searches through data for features and trends, are not designed for AMR-structured data. Without such enabling technologies, analysis and visualization of AMR data is greatly hindered. This challenge must be addressed to ensure the sustainability of scientific analysis for current- and next-generation simulations.

Therefore, this project aims to develop a holistic AMR-capable analytics framework to support simulation analysis toward extreme-scale. The proposed framework is a multi-faceted approach that targets AMR-specific challenges. First, the project will develop novel techniques for interactive querying over AMR data, which will yield near-real-time scientific insight to scientists, as well as automatic feedback at simulation runtime, allowing intelligent “steering” in response to detected phenomena appearing in the simulation. Second, the team will design a data model that will allow users (scientists) to more easily and quickly explore and manipulate AMR data. Finally, the work will operate on the unique, adaptive structure of AMR simulations in near-real time by attaching directly to the simulation itself (i.e., “in situ”), and the researchers explicitly address the extreme-scale system requirements throughout their development of the above components.