EAGER: Data-Driven Control of Power Systems using Structured Reinforcement Learning

Over the foreseeable future, the North American power grid is envisioned to evolve as the most complex Internet-of-Things, generating massive volumes of data from thousands of digital sensors. In this modern grid, conventional generation as well as distributed energy resources (DERs) in the form of renewables, energy storage, smart loads, and power electronic converters are foreseen to serve as active endpoints that react and respond proactively to commands driven by these data. To realize this vision, grid operators today must start planning for an infrastructure that not only seeks to install more sensors and actuators, but also bridges the gap between the two, and enables them to transition from data to control. The challenge, however, lies in dodging the curse of dimensionality in both data volume and controller size. Even the simplest control designs today demand cubic numerical complexity, making it almost impossible to learn them in real-time. The objective of this EAGER project is to take a step forward towards addressing this challenge. The goal is to develop machine learning algorithms that translate large volumes of power system data to real-time control actions without running into unacceptably long convergence times. Structured learning algorithms that can scan, recognize, and extract the most useful attributes of large datasets, and at the same time also identify the most vulnerable parts of the grid that need to controlled using that data will be developed to achieve this goal, reducing computation and control time by significant orders of magnitude.

On the technical front, the project will study two key questions. First, given Synchrophasor data streaming from hundreds of Phasor Measurement Units, are all spectral components of these data essential for taking a given wide-area control action Or does there exist a low-rank structure of the data that may be enough for the control goal? Second, does every actuator in the grid need to be triggered after a disturbance, or does there exist a certain neighborhood around the source of the disturbance, controlling which may be enough to stabilize the grid with a reasonably good performance? Furthermore, can the outline of this neighborhood be estimated from combinations of online Synchrophasor data and offline archived data from the energy management system? We will integrate ideas from structured compressive sensing, model reduction theory, and adaptive dynamic programming to answer these questions from core machine learning and control-theoretic points of view. Our study will address two critical control applications of Synchrophasors, namely (1) hierarchical frequency control using both conventional generators and DERs, and (2) power oscillation damping of major tie-line flows in the presence of large-scale wind and solar penetrations. The study will promote many new directions of theoretical and experimental research in the application of machine learning for tomorrow?'s power system operations. Workshops and conference tutorials will be organized to train graduate students and power system professionals in Synchrophasor data analytics.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.