Research Projects

We build controllers that act directly from data while retaining performance guarantees. Nonparametric chain policies stitch together short, locally certified trajectory segments drawn from stored data to practically stabilize nonlinear systems, with explicit sample-complexity bounds and the ability to absorb new data without retraining. The same idea accelerates model predictive control: a nonparametric policy built offline from MPC solutions reproduces near-optimal behavior between one hundred and one thousand times faster than solving the MPC online, with a provable bound on the optimality gap. Related work (currently under review) establishes when sampling-based reachability is tractable, and how physical structure such as a Hamiltonian reduces the data a controller needs.

Related publications:

1. A. Castellano, S. Pan, and E. Mallada, “Data-driven Acceleration of MPC with Guarantees,” in Proceedings of The 8th Annual Learning for Dynamics and Control Conference, 2026.
   [BibTeX] [Abstract] [Download PDF]
   

   Model Predictive Control (MPC) is a powerful framework for optimal control but can be too slow for low-latency applications. We present a data-driven framework to accelerate MPC by replacing online optimization with a nonparametric policy constructed from offline MPC solutions. Our policy is greedy with respect to a constructed upper bound on the optimal cost-to-go, and can be implemented as a nonparametric lookup rule that is orders of magnitude faster than solving MPC online. Our analysis shows that under sufficient coverage condition of the offline data, the policy is recursively feasible and admits provable, bounded optimality gap. These conditions establish an explicit trade-off between the amount of data collected and the tightness of the bounds. Our experiments show that this policy is between $100$ and $1000$ times faster than standard MPC, with only a modest hit to optimality, showing potential for real-time control tasks.

   @inproceedings{cpm2026l4dc,
     abstract = {Model Predictive Control (MPC) is a powerful framework for optimal control but can be too slow for low-latency applications. We present a data-driven framework to accelerate MPC by replacing online optimization with a nonparametric policy constructed from offline MPC solutions. Our policy is greedy with respect to a constructed upper bound on the optimal cost-to-go, and can be implemented as a nonparametric lookup rule that is orders of magnitude faster than solving MPC online. Our analysis shows that under sufficient coverage condition of the offline data, the policy is recursively feasible and admits provable, bounded optimality gap. These conditions establish an explicit trade-off between the amount of data collected and the tightness of the bounds. Our experiments show that this policy is between $100$ and $1000$ times faster than standard MPC, with only a modest hit to optimality, showing  potential for real-time control tasks.},
     author = {Castellano, Agustin and Pan, Shijie and Mallada, Enrique},
     booktitle = {Proceedings of The 8th Annual Learning for Dynamics and Control Conference},
     grants = {Global-Centers-2330450; DOE-ASCR-826565},
     month = {6},
     organization = {PMLR},
     pubstate = {accepted},
     record = {accepted Mar. 2026, submitted Nov. 2025},
     title = {Data-driven Acceleration of MPC with Guarantees},
     url = {https://mallada.ece.jhu.edu/pubs/2026-L4DC-CPM.pdf},
     year = {2026}
   }

2. R. Siegelmann and E. Mallada, “Data-driven Practical Stabilization of Nonlinear Systems via Chain Policies: Sample Complexity and Incremental Learning,” in American Control Conference (ACC), 2026, pp. 1-8.
   [BibTeX] [Abstract] [Download PDF]
   

   We propose a method for data-driven practical stabilization of nonlinear systems with provable guarantees, based on the concept of \emphNonparametric Chain Policies (NCPs). The approach employs a normalized nearest-neighbor rule to assign, at each state, a finite-duration control signal derived from stored data, after which the process repeats. Unlike recent works that model the system as linear, polynomial, or polynomial fraction, we only assume the system to be locally Lipschitz. Our analysis build son the framework of Recurrent Lyapunov Functions (RLFs), which enable data-driven certification of (practical) stability using standard norm functions instead of requiring the explicit construction of a classical Lyapunov function. To extend this framework, we introduce the concept of Recurrent Control Lyapunov Functions (R-CLFs), which can certify the existence of an NCP that practically stabilizes an arbitrarily small $c$-neighborhood of an equilibrium point. We also provide an explicit sample complexity guarantee of $\mathcalO\!łeft((3/h̊o)^d łog(R/c)\g̊ht)$ number of trajectories—where $R$ is the domain radius, $d$ the state dimension, and $\r$̊ a system-dependent constant. The proposed Chain Policies are nonparametric, thus allowing new verified data to be readily incorporated into the policy to either improve convergence rate or enlarge the certified region. Numerical experiments illustrate and validate these properties.

   @inproceedings{sm2026acc,
     abstract = {We propose a method for data-driven practical stabilization of nonlinear systems with provable guarantees, based on the concept of \emphNonparametric Chain Policies (NCPs). The approach employs a normalized nearest-neighbor rule to assign, at each state, a finite-duration control signal derived from stored data, after which the process repeats. 
   Unlike recent works that model the system as linear, polynomial, or polynomial fraction, we only assume the system to be locally Lipschitz.
   Our analysis build son the framework of Recurrent Lyapunov Functions (RLFs), which enable data-driven certification of (practical) stability using standard norm functions instead of requiring the explicit construction of a classical Lyapunov function. To extend this framework, we introduce the concept of Recurrent Control Lyapunov Functions (R-CLFs), which can certify the existence of an NCP that practically stabilizes an arbitrarily small $c$-neighborhood of an equilibrium point. 
   We also provide an explicit sample complexity guarantee of $\mathcalO\!łeft((3/h̊o)^d łog(R/c)\g̊ht)$ number of trajectories---where $R$ is the domain radius, $d$ the state dimension, and $\r$̊ a system-dependent constant. The proposed Chain Policies are nonparametric, thus allowing new verified data to be readily incorporated into the policy to either improve convergence rate or enlarge the certified region. Numerical experiments illustrate and validate these properties.},
     author = {Siegelmann, Roy and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.23919/ACC55779.2023.10156212},
     booktitle = {American Control Conference (ACC)},
     grants = {Global-Centers-2330450; DOE-ASCR-826565},
     month = {5},
     pages = {1-8},
     pubstate = {accepted},
     record = {accepted Feb. 2026, submitted Sep. 2025},
     title = {Data-driven Practical Stabilization of Nonlinear Systems via Chain Policies: Sample Complexity and Incremental Learning},
     url = {https://mallada.ece.jhu.edu/pubs/2026-ACC-SgM.pdf},
     year = {2026}
   }

Reinforcement learning is hard to trust when safety matters, because guarantees usually hold only in expectation or after unboundedly many failures. We study learning under almost-sure constraints, where a violation must never occur with probability one, and show that safety can be learned in finite time once one accepts explicit trade-offs between optimality, exposure to danger, and detection delay. A binary Bellman operator lets an agent learn the maximal safe region of the state space directly from data, and a nonparametric policy-improvement theorem extends classical guarantees to continuous action spaces using only demonstrations. Complementary results give variance-aware exploration guarantees for neural contextual bandits.

Related publications:

1. H. M. Bui, E. Mallada, and A. Liu, “Variance-Aware Linear UCB with Deep Representation for Neural Contextual Bandits,” in International Conference on Artificial Intelligence and Statistics (AISTATS), 2025.
   [BibTeX] [Abstract] [Download PDF]
   

   By leveraging the representation power of deep neural networks, neural upper confidence bound (UCB) algorithms have shown success in contextual bandits. To further balance the exploration and exploitation, we propose Neural-$σ^2$-LinearUCB, a variance-aware algorithm that utilizes $σ^2_t$, i.e., an upper bound of the reward noise variance at round $t$, to enhance the uncertainty quantification quality of the UCB, resulting in a regret performance improvement. We provide an oracle version for our algorithm characterized by an oracle variance upper bound $σ^2_t$ and a practical version with a novel estimation for this variance bound. Theoretically, we provide rigorous regret analysis for both versions and prove that our oracle algorithm achieves a better regret guarantee than other neural-UCB algorithms in the neural contextual bandits setting. Empirically, our practical method enjoys a similar computational efficiency, while outperforming state-of-the-art techniques by having a better calibration and lower regret across multiple standard settings, including on the synthetic, UCI, MNIST, and CIFAR-10 datasets.

   @inproceedings{bml2025aistats,
     abstract = {By leveraging the representation power of deep neural networks, neural upper confidence bound (UCB) algorithms have shown success in contextual bandits. To further balance the exploration and exploitation, we propose Neural-$σ^2$-LinearUCB, a variance-aware algorithm that utilizes $σ^2_t$, i.e., an upper bound of the reward noise variance at round $t$, to enhance the uncertainty quantification quality of the UCB, resulting in a regret performance improvement. We provide an oracle version for our algorithm characterized by an oracle variance upper bound $σ^2_t$ and a practical version with a novel estimation for this variance bound. Theoretically, we provide rigorous regret analysis for both versions and prove that our oracle algorithm achieves a better regret guarantee than other neural-UCB algorithms in the neural contextual bandits setting. Empirically, our practical method enjoys a similar computational efficiency, while outperforming state-of-the-art techniques by having a better calibration and lower regret across multiple standard settings, including on the synthetic, UCI, MNIST, and CIFAR-10 datasets.},
     author = {Bui, Ha Manh and Mallada, Enrique and Liu, Anqi},
     booktitle = {International Conference on Artificial Intelligence and Statistics (AISTATS)},
     grants = {No Grant},
     month = {4},
     publisher = {PMLR},
     record = {accepted Jan 2025, submitted Oct 2024},
     series = {Proceedings of Machine Learning Research},
     title = {Variance-Aware Linear UCB with Deep Representation for Neural Contextual Bandits},
     url = {https://mallada.ece.jhu.edu/pubs/2025-AISTATS-BML.pdf},
     year = {2025}
   }

2. A. Castellano, S. Rezaei, J. Markowitz, and E. Mallada, “Nonparametric Policy Improvement in Continuous Action Spaces via Expert Demonstrations,” in Reinforcement Learning Conference, 2025, pp. 1158-1179.
   [BibTeX] [Abstract] [Download PDF]
   

   The policy improvement theorem is a fundamental building block of classical reinforcement learning for discrete action spaces. Unfortunately, the lack of an analogous result for continuous action spaces with function approximation has historically limited theoretical guarantees of policy optimization algorithms, undermining their reliability. Here, we introduce a novel nonparametric policy that relies purely on data to take actions and that admits a policy improvement theorem for deterministic Markov Decision Processes (MDPs). By imposing mild regularity assumptions on the optimal policy, we show that, when data come from expert demonstrations, one can construct a nonparametric lower bound on the value of the policy, thus enabling its robust evaluation. The constructed lower bound naturally leads to a simple improvement mechanism based on adding more demonstrations. We also provide conditions to identify regions of the state space where additional demonstrations are needed to meet specific performance goals. Finally, we propose a policy optimization algorithm that ensures a monotonic improvement of the lower bound and leads to high probability performance guarantees. These contributions provide a foundational step toward establishing a rigorous framework for policy improvement in continuous action spaces.

   @inproceedings{crmm2025rlc,
     abstract = {The policy improvement theorem is a fundamental building block of classical reinforcement learning for discrete action spaces. Unfortunately, the lack of an analogous result for continuous action spaces with function approximation has historically limited theoretical guarantees of policy optimization algorithms, undermining their reliability. Here, we introduce a novel nonparametric policy that relies purely on data to take actions and that admits a 
   policy improvement theorem for deterministic Markov Decision Processes (MDPs). By imposing mild regularity assumptions on the optimal policy, we show that, when data come from expert demonstrations, one can construct a nonparametric lower bound on the value of the policy, thus enabling its robust evaluation. The constructed lower bound naturally leads to a simple improvement mechanism based on adding more demonstrations. We also provide conditions to identify regions of the state space where additional demonstrations are needed to meet specific performance goals. Finally, we propose a policy optimization algorithm that ensures a monotonic improvement of the lower bound and leads to high probability performance guarantees. These contributions provide a foundational step toward establishing a rigorous framework for policy improvement in continuous action spaces.},
     author = {Castellano, Agustin and Rezaei, Sohrab and Markowitz, Jared and Mallada, Enrique},
     booktitle = {Reinforcement Learning Conference},
     month = {8},
     pages = {1158-1179},
     record = {presented Aug. 2025, accepted May 2025, submitted Feb. 2025},
     title = {Nonparametric Policy Improvement in Continuous Action Spaces via Expert Demonstrations},
     url = {https://mallada.ece.jhu.edu/pubs/2025-RLC-CRMM.pdf},
     year = {2025}
   }

3. A. Castellano, H. Min, J. Bazerque, and E. Mallada, “Learning safety critics via a non-contractive binary Bellman operator,” in 57th Asilomar Conference on Signals, Systems, and Computers, 2023, pp. 814-821. doi:10.1109/IEEECONF59524.2023.10476995
   [BibTeX] [Abstract] [Download PDF]
   

   The inability to naturally enforce safety in Reinforcement Learning (RL), with limited failures, is a core challenge impeding its use in real-world applications. One notion of safety of vast practical relevance is the ability to avoid (unsafe) regions of the state space. Though such a safety goal can be captured by an action-value-like function, a.k.a. safety critics, the associated operator lacks the desired contraction and uniqueness properties that the classical Bellman operator enjoys. In this work, we overcome the non-contractiveness of safety critic operators by leveraging that safety is a binary property. To that end, we study the properties of the binary safety critic associated with a deterministic dynamical system that seeks to avoid reaching an unsafe region. We formulate the corresponding binary Bellman equation (B2E) for safety and study its properties. While the resulting operator is still non-contractive, we fully characterize its fixed points representing–except for a spurious solution–maximal persistently safe regions of the state space that can always avoid failure. We provide an algorithm that, by design, leverages axiomatic knowledge of safe data to avoid spurious fixed points.

   @inproceedings{cmbm2023asilomar,
     abstract = {The inability to naturally enforce safety in Reinforcement Learning (RL), with limited failures, is a core challenge impeding its use in real-world applications. One notion of safety of vast practical relevance is the ability to avoid (unsafe) regions of the state space. Though such a safety goal can be captured by an action-value-like function, a.k.a. safety critics, the associated operator lacks the desired contraction and uniqueness properties that the classical Bellman operator enjoys. In this work, we overcome the non-contractiveness of safety critic operators by leveraging that safety is a binary property. To that end, we study the properties of the binary safety critic associated with a deterministic dynamical system that seeks to avoid reaching an unsafe region. We formulate the corresponding binary Bellman equation (B2E) for safety and study its properties. While the resulting operator is still non-contractive, we fully characterize its fixed points representing--except for a spurious solution--maximal persistently safe regions of the state space that can always avoid failure. We provide an algorithm that, by design, leverages axiomatic knowledge of safe data to avoid spurious fixed points.},
     author = {Castellano, Agustin and Min, Hancheng and Bazerque, Juan and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.1109/IEEECONF59524.2023.10476995},
     booktitle = {57th Asilomar Conference on Signals, Systems, and Computers},
     doi = {10.1109/IEEECONF59524.2023.10476995},
     grants = {CAREER-1752362, CPS-2136324, Global-Centers-2330450},
     month = {11},
     organization = {IEEE},
     pages = {814--821},
     record = {presented Nov 2023, accepted Sep 2023, submitted Apr 2023},
     title = {Learning safety critics via a non-contractive binary Bellman operator},
     url = {https://mallada.ece.jhu.edu/pubs/2023-Asilomar-CMBM.pdf},
     year = {2023}
   }

4. A. Castellano, H. Min, J. Bazerque, and E. Mallada, “Learning to Act Safely with Limited Exposure and Almost Sure Certainty,” IEEE Transactions on Automatic Control, vol. 68, iss. 5, pp. 2979-2994, 2023. doi:10.1109/TAC.2023.3240925
   [BibTeX] [Abstract] [Download PDF]
   

   This paper aims to put forward the concept that learning to take safe actions in unknown environments, even with probability one guarantees, can be achieved without the need for an unbounded number of exploratory trials, provided that one is willing to navigate trade-offs between optimality, level of exposure to unsafe events, and the maximum detection time of unsafe actions. We illustrate this concept in two complementary settings. We first focus on the canonical multi-armed bandit problem and seek to study the intrinsic trade-offs of learning safety in the presence of uncertainty. Under mild assumptions on sufficient exploration, we provide an algorithm that provably detects all unsafe machines in an (expected) finite number of rounds. The analysis also unveils a trade-off between the number of rounds needed to secure the environment and the probability of discarding safe machines. We then consider the problem of finding optimal policies for a Markov Decision Process (MDP) with almost sure constraints. We show that the (action) value function satisfies a barrier-based decomposition which allows for the identification of feasible policies independently of the reward process. Using this decomposition, we develop a Barrier-learning algorithm, that identifies such unsafe state-action pairs in a finite expected number of steps. Our analysis further highlights a trade-off between the time lag for the underlying MDP necessary to detect unsafe actions, and the level of exposure to unsafe events. Simulations corroborate our theoretical findings, further illustrating the aforementioned trade-offs, and suggesting that safety constraints can further speed up the learning process.

   @article{cmbm2023tac,
     abstract = {This paper aims to put forward the concept that learning to take safe actions in unknown environments, even with probability one guarantees, can be achieved without the need for an unbounded number of exploratory trials, provided that one is willing to navigate trade-offs between optimality, level of exposure to unsafe events, and the maximum detection time of unsafe actions. We illustrate this concept in two complementary settings. We first focus on the canonical multi-armed bandit problem and seek to study the intrinsic trade-offs of learning safety in the presence of uncertainty.  Under mild assumptions on sufficient exploration, we provide an algorithm that provably detects all unsafe machines in an (expected) finite number of rounds. The analysis also unveils a trade-off between the number of rounds needed to secure the environment and the probability of discarding safe machines.  We then consider the problem of finding optimal policies for a Markov Decision Process (MDP) with almost sure constraints. 
   We show that the (action) value function satisfies a barrier-based decomposition which allows for the identification of feasible policies independently of the reward process. Using this decomposition, we develop a Barrier-learning algorithm, that identifies such unsafe state-action pairs in a finite expected number of steps. Our analysis further highlights a trade-off between the time lag for the underlying MDP necessary to detect unsafe actions, and the level of exposure to unsafe events. Simulations corroborate our theoretical findings, further illustrating the aforementioned trade-offs, and suggesting that safety constraints can further speed up the learning process.},
     author = {Castellano, Agustin and Min, Hancheng and Bazerque, Juan and Mallada, Enrique},
     bdsk-url-3 = {https://mallada.ece.jhu.edu/pubs/2023-TAC-CMBM.pdf},
     bdsk-url-4 = {http://dx.doi.org/10.1109/TAC.2023.3240925},
     doi = {10.1109/TAC.2023.3240925},
     grants = {CAREER-1752362, TRIPODS-1934979, CPS-2136324},
     journal = {IEEE Transactions on Automatic Control},
     month = {5},
     number = {5},
     pages = {2979-2994},
     record = {published, online May 2023, accepted Jan 2023, revised Oct 2022, submitted May 2021},
     title = {Learning to Act Safely with Limited Exposure and Almost Sure Certainty},
     url = {https://mallada.ece.jhu.edu/pubs/2023-TAC-CMBM.pdf},
     volume = {68},
     year = {2023}
   }

5. A. Castellano, H. Min, J. Bazerque, and E. Mallada, “Reinforcement Learning with Almost Sure Constraints,” in Proceedings of The 4th Annual Learning for Dynamics and Control Conference, 2022, pp. 559-570.
   [BibTeX] [Abstract] [Download PDF]
   

   In this work we address the problem of finding feasible policies for Constrained Markov Decision Processes under probability one constraints. We argue that stationary policies are not sufficient for solving this problem, and that a rich class of policies can be found by endowing the controller with a scalar quantity, so called budget, that tracks how close the agent is to violating the constraint. We show that the minimal budget required to act safely can be obtained as the smallest fixed point of a Bellman-like operator, for which we analyze its convergence properties. We also show how to learn this quantity when the true kernel of the Markov decision process is not known, while providing sample-complexity bounds. The utility of knowing this minimal budget relies in that it can aid in the search of optimal or near-optimal policies by shrinking down the region of the state space the agent must navigate. Simulations illustrate the different nature of probability one constraints against the typically used constraints in expectation.

   @inproceedings{cmbm2022l4dc,
     abstract = {In this work we address the problem of finding feasible policies for Constrained Markov Decision Processes under probability one constraints. We argue that stationary policies are not sufficient for solving this problem, and that a rich class of policies can be found by endowing the controller with a scalar quantity, so called budget, that tracks how close the agent is to violating the constraint. We show that the minimal budget required to act safely can be obtained as the smallest fixed point of a Bellman-like operator, for which we analyze its convergence properties. We also show how to learn this quantity when the true kernel of the Markov decision process is not known, while providing sample-complexity bounds. The utility of knowing this minimal budget relies in that it can aid in the search of optimal or near-optimal policies by shrinking down the region of the state space the agent must navigate. Simulations illustrate the different nature of probability one constraints against the typically used constraints in expectation.},
     author = {Castellano, Agustin and Min, Hancheng and Bazerque, Juan and Mallada, Enrique},
     booktitle = {Proceedings of The 4th Annual Learning for Dynamics and Control Conference},
     grants = {CAREER-1752362, TRIPODS-1934979, CPS-2136324},
     month = {6},
     organization = {PMLR},
     pages = {559--570},
     record = {presented Jun. 2022, accepted Feb. 2022, submitted Dec. 2021},
     series = {Proceedings of Machine Learning Research},
     title = {Reinforcement Learning with Almost Sure Constraints},
     url = {https://mallada.ece.jhu.edu/pubs/2022-L4DC-CMBM.pdf},
     volume = {168},
     year = {2022}
   }

6. A. Castellano, J. Bazerque, and E. Mallada, “Learning to be safe, in finite time,” in American Control Conference (ACC), 2021, pp. 909-916. doi:10.23919/ACC50511.2021.9482829
   [BibTeX] [Abstract] [Download PDF]
   

   This paper aims to put forward the concept that learning to take safe actions in unknown environments, even with probability one guarantees, can be achieved without the need for an unbounded number of exploratory trials, provided that one is willing to relax its optimality requirements mildly. We focus on the canonical multi-armed bandit problem and seek to study the exploration-preservation trade-off intrinsic within safe learning. More precisely, by defining a handicap metric that counts the number of unsafe actions, we provide an algorithm for discarding unsafe machines (or actions), with probability one, that achieves constant handicap. Our algorithm is rooted in the classical sequential probability ratio test, redefined here for continuing tasks. Under standard assumptions on sufficient exploration, our rule provably detects all unsafe machines in an (expected) finite number of rounds. The analysis also unveils a trade-off between the number of rounds needed to secure the environment and the probability of discarding safe machines. Our decision rule can wrap around any other algorithm to optimize a specific auxiliary goal since it provides a safe environment to search for (approximately) optimal policies. Simulations corroborate our theoretical findings and further illustrate the aforementioned trade-offs.

   @inproceedings{cbm2021acc,
     abstract = {This paper aims to put forward the concept that learning to take safe actions in unknown environments, even with probability one guarantees, can be achieved without the need for an unbounded number of exploratory trials, provided that one is willing to relax its optimality requirements mildly. We focus on the canonical multi-armed bandit problem and seek to study the exploration-preservation trade-off intrinsic within safe learning. More precisely, by defining a handicap metric that counts the number of unsafe actions, we provide an algorithm for discarding unsafe machines (or actions), with probability one, that achieves constant handicap.
   Our algorithm is rooted in the classical sequential probability ratio test, redefined here for continuing tasks.  Under standard assumptions on sufficient exploration, our rule provably detects all unsafe machines in an (expected) finite number of rounds. The analysis also unveils a trade-off between the number of rounds needed to secure the environment and the probability of discarding safe machines. Our decision rule can wrap around any other algorithm to optimize a specific auxiliary goal since it provides a safe environment to search for (approximately) optimal policies. Simulations corroborate our theoretical findings and further illustrate the aforementioned trade-offs.},
     author = {Castellano, Agustin and Bazerque, Juan and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.23919/ACC50511.2021.9482829},
     booktitle = {American Control Conference (ACC)},
     doi = {10.23919/ACC50511.2021.9482829},
     grants = {CPS-1544771, CAREER-1752362, TRIPODS-1934979},
     month = {5},
     pages = {909-916},
     record = {submitted Sep. 2020, accepted Jan. 2021},
     title = {Learning to be safe, in finite time},
     url = {https://mallada.ece.jhu.edu/pubs/2021-ACC-CBM.pdf},
     year = {2021}
   }

Many problems in optimization and game theory can be solved by following the flow of a dynamical system toward its equilibrium. We study saddle-flow dynamics for convex-concave problems and give an observability-based certificate for convergence that generalizes the usual strict convexity-concavity assumptions, requiring minimal conditions on the objective. Taking a control-theoretic view of these dynamics, we design dissipative and accelerated variants that damp the oscillations which plague gradient descent-ascent on min-max problems, achieving faster, provable convergence on bilinear and constrained problems, with applications to constrained reinforcement learning.

Related publications:

1. Y. Liu, E. Mallada, Z. Li, and P. You, “Accelerated Saddle Flow Dynamics for Bilinearly Coupled Minimax Problems,” in 63rd IEEE Conference on Decision and Control (CDC), 2024. doi:10.1109/CDC56724.2024.10886124
   [BibTeX] [Abstract] [Download PDF]
   

   Minimax problems have attracted much attention due to various applications in constrained optimization problems and zero-sum games. Identifying saddle points within these problems is crucial, and saddle flow dynamics offer a straightforward yet useful approach. This study focuses on a class of bilinearly coupled minimax problems and designs an accelerated algorithm based on saddle flow dynamics that achieves a convergence rate beyond stereotype limits. The algorithm is derived based on a sequential two-step transformation of the objective function. First, a change of variable is aimed at a better-conditioned saddle function. Second, a proximal regularization, when staggered with the first step, guarantees strong convexity-strong concavity of the objective function that can be tuned for accelerated exponential convergence. Besides, such an approach can be extended to a class of weakly convex-weakly concave functions and still achieve exponential convergence to one stationary point. The theory is verified by a numerical test on an affine equality-constrained convex optimization problem.

   @inproceedings{lmly2024cdc,
     abstract = {Minimax problems have attracted much attention due to various applications in constrained optimization problems and zero-sum games. Identifying saddle points within these problems is crucial, and saddle flow dynamics offer a straightforward yet useful approach. This study focuses on a class of bilinearly coupled minimax problems and designs an accelerated algorithm based on saddle flow dynamics that achieves a convergence rate beyond stereotype limits.  The algorithm is derived based on a sequential two-step transformation of the objective function. First, a change of variable is aimed at a better-conditioned saddle function. Second, a proximal regularization, when staggered with the first step, guarantees strong convexity-strong concavity of the objective function that can be tuned for accelerated exponential convergence.
   Besides, such an approach can be extended to a class of weakly convex-weakly concave functions and still achieve exponential convergence to one stationary point. The theory is verified by a numerical test on an affine equality-constrained convex optimization problem.},
     author = {Liu, Yingzhu and Mallada, Enrique and Li, Zhongkui and You, Pengcheng},
     booktitle = {63rd IEEE Conference on Decision and Control (CDC)},
     doi = {10.1109/CDC56724.2024.10886124},
     grants = {CPS-2136324, Global-Centers-2330450},
     month = {07},
     record = {presented Dec. 2024, accepted Jul. 2024, submitted Mar. 2024},
     title = {Accelerated Saddle Flow Dynamics for Bilinearly Coupled Minimax Problems},
     url = {https://mallada.ece.jhu.edu/pubs/2024-Preprint-LMLY.pdf},
     year = {2024}
   }

2. T. Zheng, N. Loizou, P. You, and E. Mallada, “Dissipative Gradient Descent Ascent Method: A Control Theory Inspired Algorithm for Min-max Optimization,” IEEE Control Systems Letters (L-CSS), vol. 8, pp. 2009-2014, 2024. doi:10.1109/LCSYS.2024.3413004
   [BibTeX] [Abstract] [Download PDF]
   

   Gradient Descent Ascent (GDA) methods for min-max optimization problems typically produce oscillatory behavior that can lead to instability, e.g., in bilinear settings. To address this problem, we introduce a dissipation term into the GDA updates to dampen these oscillations. The proposed Dissipative GDA (DGDA) method can be seen as performing standard GDA on a state-augmented and regularized saddle function that does not strictly introduce additional convexity/concavity. We theoretically show the linear convergence of DGDA in the bilinear and strongly convex-strongly concave settings and assess its performance by comparing DGDA with other methods such as GDA, Extra-Gradient (EG), and Optimistic GDA. Our findings demonstrate that DGDA surpasses these methods, achieving superior convergence rates. We support our claims with two numerical examples that showcase DGDA’s effectiveness in solving saddle point problems.

   @article{zlym2024lcss,
     abstract = {Gradient Descent Ascent (GDA) methods for min-max optimization problems typically produce oscillatory behavior that can lead to instability, e.g., in bilinear settings.
   To address this problem, we introduce a dissipation term into the GDA updates to dampen these oscillations. The proposed Dissipative GDA (DGDA) method can be seen as performing standard GDA on a state-augmented and regularized saddle function that does not strictly introduce additional convexity/concavity. We theoretically show the linear convergence of DGDA in the bilinear and strongly convex-strongly concave settings and assess its performance by comparing DGDA with other methods such as GDA, Extra-Gradient (EG), and Optimistic GDA.
   Our findings demonstrate that DGDA surpasses these methods, achieving superior convergence rates. We support our claims with two numerical examples that showcase DGDA's effectiveness in solving saddle point problems.},
     author = {Zheng, Tianqi and Loizou, Nicolas and You, Pengcheng and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.1109/LCSYS.2024.3413004},
     doi = {10.1109/LCSYS.2024.3413004},
     grants = {CPS-2136324, Global-Centers-2330450},
     journal = {IEEE Control Systems Letters (L-CSS)},
     month = {06},
     pages = {2009-2014},
     record = {published, accepted May 2024, submitted Mar 2024},
     title = {Dissipative Gradient Descent Ascent Method: A Control Theory Inspired Algorithm for Min-max Optimization},
     url = {https://mallada.ece.jhu.edu/pubs/2024-LCSS-ZLYM.pdf},
     volume = {8},
     year = {2024}
   }

3. T. Zheng, P. You, and E. Mallada, “Constrained Reinforcement Learning via Dissipative Saddle Flow Dynamics,” in 56th Asilomar Conference on Signals, Systems, and Computers, 2022, pp. 1362-1366. doi:10.1109/IEEECONF56349.2022.10052060
   [BibTeX] [Abstract] [Download PDF]
   

   In constrained reinforcement learning (C-RL), an agent seeks to learn from the environment a policy that maximizes the expected cumulative reward while satisfying minimum requirements in secondary cumulative reward constraints. Several algorithms rooted in sampled-based primal-dual methods have been recently proposed to solve this problem in policy space. However, such methods are based on stochastic gradient descent-ascent algorithms whose trajectories are connected to the optimal policy only after a mixing output stage that depends on the algorithm’s history. As a result, there is a mismatch between the behavioral policy and the optimal one. In this work, we propose a novel algorithm for constrained RL that does not suffer from these limitations. Leveraging recent results on regularized saddle-flow dynamics, we develop a novel stochastic gradient descent-ascent algorithm whose trajectories converge to the optimal policy almost surely.

   @inproceedings{zym2022asilomar,
     abstract = {In constrained reinforcement learning (C-RL), an agent seeks to learn from the environment a policy that maximizes the expected cumulative reward while satisfying minimum requirements in secondary cumulative reward constraints. Several algorithms rooted in sampled-based primal-dual methods have been recently proposed to solve this problem in policy space. However, such methods are based on stochastic gradient descent-ascent algorithms whose trajectories are connected to the optimal policy only after a mixing output stage that depends on the algorithm's history. As a result, there is a mismatch between the behavioral policy and the optimal one. In this work, we propose a novel algorithm for constrained RL that does not suffer from these limitations. Leveraging recent results on regularized saddle-flow dynamics, we develop a novel stochastic gradient descent-ascent algorithm whose trajectories converge to the optimal policy almost surely.},
     author = {Zheng, Tianqi and You, Pengcheng and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.1109/IEEECONF56349.2022.10052060},
     booktitle = {56th Asilomar Conference on Signals, Systems, and Computers},
     doi = {10.1109/IEEECONF56349.2022.10052060},
     grants = {CAREER-1752362, TRIPODS-1934979, CPS-2136324},
     month = {12},
     pages = {1362-1366},
     record = {presented Dec. 2022, accepted Sep. 2022, submitted Apr. 2022},
     title = {Constrained Reinforcement Learning via Dissipative Saddle Flow Dynamics},
     url = {https://mallada.ece.jhu.edu/pubs/2022-Asilomar-ZYM.pdf},
     year = {2022}
   }

4. P. You and E. Mallada, “Saddle Flow Dynamics: Observable Certificates and Separable Regularization,” in American Control Conference (ACC), 2021, pp. 4817-4823. doi:10.23919/ACC50511.2021.9483346
   [BibTeX] [Abstract] [Download PDF]
   

   This paper proposes a certificate, rooted in observability, for asymptotic convergence of saddle flow dynamics of convex-concave functions to a saddle point. This observable certificate directly bridges the gap between the invariant set and the equilibrium set in a LaSalle argument, and generalizes conventional conditions such as strict convexity-concavity and proximal regularization. We further build upon this certificate to propose a separable regularization method for saddle flow dynamics that makes minimal requirements on convexityconcavity and yet still guarantees asymptotic convergence to a saddle point. Our results generalize to saddle flow dynamics with projections on the vector field and have an immediate application as a distributed solution to linear programs.

   @inproceedings{ym2021acc,
     abstract = {This paper proposes a certificate, rooted in observability,
   for asymptotic convergence of saddle flow dynamics
   of convex-concave functions to a saddle point. This observable
   certificate directly bridges the gap between the invariant set
   and the equilibrium set in a LaSalle argument, and generalizes
   conventional conditions such as strict convexity-concavity and
   proximal regularization. We further build upon this certificate
   to propose a separable regularization method for saddle flow
   dynamics that makes minimal requirements on convexityconcavity
   and yet still guarantees asymptotic convergence to
   a saddle point. Our results generalize to saddle flow dynamics
   with projections on the vector field and have an immediate
   application as a distributed solution to linear programs.},
     author = {You, Pengcheng and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.23919/ACC50511.2021.9483346},
     booktitle = {American Control Conference (ACC)},
     doi = {10.23919/ACC50511.2021.9483346},
     grants = {CPS-1544771, EPCN-1711188, CAREER-1752362, TRIPODS-1934979},
     month = {5},
     pages = {4817-4823},
     record = {submitted Sep. 2020, accepted Jan. 2021},
     title = {Saddle Flow Dynamics: Observable Certificates and Separable Regularization},
     url = {https://mallada.ece.jhu.edu/pubs/2021-ACC-YM.pdf},
     year = {2021}
   }

Rather than solve an optimization problem offline and then track its solution, we design feedback controllers that are themselves optimization algorithms, steering a physical system to the solution of a possibly time-varying optimization problem in closed loop. For feedback-linearizable and differentially flat systems we prove global convergence to the optimizer, merging planning and control into a single task, and for general nonlinear plants we characterize when such an optimal-steady-state controller exists. We also develop tractable convex representations, including closed-form Minkowski-sum and positive-semidefinite cone approximations, that make these optimization-based controllers fast enough for real-time use.

Related publications:

1. J. Guthrie, M. Kobilarov, and E. Mallada, “Closed-Form Minkowski Sum Approximations for Efficient Optimization-Based Collision Avoidance,” in American Control Conference (ACC), 2022, pp. 3857-3864. doi:10.23919/ACC53348.2022.9867524
   [BibTeX] [Abstract] [Download PDF]
   

   Motion planning methods for autonomous systems based on nonlinear programming offer great flexibility in incorporating various dynamics, objectives, and constraints. One limitation of such tools is the difficulty of efficiently representing obstacle avoidance conditions for non-trivial shapes. For example, it is possible to define collision avoidance constraints suitable for nonlinear programming solvers in the canonical setting of a circular robot navigating around $M$ convex polytopes over $N$ time steps. However, it requires introducing $(2+L)MN$ additional constraints and $LMN$ additional variables, with $L$ being the number of halfplanes per polytope, leading to larger nonlinear programs with slower and less reliable solving time. In this paper, we overcome this issue by building closed-form representations of the collision avoidance conditions by outer-approximating the Minkowski sum conditions for collision. Our solution requires only $MN$ constraints (and no additional variables), leading to a smaller nonlinear program. On motion planning problems for an autonomous car and quadcopter in cluttered environments, we achieve speedups of 4.0x and 10x respectively with significantly less variance in solve times and negligible impact on performance arising from the use of outer approximations.

   @inproceedings{gkm2022acc,
     abstract = {Motion planning methods for autonomous systems based on nonlinear programming offer great flexibility in incorporating various dynamics, objectives, and constraints. One limitation of such tools is the difficulty of efficiently representing obstacle avoidance conditions for non-trivial shapes. For example, it is possible to define collision avoidance constraints suitable for nonlinear programming solvers in the canonical setting of a circular robot navigating around $M$ convex polytopes over $N$ time steps. However, it requires introducing $(2+L)MN$ additional constraints and $LMN$ additional variables, with $L$ being the number of halfplanes per polytope, leading to larger nonlinear programs with slower and less reliable solving time. In this paper, we overcome this issue by building closed-form representations of the collision avoidance conditions by outer-approximating the Minkowski sum conditions for collision. Our solution requires only $MN$ constraints (and no additional variables), leading to a smaller nonlinear program. On motion planning problems for an autonomous car and quadcopter in cluttered environments, we achieve speedups of 4.0x and 10x respectively with significantly less variance in solve times and negligible impact on performance arising from the use of outer approximations.},
     author = {Guthrie, James and Kobilarov, Marin and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.23919/ACC53348.2022.9867524},
     booktitle = {American Control Conference (ACC)},
     doi = {10.23919/ACC53348.2022.9867524},
     grants = {CAREER-1752362, CPS-2136324, TRIPODS-1934979},
     month = {6},
     pages = {3857-3864},
     record = {presented Jun. 2022, accepted Feb. 2022, submitted Oct. 2021},
     title = {Closed-Form Minkowski Sum Approximations for Efficient Optimization-Based Collision Avoidance},
     url = {https://mallada.ece.jhu.edu/pubs/2022-ACC-GKM.pdf},
     year = {2022}
   }

2. T. Zheng, J. Guthrie, and E. Mallada, “Inner Approximations of the Positive-Semidefinite Cone via Grassmannian Packings,” in 60th IEEE Conference on Decision and Control (CDC), 2021, pp. 981-986. doi:10.1109/CDC45484.2021.9682923
   [BibTeX] [Abstract] [Download PDF]
   

   We investigate the problem of finding tight inner approximations of large dimensional positive semidefinite (PSD) cones. To solve this problem, we develop a novel decomposition framework of the PSD cone by means of conical combinations of smaller dimensional sub-cones. We show that many inner approximation techniques could be summarized within this framework, including the set of (scaled) diagonally dominant matrices, Factor-width k matrices, and Chordal Sparse matrices. Furthermore, we provide a more flexible family of inner approximations of the PSD cone, where we aim to arrange the sub-cones so that they are maximally separated from each other. In doing so, these approximations tend to occupy large fractions of the volume of the PSD cone. The proposed approach is connected to a classical packing problem in Riemannian Geometry. Precisely, we show that the problem of finding maximally distant sub-cones in an ambient PSD cone is equivalent to the problem of packing sub-spaces in a Grassmannian Manifold. We further leverage the existing computational methods for constructing packings in Grassmannian manifolds to build tighter approximations of the PSD cone. Numerical experiments show how the proposed framework can balance accuracy and computational complexity, to efficiently solve positive-semidefinite programs.

   @inproceedings{zgm2021cdc,
     abstract = {We investigate the problem of finding tight inner approximations of large dimensional positive semidefinite (PSD) cones. To solve this problem, we develop a novel decomposition framework of the PSD cone by means of conical combinations of smaller dimensional sub-cones. We show that many inner approximation techniques could be summarized within this framework, including the set of (scaled) diagonally dominant matrices, Factor-width k matrices, and Chordal Sparse matrices. Furthermore, we provide a more flexible family of inner approximations of the PSD cone, where we aim to arrange the sub-cones so that they are maximally separated from each other. In doing so, these approximations tend to occupy large fractions of the volume of the PSD cone. The proposed approach is connected to a classical packing problem in Riemannian Geometry. Precisely, we show that the problem of finding maximally distant sub-cones in an ambient PSD cone is equivalent to the problem of packing sub-spaces in a Grassmannian Manifold. We further leverage the existing computational methods for constructing packings in Grassmannian manifolds to build tighter approximations of the PSD cone. Numerical experiments show how the proposed framework can balance accuracy and computational complexity, to efficiently solve positive-semidefinite programs.},
     author = {Zheng, Tianqi and Guthrie, James and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.1109/CDC45484.2021.9682923},
     booktitle = {60th IEEE Conference on Decision and Control (CDC)},
     doi = {10.1109/CDC45484.2021.9682923},
     grants = {EPCN-1711188, CAREER-1752362, AMPS-1736448, TRIPODS-1934979},
     month = {12},
     pages = {981-986},
     record = {presented Dec. 2022, accepted Jul. 2021, submitted Mar. 2021},
     title = {Inner Approximations of the Positive-Semidefinite Cone via Grassmannian Packings},
     url = {https://mallada.ece.jhu.edu/pubs/2021-CDC-ZGM.pdf},
     year = {2021}
   }

3. L. S. P. Lawrence, J. W. Simpson-Porco, and E. Mallada, “Linear-Convex Optimal Steady-State Control,” IEEE Transactions on Automatic Control, pp. 5377-5385, 2021. doi:10.1109/TAC.2020.3044275
   [BibTeX] [Abstract] [Download PDF]
   

   We consider the problem of designing a feedback controller for a multivariable nonlinear system that regulates an arbitrary subset of the system states and inputs to the solution of a constrained optimization problem, despite parametric modelling uncertainty and time-varying exogenous disturbances; we term this the optimal steady-state (OSS) control problem. We derive necessary and sufficient conditions for the existence of an OSS controller by formulating the OSS control problem as an output regulation problem wherein the regulation error is unmeasurable. We introduce the notion of an optimality model, and show that the existence of an optimality model is sufficient to reduce the OSS control problem to an output regulation problem with measurable error. This yields a design framework for OSS control that unifies and extends many existing designs in the literature. We present a complete and constructive solution of the OSS control problem for the case where the plant is linear time-invariant with structured parametric uncertainty, and disturbances are constant in time. We illustrate these results via an application to optimal frequency control of power networks, and show that our design procedure recovers several frequency controllers from the recent literature.

   @article{lsm2020tac,
     abstract = {We consider the problem of designing a feedback controller for a multivariable nonlinear system that regulates an arbitrary subset of the system states and inputs to the solution of a constrained optimization problem, despite parametric modelling uncertainty and time-varying exogenous disturbances; we term this the optimal steady-state (OSS) control problem. We derive necessary and sufficient conditions for the existence of an OSS controller by formulating the OSS control problem as an output regulation problem wherein the regulation error is unmeasurable. We introduce the notion of an optimality model, and show that the existence of an optimality model is sufficient to reduce the OSS control problem to an output regulation  problem with measurable error. This yields a design framework for OSS control that unifies and extends many existing designs in the literature. We present a complete and constructive solution of the OSS control problem for the case where the plant is linear time-invariant with structured parametric uncertainty, and disturbances are constant in time. We illustrate these results via an application to optimal frequency control of power networks, and show that our design procedure recovers several frequency controllers from the recent literature.},
     author = {Lawrence, Liam S. P. and Simpson-Porco, John W. and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.1109/TAC.2020.3044275},
     doi = {10.1109/TAC.2020.3044275},
     grants = {CAREER-1752362;TRIPODS-1934979;CPS-2136324},
     journal = {IEEE Transactions on Automatic Control},
     month = {11},
     pages = {5377-5385},
     record = {early access Dec 2020, accepted Nov 2020, conditionally accepted Aug. 2020, 2nd revision May 2020, revised Sept 2019, submitted Oct. 2018},
     title = {Linear-Convex Optimal Steady-State Control},
     url = {https://mallada.ece.jhu.edu/pubs/2020-TAC-LSM.pdf},
     year = {2021}
   }

4. J. Guthrie and E. Mallada, “Outer Approximations of Minkowski Operations on Complex Sets via Sum-of-Squares Optimization,” in American Control Conference (ACC), 2021, pp. 2367-2373. doi:10.23919/ACC50511.2021.9482940
   [BibTeX] [Abstract] [Download PDF]
   

   We study the problem of finding closed-form outerapproximations of Minkowski sums and products of sets inthe complex plane. Using polar coordinates, we pose this asan optimization problem in which we find a pair of contoursthat give lower and upper bounds on the radial distance ata given angle. Through a series of variable transformationswe rewrite this as a sum-of-squares optimization problem.Numerical examples are given to demonstrate the performance.

   @inproceedings{gm2021acc,
     abstract = {We study the problem of finding closed-form outerapproximations  of  Minkowski  sums  and  products  of  sets  inthe  complex  plane.  Using  polar  coordinates,  we  pose  this  asan  optimization  problem  in  which  we  find  a  pair  of  contoursthat  give  lower  and  upper  bounds  on  the  radial  distance  ata  given  angle.  Through  a  series  of  variable  transformationswe  rewrite  this  as  a  sum-of-squares  optimization  problem.Numerical examples are given to demonstrate the performance.},
     author = {Guthrie, James and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.23919/ACC50511.2021.9482940},
     booktitle = {American Control Conference (ACC)},
     doi = {10.23919/ACC50511.2021.9482940},
     grants = {CPS-1544771, EPCN-1711188, CAREER-1752362, TRIPODS-1934979},
     month = {5},
     pages = {2367-2373},
     record = {submitted Sep. 2020, accepted Jan. 2021},
     title = {Outer Approximations of Minkowski Operations on Complex Sets via Sum-of-Squares Optimization},
     url = {https://mallada.ece.jhu.edu/pubs/2021-ACC-GM.pdf},
     year = {2021}
   }

5. T. Zheng, J. W. Simpson-Porco, and E. Mallada, “Implicit Trajectory Planning for Feedback Linearizable Systems: A Time-varying Optimization Approach,” in American Control Conference (ACC), 2020, pp. 4677-4682. doi:10.23919/ACC45564.2020.9147997
   [BibTeX] [Abstract] [Download PDF]
   

   We develop an optimization-based framework for joint real-time trajectory planning and feedback control of feedback-linearizable systems. To achieve this goal, we define a target trajectory as the optimal solution of a time-varying optimization problem. In general, however, such trajectory may not be feasible due to , e.g., nonholonomic constraints. To solve this problem, we design a control law that generates feasible trajectories that asymptotically converge to the target trajectory. More precisely, for systems that are (dynamic) full-state linearizable, the proposed control law implicitly transforms the nonlinear system into an optimization algorithm of sufficiently high order. We prove global exponential convergence to the target trajectory for both the optimization algorithm and the original system. We illustrate the effectiveness of our proposed method on multi-target or multi-agent tracking problems with constraints.

   @inproceedings{zsm2020acc,
     abstract = { We develop an optimization-based framework for joint real-time trajectory planning and feedback control of feedback-linearizable systems. To achieve this goal, we define a target trajectory as the optimal solution of a time-varying optimization problem. In general, however, such trajectory may not be feasible due to , e.g., nonholonomic constraints. To solve this problem, we design a control law that generates feasible trajectories that asymptotically converge to the target trajectory. More precisely, for systems that are (dynamic) full-state linearizable, the proposed control law implicitly transforms the nonlinear system into an optimization algorithm of sufficiently high order. We prove global exponential convergence to the target trajectory for both the optimization algorithm and the original system. We illustrate the effectiveness of our proposed method on multi-target or multi-agent tracking problems with constraints.},
     author = {Zheng, Tianqi and Simpson-Porco, John W. and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.23919/ACC45564.2020.9147997},
     booktitle = {American Control Conference (ACC)},
     doi = {10.23919/ACC45564.2020.9147997},
     grants = {CPS-1544771, CAREER-1752362, ARO-W911NF-17-1-0092},
     month = {7},
     pages = {4677-4682},
     title = {Implicit Trajectory Planning for Feedback Linearizable Systems: A Time-varying Optimization Approach},
     url = {https://mallada.ece.jhu.edu/pubs/2020-ACC-ZSM.pdf},
     year = {2020}
   }

Gradient descent on heavily overparametrized networks converges reliably and to solutions that generalize, neither of which follows from classical optimization theory. We treat training as a dynamical system and analyze its trajectories: we give exact convergence rates and implicit-bias characterizations for overparametrized linear networks, show how the scale and balance of the initialization determine which minimum is reached, and prove that neurons in two-layer ReLU networks align with the data early in training before fitting it.

Related publications:

1. Z. Xu, H. Min, S. Tarmoun, E. Mallada, and R. Vidal, “A Local Polyak-Łojasiewicz and Descent Lemma of Gradient Descent For Overparametrized Linear Models,” Transaction on Machine Learning Research (TMLR), 2025.
   [BibTeX] [Download PDF]

   @article{xmtmv2025tmlr,
     author = {Xu, Ziqing and Min, Hancheng and Tarmoun, Salma and Mallada, Enrique and Vidal, Rene},
     grants = {Global Centers-2330450},
     issn = {2835-8856},
     journal = {Transaction on Machine Learning Research (TMLR)},
     month = {5},
     record = {accepted May 2025, submitted Feb 2025},
     title = {A Local Polyak-Łojasiewicz and Descent Lemma of Gradient Descent For Overparametrized Linear Models},
     url = {https://mallada.ece.jhu.edu/pubs/2025-TMLR-XMTMV.pdf},
     year = {2025}
   }

2. H. Min, R. Vidal, and E. Mallada, “Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization,” in International Conference on Representation Learning (ICLR), 2024.
   [BibTeX] [Abstract] [Download PDF]
   

   This paper studies the problem of training a two-layer ReLU network for binary classification using gradient flow with small initialization. We consider a training dataset with well-separated input vectors: Any pair of input data with the same label are positively correlated, and any pair with different labels are negatively correlated. Our analysis shows that, during the early phase of training, neurons in the first layer try to align with either the positive data or the negative data, depending on its corresponding weight on the second layer. A careful analysis of the neurons’ directional dynamics allows us to provide an $\mathcalO(\fracłog n\sqrtμ)$ upper bound on the time it takes for all neurons to achieve good alignment with the input data, where $n$ is the number of data points and $μ$ measures how well the data are separated. After the early alignment phase, the loss converges to zero at a $\mathcalO(\frac1t)$ rate, and the weight matrix on the first layer is approximately low-rank. Numerical experiments on the MNIST dataset illustrate our theoretical findings.

   @inproceedings{mvm2024iclr,
     abstract = {This paper studies the problem of training a two-layer ReLU network for binary classification using gradient flow with small initialization. We consider a training dataset with well-separated input vectors: Any pair of input data with the same label are positively correlated, and any pair with different labels are negatively correlated. Our analysis shows that, during the early phase of training, neurons in the first layer try to align with either the positive data or the negative data, depending on its corresponding weight on the second layer. A careful analysis of the neurons' directional dynamics allows us to provide an $\mathcalO(\fracłog n\sqrtμ)$ upper bound on the time it takes for all neurons to achieve good alignment with the input data, where $n$ is the number of data points and $μ$ measures how well the data are separated. After the early alignment phase, the loss converges to zero at a $\mathcalO(\frac1t)$ rate, and the weight matrix on the first layer is approximately low-rank. Numerical experiments on the MNIST dataset illustrate our theoretical findings.},
     author = {Min, Hancheng and Vidal, Rene and Mallada, Enrique},
     booktitle = {International Conference on Representation Learning (ICLR)},
     grants = {CAREER-1752362},
     month = {05},
     record = {published, accepted Jan 2024, submitted Sep 2023},
     title = {Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization},
     url = {https://mallada.ece.jhu.edu/pubs/2024-ICLR-MVM.pdf},
     year = {2024}
   }

3. H. Min, R. Vidal, and E. Mallada, “On the Convergence of Gradient Flow on Multi-layer Linear Models,” in International Conference on Machine Learning (ICML), 2023, pp. 1-8.
   [BibTeX] [Abstract] [Download PDF]
   

   In this paper, we analyze the convergence of gradient flow on a multi-layer linear model with a loss function of the form $f(W_1 W_2 łdots W_L)$. We show that when $f$ satisfies the gradient dominance property, proper weight initialization leads to exponential convergence of the gradient flow to a global minimum of the loss. Moreover, the convergence rate depends on two trajectory-specific quantities that are controlled by the weight initialization: the imbalance matrices, which measure the difference between the weights of adjacent layers, and the least singular value of the weight product $W = W_1 W_2 łdots W_L$. Our analysis exploits the fact that the gradient of the overparameterized loss can be written as the composition of the non-overparametrized gradient with a time-varying (weight-dependent) linear operator whose smallest eigenvalue controls the convergence rate. The key challenge we address is to derive a uniform lower bound for this time-varying eigenvalue that lead to improved rates for several multi-layer network models studied in the literature.

   @inproceedings{mvm2023icml,
     abstract = {In this paper, we analyze the convergence of gradient flow on a multi-layer linear model with a loss function of the form $f(W_1 W_2 łdots W_L)$. We show that when $f$ satisfies the gradient dominance property, proper weight initialization leads to exponential convergence of the gradient flow to a global minimum of the loss. Moreover, the convergence rate depends on two trajectory-specific quantities that are controlled by the weight initialization: the imbalance matrices, which measure the difference between the weights of adjacent layers, and the least singular value of the weight product $W = W_1 W_2 łdots W_L$. Our analysis exploits the fact that the gradient of the overparameterized loss can be written as the composition of the non-overparametrized gradient with a time-varying (weight-dependent) linear operator whose smallest eigenvalue controls the convergence rate. The key challenge we address is to derive a uniform lower bound for this time-varying eigenvalue that lead to improved rates for several multi-layer network models studied in the literature.},
     author = {Min, Hancheng and Vidal, Rene and Mallada, Enrique},
     bdsk-url-3 = {https://mallada.ece.jhu.edu/pubs/2023-ICML-MVM.pdf},
     booktitle = {International Conference on Machine Learning (ICML)},
     grants = {TRIPODS-1934979, CAREER-1752362},
     month = {4},
     pages = {1-8},
     record = {presented Jul. 2023, accepted Apr. 2023, submitted Jan. 2023},
     title = {On the Convergence of Gradient Flow on Multi-layer Linear Models},
     url = {https://mallada.ece.jhu.edu/pubs/2023-ICML-MVM.pdf},
     year = {2023}
   }

4. Z. Xu, H. Min, S. Tarmoun, E. Mallada, and R. Vidal, “Linear Convergence of Gradient Descent For Overparametrized Finite Width Two-Layer Linear Networks with General Initialization,” in International Conference on Artificial Intelligence and Statistics (AISTATS), 2023, pp. 2262-2284.
   [BibTeX] [Abstract] [Download PDF]
   

   Recent theoretical analyses of the convergence of gradient descent (GD) to a global minimum for over-parametrized neural networks make strong assumptions on the step size (infinitesimal), the hidden-layer width (infinite), or the initialization (spectral, balanced). In this work, we relax these assumptions and derive a linear convergence rate for two-layer linear networks trained using GD on the squared loss in the case of finite step size, finite width and general initialization. Despite the generality of our analysis, our rate estimates are significantly tighter than those of prior work. Moreover, we provide a time-varying step size rule that monotonically improves the convergence rate as the loss function decreases to zero. Numerical experiments validate our findings.

   @inproceedings{xmtmv2023aistats,
     abstract = {Recent theoretical analyses of the convergence of gradient descent (GD) to a global minimum for over-parametrized neural networks make strong assumptions on the step size (infinitesimal), the hidden-layer width (infinite), or the initialization (spectral, balanced). In this work, we relax these assumptions and derive a linear convergence rate for two-layer linear networks trained using GD on the squared loss in the case of finite step size, finite width and general initialization. Despite the generality of our analysis, our rate estimates are significantly tighter than those of prior work. Moreover, we provide a time-varying step size rule that monotonically improves the convergence rate as the loss function decreases to zero. Numerical experiments validate our findings.},
     author = {Xu, Ziqing and Min, Hancheng and Tarmoun, Salma and Mallada, Enrique and Vidal, Rene},
     booktitle = {International Conference on Artificial Intelligence and Statistics (AISTATS)},
     editor = {Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem},
     grants = {No Grant},
     month = {4},
     pages = {2262--2284},
     publisher = {PMLR},
     record = {published, accepted Jan 2023, submitted Oct 2022},
     series = {Proceedings of Machine Learning Research},
     title = {Linear Convergence of Gradient Descent For Overparametrized Finite Width Two-Layer Linear Networks with General Initialization},
     url = {https://mallada.ece.jhu.edu/pubs/2023-AISTATS-XMTMV.pdf},
     volume = {206},
     year = {2023}
   }

5. H. Min, S. Tarmoun, R. Vidal, and E. Mallada, “On the Explicit Role of Initialization on the Convergence and Implicit Bias of Overparametrized Linear Networks,” in International Conference on Machine Learning (ICML), 2021, pp. 7760-7768.
   [BibTeX] [Abstract] [Download PDF]
   

   Neural networks trained via gradient descent with random initialization and without any regularization enjoy good generalization performance in practice despite being highly overparametrized. A promising direction to explain this phenomenon is to study how initialization and overparametrization affect convergence and implicit bias of training algorithms. In this paper, we present a novel analysis of single-hidden-layer linear networks trained under gradient flow, which connects initialization, optimization, and overparametrization. Firstly, we show that the squared loss converges exponentially to its optimum at a rate that depends on the level of imbalance of the initialization. Secondly, we show that proper initialization constrains the dynamics of the network parameters to lie within an invariant set. In turn, minimizing the loss over this set leads to the min-norm solution. Finally, we show that large hidden layer width, together with (properly scaled) random initialization, ensures proximity to such an invariant set during training, allowing us to derive a novel non-asymptotic upper-bound on the distance between the trained network and the min-norm solution.

   @inproceedings{mtvm2021icml,
     abstract = {Neural networks trained via gradient descent with random initialization and without any regularization enjoy good generalization performance in practice despite being highly overparametrized. A promising direction to explain this phenomenon is to study how initialization and overparametrization affect convergence and implicit bias of training algorithms. In this paper, we present a novel analysis of single-hidden-layer linear networks trained under gradient flow, which connects initialization, optimization, and overparametrization. Firstly, we show that the squared loss converges exponentially to its optimum at a rate that depends on the level of imbalance of the initialization. Secondly, we show that proper initialization constrains the dynamics of the network parameters to lie within an invariant set. In turn, minimizing the loss over this set leads to the min-norm solution. Finally, we show that large hidden layer width, together with (properly scaled) random initialization, ensures proximity to such an invariant set during training, allowing us to derive a novel non-asymptotic upper-bound on the distance between the trained network and the min-norm solution.        },
     author = {Min, Hancheng and Tarmoun, Salma and Vidal, Rene and Mallada, Enrique},
     bdsk-url-3 = {https://mallada.ece.jhu.edu/pubs/2021-ICML-MTVM.pdf},
     booktitle = {International Conference on Machine Learning (ICML)},
     grants = {TRIPODS-1934979, CAREER-1752362, AMPS-1736448},
     month = {7},
     note = {(21.5$%$ acceptance)},
     pages = {7760--7768},
     publisher = {PMLR},
     record = {accepted May 2021},
     series = {Proceedings of Machine Learning Research},
     title = {On the Explicit Role of Initialization on the Convergence and Implicit Bias of Overparametrized Linear Networks},
     url = {https://mallada.ece.jhu.edu/pubs/2021-ICML-MTVM.pdf},
     volume = {139},
     year = {2021}
   }

Low-rank adaptation fine-tunes very large models by training only a small number of parameters, and works far better than its size suggests. Analyzing its learning dynamics under gradient flow, we show that the final error is governed by the misalignment between the pretrained model and the target task, that smaller initialization improves that alignment, and that a spectral initialization we propose provably converges to the target to arbitrary precision.

Related publications:

1. Z. Xu, H. Min, L. E. MacDonald, J. Luo, S. Tarmoun, E. Mallada, and R. Vidal, “Understanding the Learning Dynamics of LoRA: A Gradient Flow Perspective on Low-Rank Adaptation in Matrix Factorization,” in International Conference on Artificial Intelligence and Statistics (AISTATS), 2025.
   [BibTeX] [Abstract] [Download PDF]
   

   Despite the empirical success of Low-Rank Adaptation (LoRA) in fine-tuning pretrained models, there is little theoretical understanding of how first-order methods with carefully crafted initialization adapt models to new tasks. In this work, we take the first step towards bridging this gap by theoretically analyzing the learning dynamics of LoRA for matrix factorization (MF) under gradient flow (GF), emphasizing the crucial role of initialization. For small initialization, we theoretically show that GF converges to a neighborhood of the optimal solution, with smaller initialization leading to lower final error. Our analysis shows that the final error is affected by the misalignment between the singular spaces of the pre-trained model and the target matrix, and reducing the initialization scale improves alignment. To address this misalignment, we propose a spectral initialization for LoRA in MF and theoretically prove that GF with small spectral initialization converges to the fine-tuning task with arbitrary precision. Numerical experiments from MF and image classification validate our findings.

   @inproceedings{xmmltmv2025aistats,
     abstract = {Despite the empirical success of Low-Rank Adaptation (LoRA) in fine-tuning pretrained models, there is little theoretical understanding of how first-order methods with carefully crafted initialization adapt models to new tasks. In this work, we take the first step towards bridging this gap by theoretically analyzing the learning dynamics of LoRA for matrix factorization (MF) under gradient flow (GF), emphasizing the crucial role of initialization. For small initialization, we theoretically show that GF converges to a neighborhood of the optimal solution, with smaller initialization leading to lower final error. Our analysis shows that the final error is affected by the misalignment between the singular spaces of the pre-trained model and the target matrix, and reducing the initialization scale improves alignment. To address this misalignment, we propose a spectral initialization for LoRA in MF and theoretically prove that GF with small spectral initialization converges to the fine-tuning task with arbitrary precision. Numerical experiments from MF and image classification validate our findings.},
     author = {Xu, Ziqing and Min, Hancheng and MacDonald, Lachlan Ewen and Luo, Jinqi and Tarmoun, Salma and Mallada, Enrique and Vidal, Rene},
     booktitle = {International Conference on Artificial Intelligence and Statistics (AISTATS)},
     grants = {Global Centers},
     month = {4},
     publisher = {PMLR},
     record = {accepted Jan 2024, submitted Oct 2024},
     series = {Proceedings of Machine Learning Research},
     title = {Understanding the Learning Dynamics of LoRA: A Gradient Flow Perspective on Low-Rank Adaptation in Matrix Factorization},
     url = {https://mallada.ece.jhu.edu/pubs/2025-AISTATS-XMMLTMV.pdf},
     year = {2025}
   }

When can a sparse signal, or the sparse input driving a dynamical system, be recovered exactly? We generalize the classical nullspace property to characterize the largest sparsity pattern recoverable by L1 minimization for a given dictionary, and show that for graph incidence matrices this recoverability can be certified in polynomial time. Extending the theory to dynamical systems, we give the first necessary and sufficient conditions for recovering sparse inputs to a linear system from its outputs.

Related publications:

1. K. Poe, E. Mallada, and R. Vidal, “Invertibility of Discrete-Time Linear Systems with Sparse Inputs,” in 63rd IEEE Conference on Decision and Control (CDC), 2024. doi:10.1109/CDC56724.2024.10886207
   [BibTeX] [Abstract] [Download PDF]
   

   One of the fundamental problems of interest for discrete-time linear systems is whether its input sequence may be recovered given its output sequence, a.k.a. the left inversion problem. Many conditions on the state space geometry, dynamics, and spectral structure of a system have been used to characterize the well-posedness of this problem, without assumptions on the inputs. However, certain structural assumptions, such as input sparsity, have been shown to translate to practical gains in the performance of inversion algorithms, surpassing classical guarantees. Establishing necessary and sufficient conditions for left invertibility of systems with sparse inputs is therefore a crucial step toward understanding the performance limits of system inversion under structured input assumptions. In this work, we provide the first necessary and sufficient characterizations of left invertibility for linear systems with sparse inputs, echoing classic characterizations for standard linear systems. The key insight in deriving these results is in establishing the existence of two novel geometric invariants unique to the sparse-input setting, the weakly unobservable and strongly reachable subspace arrangements. By means of a concrete example, we demonstrate the utility of these characterizations. We conclude by discussing extensions and applications of this framework to several related problems in sparse control.

   @inproceedings{pmv2024cdc,
     abstract = {One of the fundamental problems of interest for discrete-time linear systems is whether its input sequence may be recovered given its output sequence, a.k.a. the left inversion problem. Many conditions on the state space geometry, dynamics, and spectral structure of a system have been used to characterize the well-posedness of this problem, without assumptions on the inputs. However, certain structural assumptions, such as input sparsity, have been shown to translate to practical gains in the performance of inversion algorithms, surpassing classical guarantees. Establishing necessary and sufficient conditions for left invertibility of systems with sparse inputs is therefore a crucial step toward understanding the performance limits of system inversion under structured input assumptions. In this work, we provide the first necessary and sufficient characterizations of left invertibility for linear systems with sparse inputs,  echoing classic characterizations for standard linear systems. The key insight in deriving these results is in establishing the existence of two novel geometric invariants unique to the sparse-input setting, the weakly unobservable and strongly reachable subspace arrangements. By means of a concrete example, we demonstrate the utility of these characterizations. We conclude by discussing extensions and applications of this framework to several related problems in sparse control.},
     author = {Poe, Kyle and Mallada, Enrique and Vidal, Rene},
     booktitle = {63rd IEEE Conference on Decision and Control (CDC)},
     doi = {10.1109/CDC56724.2024.10886207},
     grants = {CPS-2136324; Global-Centers-2330450},
     month = {12},
     record = {presented Dec 2024, accepted Jul 2024, submitted Mar 2024},
     title = {Invertibility of Discrete-Time Linear Systems with Sparse Inputs},
     url = {https://mallada.ece.jhu.edu/pubs/2024-CDC-PMV.pdf},
     year = {2024}
   }

2. K. Poe, E. Mallada, and R. Vidal, “Necessary and Sufficient Conditions for Simultaneous State and Input Recovery of Linear Systems with Sparse Inputs by $\ell_1$-Minimization,” in 62nd IEEE Conference on Decision and Control (CDC), 2023, pp. 6499-6506. doi:10.1109/CDC49753.2023.10383682
   [BibTeX] [Abstract] [Download PDF]
   

   The study of theoretical conditions for recovering sparse signals from compressive measurements has received a lot of attention in the research community. In parallel, there has been a great amount of work characterizing conditions for the recovery both the state and the input to a linear dynamical system (LDS), including a handful of results on recovering sparse inputs. However, existing sufficient conditions for recovering sparse inputs to an LDS are conservative and hard to interpret, while necessary and sufficient conditions have not yet appeared in the literature. In this work, we provide (1) the first characterization of necessary and sufficient conditions for the existence and uniqueness of sparse inputs to an LDS, (2) the first necessary and sufficient conditions for a linear program to recover both an unknown initial state and a sparse input, and (3) simple, interpretable recovery conditions in terms of the LDS parameters. We conclude with a numerical validation of these claims and discuss implications and future directions.

   @inproceedings{pmv2023cdc,
     abstract = {The study of theoretical conditions for recovering sparse signals from compressive measurements has received a lot of attention in the research community. In parallel, there has been a great amount of work characterizing conditions for the recovery both the state and the input to a linear dynamical system (LDS), including a handful of results on recovering sparse inputs. However, existing sufficient conditions for recovering sparse inputs to an LDS are conservative and hard to interpret, while necessary and sufficient conditions have not yet appeared in the literature. In this work, we provide (1) the first characterization of necessary and sufficient conditions for the existence and uniqueness of sparse inputs to an LDS, (2) the first necessary and sufficient conditions for a linear program to recover both an unknown initial state and a sparse input, and (3) simple, interpretable recovery conditions in terms of the LDS parameters. We conclude with a numerical validation of these claims and discuss implications and future directions.
   },
     author = {Poe, Kyle and Mallada, Enrique and Vidal, Rene},
     bdsk-url-3 = {https://doi.org/10.1109/CDC49753.2023.10383682},
     booktitle = {62nd IEEE Conference on Decision and Control (CDC)},
     doi = {10.1109/CDC49753.2023.10383682},
     grants = {CPS-2136324,CAREER-1752362},
     month = {12},
     pages = {6499--6506},
     record = {presented, accepted Jul 2023, submitted Mar 2023},
     title = {Necessary and Sufficient Conditions for Simultaneous State and Input Recovery of Linear Systems with Sparse Inputs by $\ell_1$-Minimization},
     url = {https://mallada.ece.jhu.edu/pubs/2023-CDC-PMV.pdf},
     year = {2023}
   }

3. M. D. Kaba, C. You, D. R. Robinson, E. Mallada, and R. Vidal, “Characterization of Subspace-Preserving Recovery by a Nullspace Property,” in International Conference on Machine Learning (ICML), 2021, pp. 5180-5188.
   [BibTeX] [Abstract] [Download PDF]
   

   Much of the theory for classical sparse recovery is based on conditions on the dictionary that are both necessary and sufficient (e.g., nullspace property) or only sufficient (e.g., incoherence and restricted isometry). In contrast, much of the theory for subspace-preserving recovery, the theoretical underpinnings for sparse subspace classification and clustering methods, is based on conditions on the subspaces and the data that are only sufficient (e.g., subspace incoherence and data inner-radius). This paper derives a necessary and sufficient condition for subspace-preserving recovery that is inspired by the classical nullspace property. Based on this novel condition, called here the subspace nullspace property, we derive equivalent characterizations that either admit a clear geometric interpretation that relates data distribution and subspace separation to the recovery success, or can be verified using a finite set of extreme points of a properly defined set. We further exploit these characterizations to derive new sufficient conditions, based on inner-radius and outer-radius measures and dual bounds, that generalize existing conditions and preserve the geometric interpretations. These results fill an important gap in the subspace-preserving recovery literature.

   @inproceedings{kcrmv2021icml,
     abstract = {Much of the theory for classical sparse recovery is based on conditions on the dictionary that are both necessary and sufficient (e.g., nullspace property) or only sufficient (e.g., incoherence and restricted isometry). In contrast, much of the theory for subspace-preserving recovery, the theoretical underpinnings for sparse subspace classification and clustering methods, is based on conditions on the subspaces and the data that are only sufficient (e.g., subspace incoherence and data inner-radius). This paper derives a necessary and sufficient condition for subspace-preserving recovery that is inspired by the classical nullspace property. Based on this novel condition, called here the subspace nullspace property, we derive equivalent characterizations that either admit a clear geometric interpretation that relates data distribution and subspace separation to the recovery success, or can be verified using a finite set of extreme points of a properly defined set. We further exploit these characterizations to derive new sufficient conditions, based on inner-radius and outer-radius measures and dual bounds, that generalize existing conditions and preserve the geometric interpretations. These results fill an important gap in the subspace-preserving recovery literature.},
     author = {Kaba, Mustafa Devrim and You, Chong and Robinson, Daniel R. and Mallada, Enrique and Vidal, Rene},
     booktitle = {International Conference on Machine Learning (ICML)},
     grants = {CAREER-1752362;TRIPODS-1934979;CPS-2136324},
     month = {11},
     note = {(21.5$%$ acceptance)},
     pages = {5180--5188},
     publisher = {PMLR},
     record = {accepted May 2021},
     series = {Proceedings of Machine Learning Research},
     title = {Characterization of Subspace-Preserving Recovery by a Nullspace Property},
     url = {https://mallada.ece.jhu.edu/pubs/2021-ICML-KCRMV.pdf},
     volume = {139},
     year = {2021}
   }

4. M. D. Kaba, M. Zhao, R. Vidal, D. R. Robinson, and E. Mallada, “What is the Largest Sparsity Pattern that Can Be Recovered by 1-Norm Minimization?,” IEEE Transactions on Information Theory, vol. 67, iss. 5, pp. 3060-3074, 2021. doi:10.1109/TIT.2021.3067280
   [BibTeX] [Abstract] [Download PDF]
   

   Much of the existing literature in sparse recovery is concerned with the following question: given a sparsity pattern and a corresponding regularizer, derive conditions on the dictionary under which exact recovery is possible. In this paper, we study the opposite question: given a dictionary and the `1-norm regularizer, find the largest sparsity pattern that can be recovered. We show that such a pattern is described by a mathematical object called a “maximum abstract simplicial complex,” and provide two different characterizations of this object: one based on extreme points and the other based on vectors of minimal support. In addition, we show how this new framework is useful in the study of sparse recovery problems when the dictionary takes the form of a graph incidence matrix or a partial discrete Fourier transform. In case of incidence matrices, we show that the largest sparsity pattern that can be recovered is determined by the set of simple cycles of the graph. As a byproduct, we show that standard sparse recovery can be certified in polynomial time, although this is known to be NP- hard for general matrices. In the case of the partial discrete Fourier transform, our characterization of the largest sparsity pattern that can be recovered requires the unknown signal to be real and its dimension to be a prime number.

   @article{kzvrm2021tit,
     abstract = {Much of the existing literature in sparse recovery is concerned with the following question: given a sparsity pattern and a corresponding regularizer, derive conditions on the dictionary under which exact recovery is possible. In this paper, we study the opposite question: given a dictionary and the `1-norm regularizer, find the largest sparsity pattern that can be recovered. We show that such a pattern is described by a mathematical object called a "maximum abstract simplicial complex," and provide two different characterizations of this object: one based on extreme points and the other based on vectors of minimal support. In addition, we show how this new framework is useful in the study of sparse recovery problems when the dictionary takes the form of a graph incidence matrix or a partial discrete Fourier transform. In case of incidence matrices, we show that the largest sparsity pattern that can be recovered is determined by the set of simple cycles of the graph. As a byproduct, we show that standard sparse recovery can be certified in polynomial time, although this is known to be NP- hard for general matrices. In the case of the partial discrete Fourier transform, our characterization of the largest sparsity pattern that can be recovered requires the unknown signal to be real and its dimension to be a prime number.},
     author = {Kaba, Mustafa Devrim and Zhao, Mengnan and Vidal, Rene and Robinson, Daniel R. and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.1109/TIT.2021.3067280},
     doi = {10.1109/TIT.2021.3067280},
     grants = {AMPS:1736448;CAREER-1752362},
     journal = {IEEE Transactions on Information Theory},
     month = {5},
     number = {5},
     pages = {3060-3074},
     record = {early access Mar. 2021, accepted Jan. 2021, revised Apr. 2020, submitted Oct. 2019},
     title = {What is the Largest Sparsity Pattern that Can Be Recovered by 1-Norm Minimization?},
     url = {https://mallada.ece.jhu.edu/pubs/2021-TIT-KZVRM.pdf},
     volume = {67},
     year = {2021}
   }

As inverter-based resources replace synchronous machines, the grid loses the inertia that frequency control has always relied on. The common response is to mimic that inertia with virtual inertia; we argue this is the wrong target and instead shape the grid’s frequency response directly. Our grid-forming and dynamic-droop controllers arrest disturbances faster and with less control effort than virtual inertia, can eliminate the frequency nadir using storage, and let the response of a multi-machine, multi-inverter system be shaped cluster by cluster.

Related publications:

1. B. K. Poolla, Y. Lin, A. Bernstein, E. Mallada, and D. Groß, “Dynamic Shaping of Grid Response of Multi-Machine Multi-Inverter Systems Through Grid-Forming IBRs,” in PES General Meeting, 2024, pp. 1-5. doi:10.1109/PESGM51994.2024.10688717
   [BibTeX] [Abstract] [Download PDF]
   

   We consider the problem of controlling the frequency response of weakly-coupled multi-machine multi-inverter low-inertia power systems via grid-forming inverter-based resources (IBRs). In contrast to existing methods, our approach relies on dividing the larger system into multiple strongly-coupled subsystems, without ignoring either the underlying network or approximating the subsystem response as an aggregate harmonic mean model. Rather, through a structured clustering and recursive dynamic shaping approach, the frequency response of the overall system to load perturbations is shaped appropriately. We demonstrate the proposed approach for a three-node triangular configuration and a small-scale radial network. Furthermore, previous synchronization analysis for heterogeneous systems requires the machines to satisfy certain proportionality property. In our approach, the effective transfer functions for each cluster can be tuned by the IBRs to satisfy such property, enabling us to apply the shaping control to systems with a wider range of heterogeneous machines.

   @inproceedings{plbmg2024pesgm,
     abstract = {We consider the problem of controlling the frequency response of weakly-coupled multi-machine multi-inverter low-inertia power systems via grid-forming inverter-based resources (IBRs). In contrast to existing methods, our approach relies on dividing the larger system into multiple strongly-coupled subsystems, without ignoring either the underlying network or approximating the subsystem response as an aggregate harmonic mean model. Rather, through a structured clustering and recursive dynamic shaping approach, the frequency response of the overall system to load perturbations is shaped appropriately. We demonstrate the proposed approach for a three-node triangular configuration and a small-scale radial network. Furthermore, previous synchronization analysis for heterogeneous systems requires the machines to satisfy certain proportionality property. In our approach, the effective transfer functions for each cluster can be tuned by the IBRs to satisfy such property, enabling us to apply the shaping control to systems with a wider range of heterogeneous machines.},
     author = {Poolla, Bala Kameshwar and Lin, Yashen and Bernstein, Andrey and Mallada, Enrique and Groß, Dominic},
     bdsk-url-3 = {https://doi.org/10.1109/PESGM51994.2024.10688717},
     booktitle = {PES General Meeting},
     doi = {10.1109/PESGM51994.2024.10688717},
     grants = {CPS-2136324, CAREER-1752362, Global Centers-2330450},
     month = {06},
     pages = {1-5},
     record = {presented Jun. 2024, accepted Mar. 2024, submitted Nov. 2023},
     title = {Dynamic Shaping of Grid Response of Multi-Machine Multi-Inverter Systems Through Grid-Forming IBRs},
     url = {https://mallada.ece.jhu.edu/pubs/2024-PESGM-PLBMG.pdf},
     year = {2024}
   }

2. B. K. Poolla, Y. Lin, A. Bernstein, E. Mallada, and D. Groß, “Frequency shaping control for weakly-coupled grid-forming IBRs,” IEEE Control Systems Letters (L-CSS), pp. 937-942, 2022. doi:10.1109/LCSYS.2022.3228855
   [BibTeX] [Abstract] [Download PDF]
   

   We consider the problem of controlling the fre- quency of low-inertia power systems via inverter-based resources (IBRs) that are weakly connected to the grid. We propose a novel grid-forming control strategy, the so-called frequency shaping control, that aims to shape the frequency response of synchronous generators (SGs) to load perturbations so as to efficiently arrest sudden frequency drops. Our solution relaxes several existing assumptions in the literature and is able to navigate trade- offs between peak power requirements and maximum frequency deviations. Finally, we analyze the robustness to imperfect knowl- edge of network parameters, while particularly highlighting the importance of accurate estimation of these parameters.

   @article{plbmg2023lcss,
     abstract = {We consider the problem of controlling the fre- quency of low-inertia power systems via inverter-based resources (IBRs) that are weakly connected to the grid. We propose a novel grid-forming control strategy, the so-called frequency shaping control, that aims to shape the frequency response of synchronous generators (SGs) to load perturbations so as to efficiently arrest sudden frequency drops. Our solution relaxes several existing assumptions in the literature and is able to navigate trade- offs between peak power requirements and maximum frequency deviations. Finally, we analyze the robustness to imperfect knowl- edge of network parameters, while particularly highlighting the importance of accurate estimation of these parameters.},
     author = {Poolla, Bala Kameshwar and Lin, Yashen and Bernstein, Andrey and Mallada, Enrique and Groß, Dominic},
     bdsk-url-3 = {https://doi.org/10.1109/LCSYS.2022.3228855},
     doi = {10.1109/LCSYS.2022.3228855},
     grants = {CAREER-1752362, CPS-2136324},
     journal = {IEEE Control Systems Letters (L-CSS)},
     month = {12},
     pages = {937-942},
     record = {published, online Dec 2022, accepted Nov 2022, submitted Sep 2022.},
     title = {Frequency shaping control for weakly-coupled grid-forming IBRs},
     url = {https://mallada.ece.jhu.edu/pubs/2022-LCSS-PLBMG.pdf},
     year = {2022}
   }

3. Y. Jiang, E. Cohn, P. Vorobev, and E. Mallada, “Storage-Based Frequency Shaping Control,” IEEE Transactions on Power Systems, vol. 36, iss. 6, pp. 5006-5019, 2021. doi:10.1109/TPWRS.2021.3072833
   [BibTeX] [Abstract] [Download PDF]
   

   With the decrease in system inertia, frequency security becomes an issue for power systems around the world. Energy storage systems (ESS), due to their excellent ramping capabilities, are considered as a natural choice for the improvement of frequency response following major contingencies. In this manuscript, we propose a new strategy for energy storage — frequency shaping control — that allows to completely eliminate the frequency Nadir, one of the main issue in frequency security, and at the same time tune the rate of change of frequency (RoCoF) to a desired value. With Nadir eliminated, the frequency security assessment can be performed via simple algebraic calculations, as opposed to dynamic simulations for conventional control strategies. Moreover, our proposed control is also very efficient in terms of the requirements on storage peak power, requiring up to 40% less power than conventional virtual inertia approach for the same performance.

   @article{jcvm2021tps,
     abstract = {With the decrease in system inertia, frequency security becomes an issue for power systems around the world. Energy storage systems (ESS), due to their excellent ramping capabilities, are considered as a natural choice for the improvement of frequency response following major contingencies. In this manuscript, we propose a new strategy for energy storage -- frequency shaping control -- that allows to completely eliminate the frequency Nadir, one of the main issue in frequency security, and at the same time tune the rate of change of frequency (RoCoF) to a desired value. With Nadir eliminated, the frequency security assessment can be performed via simple algebraic calculations, as opposed to dynamic simulations for conventional control strategies. Moreover, our proposed control is also very efficient in terms of the requirements on storage peak power, requiring up to 40% less power than conventional virtual inertia approach for the same performance.},
     author = {Jiang, Yan and Cohn, Eliza and Vorobev, Petr and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.1109/TPWRS.2021.3072833},
     doi = {10.1109/TPWRS.2021.3072833},
     grants = {CAREER-1752362;CPS-2136324},
     journal = {IEEE Transactions on Power Systems},
     month = {11},
     number = {6},
     pages = {5006-5019},
     record = {early access Apr 2021, accepted Mar 2021, revised Oct 2020, submitted May 2020},
     title = {Storage-Based Frequency Shaping Control},
     url = {https://mallada.ece.jhu.edu/pubs/2021-TPS-JCVM.pdf},
     volume = {36},
     year = {2021}
   }

4. Y. Jiang, R. Pates, and E. Mallada, “Dynamic Droop Control in Low Inertia Power Systems,” IEEE Transactions on Automatic Control, vol. 66, iss. 8, pp. 3518-3533, 2021. doi:10.1109/TAC.2020.3034198
   [BibTeX] [Abstract] [Download PDF]
   

   A widely embraced approach to mitigate the dynamic degradation in low-inertia power systems is to mimic generation response using grid-connected inverters to restore the grid’s stiffness. In this paper, we seek to challenge this approach and advocate for a principled design based on a systematic analysis of the performance trade-offs of inverterbased frequency control. With this aim, we perform a qualitative and quantitative study comparing the effect of conventional control strategies –droop control (DC) and virtual inertia (VI)– on several performance metrics induced by L2 and L$ınfty$ signal norms. By extending a recently proposed modal decomposition method, we capture the effect of step and stochastic power disturbances, and frequency measurement noise, on the overall transient and steady-state behavior of the system. Our analysis unveils several limitations of these solutions, such as the inability of DC to improve dynamic frequency response without increasing steady-state control effort, or the large frequency variance that VI introduces in the presence of measurement noise. We further propose a novel dynam-i-c Droop controller (iDroop) that overcomes the limitations of DC and VI. More precisely, we show that iDroop can be tuned to achieve high noise rejection, fast system-wide synchronization, or frequency overshoot (Nadir) elimination without affecting the steady-state control effort share, and propose a tuning recommendation that strikes a balance among these objectives. Extensive numerical experimentation shows that the proposed tuning is effective even when our proportionality assumptions are not valid, and that the particular tuning used for Nadir elimination strikes a good trade-off among various performance metrics.

   @article{jpm2021tac,
     abstract = {A widely embraced approach to mitigate the dynamic degradation in low-inertia power systems is to mimic generation response using grid-connected inverters to restore
   the grid's stiffness. In this paper, we seek to challenge this approach and advocate for a principled design based on a systematic analysis of the performance trade-offs of inverterbased frequency control. With this aim, we perform a qualitative
   and quantitative study comparing the effect of conventional
   control strategies --droop control (DC) and virtual inertia (VI)--
   on several performance metrics induced by L2 and L$ınfty$ signal
   norms. By extending a recently proposed modal decomposition
   method, we capture the effect of step and stochastic power
   disturbances, and frequency measurement noise, on the overall
   transient and steady-state behavior of the system. Our analysis
   unveils several limitations of these solutions, such as the inability of DC to improve dynamic frequency response without
   increasing steady-state control effort, or the large frequency
   variance that VI introduces in the presence of measurement
   noise. We further propose a novel dynam-i-c Droop controller
   (iDroop) that overcomes the limitations of DC and VI. More
   precisely, we show that iDroop can be tuned to achieve high
   noise rejection, fast system-wide synchronization, or frequency
   overshoot (Nadir) elimination without affecting the steady-state
   control effort share, and propose a tuning recommendation that
   strikes a balance among these objectives. Extensive numerical
   experimentation shows that the proposed tuning is effective even
   when our proportionality assumptions are not valid, and that
   the particular tuning used for Nadir elimination strikes a good
   trade-off among various performance metrics.},
     author = {Jiang, Yan and Pates, Richard and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.1109/TAC.2020.3034198},
     doi = {10.1109/TAC.2020.3034198},
     grants = {ENERGISE-DE-EE0008006, EPCN-1711188,AMPS-1736448, CPS-1544771, CAREER-1752362, AMPS-1736448, ARO-W911NF-17-1-0092},
     journal = {IEEE Transactions on Automatic Control},
     month = {8},
     number = {8},
     pages = {3518-3533},
     record = {available online Nov. 2020, accepted Aug. 2020, revised Mar. 2020, submitted Aug. 2019},
     title = {Dynamic Droop Control in Low Inertia Power Systems},
     url = {https://mallada.ece.jhu.edu/pubs/2021-TAC-JPM.pdf},
     volume = {66},
     year = {2021}
   }

5. Y. Jiang, A. Bernstein, P. Vorobev, and E. Mallada, “Grid-forming frequency shaping control in low inertia power systems,” in American Control Conference (ACC), 2021, pp. 4184-4189. doi:10.23919/ACC50511.2021.9482678
   [BibTeX] [Abstract] [Download PDF]
   

   As power systems transit to a state of high renewable penetration, little or no presence of synchronous generators makes the prerequisite of well-regulated fre- quency for grid-following inverters unrealistic. Thus, there is a trend to resort to grid-forming inverters which set fre- quency directly. We propose a novel grid-forming frequency shaping control that is able to shape the aggregate system frequency dynamics into a first-order one with the desired steady-state frequency deviation and Rate of Change of Frequency (RoCoF) after a sudden power imbalance. The no overshoot property resulting from the first-order dynam- ics allows the system frequency to monotonically move to- wards its new steady-state without experiencing frequency Nadir, which largely improves frequency security. We prove that our grid-forming frequency-shaping control renders the system internally stable under mild assumptions. The performance of the proposed control is verified via numeri- cal simulations on a modified Icelandic Power Network test case.

   @inproceedings{jbvm2021acc,
     abstract = {As power systems transit to a state of high renewable penetration, little or no presence of synchronous generators makes the prerequisite of well-regulated fre- quency for grid-following inverters unrealistic. Thus, there is a trend to resort to grid-forming inverters which set fre- quency directly. We propose a novel grid-forming frequency shaping control that is able to shape the aggregate system frequency dynamics into a first-order one with the desired steady-state frequency deviation and Rate of Change of Frequency (RoCoF) after a sudden power imbalance. The no overshoot property resulting from the first-order dynam- ics allows the system frequency to monotonically move to- wards its new steady-state without experiencing frequency Nadir, which largely improves frequency security. We prove that our grid-forming frequency-shaping control renders the system internally stable under mild assumptions. The performance of the proposed control is verified via numeri- cal simulations on a modified Icelandic Power Network test case.},
     author = {Jiang, Yan and Bernstein, Andrey and Vorobev, Petr and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.23919/ACC50511.2021.9482678},
     booktitle = {American Control Conference (ACC)},
     doi = {10.23919/ACC50511.2021.9482678},
     grants = {CAREER-1752362, AMPS-1736448, TRIPODS-1934979, EPCN-1711188},
     month = {5},
     pages = {4184-4189},
     record = {submitted Sep. 2020, accepted Jan. 2021},
     title = {Grid-forming frequency shaping control in low inertia power systems},
     url = {https://mallada.ece.jhu.edu/pubs/2021-ACC-JBVM.pdf},
     year = {2021}
   }

Certifying the stability of a grid with thousands of inverters is intractable if it requires a system-wide model. We develop decentralized certificates in which each component checks a local frequency-domain condition, and stability of the whole interconnection follows, robustly as components join and leave. Recent work (under review) makes these certificates explicitly dependent on the network state, capturing how reactive-power mismatch and line loading shrink the set of stabilizing controller gains, and validates grid-forming designs in power-hardware-in-the-loop experiments.

Related publications:

1. Z. Siahaan, E. Mallada, and S. Geng, “Decentralized Stability Criteria for Grid-Forming Control in Inverter-Based Power Systems,” in PES General Meeting, 2024, pp. 1-5. doi:10.1109/PESGM51994.2024.10689037
   [BibTeX] [Abstract] [Download PDF]
   

   This paper presents a decentralized stability analysis of power systems comprising grid-forming (GFM) inverters. We leverage a decentralized stability framework capable of ensuring the stability of the entire interconnection through individual assessments at each bus. The key novelty lies in incorporating voltage dynamics and their coupling with reactive power, in addition to the angle dynamics and their coupling with active power. We perform loop transformation to address the challenge posed by the non-Laplacian nature of the network Jacobian matrix in this case. This methodology is applied to characterize conditions on the droop gains of GFM controllers that can preserve system-wide stability. Our proposed stability criteria exhibit scalability and robustness, and can be extended to accommodate delays, variations in network conditions, and plug-and-play of new components in the network.

   @inproceedings{smg2024pesgm,
     abstract = {This paper presents a decentralized stability analysis of power systems comprising grid-forming (GFM) inverters. We leverage a decentralized stability framework capable of ensuring the stability of the entire interconnection through individual assessments at each bus. The key novelty lies in incorporating voltage dynamics and their coupling with reactive power, in addition to the angle dynamics and their coupling with active power. We perform loop transformation to address the challenge posed by the non-Laplacian nature of the network Jacobian matrix in this case. This methodology is applied to characterize conditions on the droop gains of GFM controllers that can preserve system-wide stability. Our proposed stability criteria exhibit scalability and robustness, and can be extended to accommodate delays, variations in network conditions, and plug-and-play of new components in the network.},
     author = {Siahaan, Zudika and Mallada, Enrique and Geng, Sijia},
     bdsk-url-3 = {https://doi.org/10.1109/PESGM51994.2024.10689037},
     booktitle = {PES General Meeting},
     doi = {10.1109/PESGM51994.2024.10689037},
     grants = {CPS-2136324, CAREER-1752362, Global Centers-2330450},
     month = {06},
     pages = {1-5},
     record = {presented Jun. 2024, accepted Mar. 2024, submitted Nov. 2023},
     title = {Decentralized Stability Criteria for Grid-Forming Control in Inverter-Based Power Systems},
     url = {https://mallada.ece.jhu.edu/pubs/2024-PESGM-SMG.pdf},
     year = {2024}
   }

2. R. Pates and E. Mallada, “Robust Scale Free Synthesis for Frequency Regulation in Power Systems,” IEEE Transactions on Control of Network Systems, vol. 6, iss. 3, pp. 1174-1184, 2019. doi:10.1109/TCNS.2019.2922503
   [BibTeX] [Abstract] [Download PDF]
   

   This paper develops a framework for power system stability analysis, that allows for the decentralised design of frequency controllers. The method builds on a novel decentralised stability criterion, expressed as a positive real requirement, that depends only on the dynamics of the components at each individual bus, and the aggregate susceptance of the transmission lines connected to it. The criterion is both robust to network uncertainties as well as heterogeneous network components, and it can be verified using several standard frequency response, state space, and circuit theory analysis tools. Moreover, it allows to formulate a scale free synthesis problem, that depends on individual bus dynamics and leverages tools from Hinf optimal control. Notably, unlike similar passivity methods, our framework certifies the stability of several existing (non-passive) power system control schemes and allows to study robustness with respect to delays.

   @article{pm2019tcns,
     abstract = {This paper develops a framework for power system stability analysis, that allows for the decentralised design of frequency controllers. The method builds on a novel decentralised stability criterion, expressed as a positive real requirement, that depends only on the dynamics of the components at each individual bus, and the aggregate susceptance of the transmission lines connected to it. The criterion is both robust to network uncertainties as well as heterogeneous network components, and it can be verified using several standard frequency response, state space, and circuit theory analysis tools. Moreover, it allows to formulate a scale free synthesis problem, that depends on individual bus dynamics and leverages tools from Hinf optimal control. Notably, unlike similar passivity methods, our framework certifies the stability of several existing (non-passive) power system control schemes and allows to study robustness with respect to delays.},
     author = {Pates, Richard and Mallada, Enrique},
     bdsk-url-3 = {https://mallada.ece.jhu.edu/pubs/2019-TCNS-PM.pdf},
     bdsk-url-4 = {http://dx.doi.org/10.1109/TCNS.2019.2922503},
     doi = {10.1109/TCNS.2019.2922503},
     grants = {CPS:1544771, EPCN-1711188, AMPS-1736448, CAREER-1752362},
     journal = {IEEE Transactions on Control of Network Systems},
     keywords = {Network Control; Power Networks},
     month = {9},
     number = {3},
     pages = {1174-1184},
     title = {Robust Scale Free Synthesis for Frequency Regulation in Power Systems},
     url = {https://mallada.ece.jhu.edu/pubs/2019-TCNS-PM.pdf},
     volume = {6},
     year = {2019}
   }

Large networks of heterogeneous machines often respond to a disturbance as if they were a single aggregate unit. We explain this coherence with a frequency-domain theory: as network connectivity grows, the system transfer matrix becomes approximately rank one, with the aggregate dynamics given by the harmonic mean of the individual units. Combined with spectral clustering, this yields structure-preserving reduced models in which each coherent group becomes one node, faithful enough to design with and small enough to be tractable. The figure shows the effect directly, as individual responses concentrate onto a single aggregate when the network grows.

Related publications:

1. H. Min, R. Pates, and E. Mallada, “A Frequency Domain Analysis of Slow Coherency in Networked Systems,” Automatica, vol. 74, pp. 1-13, 2025. doi:https://doi.org/10.1016/j.automatica.2025.112184
   [BibTeX] [Abstract] [Download PDF]
   

   Network coherence generally refers to the emergence of simple aggregated dynamical behaviors, despite heterogeneity in the dynamics of the network’s subsystems. In this paper, we develop a general frequency domain framework to analyze and quantify the level of network coherence that a system exhibits by relating coherence with a low-rank property of the system’s input-output response. More precisely, for a networked system with linear dynamics and coupling, we show that, as the network’s effective algebraic connectivity grows, the system transfer matrix converges to a rank-one transfer matrix representing the coherent behavior. Interestingly, the non-zero eigenvalue of such a rank-one matrix is given by the harmonic mean of individual nodal dynamics, and we refer to it as coherent dynamics. Our analysis unveils the frequency-dependent nature of coherence and a non-trivial interplay between dynamics and network topology. We further show that many networked systems can exhibit similar coherent behavior by establishing a concentration result in a setting with randomly chosen individual nodal dynamics.

   @article{mpm2025automatica,
     abstract = {Network coherence generally refers to the emergence of simple aggregated dynamical behaviors, despite heterogeneity in the dynamics of the network's subsystems. In this paper, we develop a general frequency domain framework to analyze and quantify the level of network coherence that a system exhibits by relating coherence with a low-rank property of the system's input-output response. More precisely, for a networked system with linear dynamics and coupling, we show that, as the network's effective algebraic connectivity grows, the system transfer matrix converges to a rank-one transfer matrix representing the coherent behavior. Interestingly, the non-zero eigenvalue of such a rank-one matrix is given by the harmonic mean of individual nodal dynamics, and we refer to it as coherent dynamics. Our analysis unveils the frequency-dependent nature of coherence and a non-trivial interplay between dynamics and network topology. We further show that many networked systems can exhibit similar coherent behavior by establishing a concentration result in a setting with randomly chosen individual nodal dynamics.},
     author = {Min, Hancheng and Pates, Richard and Mallada, Enrique},
     bdsk-url-3 = {https://mallada.ece.jhu.edu/pubs/2025-Automatica-MPM.pdf},
     bdsk-url-4 = {https://doi.org/10.1016/j.automatica.2025.112184},
     doi = {https://doi.org/10.1016/j.automatica.2025.112184},
     grants = {CAREER-1752362, TRIPODS-1934979, CPS-2136324},
     journal = {Automatica},
     month = {2},
     pages = {1-13},
     record = {published, available online Dec 2024, accepted Oct 2024, revised Feb 2024, submitted Feb 2022},
     title = {A Frequency Domain Analysis of Slow Coherency in Networked Systems},
     url = {https://mallada.ece.jhu.edu/pubs/2025-Automatica-MPM.pdf},
     volume = {74},
     year = {2025}
   }

2. H. Min and E. Mallada, “Learning Coherent Clusters in Weakly-Connected Network Systems,” in Proceedings of The 5th Annual Learning for Dynamics and Control Conference, 2023, pp. 1167-1179.
   [BibTeX] [Abstract] [Download PDF]
   

   We propose a structure-preserving model-reduction methodology for large-scale dynamic networks with tightly-connected components. First, the coherent groups are identified by a spectral clustering algorithm on the graph Laplacian matrix that models the network feedback. Then, a reduced network is built, where each node represents the aggregate dynamics of each coherent group, and the reduced network captures the dynamic coupling between the groups. We provide an upper bound on the approximation error when the network graph is randomly generated from a weight stochastic block model. Finally, numerical experiments align with and validate our theoretical findings.

   @inproceedings{mm2023l4dc,
     abstract = {We propose a structure-preserving model-reduction methodology for large-scale dynamic networks with tightly-connected components. First, the coherent groups are identified by a spectral clustering algorithm on the graph Laplacian matrix that models the network feedback. Then, a reduced network is built, where each node represents the aggregate dynamics of each coherent group, and the reduced network captures the dynamic coupling between the groups. We provide an upper bound on the approximation error when the network graph is randomly generated from a weight stochastic block model. Finally, numerical experiments align with and validate our theoretical findings.},
     author = {Min, Hancheng and Mallada, Enrique},
     booktitle = {Proceedings of The 5th Annual Learning for Dynamics and Control Conference},
     editor = {Matni, Nikolai and Morari, Manfred and Pappas, George J.},
     grants = {CAREER-1752362, TRIPODS-1934979, CPS-2136324},
     month = {6},
     pages = {1167--1179},
     publisher = {PMLR},
     record = {presented, accepted Mar 2022, submitted Nov 2022},
     series = {Proceedings of Machine Learning Research},
     title = {Learning Coherent Clusters in Weakly-Connected Network Systems},
     url = {https://mallada.ece.jhu.edu/pubs/2023-L4DC-MM.pdf},
     volume = {211},
     year = {2023}
   }

3. H. Min, F. Paganini, and E. Mallada, “Accurate Reduced Order Models for Coherent Heterogeneous Generators,” IEEE Control Systems Letters (L-CSS), vol. 5, iss. 5, pp. 1741-1746, 2021. doi:10.1109/LCSYS.2020.3043733
   [BibTeX] [Abstract] [Download PDF]
   

   We introduce a novel framework to approximate the aggregate frequency dynamics of coherent synchronous generators. By leveraging recent results on dynamics concentration of tightly connected networks, we develop a hierarchy of reduced order models –based on frequency weighted balanced truncation– that accurately approximate the aggregate system response. Our results outperform existing aggregation techniques and can be shown to monotonically improve the approximation as the hierarchy order increases.

   @article{mpm2021lcss,
     abstract = {We introduce a novel framework to approximate the aggregate frequency dynamics of coherent synchronous generators. By leveraging recent results on dynamics concentration of tightly connected networks, we develop a hierarchy of reduced order models --based on frequency weighted balanced truncation-- that accurately approximate the aggregate system response. Our results outperform existing aggregation techniques and can be shown to monotonically improve the approximation as the hierarchy order increases.},
     author = {Min, Hancheng and Paganini, Fernando and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.1109/LCSYS.2020.3043733},
     doi = {10.1109/LCSYS.2020.3043733},
     grants = {CAREER-1752362, CPS-1544771, ENERGISE-DE-EE0008006, AMPS-1736448, TRIPODS-1934979, EPCN-1711188, ARO-W911NF-17-1-0092},
     journal = {IEEE Control Systems Letters (L-CSS)},
     month = {11},
     note = {also in ACC 2021},
     number = {5},
     pages = {1741-1746},
     record = {early accesss Nov 2020, accepted Nov 2020, revised Nov 2020, submitted Sep 2020},
     title = {Accurate Reduced Order Models for Coherent Heterogeneous Generators},
     url = {https://mallada.ece.jhu.edu/pubs/2021-LCSS-MPM.pdf},
     volume = {5},
     year = {2021}
   }

4. H. Min and E. Mallada, “Dynamics Concentration of Tightly-Connected Large-Scale Networks,” in 58th IEEE Conference on Decision and Control (CDC), 2019, pp. 758-763. doi:10.1109/CDC40024.2019.9029796
   [BibTeX] [Abstract] [Download PDF]
   

   The ability to achieve coordinated behavior –engineered or emergent– on networked systems has attracted widespread interest over several fields. This has led to remarkable advances on the development of a theoretical understanding of the conditions under which agents within a network can reach agreement (consensus) or develop coordinated behaviors such as synchronization. However, fewer advances have been made toward explaining another commonly observed phenomena in tightly-connected networks systems: output responses of nodes in the networks are almost identical to each other despite heterogeneity in their individual dynamics. In this paper, we leverage tools from high-dimensional probability to provide an initial answer to this phenomena. More precisely, we show that for linear networks of nodal random transfer functions, as the networks size and connectivity grows, every node in the network follows the same response to an input or disturbance — irrespectively of the source of this input. We term this behavior as dynamics concentration as it stems from the fact that the network transfer matrix uniformly converges in probability to a unique dynamic response –i.e., it concentrates– determined by the distribution of the random transfer function of each node. We further discuss the implications of our analysis in the context of model reduction and robustness, and provide numerical evidence that similar phenomena occur in small deterministic networks over a properly defined frequency band.

   @inproceedings{mm2019cdc,
     abstract = {The ability to achieve coordinated behavior --engineered or emergent--  on networked systems has attracted widespread interest over several fields. This has led to remarkable advances on the development of a theoretical understanding of the conditions under which agents within a network can reach agreement (consensus) or develop coordinated behaviors such as synchronization. However, fewer advances have been made toward explaining another commonly observed phenomena in tightly-connected networks systems: output responses of nodes in the networks are almost identical to each other despite heterogeneity in their individual dynamics. In this paper, we leverage tools from high-dimensional probability to provide an initial answer to this phenomena. More precisely, we show that for linear networks of nodal random transfer functions, as the networks size and connectivity grows, every node in the network follows the same response to an input or disturbance -- irrespectively of the source of this input. We term this behavior as dynamics concentration as it stems from the fact that the network transfer matrix uniformly converges in probability to a unique dynamic response --i.e., it concentrates-- determined by the distribution of the random transfer function of each node. We further discuss the implications of our analysis in the context of model reduction and robustness, and provide numerical evidence that similar phenomena occur in small deterministic networks over a properly defined frequency band.},
     author = {Min, Hancheng and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.1109/CDC40024.2019.9029796},
     booktitle = {58th IEEE Conference on Decision and Control (CDC)},
     doi = {10.1109/CDC40024.2019.9029796},
     grants = {ARO-W911NF-17-1-0092, CPS-1544771, EPCN-1711188, CAREER-1752362, AMPS-1736448, ENERGISE-DE-EE0008006},
     month = {12},
     pages = {758-763},
     title = {Dynamics Concentration of Tightly-Connected Large-Scale Networks},
     url = {https://mallada.ece.jhu.edu/pubs/2019-CDC-MM.pdf},
     year = {2019}
   }

Synchronization of coupled oscillators is a canonical model of collective behavior across physics, biology, and engineering. We characterize how coupling, network topology, communication delays, and frequency heterogeneity determine whether a population synchronizes, and design controllers that guarantee phase consensus on arbitrary topologies with heterogeneous natural frequencies. The same analysis quantifies synchronization performance in power networks, connecting industry metrics such as nadir and rate-of-change-of-frequency to network structure.

Related publications:

1. F. Paganini and E. Mallada, “Global analysis of synchronization performance for power systems: bridging the theory-practice gap,” IEEE Transactions on Automatic Control, vol. 67, iss. 7, pp. 3007-3022, 2020. doi:10.1109/TAC.2019.2942536
   [BibTeX] [Abstract] [Download PDF]
   

   The issue of synchronization in the power grid is receiving renewed attention, as new energy sources with different dynamics enter the picture. Global metrics have been proposed to evaluate performance, and analyzed under highly simplified assumptions. In this paper we extend this approach to more realistic network scenarios, and more closely connect it with metrics used in power engineering practice. In particular, our analysis covers networks with generators of heterogeneous ratings and richer dynamic models of machines. Under a suitable proportionality assumption in the parameters, we show that the step response of bus frequencies can be decomposed in two components. The first component is a system-wide frequency that captures the aggregate grid behavior, and the residual component represents the individual bus frequency deviations from the aggregate. Using this decomposition, we define –and compute in closed form– several metrics that capture dynamic behaviors that are of relevance for power engineers. In particular, using the system frequency, we define industry-style metrics (Nadir, RoCoF) that are evaluated through a representative machine. We further use the norm of the residual component to define a synchronization cost that can appropriately quantify inter-area oscillations. Finally, we employ robustness analysis tools to evaluate deviations from our proportionality assumption. We show that the system frequency still captures the grid steady-state deviation, and becomes an accurate reduced-order model of the grid as the network connectivity grows. Simulation studies with practically relevant data are included to validate the theory and further illustrate the impact of network structure and parameters on synchronization. Our analysis gives conclusions of practical interest, sometimes challenging the conventional wisdom in the field.

   @article{pm2020tac,
     abstract = {The issue of synchronization in the power grid is receiving renewed attention, as new energy sources with different dynamics enter the picture. Global metrics have been proposed to evaluate performance, and analyzed under highly simplified assumptions. In this paper we extend this approach to more realistic network scenarios, and more closely connect it with metrics used in power engineering practice. In particular, our analysis covers networks with generators of heterogeneous ratings and richer dynamic models of machines. Under a suitable proportionality assumption in the parameters, we show that the step response of bus frequencies can be decomposed in two components. The first component is a system-wide frequency that captures the aggregate grid behavior, and the residual component represents the individual bus frequency deviations from the aggregate. Using this decomposition, we define --and compute in closed form-- several metrics that capture dynamic behaviors that are of relevance for power engineers. In particular, using the system frequency, we define industry-style metrics (Nadir, RoCoF) that are evaluated through a representative machine. We further use the norm of the residual component to define a synchronization cost that can appropriately quantify inter-area oscillations. Finally, we employ robustness analysis tools to evaluate deviations from our proportionality assumption. We show that the system frequency still captures the grid steady-state deviation, and becomes an accurate reduced-order model of the grid as the network connectivity grows. Simulation studies with practically relevant data are included to validate the theory and further illustrate the impact of network structure and parameters on synchronization. Our analysis gives conclusions of practical interest, sometimes challenging the conventional wisdom in the field.},
     author = {Paganini, Fernando and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.1109/TAC.2019.2942536},
     doi = {10.1109/TAC.2019.2942536},
     grants = {CPS-1544771, AMPS-1736448, EPCN-1711188, CAREER-1752362, ENERGISE-DE-EE0008006},
     journal = {IEEE Transactions on Automatic Control},
     month = {7},
     number = {7},
     pages = {3007-3022},
     title = {Global analysis of synchronization performance for power systems: bridging the theory-practice gap},
     url = {https://mallada.ece.jhu.edu/pubs/2020-TAC-PM.pdf},
     volume = {67},
     year = {2020}
   }

2. A. Gushchin, E. Mallada, and A. Tang, “Phase-coupled oscillators with plastic coupling: Synchronization and stability,” IEEE Transactions on Network Science and Engineering, vol. 3, iss. 4, pp. 240-256, 2016. doi:10.1109/TNSE.2016.2605096
   [BibTeX] [Abstract] [Download PDF]
   

   In this article we study synchronization of systems of homogeneous phase-coupled oscillators with plastic coupling strengths and arbitrary underlying topology. The dynamics of the coupling strength between two oscillators is governed by the phase difference between these oscillators. We show that, under mild assumptions, such systems are gradient systems, and always achieve frequency synchronization. Furthermore, we provide sufficient stability and instability conditions that are based on results from algebraic graph theory. For a special case when underlying topology is a tree, we formulate a criterion (necessary and sufficient condition) of stability of equilibria. For both, tree and arbitrary topologies, we provide sufficient conditions for phase-locking, i.e. convergence to a stable equilibrium almost surely. We additionally find conditions when the system possesses a unique stable equilibrium, and thus, almost global stability follows. Several examples are used to demonstrate variety of equilibria the system has, their dependence on system’s parameters, and to illustrate differences in behavior of systems with constant and plastic coupling strengths.

   @article{gmt2016tnse,
     abstract = {In this article we study synchronization of systems of homogeneous phase-coupled oscillators with plastic coupling strengths and arbitrary underlying topology. The dynamics of the coupling strength between two oscillators is governed by the phase difference between these oscillators. We show that, under mild assumptions, such systems are gradient systems, and always achieve frequency synchronization. Furthermore, we provide sufficient stability and instability conditions that are based on results from algebraic graph theory. For a special case when underlying topology is a tree, we formulate a criterion (necessary and sufficient condition) of stability of equilibria. For both, tree and arbitrary topologies, we provide sufficient conditions for phase-locking, i.e. convergence to a stable equilibrium almost surely. We additionally find conditions when the system possesses a unique stable equilibrium, and thus, almost global stability follows. Several examples are used to demonstrate variety of equilibria the system has, their dependence on system's parameters, and to illustrate differences in behavior of systems with constant and plastic coupling strengths.},
     author = {Gushchin, Andrey and Mallada, Enrique and Tang, Ao},
     doi = {10.1109/TNSE.2016.2605096},
     journal = {IEEE Transactions on Network Science and Engineering},
     keywords = {Synchronization},
     month = {09},
     number = {4},
     pages = {240-256},
     title = {Phase-coupled oscillators with plastic coupling: Synchronization and stability},
     url = {https://mallada.ece.jhu.edu/pubs/2016-TNSE-GMT.pdf},
     volume = {3},
     year = {2016}
   }

3. E. Mallada, R. A. Freeman, and A. Tang, “Distributed synchronization of heterogeneous oscillators on networks with arbitrary topology,” IEEE Transactions on Control of Network Systems, vol. 3, iss. 1, pp. 12-23, 2016. doi:10.1109/TCNS.2015.2428371
   [BibTeX] [Abstract] [Download PDF]
   

   Many network applications rely on the synchronization of coupled oscillators. For example, such synchronization can provide networked devices with a common temporal reference necessary for coordinating actions or decoding transmitted messages. In this paper, we study the problem of using distributed control to achieve both phase and frequency synchronization of a network of coupled heterogeneous nonlinear oscillators. Not only do our controllers guarantee zero phase error in steady state under arbitrary frequency heterogeneity, but they also require little knowledge of the oscillator nonlinearities and network topology. Furthermore, we provide a global convergence analysis, in the absence of noise and propagation delay, for the resulting nonlinear system whose phase vector evolves on the n-torus.

   @article{mft2016tcns,
     abstract = {Many network applications rely on the synchronization of coupled oscillators. For example, such synchronization can provide networked devices with a common temporal reference necessary for coordinating actions or decoding transmitted messages. In this paper, we study the problem of using distributed control to achieve both phase and frequency synchronization of a network of coupled heterogeneous nonlinear oscillators. Not only do our controllers guarantee zero phase error in steady state under arbitrary frequency heterogeneity, but they also require little knowledge of the oscillator nonlinearities and network topology. Furthermore, we provide a global convergence analysis, in the absence of noise and propagation delay, for the resulting nonlinear system whose phase vector evolves on the n-torus.},
     author = {Mallada, Enrique and Freeman, Randy A and Tang, Ao},
     doi = {10.1109/TCNS.2015.2428371},
     journal = {IEEE Transactions on Control of Network Systems},
     keywords = {Coupled Oscillators; Synchronization},
     month = {3},
     number = {1},
     pages = {12-23},
     title = {Distributed synchronization of heterogeneous oscillators on networks with arbitrary topology},
     url = {https://mallada.ece.jhu.edu/pubs/2016-TCNS-MFT.pdf},
     volume = {3},
     year = {2016}
   }

4. E. Mallada and A. Tang, “Synchronization of weakly coupled oscillators: coupling, delay and topology,” Journal of Physics A: Mathematical and Theoretical, vol. 46, iss. 50, p. 505101, 2013. doi:10.1088/1751-8113/46/50/505101
   [BibTeX] [Abstract] [Download PDF]
   

   There are three key factors in a system of coupled oscillators that characterize the interaction between them: coupling (how to affect), delay (when to affect) and topology (whom to affect). The existing work on each of these factors has mainly focused on special cases. With new angles and tools, this paper makes progress in relaxing some assumptions on these factors. There are three main results in this paper. Firstly, by using results from algebraic graph theory, a sufficient condition is obtained that can be used to check equilibrium stability. This condition works for arbitrary topology, generalizing existing results and also leading to a sufficient condition on the coupling function which guarantees that the system will reach synchronization. Secondly, it is known that identical oscillators with sin () coupling functions are guaranteed to synchronize in phase on a complete graph. Our results prove that in many cases certain structures such as symmetry and concavity, rather than the exact shape of the coupling function, are the keys for global synchronization. Finally, the effect of heterogenous delays is investigated. Using mean field theory, a system of delayed coupled oscillators is approximated by a non-delayed one whose coupling depends on the delay distribution. This shows how the stability properties of the system depend on the delay distribution and allows us to predict its behavior. In particular, we show that for sin () coupling, heterogeneous delays are equivalent to homogeneous delays. Furthermore, we can use our novel sufficient instability condition to show that heterogeneity, i.e. wider delay distribution, can help reach in-phase synchronization.

   @article{mt2013jopa,
     abstract = {There are three key factors in a system of coupled oscillators that characterize the interaction between them: coupling (how to affect), delay (when to affect) and topology (whom to affect). The existing work on each of these factors has mainly focused on special cases. With new angles and tools, this paper makes progress in relaxing some assumptions on these factors. There are three main results in this paper. Firstly, by using results from algebraic graph theory, a sufficient condition is obtained that can be used to check equilibrium stability. This condition works for arbitrary topology, generalizing existing results and also leading to a sufficient condition on the coupling function which guarantees that the system will reach synchronization. Secondly, it is known that identical oscillators with sin () coupling functions are guaranteed to synchronize in phase on a complete graph. Our results prove that in many cases certain structures such as symmetry and concavity, rather than the exact shape of the coupling function, are the keys for global synchronization. Finally, the effect of heterogenous delays is investigated. Using mean field theory, a system of delayed coupled oscillators is approximated by a non-delayed one whose coupling depends on the delay distribution. This shows how the stability properties of the system depend on the delay distribution and allows us to predict its behavior. In particular, we show that for sin () coupling, heterogeneous delays are equivalent to homogeneous delays. Furthermore, we can use our novel sufficient instability condition to show that heterogeneity, i.e. wider delay distribution, can help reach in-phase synchronization.},
     author = {Mallada, Enrique and Tang, Ao},
     doi = {10.1088/1751-8113/46/50/505101},
     journal = {Journal of Physics A: Mathematical and Theoretical},
     keywords = {Coupled Oscillators; Synchronization},
     month = {12},
     number = {50},
     pages = {505101},
     title = {Synchronization of weakly coupled oscillators: coupling, delay and topology},
     url = {https://mallada.ece.jhu.edu/pubs/2013-JOPA-MT.pdf},
     volume = {46},
     year = {2013}
   }

Foundational work in the group underpins the topics above. We showed that flexible loads can participate in frequency regulation through distributed controllers that restore nominal frequency while respecting network and thermal limits, turning demand into a fast reserve. We also studied voltage collapse, the canonical blackout mechanism, casting it as the unique equilibrium of a load dynamic game and designing cooperative controllers that stabilize otherwise unstable operating points.

Related publications:

1. C. Avraam, J. Rines, A. Sarker, F. Paganini, and E. Mallada, “Voltage Collapse Stabilization in Star DC Networks,” in American Control Conference (ACC), 2019, pp. 1957-1964. doi:10.23919/ACC.2019.8814708
   [BibTeX] [Abstract] [Download PDF]
   

   Voltage collapse is a type of blackout-inducing dynamic instability that occurs when the power demand exceeds the maximum power that can be transferred through the network. The traditional (preventive) approach to avoid voltage collapse is based on ensuring that the network never reaches its maximum capacity. However, such an approach leads to inefficiencies as it prevents operators to fully utilize the network resources and does not account for unprescribed events. To overcome this limitation, this paper seeks to initiate the study of voltage collapse stabilization. More precisely, for a DC network, we formulate the problem of voltage stability as a dynamic problem where each load seeks to achieve a constant power consumption by updating its conductance as the voltage changes. We show that such a system can be interpreted as a dynamic game, where each player (load) seeks to myopically maximize their utility, and where every stable power flow solution amounts to a Local Nash Equilibrium. Using this framework, we show that voltage collapse is equivalent to the non-existence of a Local Nash Equilibrium in the game and, as a result, it is caused by the lack of cooperation between loads. Finally, we propose a Voltage Collapse Stabilizer (VCS) controller that uses (flexible) loads that are willing to cooperate and provides a fair allocation of the curtailed demand. Our solution stabilizes voltage collapse even in the presence of non-cooperative loads. Numerical simulations validate several features of our controllers.

   @inproceedings{arspm2019acc,
     abstract = {Voltage collapse is a type of blackout-inducing dynamic instability that occurs when the power demand exceeds the maximum power that can be transferred through the network. The traditional (preventive) approach to avoid voltage collapse is based on ensuring that the network never reaches its maximum capacity. However, such an approach leads to inefficiencies as it prevents operators to fully utilize the network resources and does not account for unprescribed events. To overcome this limitation, this paper seeks to initiate the study of voltage collapse stabilization.
   
   More precisely, for a DC network, we formulate the problem of voltage stability as a dynamic problem where each load seeks to achieve a constant power consumption by updating its conductance as the voltage changes. We show that such a system can be interpreted as a dynamic game, where each player (load) seeks to myopically maximize their utility, and where every stable power flow solution amounts to a Local Nash Equilibrium.
   
   Using this framework, we show that voltage collapse is equivalent to the non-existence of a Local Nash Equilibrium in the game and, as a result, it is caused by the lack of cooperation
   between loads. Finally, we propose a Voltage Collapse Stabilizer (VCS) controller that uses (flexible) loads that are willing to cooperate and provides a fair allocation of the curtailed demand. Our solution stabilizes voltage collapse even in the presence of non-cooperative loads. Numerical simulations validate several features of our controllers.},
     author = {Avraam, Charalampos and Rines, Jesse and Sarker, Aurik and Paganini, Fernando and Mallada, Enrique},
     booktitle = {American Control Conference (ACC)},
     doi = {10.23919/ACC.2019.8814708},
     grants = {CAREER-1752362,EPCN-1711188,ENERGISE-DE-EE0008006,ARO-W911NF-17-1-0092,EPCN-1711188,CPS-1544771},
     keywords = {Power Networks},
     month = {06},
     pages = {1957-1964},
     title = {Voltage Collapse Stabilization in Star DC Networks},
     url = {https://mallada.ece.jhu.edu/pubs/2019-ACC-ARSPM.pdf},
     year = {2019}
   }

2. C. Zhao, E. Mallada, S. H. Low, and J. W. Bialek, “Distributed plug-and-play optimal generator and load control for power system frequency regulation,” International Journal of Electric Power and Energy Systems, vol. 101, pp. 1-12, 2018. doi:https://doi.org/10.1016/j.ijepes.2018.03.014
   [BibTeX] [Abstract] [Download PDF]
   

   A distributed control scheme, which can be implemented on generators and controllable loads in a plug-and-play manner, is proposed for power system frequency regulation. The proposed scheme is based on local measurements, local computation, and neighborhood information exchanges over a communication network with an arbitrary (but connected) topology. In the event of a sudden change in generation or load, the proposed scheme can restore the nominal frequency and the reference inter-area power flows, while minimizing the total cost of control for participating generators and loads. Power network stability under the proposed control is proved with a relatively realistic model which includes nonlinear power flow and a generic (potentially nonlinear or high-order) turbine-governor model, and further with first- and second-order turbine-governor models as special cases. In simulations, the proposed control scheme shows a comparable performance to the existing automatic generation control (AGC) when implemented only on the generator side, and demonstrates better dynamic characteristics that AGC when each scheme is implemented on both generators and controllable loads.

   @article{zmlb2018ijepes,
     abstract = {A distributed control scheme, which can be implemented on generators and controllable loads in a plug-and-play manner, is proposed for power system frequency regulation. The proposed scheme is based on local measurements, local computation, and neighborhood information exchanges over a communication network with an arbitrary (but connected) topology. In the event of a sudden change in generation or load, the proposed scheme can restore the nominal frequency and the reference inter-area power flows, while minimizing the total cost of control for participating generators and loads. Power network stability under the proposed control is proved with a relatively realistic model which includes nonlinear power flow and a generic (potentially nonlinear or high-order) turbine-governor model, and further with first- and second-order turbine-governor models as special cases. In simulations, the proposed control scheme shows a comparable performance to the existing automatic generation control (AGC) when implemented only on the generator side, and demonstrates better dynamic characteristics that AGC when each scheme is implemented on both generators and controllable loads.},
     author = {Zhao, Changhong and Mallada, Enrique and Low, Steven H and Bialek, Janusz W},
     bdsk-url-3 = {https://doi.org/10.1016/j.ijepes.2018.03.014},
     doi = {https://doi.org/10.1016/j.ijepes.2018.03.014},
     grants = {W911NF-17-1-0092, 1544771, 1711188, 1736448, 1752362},
     issn = {0142-0615},
     journal = {International Journal of Electric Power and Energy Systems},
     keywords = {Power Networks; Frequency Control},
     month = {10},
     pages = {1 -12},
     title = {Distributed plug-and-play optimal generator and load control for power system frequency regulation},
     url = {https://mallada.ece.jhu.edu/pubs/2018-IJEPES-ZMLB.pdf},
     volume = {101},
     year = {2018}
   }

3. E. Mallada, C. Zhao, and S. H. Low, “Optimal load-side control for frequency regulation in smart grids,” IEEE Transactions on Automatic Control, vol. 62, iss. 12, pp. 6294-6309, 2017. doi:10.1109/TAC.2017.2713529
   [BibTeX] [Abstract] [Download PDF]
   

   Frequency control rebalances supply and demand while maintaining the network state within operational margins. It is implemented using fast ramping reserves that are expensive and wasteful, and which are expected to grow with the increasing penetration of renewables. The most promising solution to this problem is the use of demand response, i.e. load participation in frequency control. Yet it is still unclear how to efficiently integrate load participation without introducing instabilities and violating operational constraints. In this paper we present a comprehensive load-side frequency control mechanism that can maintain the grid within operational constraints. Our controllers can rebalance supply and demand after disturbances, restore the frequency to its nominal value and preserve inter-area power flows. Furthermore, our controllers are distributed (unlike generation-side), can allocate load updates optimally, and can maintain line flows within thermal limits. We prove that such a distributed load-side control is globally asymptotically stable and robust to unknown load parameters. Simulations are used to illustrate the properties of our solution.

   @article{mzl2017tac,
     abstract = {Frequency control rebalances supply and demand while maintaining the network state within operational margins. It is implemented using fast ramping reserves that are expensive and wasteful, and which are expected to grow with the increasing penetration of renewables. The most promising solution to this problem is the use of demand response, i.e. load participation in frequency control. Yet it is still unclear how to efficiently integrate load participation without introducing instabilities and violating operational constraints.
   In this paper we present a comprehensive load-side frequency control mechanism that can maintain the grid within operational constraints. Our controllers can rebalance supply and demand after disturbances, restore the frequency to its nominal value and preserve inter-area power flows. Furthermore, our controllers are distributed (unlike generation-side), can allocate load updates optimally, and can maintain line flows within thermal limits. We prove that such a distributed load-side control is globally asymptotically stable and robust to unknown load parameters. Simulations are used to illustrate the properties of our solution.},
     author = {Mallada, Enrique and Zhao, Changhong and Low, Steven H},
     doi = {10.1109/TAC.2017.2713529},
     grants = {1544771},
     journal = {IEEE Transactions on Automatic Control},
     keywords = {Power Networks},
     month = {12},
     number = {12},
     pages = {6294-6309},
     title = {Optimal load-side control for frequency regulation in smart grids},
     url = {https://mallada.ece.jhu.edu/pubs/2017-TAC-MZL.pdf},
     volume = {62},
     year = {2017}
   }

Electricity markets increasingly clear at timescales fast enough to interact with the physical grid. We model the market-clearing process as a dynamical system coupled to grid frequency dynamics and ask when it converges to an efficient, operationally feasible operating point. We identify an alignment condition between participants’ incentives and the operator’s objective under which a saddle-based market design is provably stable, and a regularized design that recovers stability when that alignment fails. This builds on earlier work decomposing joint economic dispatch and frequency regulation across slow and fast timescales.

Related publications:

1. P. You, Y. Jiang, E. Yeung, D. Gayme, and E. Mallada, “On the Stability, Economic Efficiency and Incentive Compatibility of Electricity Market Dynamics,” IEEE Transactions on Automatic Control, vol. 70, iss. 10, pp. 6815-6830, 2025. doi:10.1109/TAC.2025.3589447
   [BibTeX] [Abstract] [Download PDF]
   

   This paper focuses on the operation of an electricity market that accounts for participants that bid at a sub-minute timescale. To that end, we model the market-clearing process as a dynamical system, called market dynamics, which is temporally coupled with the grid frequency dynamics and is thus required to guarantee system-wide stability while meeting the system operational constraints. We characterize participants as price-takers who rationally update their bids to maximize their utility in response to real-time schedules of prices and dispatch. For two common bidding mechanisms, based on quantity and price, we identify a notion of alignment between participants’ behavior and planners’ goals that leads to a saddle-based design of the market that guarantees convergence to a point meeting all operational constraints. We further explore cases where this alignment property does not hold and observe that misaligned participants’ bidding can destabilize the closed-loop system. We thus design a regularized version of the market dynamics that recovers all the desirable stability and steady-state performance guarantees. Numerical tests validate our results on the IEEE 39-bus system.

   @article{yjygm2025tac,
     abstract = {This paper focuses on the operation of an electricity market that accounts for participants that bid at a sub-minute timescale. To that end, we model the market-clearing process as a dynamical system, called market dynamics, which is temporally coupled with the grid frequency dynamics and is thus required to guarantee system-wide stability while meeting the system operational constraints. We characterize participants as price-takers who rationally update their bids to maximize their utility in response to real-time schedules of prices and dispatch. For two common bidding mechanisms, based on quantity and price, we identify a notion of alignment between participants' behavior and planners' goals that leads to a saddle-based design of the market that guarantees convergence to a point meeting all operational constraints. We further explore cases where this alignment property does not hold and observe that misaligned participants' bidding can destabilize the closed-loop system.  We thus design a regularized version of the market dynamics that recovers all the desirable stability and steady-state performance guarantees. Numerical tests validate our results on the IEEE 39-bus system.},
     author = {You, Pengcheng and Jiang, Yan and Yeung, Enoch and Gayme, Dennice and Mallada, Enrique},
     bdsk-url-3 = {https://mallada.ece.jhu.edu/pubs/2024-TAC-YJYGM.pdf},
     doi = {10.1109/TAC.2025.3589447},
     grants = {CPS-2136324, Global Centers-2330450},
     journal = {IEEE Transactions on Automatic Control},
     month = {10},
     number = {10},
     pages = {6815-6830},
     record = {published Oct 2025, accepted Aug 2024, revised Dec 2023, submitted Dec 2021},
     title = {On the Stability, Economic Efficiency and Incentive Compatibility of Electricity Market Dynamics},
     url = {https://mallada.ece.jhu.edu/pubs/2025-TAC-YJYGM.pdf},
     volume = {70},
     year = {2025}
   }

2. D. Cai, E. Mallada, and A. Wierman, “Distributed optimization decomposition for joint economic dispatch and frequency regulation,” IEEE Transactions on Power Systems, vol. 32, iss. 6, pp. 4370-4385, 2017. doi:10.1109/TPWRS.2017.2682235
   [BibTeX] [Abstract] [Download PDF]
   

   Economic dispatch and frequency regulation are typically viewed as fundamentally different problems in power systems and, hence, are typically studied separately. In this paper, we frame and study a joint problem that co-optimizes both slow timescale economic dispatch resources and fast timescale frequency regulation resources. We show how the joint problem can be decomposed without loss of optimality into slow and fast timescale sub-problems that have appealing interpretations as the economic dispatch and frequency regulation problems respectively. We solve the fast timescale sub-problem using a distributed frequency control algorithm that preserves the stability of the network during transients. We solve the slow timescale sub-problem using an efficient market mechanism that coordinates with the fast timescale sub-problem. We investigate the performance of the decomposition on the IEEE 24-bus reliability test system.

   @article{cmw2017tps,
     abstract = {Economic dispatch and frequency regulation are typically viewed as fundamentally different problems in power systems and, hence, are typically studied separately. In this paper, we frame and study a joint problem that co-optimizes both slow timescale economic dispatch resources and fast timescale frequency regulation resources. We show how the joint problem can be decomposed without loss of optimality into slow and fast timescale sub-problems that have appealing interpretations as the economic dispatch and frequency regulation problems respectively. We solve the fast timescale sub-problem using a distributed frequency control algorithm that preserves the stability of the network during transients. We solve the slow timescale sub-problem using an efficient market mechanism that coordinates with the fast timescale sub-problem. We investigate the performance of the decomposition on the IEEE
   24-bus reliability test system.},
     author = {Cai, Desmond and Mallada, Enrique and Wierman, Adam},
     doi = {10.1109/TPWRS.2017.2682235},
     grants = {1544771},
     journal = {IEEE Transactions on Power Systems},
     keywords = {Power Networks; Markets},
     month = {11},
     number = {6},
     pages = {4370-4385},
     title = {Distributed optimization decomposition for joint economic dispatch and frequency regulation},
     url = {https://mallada.ece.jhu.edu/pubs/2017-TPS-CMW.pdf},
     volume = {32},
     year = {2017}
   }

Most electricity is traded through a two-stage, day-ahead and real-time settlement, where insufficient competition opens the door to price manipulation. We analyze the equilibria of these markets when generators and price-inelastic loads bid strategically, and evaluate the market-power-mitigation policies that operators use to substitute default bids for non-competitive offers. Our analysis shows these policies can backfire, shifting market power to loads or eliminating a stable equilibrium, and identifies bidding mechanisms and policy combinations that mitigate market power more effectively.

Related publications:

1. R. K. Bansal, P. You, Y. Chen, and E. Mallada, “Counterfactual analysis of default bid market power mitigation strategies in two-stage electricity markets,” European Journal of Operational Research, pp. 1-18, 2025. doi:https://doi.org/10.1016/j.ejor.2025.12.030
   [BibTeX] [Abstract] [Download PDF]
   

   Market power remains a persistent challenge in many liberalized electricity markets worldwide, driving the adoption of ex-ante and ex-post mitigation measures. Despite locational mitigation tools (e.g., cost-based reference levels or default energy bids), evidence of price manipulation has motivated system-level market power mitigation (MPM) policies. However, the full implications of these rules are not well understood, and limited insight into participant behavior can lead to unintended consequences, including increased market power and welfare losses. We study sequentially cleared electricity markets and analyze a two-stage settlement structure commonly used by system operators (e.g., day-ahead and real-time markets in North America). Our focus is on MPM policies that replace noncompetitive generator offers with operator-estimated default bids, and we model competition between generators and loads with inelastic energy requirements who act strategically in allocating demand across stages under real-time, day-ahead, and simultaneous applications of MPM policies. Motivated by the loss of Nash equilibrium under conventional supply-function bidding, we adopt an alternative mechanism in which generators bid the intercept of an affine supply function. Under real-time MPM, strategic interaction in the day-ahead market drives all demand to real time, producing an undesirable outcome. To test robustness, we incorporate demand uncertainty using a variance-penalized expectation framework. Low risk aversion still leads to substantial real-time clearing, while imbalances in risk preferences further amplify market power. Overall, intercept-function bidding combined with day-ahead and simultaneous MPM policies mitigates generator market power more effectively than real-time substitution alone, although these policies shift some market power toward loads.

   @article{bcym2025ejor,
     abstract = {Market power remains a persistent challenge in many liberalized electricity markets worldwide, driving the adoption of ex-ante and ex-post mitigation measures. Despite locational mitigation tools (e.g., cost-based reference levels or default energy bids), evidence of price manipulation has motivated system-level market power mitigation (MPM) policies. However, the full implications of these rules are not well understood, and limited insight into participant behavior can lead to unintended consequences, including increased market power and welfare losses. We study sequentially cleared electricity markets and analyze a two-stage settlement structure commonly used by system operators (e.g., day-ahead and real-time markets in North America). Our focus is on MPM policies that replace noncompetitive generator offers with operator-estimated default bids, and we model competition between generators and loads with inelastic energy requirements who act strategically in allocating demand across stages under real-time, day-ahead, and simultaneous applications of MPM policies. Motivated by the loss of Nash equilibrium under conventional supply-function bidding, we adopt an alternative mechanism in which generators bid the intercept of an affine supply function. Under real-time MPM, strategic interaction in the day-ahead market drives all demand to real time, producing an undesirable outcome. To test robustness, we incorporate demand uncertainty using a variance-penalized expectation framework. Low risk aversion still leads to substantial real-time clearing, while imbalances in risk preferences further amplify market power. Overall, intercept-function bidding combined with day-ahead and simultaneous MPM policies mitigates generator market power more effectively than real-time substitution alone, although these policies shift some market power toward loads.},
     author = {Bansal, Rajni Kant and You, Pengcheng and Chen, Yue and Mallada, Enrique},
     doi = {https://doi.org/10.1016/j.ejor.2025.12.030},
     grants = {CAREER-1752362; CPS-2136324; Global-Centers-2330450},
     issn = {0377-2217},
     journal = {European Journal of Operational Research},
     month = {12},
     pages = {1-18},
     record = {online 12 2025, accepted Dec 2025, under revision Jan 2024, submitted Aug 2023},
     title = {Counterfactual analysis of default bid market power mitigation strategies in two-stage electricity markets},
     url = {https://mallada.ece.jhu.edu/pubs/2025-EJOR-BCYM.pdf},
     year = {2025}
   }

2. R. K. Bansal, Y. Chen, P. You, and E. Mallada, “Market Power Mitigation in Two-stage Electricity Market with Supply Function and Quantity Bidding,” IEEE Transactions on Energy Markets, Policy and Regulation, vol. 1, iss. 4, pp. 512-522, 2023. doi:10.1109/TEMPR.2023.3318149
   [BibTeX] [Abstract] [Download PDF]
   

   Two-stage settlement electricity markets, which in- clude day-ahead and real-time markets, often observe unde- sirable price manipulation due to the price difference across stages, inadequate competition, and unforeseen circumstances. To mitigate this, some Independent System Operators (ISOs) have proposed system-level market power mitigation (MPM) policies in addition to existing local policies. These system-level policies aim to substitute noncompetitive bids with a default bid based on estimated generator costs. However, without accounting for the conflicting interest of participants, they may lead to unintended consequences when implemented. In this paper, we model the competition between generators (bidding supply functions) and loads (bidding quantity) in a two-stage market with a stage- wise MPM policy. An equilibrium analysis shows that a real- time MPM policy leads to equilibrium loss, meaning no stable market outcome (Nash equilibrium) exists. A day-ahead MPM policy leads to Stackelberg-Nash game, with loads acting as leaders and generators as followers. Despite estimation errors, the competitive equilibrium is efficient, while the Nash equilibrium is comparatively robust to price manipulations. Moreover, analysis of inelastic loads shows their tendency to shift allocation and manipulate prices in the market. Numerical studies illustrate the impact of cost estimation errors, heterogeneity in generation cost, and load size on market equilibrium.

   @article{bcym2023tempr,
     abstract = {Two-stage settlement electricity markets, which in- clude day-ahead and real-time markets, often observe unde- sirable price manipulation due to the price difference across stages, inadequate competition, and unforeseen circumstances. To mitigate this, some Independent System Operators (ISOs) have proposed system-level market power mitigation (MPM) policies in addition to existing local policies. These system-level policies aim to substitute noncompetitive bids with a default bid based on estimated generator costs. However, without accounting for the conflicting interest of participants, they may lead to unintended consequences when implemented. In this paper, we model the competition between generators (bidding supply functions) and loads (bidding quantity) in a two-stage market with a stage- wise MPM policy. An equilibrium analysis shows that a real- time MPM policy leads to equilibrium loss, meaning no stable market outcome (Nash equilibrium) exists. A day-ahead MPM policy leads to Stackelberg-Nash game, with loads acting as leaders and generators as followers. Despite estimation errors, the competitive equilibrium is efficient, while the Nash equilibrium is comparatively robust to price manipulations. Moreover, analysis of inelastic loads shows their tendency to shift allocation and manipulate prices in the market. Numerical studies illustrate the impact of cost estimation errors, heterogeneity in generation cost, and load size on market equilibrium.},
     author = {Bansal, Rajni Kant and Chen, Yue and You, Pengcheng and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.1109/TEMPR.2023.3318149},
     doi = {10.1109/TEMPR.2023.3318149},
     grants = {CAREER-1752362, CPS-2136324, EPICS-2330450},
     journal = {IEEE Transactions on Energy Markets, Policy and Regulation},
     month = {12},
     number = {4},
     pages = {512-522},
     record = {published, online Sep 2023, revised July 2023, under revision May 2023, submitted Jan 2023},
     title = {Market Power Mitigation in Two-stage Electricity Market with Supply Function and Quantity Bidding},
     url = {https://mallada.ece.jhu.edu/pubs/2023-TEMPR-BCYM.pdf},
     volume = {1},
     year = {2023}
   }

3. R. K. Bansal, Y. Chen, P. You, and E. Mallada, “Equilibrium Analysis of Electricity Markets with Day-Ahead Market Power Mitigation and Real-Time Intercept Bidding,” in Proceedings of the Thirteenth ACM International Conference on Future Energy Systems (e-Energy), 2022, pp. 47-62. doi:https://doi.org/10.1145/3538637.3538839
   [BibTeX] [Abstract] [Download PDF]
   

   Electricity markets are cleared by a two-stage, sequential process consisting of a forward (day-ahead) market and a spot (real-time) market. While their design goal is to achieve efficiency, the lack of sufficient competition introduces many opportunities for price manipulation. To discourage this phenomenon, some Independent System Operators (ISOs) mandate generators to submit (approximately) truthful bids in the day-ahead market. However, without fully accounting for all participants’ incentives (generators and loads), the application of such a mandate may lead to unintended consequences. In this paper, we model and study the interactions of generators and inelastic loads in a two-stage settlement where generators are required to bid truthfully in the day-ahead market. We show that such mandate, when accounting for generator and load incentives, leads to a generalized Stackelberg-Nash game where load decisions (leaders) are performed in day-ahead market and generator decisions (followers) are relegated to the real-time market. Furthermore, the use of conventional supply function bidding for generators in real-time, does not guarantee the existence of a Nash equilibrium. This motivates the use of intercept bidding, as an alternative bidding mechanism for generators in the real-time market. An equilibrium analysis in this setting, leads to a closed-form solution that unveils several insights. Particularly, it shows that, unlike standard two-stage markets, loads are the winners of the competition in the sense that their aggregate payments are less than that of the competitive equilibrium. Moreover, heterogeneity in generators cost has the unintended effect of mitigating loads market power. Numerical studies validate and further illustrate these insights.

   @inproceedings{bcym2022e-energy,
     abstract = {Electricity markets are cleared by a two-stage, sequential process consisting of a forward (day-ahead) market and a spot (real-time) market. While their design goal is to achieve efficiency, the lack of sufficient competition introduces many opportunities for price manipulation. To discourage this phenomenon, some Independent System Operators (ISOs) mandate generators to submit (approximately) truthful bids in the day-ahead market. However, without fully accounting for all participants' incentives (generators and loads), the application of such a mandate may lead to unintended consequences. In this paper, we model and study the interactions of generators and inelastic loads in a two-stage settlement where generators are required to bid truthfully in the day-ahead market. We show that such mandate, when accounting for generator and load incentives, leads to a generalized Stackelberg-Nash game where load decisions (leaders) are performed in day-ahead market and generator decisions (followers) are relegated to the real-time market. Furthermore, the use of conventional supply function bidding for generators in real-time, does not guarantee the existence of a Nash equilibrium. This motivates the use of intercept bidding, as an alternative bidding mechanism for generators in the real-time market. An equilibrium analysis in this setting, leads to a closed-form solution that unveils several insights. Particularly, it shows that, unlike standard two-stage markets, loads are the winners of the competition in the sense that their aggregate payments are less than that of the competitive equilibrium. Moreover, heterogeneity in generators cost has the unintended effect of mitigating loads market power. Numerical studies validate and further illustrate these insights.},
     author = {Bansal, Rajni Kant and Chen, Yue and You, Pengcheng and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.1145/3538637.3538839},
     booktitle = {Proceedings of the Thirteenth ACM International Conference on Future Energy Systems (e-Energy)},
     doi = {https://doi.org/10.1145/3538637.3538839},
     grants = {CAREER-1752362,EPCN-1711188,CPS-2136324},
     month = {6},
     pages = {47--62},
     record = {published, accepted Jun 2022, submitted Feb 2022.},
     title = {Equilibrium Analysis of Electricity Markets with Day-Ahead Market Power Mitigation and Real-Time Intercept Bidding},
     url = {https://mallada.ece.jhu.edu/pubs/2022-e-Energy-BCYM.pdf},
     year = {2022}
   }

4. P. You, D. F. Gayme, and E. Mallada, “The Role of Strategic Load Participants in Two-Stage Settlement Electricity Markets,” in 58th IEEE Conference on Decision and Control (CDC), 2019, pp. 8416-8422. doi:10.1109/CDC40024.2019.9029514
   [BibTeX] [Abstract] [Download PDF]
   

   Two-stage electricity market clearing is designed to maintain market efficiency under ideal conditions, e.g., perfect forecast and nonstrategic generation. This work demon- strates that the individual strategic behavior of inelastic load participants in a two-stage settlement electricity market can deteriorate efficiency. Our analysis further implies that virtual bidding can play a role in alleviating this loss of efficiency by mitigating market power of strategic load participants. We use real-world market data from New York ISO to validate our theory.

   @inproceedings{ygm2019cdc,
     abstract = {Two-stage electricity market clearing is designed to maintain market efficiency under ideal conditions, e.g., perfect forecast and nonstrategic generation. This work demon- strates that the individual strategic behavior of inelastic load participants in a two-stage settlement electricity market can deteriorate efficiency. Our analysis further implies that virtual bidding can play a role in alleviating this loss of efficiency by mitigating market power of strategic load participants. We use real-world market data from New York ISO to validate our theory.},
     author = {You, Pengcheng and Gayme, Dennice F. and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.1109/CDC40024.2019.9029514},
     booktitle = {58th IEEE Conference on Decision and Control (CDC)},
     doi = {10.1109/CDC40024.2019.9029514},
     grants = {ARO-W911NF-17-1-0092, CPS-1544771, EPCN-1711188, CAREER-1752362, AMPS-1736448, ENERGISE-DE-EE0008006},
     month = {12},
     pages = {8416-8422},
     title = {The Role of Strategic Load Participants in Two-Stage Settlement Electricity Markets},
     url = {https://mallada.ece.jhu.edu/pubs/2019-CDC-YGM.pdf},
     year = {2019}
   }

Storage and other flexible resources do not fit the cost model markets were designed around, since their cost is degradation accrued cycle by cycle rather than fuel burned. We design market mechanisms in which storage bids energy-cycling functions that map prices to cycle depths, making truthful bidding optimal and yielding an efficient competitive equilibrium without requiring price prediction. Related work quantifies the value of storage in reducing generation cost and develops online scheduling algorithms with optimal competitive ratios, from procuring and storing energy under time-varying prices to charging electric-vehicle fleets under peak constraints.

Related publications:

1. R. K. Bansal, P. You, D. F. Gayme, and E. Mallada, “A Market Mechanism for Truthful Bidding with Energy Storage,” in Power Systems Computation Conference (PSCC), 2022, pp. 1-9.
   [BibTeX] [Abstract] [Download PDF]
   

   This paper proposes a market mechanism for multi-interval electricity markets with generator and storage participants. Drawing ideas from supply function bidding, we introduce a novel bid structure for storage participation that allows storage units to communicate their cost to the market using energy cycling functions that map prices to cycle depths. The resulting market-clearing process–implemented via convex programming–yields corresponding schedules and payments based on traditional energy prices for power supply and per-cycle prices for storage utilization. We illustrate the benefits of our solution by comparing the competitive equilibrium of the resulting mechanism to that of an alternative solution that uses prosumer-based bids. Our solution shows several advantages over the prosumer-based approach. It does not require a priori price estimation. It also incentivizes participants to reveal their truthful costs, thus leading to an efficient, competitive equilibrium. Numerical experiments using New York Independent System Operator (NYISO) data validate our findings.

   @inproceedings{bygm2022pscc,
     abstract = {This paper proposes a market mechanism for multi-interval electricity markets with generator and storage participants. Drawing ideas from supply function bidding, we introduce a novel bid structure for storage participation that allows storage units to communicate their cost to the market using energy cycling functions that map prices to cycle depths. The resulting market-clearing process--implemented via convex programming--yields corresponding schedules and payments based on traditional energy prices for power supply and per-cycle prices for storage utilization. We illustrate the benefits of our solution by comparing the competitive equilibrium of the resulting mechanism to that of an alternative solution that uses prosumer-based bids. Our solution shows several advantages over the prosumer-based approach. It does not require a priori price estimation. It also incentivizes participants to reveal their truthful costs, thus leading to an efficient, competitive equilibrium. Numerical experiments using New York Independent System Operator (NYISO) data validate our findings.},
     author = {Bansal, Rajni Kant and You, Pengcheng and Gayme, Dennice F. and Mallada, Enrique},
     booktitle = {Power Systems Computation Conference (PSCC)},
     grants = {CAREER-1752362, TRIPODS-1934979, CPS-2136324},
     month = {7},
     pages = {1-9},
     record = {presented, accepted Feb. 2022, submitted Oct. 2021},
     title = {A Market Mechanism for Truthful Bidding with Energy Storage},
     url = {https://mallada.ece.jhu.edu/pubs/2022-PSCC-BYGM.pdf},
     year = {2022}
   }

2. R. K. Bansal, P. You, D. F. Gayme, and E. Mallada, “Storage Degradation Aware Economic Dispatch,” in American Control Conference (ACC), 2021, pp. 589-595. doi:10.23919/ACC50511.2021.9482838
   [BibTeX] [Abstract] [Download PDF]
   

   In this paper, we formulate a cycling cost aware economic dispatch problem that co-optimizes generation and storage dispatch while taking into account cycle based storage degradation cost. Our approach exploits the Rainflow cycle counting algorithm to quantify storage degradation for each charging and discharging half-cycle based on its depth. We show that the dispatch is optimal for individual participants in the sense that it maximizes the profit of generators and storage units, under price taking assumptions. We further provide a condition under which the optimal storage response is unique for given market clearing prices. Simulations using data from the New York Independent System Operator (NYISO) illustrate the optimization framework. In particular, they show that the generation-centric dispatch that does not account for storage degradation is insufficient to guarantee storage profitability.

   @inproceedings{bygm2021acc,
     abstract = {In this paper, we formulate a cycling cost aware economic dispatch problem that co-optimizes generation and storage dispatch while taking into account cycle based storage degradation cost. Our approach exploits the Rainflow cycle counting algorithm to quantify storage degradation for each charging and discharging half-cycle based on its depth. We show that the dispatch is optimal for individual participants in the sense that it maximizes the profit of generators and storage units, under price taking assumptions. We further provide a condition under which the optimal storage response is unique for given market clearing prices. Simulations using data from the New York Independent System Operator (NYISO) illustrate the optimization framework. In particular, they show that the generation-centric dispatch that does not account for storage degradation is insufficient to guarantee storage profitability. },
     author = {Bansal, Rajni Kant and You, Pengcheng and Gayme, Dennice F. and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.23919/ACC50511.2021.9482838},
     booktitle = {American Control Conference (ACC)},
     doi = {10.23919/ACC50511.2021.9482838},
     grants = {CPS-1544771, EPCN-1711188, CAREER-1752362, TRIPODS-1934979},
     month = {5},
     pages = {589-595},
     record = {submitted Sep. 2020, accepted Jan. 2021},
     title = {Storage Degradation Aware Economic Dispatch},
     url = {https://mallada.ece.jhu.edu/pubs/2021-ACC-BYGM.pdf},
     year = {2021}
   }

3. B. Alinia, M. H. Hajiesmaili, Z. Lee, N. Crespi, and E. Mallada, “Online EV Scheduling Algorithms for Adaptive Charging Networks with Global Peak Constraints,” IEEE Transactions on Sustainable Computing, 2020. doi:10.1109/TSUSC.2020.2979854
   [BibTeX] [Abstract] [Download PDF]
   

   This paper tackles online scheduling of electric vehicles (EVs) in an adaptive charging network (ACN) with local and global peak constraints. Given the aggregate charging demand of the EVs and the peak constraints of the ACN, it might be infeasible to fully charge all the EVs according to their charging demand. Two alternatives in such resource-limited scenarios are to maximize the social welfare by partially charging the EVs (fractional model) or selecting a subset of EVs and fully charge them (integral model). The critical challenge is the need for online solution design since in practical scenarios the scheduler has no information of future arrivals of EVs in a time- coupled underlying problem. For the fractional model, we devise both offline and online algorithms. We prove that the offline algorithm is optimal. Using competitive ratio as the performance measure, we prove the online algorithm achieves a competitive ratio of 2. The integral model, however, is more challenging since the underlying problem is strongly NP-hard due to 0/1 selection criteria of EVs. Hence, efficient solution design is challenging even in offline setting. We devise a low-complexity primal-dual scheduling algorithm that achieves a bounded approximation ratio. Built upon the offline approximate algorithm, we propose an online algorithm and analyze its competitive ratio in special cases.

   @article{bhlcm2019tsusc,
     abstract = {This paper tackles online scheduling of electric vehicles (EVs) in an adaptive charging network (ACN) with local and global peak constraints. Given the aggregate charging demand of the EVs and the peak constraints of the ACN, it might be infeasible to fully charge all the EVs according to their charging demand. Two alternatives in such resource-limited scenarios are to maximize the social welfare by partially charging the EVs (fractional model) or selecting a subset of EVs and fully charge them (integral model). The critical challenge is the need for online solution design since in practical scenarios the scheduler has no information of future arrivals of EVs in a time- coupled underlying problem. For the fractional model, we devise both offline and online algorithms. We prove that the offline algorithm is optimal. Using competitive ratio as the performance measure, we prove the online algorithm achieves a competitive ratio of 2. The integral model, however, is more challenging since the underlying problem is strongly NP-hard due to 0/1 selection criteria of EVs. Hence, efficient solution design is challenging even in offline setting. We devise a low-complexity primal-dual scheduling algorithm that achieves a bounded approximation ratio. Built upon the offline approximate algorithm, we propose an online algorithm and analyze its competitive ratio in special cases.},
     author = {Alinia, Bahram and Hajiesmaili, Mohammad H. and Lee, Zachary and Crespi, Noel and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.1109/TSUSC.2020.2979854},
     doi = {10.1109/TSUSC.2020.2979854},
     grants = {CAREER-1752362, CPS-1544771, ENERGISE-DE-EE0008006, AMPS-1736448, TRIPODS-1934979,EPCN-1711188,},
     journal = {IEEE Transactions on Sustainable Computing},
     month = {1},
     title = {Online EV Scheduling Algorithms for Adaptive Charging Networks with Global Peak Constraints},
     url = {https://mallada.ece.jhu.edu/pubs/2020-TSUSC-BHLCM.pdf},
     year = {2020}
   }

4. Y. Shen, M. Bichuch, and E. Mallada, “On the Value of Energy Storage in Generation Cost Reduction,” in 19th IEEE European Control Conference (ECC), 2020, pp. 1526-1532. doi:10.23919/ECC51009.2020.9143772
   [BibTeX] [Abstract] [Download PDF]
   

   This work seeks to quantify the benefits of using energy storage toward the reduction of the energy generation cost of a power system. A two-fold optimization framework is provided where the first optimization problem seeks to find the optimal storage schedule that minimizes operational costs. Since the operational cost depends on the storage capacity, a second optimization problem is then formulated with the aim of finding the optimal storage capacity to be deployed. Although, in general, these problems are difficult to solve, we provide a lower bound on the cost savings for a parametrized family of demand profiles. The optimization framework is numerically illustrated using real-world demand data from ISO New England. Numerical results show that energy storage can reduce energy generation costs by at least 2.5 percent.

   @inproceedings{sbm2020ecc,
     abstract = {This work seeks to quantify the benefits of using energy storage toward the reduction of the energy generation cost of a power system.  A two-fold optimization framework is provided where the first optimization problem seeks to find the optimal storage schedule that minimizes operational costs. Since the operational cost depends on the storage capacity, a second optimization problem is then formulated with the aim of finding the optimal storage capacity to be deployed. Although, in general, these problems are difficult to solve, we provide a lower bound on the cost savings for a  parametrized family of demand profiles.
   The optimization framework is numerically illustrated using real-world demand data from ISO New England. Numerical results show that energy storage can reduce energy generation costs by at least 2.5 percent.},
     author = {Shen, Yue and Bichuch, Maxim and Mallada, Enrique},
     bdsk-url-3 = {https://doi.org/10.23919/ECC51009.2020.9143772},
     booktitle = {19th IEEE European Control Conference (ECC)},
     doi = {10.23919/ECC51009.2020.9143772},
     grants = {CAREER-1752362, CPS-1544771, ENERGISE-DE-EE0008006, AMPS-1736448, TRIPODS-1934979, EPCN-1711188, ARO-W911NF-17-1-0092},
     keywords = {Power Networks},
     month = {5},
     pages = {1526-1532},
     title = {On the Value of Energy Storage in Generation Cost Reduction},
     url = {https://mallada.ece.jhu.edu/pubs/2020-ECC-SBM.pdf},
     year = {2020}
   }

5. L. Yang, M. H. Hajiesmaili, R. Sitaraman, A. Wierman, E. Mallada, and W. S. Wong, “Online Linear Optimization with Inventory Management Constraints,” in ACM Sigmetrics, 2020, pp. 1-19. doi:10.1145/3393691.3394207
   [BibTeX] [Abstract] [Download PDF]
   

   This paper considers the problem of online linear optimization with inventory management constraints. Specifically, we consider an online scenario where a decision maker needs to satisfy her time-varying demand for some units of an asset, either from a market with a time-varying price or from her own inventory. In each time slot, the decision maker is presented a (linear) price and must immediately decide the amount to purchase for covering the demand and/or for storing in the inventory for future use. The inventory has a limited capacity and can be used to buy and store assets at low price and cover the demand when the price is high. The ultimate goal of the decision maker is to cover the demand at each time slot while minimizing the cost of buying assets from the market. We propose ARP, an online algorithm for linear programming with inventory constraints, and ARPRate, an extended version that handles rate constraints to/from the inventory. Both ARP and ARPRate achieve optimal competitive ratios, meaning that no other online algorithm can achieve a better theoretical guarantee. To illustrate the results, we use the proposed algorithms in a case study focused on energy procurement and storage management strategies for data centers.

   @inproceedings{yhswmw2020sigmetrics,
     abstract = {This paper considers the problem of online linear optimization with inventory management constraints. Specifically, we consider an online scenario where a decision maker needs to satisfy her time-varying demand for some units of an asset, either from a market with a time-varying price or from her own inventory. In each time slot, the decision maker is presented a (linear) price and must immediately decide the amount to purchase for covering the demand and/or for storing in the inventory for future use. The inventory has a limited capacity and can be used to buy and store assets at low price and cover the demand when the price is high. The ultimate goal of the decision maker is to cover the demand at each time slot while minimizing the cost of buying assets from the market. We propose ARP, an online algorithm for linear programming with inventory constraints, and ARPRate, an extended version that handles rate constraints to/from the inventory. Both ARP and ARPRate achieve optimal competitive ratios, meaning that no other online algorithm can achieve a better theoretical guarantee. To illustrate the results, we use the proposed algorithms in a case study focused on energy procurement and storage management strategies for data centers.},
     author = {Yang, Lin and Hajiesmaili, Mohammad H. and Sitaraman, Ramesh and Wierman, Adam and Mallada, Enrique and Wong, Wing S.},
     booktitle = {ACM Sigmetrics},
     doi = {10.1145/3393691.3394207},
     grants = {CAREER-1752362, CPS-1544771, ENERGISE-DE-EE0008006, AMPS-1736448,EPCN-1711188,},
     month = {6},
     pages = {1-19},
     title = {Online Linear Optimization with Inventory Management Constraints},
     url = {https://mallada.ece.jhu.edu/pubs/2020-Sigmetrics-YHSWMW.pdf},
     year = {2020}
   }