Home

Enrique Mallada

Associate Professor Electrical & Computer Engineering (he/him/his)

Barton Hall 312 3400 N Charles St  Baltimore MD 21218
phone: 410-516-7018 fax: 410-516-5566 mallada [at] jhu [dot] edu

            

I have been an associate professor in Electrical and Computer Engineering (ECE) at Johns Hopkins University (JHU) since July 2022. I earned my Ph.D. in ECE with a minor in Applied Mathematics from Cornell University in Jan 2014 under the supervision of an awesome advisor and person, Prof. A. Kevin Tang. Before joining JHU as an assistant professor in Jan 2016, I was a postdoctoral scholar at the Center for the Mathematics of Information (CMI) in the Computational and Mathematical Sciences (CMS) department at Caltech from 2013 to 2015, where I had the pleasure to be mentored by Prof. Steven Low and Prof. Adam Wierman.


Research Interests

  • Networked Systems: coupled oscillators, clock synchronization, saddle-flows, network coherence, distributed coordination, consensus
  • Power Systems: frequency control, inverter-based control, real-time congestion management, electricity markets, reduced-order models
  • Optimization: time-varying optimization, primal-dual algorithms, semidefinite programming, sum-of-squares optimization
  • Machine Learning: reinforcement learning, sparse recovery, subspace preserving recovery, network tomography, multi-armed bandits

Recent Talks

A complete list of talks can be found here.

  1. 2024-03-20: Model-Free Analysis of Dynamical Systems Using Recurrent Sets, ECE Colloquium, Rutgers University.
    [BibTeX] [Abstract] [Download PDF]

    In this talk, we develop model-free methods for analyzing dynamical systems using trajectory data. Our critical insight is to replace the notion of invariance, a core concept in Lyapunov Theory, with the more relaxed notion of recurrence. Specifically, a set is τ-recurrent (resp. k-recurrent) if every trajectory that starts within the set returns to it after at most τ seconds (resp. k steps). We leverage this notion of recurrence to develop several analysis tools and algorithms to study dynamical systems. Firstly, we consider the problem of learning an inner approximation of the region of attraction (ROA) of an asymptotically stable equilibrium point using trajectory data. We show that a τ-recurrent set containing a stable equilibrium must be a subset of its ROA under mild assumptions. We then develop algorithms that compute inner approximations of the ROA using counter-examples of recurrence that are obtained by sampling finite-length trajectories. Secondly, we generalize Lyapunov’s Direct Method to allow for non-monotonic evolution of the function values by only requiring sub-level sets to be τ-recurrent (instead of invariant). We provide conditions for stability, asymptotic stability, and exponential stability of an equilibrium using τ-decreasing functions (functions whose value along trajectories decrease after at most τ seconds) and develop a verification algorithm that leverages GPU parallel processing to verify such conditions using trajectories. We finalize discussing future research directions and possible extensions for control.

    @talk{rutgers24,
      abstract = {In this talk, we develop model-free methods for analyzing dynamical systems using trajectory data. Our critical insight is to replace the notion of invariance, a core concept in Lyapunov Theory, with the more relaxed notion of recurrence. Specifically, a set is τ-recurrent (resp. k-recurrent) if every trajectory that starts within the set returns to it after at most τ seconds (resp. k steps). We leverage this notion of recurrence to develop several analysis tools and algorithms to study dynamical systems. Firstly, we consider the problem of learning an inner approximation of the region of attraction (ROA) of an asymptotically stable equilibrium point using trajectory data. We show that a τ-recurrent set containing a stable equilibrium must be a subset of its ROA under mild assumptions. We then develop algorithms that compute inner approximations of the ROA using counter-examples of recurrence that are obtained by sampling finite-length trajectories. Secondly, we generalize Lyapunov's Direct Method to allow for non-monotonic evolution of the function values by only requiring sub-level sets to be τ-recurrent (instead of invariant). We provide conditions for stability, asymptotic stability, and exponential stability of an equilibrium using τ-decreasing functions (functions whose value along trajectories decrease after at most τ seconds) and develop a verification algorithm that leverages GPU parallel processing to verify such conditions using trajectories. We finalize discussing future research directions and possible extensions for control.},
      date = {03/20/2024},
      day = {20},
      event = {ECE Colloquium, Rutgers University},
      host = {Daniel Burbano (Rutgers)},
      month = {03},
      role = {Lecture},
      title = {Model-Free Analysis of Dynamical Systems Using Recurrent Sets},
      url = {https://mallada.ece.jhu.edu/talks/202403-Rutgers.pdf},
      year = {2024}
    }

  2. 2024-02-16: Reinforcement Learning for Safety Critical Applications, George Mason University.
    [BibTeX] [Abstract] [Download PDF]

    Integrating Reinforcement Learning (RL) in safety-critical applications, such as autonomous vehicles, healthcare, and industrial automation, necessitates an increased focus on safety and reliability. In this talk, we consider two complementary mechanisms to augment RL’s suitability for safety-critical systems. Firstly, we consider a constrained reinforcement learning (C-RL) setting, wherein agents aim to maximize rewards while adhering to required constraints on secondary specifications. Several algorithms rooted in sampled-based primal-dual methods have been recently proposed to solve this problem in policy space. However, such methods exhibit a discrepancy between the behavioral and optimal policies due to their reliance on stochastic gradient descent-ascent algorithms. We propose a novel algorithm for constrained RL that does not suffer from these limitations. Leveraging recent results on regularized saddle-flow dynamics, we develop a novel stochastic gradient descent-ascent algorithm whose trajectories almost surely converge to the optimal policy. Secondly, we study the problem of incorporating safety-critical constraints to RL that allow an agent to avoid (unsafe) regions of the state space. Though such a safety goal can be captured by an action-value-like function, a.k.a. safety critics, the associated operator lacks the desired contraction and uniqueness properties that the classical Bellman operator enjoys. In this work, we overcome the non-contractiveness of safety critic operators by leveraging that safety is a binary property. To that end, we study the properties of the binary safety critic associated with a deterministic dynamical system that seeks to avoid reaching an unsafe region. We formulate the corresponding binary Bellman equation (B2E) for safety and study its properties. While the resulting operator is still non-contractive, we fully characterize its fixed points representing–except for a spurious solution–maximal persistently safe regions of the state space that can always avoid failure. We provide an algorithm that, by design, leverages axiomatic knowledge of safe data to avoid spurious fixed points.

    @talk{gmu24,
      abstract = {Integrating Reinforcement Learning (RL) in safety-critical applications, such as autonomous vehicles, healthcare, and industrial automation, necessitates an increased focus on safety and reliability. In this talk, we consider two complementary mechanisms to augment RL's suitability for safety-critical systems.
    
    Firstly, we consider a constrained reinforcement learning (C-RL) setting, wherein agents aim to maximize rewards while adhering to required constraints on secondary specifications. Several algorithms rooted in sampled-based primal-dual methods have been recently proposed to solve this problem in policy space. However, such methods exhibit a discrepancy between the behavioral and optimal policies due to their reliance on stochastic gradient descent-ascent algorithms. We propose a novel algorithm for constrained RL that does not suffer from these limitations. Leveraging recent results on regularized saddle-flow dynamics, we develop a novel stochastic gradient descent-ascent algorithm whose trajectories almost surely converge to the optimal policy.
    
    Secondly, we study the problem of incorporating safety-critical constraints to RL that allow an agent to avoid (unsafe) regions of the state space. Though such a safety goal can be captured by an action-value-like function, a.k.a. safety critics, the associated operator lacks the desired contraction and uniqueness properties that the classical Bellman operator enjoys. In this work, we overcome the non-contractiveness of safety critic operators by leveraging that safety is a binary property. To that end, we study the properties of the binary safety critic associated with a deterministic dynamical system that seeks to avoid reaching an unsafe region. We formulate the corresponding binary Bellman equation (B2E) for safety and study its properties. While the resulting operator is still non-contractive, we fully characterize its fixed points representing--except for a spurious solution--maximal persistently safe regions of the state space that can always avoid failure. We provide an algorithm that, by design, leverages axiomatic knowledge of safe data to avoid spurious fixed points.},
      date = {02/2024},
      day = {16},
      event = {George Mason University},
      host = {Ningshi Yao (GMU)},
      month = {02},
      role = {Lecture},
      title = {Reinforcement Learning for Safety Critical Applications},
      url = {https://mallada.ece.jhu.edu/talks/202402-GMU.pdf},
      year = {2024}
    }

  3. 2024-01-11: Reinforcement Learning for Safety Critical Applications, Applied Physics Laboratory, JHU.
    [BibTeX] [Abstract] [Download PDF]

    Integrating Reinforcement Learning (RL) in safety-critical applications, such as autonomous vehicles, healthcare, and industrial automation, necessitates an increased focus on safety and reliability. In this talk, we consider two complementary mechanisms to augment RL’s suitability for safety-critical systems. Firstly, we consider a constrained reinforcement learning (C-RL) setting, wherein agents aim to maximize rewards while adhering to required constraints on secondary specifications. Several algorithms rooted in sampled-based primal-dual methods have been recently proposed to solve this problem in policy space. However, such methods exhibit a discrepancy between the behavioral and optimal policies due to their reliance on stochastic gradient descent-ascent algorithms. We propose a novel algorithm for constrained RL that does not suffer from these limitations. Leveraging recent results on regularized saddle-flow dynamics, we develop a novel stochastic gradient descent-ascent algorithm whose trajectories almost surely converge to the optimal policy. Secondly, we study the problem of incorporating safety-critical constraints to RL that allow an agent to avoid (unsafe) regions of the state space. Though such a safety goal can be captured by an action-value-like function, a.k.a. safety critics, the associated operator lacks the desired contraction and uniqueness properties that the classical Bellman operator enjoys. In this work, we overcome the non-contractiveness of safety critic operators by leveraging that safety is a binary property. To that end, we study the properties of the binary safety critic associated with a deterministic dynamical system that seeks to avoid reaching an unsafe region. We formulate the corresponding binary Bellman equation (B2E) for safety and study its properties. While the resulting operator is still non-contractive, we fully characterize its fixed points representing–except for a spurious solution–maximal persistently safe regions of the state space that can always avoid failure. We provide an algorithm that, by design, leverages axiomatic knowledge of safe data to avoid spurious fixed points.

    @talk{apl24,
      abstract = {Integrating Reinforcement Learning (RL) in safety-critical applications, such as autonomous vehicles, healthcare, and industrial automation, necessitates an increased focus on safety and reliability. In this talk, we consider two complementary mechanisms to augment RL's suitability for safety-critical systems.
    
    Firstly, we consider a constrained reinforcement learning (C-RL) setting, wherein agents aim to maximize rewards while adhering to required constraints on secondary specifications. Several algorithms rooted in sampled-based primal-dual methods have been recently proposed to solve this problem in policy space. However, such methods exhibit a discrepancy between the behavioral and optimal policies due to their reliance on stochastic gradient descent-ascent algorithms. We propose a novel algorithm for constrained RL that does not suffer from these limitations. Leveraging recent results on regularized saddle-flow dynamics, we develop a novel stochastic gradient descent-ascent algorithm whose trajectories almost surely converge to the optimal policy.
    
    Secondly, we study the problem of incorporating safety-critical constraints to RL that allow an agent to avoid (unsafe) regions of the state space. Though such a safety goal can be captured by an action-value-like function, a.k.a. safety critics, the associated operator lacks the desired contraction and uniqueness properties that the classical Bellman operator enjoys. In this work, we overcome the non-contractiveness of safety critic operators by leveraging that safety is a binary property. To that end, we study the properties of the binary safety critic associated with a deterministic dynamical system that seeks to avoid reaching an unsafe region. We formulate the corresponding binary Bellman equation (B2E) for safety and study its properties. While the resulting operator is still non-contractive, we fully characterize its fixed points representing--except for a spurious solution--maximal persistently safe regions of the state space that can always avoid failure. We provide an algorithm that, by design, leverages axiomatic knowledge of safe data to avoid spurious fixed points.},
      date = {02/2024},
      day = {11},
      event = {Applied Physics Laboratory, JHU},
      host = {Jared Markowitz},
      month = {01},
      role = {Lecture},
      title = {Reinforcement Learning for Safety Critical Applications},
      url = {https://mallada.ece.jhu.edu/talks/202401-JHUAPL.pdf},
      year = {2024}
    }

  4. 2023-12-11: Unintended Consequences of Market Designs, IHPC’s Workshop of Power and Energy Systems of the (near) Future, ASTAR.
    [BibTeX] [Abstract] [Download PDF]

    In this talk, we seek to highlight the importance of accounting for the incentives of *all* market participants when designing market mechanisms for electricity. To this end, we perform a Nash equilibrium analysis of two different market mechanisms that aim to illustrate the critical role that the incentives of consumers and other new types of participants, such as storage, play in the equilibrium outcome. Firstly, we study the incentives of heterogeneous participants (generators and consumers) in a two-stage settlement market, where generators participate using a supply function bid and consumers use a quantity bid. We show that strategic consumers are able to exploit generators’ strategic behavior to maintain a systematic difference between the forward and spot prices, with the latter being higher. Notably, such a strategy does bring down consumer payments and undermines the supply-side market power. We further observe situations where generators lose profit by behaving strategically, a sign of overturn of the conventional supply-side market power. Secondly, we study a market mechanism for multi-interval electricity markets with generator and storage participants. Drawing ideas from supply function bidding, we introduce a novel bid structure for storage participation that allows storage units to communicate their cost to the market using energy-cycling functions that map prices to cycle depths. The resulting market-clearing process — implemented via convex programming — yields corresponding schedules and payments based on traditional energy prices for power supply and per-cycle prices for storage utilization. Our solution shows several advantages over the standard prosumer-based approach that prices energy per slot. In particular, it does not require a priori estimation of future prices and leads to an efficient, competitive equilibrium.

    @talk{astar23,
      abstract = {In this talk, we seek to highlight the importance of accounting for the incentives of *all* market participants when designing market mechanisms for electricity. To this end, we perform a Nash equilibrium analysis of two different market mechanisms that aim to illustrate the critical role that the incentives of consumers and other new types of participants, such as storage, play in the equilibrium outcome. Firstly, we study the incentives of heterogeneous participants (generators and consumers) in a two-stage settlement market, where generators participate using a supply function bid and consumers use a quantity bid. We show that strategic consumers are able to exploit generators' strategic behavior to maintain a systematic difference between the forward and spot prices, with the latter being higher.  Notably, such a strategy does bring down consumer payments and undermines the supply-side market power. We further observe situations where generators lose profit by behaving strategically, a sign of overturn of the conventional supply-side market power. Secondly, we study a market mechanism for multi-interval electricity markets with generator and storage participants. Drawing ideas from supply function bidding, we introduce a novel bid structure for storage participation that allows storage units to communicate their cost to the market using energy-cycling functions that map prices to cycle depths. The resulting market-clearing process -- implemented via convex programming -- yields corresponding schedules and payments based on traditional energy prices for power supply and per-cycle prices for storage utilization. Our solution shows several advantages over the standard prosumer-based approach that prices energy per slot. In particular, it does not require a priori estimation of future prices and leads to an efficient, competitive equilibrium.},
      date = {12/11/2023},
      day = {11},
      event = {IHPC's Workshop of Power and Energy Systems of the (near) Future, ASTAR},
      host = {John Pang (ASTAR)},
      month = {12},
      role = {Speaker},
      title = {Unintended Consequences of Market Designs},
      url = {https://mallada.ece.jhu.edu/talks/202312-ASTAR.pdf},
      year = {2023}
    }

  5. 2023-11-04: Model-Free Analysis of Dynamical Systems Using Recurrent Sets, FIND Seminar, Cornell University.
    [BibTeX] [Abstract] [Download PDF]

    In this talk, we develop model-free methods for analyzing dynamical systems using trajectory data. Our critical insight is to replace the notion of invariance, a core concept in Lyapunov Theory, with the more relaxed notion of recurrence. Specifically, a set is τ-recurrent (resp. k-recurrent) if every trajectory that starts within the set returns to it after at most τ seconds (resp. k steps). We leverage this notion of recurrence to develop several analysis tools and algorithms to study dynamical systems. Firstly, we consider the problem of learning an inner approximation of the region of attraction (ROA) of an asymptotically stable equilibrium point using trajectory data. We show that a τ-recurrent set containing a stable equilibrium must be a subset of its ROA under mild assumptions. We then develop algorithms that compute inner approximations of the ROA using counter-examples of recurrence that are obtained by sampling finite-length trajectories. Secondly, we generalize Lyapunov’s Direct Method to allow for non-monotonic evolution of the function values by only requiring sub-level sets to be τ-recurrent (instead of invariant). We provide conditions for stability, asymptotic stability, and exponential stability of an equilibrium using τ-decreasing functions (functions whose value along trajectories decrease after at most τ seconds) and develop a verification algorithm that leverages GPU parallel processing to verify such conditions using trajectories. We finalize discussing future research directions and possible extensions for control.

    @talk{cornell23,
      abstract = {In this talk, we develop model-free methods for analyzing dynamical systems using trajectory data. Our critical insight is to replace the notion of invariance, a core concept in Lyapunov Theory, with the more relaxed notion of recurrence. Specifically, a set is τ-recurrent (resp. k-recurrent) if every trajectory that starts within the set returns to it after at most τ seconds (resp. k steps). We leverage this notion of recurrence to develop several analysis tools and algorithms to study dynamical systems. Firstly, we consider the problem of learning an inner approximation of the region of attraction (ROA) of an asymptotically stable equilibrium point using trajectory data. We show that a τ-recurrent set containing a stable equilibrium must be a subset of its ROA under mild assumptions. We then develop algorithms that compute inner approximations of the ROA using counter-examples of recurrence that are obtained by sampling finite-length trajectories. Secondly, we generalize Lyapunov's Direct Method to allow for non-monotonic evolution of the function values by only requiring sub-level sets to be τ-recurrent (instead of invariant). We provide conditions for stability, asymptotic stability, and exponential stability of an equilibrium using τ-decreasing functions (functions whose value along trajectories decrease after at most τ seconds) and develop a verification algorithm that leverages GPU parallel processing to verify such conditions using trajectories. We finalize discussing future research directions and possible extensions for control.},
      date = {11/04/2023},
      day = {04},
      event = {FIND Seminar, Cornell University},
      host = {Kevin A. Tang (Cornell)},
      month = {11},
      role = {Lecture},
      title = {Model-Free Analysis of Dynamical Systems Using Recurrent Sets},
      url = {https://mallada.ece.jhu.edu/talks/202311-Cornell.pdf},
      year = {2023}
    }

  6. 2023-10-12: Reinforcement Learning with Almost Sure Constraints, MURI Workshop.
    [BibTeX] [Abstract] [Download PDF]

    In this work, we study how to tackle decision-making for safety-critical systems under uncertainty. To that end, we formulate a Reinforcement Learning problem with Almost Sure constraints, in which one seeks a policy that allows no more than $Δınℕ$ unsafe events in any trajectory, with probability one. We argue that this type of constraint might be better suited for safety-critical systems as opposed to the usual average constraint employed in Constrained Markov Decision Processes and that, moreover, having constraints of this kind makes feasible policies much easier to find. The talk is didactically split into two parts, first considering $Δ=0$ and then the $Δ≥ 0$ case. At the core of our theory is a barrier-based decomposition of the Q-function that decouples the problems of optimality and feasibility and allows them to be learned either independently or in conjunction. We develop an algorithm for characterizing the set of all feasible policies that provably converges in expected finite time. We further develop sample-complexity bounds for learning this set with high probability. Simulations corroborate our theoretical findings and showcase how our algorithm can be wrapped around other learning algorithms to hasten the search for first feasible and then optimal policies.

    @talk{muri23,
      abstract = {In this work, we study how to tackle decision-making for safety-critical systems under uncertainty. To that end, we formulate a Reinforcement Learning problem with Almost Sure constraints, in which one seeks a policy that allows no more than $Δınℕ$ unsafe events in any trajectory, with probability one. We argue that this type of constraint might be better suited for safety-critical systems as opposed to the usual average constraint employed in Constrained Markov Decision Processes and that, moreover, having constraints of this kind makes feasible policies much easier to find. The talk is didactically split into two parts, first considering $Δ=0$ and then the $Δ≥ 0$ case. At the core of our theory is a barrier-based decomposition of the Q-function that decouples the problems of optimality and feasibility and allows them to be learned either independently or in conjunction. We develop an algorithm for characterizing the set of all feasible policies that provably converges in expected finite time. We further develop sample-complexity bounds for learning this set with high probability. Simulations corroborate our theoretical findings and showcase how our algorithm can be wrapped around other learning algorithms to hasten the search for first feasible and then optimal policies.},
      date = {10/2023},
      day = {12},
      event = {MURI Workshop},
      host = {Mario Sznaier (Northeastern)},
      month = {10},
      role = {Speaker},
      title = {Reinforcement Learning with Almost Sure Constraints},
      url = {https://mallada.ece.jhu.edu/talks/202310-MURI.pdf},
      year = {2023}
    }

  7. 2023-09-07: Grid Shaping Control for High-IBR Power Systems: Stability Analysis and Control Design, GE EDGE Symposium.
    [BibTeX] [Abstract] [Download PDF]

    The transition of power systems from conventional synchronous generation towards renewable energy sources -with little or no inertia- is gradually threatening classical methods for achieving grid synchronization. A widely embraced approach to mitigate this problem is to mimic inertial response using grid-connected inverters. That is, to introduce virtual inertia to restore the stiffness that the system used to enjoy. In this talk, we seek to challenge this approach. We advocate taking advantage of the system’s low inertia to restore grid synchronism without incurring excessive control efforts. To this end, we develop an analysis and design framework for inverter-based frequency control. First, we develop novel stability analysis tools for power systems, which allow for the decentralized design of inverter-based controllers. The method requires that each inverter satisfies a standard H-infinity design requirement that depends on the dynamics of the components and inverters at each bus and the aggregate susceptance of the transmission lines connected to it. It is robust to network and delay uncertainty and, when no network information is available, reduces to the standard passivity condition for stability. Then, we propose a novel grid-forming control strategy, so-called grid shaping control, that aims to shape the frequency response of synchronous generators (SGs) to load perturbations so as to efficiently arrest sudden frequency drops. The approach builds on novel analysis tools that can characterize the Center of Inertia (CoI) response of a system with both IBRs and SGs and use this characterization to reshape it.

    @talk{ge-edge23,
      abstract = {The transition of power systems from conventional synchronous generation towards renewable energy sources -with little or no inertia- is gradually threatening classical methods for achieving grid synchronization. A widely embraced approach to mitigate this problem is to mimic inertial response using grid-connected inverters. That is, to introduce virtual inertia to restore the stiffness that the system used to enjoy. In this talk, we seek to challenge this approach. We advocate taking advantage of the system's low inertia to restore grid synchronism without incurring excessive control efforts. To this end, we develop an analysis and design framework for inverter-based frequency control. First, we develop novel stability analysis tools for power systems, which allow for the decentralized design of inverter-based controllers. The method requires that each inverter satisfies a standard H-infinity design requirement that depends on the dynamics of the components and inverters at each bus and the aggregate susceptance of the transmission lines connected to it. It is robust to network and delay uncertainty and, when no network information is available, reduces to the standard passivity condition for stability. Then, we propose a novel grid-forming control strategy, so-called grid shaping control, that aims to shape the frequency response of synchronous generators (SGs) to load perturbations so as to efficiently arrest sudden frequency drops. The approach builds on novel analysis tools that can characterize the Center of Inertia (CoI) response of a system with both IBRs and SGs and use this characterization to reshape it.},
      date = {09/20/2023},
      day = {07},
      event = {GE EDGE Symposium},
      host = {Aditya Kumar (GE)},
      month = {09},
      role = {Speaker},
      title = {Grid Shaping Control for High-IBR Power Systems: Stability Analysis and Control Design},
      url = {https://mallada.ece.jhu.edu/talks/202309-GE-EDGE.pdf},
      year = {2023}
    }

  8. 2023-09-07: Learning Coherent Clusters in Weakly Connected Power Networks, 6th Workshop on Autonomous Energy Systems.
    [BibTeX] [Abstract] [Download PDF]

    Network coherence generally refers to the emergence of a simple aggregated dynamic response of generator units, despite heterogeneity in the unit’s location and dynamic constitution. In this talk, we develop a general frequency domain framework to analyze and quantify the level of network coherence that a system exhibits by relating coherence with a low-rank property of the system’s input-output response. Our analysis unveils the frequency-dependent nature of coherence and a non-trivial interplay between dynamics, network topology, and the type of disturbance. We further leverage this framework to build structure-preserving model-reduction methodology for large-scale dynamic networks with tightly-connected components and provide time-domain bounds on the approximation error of our model. Our work provides new avenues for analysis and control designs of IBR-rich power systems.

    @talk{nrel23,
      abstract = {Network coherence generally refers to the emergence of a simple aggregated dynamic response of generator units, despite heterogeneity in the unit's location and dynamic constitution. In this talk, we develop a general frequency domain framework to analyze and quantify the level of network coherence that a system exhibits by relating coherence with a low-rank property of the system's input-output response. Our analysis unveils the frequency-dependent nature of coherence and a non-trivial interplay between dynamics, network topology, and the type of disturbance. We further leverage this framework to build structure-preserving model-reduction methodology for large-scale dynamic networks with tightly-connected components and provide time-domain bounds on the approximation error of our model. Our work provides new avenues for analysis and control designs of IBR-rich power systems. },
      date = {09/07/2023},
      day = {07},
      event = {6th Workshop on Autonomous Energy Systems},
      host = {Andrey Berstein (NREL), Guido Carvaro (NREL)},
      month = {09},
      role = {Speaker},
      title = {Learning Coherent Clusters in Weakly Connected Power Networks},
      url = {https://mallada.ece.jhu.edu/talks/202309-NREL.pdf},
      year = {2023}
    }


Recent News

A complete list of updates can be found here.


Preprints

  1. R. K. Bansal, P. You, Y. Chen, and E. Mallada, Intercept Function and Quantity Bidding in Two-stage Electricity Market with Market Power Mitigation, 2023, under revision, submitted Aug 2023.
    [BibTeX] [Abstract] [Download PDF]

    Electricity markets typically operate in two stages, day-ahead and real-time. Despite best efforts striving efficiency, evidence of price manipulation has called for system-level market power mitigation (MPM) initiatives that substitute noncompetitive bids with default bids. Implementing these policies with a limited understanding of participant behavior may lead to unintended economic losses. In this paper, we model the competition between generators and inelastic loads in a two-stage market with stage-wise MPM policies. The loss of Nash equilibrium and lack of guarantee of stable market outcome in the case of conventional supply function bidding motivates the use of an alternative market mechanism where generators bid an intercept function. A Nash equilibrium analysis for a day-ahead MPM policy leads to a Stackelberg-Nash game with loads exercising market power at the expense of generators. A comparison of the resulting equilibrium with the standard market (not implementing any MPM policy) shows that a day-ahead policy completely mitigates the market power of generators. On the other hand, the real-time MPM policy increases demand allocation to real-time, contrary to current market practice with most electricity trades in the day-ahead market. Numerical studies illustrate the impact of the slope of the intercept function on the standard market.

    @unpublished{bcym2023a-preprint,
      abstract = {Electricity markets typically operate in two stages, day-ahead and real-time. Despite best efforts striving efficiency, evidence of price manipulation has called for system-level market power mitigation (MPM) initiatives that substitute noncompetitive bids with default bids. Implementing these policies with a limited understanding of participant behavior may lead to unintended economic losses. In this paper, we model the competition between generators and inelastic loads in a two-stage market with stage-wise MPM policies. The loss of Nash equilibrium and lack of guarantee of stable market outcome in the case of conventional supply function bidding motivates the use of an alternative market mechanism where generators bid an intercept function. A Nash equilibrium analysis for a day-ahead MPM policy leads to a Stackelberg-Nash game with loads exercising market power at the expense of generators. A comparison of the resulting equilibrium with the standard market (not implementing any MPM policy) shows that a day-ahead policy completely mitigates the market power of generators. On the other hand, the real-time MPM policy increases demand allocation to real-time, contrary to current market practice with most electricity trades in the day-ahead market. Numerical studies illustrate the impact of the slope of the intercept function on the standard market.},
      author = {Bansal, Rajni Kant and You, Pengcheng and Chen, Yue and Mallada, Enrique},
      month = {11},
      pages = {1-14},
      title = {Intercept Function and Quantity Bidding in Two-stage Electricity Market with Market Power Mitigation},
      url = {https://mallada.ece.jhu.edu/pubs/2023-Preprint-BCYMb.pdf},
      year = {2023, under revision, submitted Aug 2023}
    }

  2. H. Min, R. Pates, and E. Mallada, A Frequency Domain Analysis of Slow Coherency in Networked Systems, 2024, revised, submitted Feb 2022.
    [BibTeX] [Abstract] [Download PDF]

    Network coherence generally refers to the emergence of simple aggregated dynamical behaviors, despite heterogeneity in the dynamics of the network’s subsystems. In this paper, we develop a general frequency domain framework to analyze and quantify the level of network coherence that a system exhibits by relating coherence with a low-rank property of the system’s input-output response. More precisely, for a networked system with linear dynamics and coupling, we show that, as the network’s effective algebraic connectivity grows, the system transfer matrix converges to a rank-one transfer matrix representing the coherent behavior. Interestingly, the non-zero eigenvalue of such a rank-one matrix is given by the harmonic mean of individual nodal dynamics, and we refer to it as coherent dynamics. Our analysis unveils the frequency-dependent nature of coherence and a non-trivial interplay between dynamics and network topology. We further show that many networked systems can exhibit similar coherent behavior by establishing a concentration result in a setting with randomly chosen individual nodal dynamics.

    @unpublished{mpm2023a-preprint,
      abstract = {Network coherence generally refers to the emergence of simple aggregated dynamical behaviors, despite heterogeneity in the dynamics of the network's subsystems. In this paper, we develop a general frequency domain framework to analyze and quantify the level of network coherence that a system exhibits by relating coherence with a low-rank property of the system's input-output response. More precisely, for a networked system with linear dynamics and coupling, we show that, as the network's effective algebraic connectivity grows, the system transfer matrix converges to a rank-one transfer matrix representing the coherent behavior. Interestingly, the non-zero eigenvalue of such a rank-one matrix is given by the harmonic mean of individual nodal dynamics, and we refer to it as coherent dynamics. Our analysis unveils the frequency-dependent nature of coherence and a non-trivial interplay between dynamics and network topology. We further show that many networked systems can exhibit similar coherent behavior by establishing a concentration result in a setting with randomly chosen individual nodal dynamics.},
      author = {Min, Hancheng and Pates, Richard and Mallada, Enrique},
      grants = {CAREER-1752362, TRIPODS-1934979, CPS-2136324},
      month = {2},
      pages = {1-15},
      title = {A Frequency Domain Analysis of Slow Coherency in Networked Systems},
      url = {https://mallada.ece.jhu.edu/pubs/2023-Preprint-MPM.pdf},
      year = {2024, revised, submitted Feb 2022}
    }

  3. H. Min, S. Tarmoun, R. Vidal, and E. Mallada, Convergence and Implicit Bias of Gradient Flow on Overparametrized Linear Networks, 2023, submitted.
    [BibTeX] [Abstract] [Download PDF]

    Neural networks trained via gradient descent with random initialization and without any regularization enjoy good generalization performance in practice despite being highly overparametrized. A promising direction to explain this phenomenon is to study how initialization and overparametrization affect convergence and implicit bias of training algorithms. In this paper, we present a novel analysis of single-hidden-layer linear networks trained under gradient flow, which connects initialization, optimization, and overparametrization. Firstly, we show that the squared loss converges exponentially to its optimum at a rate that depends on the level of imbalance of the initialization. Secondly, we show that proper initialization constrains the dynamics of the network parameters to lie within an invariant set. In turn, minimizing the loss over this set leads to the min-norm solution. Finally, we show that large hidden layer width, together with (properly scaled) random initialization, ensures proximity to such an invariant set during training, allowing us to derive a novel non-asymptotic upper-bound on the distance between the trained network and the min-norm solution.

    @unpublished{mtvm2023a-preprint,
      abstract = {Neural networks trained via gradient descent with random initialization and without any regularization enjoy good generalization performance in practice despite being highly overparametrized. A promising direction to explain this phenomenon is to study how initialization and overparametrization affect convergence and implicit bias of training algorithms. In this paper, we present a novel analysis of single-hidden-layer linear networks trained under gradient flow, which connects initialization, optimization, and overparametrization. Firstly, we show that the squared loss converges exponentially to its optimum at a rate that depends on the level of imbalance of the initialization. Secondly, we show that proper initialization constrains the dynamics of the network parameters to lie within an invariant set. In turn, minimizing the loss over this set leads to the min-norm solution. Finally, we show that large hidden layer width, together with (properly scaled) random initialization, ensures proximity to such an invariant set during training, allowing us to derive a novel non-asymptotic upper-bound on the distance between the trained network and the min-norm solution.        },
      author = {Min, Hancheng and Tarmoun, Salma and Vidal, Rene and Mallada, Enrique},
      grants = {CAREER-1752362, TRIPODS-1934979, CPS-2136324},
      month = {02},
      title = {Convergence and Implicit Bias of Gradient Flow on Overparametrized Linear Networks},
      url = {https://mallada.ece.jhu.edu/pubs/2022-Preprint-MTVM.pdf},
      year = {2023, submitted}
    }

  4. P. You, M. Fernandez, D. F. Gayme, and E. Mallada, Mixed Supply Function and Quantity Bidding in Two-Stage Settlement Markets, 2023, under revision, submitted Mar 2023.
    [BibTeX] [Abstract] [Download PDF]

    Motivated by electricity markets, we study the incentives of heterogeneous participants (firms and consumers) in a two-stage settlement market with a mixed bidding mechanism, in which firms participate using supply function bids and consumers use quantity bids. We carry out an equilibrium analysis of the market outcome and obtain closed-form solutions. The characterization of the equilibria allows us to gain insights into the market-power implications of mixed bidding and uncover the importance of accounting for consumers’ strategic behavior in a two-stage market, even when their demand is completely inelastic with respect to price. We show that strategic consumers are able to exploit firms’ strategic behavior to maintain a systematic difference between the forward and spot prices, with the latter being higher. Notably, such a strategy does bring down consumer payment and undermines the supply-side market power. However, it is only effective when firms are behaving strategically. We also observe situations where firms lose profit by behaving strategically, a sign of overturn of the conventional supply-side market power. Our results further suggest that market competition has a heterogeneous impact across consumer sizes, particularly benefiting small consumers. Our analysis can accommodate other market policies, and we demonstrate this versatility by examining the impact of some example policies, including virtual bidding, on the market outcome.

    @unpublished{yfgm2023a-preprint,
      abstract = {Motivated by electricity markets, we study the incentives of heterogeneous participants (firms and consumers) in a two-stage settlement market with a mixed bidding mechanism, in which firms participate using supply function bids and consumers use quantity bids. We carry out an equilibrium analysis of the market outcome and obtain closed-form solutions.  The characterization of the equilibria allows us to gain insights into the market-power implications of mixed bidding and uncover the importance of accounting for consumers' strategic behavior in a two-stage market, even when their demand is completely inelastic with respect to price. We show that strategic consumers are able to exploit firms' strategic behavior to maintain a systematic difference between the forward and spot prices, with the latter being higher. Notably, such a strategy does bring down consumer payment and undermines the supply-side market power. However, it is only effective when firms are behaving strategically. We also observe situations where firms lose profit by behaving strategically, a sign of overturn of the conventional supply-side market power. Our results further suggest that market competition has a heterogeneous impact across consumer sizes, particularly benefiting small consumers. Our analysis can accommodate other market policies, and we demonstrate this versatility by examining the impact of some example policies, including virtual bidding, on the market outcome.},
      author = {You, Pengcheng and Fernandez, Marcelo and Gayme, Dennice F. and Mallada, Enrique},
      grants = {CAREER-1752362;TRIPODS-1934979;CPS-2136324},
      month = {8},
      pages = {1-45},
      title = {Mixed Supply Function and Quantity Bidding in Two-Stage Settlement Markets},
      url = {https://mallada.ece.jhu.edu/pubs/2023-Preprint-YFGM.pdf},
      year = {2023, under revision, submitted Mar 2023}
    }

  5. P. You, Y. Jiang, E. Yeung, D. Gayme, and E. Mallada, On the Stability, Economic Efficiency and Incentive Compatibility of Electricity Market Dynamics, 2023, under revision, submitted Dec 2021.
    [BibTeX] [Abstract] [Download PDF]

    This paper focuses on the operation of an electricity market that accounts for participants that bid at a sub-minute timescale. To that end, we model the market-clearing process as a dynamical system, called market dynamics, which is temporally coupled with the grid frequency dynamics and is thus required to guarantee system-wide stability while meeting the system operational constraints. We characterize participants as price-takers who rationally update their bids to maximize their utility in response to real-time schedules of prices and dispatch. For two common bidding mechanisms, based on quantity and price, we identify a notion of alignment between participants’ behavior and planners’ goals that leads to a saddle-based design of the market that guarantees convergence to a point meeting all operational constraints. We further explore cases where this alignment property does not hold and observe that misaligned participants’ bidding can destabilize the closed-loop system. We thus design a regularized version of the market dynamics that recovers all the desirable stability and steady-state performance guarantees. Numerical tests validate our results on the IEEE 39-bus system.

    @unpublished{yjygm2023a-preprint,
      abstract = {This paper focuses on the operation of an electricity market that accounts for participants that bid at a sub-minute timescale. To that end, we model the market-clearing process as a dynamical system, called market dynamics, which is temporally coupled with the grid frequency dynamics and is thus required to guarantee system-wide stability while meeting the system operational constraints. We characterize participants as price-takers who rationally update their bids to maximize their utility in response to real-time schedules of prices and dispatch. For two common bidding mechanisms, based on quantity and price, we identify a notion of alignment between participants' behavior and planners' goals that leads to a saddle-based design of the market that guarantees convergence to a point meeting all operational constraints. We further explore cases where this alignment property does not hold and observe that misaligned participants' bidding can destabilize the closed-loop system.  We thus design a regularized version of the market dynamics that recovers all the desirable stability and steady-state performance guarantees. Numerical tests validate our results on the IEEE 39-bus system.},
      author = {You, Pengcheng and Jiang, Yan and Yeung, Enoch and Gayme, Dennice and Mallada, Enrique},
      grants = {CAREER-1752362, CPS-2136324},
      month = {10},
      pages = {1-16},
      title = {On the Stability, Economic Efficiency and Incentive Compatibility of Electricity Market Dynamics},
      url = {https://mallada.ece.jhu.edu/pubs/2021-Preprint-YJYGM.pdf},
      year = {2023, under revision, submitted Dec 2021}
    }

  6. T. Zheng, N. Loizou, P. You, and E. Mallada, Dissipative Gradient Descent Ascent Method: A Control Theory Inspired Algorithm for Min-max Optimization, 2024, submitted.
    [BibTeX] [Abstract] [Download PDF]

    Gradient Descent Ascent (GDA) methods for min-max optimization problems typically produce oscillatory behavior that can lead to instability, e.g., in bilinear settings. To address this problem, we introduce a dissipation term into the GDA updates to dampen these oscillations. The proposed Dissipative GDA (DGDA) method can be seen as performing standard GDA on a state-augmented and regularized saddle function that does not strictly introduce additional convexity/concavity. We theoretically show the linear convergence of DGDA in the bilinear and strongly convex-strongly concave settings and assess its performance by comparing DGDA with other methods such as GDA, Extra-Gradient (EG), and Optimistic GDA. Our findings demonstrate that DGDA surpasses these methods, achieving superior convergence rates. We support our claims with two numerical examples that showcase DGDA’s effectiveness in solving saddle point problems.

    @unpublished{zlym2024a-preprint,
      abstract = {Gradient Descent Ascent (GDA) methods for min-max optimization problems typically produce oscillatory behavior that can lead to instability, e.g., in bilinear settings.
    To address this problem, we introduce a dissipation term into the GDA updates to dampen these oscillations. The proposed Dissipative GDA (DGDA) method can be seen as performing standard GDA on a state-augmented and regularized saddle function that does not strictly introduce additional convexity/concavity. We theoretically show the linear convergence of DGDA in the bilinear and strongly convex-strongly concave settings and assess its performance by comparing DGDA with other methods such as GDA, Extra-Gradient (EG), and Optimistic GDA.
    Our findings demonstrate that DGDA surpasses these methods, achieving superior convergence rates. We support our claims with two numerical examples that showcase DGDA's effectiveness in solving saddle point problems.},
      author = {Zheng, Tianqi and Loizou, Nicolas and You, Pengcheng and Mallada, Enrique},
      grants = {CPS-2136324, Global-Centers-2330450},
      month = {03},
      title = {Dissipative Gradient Descent Ascent Method: A Control Theory Inspired Algorithm for Min-max Optimization},
      url = {https://mallada.ece.jhu.edu/pubs/2024-Preprint-ZLYM.pdf},
      year = {2024, submitted}
    }

  7. T. Zheng, J. W. Simpson-Porco, and E. Mallada, Closed-Loop Motion Planning for Differentially Flat Systems: A Time-Varying Optimization Framework, 2023, submitted.
    [BibTeX] [Abstract] [Download PDF]

    Motion planning and control are two core components of the robotic systems autonomy stack. The standard approach to combine these methodologies comprises an offline/open-loop stage, \textitplanning, that designs a feasible and safe trajectory to follow, and an online/closed-loop stage, \textittracking, that corrects for unmodeled dynamics and disturbances. Such an approach generally introduces conservativeness into the planning stage, which becomes difficult to overcome as the model complexity increases and real-time decisions need to be made in a changing environment. This work addresses these challenges for the class of differentially flat nonlinear systems by integrating planning and control into a cohesive closed-loop task. Precisely, we develop an optimization-based framework that aims to steer a differentially flat system to a trajectory implicitly defined via a constrained \textittime-varying optimization problem. To that end, we generalize the notion of feedback linearization, which makes non-linear systems behave as linear systems, and develop controllers that effectively transform a differentially flat system into an optimization algorithm that seeks to find the optimal solution of a (possibly time-varying) optimization problem. Under sufficient regularity assumptions, we prove global asymptotic convergence for the optimization dynamics to the minimizer of the time-varying optimization problem. We illustrate the effectiveness of our method with two numerical examples: a multi-robot tracking problem and a robot obstacle avoidance problem.

    @unpublished{zsm2023a-preprint,
      abstract = {Motion planning and control are two core components of the robotic systems autonomy stack. The standard approach to combine these methodologies comprises an offline/open-loop stage, \textitplanning, that designs a feasible and safe trajectory to follow, and an online/closed-loop stage, \textittracking, that corrects for unmodeled dynamics and disturbances. Such an approach generally introduces conservativeness into the planning stage, which becomes difficult to overcome as the model complexity increases and real-time decisions need to be made in a changing environment. This work addresses these challenges for the class of differentially flat nonlinear systems by integrating planning and control into a cohesive closed-loop task. Precisely, we develop an optimization-based framework that aims to steer a differentially flat system to a trajectory implicitly defined via a constrained \textittime-varying optimization problem. To that end, we generalize the notion of feedback linearization, which makes non-linear systems behave as linear systems, and develop controllers that effectively transform a differentially flat system into an optimization algorithm that seeks to find the optimal solution of a (possibly time-varying) optimization problem. Under sufficient regularity assumptions, we prove global asymptotic convergence for the optimization dynamics to the minimizer of the time-varying optimization problem. We illustrate the effectiveness of our method with two numerical examples: a multi-robot tracking problem and a robot obstacle avoidance problem.},
      author = {Zheng, Tianqi and Simpson-Porco, John W. and Mallada, Enrique},
      grants = {CAREER-1752362,CPS-2136324,EPICS-2330450},
      month = {10},
      pages = {1-14},
      title = {Closed-Loop Motion Planning for Differentially Flat Systems: A Time-Varying Optimization Framework},
      url = {https://mallada.ece.jhu.edu/pubs/2023-Preprint-ZSM.pdf},
      year = {2023, submitted}
    }

Recent Publications

  1. R. K. Bansal, Y. Chen, P. You, and E. Mallada, “Market Power Mitigation in Two-stage Electricity Market with Supply Function and Quantity Bidding,” IEEE Transactions on Energy Markets, Policy and Regulation, vol. 1, iss. 4, pp. 512-522, 2023. doi:10.1109/TEMPR.2023.3318149
    [BibTeX] [Abstract] [Download PDF]

    The main goal of a sequential two-stage electricity market—e.g., day-ahead and real-time markets—is to operate efficiently. However, the price difference across stages due to inadequate competition and unforeseen circumstances leads to undesirable price manipulation. To mitigate this, some Inde- pendent System Operators (ISOs) proposed system-level market power mitigation (MPM) policies in addition to existing local policies. These policies aim to substitute noncompetitive bids with a default bid based on estimated generator costs. However, these policies may lead to unintended consequences when implemented without accounting for the conflicting interest of participants. In this paper, we model the competition between generators (bidding supply functions) and loads (bidding quantity) in a two-stage market with a stage-wise MPM policy. An equilibrium analysis shows that a real-time MPM policy leads to equilibrium loss, meaning no stable market outcome (Nash equilibrium) exists. A day-ahead MPM policy, besides, leads to a Stackelberg-Nash game with loads acting as leaders and generators as followers. In this setting, loads become winners, i.e., their aggregate payment is always less than competitive payments. Moreover, comparison with standard market equilibrium highlights that markets are better off without such policies. Finally, numerical studies highlight the impact of heterogeneity and load size on market equilibrium.

    @article{bcym2023tempr,
      abstract = {The main goal of a sequential two-stage electricity market---e.g., day-ahead and real-time markets---is to operate efficiently. However, the price difference across stages due to inadequate competition and unforeseen circumstances leads to undesirable price manipulation. To mitigate this, some Inde- pendent System Operators (ISOs) proposed system-level market power mitigation (MPM) policies in addition to existing local policies. These policies aim to substitute noncompetitive bids with a default bid based on estimated generator costs. However, these policies may lead to unintended consequences when implemented without accounting for the conflicting interest of participants. In this paper, we model the competition between generators (bidding supply functions) and loads (bidding quantity) in a two-stage market with a stage-wise MPM policy. An equilibrium analysis shows that a real-time MPM policy leads to equilibrium loss, meaning no stable market outcome (Nash equilibrium) exists. A day-ahead MPM policy, besides, leads to a Stackelberg-Nash game with loads acting as leaders and generators as followers. In this setting, loads become winners, i.e., their aggregate payment is always less than competitive payments. Moreover, comparison with standard market equilibrium highlights that markets are better off without such policies. Finally, numerical studies highlight the impact of heterogeneity and load size on market equilibrium.},
      author = {Bansal, Rajni Kant and Chen, Yue and You, Pengcheng and Mallada, Enrique},
      doi = {10.1109/TEMPR.2023.3318149},
      grants = {CAREER-1752362, CPS-2136324, EPICS-2330450},
      journal = {IEEE Transactions on Energy Markets, Policy and Regulation},
      month = {12},
      number = {4},
      pages = {512-522},
      record = {published, online Sep 2023, revised July 2023, under revision May 2023, submitted Jan 2023},
      title = {Market Power Mitigation in Two-stage Electricity Market with Supply Function and Quantity Bidding},
      url = {https://mallada.ece.jhu.edu/pubs/2023-TEMPR-BCYM.pdf},
      volume = {1},
      year = {2023}
    }

  2. A. Castellano, H. Min, J. Bazerque, and E. Mallada, “Learning to Act Safely with Limited Exposure and Almost Sure Certainty,” IEEE Transactions on Automatic Control, vol. 68, iss. 5, pp. 2979-2994, 2023. doi:10.1109/TAC.2023.3240925
    [BibTeX] [Abstract] [Download PDF]

    This paper aims to put forward the concept that learning to take safe actions in unknown environments, even with probability one guarantees, can be achieved without the need for an unbounded number of exploratory trials, provided that one is willing to navigate trade-offs between optimality, level of exposure to unsafe events, and the maximum detection time of unsafe actions. We illustrate this concept in two complementary settings. We first focus on the canonical multi-armed bandit problem and seek to study the intrinsic trade-offs of learning safety in the presence of uncertainty. Under mild assumptions on sufficient exploration, we provide an algorithm that provably detects all unsafe machines in an (expected) finite number of rounds. The analysis also unveils a trade-off between the number of rounds needed to secure the environment and the probability of discarding safe machines. We then consider the problem of finding optimal policies for a Markov Decision Process (MDP) with almost sure constraints. We show that the (action) value function satisfies a barrier-based decomposition which allows for the identification of feasible policies independently of the reward process. Using this decomposition, we develop a Barrier-learning algorithm, that identifies such unsafe state-action pairs in a finite expected number of steps. Our analysis further highlights a trade-off between the time lag for the underlying MDP necessary to detect unsafe actions, and the level of exposure to unsafe events. Simulations corroborate our theoretical findings, further illustrating the aforementioned trade-offs, and suggesting that safety constraints can further speed up the learning process.

    @article{cmbm2023tac,
      abstract = {This paper aims to put forward the concept that learning to take safe actions in unknown environments, even with probability one guarantees, can be achieved without the need for an unbounded number of exploratory trials, provided that one is willing to navigate trade-offs between optimality, level of exposure to unsafe events, and the maximum detection time of unsafe actions. We illustrate this concept in two complementary settings. We first focus on the canonical multi-armed bandit problem and seek to study the intrinsic trade-offs of learning safety in the presence of uncertainty.  Under mild assumptions on sufficient exploration, we provide an algorithm that provably detects all unsafe machines in an (expected) finite number of rounds. The analysis also unveils a trade-off between the number of rounds needed to secure the environment and the probability of discarding safe machines.  We then consider the problem of finding optimal policies for a Markov Decision Process (MDP) with almost sure constraints. 
    We show that the (action) value function satisfies a barrier-based decomposition which allows for the identification of feasible policies independently of the reward process. Using this decomposition, we develop a Barrier-learning algorithm, that identifies such unsafe state-action pairs in a finite expected number of steps. Our analysis further highlights a trade-off between the time lag for the underlying MDP necessary to detect unsafe actions, and the level of exposure to unsafe events. Simulations corroborate our theoretical findings, further illustrating the aforementioned trade-offs, and suggesting that safety constraints can further speed up the learning process.},
      author = {Castellano, Agustin and Min, Hancheng and Bazerque, Juan and Mallada, Enrique},
      doi = {10.1109/TAC.2023.3240925},
      grants = {CAREER-1752362, TRIPODS-1934979, CPS-2136324},
      journal = {IEEE Transactions on Automatic Control},
      month = {5},
      number = {5},
      pages = {2979-2994},
      record = {published, online May 2023, accepted Jan 2023, revised Oct 2022, submitted May 2021},
      title = {Learning to Act Safely with Limited Exposure and Almost Sure Certainty},
      url = {https://mallada.ece.jhu.edu/pubs/2023-TAC-CMBM.pdf},
      volume = {68},
      year = {2023}
    }

  3. B. K. Poolla, Y. Lin, A. Bernstein, E. Mallada, and D. Groß, “Frequency shaping control for weakly-coupled grid-forming IBRs,” IEEE Control Systems Letters (L-CSS), pp. 937-942, 2022. doi:10.1109/LCSYS.2022.3228855
    [BibTeX] [Abstract] [Download PDF]

    We introduce a novel framework to approximate the aggregate frequency dynamics of coherent synchronous generators. By leveraging recent results on dynamics concentration of tightly connected networks, we develop a hierarchy of reduced order models –based on frequency weighted balanced truncation– that accurately approximate the aggregate system response. Our results outperform existing aggregation techniques and can be shown to monotonically improve the approximation as the hierarchy order increases.

    @article{plbmg2023lcss,
      abstract = {We introduce a novel framework to approximate the aggregate frequency dynamics of coherent synchronous generators. By leveraging recent results on dynamics concentration of tightly connected networks, we develop a hierarchy of reduced order models --based on frequency weighted balanced truncation-- that accurately approximate the aggregate system response. Our results outperform existing aggregation techniques and can be shown to monotonically improve the approximation as the hierarchy order increases.},
      author = {Poolla, Bala Kameshwar and Lin, Yashen and Bernstein, Andrey and Mallada, Enrique and Groß, Dominic},
      doi = {10.1109/LCSYS.2022.3228855},
      grants = {CAREER-1752362, CPS-2136324},
      journal = {IEEE Control Systems Letters (L-CSS)},
      month = {12},
      pages = {937-942},
      record = {published, online Dec 2022, accepted Nov 2022, submitted Sep 2022.},
      title = {Frequency shaping control for weakly-coupled grid-forming IBRs},
      url = {https://mallada.ece.jhu.edu/pubs/2022-LCSS-PLBMG.pdf},
      year = {2022}
    }

  4. G. H. Oral, E. Mallada, and D. Gayme, “On the Role of Interconnection Directionality in the Quadratic Performance of Double-Integrator Networks,” IEEE Transactions on Automatic Control, vol. 67, iss. 11, pp. 6211-6218, 2022. doi:10.1109/TAC.2021.3135358
    [BibTeX] [Abstract] [Download PDF]

    This paper provides a framework to evaluate the performance of single and double integrator networks over arbitrary directed graphs. Adopting vehicular network terminology, we consider quadratic performance metrics defined by the L2-norm of position and velocity based response functions given impulsive inputs to each vehicle. We exploit the spectral properties of weighted graph Laplacians and output performance matrices to derive a novel method of computing the closed-form solutions for this general class of performance metrics, which include H2-norm based quantities as special cases. We then explore the effect of the interplay between network properties (such as edge directionality and connectivity) and the control strategy on the overall network performance. More precisely, for systems whose interconnection is described by graphs with normal Laplacian L, we characterize the role of directionality by comparing their performance with that of their undirected counterparts, represented by the Hermitian part of L. We show that, for single-integrator networks, directed and undirected graphs perform identically. However, for double-integrator networks, graph directionality -expressed by the eigenvalues of L with nonzero imaginary part- can significantly degrade performance. Interestingly, in many cases, well-designed feedback can also exploit directionality to mitigate degradation or even improve the performance to exceed that of the undirected case. Finally we focus on a system coherence metric -aggregate deviation from the state average- to investigate the relationship between performance and degree of connectivity, leading to somewhat surprising findings. For example, increasing the number of neighbors on a ω-nearest neighbor directed graph does not necessarily improve performance. Similarly, we demonstrate equivalence in performance between all-to-one and all-to-all communication graphs.

    @article{omg2022tac,
      abstract = {This paper provides a framework to evaluate the performance of single and double integrator networks over arbitrary directed graphs. Adopting vehicular network terminology, we consider quadratic performance metrics defined by the L2-norm of position and velocity based response functions given impulsive inputs to each vehicle. We exploit the spectral properties of weighted graph Laplacians and output performance matrices to derive a novel method of computing the closed-form solutions for this general class of performance metrics, which include H2-norm based quantities as special cases. We then explore the effect of the interplay between network properties (such as edge directionality and connectivity) and the control strategy on the overall network performance. More precisely, for systems whose interconnection is described by graphs with normal Laplacian L, we characterize the role of directionality by comparing their performance with that of their undirected counterparts, represented by the Hermitian part of L. We show that, for single-integrator networks, directed and undirected graphs perform identically. However, for double-integrator networks, graph directionality -expressed by the eigenvalues of L with nonzero imaginary part- can significantly degrade performance. Interestingly, in many cases, well-designed feedback can also exploit directionality to mitigate degradation or even improve the performance to exceed that of the undirected case. Finally we focus on a system coherence metric -aggregate deviation from the state average- to investigate the relationship between performance and degree of connectivity, leading to somewhat surprising findings. For example, increasing the number of neighbors on a ω-nearest neighbor directed graph does not necessarily improve performance. Similarly, we demonstrate equivalence in performance between all-to-one and all-to-all communication graphs.},
      author = {Oral, H. Giray and Mallada, Enrique and Gayme, Dennice},
      doi = {10.1109/TAC.2021.3135358},
      grants = {ENERGISE-DE-EE0008006, EPCN-1711188,AMPS-1736448, CPS-1544771, CAREER-1752362, AMPS-1736448, ARO-W911NF-17-1-0092},
      journal = {IEEE Transactions on Automatic Control},
      month = {11},
      number = {11},
      pages = {6211-6218},
      record = {published, online Dec. 2021, accepted Nov. 2021, conditionally accepted Apr 2021, revised Nov. 2020, submitted Nov. 2019},
      title = {On the Role of Interconnection Directionality in the Quadratic Performance of Double-Integrator Networks},
      url = {https://mallada.ece.jhu.edu/pubs/2019-Preprint-OMG.pdf},
      volume = {67},
      year = {2022}
    }

  5. R. K. Bansal, P. You, D. F. Gayme, and E. Mallada, “A Market Mechanism for Truthful Bidding with Energy Storage,” Electric Power Systems Research, vol. 211, iss. 108284, pp. 1-7, 2022. doi:https://doi.org/10.1016/j.epsr.2022.108284
    [BibTeX] [Abstract] [Download PDF]

    This paper proposes a market mechanism for multiinterval electricity markets with generator and storage participants. Drawing ideas from supply function bidding, we introduce a novel bid structure for storage participation that allows storage units to communicate their cost to the market using energycycling functions that map prices to cycle depths. The resulting market-clearing process–implemented via convex programming–yields corresponding schedules and payments based on traditional energy prices for power supply and per-cycle prices for storage utilization. We illustrate the benefits of our solution by comparing the competitive equilibrium of the resulting mechanism to that of an alternative solution that uses prosumer-based bids. Our solution shows several advantages over the prosumerbased approach. It does not require a priori price estimation. It also incentivizes participants to reveal their truthful cost, thus leading to an efficient, competitive equilibrium. Numerical experiments using New York Independent System Operator (NYISO) data validate our findings.

    @article{bygm2022epsr,
      abstract = {This paper proposes a market mechanism for multiinterval electricity markets with generator and storage participants. Drawing ideas from supply function bidding, we introduce a novel bid structure for storage participation that allows storage units to communicate their cost to the market using energycycling functions that map prices to cycle depths. The resulting market-clearing process--implemented via convex programming--yields corresponding schedules and payments based on traditional energy prices for power supply and per-cycle prices for storage utilization. We illustrate the benefits of our solution by comparing the competitive equilibrium of the resulting mechanism to that of an alternative solution that uses prosumer-based bids. Our solution shows several advantages over the prosumerbased approach. It does not require a priori price estimation. It also incentivizes participants to reveal their truthful cost, thus leading to an efficient, competitive equilibrium. Numerical experiments using New York Independent System Operator (NYISO) data validate our findings.},
      author = {Bansal, Rajni Kant and You, Pengcheng and Gayme, Dennice F. and Mallada, Enrique},
      doi = {https://doi.org/10.1016/j.epsr.2022.108284},
      grants = {CAREER-1752362, TRIPODS-1934979, CPS-2136324},
      issn = {0378-7796},
      journal = {Electric Power Systems Research},
      month = {10},
      note = {also in PSCC 2022},
      number = {108284},
      pages = {1-7},
      record = {published, accepted Feb. 2022, submitted Oct. 2021},
      title = {A Market Mechanism for Truthful Bidding with Energy Storage},
      url = {https://mallada.ece.jhu.edu/pubs/2022-EPSR-BYGM.pdf},
      volume = {211},
      year = {2022}
    }

  6. R. Pates, A. Ferragut, E. Pivo, P. You, F. Paganini, and E. Mallada, “Respect the Unstable: Delays and Saturation in Contact Tracing for Disease Control,” SIAM Journal on Control and Optimization, vol. 60, iss. 2, p. S196-S220, 2022. doi:https://doi.org/10.1137/20M1377825
    [BibTeX] [Abstract] [Download PDF]

    Motivated by the novel coronavirus disease (COVID-19) pandemic, this paper aims to apply Gunter Stein’s cautionary message of respecting the unstable to the problem of controlling the spread of an infectious disease. With this goal, we study the effect that delays and capacity constraints in the TeTrIs process have on preventing exponential disease spread. Our analysis highlights the critical importance of speed and scale in the TeTrIs process. Precisely, having a delay in the TeTrIs process smaller than the doubling time of the disease spread is necessary for achieving acceptable performance. Similarly, limited TeTrIs capacity introduces a threshold on the size of an outbreak beyond which the disease spreads almost like the uncontrolled case. Along the way, we provide numerical illustrations to highlight these points.

    @article{pfpypm2022sicon,
      abstract = {Motivated by the novel coronavirus disease (COVID-19) pandemic, this paper aims to apply Gunter Stein's cautionary message of respecting the unstable to the problem of controlling the spread of an infectious disease. With this goal, we study the effect that delays and capacity constraints in the TeTrIs process have on preventing exponential disease spread. Our analysis highlights the critical importance of speed and scale in the TeTrIs process. Precisely, having a delay in the TeTrIs process  smaller than the doubling time of the disease spread is necessary for achieving acceptable performance. Similarly, limited TeTrIs capacity introduces a threshold on the size of an outbreak beyond which the disease spreads almost like the uncontrolled case. Along the way, we provide numerical illustrations to highlight these points.},
      author = {Pates, Richard and Ferragut, Andres and Pivo, Elijah and You, Pengcheng and Paganini, Fernando and Mallada, Enrique},
      doi = {https://doi.org/10.1137/20M1377825},
      grants = {CAREER-1752362, EPCN-1711188, AMPS-1736448, TRIPODS-1934979},
      journal = {SIAM Journal on Control and Optimization},
      month = {4},
      number = {2},
      pages = {S196-S220},
      record = {accepted Dec 2021, revised Nov 2021, submitted Nov 2020},
      title = {Respect the Unstable: Delays and Saturation in Contact Tracing for Disease Control},
      url = {https://mallada.ece.jhu.edu/pubs/2022-SICON-PFPYPM.pdf},
      volume = {60},
      year = {2022}
    }

  7. Y. Jiang, A. Bernstein, P. Vorobev, and E. Mallada, “Grid-forming frequency shaping control in low inertia power systems,” IEEE Control Systems Letters (L-CSS), vol. 5, iss. 6, pp. 1988-1993, 2021. doi:10.1109/LCSYS.2020.3044551
    [BibTeX] [Abstract] [Download PDF]

    We introduce a novel framework to approximate the aggregate frequency dynamics of coherent synchronous generators. By leveraging recent results on dynamics concentration of tightly connected networks, we develop a hierarchy of reduced order models –based on frequency weighted balanced truncation– that accurately approximate the aggregate system response. Our results outperform existing aggregation techniques and can be shown to monotonically improve the approximation as the hierarchy order increases.

    @article{jbvm2021lcss,
      abstract = {We introduce a novel framework to approximate the aggregate frequency dynamics of coherent synchronous generators. By leveraging recent results on dynamics concentration of tightly connected networks, we develop a hierarchy of reduced order models --based on frequency weighted balanced truncation-- that accurately approximate the aggregate system response. Our results outperform existing aggregation techniques and can be shown to monotonically improve the approximation as the hierarchy order increases.},
      author = {Jiang, Yan and Bernstein, Andrey and Vorobev, Petr and Mallada, Enrique},
      doi = {10.1109/LCSYS.2020.3044551},
      grants = {CAREER-1752362, AMPS-1736448, TRIPODS-1934979, EPCN-1711188, CPS-2136324},
      journal = {IEEE Control Systems Letters (L-CSS)},
      month = {12},
      note = {also in ACC 2021},
      number = {6},
      pages = {1988-1993},
      record = {early access Dec 2020, accepted Nov 2020, revised Nov 2020, submitted Sep 2020},
      title = {Grid-forming frequency shaping control in low inertia power systems},
      url = {https://mallada.ece.jhu.edu/pubs/2021-LCSS-JBVM.pdf},
      volume = {5},
      year = {2021}
    }

  8. Y. Jiang, E. Cohn, P. Vorobev, and E. Mallada, “Storage-Based Frequency Shaping Control,” IEEE Transactions on Power Systems, vol. 36, iss. 6, pp. 5006-5019, 2021. doi:10.1109/TPWRS.2021.3072833
    [BibTeX] [Abstract] [Download PDF]

    With the decrease in system inertia, frequency security becomes an issue for power systems around the world. Energy storage systems (ESS), due to their excellent ramping capabilities, are considered as a natural choice for the improvement of frequency response following major contingencies. In this manuscript, we propose a new strategy for energy storage — frequency shaping control — that allows to completely eliminate the frequency Nadir, one of the main issue in frequency security, and at the same time tune the rate of change of frequency (RoCoF) to a desired value. With Nadir eliminated, the frequency security assessment can be performed via simple algebraic calculations, as opposed to dynamic simulations for conventional control strategies. Moreover, our proposed control is also very efficient in terms of the requirements on storage peak power, requiring up to 40% less power than conventional virtual inertia approach for the same performance.

    @article{jcvm2021tps,
      abstract = {With the decrease in system inertia, frequency security becomes an issue for power systems around the world. Energy storage systems (ESS), due to their excellent ramping capabilities, are considered as a natural choice for the improvement of frequency response following major contingencies. In this manuscript, we propose a new strategy for energy storage -- frequency shaping control -- that allows to completely eliminate the frequency Nadir, one of the main issue in frequency security, and at the same time tune the rate of change of frequency (RoCoF) to a desired value. With Nadir eliminated, the frequency security assessment can be performed via simple algebraic calculations, as opposed to dynamic simulations for conventional control strategies. Moreover, our proposed control is also very efficient in terms of the requirements on storage peak power, requiring up to 40% less power than conventional virtual inertia approach for the same performance.},
      author = {Jiang, Yan and Cohn, Eliza and Vorobev, Petr and Mallada, Enrique},
      doi = {10.1109/TPWRS.2021.3072833},
      grants = {CAREER-1752362;CPS-2136324},
      journal = {IEEE Transactions on Power Systems},
      month = {11},
      number = {6},
      pages = {5006-5019},
      record = {early access Apr 2021, accepted Mar 2021, revised Oct 2020, submitted May 2020},
      title = {Storage-Based Frequency Shaping Control},
      url = {https://mallada.ece.jhu.edu/pubs/2021-TPS-JCVM.pdf},
      volume = {36},
      year = {2021}
    }