Year: 2022

1 paper accepted to LCSS

by

Our paper on frequency shaping control for grid-forming IBRs [1] has been accepted to Control Systems Letters!

[1] [doi] B. K. Poolla, Y. Lin, A. Bernstein, E. Mallada, and D. Groß, “Frequency shaping control for weakly-coupled grid-forming IBRs,” IEEE Control Systems Letters (L-CSS), pp. 937-942, 2022.
[Bibtex] [Abstract] [Download PDF]

We consider the problem of controlling the fre- quency of low-inertia power systems via inverter-based resources (IBRs) that are weakly connected to the grid. We propose a novel grid-forming control strategy, the so-called frequency shaping control, that aims to shape the frequency response of synchronous generators (SGs) to load perturbations so as to efficiently arrest sudden frequency drops. Our solution relaxes several existing assumptions in the literature and is able to navigate trade- offs between peak power requirements and maximum frequency deviations. Finally, we analyze the robustness to imperfect knowl- edge of network parameters, while particularly highlighting the importance of accurate estimation of these parameters.

@article{plbmg2023lcss,
  abstract = {We consider the problem of controlling the fre- quency of low-inertia power systems via inverter-based resources (IBRs) that are weakly connected to the grid. We propose a novel grid-forming control strategy, the so-called frequency shaping control, that aims to shape the frequency response of synchronous generators (SGs) to load perturbations so as to efficiently arrest sudden frequency drops. Our solution relaxes several existing assumptions in the literature and is able to navigate trade- offs between peak power requirements and maximum frequency deviations. Finally, we analyze the robustness to imperfect knowl- edge of network parameters, while particularly highlighting the importance of accurate estimation of these parameters.},
  author = {Poolla, Bala Kameshwar and Lin, Yashen and Bernstein, Andrey and Mallada, Enrique and Groß, Dominic},
  bdsk-url-3 = {https://doi.org/10.1109/LCSYS.2022.3228855},
  doi = {10.1109/LCSYS.2022.3228855},
  grants = {CAREER-1752362, CPS-2136324},
  journal = {IEEE Control Systems Letters (L-CSS)},
  month = {12},
  pages = {937-942},
  record = {published, online Dec 2022, accepted Nov 2022, submitted Sep 2022.},
  title = {Frequency shaping control for weakly-coupled grid-forming IBRs},
  url = {https://mallada.ece.jhu.edu/pubs/2022-LCSS-PLBMG.pdf},
  year = {2022}
}

1 paper published in TAC

by

Our paper on the role of interconnection directionality in the quadratic performance of double-integrator networks [1] has been published in the IEEE Transactions on Automatic Control.

[1] [doi] G. H. Oral, E. Mallada, and D. Gayme, “On the Role of Interconnection Directionality in the Quadratic Performance of Double-Integrator Networks,” IEEE Transactions on Automatic Control, vol. 67, iss. 11, pp. 6211-6218, 2022.
[Bibtex] [Abstract] [Download PDF]

This note provides a quantitative and qualitative eval- uation of the role of interconnection directionality in a general class of quadratic performance metrics for double-integrator net- works. We first develop an analysis framework that can be used to evaluate the quadratic performance metrics of networks defined over a general class of directed graphs. A comparison between systems whose directed graph Laplacians are normal and their undirected counterparts unveils an interplay between the inter- connection directionality and the control strategy that determines network performance. We show that directionality can significantly degrade performance; however well-designed feedback can exploit directionality to mitigate this degradation or even improve perfor- mance.

@article{omg2022tac,
  abstract = {This note provides a quantitative and qualitative eval- uation of the role of interconnection directionality in a general class of quadratic performance metrics for double-integrator net- works. We first develop an analysis framework that can be used to evaluate the quadratic performance metrics of networks defined over a general class of directed graphs. A comparison between systems whose directed graph Laplacians are normal and their undirected counterparts unveils an interplay between the inter- connection directionality and the control strategy that determines network performance. We show that directionality can significantly degrade performance; however well-designed feedback can exploit directionality to mitigate this degradation or even improve perfor- mance.},
  author = {Oral, H. Giray and Mallada, Enrique and Gayme, Dennice},
  bdsk-url-3 = {https://doi.org/10.1109/TAC.2021.3135358},
  doi = {10.1109/TAC.2021.3135358},
  grants = {ENERGISE-DE-EE0008006, EPCN-1711188,AMPS-1736448, CPS-1544771, CAREER-1752362, AMPS-1736448, ARO-W911NF-17-1-0092},
  journal = {IEEE Transactions on Automatic Control},
  month = {11},
  number = {11},
  pages = {6211-6218},
  record = {published, online Dec. 2021, accepted Nov. 2021, conditionally accepted Apr 2021, revised Nov. 2020, submitted Nov. 2019},
  title = {On the Role of Interconnection Directionality in the Quadratic Performance of Double-Integrator Networks},
  url = {https://mallada.ece.jhu.edu/pubs/2022-TAC-OMG.pdf},
  volume = {67},
  year = {2022}
}

Jay defended his dissertation

by

Jay Guthrie, an ECE Ph.D. student in our lab, defended his dissertation entitled “Novel Representations of Semialgebraic Sets Arising in Planning and Control” on Friday, October 21st. Congratulations!

1 paper accepted to Asilomar

by

Our paper on constrained reinforcement learning via dissipative saddle flow dynamics [1] has been accepted to the 56th Asilomar Conference on Signals, Systems, and Computers. Congrats Tianqi!

[1] [doi] T. Zheng, P. You, and E. Mallada, “Constrained Reinforcement Learning via Dissipative Saddle Flow Dynamics,” in 56th Asilomar Conference on Signals, Systems, and Computers, 2022, pp. 1362-1366.
[Bibtex] [Abstract] [Download PDF]

In constrained reinforcement learning (C-RL), an agent seeks to learn from the environment a policy that maximizes the expected cumulative reward while satisfying minimum requirements in secondary cumulative reward con- straints. Several algorithms rooted in sampled-based primal- dual methods have been recently proposed to solve this problem in policy space. However, such methods are based on stochastic gradient descent-ascent algorithms whose trajectories are con- nected to the optimal policy only after a mixing output stage that depends on the algorithm’s history. As a result, there is a mismatch between the behavioral policy and the optimal one. In this work, we propose a novel algorithm for constrained RL that does not suffer from these limitations. Leveraging recent results on regularized saddle-flow dynamics, we develop a novel stochastic gradient descent-ascent algorithm whose trajectories converge to the optimal policy almost surely.

@inproceedings{zym2022asilomar,
  abstract = {In constrained reinforcement learning (C-RL), an agent seeks to learn from the environment a policy that maximizes the expected cumulative reward while satisfying minimum requirements in secondary cumulative reward con- straints. Several algorithms rooted in sampled-based primal- dual methods have been recently proposed to solve this problem in policy space. However, such methods are based on stochastic gradient descent-ascent algorithms whose trajectories are con- nected to the optimal policy only after a mixing output stage that depends on the algorithm's history. As a result, there is a mismatch between the behavioral policy and the optimal one. In this work, we propose a novel algorithm for constrained RL that does not suffer from these limitations. Leveraging recent results on regularized saddle-flow dynamics, we develop a novel stochastic gradient descent-ascent algorithm whose trajectories converge to the optimal policy almost surely.},
  author = {Zheng, Tianqi and You, Pengcheng and Mallada, Enrique},
  bdsk-url-3 = {https://doi.org/10.1109/IEEECONF56349.2022.10052060},
  booktitle = {56th Asilomar Conference on Signals, Systems, and Computers},
  doi = {10.1109/IEEECONF56349.2022.10052060},
  grants = {CAREER-1752362, TRIPODS-1934979, CPS-2136324},
  month = {12},
  pages = {1362-1366},
  record = {presented Dec. 2022, accepted Sep. 2022, submitted Apr. 2022},
  title = {Constrained Reinforcement Learning via Dissipative Saddle Flow Dynamics},
  url = {https://mallada.ece.jhu.edu/pubs/2022-Asilomar-ZYM.pdf},
  year = {2022}
}

1 paper accepted to CDC

by

Our paper on model-free learning of regions of attraction via recurrent sets [1] has been accepted to the 61st IEEE Conference on Decision and Control. Congrats Yue!

[1] [doi] Y. Shen, M. Bichuch, and E. Mallada, “Model-free Learning of Regions of Attraction via Recurrent Sets,” in 61st IEEE Conference on Decision and Control (CDC), 2022, pp. 4714-4719.
[Bibtex] [Abstract] [Download PDF]

We consider the problem of learning an inner approximation of the region of attraction (ROA) of an asymptotically stable equilibrium point without an explicit model of the dynamics. Rather than leveraging approximate models with bounded uncertainty to find a (robust) invariant set contained in the ROA, we propose to learn sets that satisfy a more relaxed notion of containment known as recurrence. We define a set to be $τ$-recurrent (resp. $k$-recurrent) if every trajectory that starts within the set, returns to it after at most $τ$ seconds (resp. $k$ steps). We show that under mild assumptions a $τ$-recurrent set containing a stable equilibrium must be a subset of its ROA. We then leverage this property to develop algorithms that compute inner approximations of the ROA using counter-examples of recurrence that are obtained by sampling finite-length trajectories. Our algorithms process samples sequentially, which allow them to continue being executed even after an initial offline training stage. We further provide an upper bound on the number of counter-examples used by the algorithm, and almost sure convergence guarantees.

@inproceedings{sbm2022cdc,
  abstract = {We consider the problem of learning an inner approximation of the region of attraction (ROA) of an asymptotically stable equilibrium point without an explicit model of the dynamics. Rather than leveraging approximate models with bounded uncertainty to find a (robust) invariant set contained in the ROA, we propose to learn sets that satisfy a more relaxed notion of containment known as recurrence. We define a set to be $τ$-recurrent (resp. $k$-recurrent) if every trajectory that starts within the set, returns to it after at most $τ$ seconds (resp. $k$ steps). We show that under mild assumptions a $τ$-recurrent set containing a stable equilibrium must be a subset of its ROA. We then leverage this property to develop algorithms that compute inner approximations of the ROA using  counter-examples of recurrence that are obtained by sampling finite-length trajectories. Our algorithms process samples sequentially, which allow them to continue being executed even after an initial offline training stage. We further provide an upper bound on the number of counter-examples used by the algorithm, and almost sure convergence guarantees.},
  author = {Shen, Yue and Bichuch, Maxim and Mallada, Enrique},
  bdsk-url-3 = {https://doi.org/10.1109/CDC51059.2022.9993280},
  booktitle = {61st IEEE Conference on Decision and Control (CDC)},
  doi = {10.1109/CDC51059.2022.9993280},
  grants = {CAREER-1752362, TRIPODS-1934979, CPS-2136324},
  month = {12},
  pages = {4714-4719},
  record = {presented Dec. 2022, accepted Sep 2022, submitted Mar 2022},
  title = {Model-free Learning of Regions of Attraction via Recurrent Sets},
  url = {https://mallada.ece.jhu.edu/pubs/2022-CDC-SBM.pdf},
  year = {2022}
}

I got tenure! :P

by

I was promoted to Associate Professor with tenure! Thanks to all students, collaborators, mentors, and sponsors that helped make this possible.

1 paper accepted to ACM e-Energy

by

Our paper on equilibrium analysis of electricity markets with day-ahead market power mitigation and real-time intercept bidding [1] has been accepted to ACM e-Energy. Congrats Rajni!

[1] [doi] R. K. Bansal, Y. Chen, P. You, and E. Mallada, “Equilibrium Analysis of Electricity Markets with Day-Ahead Market Power Mitigation and Real-Time Intercept Bidding,” in Proceedings of the Thirteenth ACM International Conference on Future Energy Systems (e-Energy), 2022, pp. 47-62.
[Bibtex] [Abstract] [Download PDF]

Electricity markets are cleared by a two-stage, sequential process consisting of a forward (day-ahead) market and a spot (real-time) market. While their design goal is to achieve efficiency, the lack of sufficient competition introduces many opportunities for price manipulation. To discourage this phenomenon, some Independent System Operators (ISOs) mandate generators to submit (approximately) truthful bids in the day-ahead market. However, without fully accounting for all participants’ incentives (generators and loads), the application of such a mandate may lead to unintended consequences. In this paper, we model and study the interactions of generators and inelastic loads in a two-stage settlement where generators are required to bid truthfully in the day-ahead market. We show that such mandate, when accounting for generator and load incentives, leads to a generalized Stackelberg-Nash game where load decisions (leaders) are performed in day-ahead market and generator decisions (followers) are relegated to the real-time market. Furthermore, the use of conventional supply function bidding for generators in real-time, does not guarantee the existence of a Nash equilibrium. This motivates the use of intercept bidding, as an alternative bidding mechanism for generators in the real-time market. An equilibrium analysis in this setting, leads to a closed-form solution that unveils several insights. Particularly, it shows that, unlike standard two-stage markets, loads are the winners of the competition in the sense that their aggregate payments are less than that of the competitive equilibrium. Moreover, heterogeneity in generators cost has the unintended effect of mitigating loads market power. Numerical studies validate and further illustrate these insights.

@inproceedings{bcym2022e-energy,
  abstract = {Electricity markets are cleared by a two-stage, sequential process consisting of a forward (day-ahead) market and a spot (real-time) market. While their design goal is to achieve efficiency, the lack of sufficient competition introduces many opportunities for price manipulation. To discourage this phenomenon, some Independent System Operators (ISOs) mandate generators to submit (approximately) truthful bids in the day-ahead market. However, without fully accounting for all participants' incentives (generators and loads), the application of such a mandate may lead to unintended consequences. In this paper, we model and study the interactions of generators and inelastic loads in a two-stage settlement where generators are required to bid truthfully in the day-ahead market. We show that such mandate, when accounting for generator and load incentives, leads to a generalized Stackelberg-Nash game where load decisions (leaders) are performed in day-ahead market and generator decisions (followers) are relegated to the real-time market. Furthermore, the use of conventional supply function bidding for generators in real-time, does not guarantee the existence of a Nash equilibrium. This motivates the use of intercept bidding, as an alternative bidding mechanism for generators in the real-time market. An equilibrium analysis in this setting, leads to a closed-form solution that unveils several insights. Particularly, it shows that, unlike standard two-stage markets, loads are the winners of the competition in the sense that their aggregate payments are less than that of the competitive equilibrium. Moreover, heterogeneity in generators cost has the unintended effect of mitigating loads market power. Numerical studies validate and further illustrate these insights.},
  author = {Bansal, Rajni Kant and Chen, Yue and You, Pengcheng and Mallada, Enrique},
  bdsk-url-3 = {https://doi.org/10.1145/3538637.3538839},
  booktitle = {Proceedings of the Thirteenth ACM International Conference on Future Energy Systems (e-Energy)},
  doi = {https://doi.org/10.1145/3538637.3538839},
  grants = {CAREER-1752362,EPCN-1711188,CPS-2136324},
  month = {6},
  pages = {47--62},
  record = {published, accepted Jun 2022, submitted Feb 2022.},
  title = {Equilibrium Analysis of Electricity Markets with Day-Ahead Market Power Mitigation and Real-Time Intercept Bidding},
  url = {https://mallada.ece.jhu.edu/pubs/2022-e-Energy-BCYM.pdf},
  year = {2022}
}