News

1 paper accepted to RLC

by

Our paper on nonparametric policy improvement in continuous action spaces via expert demonstrations [1] has been accepted to the Reinforcement Learning Conference. Congrats Agustin!

[1] A. Castellano, S. Rezaei, J. Markowitz, and E. Mallada, “Nonparametric Policy Improvement in Continuous Action Spaces via Expert Demonstrations,” in Reinforcement Learning Conference, 2025, pp. 1158-1179.
[Bibtex] [Abstract] [Download PDF]

The policy improvement theorem is a fundamental building block of classical reinforcement learning for discrete action spaces. Unfortunately, the lack of an analogous result for continuous action spaces with function approximation has historically limited theoretical guarantees of policy optimization algorithms, undermining their reliability. Here, we introduce a novel nonparametric policy that relies purely on data to take actions and that admits a policy improvement theorem for deterministic Markov Decision Processes (MDPs). By imposing mild regularity assumptions on the optimal policy, we show that, when data come from expert demonstrations, one can construct a nonparametric lower bound on the value of the policy, thus enabling its robust evaluation. The constructed lower bound naturally leads to a simple improvement mechanism based on adding more demonstrations. We also provide conditions to identify regions of the state space where additional demonstrations are needed to meet specific performance goals. Finally, we propose a policy optimization algorithm that ensures a monotonic improvement of the lower bound and leads to high probability performance guarantees. These contributions provide a foundational step toward establishing a rigorous framework for policy improvement in continuous action spaces.

@inproceedings{crmm2025rlc,
  abstract = {The policy improvement theorem is a fundamental building block of classical reinforcement learning for discrete action spaces. Unfortunately, the lack of an analogous result for continuous action spaces with function approximation has historically limited theoretical guarantees of policy optimization algorithms, undermining their reliability. Here, we introduce a novel nonparametric policy that relies purely on data to take actions and that admits a 
policy improvement theorem for deterministic Markov Decision Processes (MDPs). By imposing mild regularity assumptions on the optimal policy, we show that, when data come from expert demonstrations, one can construct a nonparametric lower bound on the value of the policy, thus enabling its robust evaluation. The constructed lower bound naturally leads to a simple improvement mechanism based on adding more demonstrations. We also provide conditions to identify regions of the state space where additional demonstrations are needed to meet specific performance goals. Finally, we propose a policy optimization algorithm that ensures a monotonic improvement of the lower bound and leads to high probability performance guarantees. These contributions provide a foundational step toward establishing a rigorous framework for policy improvement in continuous action spaces.},
  author = {Castellano, Agustin and Rezaei, Sohrab and Markowitz, Jared and Mallada, Enrique},
  booktitle = {Reinforcement Learning Conference},
  month = {8},
  pages = {1158-1179},
  record = {presented Aug. 2025, accepted May 2025, submitted Feb. 2025},
  title = {Nonparametric Policy Improvement in Continuous Action Spaces via Expert Demonstrations},
  url = {https://mallada.ece.jhu.edu/pubs/2025-RLC-CRMM.pdf},
  year = {2025}
}

1 paper accepted to ACC

by

Our tutorial paper on safe physics-informed machine learning for dynamics and control [1] has been accepted to the American Control Conference.

[1] [doi] J. Drgona, T. X. Nghiem, T. Beckers, M. Fazlyab, E. Mallada, C. Jones, D. Vrabie, S. L. Brunton, and R. Findeisen, “Safe Physics-informed Machine Learning for Dynamics and Control,” in American Control Conference (ACC), 2025, pp. 591-606.
[Bibtex] [Abstract] [Download PDF]

This tutorial paper focuses on safe physics-informed machine learning in the context of dynamics and control, providing a comprehensive overview of how to integrate physical models and safety guarantees. As machine learning techniques enhance the modeling and control of complex dynamical systems, ensuring safety and stability remains a critical challenge, especially in safety-critical applications like autonomous vehicles, robotics, medical decision-making, and energy systems. We explore various approaches for embedding and ensuring safety constraints, including structural priors, Lyapunov and Control Barrier Functions, predictive control, projections, and robust optimization techniques. Additionally, we delve into methods for uncertainty quantification and safety verification, including reachability analysis and neural network verification tools, which help validate that control policies remain within safe operating bounds even in uncertain environments. The paper includes illustrative examples demonstrating the implementation aspects of safe learning frameworks that combine the strengths of data-driven approaches with the rigor of physical principles, offering a path toward the safe control of complex dynamical systems.

@inproceedings{dnbetal2025acc,
  abstract = {This tutorial paper focuses on safe physics-informed machine learning in the context of dynamics and control, providing a comprehensive overview of how to integrate physical models and safety guarantees. As machine learning techniques enhance the modeling and control of complex dynamical systems, ensuring safety and stability remains a critical challenge, especially in safety-critical applications like autonomous vehicles, robotics, medical decision-making, and energy systems. We explore various approaches for embedding and ensuring safety constraints, including structural priors, Lyapunov and Control Barrier Functions, predictive control, projections, and robust optimization techniques. Additionally, we delve into methods for uncertainty quantification and safety verification, including reachability analysis and neural network verification tools, which help validate that control policies remain within safe operating bounds even in uncertain environments. The paper includes illustrative examples demonstrating the implementation aspects of safe learning frameworks that combine the strengths of data-driven approaches with the rigor of physical principles, offering a path toward the safe control of complex dynamical systems.},
  author = {Drgona, Jan and Nghiem, Truong X. and Beckers, Thomas and Fazlyab, Mahyar and Mallada, Enrique and Jones, Colin and Vrabie, Draguna and Brunton, Steven L. and Findeisen, Rolf},
  booktitle = {American Control Conference (ACC)},
  doi = {10.23919/ACC63710.2025.11107836},
  month = {7},
  pages = {591-606},
  record = {presented Jul. 2025, accepted May 2025, submitted Mar. 2025},
  title = {Safe Physics-informed Machine Learning for Dynamics and Control},
  url = {https://mallada.ece.jhu.edu/pubs/2025-ACC-Tutorial-DNBetal.pdf},
  year = {2025}
}

Agustin received the WSE Teaching Assistant Award

by

Agustin Castellano received the Whiting School of Engineering Teaching Assistant Award, recognizing his exemplary support of students and faculty and his commitment to academic excellence as a graduate teaching assistant. Congrats Agustin!

2 papers accepted to AISTATS

by

Our papers on variance-aware linear UCB for neural contextual bandits [1] and on the learning dynamics of LoRA [2] have been accepted to the International Conference on Artificial Intelligence and Statistics. Congrats Ziqing!

[1] H. M. Bui, E. Mallada, and A. Liu, “Variance-Aware Linear UCB with Deep Representation for Neural Contextual Bandits,” in International Conference on Artificial Intelligence and Statistics (AISTATS), 2025.
[Bibtex] [Abstract] [Download PDF]

By leveraging the representation power of deep neural networks, neural upper confidence bound (UCB) algorithms have shown success in contextual bandits. To further balance the exploration and exploitation, we propose Neural-$σ^2$-LinearUCB, a variance-aware algorithm that utilizes $σ^2_t$, i.e., an upper bound of the reward noise variance at round $t$, to enhance the uncertainty quantification quality of the UCB, resulting in a regret performance improvement. We provide an oracle version for our algorithm characterized by an oracle variance upper bound $σ^2_t$ and a practical version with a novel estimation for this variance bound. Theoretically, we provide rigorous regret analysis for both versions and prove that our oracle algorithm achieves a better regret guarantee than other neural-UCB algorithms in the neural contextual bandits setting. Empirically, our practical method enjoys a similar computational efficiency, while outperforming state-of-the-art techniques by having a better calibration and lower regret across multiple standard settings, including on the synthetic, UCI, MNIST, and CIFAR-10 datasets.

@inproceedings{bml2025aistats,
  abstract = {By leveraging the representation power of deep neural networks, neural upper confidence bound (UCB) algorithms have shown success in contextual bandits. To further balance the exploration and exploitation, we propose Neural-$σ^2$-LinearUCB, a variance-aware algorithm that utilizes $σ^2_t$, i.e., an upper bound of the reward noise variance at round $t$, to enhance the uncertainty quantification quality of the UCB, resulting in a regret performance improvement. We provide an oracle version for our algorithm characterized by an oracle variance upper bound $σ^2_t$ and a practical version with a novel estimation for this variance bound. Theoretically, we provide rigorous regret analysis for both versions and prove that our oracle algorithm achieves a better regret guarantee than other neural-UCB algorithms in the neural contextual bandits setting. Empirically, our practical method enjoys a similar computational efficiency, while outperforming state-of-the-art techniques by having a better calibration and lower regret across multiple standard settings, including on the synthetic, UCI, MNIST, and CIFAR-10 datasets.},
  author = {Bui, Ha Manh and Mallada, Enrique and Liu, Anqi},
  booktitle = {International Conference on Artificial Intelligence and Statistics (AISTATS)},
  grants = {No Grant},
  month = {4},
  publisher = {PMLR},
  record = {accepted Jan 2025, submitted Oct 2024},
  series = {Proceedings of Machine Learning Research},
  title = {Variance-Aware Linear UCB with Deep Representation for Neural Contextual Bandits},
  url = {https://mallada.ece.jhu.edu/pubs/2025-AISTATS-BML.pdf},
  year = {2025}
}
[2] Z. Xu, H. Min, L. E. MacDonald, J. Luo, S. Tarmoun, E. Mallada, and R. Vidal, “Understanding the Learning Dynamics of LoRA: A Gradient Flow Perspective on Low-Rank Adaptation in Matrix Factorization,” in International Conference on Artificial Intelligence and Statistics (AISTATS), 2025.
[Bibtex] [Abstract] [Download PDF]

Despite the empirical success of Low-Rank Adaptation (LoRA) in fine-tuning pretrained models, there is little theoretical understanding of how first-order methods with carefully crafted initialization adapt models to new tasks. In this work, we take the first step towards bridging this gap by theoretically analyzing the learning dynamics of LoRA for matrix factorization (MF) under gradient flow (GF), emphasizing the crucial role of initialization. For small initialization, we theoretically show that GF converges to a neighborhood of the optimal solution, with smaller initialization leading to lower final error. Our analysis shows that the final error is affected by the misalignment between the singular spaces of the pre-trained model and the target matrix, and reducing the initialization scale improves alignment. To address this misalignment, we propose a spectral initialization for LoRA in MF and theoretically prove that GF with small spectral initialization converges to the fine-tuning task with arbitrary precision. Numerical experiments from MF and image classification validate our findings.

@inproceedings{xmmltmv2025aistats,
  abstract = {Despite the empirical success of Low-Rank Adaptation (LoRA) in fine-tuning pretrained models, there is little theoretical understanding of how first-order methods with carefully crafted initialization adapt models to new tasks. In this work, we take the first step towards bridging this gap by theoretically analyzing the learning dynamics of LoRA for matrix factorization (MF) under gradient flow (GF), emphasizing the crucial role of initialization. For small initialization, we theoretically show that GF converges to a neighborhood of the optimal solution, with smaller initialization leading to lower final error. Our analysis shows that the final error is affected by the misalignment between the singular spaces of the pre-trained model and the target matrix, and reducing the initialization scale improves alignment. To address this misalignment, we propose a spectral initialization for LoRA in MF and theoretically prove that GF with small spectral initialization converges to the fine-tuning task with arbitrary precision. Numerical experiments from MF and image classification validate our findings.},
  author = {Xu, Ziqing and Min, Hancheng and MacDonald, Lachlan Ewen and Luo, Jinqi and Tarmoun, Salma and Mallada, Enrique and Vidal, Rene},
  booktitle = {International Conference on Artificial Intelligence and Statistics (AISTATS)},
  grants = {Global Centers},
  month = {4},
  publisher = {PMLR},
  record = {accepted Jan 2024, submitted Oct 2024},
  series = {Proceedings of Machine Learning Research},
  title = {Understanding the Learning Dynamics of LoRA: A Gradient Flow Perspective on Low-Rank Adaptation in Matrix Factorization},
  url = {https://mallada.ece.jhu.edu/pubs/2025-AISTATS-XMMLTMV.pdf},
  year = {2025}
}

1 paper published in Automatica

by

Our paper on a frequency domain analysis of slow coherency in networked systems [1] has been published in Automatica. Congrats Hancheng!

[1] [doi] H. Min, R. Pates, and E. Mallada, “A Frequency Domain Analysis of Slow Coherency in Networked Systems,” Automatica, vol. 74, pp. 1-13, 2025.
[Bibtex] [Abstract] [Download PDF]

Network coherence generally refers to the emergence of simple aggregated dynamical behaviors, despite heterogeneity in the dynamics of the network’s subsystems. In this paper, we develop a general frequency domain framework to analyze and quantify the level of network coherence that a system exhibits by relating coherence with a low-rank property of the system’s input-output response. More precisely, for a networked system with linear dynamics and coupling, we show that, as the network’s effective algebraic connectivity grows, the system transfer matrix converges to a rank-one transfer matrix representing the coherent behavior. Interestingly, the non-zero eigenvalue of such a rank-one matrix is given by the harmonic mean of individual nodal dynamics, and we refer to it as coherent dynamics. Our analysis unveils the frequency-dependent nature of coherence and a non-trivial interplay between dynamics and network topology. We further show that many networked systems can exhibit similar coherent behavior by establishing a concentration result in a setting with randomly chosen individual nodal dynamics.

@article{mpm2025automatica,
  abstract = {Network coherence generally refers to the emergence of simple aggregated dynamical behaviors, despite heterogeneity in the dynamics of the network's subsystems. In this paper, we develop a general frequency domain framework to analyze and quantify the level of network coherence that a system exhibits by relating coherence with a low-rank property of the system's input-output response. More precisely, for a networked system with linear dynamics and coupling, we show that, as the network's effective algebraic connectivity grows, the system transfer matrix converges to a rank-one transfer matrix representing the coherent behavior. Interestingly, the non-zero eigenvalue of such a rank-one matrix is given by the harmonic mean of individual nodal dynamics, and we refer to it as coherent dynamics. Our analysis unveils the frequency-dependent nature of coherence and a non-trivial interplay between dynamics and network topology. We further show that many networked systems can exhibit similar coherent behavior by establishing a concentration result in a setting with randomly chosen individual nodal dynamics.},
  author = {Min, Hancheng and Pates, Richard and Mallada, Enrique},
  bdsk-url-3 = {https://mallada.ece.jhu.edu/pubs/2025-Automatica-MPM.pdf},
  bdsk-url-4 = {https://doi.org/10.1016/j.automatica.2025.112184},
  doi = {https://doi.org/10.1016/j.automatica.2025.112184},
  grants = {CAREER-1752362, TRIPODS-1934979, CPS-2136324},
  journal = {Automatica},
  month = {2},
  pages = {1-13},
  record = {published, available online Dec 2024, accepted Oct 2024, revised Feb 2024, submitted Feb 2022},
  title = {A Frequency Domain Analysis of Slow Coherency in Networked Systems},
  url = {https://mallada.ece.jhu.edu/pubs/2025-Automatica-MPM.pdf},
  volume = {74},
  year = {2025}
}

3 papers accepted to CDC

by

Our papers on Dissipative Gradient Descent-Ascent algorithms for min-max problems [1], Accelerated saddle flow dynamics [2], and invertibility of sparse linear dynamical systems [3] have been accepted to the 63rd IEEE Conference on Decision and Control. Congrats to the leading students Tianqi, Yingzhu, Kyle, and to all the rest of the team!

[1] [doi] T. Zheng, N. Loizou, P. You, and E. Mallada, “Dissipative Gradient Descent Ascent Method: A Control Theory Inspired Algorithm for Min-max Optimization,” in 63rd IEEE Conference on Decision and Control (CDC), 2024.
[Bibtex] [Abstract] [Download PDF]

Gradient Descent Ascent (GDA) methods for min-max optimization problems typically produce oscillatory behavior that can lead to instability, e.g., in bilinear settings. To address this problem, we introduce a dissipation term into the GDA updates to dampen these oscillations. The proposed Dissipative GDA (DGDA) method can be seen as performing standard GDA on a state-augmented and regularized saddle function that does not strictly introduce additional convexity/concavity. We theoretically show the linear convergence of DGDA in the bilinear and strongly convex-strongly concave settings and assess its performance by comparing DGDA with other methods such as GDA, Extra-Gradient (EG), and Optimistic GDA. Our findings demonstrate that DGDA surpasses these methods, achieving superior convergence rates. We support our claims with two numerical examples that showcase DGDA’s effectiveness in solving saddle point problems.

@inproceedings{zlym2024cdc,
  abstract = {Gradient Descent Ascent (GDA) methods for min-max optimization problems typically produce oscillatory behavior that can lead to instability, e.g., in bilinear settings.
To address this problem, we introduce a dissipation term into the GDA updates to dampen these oscillations. The proposed Dissipative GDA (DGDA) method can be seen as performing standard GDA on a state-augmented and regularized saddle function that does not strictly introduce additional convexity/concavity. We theoretically show the linear convergence of DGDA in the bilinear and strongly convex-strongly concave settings and assess its performance by comparing DGDA with other methods such as GDA, Extra-Gradient (EG), and Optimistic GDA.
Our findings demonstrate that DGDA surpasses these methods, achieving superior convergence rates. We support our claims with two numerical examples that showcase DGDA's effectiveness in solving saddle point problems.},
  author = {Zheng, Tianqi and Loizou, Nicolas and You, Pengcheng and Mallada, Enrique},
  booktitle = {63rd IEEE Conference on Decision and Control (CDC)},
  doi = {10.1109/LCSYS.2024.3413004},
  grants = {CPS-2136324, Global-Centers-2330450},
  month = {12},
  note = {also in L-CSS},
  record = {presented Dec 2024, accepted Jul 2024, submitted Mar 2024},
  title = {Dissipative Gradient Descent Ascent Method: A Control Theory Inspired Algorithm for Min-max Optimization},
  url = {https://mallada.ece.jhu.edu/pubs/2024-CDC-ZLYM.pdf},
  year = {2024}
}
[2] [doi] Y. Liu, E. Mallada, Z. Li, and P. You, “Accelerated Saddle Flow Dynamics for Bilinearly Coupled Minimax Problems,” in 63rd IEEE Conference on Decision and Control (CDC), 2024.
[Bibtex] [Abstract] [Download PDF]

Minimax problems have attracted much attention due to various applications in constrained optimization problems and zero-sum games. Identifying saddle points within these problems is crucial, and saddle flow dynamics offer a straightforward yet useful approach. This study focuses on a class of bilinearly coupled minimax problems and designs an accelerated algorithm based on saddle flow dynamics that achieves a convergence rate beyond stereotype limits. The algorithm is derived based on a sequential two-step transformation of the objective function. First, a change of variable is aimed at a better-conditioned saddle function. Second, a proximal regularization, when staggered with the first step, guarantees strong convexity-strong concavity of the objective function that can be tuned for accelerated exponential convergence. Besides, such an approach can be extended to a class of weakly convex-weakly concave functions and still achieve exponential convergence to one stationary point. The theory is verified by a numerical test on an affine equality-constrained convex optimization problem.

@inproceedings{lmly2024cdc,
  abstract = {Minimax problems have attracted much attention due to various applications in constrained optimization problems and zero-sum games. Identifying saddle points within these problems is crucial, and saddle flow dynamics offer a straightforward yet useful approach. This study focuses on a class of bilinearly coupled minimax problems and designs an accelerated algorithm based on saddle flow dynamics that achieves a convergence rate beyond stereotype limits.  The algorithm is derived based on a sequential two-step transformation of the objective function. First, a change of variable is aimed at a better-conditioned saddle function. Second, a proximal regularization, when staggered with the first step, guarantees strong convexity-strong concavity of the objective function that can be tuned for accelerated exponential convergence.
Besides, such an approach can be extended to a class of weakly convex-weakly concave functions and still achieve exponential convergence to one stationary point. The theory is verified by a numerical test on an affine equality-constrained convex optimization problem.},
  author = {Liu, Yingzhu and Mallada, Enrique and Li, Zhongkui and You, Pengcheng},
  booktitle = {63rd IEEE Conference on Decision and Control (CDC)},
  doi = {10.1109/CDC56724.2024.10886124},
  grants = {CPS-2136324, Global-Centers-2330450},
  month = {07},
  record = {presented Dec. 2024, accepted Jul. 2024, submitted Mar. 2024},
  title = {Accelerated Saddle Flow Dynamics for Bilinearly Coupled Minimax Problems},
  url = {https://mallada.ece.jhu.edu/pubs/2024-Preprint-LMLY.pdf},
  year = {2024}
}
[3] [doi] K. Poe, E. Mallada, and R. Vidal, “Invertibility of Discrete-Time Linear Systems with Sparse Inputs,” in 63rd IEEE Conference on Decision and Control (CDC), 2024.
[Bibtex] [Abstract] [Download PDF]

One of the fundamental problems of interest for discrete-time linear systems is whether its input sequence may be recovered given its output sequence, a.k.a. the left inversion problem. Many conditions on the state space geometry, dynamics, and spectral structure of a system have been used to characterize the well-posedness of this problem, without assumptions on the inputs. However, certain structural assumptions, such as input sparsity, have been shown to translate to practical gains in the performance of inversion algorithms, surpassing classical guarantees. Establishing necessary and sufficient conditions for left invertibility of systems with sparse inputs is therefore a crucial step toward understanding the performance limits of system inversion under structured input assumptions. In this work, we provide the first necessary and sufficient characterizations of left invertibility for linear systems with sparse inputs, echoing classic characterizations for standard linear systems. The key insight in deriving these results is in establishing the existence of two novel geometric invariants unique to the sparse-input setting, the weakly unobservable and strongly reachable subspace arrangements. By means of a concrete example, we demonstrate the utility of these characterizations. We conclude by discussing extensions and applications of this framework to several related problems in sparse control.

@inproceedings{pmv2024cdc,
  abstract = {One of the fundamental problems of interest for discrete-time linear systems is whether its input sequence may be recovered given its output sequence, a.k.a. the left inversion problem. Many conditions on the state space geometry, dynamics, and spectral structure of a system have been used to characterize the well-posedness of this problem, without assumptions on the inputs. However, certain structural assumptions, such as input sparsity, have been shown to translate to practical gains in the performance of inversion algorithms, surpassing classical guarantees. Establishing necessary and sufficient conditions for left invertibility of systems with sparse inputs is therefore a crucial step toward understanding the performance limits of system inversion under structured input assumptions. In this work, we provide the first necessary and sufficient characterizations of left invertibility for linear systems with sparse inputs,  echoing classic characterizations for standard linear systems. The key insight in deriving these results is in establishing the existence of two novel geometric invariants unique to the sparse-input setting, the weakly unobservable and strongly reachable subspace arrangements. By means of a concrete example, we demonstrate the utility of these characterizations. We conclude by discussing extensions and applications of this framework to several related problems in sparse control.},
  author = {Poe, Kyle and Mallada, Enrique and Vidal, Rene},
  booktitle = {63rd IEEE Conference on Decision and Control (CDC)},
  doi = {10.1109/CDC56724.2024.10886207},
  grants = {CPS-2136324; Global-Centers-2330450},
  month = {12},
  record = {presented Dec 2024, accepted Jul 2024, submitted Mar 2024},
  title = {Invertibility of Discrete-Time Linear Systems with Sparse Inputs},
  url = {https://mallada.ece.jhu.edu/pubs/2024-CDC-PMV.pdf},
  year = {2024}
}

1 paper accepted to L-CSS

by

Our paper on Dissipative Gradient Descent-Ascent algorithms for min-max problems [1] has been accepted to the IEEE Control Systems Letters. Congrats Tianqi!

[1] [doi] T. Zheng, N. Loizou, P. You, and E. Mallada, “Dissipative Gradient Descent Ascent Method: A Control Theory Inspired Algorithm for Min-max Optimization,” IEEE Control Systems Letters (L-CSS), vol. 8, pp. 2009-2014, 2024.
[Bibtex] [Abstract] [Download PDF]

Gradient Descent Ascent (GDA) methods for min-max optimization problems typically produce oscillatory behavior that can lead to instability, e.g., in bilinear settings. To address this problem, we introduce a dissipation term into the GDA updates to dampen these oscillations. The proposed Dissipative GDA (DGDA) method can be seen as performing standard GDA on a state-augmented and regularized saddle function that does not strictly introduce additional convexity/concavity. We theoretically show the linear convergence of DGDA in the bilinear and strongly convex-strongly concave settings and assess its performance by comparing DGDA with other methods such as GDA, Extra-Gradient (EG), and Optimistic GDA. Our findings demonstrate that DGDA surpasses these methods, achieving superior convergence rates. We support our claims with two numerical examples that showcase DGDA’s effectiveness in solving saddle point problems.

@article{zlym2024lcss,
  abstract = {Gradient Descent Ascent (GDA) methods for min-max optimization problems typically produce oscillatory behavior that can lead to instability, e.g., in bilinear settings.
To address this problem, we introduce a dissipation term into the GDA updates to dampen these oscillations. The proposed Dissipative GDA (DGDA) method can be seen as performing standard GDA on a state-augmented and regularized saddle function that does not strictly introduce additional convexity/concavity. We theoretically show the linear convergence of DGDA in the bilinear and strongly convex-strongly concave settings and assess its performance by comparing DGDA with other methods such as GDA, Extra-Gradient (EG), and Optimistic GDA.
Our findings demonstrate that DGDA surpasses these methods, achieving superior convergence rates. We support our claims with two numerical examples that showcase DGDA's effectiveness in solving saddle point problems.},
  author = {Zheng, Tianqi and Loizou, Nicolas and You, Pengcheng and Mallada, Enrique},
  bdsk-url-3 = {https://doi.org/10.1109/LCSYS.2024.3413004},
  doi = {10.1109/LCSYS.2024.3413004},
  grants = {CPS-2136324, Global-Centers-2330450},
  journal = {IEEE Control Systems Letters (L-CSS)},
  month = {06},
  pages = {2009-2014},
  record = {published, accepted May 2024, submitted Mar 2024},
  title = {Dissipative Gradient Descent Ascent Method: A Control Theory Inspired Algorithm for Min-max Optimization},
  url = {https://mallada.ece.jhu.edu/pubs/2024-LCSS-ZLYM.pdf},
  volume = {8},
  year = {2024}
}

1 paper accepted to ICLR

by

Our paper on Early Neuron Alignment in Two-layer ReLU Networks [1] has been accepted to the International Conference on Representation Learning. Congrats Hancheng!

[1] H. Min, R. Vidal, and E. Mallada, “Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization,” in International Conference on Representation Learning (ICLR), 2024.
[Bibtex] [Abstract] [Download PDF]

This paper studies the problem of training a two-layer ReLU network for binary classification using gradient flow with small initialization. We consider a training dataset with well-separated input vectors: Any pair of input data with the same label are positively correlated, and any pair with different labels are negatively correlated. Our analysis shows that, during the early phase of training, neurons in the first layer try to align with either the positive data or the negative data, depending on its corresponding weight on the second layer. A careful analysis of the neurons’ directional dynamics allows us to provide an $\mathcalO(\fracłog n\sqrtμ)$ upper bound on the time it takes for all neurons to achieve good alignment with the input data, where $n$ is the number of data points and $μ$ measures how well the data are separated. After the early alignment phase, the loss converges to zero at a $\mathcalO(\frac1t)$ rate, and the weight matrix on the first layer is approximately low-rank. Numerical experiments on the MNIST dataset illustrate our theoretical findings.

@inproceedings{mvm2024iclr,
  abstract = {This paper studies the problem of training a two-layer ReLU network for binary classification using gradient flow with small initialization. We consider a training dataset with well-separated input vectors: Any pair of input data with the same label are positively correlated, and any pair with different labels are negatively correlated. Our analysis shows that, during the early phase of training, neurons in the first layer try to align with either the positive data or the negative data, depending on its corresponding weight on the second layer. A careful analysis of the neurons' directional dynamics allows us to provide an $\mathcalO(\fracłog n\sqrtμ)$ upper bound on the time it takes for all neurons to achieve good alignment with the input data, where $n$ is the number of data points and $μ$ measures how well the data are separated. After the early alignment phase, the loss converges to zero at a $\mathcalO(\frac1t)$ rate, and the weight matrix on the first layer is approximately low-rank. Numerical experiments on the MNIST dataset illustrate our theoretical findings.},
  author = {Min, Hancheng and Vidal, Rene and Mallada, Enrique},
  booktitle = {International Conference on Representation Learning (ICLR)},
  grants = {CAREER-1752362},
  month = {05},
  record = {published, accepted Jan 2024, submitted Sep 2023},
  title = {Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization},
  url = {https://mallada.ece.jhu.edu/pubs/2024-ICLR-MVM.pdf},
  year = {2024}
}