arXiv subjects

Juan Ramirez

Publications and source records attributed to Juan Ramirez.

14 recordsLinked to original sources

arXiv2025

Dual Optimistic Ascent (PI Control) is the Augmented Lagrangian Method in Disguise

Constrained optimization is a powerful framework for enforcing requirements on neural networks. These constrained deep learning problems are typically solved using first-order methods on their min-max Lagrangian formulation, but such approaches often suffer from oscillations and can fail to find all local solutions. While the Augmented Lagrangian method (ALM) addresses these issues, practitioners often favor dual optimistic ascent schemes (PI control) on the standard Lagrangian, which perform well empirically but lack formal guarantees. In this paper, we establish a previously unknown equivalence between these approaches: dual optimistic ascent on the Lagrangian is equivalent to gradient descent-ascent on the Augmented Lagrangian. This finding allows us to transfer the robust theoretical guarantees of the ALM to the dual optimistic setting, proving it converges linearly to all local solutions. Furthermore, the equivalence provides principled guidance for tuning the optimism hyper-parameter. Our work closes a critical gap between the empirical success of dual optimistic methods and their theoretical foundation in the single-step, first-order regime commonly used in constrained deep learning.

cs.LG↗

arXiv2025

A mathematical model of vaccine hesitancy: Analysing the impact of political trends and the interaction across age and education groups in the USA

Vaccination against the SARS-CoV-2 disease has significantly reduced its mortality rate and spread. However, despite its availability, a considerable proportion of the public has either refused or delayed getting vaccinated. This reluctance is known as vaccine hesitancy. The aim of this paper is to present a mathematical model to investigate how social interaction can impact vaccine hesitancy. The model describes the temporal transitions between different vaccination classes of the population (those vaccinated, those who are not yet vaccinated but agree to be vaccinated, and those who refuse). We apply the model to state and national survey data from the USA to estimate model parameters that quantify the rates at which public opinion on vaccination changes. Moreover, we investigate how political trends and demographic factors, such as age and education, impact these parameters. Our results show that state-level political affiliation, age, and educational level shape opinions on vaccination and have a strong influence on the temporal dynamics of attitude changes.

q-bio.PE↗

arXiv2025

Position: Adopt Constraints Over Fixed Penalties in Deep Learning

Recent efforts to develop trustworthy AI systems have increased interest in learning problems with explicit requirements, or constraints. In deep learning, however, such problems are often handled through fixed weighted-sum penalization: the constraints are added to the task loss with fixed coefficients, and the resulting scalarized objective is minimized. This position paper argues that fixed penalization is often ill-suited for deep learning problems with non-negotiable requirements for several reasons. First, in non-convex settings, the penalized and constrained problems are generally not equivalent, so solving the former need not solve the latter. Second, fixed penalization weakens hard requirements into soft penalties to be traded off against task performance. Third, choosing penalty coefficients to indirectly solve the constrained problem often involves costly trial and error, because changing them alters the penalized objective itself, and hence can mean solving the wrong problem altogether. We therefore argue that, when a deep learning problem specifies non-negotiable requirements, the constrained formulation itself should be the starting point, not the surrogate problem defined by fixed penalization. The appropriate solution strategy should then be chosen based on the problem's structure and scale.

cs.LG↗

arXiv2025

Cooper: A Library for Constrained Optimization in Deep Learning

Cooper is an open-source package for solving constrained optimization problems involving deep learning models. Cooper implements several Lagrangian-based first-order update schemes, making it easy to combine constrained optimization algorithms with high-level features of PyTorch such as automatic differentiation, and specialized deep learning architectures and optimizers. Although Cooper is specifically designed for deep learning applications where gradients are estimated based on mini-batches, it is suitable for general non-convex continuous constrained optimization. Cooper's source code is available at https://github.com/cooper-org/cooper.

cs.LG↗

arXiv2025

Feasible Learning

We introduce Feasible Learning (FL), a sample-centric learning paradigm where models are trained by solving a feasibility problem that bounds the loss for each training sample. In contrast to the ubiquitous Empirical Risk Minimization (ERM) framework, which optimizes for average performance, FL demands satisfactory performance on every individual data point. Since any model that meets the prescribed performance threshold is a valid FL solution, the choice of optimization algorithm and its dynamics play a crucial role in shaping the properties of the resulting solutions. In particular, we study a primal-dual approach which dynamically re-weights the importance of each sample during training. To address the challenge of setting a meaningful threshold in practice, we introduce a relaxation of FL that incorporates slack variables of minimal norm. Our empirical analysis, spanning image classification, age regression, and preference optimization in large language models, demonstrates that models trained via FL can learn from data while displaying improved tail behavior compared to ERM, with only a marginal impact on average performance.

cs.LG↗

arXiv2024

On PI Controllers for Updating Lagrange Multipliers in Constrained Optimization

Constrained optimization offers a powerful framework to prescribe desired behaviors in neural network models. Typically, constrained problems are solved via their min-max Lagrangian formulations, which exhibit unstable oscillatory dynamics when optimized using gradient descent-ascent. The adoption of constrained optimization techniques in the machine learning community is currently limited by the lack of reliable, general-purpose update schemes for the Lagrange multipliers. This paper proposes the $\nu$PI algorithm and contributes an optimization perspective on Lagrange multiplier updates based on PI controllers, extending the work of Stooke, Achiam and Abbeel (2020). We provide theoretical and empirical insights explaining the inability of momentum methods to address the shortcomings of gradient descent-ascent, and contrast this with the empirical success of our proposed $\nu$PI controller. Moreover, we prove that $\nu$PI generalizes popular momentum methods for single-objective minimization. Our experiments demonstrate that $\nu$PI reliably stabilizes the multiplier dynamics and its hyperparameters enjoy robust and predictable behavior.

cs.LG↗

arXiv2023

Balancing Act: Constraining Disparate Impact in Sparse Models

Model pruning is a popular approach to enable the deployment of large deep learning models on edge devices with restricted computational or storage capacities. Although sparse models achieve performance comparable to that of their dense counterparts at the level of the entire dataset, they exhibit high accuracy drops for some data sub-groups. Existing methods to mitigate this disparate impact induced by pruning (i) rely on surrogate metrics that address the problem indirectly and have limited interpretability; or (ii) scale poorly with the number of protected sub-groups in terms of computational cost. We propose a constrained optimization approach that directly addresses the disparate impact of pruning: our formulation bounds the accuracy change between the dense and sparse models, for each sub-group. This choice of constraints provides an interpretable success criterion to determine if a pruned model achieves acceptable disparity levels. Experimental results demonstrate that our technique scales reliably to problems involving large models and hundreds of protected sub-groups.

cs.LG↗

arXiv2023

Omega: Optimistic EMA Gradients

Stochastic min-max optimization has gained interest in the machine learning community with the advancements in GANs and adversarial training. Although game optimization is fairly well understood in the deterministic setting, some issues persist in the stochastic regime. Recent work has shown that stochastic gradient descent-ascent methods such as the optimistic gradient are highly sensitive to noise or can fail to converge. Although alternative strategies exist, they can be prohibitively expensive. We introduce Omega, a method with optimistic-like updates that mitigates the impact of noise by incorporating an EMA of historic gradients in its update rule. We also explore a variation of this algorithm that incorporates momentum. Although we do not provide convergence guarantees, our experiments on stochastic games show that Omega outperforms the optimistic gradient method when applied to linear players.

cs.LG↗

arXiv2022

Controlled Sparsity via Constrained Optimization or: How I Learned to Stop Tuning Penalties and Love Constraints

The performance of trained neural networks is robust to harsh levels of pruning. Coupled with the ever-growing size of deep learning models, this observation has motivated extensive research on learning sparse models. In this work, we focus on the task of controlling the level of sparsity when performing sparse learning. Existing methods based on sparsity-inducing penalties involve expensive trial-and-error tuning of the penalty factor, thus lacking direct control of the resulting model sparsity. In response, we adopt a constrained formulation: using the gate mechanism proposed by Louizos et al. (2018), we formulate a constrained optimization problem where sparsification is guided by the training objective and the desired sparsity target in an end-to-end fashion. Experiments on CIFAR-{10, 100}, TinyImageNet, and ImageNet using WideResNet and ResNet{18, 50} models validate the effectiveness of our proposal and demonstrate that we can reliably achieve pre-determined sparsity targets without compromising on predictive performance.

cs.LG↗

arXiv2022

L$_0$onie: Compressing COINs with L$_0$-constraints

Advances in Implicit Neural Representations (INR) have motivated research on domain-agnostic compression techniques. These methods train a neural network to approximate an object, and then store the weights of the trained model. For example, given an image, a network is trained to learn the mapping from pixel locations to RGB values. In this paper, we propose L$_0$onie, a sparsity-constrained extension of the COIN compression method. Sparsity allows to leverage the faster learning of overparameterized networks, while retaining the desirable compression rate of smaller models. Moreover, our constrained formulation ensures that the final model respects a pre-determined compression rate, dispensing of the need for expensive architecture search.

cs.LG↗

arXiv2021

Covariance Estimation from Compressive Data Partitions using a Projected Gradient-based Algorithm

Compressive covariance estimation has arisen as a class of techniques whose aim is to obtain second-order statistics of stochastic processes from compressive measurements. Recently, these methods have been used in various image processing and communications applications, including denoising, spectrum sensing, and compression. Notice that estimating the covariance matrix from compressive samples leads to ill-posed minimizations with severe performance loss at high compression rates. In this regard, a regularization term is typically aggregated to the cost function to consider prior information about a particular property of the covariance matrix. Hence, this paper proposes an algorithm based on the projected gradient method to recover low-rank or Toeplitz approximations of the covariance matrix from compressive measurements. The algorithm divides the compressive measurements into data subsets projected onto different subspaces and accurately estimates the covariance matrix by solving a single optimization problem assuming that each data subset contains an approximation of the signal statistics. Furthermore, gradient filtering is included at every iteration of the proposed algorithm to minimize the estimation error. The error induced by the proposed splitting approach is analytically derived along with the convergence guarantees of the proposed method. The algorithm estimates the covariance matrix of hyperspectral images from synthetic and real compressive samples. Extensive simulations show that the proposed algorithm can effectively recover the covariance matrix of hyperspectral images from compressive measurements (8-15% approx). Moreover, simulations and theoretical results show that the filtering step reduces the recovery error up to twice the number of eigenvectors. Finally, an optical implementation is proposed, and real measurements are used to validate the theoretical findings.

eess.IV↗

arXiv2019

Visualizing the Support of Kostant's Weight Multiplicity Formula for the Rank Two Lie Algebras

The multiplicity of a weight in a finite-dimensional irreducible representation of a simple Lie algebra g can be computed via Kostant's weight multiplicity formula. This formula consists of an alternating sum over the Weyl group (a finite group) and involves a partition function known as Kostant's partition function. Motivated by the observation that, in practice, most terms in the sum are zero, our main results describe the elements of the Weyl alternation sets. The Weyl alternation sets are subsets of the Weyl group which contributes nontrivially to the multiplicity of a weight in a highest weight representation of the Lie algebras so_4(C), so_5(C), sp_4(C), and the exceptional Lie algebra g_2. By taking a geometric approach, we extend the work of Harris, Lescinsky, and Mabie on sl_3(C), to provide visualizations of these Weyl alternation sets for all pairs of integral weights \lambda and \mu of the Lie algebras considered.

math.CO↗

arXiv2004

A Conceptual Tour About the Standard Cosmological Model

With the beginning of the XXIst century, a physical model of our Universe, usually called the Standard Cosmological Model (SCM), is reaching an important level of consolidation, based on accurate astrophysical data and also on theoretical developments. In this paper we review the interplay between the basic concepts and observations underlying this model. The SCM is a complex and beautiful building, recieving inputs from many branches of physics. Major topics reviewed are: General Relativity and the cosmological constant, the Cosmological Principle and Friedmann-Robertson-Walker-Lemaitre models, Hubble diagrams and dark energy, large scale structure and dark matter, the cosmic microwave background, Big Bang nucleosynthesis, and inflation.

astro-ph↗

arXiv2001

Cosmological perturbations: a new gauge-invariant approach

A new gauge-invariant approach for describing cosmological perturbations is developed. It is based on a physically motivated splitting of the stress-energy tensor of the perturbation into two parts - the bare perturbation and the complementary perturbation associated with stresses in the background gravitational field induced by the introduction of the bare perturbation. The complementary perturbation of the stress-energy tensor is explicitly singled out and taken to the left side of the perturbed Einstein equations so that the bare stress-energy tensor is the sole source for the perturbation of the metric tensor and both sides of these equations are gauge invariant with respect to infinitesimal coordinate transformations. For simplicity we analyze the perturbations of the spatially-flat Friedmann-Lemaitre-Robertson-Walker dust model. A cosmological gauge can be chosen such that the equations for the perturbations of the metric tensor are completely decoupled for the h_{00}, h_{0i}, and h_{ij} metric components and explicitly solvable in terms of retarded integrals.

gr-qc↗