arXiv subjects

Laura Selicato

Publications and source records attributed to Laura Selicato.

4 recordsLinked to original sources

arXiv2025

Binno: A 1st-order method for Bi-level Nonconvex Nonsmooth Optimization for Matrix Factorizations

Nonconvex and nonsmooth bi-level optimization poses critical theoretical challenges, while arising in several applications. In this work, we develop a method for nonconvex, nonsmooth bi-level optimization and introduce Binno, a first-order method that builds on proximal-gradient updates within the the proximal alternate minimization framework with descent conditions from variational analysis. Binno couples the two levels via a descent-driven averaging mechanism, extending single-level proximal schemes to the nonconvex nonsmooth bi-level setting. We show thatBinno induces a descent property for a suitable surrogate of the bi-level objective. Each iteration performs blockwise proximal-gradient updates for the upper and lower problems separately, then forms a calibrated, block-diagonal convex combination of the two iterates. A linesearch selects combination weights enforcing simultaneous descent of both objectives. We give conditions ensuring that both these weights and the descent directions induced by the associated proximal-gradient maps exist. We apply Binno to sparse low-rank factorization, where the upper level uses elementwise l1 penalties and the lower level uses nuclear norms, coupled via a Frobenius data term. We test Binno on synthetic matrices and a real traffic-video dataset, achieving lower reconstruction error and higher peak signal-to-noise ratio than standard methods. We also validate it on a regularized market-clearing problem, where it selects policy-preferred equilibria, and compare it with bi-level baseline, showing consistent improvements

math.OC↗

arXiv2025

Sparse Hyperparametric Itakura-Saito Nonnegative Matrix Factorization via Bi-Level Optimization

The selection of penalty hyperparameters is a critical aspect in Nonnegative Matrix Factorization (NMF), since these values control the trade-off between reconstruction accuracy and adherence to desired constraints. In this work, we focus on an NMF problem involving the Itakura-Saito (IS) divergence, which is particularly effective for extracting low spectral density components from spectrograms of mixed signals, and benefits from the introduction of sparsity constraints. We propose a new algorithm called SHINBO, which introduces a bi-level optimization framework to automatically and adaptively tune the row-dependent penalty hyperparameters, enhancing the ability of IS-NMF to isolate sparse, periodic signals in noisy environments. Experimental results demonstrate that SHINBO achieves accurate spectral decompositions and demonstrates superior performance in both synthetic and real-world applications. In the latter case, SHINBO is particularly useful for noninvasive vibration-based fault detection in rolling bearings, where the desired signal components often reside in high-frequency subbands but are obscured by stronger, spectrally broader noise. By addressing the critical issue of hyperparameter selection, SHINBO improves the state-of-the-art in signal recovery for complex, noise-dominated environments.

cs.LG↗

arXiv2023

Bi-level algorithm for optimizing hyperparameters in penalized nonnegative matrix factorization

Learning approaches rely on hyperparameters that impact the algorithm's performance and affect the knowledge extraction process from data. Recently, Nonnegative Matrix Factorization (NMF) has attracted a growing interest as a learning algorithm. This technique captures the latent information embedded in large datasets while preserving feature properties. NMF can be formalized as a penalized optimization task in which tuning the penalty hyperparameters is an open issue. The current literature does not provide any general framework addressing this task. This study proposes to express the penalty hyperparameters problem in NMF in terms of a bi-level optimization. We design a novel algorithm, named Alternating Bi-level (AltBi), which incorporates the hyperparameters tuning procedure into the updates of NMF factors. Results of the existence and convergence of numerical solutions, under appropriate assumptions, are studied, and numerical experiments are provided.

math.NA↗

arXiv2023

Theoretical aspects in penalty hyperparameters optimization

Learning processes are useful methodologies able to improve knowledge of real phenomena. These are often dependent on hyperparameters, variables set before the training process and regulating the learning procedure. Hyperparameters optimization problem is an open issue in learning approaches since it can strongly affect any real data analysis. They are usually selected using Grid-Search or Cross Validation techniques. No automatic tuning procedure exists especially if we focus on an unsupervised learning scenario. This study aims to assess some theoretical considerations for tuning penalty hyperparameters in optimization problems. It considers a bi-level formulation tuning problem in an unsupervised context, by using Gradient-based methods. Suitable conditions for the existence of a minimizer in an infinite-dimensional Hilbert space are outlined, together with some theoretical results, applicable in all those situations when it is unnecessary or not possible obtaining an exact minimizer. An iterative algorithmic strategy is considered, equipped with a stopping criterion via Ekeland's variational principle.

math.OC↗