arXiv subjects

Valentin Leplat

Publications and source records attributed to Valentin Leplat.

28 records · Page 2Linked to original sources

arXiv2024

An Alternating Minimization Algorithm with Trajectory for Direct Exoplanet Detection -- The AMAT Algorithm

Effective image post-processing algorithms are vital for the successful direct imaging of exoplanets. Standard PSF subtraction methods use techniques based on a low-rank approximation to separate the rotating planet signal from the quasi-static speckles, and rely on signal-to-noise ratio maps to detect the planet. These steps do not interact or feed each other, leading to potential limitations in the accuracy and efficiency of exoplanet detection. We aim to develop a novel approach that iteratively finds the flux of the planet and the low-rank approximation of quasi-static signals, in an attempt to improve upon current PSF subtraction techniques. In this study, we extend the standard L2 norm minimization paradigm to an L1 norm minimization framework to better account for noise statistics in the high contrast images. Then, we propose a new method, referred to as Alternating Minimization Algorithm with Trajectory, that makes a more advanced use of estimating the low-rank approximation of the speckle field and the planet flux by alternating between them and utilizing both L1 and L2 norms. For the L1 norm minimization, we propose using L1 norm low-rank approximation, a low-rank approximation computed using an exact block-cyclic coordinate descent method, while we use randomized singular value decomposition for the L2 norm minimization. Additionally, we enhance the visibility of the planet signal using a likelihood ratio as a postprocessing step. Numerical experiments performed on a VLT/SPHERE-IRDIS dataset show the potential of AMAT to improve upon the existing approaches in terms of higher S/N, sensitivity limits, and ROC curves. Moreover, for a systematic comparison, we used datasets from the exoplanet data challenge to compare our algorithm to other algorithms in the challenge, and AMAT with likelihood ratio map performs better than most algorithms tested on the exoplanet data challenge.

astro-ph.IM↗

arXiv2024

Deep Nonnegative Matrix Factorization with Beta Divergences

Deep Nonnegative Matrix Factorization (deep NMF) has recently emerged as a valuable technique for extracting multiple layers of features across different scales. However, all existing deep NMF models and algorithms have primarily centered their evaluation on the least squares error, which may not be the most appropriate metric for assessing the quality of approximations on diverse datasets. For instance, when dealing with data types such as audio signals and documents, it is widely acknowledged that $β$-divergences offer a more suitable alternative. In this paper, we develop new models and algorithms for deep NMF using some $β$-divergences, with a focus on the Kullback-Leibler divergence. Subsequently, we apply these techniques to the extraction of facial features, the identification of topics within document collections, and the identification of materials within hyperspectral images.

cs.LG↗

arXiv2023

Direct Exoplanet Detection Using L1 Norm Low-Rank Approximation

We propose to use low-rank matrix approximation using the component-wise L1-norm for direct imaging of exoplanets. Exoplanet detection by direct imaging is a challenging task for three main reasons: (1) the host star is several orders of magnitude brighter than exoplanets, (2) the angular distance between exoplanets and star is usually very small, and (3) the images are affected by the noises called speckles that are very similar to the exoplanet signal both in shape and intensity. We first empirically examine the statistical noise assumptions of the L1 and L2 models, and then we evaluate the performance of the proposed L1 low-rank approximation (L1-LRA) algorithm based on visual comparisons and receiver operating characteristic (ROC) curves. We compare the results of the L1-LRA with the widely used truncated singular value decomposition (SVD) based on the L2 norm in two different annuli, one close to the star and one far away.

astro-ph.IM↗

arXiv2023

NAG-GS: Semi-Implicit, Accelerated and Robust Stochastic Optimizer

Classical machine learning models such as deep neural networks are usually trained by using Stochastic Gradient Descent-based (SGD) algorithms. The classical SGD can be interpreted as a discretization of the stochastic gradient flow. In this paper we propose a novel, robust and accelerated stochastic optimizer that relies on two key elements: (1) an accelerated Nesterov-like Stochastic Differential Equation (SDE) and (2) its semi-implicit Gauss-Seidel type discretization. The convergence and stability of the obtained method, referred to as NAG-GS, are first studied extensively in the case of the minimization of a quadratic function. This analysis allows us to come up with an optimal learning rate in terms of the convergence rate while ensuring the stability of NAG-GS. This is achieved by the careful analysis of the spectral radius of the iteration matrix and the covariance matrix at stationarity with respect to all hyperparameters of our method. Further, we show that NAG- GS is competitive with state-of-the-art methods such as momentum SGD with weight decay and AdamW for the training of machine learning models such as the logistic regression model, the residual networks models on standard computer vision datasets, Transformers in the frame of the GLUE benchmark and the recent Vision Transformers.

math.OC↗

arXiv2023

Conic-Optimization Based Algorithms for Nonnegative Matrix Factorization

Nonnegative matrix factorization is the following problem: given a nonnegative input matrix $V$ and a factorization rank $K$, compute two nonnegative matrices, $W$ with $K$ columns and $H$ with $K$ rows, such that $WH$ approximates $V$ as well as possible. In this paper, we propose two new approaches for computing high-quality NMF solutions using conic optimization. These approaches rely on the same two steps. First, we reformulate NMF as minimizing a concave function over a product of convex cones--one approach is based on the exponential cone, and the other on the second-order cone. Then, we solve these reformulations iteratively: at each step, we minimize exactly, over the feasible set, a majorization of the objective functions obtained via linearization at the current iterate. Hence these subproblems are convex conic programs and can be solved efficiently using dedicated algorithms. We prove that our approaches reach a stationary point with an accuracy decreasing as $\mathcal{O}(\frac{1}{i})$, where $i$ denotes the iteration number. To the best of our knowledge, our analysis is the first to provide a convergence rate to stationary points for NMF. Furthermore, in the particular cases of rank-one factorizations (that is, $K=1$), we show that one of our formulations can be expressed as a convex optimization problem implying that optimal rank-one approximations can be computed efficiently. Finally, we show on several numerical examples that our approaches are able to frequently compute exact NMFs (that is, with $V = WH$), and compete favorably with the state of the art.

math.OC↗

arXiv2022

Nonnegative Tucker Decomposition with Beta-divergence for Music Structure Analysis of Audio Signals

Nonnegative Tucker decomposition (NTD), a tensor decomposition model, has received increased interest in the recent years because of its ability to blindly extract meaningful patterns, in particular in Music Information Retrieval. Nevertheless, existing algorithms to compute NTD are mostly designed for the Euclidean loss. This work proposes a multiplicative updates algorithm to compute NTD with the beta-divergence loss, often considered a better loss for audio processing. We notably show how to implement efficiently the multiplicative rules using tensor algebra. Finally, we show on a music structure analysis task that unsupervised NTD fitted with beta-divergence loss outperforms earlier results obtained with the Euclidean loss.

cs.SD↗

arXiv2022

Multiplicative Updates for NMF with $β$-Divergences under Disjoint Equality Constraints

Nonnegative matrix factorization (NMF) is the problem of approximating an input nonnegative matrix, $V$, as the product of two smaller nonnegative matrices, $W$ and $H$. In this paper, we introduce a general framework to design multiplicative updates (MU) for NMF based on $β$-divergences ($β$-NMF) with disjoint equality constraints, and with penalty terms in the objective function. By disjoint, we mean that each variable appears in at most one equality constraint. Our MU satisfy the set of constraints after each update of the variables during the optimization process, while guaranteeing that the objective function decreases monotonically. We showcase this framework on three NMF models, and show that it competes favorably the state of the art: (1)~$β$-NMF with sum-to-one constraints on the columns of $H$, (2) minimum-volume $β$-NMF with sum-to-one constraints on the columns of $W$, and (3) sparse $β$-NMF with $\ell_2$-norm constraints on the columns of $W$.

cs.LG↗

arXiv2021

Multi-Resolution Beta-Divergence NMF for Blind Spectral Unmixing

Many datasets are obtained as a resolution trade-off between two adversarial dimensions; for example between the frequency and the temporal resolutions for the spectrogram of an audio signal, and between the number of wavelengths and the spatial resolution for a hyper/multi-spectral image. To perform blind source separation using observations with different resolutions, a standard approach is to use coupled nonnegative matrix factorizations (NMF). Most previous works have focused on the least squares error measure, which is the $β$-divergence for $β= 2$. In this paper, we formulate this multi-resolution NMF problem for any $β$-divergence, and propose an algorithm based on multiplicative updates (MU). We show on numerical experiments that the MU are able to obtain high resolutions in both dimensions on two applications: (1) blind unmixing of audio spectrograms: to the best of our knowledge, this is the first time a coupled NMF model is used in this context, and (2) the fusion of hyperspectral and multispectral images: we show that the MU compete favorable with state-of-the-art algorithms in particular in the presence of non-Gaussian noise.

eess.SP↗

arXiv2021

Distributionally Robust and Multi-Objective Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is a linear dimensionality reduction technique for analyzing nonnegative data. A key aspect of NMF is the choice of the objective function that depends on the noise model (or statistics of the noise) assumed on the data. In many applications, the noise model is unknown and difficult to estimate. In this paper, we define a multi-objective NMF (MO-NMF) problem, where several objectives are combined within the same NMF model. We propose to use Lagrange duality to judiciously optimize for a set of weights to be used within the framework of the weighted-sum approach, that is, we minimize a single objective function which is a weighted sum of the all objective functions. We design a simple algorithm based on multiplicative updates to minimize this weighted sum. We show how this can be used to find distributionally robust NMF (DR-NMF) solutions, that is, solutions that minimize the largest error among all objectives, using a dual approach solved via a heuristic inspired from the Frank-Wolfe algorithm. We illustrate the effectiveness of this approach on synthetic, document and audio data sets. The results show that DR-NMF is robust to our incognizance of the noise model of the NMF problem.

cs.LG↗

arXiv2020

Blind Audio Source Separation with Minimum-Volume Beta-Divergence NMF

Considering a mixed signal composed of various audio sources and recorded with a single microphone, we consider on this paper the blind audio source separation problem which consists in isolating and extracting each of the sources. To perform this task, nonnegative matrix factorization (NMF) based on the Kullback-Leibler and Itakura-Saito $β$-divergences is a standard and state-of-the-art technique that uses the time-frequency representation of the signal. We present a new NMF model better suited for this task. It is based on the minimization of $β$-divergences along with a penalty term that promotes the columns of the dictionary matrix to have a small volume. Under some mild assumptions and in noiseless conditions, we prove that this model is provably able to identify the sources. In order to solve this problem, we propose multiplicative updates whose derivations are based on the standard majorization-minimization framework. We show on several numerical experiments that our new model is able to obtain more interpretable results than standard NMF models. Moreover, we show that it is able to recover the sources even when the number of sources present into the mixed signal is overestimated. In fact, our model automatically sets sources to zero in this situation, hence performs model order selection automatically.

eess.SP↗