arXiv subjects
Publications and source records attributed to Jean-François Cardoso.
I recall my first encounter with Professor Shun-ichi Amari who, once upon a time in Las Vegas, gave me a precious hint about connecting Independent Component Analysis (ICA) to Information Geometry. The paper sketches, rather informally, some of the insights gained in following this lead.
Background: Independent Component Analysis (ICA) is a widespread tool for exploration and denoising of electroencephalography (EEG) or magnetoencephalography (MEG) signals. In its most common formulation, ICA assumes that the signal matrix is a noiseless linear mixture of independent sources that are assumed non-Gaussian. A limitation is that it enforces to estimate as many sources as sensors or to rely on a detrimental PCA step. Methods: We present the Spectral Matching ICA (SMICA) model. Signals are modelled as a linear mixing of independent sources corrupted by additive noise, where sources and the noise are stationary Gaussian time series. Thanks to the Gaussian assumption, the negative log-likelihood has a simple expression as a sum of divergences between the empirical spectral covariance matrices of the signals and those predicted by the model. The model parameters can then be estimated by the expectation-maximization (EM) algorithm. Results: Experiments on phantom MEG datasets show that SMICA can recover dipole locations more precisely than usual ICA algorithms or Maxwell filtering when the dipole amplitude is low. Experiments on EEG datasets show that SMICA identifies a source subspace which contains sources that have less pairwise mutual information, and are better explained by the projection of a single dipole on the scalp. Comparison with existing methods: Noiseless ICA models lead to degenerate likelihood when there are fewer sources than sensors, while SMICA succeeds without resorting to prior dimension reduction. Conclusions: SMICA is a promising alternative to other noiseless ICA models based on non-Gaussian assumptions.
The merging rate of cosmic structures is computed, relying on the Ansatz that they can be predicted in the initial linear density field from the coalescence of critical points with increasing smoothing scale, used here as a proxy for cosmic time. Beyond the mergers of peaks with saddle points (a proxy for halo mergers), we consider the coalescence and nucleation of all sets of critical points, including wall-saddle to filament-saddle and wall-saddle to minima (a proxy for filament and void mergers respectively), as they impact the geometry of galactic infall, and in particular filament disconnection. Analytical predictions of the one-point statistics are validated against multiscale measurements in 2D and 3D realisations of Gaussian random fields (the corresponding code being available upon request) and compared qualitatively to cosmological $N$-body simulations at early times ($z\geq 10$) and large scales ($\geq 5\, \mathrm{Mpc}/h$). The rate of filament coalescence is compared to the merger rate of haloes and the two-point clustering of these events is computed, along with their cross-correlations with critical points. These correlations are qualitatively consistent with the preservation of the connectivity of dark matter haloes, and the impact of the large scale structures on assembly bias. The destruction rate of haloes and voids as a function of mass and redshift is quantified down to $z=0$ for a $Λ$CDM cosmology. The one-point statistics in higher dimensions are also presented, together with consistency relations between critical point and critical event counts.
Independent component analysis (ICA) is a widespread data exploration technique, where observed signals are modeled as linear mixtures of independent components. From a machine learning point of view, it amounts to a matrix factorization problem with a statistical independence criterion. Infomax is one of the most used ICA algorithms. It is based on a loss function which is a non-convex log-likelihood. We develop a new majorization-minimization framework adapted to this loss function. We derive an online algorithm for the streaming setting, and an incremental algorithm for the finite sum setting, with the following benefits. First, unlike most algorithms found in the literature, the proposed methods do not rely on any critical hyper-parameter like a step size, nor do they require a line-search technique. Second, the algorithm for the finite sum setting, although stochastic, guarantees a decrease of the loss function at each iteration. Experiments demonstrate progress on the state-of-the-art for large scale datasets, without the necessity for any manual parameter tuning.
The approximate joint diagonalization of a set of matrices consists in finding a basis in which these matrices are as diagonal as possible. This problem naturally appears in several statistical learning tasks such as blind signal separation. We consider the diagonalization criterion studied in a seminal paper by Pham (2001), and propose a new quasi-Newton method for its optimization. Through numerical experiments on simulated and real datasets, we show that the proposed method outper-forms Pham's algorithm. An open source Python package is released.
We study optimization methods for solving the maximum likelihood formulation of independent component analysis (ICA). We consider both the the problem constrained to white signals and the unconstrained problem. The Hessian of the objective function is costly to compute, which renders Newton's method impractical for large data sets. Many algorithms proposed in the literature can be rewritten as quasi-Newton methods, for which the Hessian approximation is cheap to compute. These algorithms are very fast on simulated data where the linear mixture assumption really holds. However, on real signals, we observe that their rate of convergence can be severely impaired. In this paper, we investigate the origins of this behavior, and show that the recently proposed Preconditioned ICA for Real Data (Picard) algorithm overcomes this issue on both constrained and unconstrained problems.
Independent Component Analysis (ICA) is a technique for unsupervised exploration of multi-channel data widely used in observational sciences. In its classical form, ICA relies on modeling the data as a linear mixture of non-Gaussian independent sources. The problem can be seen as a likelihood maximization problem. We introduce Picard-O, a preconditioned L-BFGS strategy over the set of orthogonal matrices, which can quickly separate both super- and sub-Gaussian signals. It returns the same set of sources as the widely used FastICA algorithm. Through numerical experiments, we show that our method is faster and more robust than FastICA on real data.
Independent Component Analysis (ICA) is a technique for unsupervised exploration of multi-channel data that is widely used in observational sciences. In its classic form, ICA relies on modeling the data as linear mixtures of non-Gaussian independent sources. The maximization of the corresponding likelihood is a challenging problem if it has to be completed quickly and accurately on large sets of real data. We introduce the Preconditioned ICA for Real Data (Picard) algorithm, which is a relative L-BFGS algorithm preconditioned with sparse Hessian approximations. Extensive numerical comparisons to several algorithms of the same class demonstrate the superior performance of the proposed technique, especially on real data, for which the ICA model does not necessarily hold.
We present a new blind formulation of the Cosmic Microwave Background (CMB) inference problem. The approach relies on a phenomenological model of the multi-frequency microwave sky without the need for physical models of the individual components. For all-sky and high resolution data, it unifies parts of the analysis that have previously been treated separately, such as component separation and power spectrum inference. We describe an efficient sampling scheme that fully explores the component separation uncertainties on the inferred CMB products such as maps and/or power spectra. External information about individual components can be incorporated as a prior giving a flexible way to progressively and continuously introduce physical component separation from a maximally blind approach. We connect our Bayesian formalism to existing approaches such as Commander, SMICA and ILC, and discuss possible future extensions.
A large-scale hydrodynamical cosmological simulation, Horizon-AGN, is used to investigate the alignment between the spin of galaxies and the cosmic filaments above redshift 1.2. The analysis of more than 150 000 galaxies per time step in the redshift range 1.2<z<1.8 with morphological diversity shows that the spin of low-mass blue galaxies is preferentially aligned with their neighbouring filaments, while high-mass red galaxies tend to have a perpendicular spin. The reorientation of the spin of massive galaxies is provided by galaxy mergers, which are significant in their mass build-up. We find that the stellar mass transition from alignment to misalignment happens around 3.10^10 M_sun. Galaxies form in the vorticity-rich neighbourhood of filaments, and migrate towards the nodes of the cosmic web as they convert their orbital angular momentum into spin. The signature of this process can be traced to the properties of galaxies, as measured relative to the cosmic web. We argue that a strong source of feedback such as active galactic nuclei is mandatory to quench in situ star formation in massive galaxies and promote various morphologies. It allows mergers to play their key role by reducing post-merger gas inflows and, therefore, keeping spins misaligned with cosmic filaments.
We demonstrate that for a cosmic variance limited experiment, CMB E polarization alone places stronger constraints on cosmological parameters than CMB temperature. For example, we show that EE can constrain parameters better than TT by up to a factor 2.8 when a multipole range of l=30-2500 is considered. We expose the physical effects at play behind this remarkable result and study how it depends on the multipole range included in the analysis. In most relevant cases, TE or EE surpass the TT based cosmological constraints. This result is important as the small scale astrophysical foregrounds are expected to have a much reduced impact on polarization, thus opening the possibility of building cleaner and more stringent constraints of the LCDM model. This is relevant specially for proposed future CMB satellite missions, such as CORE or PRISM, that are designed to be cosmic variance limited in polarization till very large multipoles. We perform the same analysis for a Planck-like experiment, and conclude that even in this case TE alone should determine the constraint on $Ω_ch^2$ better than TT by 15%, while determining $Ω_bh^2$, $n_s$ and $θ$ with comparable accuracy. Finally, we explore a few classical extensions of the LCDM model and show again that CMB polarization alone provides more stringent constraints than CMB temperature in case of a cosmic variance limited experiment.
We consider the reconstruction of the CMB lensing potential and its power spectrum of the full sphere in presence of sky-cuts due to point sources and Galactic contaminations. Those two effects are treated separately. Small regions contaminated by point sources are filled in using Gaussian constrained realizations. The Galactic plane is simply masked using an apodized mask before lensing reconstruction. This algorithm recovers the power spectrum of the lensing potential with no significant bias.
We present a Bayesian sampling algorithm called adaptive importance sampling or Population Monte Carlo (PMC), whose computational workload is easily parallelizable and thus has the potential to considerably reduce the wall-clock time required for sampling, along with providing other benefits. To assess the performance of the approach for cosmological problems, we use simulated and actual data consisting of CMB anisotropies, supernovae of type Ia, and weak cosmological lensing, and provide a comparison of results to those obtained using state-of-the-art Markov Chain Monte Carlo (MCMC). For both types of data sets, we find comparable parameter estimates for PMC and MCMC, with the advantage of a significantly lower computational time for PMC. In the case of WMAP5 data, for example, the wall-clock time reduces from several days for MCMC to a few hours using PMC on a cluster of processors. Other benefits of the PMC approach, along with potential difficulties in using the approach, are analysed and discussed.
Abreg: We investigate the impact of polarized foreground emission on the performances of future CMB experiments in measuring the tensor-to-scalar ratio r. We design a component separation pipeline, based on the Smica method, aimed at estimating r and the foreground contamination from the data with no prior assumption on the frequency dependence or spatial distribution of the foregrounds. We derive error bars accounting for the uncertainty on foreground contribution. We use the current knowledge of galactic and extra-galactic foregrounds as implemented in the Planck Sky Model (PSM), to build simulations of the sky emission for various experimental setups. Our method, permits us to detect r = 0.1 at more than 3 sigma from B-modes only with Planck data, and r = 0.001 at 6 sigma for the most ambitious designs of the future EPIC probe. We find that all-sky experiments permit a proper measurement of the reionization bump despite the large scale foreground emission and are nearly insensitive to contamination from point sources and lensing if their statistical contribution can be modelled accurately. Investigating the observation of a small but clean part of the sky, we show that diffuse foregrounds remain a concern for a sensitive ground-based experiment with a limited frequency coverage when measuring r < 0.1, but are dealt with efficiently by a deep field space mission which is in return quite sensitive to lensing. Our results do not significantly depend on the overall level and frequency dependence of the diffused foreground model.
We address the question of designing isotropic analysis functions on the sphere which are perfectly limited in the spectral domain and optimally localized in the spatial domain. This work is motivated by the need of localized analysis tools in domains where the data is lying on the sphere, e.g.{} the science of the Cosmic Microwave Background. Our construction is derived from the localized frames introduced by Narcowich, Petrushev, Ward, 2006. The analysis frames are optimized for given applications and compared numerically using various criteria.