arXiv subjects

Franca Hoffmann

Publications and source records attributed to Franca Hoffmann.

34 records · Page 2Linked to original sources

arXiv2023

Strategic Distribution Shift of Interacting Agents via Coupled Gradient Flows

We propose a novel framework for analyzing the dynamics of distribution shift in real-world systems that captures the feedback loop between learning algorithms and the distributions on which they are deployed. Prior work largely models feedback-induced distribution shift as adversarial or via an overly simplistic distribution-shift structure. In contrast, we propose a coupled partial differential equation model that captures fine-grained changes in the distribution over time by accounting for complex dynamics that arise due to strategic responses to algorithmic decision-making, non-local endogenous population interactions, and other exogenous sources of distribution shift. We consider two common settings in machine learning: cooperative settings with information asymmetries, and competitive settings where a learner faces strategic users. For both of these settings, when the algorithm retrains via gradient descent, we prove asymptotic convergence of the retraining procedure to a steady-state, both in finite and in infinite dimensions, obtaining explicit rates in terms of the model parameters. To do so we derive new results on the convergence of coupled PDEs that extends what is known on multi-species systems. Empirically, we show that our approach captures well-documented forms of distribution shifts like polarization and disparate impacts that simpler models cannot capture.

cs.LG↗

arXiv2023

Bayesian Posterior Perturbation Analysis with Integral Probability Metrics

In recent years, Bayesian inference in large-scale inverse problems found in science, engineering and machine learning has gained significant attention. This paper examines the robustness of the Bayesian approach by analyzing the stability of posterior measures in relation to perturbations in the likelihood potential and the prior measure. We present new stability results using a family of integral probability metrics (divergences) akin to dual problems that arise in optimal transport. Our results stand out from previous works in three directions: (1) We construct new families of integral probability metrics that are adapted to the problem at hand; (2) These new metrics allow us to study both likelihood and prior perturbations in a convenient way; and (3) our analysis accommodates likelihood potentials that are only locally Lipschitz, making them applicable to a wide range of nonlinear inverse problems. Our theoretical findings are further reinforced through specific and novel examples where the approximation rates of posterior measures are obtained for different types of perturbations and provide a path towards the convergence analysis of recently adapted machine learning techniques for Bayesian inverse problems such as data-driven priors and neural network surrogates.

stat.ML↗

arXiv2022

Convergence rates for ansatz-free data-driven inference in physically constrained problems

We study a Data-Driven approach to inference in physical systems in a measure-theoretic framework. The systems under consideration are characterized by two measures defined over the phase space: i) A physical likelihood measure expressing the likelihood that a state of the system be admissible, in the sense of satisfying all governing physical laws; ii) A material likelihood measure expressing the likelihood that a local state of the material be observed in the laboratory. We assume deterministic loading, which means that the first measure is supported on a linear subspace. We additionally assume that the second measure is only known approximately through a sequence of empirical (discrete) measures. We develop a method for the quantitative analysis of convergence based on the flat metric and obtain error bounds both for annealing and the discretization or sampling procedure, leading to the determination of appropriate quantitative annealing rates. Finally, we provide an example illustrating the application of the theory to transportation networks.

math.OC↗

arXiv2021

Model-free Data-Driven Inference

We present a model-free data-driven inference method that enables inferences on system outcomes to be derived directly from empirical data without the need for intervening modeling of any type, be it modeling of a material law or modeling of a prior distribution of material states. We specifically consider physical systems with states characterized by points in a phase space determined by the governing field equations. We assume that the system is characterized by two likelihood measures: one $μ_D$ measuring the likelihood of observing a material state in phase space; and another $μ_E$ measuring the likelihood of states satisfying the field equations, possibly under random actuation. We introduce a notion of intersection between measures which can be interpreted to quantify the likelihood of system outcomes. We provide conditions under which the intersection can be characterized as the athermal limit $μ_\infty$ of entropic regularizations $μ_β$, or thermalizations, of the product measure $μ= μ_D\times μ_E$ as $β\to +\infty$. We also supply conditions under which $μ_\infty$ can be obtained as the athermal limit of carefully thermalized $(μ_{h,β_h})$ sequences of empirical data sets $(μ_h)$ approximating weakly an unknown likelihood function $μ$. In particular, we find that the cooling sequence $β_h \to +\infty$ must be slow enough, corresponding to quenching, in order for the proper limit $μ_\infty$ to be delivered. Finally, we derive explicit analytic expressions for expectations $\mathbb{E}[f]$ of outcomes $f$ that are explicit in the data, thus demonstrating the feasibility of the model-free data-driven paradigm as regards making convergent inferences directly from the data without recourse to intermediate modeling steps.

math.FA↗

arXiv2020

Nonlinear stability of chemotactic clustering with discontinuous advection

We perform the nonlinear stability analysis of a chemotaxis model of bacterial self-organization, assuming that bacteria respond sharply to chemical signals. The resulting discontinuous advection speed represents the key challenge for the stability analysis. We follow a perturbative approach, where the shape of the cellular profile is clearly separated from its global motion, allowing us to circumvent the discontinuity issue. Further, the homogeneity of the problem leads to two conservation laws, which express themselves in differently weighted functional spaces. This discrepancy between the weights represents another key methodological challenge. We derive an improved Poincaré inequality that allows to transfer the information encoded in the conservation laws to the appropriately weighted spaces. As a result, we obtain exponential relaxation to equilibrium with an explicit rate. A numerical investigation illustrates our results.

math.AP↗

arXiv2020

Spectral Analysis Of Weighted Laplacians Arising In Data Clustering

Graph Laplacians computed from weighted adjacency matrices are widely used to identify geometric structure in data, and clusters in particular; their spectral properties play a central role in a number of unsupervised and semi-supervised learning algorithms. When suitably scaled, graph Laplacians approach limiting continuum operators in the large data limit. Studying these limiting operators, therefore, sheds light on learning algorithms. This paper is devoted to the study of a parameterized family of divergence form elliptic operators that arise as the large data limit of graph Laplacians. The link between a three-parameter family of graph Laplacians and a three-parameter family of differential operators is explained. The spectral properties of these differential operators are analyzed in the situation where the data comprises two nearly separated clusters, in a sense which is made precise. In particular, we investigate how the spectral gap depends on the three parameters entering the graph Laplacian, and on a parameter measuring the size of the perturbation from the perfectly clustered case. Numerical results are presented which exemplify and extend the analysis: the computations study situations in which there are two nearly separated clusters, but which violate the assumptions used in our theory; situations in which more than two clusters are present, also going beyond our theory; and situations which demonstrate the relevance of our studies of differential operators for the understanding of finite data problems via the graph Laplacian. The findings provide insight into parameter choices made in learning algorithms which are based on weighted adjacency matrices; they also provide the basis for analysis of the consistency of various unsupervised and semi-supervised learning algorithms, in the large data limit.

math.SP↗

arXiv2020

Uniqueness of stationary states for singular Keller-Segel type models

We consider a generalised Keller-Segel model with non-linear porous medium type diffusion and non-local attractive power law interaction, focusing on potentials that are more singular than Newtonian interaction. We show uniqueness of stationary states (if they exist) in any dimension both in the diffusion-dominated regime and in the fair-competition regime when attraction and repulsion are in balance. As stationary states are radially symmetric decreasing, the question of uniqueness reduces to the radial setting. Our key result is a sharp generalised Hardy-Littlewood-Sobolev type functional inequality in the radial setting.

math.AP↗

arXiv2020

Consistency of semi-supervised learning algorithms on graphs: Probit and one-hot methods

Graph-based semi-supervised learning is the problem of propagating labels from a small number of labelled data points to a larger set of unlabelled data. This paper is concerned with the consistency of optimization-based techniques for such problems, in the limit where the labels have small noise and the underlying unlabelled data is well clustered. We study graph-based probit for binary classification, and a natural generalization of this method to multi-class classification using one-hot encoding. The resulting objective function to be optimized comprises the sum of a quadratic form defined through a rational function of the graph Laplacian, involving only the unlabelled data, and a fidelity term involving only the labelled data. The consistency analysis sheds light on the choice of the rational function defining the optimization.

stat.ML↗

arXiv2019

Interacting Langevin Diffusions: Gradient Structure And Ensemble Kalman Sampler

Solving inverse problems without the use of derivatives or adjoints of the forward model is highly desirable in many applications arising in science and engineering. In this paper, we propose a new version of such a methodology, a framework for its analysis, and numerical evidence of the practicality of the method proposed. Our starting point is an ensemble of over-damped Langevin diffusions which interact through a single preconditioner computed as the empirical ensemble covariance. We demonstrate that the nonlinear Fokker-Planck equation arising from the mean-field limit of the associated stochastic differential equation (SDE) has a novel gradient flow structure, built on the Wasserstein metric and the covariance matrix of the noisy flow. Using this structure, we investigate large time properties of the Fokker-Planck equation, showing that its invariant measure coincides with that of a single Langevin diffusion, and demonstrating exponential convergence to the invariant measure in a number of settings. We introduce a new noisy variant on ensemble Kalman inversion (EKI) algorithms found from the original SDE by replacing exact gradients with ensemble differences; this defines the ensemble Kalman sampler (EKS). Numerical results are presented which demonstrate its efficacy as a derivative-free approximate sampler for the Bayesian posterior arising from inverse problems.

math.DS↗

arXiv2019

Reverse Hardy-Littlewood-Sobolev inequalities

This paper is devoted to a new family of reverse Hardy-Littlewood-Sobolev inequalities which involve a power law kernel with positive exponent. We investigate the range of the admissible parameters and the properties of the optimal functions. A striking open question is the possibility of concentration which is analyzed and related with free energy functionals and nonlinear diffusion equations involving mean field drifts.

math.AP↗

arXiv2019

Geometric structure of graph Laplacian embeddings

We analyze the spectral clustering procedure for identifying coarse structure in a data set $x_1, \dots, x_n$, and in particular study the geometry of graph Laplacian embeddings which form the basis for spectral clustering algorithms. More precisely, we assume that the data is sampled from a mixture model supported on a manifold $\mathcal{M}$ embedded in $\mathbb{R}^d$, and pick a connectivity length-scale $\varepsilon>0$ to construct a kernelized graph Laplacian. We introduce a notion of a well-separated mixture model which only depends on the model itself, and prove that when the model is well separated, with high probability the embedded data set concentrates on cones that are centered around orthogonal vectors. Our results are meaningful in the regime where $\varepsilon = \varepsilon(n)$ is allowed to decay to zero at a slow enough rate as the number of data points grows. This rate depends on the intrinsic dimension of the manifold on which the data is supported.

math.SP↗

arXiv2018

Reverse Hardy-Littlewood-Sobolev inequalities

This paper is devoted to a new family of reverse Hardy-Littlewood-Sobolev inequalities which involve a power law kernel with positive exponent. We investigate the range of the admissible parameters and characterize the optimal functions. A striking open question is the possibility of concentration which is analyzed and related with nonlinear diffusion equations involving mean field drifts.

math.AP↗

arXiv2017

Ground States in the Diffusion-Dominated Regime

We consider macroscopic descriptions of particles where repulsion is modelled by non-linear power-law diffusion and attraction by a homogeneous singular kernel leading to variants of the Keller-Segel model of chemotaxis. We analyse the regime in which diffusive forces are stronger than attraction between particles, known as the diffusion-dominated regime, and show that all stationary states of the system are radially symmetric decreasing and compactly supported. The model can be formulated as a gradient flow of a free energy functional for which the overall convexity properties are not known. We show that global minimisers of the free energy always exist. Further, they are radially symmetric, compactly supported, uniformly bounded and $C^\infty$ inside their support. Global minimisers enjoy certain regularity properties if the diffusion is not too slow, and in this case, provide stationary states of the system. In one dimension, stationary states are characterised as optimisers of a functional inequality which establishes equivalence between global minimisers and stationary states, and allows to deduce uniqueness.

math.AP↗

arXiv2017

Exponential decay to equilibrium for a fibre lay-down process on a moving conveyor belt

We show existence and uniqueness of a stationary state for a kinetic Fokker-Planck equation modelling the fibre lay-down process in the production of non-woven textiles. Following a micro-macro decomposition, we use hypocoercivity techniques to show exponential convergence to equilibrium with an explicit rate assuming the conveyor belt moves slow enough. This work is an extension of (Dolbeault et al., 2013), where the authors consider the case of a stationary conveyor belt. Adding the movement of the belt, the global Gibbs state is not known explicitly. We thus derive a more general hypocoercivity estimate from which existence, uniqueness and exponential convergence can be derived. To treat the same class of potentials as in (Dolbeault et al., 2013), we make use of an additional weight function following the Lyapunov functional approach in (Kolb et al., 2013).

math.AP↗

arXiv2016

The geometry of diffusing and self-attracting particles in a one-dimensional fair-competition regime

We consider an aggregation-diffusion equation modelling particle interaction with non-linear diffusion and non-local attractive interaction using a homogeneous kernel (singular and non-singular) leading to variants of the Keller-Segel model of chemotaxis. We analyse the fair-competition regime in which both homogeneities scale the same with respect to dilations. Our analysis here deals with the one-dimensional case and provides an almost complete classification. In the singular kernel case and for critical interaction strength, we prove uniqueness of stationary states via a variant of the Hardy-Littlewood-Sobolev inequality. Using the same methods, we show uniqueness of self-similar profiles in the sub-critical case by proving a new type of functional inequality. Surprisingly, the same results hold true for any interaction strength in the non-singular kernel case. Further, we investigate the asymptotic behaviour of solutions, proving convergence to equilibrium in Wasserstein distance in the critical singular kernel case, and convergence to self-similarity for sub-critical interaction strength, both under a uniform stability condition. Moreover, solutions converge to a unique self-similar profile in the non-singular kernel case. Finally, we provide a numerical overview for the asymptotic behaviour of solutions in the full parameter space demonstrating the above results. We also discuss a number of phenomena appearing in the numerical explorations for the diffusion-dominated and attraction-dominated regimes.

math.AP↗

arXiv2016

Equilibria of homogeneous functionals in the fair-competition regime

We consider macroscopic descriptions of particles where repulsion is modelled by non-linear power-law diffusion and attraction by a homogeneous singular/smooth kernel leading to variants of the Keller-Segel model of chemotaxis. We analyse the regime in which both homogeneities scale the same with respect to dilations, that we coin as fair-competition. In the singular kernel case, we show that existence of global equilibria can only happen at a certain critical value and they are characterised as optimisers of a variant of HLS inequalities. We also study the existence of self-similar solutions for the sub-critical case, or equivalently of optimisers of rescaled free energies. These optimisers are shown to be compactly supported radially symmetric and non-increasing stationary solutions of the non-linear Keller-Segel equation. On the other hand, we show that no radially symmetric non-increasing stationary solutions exist in the smooth kernel case, implying that there is no criticality. However, we show the existence of positive self-similar solutions for all values of the parameter under the condition that diffusion is not too fast. We finally illustrate some of the open problems in the smooth kernel case by numerical experiments.

math.AP↗