Searcharxiv⌕ Search

arXiv subjects

Gilad Lerman

Publications and source records attributed to Gilad Lerman.

At least 19 recordsLinked to original sources

Recovery Theory for Projected Power Iterations in Permutation Synchronization

We study the projected power method for permutation synchronization: recovering $n$ unknown permutations of $m$ objects from pairwise relative permutations, observed independently with probability $p$ and independently corrupted by uniform permutations with probability $1-π_0$. Our main theorem establishes uniform one-step contraction for all estimates with optimally aligned block-error fraction at most $1/2-ε$, for fixed $ε>0$, including measurement-dependent estimates. When $np\ge C_0\log(n)$ and $m=o(npπ_0^2)$, one update contracts the error by a factor $O(m/(npπ_0^2))$, up to a floor of $O(e^{-c\min\{m\,npπ_0^2,\,npπ_0\}} + \log(n) / n)$. Thus, every initialization in this majority basin achieves $o(1)$ error after one update and maintains it uniformly over subsequent iterations. A reference-block spectral initializer enters the basin with probability $1-o(1)$, yielding end-to-end almost-exact recovery. If additionally $\min\{m\,npπ_0^2,\,npπ_0\} \ge C\log(n)$ for sufficiently large $C$, the method achieves exact recovery in finitely many iterations. We also establish prescribed-block recovery under weaker conditions for measurement-independent estimates and extend the guarantees to independent permutation-valued corruptions with uniform mean and to partial permutations with common visible support.

stat.ML↗

Global Convergence of Iteratively Reweighted Least Squares for Robust Subspace Recovery

Robust subspace estimation is fundamental to many machine learning and data analysis tasks. Iteratively Reweighted Least Squares (IRLS) is an elegant and empirically effective approach to this problem, yet its theoretical properties remain poorly understood. This paper establishes that, under deterministic conditions, a variant of IRLS with dynamic smoothing regularization converges linearly to the underlying subspace from any initialization. We extend these guarantees to affine subspace estimation, a setting that lacks prior recovery theory. Additionally, we illustrate the practical benefits of IRLS through an application to low-dimensional neural network training. Our results provide the first global convergence guarantees for IRLS in robust subspace recovery and, more broadly, for nonconvex IRLS on a Riemannian manifold.

stat.ML↗

Theoretical Guarantees for the Subspace-Constrained Tyler's Estimator

We study the subspace-constrained Tyler's estimator (STE) for recovering a low-dimensional linear subspace from data contaminated by outliers. Under deterministic conditions on inlier spread and outlier alignment, we prove linear convergence to the true subspace from a sufficiently accurate initialization. The guarantee also applies when the dimension-scaled signal-to-noise ratio is below one, where general robust subspace recovery is SSE-hard. When initializing STE from a candidate subspace, we provide a sufficient recovery condition based on the initial angular error and the size of the identity shift applied to the subspace projector. For noisy inliers, we prove a subspace error proportional to the noise level within an explicit iteration bound. Finally, under an asymptotic, anisotropic Gaussian inlier--outlier model, we show that Tyler's M-estimator (TME) provides a successful STE initialization even when TME's own leading eigenspace differs from the target subspace.

math.ST↗

Learning to Charge More: How Q-Learning Firms Lock In Supracompetitive Prices

There is growing experimental evidence that $Q$-learning agents can learn to charge supracompetitive prices. We ask when such behavior persists, whether it emerges, and which strategies support it. In a repeated pricing game in which firms condition on the previous period's price profile, we give conditions on the learned action values under which supracompetitive prices become an absorbing state once experimentation ends. We then show that under softmax exploration these conditions are reached with positive and explicitly bounded probability, both from a random and from an all-zero initial $Q$-table. In the latter case firms lock in supracompetitive prices because they never learn that undercutting pays; the same mechanism can lock in the competitive price, so the long-run outcome is selected by the exploration path. Finally, we give sufficient conditions on the learned table under which the induced strategy is naive collusion, a grim trigger strategy or an increasing-price strategy; grim trigger strategies are subgame perfect under a standard discounting condition, whereas naive collusion is generally not an equilibrium.

econ.GN↗

When Windowed Anderson Acceleration Converges Faster

Windowed Anderson acceleration AA($m$) combines at most $m+1$ recent iterates by least squares. For a real affine contraction $x\mapsto Wx+b$ with $q=\|W\| \in(0,1)$, we determine exactly when AA($m$) converges from every starting point with an asymptotic factor below $q$. We give a polynomial criterion for this improvement and an equivalent spectral condition: there is no probability distribution on the phases of the eigenvalues of modulus $q$ whose first $m$ cosine moments are $q,q^2,\ldots,q^m$. If such a distribution exists, we construct a starting point from which AA($m$) reproduces fixed-point iteration at every step. Otherwise, over a fixed number of consecutive steps, every run contracts the residual norm by a common factor strictly smaller than the corresponding power of $q$. The polynomial criterion also gives explicit convergence-rate bounds with constants independent of the dimension. When $q$ equals the spectral radius, this characterizes improvement over the asymptotic convergence factor of fixed-point iteration. For general affine contractions, the asymptotic factor never exceeds the numerical radius of $W$, and a related moment condition gives a uniform upper bound strictly below that radius.

math.NA↗

Specific Algorithmic Interpretability of Neural Networks: A Case Study on Textures

We develop a principled framework for constructing neural networks whose specific parameter realizations admit an explicit algorithmic interpretation. Existing algorithm-inspired architectures can explain the computational structure of a network, yet after standard training the learned parameters need not retain a clear relation to the motivating algorithm. We address this gap as follows. First, we model each data point as a sample of a class-dependent stochastic process and assume that statistics of this process can be estimated from a single sample and these statistics are sufficient to distinguish the classes. We then construct a neural network whose initial parameters exactly implement an algorithm for estimating these statistics, making the network fully interpretable. To account for mismatch between the idealized model and real data, we fine-tune this network while controlling its deviation from the algorithmic initialization. The trained network hence roughly retains the interpretation of the initial network. A PAC-Bayesian analysis yields a uniform generalization bound whose complexity term scales with the fine-tuning radius, providing a statistical motivation for our approach. We instantiate the framework for texture classification using the scattering transform to estimate the discriminative statistics.

cs.LG↗

Understanding Latent Diffusability via Fisher Geometry

Diffusion models often degrade in latent spaces, yet the formal causes remain poorly understood. We quantify latent-space diffusability via the rate of change of the Minimum Mean Squared Error (MMSE) along the diffusion trajectory. Our framework decomposes this MMSE rate into contributions from Fisher Information (FI) and Fisher Information Rate (FIR). We show that isometric embeddings preserve intrinsic FI and establish quantitative intrinsic-FI bounds for a broader class of bi-Lipschitz encoders with controlled weak volume distortion, whereas FIR is governed by the interplay between encoder and data geometries. Our analysis separates four geometric contributions in local stability bounds for Gaussian-smoothed FIR: dimensional compression, tangential distortion, high-frequency encoder curvature, and curvature of data manifold. Experiments across diverse autoencoding architectures provide qualitative support for the geometric mechanisms identified by the theory and show that empirical FI and FIR track several measures of generation quality and latent-space geometry in the settings tested. We establish FI and FIR as a comprehensive analytical framework for understanding latent diffusability.

cs.LG↗

Feature Learning for the High Dimensional Stationary Schödinger Equation with Deep Ritz Method

This paper investigates feature learning within the framework of the deep Ritz method for solving the stationary Schrödinger equation with Neumann boundary conditions. We first analyze the convergence of Riemannian gradient descent in an agnostic setting, where the hypothesis function is restricted to a single-index model while the PDE solution is arbitrary. We prove that gradient descent reaches an approximate global minimum: after T = O(log(1/ε)) iterations, the loss is within εof a constant multiple of the optimal loss. We then examine the loss landscape when the source term of the PDE itself follows a single-index model, considering hypothesis functions given by either a single-index model or a two-neuron multi-index model. In the single-index case, we show that the minimum Ritz energy is attained at the feature vector aligned with that of the source term. In the two-neuron case, we study the landscape of regularized Ritz losses and characterize how a second feature emerges, given that the first feature is aligned with the source, as the regularization parameter varies. Finally, numerical experiments are presented to validate the feature emergence theory in the two-neuron setting.

math.OC↗

Geometry-Preserving Encoder/Decoder in Latent Generative Models

Generative modeling aims to generate new data samples that resemble a given dataset. When using diffusion models for this task, one of the main challenges is solving the problem in the input space, which tends to be very high-dimensional. To address this, recent approaches solve diffusion models in the latent space through an encoder that maps from the data space to a lower-dimensional latent space, improving training efficiency and achieving state-of-the-art results. The variational autoencoder (VAE) is the most commonly used encoder/decoder framework in this domain, known for its ability to learn latent representations and generate data samples. In this paper, we introduce a novel encoder/decoder framework with theoretical properties distinct from those of the VAE, specifically designed to preserve the geometric structure of the data distribution. We demonstrate the significant advantages of this geometry-preserving encoder in the training process of both the encoder and decoder. Additionally, we provide theoretical results proving convergence of the training process, including convergence guarantees for encoder training, and results showing faster convergence of decoder training when using the geometry-preserving encoder.

math.NA↗

The Sharp Phase Transition of Tyler's M-Estimator for Robust Subspace Recovery

Robust Subspace Recovery (RSR) aims to identify an underlying d-dimensional subspace from a dataset heavily corrupted by outliers. Complexity-theoretic results establish a threshold for the problem's computational hardness based on the dimension-scaled signal-to-noise ratio (DS-SNR): the problem is SSE-hard when the DS-SNR is strictly less than 1, and solvable via practical algorithms when it is greater than 1 under general position assumptions. However, the exact behavior of practical algorithms at the critical boundary DS-SNR = 1 has remained unknown. This work resolves the behavior of Tyler's M-estimator (TME) at this critical boundary, consequently establishing a sharp phase transition. Specifically, we prove that TME converges exactly to the true subspace for DS-SNR \geq 1 under a new stability condition, which is less restrictive than the general position assumptions used in prior literature. Our analysis utilizes a decomposition of the TME iterates within a majorization-minimization framework.

cs.IT↗

QuadSync: Quadrifocal Tensor Synchronization via Tucker Decomposition

In structure from motion, quadrifocal tensors capture more information than their pairwise counterparts (essential matrices), yet they have often been thought of as impractical and only of theoretical interest. In this work, we challenge such beliefs by providing a new framework to recover $n$ cameras from the corresponding collection of quadrifocal tensors. We form the block quadrifocal tensor and show that it admits a Tucker decomposition whose factor matrices are the stacked camera matrices, and which thus has a multilinear rank of (4,~4,~4,~4) independent of $n$. We develop the first synchronization algorithm for quadrifocal tensors, using Tucker decomposition, alternating direction method of multipliers, and iteratively reweighted least squares. We further establish relationships between the block quadrifocal, trifocal, and bifocal tensors, and introduce an algorithm that jointly synchronizes these three entities. Numerical experiments demonstrate the effectiveness of our methods on modern datasets, indicating the potential and importance of using higher-order information in synchronization.

cs.CV↗

Stein Discrepancy for Unsupervised Domain Adaptation

Unsupervised domain adaptation (UDA) aims to improve model performance on an unlabeled target domain using a related, labeled source domain. A common approach aligns source and target feature distributions by minimizing a distance between them, often using symmetric measures such as maximum mean discrepancy (MMD). However, these methods struggle when target data is scarce. We propose a novel UDA framework that leverages Stein discrepancy, an asymmetric measure that depends on the target distribution only through its score function, making it particularly suitable for low-data target regimes. Our proposed method has kernelized and adversarial forms and supports flexible modeling of the target distribution via Gaussian, GMM, or VAE models. We derive a generalization bound on the target error and a convergence rate for the empirical Stein discrepancy in the two-sample setting. Empirically, our method consistently outperforms prior UDA approaches under limited target data across multiple benchmarks.

cs.LG↗

Cycle-Sync: Robust Global Camera Pose Estimation through Enhanced Cycle-Consistent Synchronization

We introduce Cycle-Sync, a robust and global framework for estimating camera poses (both rotations and locations). Our core innovation is a location solver that adapts message-passing least squares (MPLS) -- originally developed for group synchronization -- to camera location estimation. We modify MPLS to emphasize cycle-consistent information, redefine cycle consistencies using estimated distances from previous iterations, and incorporate a Welsch-type robust loss. We establish the strongest known deterministic exact-recovery guarantee for camera location estimation, showing that cycle consistency alone -- without access to inter-camera distances -- suffices to achieve the lowest sample complexity currently known. To further enhance robustness, we introduce a plug-and-play outlier rejection module inspired by robust subspace recovery, and we fully integrate cycle consistency into MPLS for rotation synchronization. Our global approach avoids the need for bundle adjustment. Experiments on synthetic and real datasets show that Cycle-Sync consistently outperforms leading pose estimators, including full structure-from-motion pipelines with bundle adjustment.

cs.CV↗

Improved Convergence Factor of Windowed Anderson Acceleration for Symmetric Fixed-Point Iterations

This paper studies the commonly utilized windowed Anderson acceleration (AA) algorithm for fixed-point methods, $x^{(k+1)}=q(x^{(k)})$. It provides the first proof that when the operator $q$ is linear and symmetric the windowed AA, which uses a sliding window of prior iterates, improves the root-linear convergence factor over the fixed-point iterations. When $q$ is nonlinear, yet has a symmetric Jacobian at a fixed point, a slightly modified AA algorithm is proved to have an analogous root-linear convergence factor improvement over fixed-point iterations. Simulations verify our observations. Furthermore, experiments with different data models demonstrate AA is significantly superior to the standard fixed-point methods for Tyler's M-estimation.

math.NA↗

Competition and Collusion in Two-Sided Markets with an Outside Option

We introduce pricing formulas for competition and collusion models of two-sided markets with an outside option. For the competition model, we find conditions under which prices and consumer surplus may increase or decrease if the outside option utility increases. Therefore, neglecting the outside option can lead to either overestimation or underestimation of these equilibrium outputs. Comparing collusion to competition, we find that in cases of small cross-side externalities, collusion results in decreased normalized net deterministic utilities, reduced market participation and increased price, on both sides of the market. Additionally, we observe that as the number of platforms increases in the competition model, market participation rises. Profits, however, decrease when the net normalized deterministic utility is sufficiently low but increase when it is high. Furthermore, we identify specific conditions that quantify the change of price and consumer surplus when the competition increases.

econ.GN↗

Tensor-Based Synchronization and the Low-Rankness of the Block Trifocal Tensor

The block tensor of trifocal tensors provides crucial geometric information on the three-view geometry of a scene. The underlying synchronization problem seeks to recover camera poses (locations and orientations up to a global transformation) from the block trifocal tensor. We establish an explicit Tucker factorization of this tensor, revealing a low multilinear rank of $(6,4,4)$ independent of the number of cameras under appropriate scaling conditions. We prove that this rank constraint provides sufficient information for camera recovery in the noiseless case. The constraint motivates a synchronization algorithm based on the higher-order singular value decomposition of the block trifocal tensor. Experimental comparisons with state-of-the-art global synchronization methods on real datasets demonstrate the potential of this algorithm for significantly improving location estimation accuracy. Overall this work suggests that higher-order interactions in synchronization problems can be exploited to improve performance, beyond the usual pairwise-based approaches.

cs.CV↗

General Constrained Matrix Optimization

This paper presents and analyzes the first matrix optimization model which allows general coordinate and spectral constraints. The breadth of problems our model covers is exemplified by a lengthy list of examples from the literature, including semidefinite programming, matrix completion, and quadratically constrained quadratic programs (QCQPs), and we demonstrate our model enables completely novel formulations of numerous problems. Our solution methodology leverages matrix factorization and constrained manifold optimization to develop an equivalent reformulation of our general matrix optimization model for which we design a feasible, first-order algorithm. We prove our algorithm converges to $(ε,ε)$-approximate first-order KKT points of our reformulation in $\mathcal{O}(1/ε^2)$ iterations. The method we developed applies to a special class of constrained manifold optimization problems and is one of the first which generates a sequence of feasible points which converges to a KKT point. We validate our model and method through numerical experimentation. Our first experiment presents a generalized version of semidefinite programming which allows novel eigenvalue constraints, and our second numerical experiment compares our method to the classical semidefinite relaxation approach for solving QCQPs. For the QCQP numerical experiments, we demonstrate our method is able to dominate the classical state-of-the-art approach, solving more than ten times as many problems compared to the standard solution procedure.

math.OC↗

Improving Hyperbolic Representations via Gromov-Wasserstein Regularization

Hyperbolic representations have shown remarkable efficacy in modeling inherent hierarchies and complexities within data structures. Hyperbolic neural networks have been commonly applied for learning such representations from data, but they often fall short in preserving the geometric structures of the original feature spaces. In response to this challenge, our work applies the Gromov-Wasserstein (GW) distance as a novel regularization mechanism within hyperbolic neural networks. The GW distance quantifies how well the original data structure is maintained after embedding the data in a hyperbolic space. Specifically, we explicitly treat the layers of the hyperbolic neural networks as a transport map and calculate the GW distance accordingly. We validate that the GW distance computed based on a training set well approximates the GW distance of the underlying data distribution. Our approach demonstrates consistent enhancements over current state-of-the-art methods across various tasks, including few-shot image classification, as well as semi-supervised graph link prediction and node classification.

cs.LG↗