arXiv subjects

Mircea Petrache

Publications and source records attributed to Mircea Petrache.

At least 19 recordsLinked to original sources

arXiv2026

Recurrent Equivariant Constraint Modulation: Learning Per-Layer Symmetry Relaxation from Data

Equivariant neural networks exploit underlying task symmetries to improve generalization, but strict equivariance constraints can induce more complex optimization dynamics that can hinder learning. Prior work addresses these limitations by relaxing strict equivariance during training, but typically relies on prespecified, explicit, or implicit target levels of relaxation for each network layer, which are task-dependent and costly to tune. We propose Recurrent Equivariant Constraint Modulation (RECM), a layer-wise constraint modulation mechanism that learns appropriate relaxation levels solely from the training signal and the symmetry properties of each layer's input-target distribution, without requiring any prior knowledge about the task-dependent target relaxation level. We demonstrate that under the proposed RECM update, the relaxation level of each layer provably converges to a value upper-bounded by its symmetry gap, namely the degree to which its input-target distribution deviates from exact symmetry. Consequently, layers processing symmetric distributions recover full equivariance, while those with approximate symmetries retain sufficient flexibility to learn non-symmetric solutions when warranted by the data. Empirically, RECM outperforms prior methods across diverse exact and approximate equivariant tasks, including the challenging molecular conformer generation on the GEOM-Drugs dataset.

cs.LG↗

arXiv2025

On Universality of Deep Equivariant Networks

Universality results for equivariant neural networks remain rare. Those that do exist typically hold only in restrictive settings: either they rely on regular or higher-order tensor representations, leading to impractically high-dimensional hidden spaces, or they target specialized architectures, often confined to the invariant setting. This work develops a more general account. For invariant networks, we establish a universality theorem under separation constraints, showing that the addition of a fully connected readout layer secures approximation within the class of separation-constrained continuous functions. For equivariant networks, where results are even scarcer, we demonstrate that standard separability notions are inadequate and introduce the sharper criterion of $\textit{entry-wise separability}$. We show that with sufficient depth or with the addition of appropriate readout layers, equivariant networks attain universality within the entry-wise separable regime. Together with prior results showing the failure of universality for shallow models, our findings identify depth and readout layers as a decisive mechanism for universality, additionally offering a unified perspective that subsumes and extends earlier specialized results.

stat.ML↗

arXiv2025

A Compressive-Expressive Communication Framework for Compositional Representations

Compositionality in knowledge and language--the ability to represent complex concepts as a combination of simpler ones--is a hallmark of human cognition and communication. Despite recent advances, deep neural networks still struggle to acquire this property reliably. Neural models for emergent communication look to endow artificial agents with compositional language by simulating the pressures that form human language. In this work, we introduce CELEBI (Compressive-Expressive Language Emergence through a discrete Bottleneck and Iterated learning), a novel self-supervised framework for inducing compositional representations through a reconstruction-based communication game between a sender and a receiver. Building on theories of language emergence and the iterated learning framework, we integrate three mechanisms that jointly promote compressibility, expressivity, and efficiency in the emergent language. First, Progressive Decoding incentivizes intermediate reasoning by requiring the receiver to produce partial reconstructions after each symbol. Second, Final-State Imitation trains successive generations of agents to imitate reconstructions rather than messages, enforcing a tighter communication bottleneck. Third, Pairwise Distance Maximization regularizes message diversity by encouraging high distances between messages, with formal links to entropy maximization. Our method significantly improves both the efficiency and compositionality of the learned messages on the Shapes3D and MPI3D datasets, surpassing prior discrete communication frameworks in both reconstruction accuracy and topographic similarity. This work provides new theoretical and empirical evidence for the emergence of structured, generalizable communication protocols from simplicity-based inductive biases.

cs.LG↗

arXiv2024

Symmetry-Based Structured Matrices for Efficient Approximately Equivariant Networks

There has been much recent interest in designing neural networks (NNs) with relaxed equivariance, which interpolate between exact equivariance and full flexibility for consistent performance gains. In a separate line of work, structured parameter matrices with low displacement rank (LDR) -- which permit fast function and gradient evaluation -- have been used to create compact NNs, though primarily benefiting classical convolutional neural networks (CNNs). In this work, we propose a framework based on symmetry-based structured matrices to build approximately equivariant NNs with fewer parameters. Our approach unifies the aforementioned areas using Group Matrices (GMs), a forgotten precursor to the modern notion of regular representations of finite groups. GMs allow the design of structured matrices similar to LDR matrices, which can generalize all the elementary operations of a CNN from cyclic groups to arbitrary finite groups. We show GMs can also generalize classical LDR theory to general discrete groups, enabling a natural formalism for approximate equivariance. We test GM-based architectures on various tasks with relaxed symmetry and find that our framework performs competitively with approximately equivariant NNs and other structured matrix-based methods, often with one to two orders of magnitude fewer parameters.

stat.ML↗

arXiv2024

Clearing-out of dipoles for minimisers of 2-dimensional discrete energies with topological singularities

A key question in the analysis of discrete models for material defects, such as vortices in spin systems and superconductors or isolated dislocations in metals, is whether information on boundary energy for a domain can be sufficient for controlling the number of defects in the interior. We present a general combinatorial dipole-removal argument for a large class of discrete models including XY systems and screw dislocation models, allowing to prove sharp conditions under which controlled flux and boundary energy guarantee to have minimizers with zero or one charges in the interior. The argument uses the max-flow min-cut theorem in combination with an ad-hoc duality for planar graphs, and is robust with respect to changes of the function defining the interaction energies.

math.AP↗

arXiv2024

Fisher Flow Matching for Generative Modeling over Discrete Data

Generative modeling over discrete data has recently seen numerous success stories, with applications spanning language modeling, biological sequence design, and graph-structured molecular data. The predominant generative modeling paradigm for discrete data is still autoregressive, with more recent alternatives based on diffusion or flow-matching falling short of their impressive performance in continuous data settings, such as image or video generation. In this work, we introduce Fisher-Flow, a novel flow-matching model for discrete data. Fisher-Flow takes a manifestly geometric perspective by considering categorical distributions over discrete data as points residing on a statistical manifold equipped with its natural Riemannian metric: the $\textit{Fisher-Rao metric}$. As a result, we demonstrate discrete data itself can be continuously reparameterised to points on the positive orthant of the $d$-hypersphere $\mathbb{S}^d_+$, which allows us to define flows that map any source distribution to target in a principled manner by transporting mass along (closed-form) geodesics of $\mathbb{S}^d_+$. Furthermore, the learned flows in Fisher-Flow can be further bootstrapped by leveraging Riemannian optimal transport leading to improved training dynamics. We prove that the gradient flow induced by Fisher-Flow is optimal in reducing the forward KL divergence. We evaluate Fisher-Flow on an array of synthetic and diverse real-world benchmarks, including designing DNA Promoter, and DNA Enhancer sequences. Empirically, we find that Fisher-Flow improves over prior diffusion and flow-matching models on these benchmarks.

cs.LG↗

arXiv2024

Understanding the dynamics of the frequency bias in neural networks

Recent works have shown that traditional Neural Network (NN) architectures display a marked frequency bias in the learning process. Namely, the NN first learns the low-frequency features before learning the high-frequency ones. In this study, we rigorously develop a partial differential equation (PDE) that unravels the frequency dynamics of the error for a 2-layer NN in the Neural Tangent Kernel regime. Furthermore, using this insight, we explicitly demonstrate how an appropriate choice of distributions for the initialization weights can eliminate or control the frequency bias. We focus our study on the Fourier Features model, an NN where the first layer has sine and cosine activation functions, with frequencies sampled from a prescribed distribution. In this setup, we experimentally validate our theoretical results and compare the NN dynamics to the solution of the PDE using the finite element method. Finally, we empirically show that the same principle extends to multi-layer NNs.

cs.LG↗

arXiv2024

Long Tail Image Generation Through Feature Space Augmentation and Iterated Learning

Image and multimodal machine learning tasks are very challenging to solve in the case of poorly distributed data. In particular, data availability and privacy restrictions exacerbate these hurdles in the medical domain. The state of the art in image generation quality is held by Latent Diffusion models, making them prime candidates for tackling this problem. However, a few key issues still need to be solved, such as the difficulty in generating data from under-represented classes and a slow inference process. To mitigate these issues, we propose a new method for image augmentation in long-tailed data based on leveraging the rich latent space of pre-trained Stable Diffusion Models. We create a modified separable latent space to mix head and tail class examples. We build this space via Iterated Learning of underlying sparsified embeddings, which we apply to task-specific saliency maps via a K-NN approach. Code is available at https://github.com/SugarFreeManatee/Feature-Space-Augmentation-and-Iterated-Learning

cs.CV↗

arXiv2024

CConnect: Synergistic Convolutional Regularization for Cartesian T2* Mapping

Magnetic resonance imaging (MRI) is fundamental for the assessment of many diseases, due to its excellent tissue contrast characterization. This is based on quantitative techniques, such as T1 , T2 , and T2* mapping. Quantitative MRI requires the acquisition of several contrast-weighed images followed by a fitting to an exponential model or dictionary matching, which results in undesirably long acquisition times. Undersampling reconstruction techniques are commonly employed to speed up the scan, with the drawback of introducing aliasing artifacts. However, most undersampling reconstruction techniques require long computational times or do not exploit redundancies across the different contrast-weighted images. This study introduces a new regularization technique to overcome aliasing artifacts, namely CConnect, which uses an innovative regularization term that leverages several trained convolutional neural networks (CNNs) to connect and exploit information across image contrasts in a latent space. We validate our method using in-vivo T2* mapping of the brain, with retrospective undersampling factors of 4, 5 and 6, demonstrating its effectiveness in improving reconstruction in comparison to state-of-the-art techniques. Comparisons against joint total variation, nuclear low rank and a deep learning (DL) de-aliasing post-processing method, with respect to structural similarity index measure (SSIM) and peak signal-to-noise ratio (PSNR) metrics are presented.

physics.med-ph↗

arXiv2024

A Class of Topological Pseudodistances for Fast Comparison of Persistence Diagrams

Persistence diagrams (PD)s play a central role in topological data analysis, and are used in an ever increasing variety of applications. The comparison of PD data requires computing comparison metrics among large sets of PDs, with metrics which are accurate, theoretically sound, and fast to compute. Especially for denser multi-dimensional PDs, such comparison metrics are lacking. While on the one hand, Wasserstein-type distances have high accuracy and theoretical guarantees, they incur high computational cost. On the other hand, distances between vectorizations such as Persistence Statistics (PS)s have lower computational cost, but lack the accuracy guarantees and in general they are not guaranteed to distinguish PDs (i.e. the two PS vectors of different PDs may be equal). In this work we introduce a class of pseudodistances called Extended Topological Pseudodistances (ETD)s, which have tunable complexity, and can approximate Sliced and classical Wasserstein distances at the high-complexity extreme, while being computationally lighter and close to Persistence Statistics at the lower complexity extreme, and thus allow users to interpolate between the two metrics. We build theoretical comparisons to show how to fit our new distances at an intermediate level between persistence vectorizations and Wasserstein distances. We also experimentally verify that ETDs outperform PSs in terms of accuracy and outperform Wasserstein and Sliced Wasserstein distances in terms of computational complexity.

cs.CG↗

arXiv2024

Position Paper: Generalized grammar rules and structure-based generalization beyond classical equivariance for lexical tasks and transduction

Compositional generalization is one of the main properties which differentiates lexical learning in humans from state-of-art neural networks. We propose a general framework for building models that can generalize compositionally using the concept of Generalized Grammar Rules (GGRs), a class of symmetry-based compositional constraints for transduction tasks, which we view as a transduction analogue of equivariance constraints in physics-inspired tasks. Besides formalizing generalized notions of symmetry for language transduction, our framework is general enough to contain many existing works as special cases. We present ideas on how GGRs might be implemented, and in the process draw connections to reinforcement learning and other areas of research.

cs.CL↗

arXiv2024

Existence and uniqueness of Monge minimizers for a Multi-marginal Optimal Transport problem with intermolecular interactions cost

We investigate a new multi-marginal optimal transport problem arising from a dissociation model in the Strong Interaction Limit of Density Functional Theory. In this short note, we introduce such dissociation model, the corresponding optimal transport problem as well as show preliminary results on the existence and uniqueness of Monge solutions assuming absolute continuity of at least two of the marginals. Finally, we show that such marginal regularity conditions are necessary for the existence of an unique Monge solution.

math.AP↗

arXiv2023

The search for the lost attractor

N-body systems characterized by inverse square attractive forces may display a self similar collapse known as the gravo-thermal catastrophe. In star clusters, collapse is halted by binary stars, and a large fraction of Milky Way clusters may have already reached this phase. It has been speculated -- with guidance from simulations -- that macroscopic variables such as central density and velocity dispersion are governed post-collapse by an effective, low-dimensional system of ODEs. It is still hard to distinguish chaotic, low dimensional motion, from high dimensional stochastic noise. Here we apply three machine learning tools to state-of-the-art dynamical simulations to constrain the post collapse dynamics: topological data analysis (TDA) on a lag embedding of the relevant time series, Sparse Identification of Nonlinear Dynamics (SINDY), and Tests of Accuracy with Random Points (TARP).

astro-ph.GA↗

arXiv2023

Optimal quantization with branched optimal transport distances

We consider the problem of optimal approximation of a target measure by an atomic measure with $N$ atoms, in branched optimal transport distance. This is a new branched transport version of optimal quantization problems. New difficulties arise, since in classical semi-discrete optimal transport with Wasserstein distance, the interfaces between cells associated with neighboring atoms have Voronoi structure and satisfy an explicit description. This description is missing for our problem, in which the cell interfaces are thought to have fractal boundary. We study the asymptotic behaviour of optimal quantizers for absolutely continuous measures as the number $N$ of atoms grows to infinity. We compute the limit distribution of the corresponding point clouds and show in particular a branched transport version of Zador's theorem. Moreover, we establish uniformity bounds of optimal quantizers in terms of separation distance and covering radius of the atoms, when the measure is $d$-Ahlfors regular. A crucial technical tool is the uniform in $N$ H\"older regularity of the landscape function, a branched transport analog to Kantorovich potentials in classical optimal transport.

math.OC↗

arXiv2023

Effective recovery of Fourier spectra and spectral approximation by finite groups

We prove a result on approximate recovery, with high probability, of subgroups of a finite nonabelian group $\Gamma$ from their random perturbations. We use this for ad-hoc sequences of $\Gamma_n$ while passing to the continuum limit, in order to obtain asymptotic almost sure recovery for rational lcsc nilpotent Lie groups. By comparison to limit theorems for groups of polynomial growth, it turns out that this setting is the natural general setting for recovery results, under polynomial growth assumptions on the $\Gamma_n$. This approach makes effective the convergence rate in previous Fourier recovery theorems in Euclidean space, and extends them to the nonabelian setting. A series of interesting further directions are highlighted by this approach.

math.PR↗

arXiv2023

Approximation-Generalization Trade-offs under (Approximate) Group Equivariance

The explicit incorporation of task-specific inductive biases through symmetry has emerged as a general design precept in the development of high-performance machine learning models. For example, group equivariant neural networks have demonstrated impressive performance across various domains and applications such as protein and drug design. A prevalent intuition about such models is that the integration of relevant symmetry results in enhanced generalization. Moreover, it is posited that when the data and/or the model may only exhibit $\textit{approximate}$ or $\textit{partial}$ symmetry, the optimal or best-performing model is one where the model symmetry aligns with the data symmetry. In this paper, we conduct a formal unified investigation of these intuitions. To begin, we present general quantitative bounds that demonstrate how models capturing task-specific symmetries lead to improved generalization. In fact, our results do not require the transformations to be finite or even form a group and can work with partial or approximate equivariance. Utilizing this quantification, we examine the more general question of model mis-specification i.e. when the model symmetries don't align with the data symmetries. We establish, for a given symmetry group, a quantitative comparison between the approximate/partial equivariance of the model and that of the data distribution, precisely connecting model equivariance error and data equivariance error. Our result delineates conditions under which the model equivariance error is optimal, thereby yielding the best-performing model for the given task and data. Our results are the most general results of their type in the literature.

cs.LG↗

arXiv2023

Three iterations of $(d-1)$-WL test distinguish non isometric clouds of $d$-dimensional points

The Weisfeiler--Lehman (WL) test is a fundamental iterative algorithm for checking isomorphism of graphs. It has also been observed that it underlies the design of several graph neural network architectures, whose capabilities and performance can be understood in terms of the expressive power of this test. Motivated by recent developments in machine learning applications to datasets involving three-dimensional objects, we study when the WL test is {\em complete} for clouds of euclidean points represented by complete distance graphs, i.e., when it can distinguish, up to isometry, any arbitrary such cloud. %arbitrary clouds of euclidean points represented by complete distance graphs. % How many dimensions of the Weisfeiler--Lehman test is enough to distinguish any two non-isometric point clouds in $d$-dimensional Euclidean space, assuming that these point clouds are given as complete graphs labeled by distances between the points? This question is important for understanding, which architectures of graph neural networks are capable of fully exploiting the spacial structure of a point cloud. Our main result states that the $(d-1)$-dimensional WL test is complete for point clouds in $d$-dimensional Euclidean space, for any $d\ge 2$, and that only three iterations of the test suffice. We also observe that the $d$-dimensional WL test only requires one iteration to achieve completeness. Our paper thus provides complete understanding of the 3-dimensional case: it was shown in previous works that 1-WL is not complete in $\mathbb{R}^3$, and we show that 2-WL is complete there. We also strengthen the lower bound for 1-WL by showing that it is unable to recognize planar point clouds in $\mathbb{R}^3$. Finally, we show that 2-WL is not complete in $\mathbb{R}^6$, leaving as an open question, whether it is complete in $\mathbb{R}^{d}$ for $d = 4,5$.

cs.LG↗

arXiv2021

Almost sure recovery in quasi-periodic structures

We study random perturbations of quasi-periodic uniformly discrete sets in the $d$-dimensional euclidean space. By means of Diffraction Theory, we find conditions under which a quasi-periodic set $X$ can be almost surely recovered from its random perturbations. This extends the recent periodic case result of Yakir from "Recovery the lattice from its random perturbations" IMRN 2020 (arXiv:200201508).

math.PR↗