arXiv subjects

Efstratios Tsoukanis

Publications and source records attributed to Efstratios Tsoukanis.

7 recordsLinked to original sources

arXiv2026

Quantitative Bounds for Sorting-Based Permutation-Invariant Embeddings

We study permutation-invariant embeddings of $d$-dimensional point sets, which are defined by sorting $D$ independent one-dimensional projections of the input. Such embeddings arise in graph deep learning where outputs should be invariant to permutations of graph nodes. Previous work showed that for large enough $D$ and projections in general position, this mapping is injective, and moreover satisfies a bi-Lipschitz condition. However, two gaps remain: firstly, the optimal size $D$ required for injectivity is not yet known, and secondly, no estimates of the bi-Lipschitz constants of the mapping are known. In this paper, we make substantial progress in addressing both of these gaps. Regarding the first gap, we improve upon the best known upper bounds for the embedding dimension $D$ necessary for injectivity, and also provide a lower bound on the minimal injectivity dimension. Regarding the second gap, we construct matrices of projection vectors, so that the bi-Lipschitz distortion of the mapping depends quadratically on the number of points $n$, and is completely independent of the dimension $d$. We also show that for any choice of projection vectors, the distortion of the mapping will never be better than a bound proportional to the square root of $n$. Finally, we show that similar guarantees can be provided even when linear projections are applied to the mapping to reduce its dimension.

cs.LG↗

arXiv2026

An Approximation Theory Perspective on Machine Learning

A central problem in machine learning is often formulated as follows: Given a dataset $\{(x_j, y_j)\}_{j=1}^M$, which is a sample drawn from an unknown probability distribution, the goal is to construct a functional model $f$ such that $f(x) \approx y$ for any $(x, y)$ drawn from the same distribution. Neural networks and kernel-based methods are commonly employed for this task due to their capacity for fast and parallel computation. The approximation capabilities, or expressive power, of these methods have been extensively studied over the past 35 years. In this paper, we will present examples of key ideas in this area found in the literature. We will discuss emerging trends in machine learning including the role of shallow/deep networks, approximation on manifolds, physics-informed neural surrogates, neural operators, and transformer architectures. Despite function approximation being a fundamental problem in machine learning, approximation theory does not play a central role in the theoretical foundations of the field. One unfortunate consequence of this disconnect is that it is often unclear how well trained models will generalize to unseen or unlabeled data. In this review, we examine some of the shortcomings of the current machine learning framework and explore the reasons for the gap between approximation theory and machine learning practice. We will then introduce our novel research to achieve function approximation on unknown manifolds without the need to learn specific manifold features, such as the eigen-decomposition of the Laplace-Beltrami operator or atlas construction. In many machine learning problems, particularly classification tasks, the labels $y_j$ are drawn from a finite set of values.

cs.LG↗

arXiv2025

G-Invariant Representations using Coorbits: Bi-Lipschitz Properties

Consider a finite dimensional real vector space and a finite group acting unitarily on it. We study the general problem of constructing Euclidean stable embeddings of the quotient space of orbits. Our embedding is based on subsets of sorted coorbits. Our main result shows that, whenever such embeddings are injective, they are automatically bi-Lipschitz. Additionally, we demonstrate that stable embeddings can be achieved with reduced dimensionality, and that any continuous or Lipschitz $G$-invariant map can be factorized through these embeddings.

math.RT↗

arXiv2025

G-Invariant Representations using Coorbits: Injectivity Properties

Consider a finite-dimensional real vector space equipped with a finite group acting unitarily on it. We address the general problem of constructing Euclidean stable embeddings of the quotient space of orbits. Our approach is based on subsets of sorted coorbits with respect to selected window vectors. We derive conditions under which such embeddings are injective, using techniques from semi-algebraic analysis. These results are then applied to the specific case of planar rotation invariance.

math.RT↗

arXiv2025

Active Learning Classification from a Signal Separation Perspective

In machine learning, classification is usually seen as a function approximation problem, where the goal is to learn a function that maps input features to class labels. In this paper, we propose a novel clustering and classification framework inspired by the principles of signal separation. This approach enables efficient identification of class supports, even in the presence of overlapping distributions. We validate our method on real-world hyperspectral datasets Salinas and Indian Pines. The experimental results demonstrate that our method is competitive with the state of the art active learning algorithms by using a very small subset of data set as training points.

cs.LG↗

arXiv2024

Stability of sorting based embeddings

Consider a group $G$ of order $M$ acting unitarily on a real inner product space $V$. We show that the sorting based embedding obtained by applying a general linear map $α: \mathbb{R}^{M \times N} \to \mathbb{R}^D$ to the invariant map $β_Φ: V \to \mathbb{R}^{M \times N}$ given by sorting the coorbits $(\langle v, g ϕ_i \rangle_V)_{g \in G}$, where $(ϕ_i)_{i=1}^N \in V$, satisfies a bi-Lipschitz condition if and only if it separates orbits. Additionally, we note that any invariant Lipschitz continuous map (into a Hilbert space) factors through the sorting based embedding, and that any invariant continuous map (into a locally convex space) factors through the sorting based embedding as well.

math.FA↗

arXiv2023

Relationships between the Phase Retrieval Problem and Permutation Invariant Embeddings

This paper discusses the connection between the phase retrieval problem and permutation invariant embeddings. We show that the real phase retrieval problem for $\mathbb{R}^d/O(1)$ is equivalent to Euclidean embeddings of the quotient space $\mathbb{R}^{2\times d}/S_2$ performed by the sorting encoder introduced in an earlier work. In addition, this relationship provides us with inversion algorithms of the orbits induced by the group of permutation matrices.

math.FA↗