arXiv subjects

The-Anh Ta

Publications and source records attributed to The-Anh Ta.

14 recordsLinked to original sources

arXiv2026

Pruning at Initialisation through the lens of Graphon Limit: Convergence, Expressivity, and Generalisation

Pruning at Initialisation methods discover sparse, trainable subnetworks before training, but their theoretical mechanisms remain elusive. Existing analyses are often limited to finite-width statistics, lacking a rigorous characterisation of the global sparsity patterns that emerge as networks grow large. In this work, we connect discrete pruning heuristics to graph limit theory via graphons, establishing the graphon limit of PaI masks. We introduce a Factorised Saliency Model that encompasses popular pruning criteria and prove that, under regularity conditions, the discrete masks generated by these algorithms converge to deterministic bipartite graphons. This limit framework establishes a novel topological taxonomy for sparse networks: while unstructured methods (e.g., Random, Magnitude) converge to homogeneous graphons representing uniform connectivity, data-driven methods (e.g., SNIP, GraSP) converge to heterogeneous graphons that encode implicit feature selection. Leveraging this continuous characterisation, we derive two fundamental theoretical results: (i) a Universal Approximation Theorem for sparse networks that depends only on the intrinsic dimension of active coordinate subspaces; and (ii) a Graphon-NTK generalisation bound demonstrating how the limit graphon modulates the kernel geometry to align with informative features. Our results transform the study of sparse neural networks from combinatorial graph problems into a rigorous framework of continuous operators, offering a new mechanism for analysing expressivity and generalisation in sparse neural networks.

cs.LG↗

arXiv2026

NADD: Amplifying Noise for Effective Diffusion-based Adversarial Purification

The strategy of combining diffusion-based generative models with classifiers continues to demonstrate state-of-the-art performance on adversarial robustness benchmarks. Known as adversarial purification, this exploits a diffusion model's capability of identifying high density regions in data distributions to purify adversarial perturbations from inputs. However, existing diffusion-based purification defenses are impractically slow and limited in robustness due to the low levels of noise used in the diffusion process. This low noise design aims to preserve the semantic features of the original input, thereby minimizing utility loss for benign inputs. Our findings indicate that systematic amplification of noise throughout the diffusion process improves the robustness of adversarial purification. However, this approach presents a key challenge, as noise levels cannot be arbitrarily increased without risking distortion of the input. To address this key problem, we introduce high levels of noise during the forward process and propose the ring proximity correction to gradually eliminate adversarial perturbations whilst closely preserving the original data sample. As a second contribution, we propose a new stochastic sampling method which introduces additional noise during the reverse diffusion process to dilute adversarial perturbations. Without relying on gradient obfuscation, these contributions result in a new robustness accuracy record of 44.23% on ImageNet using AutoAttack ($\ell_{\infty}=4/255$), an improvement of +2.07% over the previous best work. Furthermore, our method reduces inference time to 1.08 seconds per sample on ImageNet, a $47\times$ improvement over the existing state-of-the-art approach, making it far more practical for real-world defensive scenarios.

cs.CR↗

arXiv2025

The Graphon Limit Hypothesis: Understanding Neural Network Pruning via Infinite Width Analysis

Sparse neural networks promise efficiency, yet training them effectively remains a fundamental challenge. Despite advances in pruning methods that create sparse architectures, understanding why some sparse structures are better trainable than others with the same level of sparsity remains poorly understood. Aiming to develop a systematic approach to this fundamental problem, we propose a novel theoretical framework based on the theory of graph limits, particularly graphons, that characterizes sparse neural networks in the infinite-width regime. Our key insight is that connectivity patterns of sparse neural networks induced by pruning methods converge to specific graphons as networks' width tends to infinity, which encodes implicit structural biases of different pruning methods. We postulate the Graphon Limit Hypothesis and provide empirical evidence to support it. Leveraging this graphon representation, we derive a Graphon Neural Tangent Kernel (Graphon NTK) to study the training dynamics of sparse networks in the infinite width limit. Graphon NTK provides a general framework for the theoretical analysis of sparse networks. We empirically show that the spectral analysis of Graphon NTK correlates with observed training dynamics of sparse networks, explaining the varying convergence behaviours of different pruning methods. Our framework provides theoretical insights into the impact of connectivity patterns on the trainability of various sparse network architectures.

cs.LG↗

arXiv2024

Flatness-aware Sequential Learning Generates Resilient Backdoors

Recently, backdoor attacks have become an emerging threat to the security of machine learning models. From the adversary's perspective, the implanted backdoors should be resistant to defensive algorithms, but some recently proposed fine-tuning defenses can remove these backdoors with notable efficacy. This is mainly due to the catastrophic forgetting (CF) property of deep neural networks. This paper counters CF of backdoors by leveraging continual learning (CL) techniques. We begin by investigating the connectivity between a backdoored and fine-tuned model in the loss landscape. Our analysis confirms that fine-tuning defenses, especially the more advanced ones, can easily push a poisoned model out of the backdoor regions, making it forget all about the backdoors. Based on this finding, we re-formulate backdoor training through the lens of CL and propose a novel framework, named Sequential Backdoor Learning (SBL), that can generate resilient backdoors. This framework separates the backdoor poisoning process into two tasks: the first task learns a backdoored model, while the second task, based on the CL principles, moves it to a backdoored region resistant to fine-tuning. We additionally propose to seek flatter backdoor regions via a sharpness-aware minimizer in the framework, further strengthening the durability of the implanted backdoor. Finally, we demonstrate the effectiveness of our method through extensive empirical experiments on several benchmark datasets in the backdoor domain. The source code is available at https://github.com/mail-research/SBL-resilient-backdoors

cs.LG↗

arXiv2024

Wicked Oddities: Selectively Poisoning for Effective Clean-Label Backdoor Attacks

Deep neural networks are vulnerable to backdoor attacks, a type of adversarial attack that poisons the training data to manipulate the behavior of models trained on such data. Clean-label attacks are a more stealthy form of backdoor attacks that can perform the attack without changing the labels of poisoned data. Early works on clean-label attacks added triggers to a random subset of the training set, ignoring the fact that samples contribute unequally to the attack's success. This results in high poisoning rates and low attack success rates. To alleviate the problem, several supervised learning-based sample selection strategies have been proposed. However, these methods assume access to the entire labeled training set and require training, which is expensive and may not always be practical. This work studies a new and more practical (but also more challenging) threat model where the attacker only provides data for the target class (e.g., in face recognition systems) and has no knowledge of the victim model or any other classes in the training set. We study different strategies for selectively poisoning a small set of training samples in the target class to boost the attack success rate in this setting. Our threat model poses a serious threat in training machine learning models with third-party datasets, since the attack can be performed effectively with limited information. Experiments on benchmark datasets illustrate the effectiveness of our strategies in improving clean-label backdoor attacks.

cs.LG↗

arXiv2023

Improving Heterogeneous Graph Learning with Weighted Mixed-Curvature Product Manifold

In graph representation learning, it is important that the complex geometric structure of the input graph, e.g. hidden relations among nodes, is well captured in embedding space. However, standard Euclidean embedding spaces have a limited capacity in representing graphs of varying structures. A promising candidate for the faithful embedding of data with varying structure is product manifolds of component spaces of different geometries (spherical, hyperbolic, or euclidean). In this paper, we take a closer look at the structure of product manifold embedding spaces and argue that each component space in a product contributes differently to expressing structures in the input graph, hence should be weighted accordingly. This is different from previous works which consider the roles of different components equally. We then propose WEIGHTED-PM, a data-driven method for learning embedding of heterogeneous graphs in weighted product manifolds. Our method utilizes the topological information of the input graph to automatically determine the weight of each component in product spaces. Extensive experiments on synthetic and real-world graph datasets demonstrate that WEIGHTED-PM is capable of learning better graph representations with lower geometric distortion from input data, and performs better on multiple downstream tasks, such as word similarity learning, top-$k$ recommendation, and knowledge graph embedding.

cs.LG↗

arXiv2023

A Cosine Similarity-based Method for Out-of-Distribution Detection

The ability to detect OOD data is a crucial aspect of practical machine learning applications. In this work, we show that cosine similarity between the test feature and the typical ID feature is a good indicator of OOD data. We propose Class Typical Matching (CTM), a post hoc OOD detection algorithm that uses a cosine similarity scoring function. Extensive experiments on multiple benchmarks show that CTM outperforms existing post hoc OOD detection methods.

cs.LG↗

arXiv2021

On Convergent Poincaré-Moser Reduction for Levi Degenerate Embedded $5$-Dimensional CR Manifolds

Applying Lie's theory, we show that any $\mathcal{C}^ω$ hypersurface $M^5 \subset \mathbb{C}^3$ in the class $\mathfrak{C}_{2,1}$ carries Cartan-Moser chains of orders $1$ and $2$. Integrating and straightening any order $2$ chain at any point $p \in M$ to be the $v$-axis in coordinates $(z, ζ, w = u + i\, v)$ centered at $p$, we show that there exists a (unique up to 5 parameters) convergent change of complex coordinates fixing the origin in which $γ$ is the $v$-axis so that $M = \{u=F(z,ζ,\overline{z},\overlineζ,v)\}$ has Poincaré-Moser reduced equation: \begin{align} u & = z\overline{z} + \tfrac{1}{2}\,\overline{z}^2ζ+ \tfrac{1}{2}\,z^2\overlineζ + z\overline{z}ζ\overlineζ + \tfrac{1}{2}\,\overline{z}^2ζζ\overlineζ + \tfrac{1}{2}\,z^2\overlineζζ\overlineζ + z\overline{z}ζ\overlineζζ\overlineζ \\ & + 2{\rm Re} \{ z^3\overlineζ^2 F_{3,0,0,2}(v) + ζ\overlineζ ( 3\,{z}^2\overline{z}\overlineζ F_{3,0,0,2}(v) ) \} \\ & + 2{\rm Re} \{ z^5\overlineζ F_{5,0,0,1}(v) + z^4\overlineζ^2 F_{4,0,0,2}(v) + z^3\overline{z}^2\overlineζ F_{3,0,2,1}(v) + z^3\overline{z}\overlineζ^2 F_{3,0,1,2}(v) + z^3{\overlineζ}^3 F_{3,0,0,3}(v) \} \\ & + z^3\overline{z}^3 {\rm O}_{z,\overline{z}}(1) + 2{\rm Re} ( \overline{z}^3ζ{\rm O}_{z,ζ,\overline{z}}(3) ) + ζ\overlineζ\, {\rm O}_{z,ζ,\overline{z},\overlineζ}(5). \end{align} The values at the origin of Pocchiola's two primary invariants are: \[ W_0 = 4\overline{F_{3,0,0,2}(0)}, \quad\quad J_0 = 20\, F_{5,0,0,1}(0). \] The proofs are detailed, accessible to non-experts. The computer-generated aspects (upcoming) have been reduced to a minimum.

math.CV↗

arXiv2021

Homogeneous C21 Models

Fels-Kaup (Acta Mathematica 2008) classified homogeneous $\mathfrak{C}_{2,1}$ hypersurfaces $M^5 \subset \mathbb{C}^3$ and discovered that they are all biholomorphic to tubes $S^2 \times i \mathbb{R}^3$ over some affinely homogeneous surface $S^2 \subset \mathbb{R}^3$. The second and third authors in 2003.08166, by performing highly non-straightforward calculations, conducted the Cartan method of equivalence to classify homogeneous models of PDE systems related to such $\mathfrak{C}^{2,1}$ hypersurfaces $M^5 \subset \mathbb{C}^3$. Kolar-Kossovskiy 1905.05629 and the authors 2003.01952 constructed a formal and a convergent Poincaré-Moser normal form for $\mathfrak{C}_{2,1}$ hypersurfaces $M^5 \subset \mathbb{C}^3$. But this was only a first, preliminary step. Indeed, the invariant branching tree underlying Fels-Kaup's classification was still missing in the literature, due to computational obstacles. The present work applies the power series method of equivalence, confirms Fels-Kaup 2008, and finds a differential-invariant tree. To terminate the middle (thickest) branch, it is necessary to compute up to order $10$ with $5$ variables. Again, calculations, done by hand, are non-straightforward.

math.CV↗

arXiv2020

Normal Forms for Rigid $\mathfrak{C}_{2,1}$ Hypersurfaces $M^5 \subset \mathbb{C}^3$

Consider a $2$-nondegenerate constant Levi rank $1$ rigid $\mathcal{C}^ω$ hypersurface $M^5 \subset \mathbb{C}^3$ in coordinates $(z, ζ, w = u + iv)$: \[ u = F\big(z,ζ,\bar{z},\barζ\big). \] The Gaussier-Merker model $u=\frac{z\bar{z}+ \frac{1}{2}z^2\barζ+\frac{1}{2} \bar{z}^2 ζ}{1-ζ\barζ}$ was shown by Fels-Kaup 2007 to be locally CR-equivalent to the light cone $\{x_1^2+x_2^2-x_3^2=0\}$. Another representation is the tube $u=\frac{x^2}{1-y}$. Inspired by Alexander Isaev, we study rigid biholomorphisms: \[ (z,ζ,w) \longmapsto \big( f(z,ζ), g(z,ζ), ρ\,w+h(z,ζ) \big) =: (z',ζ',w'). \] The G-M model has 7-dimensional rigid automorphisms group. A Cartan-type reduction to an e-structure was done by Foo-Merker-Ta in 1904.02562. Three relative invariants appeared: $V_0$, $I_0$ (primary) and $Q_0$ (derived). In Pocchiola's formalism, Section 8 provides a finalized expression for $Q_0$. The goal is to establish the Poincaré-Moser complete normal form: \[ u = \frac{z\bar{z}+\frac{1}{2}\,z^2\barζ +\frac{1}{2}\,\bar{z}^2ζ}{ 1-ζ\barζ} + \sum_{a,b,c,d \atop a+c\geqslant 3}\, G_{a,b,c,d}\, z^aζ^b\bar{z}^c\barζ^d, \] with $0 = G_{a,b,0,0} = G_{a,b,1,0} = G_{a,b,2,0}$ and $0 = G_{3,0,0,1} = {\rm Im}\, G_{3,0,1,1}$. We apply the method of Chen-Merker 1908.07867 to catch (relative) invariants at every point, not only at the central point, as the coefficients $G_{0,1,4,0}$, $G_{0, 2, 3, 0}$, ${\rm Re} G_{3,0,1,1}$. With this, a brige Poincaré $\longleftrightarrow$ Cartan is constructed. In terms of $F$, the numerators of $V_0$, $I_0$, $Q_0$ incorporate 11, 52, 824 differential monomials.

math.CV↗

arXiv2019

Rigid equivalences of $5$-dimensional $2$-nondegenerate rigid real hypersurfaces $M^5 \subset \mathbb{C}^ 3$ of constant Levi rank $1$

We study the local equivalence problem for real-analytic ($\mathcal{C}^ω$) hypersurfaces $M^5 \subset \mathbb{C}^3$ which, in coordinates $(z_1, z_2, w) \in \mathbb{C}^3$ with $w = u+i\, v$, are rigid: \[ u \,=\, F\big(z_1,z_2,\overline{z}_1,\overline{z}_2\big), \] with $F$ independent of $v$. Specifically, we study the group ${\sf Hol}_{\sf rigid}(M)$ of rigid local biholomorphic transformations of the form: \[ \big(z_1,z_2,w\big) \longmapsto \Big( f_1(z_1,z_2), f_2(z_1,z_2), a\,w + g(z_1,z_2) \Big), \] where $a \in \mathbb{R} \backslash \{0\}$ and $\frac{D(f_1,f_2)}{D(z_1,z_2)} \neq 0$, which preserve rigidity of hypersurfaces. After performing a Cartan-type reduction to an appropriate $\{e\}$-structure, we find exactly two primary invariants $I_0$ and $V_0$, which we express explicitly in terms of the $5$-jet of the graphing function $F$ of $M$. The identical vanishing $0 \equiv I_0 \big( J^5F \big) \equiv V_0 \big( J^5F \big)$ then provides a necessary and sufficient condition for $M$ to be locally rigidly-biholomorphic to the known model hypersurface: \[ M_{\sf LC} \colon \ \ \ \ \ u \,=\, \frac{z_1\,\overline{z}_1 +\frac{1}{2}\,z_1^2\overline{z}_2 +\frac{1}{2}\,\overline{z}_1^2z_2}{ 1-z_2\overline{z}_2}. \] We establish that $\dim\, {\sf Hol}_{\sf rigid} (M) \leq 7 = \dim\, {\sf Hol}_{\sf rigid} \big( M_{\sf LC} \big)$ always. If one of these two primary invariants $I_0 \not\equiv 0$ or $V_0 \not\equiv 0$ does not vanish identically, we show that this rigid equivalence problem between rigid hypersurfaces reduces to an equivalence problem for a certain $5$-dimensional $\{e\}$-structure on $M$.

math.DG↗

arXiv2019

Nonvanishing of Cartan CR curvature on boundaries of Grauert tubes around hyperbolic surfaces

We show that the boundaries of thin strongly pseudoconvex Grauert tubes, with respect to the Guillemin-Stenzel Kähler metric canonically associated with the Poincaré metric on closed hyperbolic real-analytic surfaces, has nowhere vanishing Cartan CR-curvature. This result provides a wealth of examples of compact $3$-dimensional Levi nondegenerate CR manifolds having no CR-umbilical point. We provide two proofs utilizing two recent formulas for determining the Cartan CR-curvature of any local $\mathcal{C}^6$-smooth hypersurfaces in $\mathbb{C}^2$. One was obtained in 2012 by the second named author joint with Sabzevari, and it is an expanded explicit formula, valid for locally graphed hypersurfaces, containing millions of terms. The other formula, which we published in 2018 when studying Webster's ellipsoidal hypersurfaces, is not expanded, but more suitable for calculations with a hypersurface in $\mathbb{C}^2$ that is represented as the zero locus of some implicit (but simple in some sense, e.g. quadratic) defining function. We also discuss Grauert tubes constructed with respect to extrinsic metrics depending on embeddings in complex surfaces, together with a certain combinatorics of product metrics.

math.CV↗

arXiv2019

Degrees $d \geqslant \big( \sqrt{n}\, \log\, n\big)^n$ and $d \geqslant \big( n\, \log\, n\big)^n$ in the Conjectures of Green-Griffiths and of Kobayashi

Once first answers in any dimension to the Green-Griffiths and Kobayashi conjectures for generic algebraic hypersurfaces $\mathbb{X}^{n-1} \subset \mathbb{P}^n(\mathbb{C})$ have been reached, the principal goal is to decrease (to improve) the degree bounds, knowing that the `celestial' horizon lies near $d \geqslant 2n$. For Green-Griffiths algebraic degeneracy of entire holomorphic curves, we obtain: \[ d \,\geqslant\, \big(\sqrt{n}\,{\sf log}\,n\big)^n, \] and for Kobayashi-hyperbolicity (constancy of entire curves), we obtain: \[ d \,\geqslant\, \big(n\,{\sf log}\,n\big)^n. \] The latter improves $d \geqslant n^{2n}$ obtained by Merker in arxiv.org/1807/11309/. Admitting a certain technical conjecture $I_0 \geqslant \widetilde{I}_0$, the method employed (Diverio-Merker-Rousseau, Bérczi, Darondeau) conducts to constant power $n$, namely to: \[ d\ ,\geqslant\, 2^{5n} \qquad \text{and, respectively, to:} \qquad d \,\geqslant\, 4^{5n}. \] In Spring 2019, a forthcoming prepublication based on intensive computer explorations will present several subconjectures supporting the belief that $I_0 \geqslant \widetilde{I}_0$, a conjecture which will be established up to dimension $n = 50$.

math.AG↗

arXiv2017

Parametric CR-umbilical Locus of Ellipsoids in $\mathbb{C}^2$

For every real numbers $a \geqslant 1$, $b \geqslant 1$ with $(a,b) \neq (1,1)$, the curve parametrized by $θ\in \mathbb{R}$ valued in $\mathbb{C}^2 \cong \mathbb{R}^4$ \[ γ\, \colon \ \ \ θ\,\,\,\longmapsto\,\,\, \big( x(θ)+{\scriptstyle{\sqrt{-1}}}\,y(θ),\,\, u(θ)+{\scriptstyle{\sqrt{-1}}}\,v(θ) \big) \] with components: \[ x(θ) \,:=\, {\textstyle{\sqrt{\frac{a-1}{a\,(ab-1)}}}}\, \cos\,θ, \ \ \ \ \ y(θ) \,:=\, {\textstyle{\sqrt{\frac{b\,(a-1)}{ab-1}}}}\, \sin\,θ, \ \ \ \ \ u(θ) \,:=\, {\textstyle{\sqrt{\frac{b-1}{b\,(ab-1)}}}}\, \sin\,θ, \ \ \ \ \ v(θ) \,:=\, -\, {\textstyle{\sqrt{\frac{a\,(b-1)}{ab-1}}}}\, \cos\,θ, \] has image contained in the CR-umbilical locus: \[ γ(\mathbb{R}) \,\subset\, {\sf UmbCR} \big({\sf E}_{a,b}\big) \,\subset\, {\sf E}_{a,b} \] of the ellipsoid ${\sf E}_{a,b} \subset \mathbb{C}^2$ of equation $a\,x^2+y^2+b\,u^2+y^2 = 1$.

math.CV↗