arXiv subjects

Qiquan Wang

Publications and source records attributed to Qiquan Wang.

6 recordsLinked to original sources

arXiv2026

Topological Signatures of Grokking

We study the grokking phenomenon through the lens of topology. Using persistent homology on point clouds derived from the embedding matrices of a range of models trained on modular arithmetic with varying primes, we identify a clear and consistent topological signature of grokking: a sharp increase in both the maximum and total persistence of first homology ($H_1$). Persistence diagrams reveal the emergence of a dominant long-lived topological feature together with increasingly structured secondary features, reflecting the underlying cyclic structure of the task. Compared to existing spectral and geometric diagnostics -- specifically, Fourier analysis and local intrinsic dimension -- persistent homology provides a unified geometric and topological characterization of representation learning, capturing both local and global multi-scale structure. Ablations across data regimes and control settings show that these topological transitions are tied to generalization rather than memorization. Our results suggest that persistent homology offers a principled and interpretable framework for analyzing how neural networks internalize latent structure during training.

cs.LG↗

arXiv2026

The Shape of Adversarial Influence: Characterizing LLM Latent Spaces with Persistent Homology

Existing interpretability methods for Large Language Models (LLMs) predominantly capture linear directions or isolated features. This overlooks the high-dimensional, relational, and nonlinear geometry of model representations. We apply persistent homology (PH) to characterize how adversarial inputs reshape the geometry and topology of internal representation spaces of LLMs. This phenomenon, especially when considered across operationally different attack modes, remains poorly understood. We analyze six models (3.8B to 70B parameters) under two distinct attacks, indirect prompt injection and backdoor fine--tuning, and show that a consistent topological signature persists throughout. Adversarial inputs induce topological compression, where the latent space becomes structurally simpler, collapsing the latent space from varied, compact, small-scale features into fewer, dominant, large-scale ones. This signature is architecture-agnostic, emerges early in the network, and is highly discriminative across layers. By quantifying the shape of activation point clouds and neuron-level information flow, our framework reveals geometric invariants of representational change that complement existing linear interpretability methods.

cs.LG↗

arXiv2025

A Topological Gaussian Mixture Model for Bone Marrow Morphology in Leukaemia

Acute myeloid leukaemia (AML) is a type of blood and bone marrow cancer characterized by the proliferation of abnormal clonal haematopoietic cells in the bone marrow leading to bone marrow failure. Over the course of the disease, angiogenic factors released by leukaemic cells drastically alter the bone marrow vascular niches resulting in observable structural abnormalities. We use a technique from topological data analysis - persistent homology - to quantify the images and infer on the disease through the imaged morphological features. We find that persistent homology uncovers succinct dissimilarities between the control, early, and late stages of AML development. We then integrate persistent homology into stage-dependent Gaussian mixture models for the first time, proposing a new class of models which are applicable to persistent homology summaries and able to both infer patterns in morphological changes between different stages of progression as well as provide a basis for prediction.

stat.AP↗

arXiv2025

Confidence Bands for Multiparameter Persistence Landscapes

Multiparameter persistent homology is a generalization of classical persistent homology, a central and widely-used methodology from topological data analysis, which takes into account density estimation and is an effective tool for data analysis in the presence of noise. Similar to its classical single-parameter counterpart, however, it is challenging to compute and use in practice due to its complex algebraic construction. In this paper, we study a popular and tractable invariant for multiparameter persistent homology in a statistical setting: the multiparameter persistence landscape. We derive a functional central limit theorem for multiparameter persistence landscapes, from which we compute confidence bands, giving rise to one of the first statistical inference methodologies for multiparameter persistence landscapes. We provide an implementation of confidence bands and demonstrate their application in a machine learning task on synthetic data.

math.ST↗

arXiv2024

On the Limitations of Fractal Dimension as a Measure of Generalization

Bounding and predicting the generalization gap of overparameterized neural networks remains a central open problem in theoretical machine learning. There is a recent and growing body of literature that proposes the framework of fractals to model optimization trajectories of neural networks, motivating generalization bounds and measures based on the fractal dimension of the trajectory. Notably, the persistent homology dimension has been proposed to correlate with the generalization gap. This paper performs an empirical evaluation of these persistent homology-based generalization measures, with an in-depth statistical analysis. Our study reveals confounding effects in the observed correlation between generalization and topological measures due to the variation of hyperparameters. We also observe that fractal dimension fails to predict generalization of models trained from poor initializations. We lastly reveal the intriguing manifestation of model-wise double descent in these topological generalization measures. Our work forms a basis for a deeper investigation of the causal relationships between fractal geometry, topological data analysis, and neural network optimization.

cs.LG↗

arXiv2024

Stability for Inference with Persistent Homology Rank Functions

Persistent homology barcodes and diagrams are a cornerstone of topological data analysis that capture the "shape" of a wide range of complex data structures, such as point clouds, networks, and functions. However, their use in statistical settings is challenging due to their complex geometric structure. In this paper, we revisit the persistent homology rank function, which is mathematically equivalent to a barcode and persistence diagram, as a tool for statistics and machine learning. Rank functions, being functions, enable the direct application of the statistical theory of functional data analysis (FDA)-a domain of statistics adapted for data in the form of functions. A key challenge they present over barcodes in practice, however, is their lack of stability-a property that is crucial to validate their use as a faithful representation of the data and therefore a viable summary statistic. In this paper, we fill this gap by deriving two stability results for persistent homology rank functions under a suitable metric for FDA integration. We then study the performance of rank functions in functional inferential statistics and machine learning on real data applications, in both single and multiparameter persistent homology. We find that the use of persistent homology captured by rank functions offers a clear improvement over existing non-persistence-based approaches.

math.AT↗