arXiv subjects

Yuri Gardinazzi

Publications and source records attributed to Yuri Gardinazzi.

4 recordsLinked to original sources

arXiv2026

The Intrinsic Dimension of Prompts in Internal Representations of Large Language Models

We study the geometry of token representations at the prompt level in large language models through the lens of intrinsic dimension. Viewing transformers as mean-field particle systems, we estimate the intrinsic dimension of the empirical measure at each layer and demonstrate that it correlates with next-token uncertainty. Across models and intrinsic dimension estimators, we find that intrinsic dimension peaks in early to middle layers and increases under syntactic and semantic disruption (by shuffling tokens), and that it is strongly correlated with average surprisal, with a simple analysis linking logits geometry to entropy via softmax. As a case study in practical interpretability and safety, we train a linear probe on the per-layer intrinsic dimension profile to distinguish malicious from benign prompts before generation. This probe achieves accuracy of 90 to 95\% in different datasets, outperforming widely used guardrails such as Llama Guard and Shield Gemma. We further compare against linear probes built from layerwise entropy derived via the Tuned Lens and find that the intrinsic dimension-based probe is competitive and complementary, offering a compact, interpretable signal distributed across layers. Our findings suggest that prompt-level geometry provides actionable signals for monitoring and controlling LLM behavior, and offers a bridge between mechanistic insights and practical safety tools.

cs.CL↗

arXiv2026

Zigzag Persistence of Neural Responses to Time-Varying Stimuli

We use topological data analysis to study neural population activity in the Sensorium 2023 dataset, which records responses from thousands of mouse visual cortex neurons to diverse video stimuli. For each video, we build frame-by-frame cubical complexes from neuronal activity and apply zigzag persistent homology to capture how topological structure evolves over time. These dynamics are summarized with persistence landscapes, providing a compact vectorized representation of temporal features. We focus on one-dimensional topological features-loops in the data-that reflect coordinated, cyclical patterns of neural co-activation. To test their informativeness, we compare repeated trials of different videos by clustering their resulting topological neural representations. Our results show that these topological descriptors reliably distinguish neural responses to distinct stimuli. This work highlights a connection between evolving neuronal activity and interpretable topological signatures, advancing the use of topological data analysis for uncovering neural coding in complex dynamical systems.

math.AT↗

arXiv2025

Characterization Of Diseases In Temporal Comorbidity Networks

Comorbidity networks, which capture disease-disease co-occurrence usually based on electronic health records, reveal structured patterns in how diseases cluster and progress across individuals. However, how these networks evolve across different age groups and how this evolution relates to properties like disease prevalence and mortality remains understudied. To address these issues, we used publicly available comorbidity networks extracted from a comprehensive dataset of 45 million Austrian hospital stays from 1997 to 2014, covering 8.9 million patients. These networks grow and become denser with age. We identified groups of diseases that exhibit similar patterns of structural centrality throughout the lifespan, revealing three dominant age-related components with peaks in early childhood, midlife, and late life. To uncover the drivers of this structural change, we examined the relationship between prevalence and degree. This allowed us to identify conditions that were disproportionately connected to other diseases. Using betweenness centrality in combination with mortality data, we further identified high-mortality bridging diseases. Several diseases show high connectivity relative to their prevalence, such as iron deficiency anemia (D50) in children, nicotine dependence (F17), and lipoprotein metabolism disorders (E78) in adults. We also highlight structurally central diseases with high mortality that emerge at different life stages, including cancers (C group), liver cirrhosis (K74), subarachnoid hemorrhage (I60), and chronic kidney disease (N18). These findings underscore the importance of targeting age-specific, network-central conditions with high mortality for prevention and integrated care.

physics.soc-ph↗

arXiv2025

Persistent Topological Features in Large Language Models

Understanding the decision-making processes of large language models is critical given their widespread applications. To achieve this, we aim to connect a formal mathematical framework - zigzag persistence from topological data analysis - with practical and easily applicable algorithms. Zigzag persistence is particularly effective for characterizing data as it dynamically transforms across model layers. Within this framework, we introduce topological descriptors that measure how topological features, $p$-dimensional holes, persist and evolve throughout the layers. Unlike methods that assess each layer individually and then aggregate the results, our approach directly tracks the full evolutionary path of these features. This offers a statistical perspective on how prompts are rearranged and their relative positions changed in the representation space, providing insights into the system's operation as an integrated whole. To demonstrate the expressivity and applicability of our framework, we highlight how sensitive these descriptors are to different models and a variety of datasets. As a showcase application to a downstream task, we use zigzag persistence to establish a criterion for layer pruning, achieving results comparable to state-of-the-art methods while preserving the system-level perspective.

cs.CL↗