arXiv subjects

Armanouche Gasparian

Publications and source records attributed to Armanouche Gasparian.

4 recordsLinked to original sources

arXiv2025

Polysemy of Synthetic Neurons Towards a New Type of Explanatory Categorical Vector Spaces

The polysemantic nature of synthetic neurons in artificial intelligence language models is currently understood as the result of a necessary superposition of distributed features within the latent space. We propose an alternative approach, geometrically defining a neuron in layer n as a categorical vector space with a non-orthogonal basis, composed of categorical sub-dimensions extracted from preceding neurons in layer n-1. This categorical vector space is structured by the activation space of each neuron and enables, via an intra-neuronal attention process, the identification and utilization of a critical categorical zone for the efficiency of the language model - more homogeneous and located at the intersection of these different categorical sub-dimensions.

cs.CL↗

arXiv2025

Intra-neuronal attention within language models Relationships between activation and semantics

This study investigates the ability of perceptron-type neurons in language models to perform intra-neuronal attention; that is, to identify different homogeneous categorical segments within the synthetic thought category they encode, based on a segmentation of specific activation zones for the tokens to which they are particularly responsive. The objective of this work is therefore to determine to what extent formal neurons can establish a homomorphic relationship between activation-based and categorical segmentations. The results suggest the existence of such a relationship, albeit tenuous, only at the level of tokens with very high activation levels. This intra-neuronal attention subsequently enables categorical restructuring processes at the level of neurons in the following layer, thereby contributing to the progressive formation of high-level categorical abstractions.

cs.AI↗

arXiv2025

Synthetic Categorical Restructuring large Or How AIs Gradually Extract Efficient Regularities from Their Experience of the World

How do language models segment their internal experience of the world of words to progressively learn to interact with it more efficiently? This study in the neuropsychology of artificial intelligence investigates the phenomenon of synthetic categorical restructuring, a process through which each successive perceptron neural layer abstracts and combines relevant categorical sub-dimensions from the thought categories of its previous layer. This process shapes new, even more efficient categories for analyzing and processing the synthetic system's own experience of the linguistic external world to which it is exposed. Our genetic neuron viewer, associated with this study, allows visualization of the synthetic categorical restructuring phenomenon occurring during the transition from perceptron layer 0 to 1 in GPT2-XL.

cs.CL↗

arXiv2024

Neuropsychology of AI: Relationship Between Activation Proximity and Categorical Proximity Within Neural Categories of Synthetic Cognition

Neuropsychology of artificial intelligence focuses on synthetic neural cog nition as a new type of study object within cognitive psychology. With the goal of making artificial neural networks of language models more explainable, this approach involves transposing concepts from cognitive psychology to the interpretive construction of artificial neural cognition. The human cognitive concept involved here is categorization, serving as a heuristic for thinking about the process of segmentation and construction of reality carried out by the neural vectors of synthetic cognition.

q-bio.NC↗