arXiv subjects

Carlo Vittorio Cannistraci

Publications and source records attributed to Carlo Vittorio Cannistraci.

17 recordsLinked to original sources

arXiv2026

Brain network science modelling of sparse neural networks enables Transformers and LLMs to perform as fully connected

Dynamic sparse training (DST) can reduce the computational demands in ANNs, but faces difficulties in keeping peak performance at high sparsity levels. The Cannistraci-Hebb training (CHT) is a brain-inspired method for growing connectivity in DST. CHT leverages a gradient-free, topology-driven link regrowth, which has shown ultra-sparse (less than 1% connectivity) advantage across various tasks compared to fully connected networks. Yet, CHT suffers two main drawbacks: (i) its time complexity is $O(Nd^3)$ - N node network size, d node degree - restricting it to ultra-sparse regimes. (ii) it selects top link prediction scores, which is inappropriate for the early training epochs, when the network presents unreliable connections. Here, we design the first brain-inspired network model - termed bipartite receptive field (BRF) - to initialize the connectivity of sparse artificial neural networks. We further introduce a GPU-friendly matrix-based approximation of CH link prediction, reducing complexity to $O(N^3)$. We introduce the Cannistraci-Hebb training soft rule (CHTs), which adopts a flexible strategy for sampling connections in both link removal and regrowth, balancing the exploration and exploitation of network topology. Additionally, we integrate CHTs with a sigmoid gradual density decay (CHTss). Empirical results show that BRF offers performance advantages over previous network science models. Using 1% of connections, CHTs outperforms fully connected networks in MLP architectures on image classification tasks, compressing some networks to less than 30% of the nodes. Using 5% of the connections, CHTss outperforms fully connected networks in two Transformer-based machine translation tasks. Finally, at 30% connectivity, both CHTs and CHTss outperform other DST methods in language modeling task.

cs.LG↗

arXiv2025

Cannistraci-Hebb Training on Ultra-Sparse Spiking Neural Networks

Inspired by the brain's spike-based computation, spiking neural networks (SNNs) inherently possess temporal activation sparsity. However, when it comes to the sparse training of SNNs in the structural connection domain, existing methods fail to achieve ultra-sparse network structures without significant performance loss, thereby hindering progress in energy-efficient neuromorphic computing. This limitation presents a critical challenge: how to achieve high levels of structural connection sparsity while maintaining performance comparable to fully connected networks. To address this challenge, we propose the Cannistraci-Hebb Spiking Neural Network (CH-SNN), a novel and generalizable dynamic sparse training framework for SNNs consisting of four stages. First, we propose a sparse spike correlated topological initialization (SSCTI) method to initialize a sparse network based on node correlations. Second, temporal activation sparsity and structural connection sparsity are integrated via a proposed sparse spike weight initialization (SSWI) method. Third, a hybrid link removal score (LRS) is applied to prune redundant weights and inactive neurons, improving information flow. Finally, the CH3-L3 network automaton framework inspired by Cannistraci-Hebb learning theory is incorporated to perform link prediction for potential synaptic regrowth. These mechanisms enable CH-SNN to achieve sparsification across all linear layers. We have conducted extensive experiments on six datasets including CIFAR-10 and CIFAR-100, evaluating various network architectures such as spiking convolutional neural networks and Spikformer.

cs.NE↗

arXiv2025

Sparse Spectral Training and Inference on Euclidean and Hyperbolic Neural Networks

The growing demands on GPU memory posed by the increasing number of neural network parameters call for training approaches that are more memory-efficient. Previous memory reduction training techniques, such as Low-Rank Adaptation (LoRA) and ReLoRA, face challenges, with LoRA being constrained by its low-rank structure, particularly during intensive tasks like pre-training, and ReLoRA suffering from saddle point issues. In this paper, we propose Sparse Spectral Training (SST) to optimize memory usage for pre-training. SST updates all singular values and selectively updates singular vectors through a multinomial sampling method weighted by the magnitude of the singular values. Furthermore, SST employs singular value decomposition to initialize and periodically reinitialize low-rank parameters, reducing distortion relative to full-rank training compared to other low-rank methods. Through comprehensive testing on both Euclidean and hyperbolic neural networks across various tasks, SST demonstrates its ability to outperform existing memory reduction training methods and is comparable to full-rank training in various cases. On LLaMA-1.3B, with only 18.7\% of the parameters trainable compared to full-rank training (using a rank equivalent to 6\% of the embedding dimension), SST reduces the perplexity gap between other low-rank methods and full-rank training by 97.4\%. This result highlights SST as an effective parameter-efficient technique for model pre-training.

cs.LG↗

arXiv2025

Pivoting Factorization: A Compact Meta Low-Rank Representation of Sparsity for Efficient Inference in Large Language Models

The rapid growth of Large Language Models has driven demand for effective model compression techniques to reduce memory and computation costs. Low-rank pruning has gained attention for its GPU compatibility across all densities. However, low-rank pruning struggles to match the performance of semi-structured pruning, often doubling perplexity at similar densities. In this paper, we propose Pivoting Factorization (PIFA), a novel lossless meta low-rank representation that unsupervisedly learns a compact form of any low-rank representation, effectively eliminating redundant information. PIFA identifies pivot rows (linearly independent rows) and expresses non-pivot rows as linear combinations, achieving 24.2% additional memory savings and 24.6% faster inference over low-rank layers at rank = 50% of dimension. To mitigate the performance degradation caused by low-rank pruning, we introduce a novel, retraining-free reconstruction method that minimizes error accumulation (M). MPIFA, combining M and PIFA into an end-to-end framework, significantly outperforms existing low-rank pruning methods, and achieves performance comparable to semi-structured pruning, while surpassing it in GPU efficiency and compatibility. Our code is available at https://github.com/biomedical-cybernetics/pivoting-factorization.

cs.LG↗

arXiv2020

Geometrical congruence and efficient greedy navigability of complex networks

Hyperbolic networks are supposed to be congruent with their underlying latent geometry and following geodesics in the hyperbolic space is believed equivalent to navigate through topological shortest paths (TSP). This assumption of geometrical congruence is considered the reason for nearly maximally efficient greedy navigation of hyperbolic networks. Here, we propose a complex network measure termed geometrical congruence (GC) and we show that there might exist different TSP, whose projections (pTSP) in the hyperbolic space largely diverge, and significantly differ from the respective geodesics. We discover that, contrary to current belief, hyperbolic networks do not demonstrate in general geometrical congruence and efficient navigability which, in networks generated with nPSO model, seem to emerge only for power-law exponent close to 2. We conclude by showing that GC measure can impact also real networks analysis, indeed it significantly changes in structural brain connectomes grouped by gender or age.

cs.SI↗

arXiv2020

Modular gateway-ness connectivity and structural core organization in maritime network science

Around 80% of global trade by volume is transported by sea, and thus the maritime transportation system is fundamental to the world economy. To better exploit new international shipping routes, we need to understand the current ones and their complex systems association with international trade. We investigate the structure of the global liner shipping network (GLSN), finding it is an economic small-world network with a trade-off between high transportation efficiency and low wiring cost. To enhance understanding of this trade-off, we examine the modular segregation of the GLSN; we study provincial-, connector-hub ports and propose the definition of gateway-hub ports, using three respective structural measures. The gateway-hub structural-core organization seems a salient property of the GLSN, which proves importantly associated to network integration and function in realizing the cargo transportation of international trade. This finding offers new insights into the GLSN's structural organization complexity and its relevance to international trade.

physics.soc-ph↗

arXiv2020

Cell Mechanics Based Computational Classification of Red Blood Cells Via Machine Intelligence Applied to Morpho-Rheological Markers

Despite fluorescent cell-labelling being widely employed in biomedical studies, some of its drawbacks are inevitable, with unsuitable fluorescent probes or probes inducing a functional change being the main limitations. Consequently, the demand for and development of label-free methodologies to classify cells is strong and its impact on precision medicine is relevant. Towards this end, high-throughput techniques for cell mechanical phenotyping have been proposed to get a multidimensional biophysical characterization of single cells. With this motivation, our goal here is to investigate the extent to which an unsupervised machine learning methodology, which is applied exclusively on morpho-rheological markers obtained by real-time deformability and fluorescence cytometry (RT-FDC), can address the difficult task of providing label-free discrimination of reticulocytes from mature red blood cells. We focused on this problem, since the characterization of reticulocytes (their percentage and cellular features) in the blood is vital in multiple human disease conditions, especially bone-marrow disorders such as anemia and leukemia. Our approach reports promising label-free results in the classification of reticulocytes from mature red blood cells, and it represents a step forward in the development of high-throughput morpho-rheological-based methodologies for the computational categorization of single cells. Besides, our methodology can be an alternative but also a complementary method to integrate with existing cell-labelling techniques.

q-bio.QM↗

arXiv2019

Angular separability of data clusters or network communities in geometrical space and its relevance to hyperbolic embedding

Analysis of 'big data' characterized by high-dimensionality such as word vectors and complex networks requires often their representation in a geometrical space by embedding. Recent developments in machine learning and network geometry have pointed out the hyperbolic space as a useful framework for the representation of this data derived by real complex physical systems. In the hyperbolic space, the radial coordinate of the nodes characterizes their hierarchy, whereas the angular distance between them represents their similarity. Several studies have highlighted the relationship between the angular coordinates of the nodes embedded in the hyperbolic space and the community metadata available. However, such analyses have been often limited to a visual or qualitative assessment. Here, we introduce the angular separation index (ASI), to quantitatively evaluate the separation of node network communities or data clusters over the angular coordinates of a geometrical space. ASI is particularly useful in the hyperbolic space - where it is extensively tested along this study - but can be used in general for any assessment of angular separation regardless of the adopted geometry. ASI is proposed together with an exact test statistic based on a uniformly random null model to assess the statistical significance of the separation. We show that ASI allows to discover two significant phenomena in network geometry. The first is that the increase of temperature in 2D hyperbolic network generative models, not only reduces the network clustering but also induces a 'dimensionality jump' of the network to dimensions higher than two. The second is that ASI can be successfully applied to detect the intrinsic dimensionality of network structures that grow in a hidden geometrical space.

cs.LG↗

arXiv2019

Navigability evaluation of complex networks by greedy routing efficiency

Network navigability is a key feature of complex networked systems. For a network embedded in a geometrical space, maximization of greedy routing (GR) measures based on the node geometrical coordinates can ensure efficient greedy navigability. In PNAS, Seguin et al. (PNAS 2018, vol. 115, no. 24) define a measure for quantifying the efficiency of brain network navigability in the Euclidean space, referred to as the efficiency ratio, whose formula exactly coincides with the GR-score (GR-efficiency) previously published by Muscoloni et al. (Nature Communications 2017, vol. 8, no. 1615). In this Letter, we point out potential flaws in the study of Seguin et al. regarding the discussion of the GR evaluation. In particular, we revise the concept of GR navigability, together with a careful discussion of the advantage offered by the new proposed GR-efficiency measure in comparison to the main measures previously adopted in literature. Finally, we clarify and standardize the GR-efficiency terminology in order to simplify and facilitate the discussion in future studies.

physics.soc-ph↗

arXiv2018

Local-ring network automata and the impact of hyperbolic geometry in complex network link-prediction

Topological link-prediction can exploit the entire network topology (global methods) or only the neighbourhood (local methods) of the link to predict. Global methods are believed the best. Is this common belief well-founded? Stochastic-Block-Model (SBM) is a global method believed as one of the best link-predictors, therefore it is considered a reference for comparison. But, our results suggest that SBM, whose computational time is high, cannot in general overcome the Cannistraci-Hebb (CH) network automaton model that is a simple local-learning-rule of topological self-organization proved as the current best local-based and parameter-free deterministic rule for link-prediction. To elucidate the reasons of this unexpected result, we formally introduce the notion of local-ring network automata models and their relation with the nature of common-neighbours' definition in complex network theory. After extensive tests, we recommend Structural-Perturbation-Method (SPM) as the new best global method baseline. However, even SPM overall does not outperform CH and in several evaluation frameworks we astonishingly found the opposite. In particular, CH was the best predictor for synthetic networks generated by the Popularity-Similarity-Optimization (PSO) model, and its performance in PSO networks with community structure was even better than using the original internode-hyperbolic-distance as link-predictor. Interestingly, when tested on non-hyperbolic synthetic networks the performance of CH significantly dropped down indicating that this rule of network self-organization could be strongly associated to the rise of hyperbolic geometry in complex networks. The superiority of global methods seems a "misleading belief" caused by a latent geometry bias of the few small networks used as benchmark in previous studies. We propose to found a latent geometry theory of link-prediction in complex networks.

physics.soc-ph↗

arXiv2018

Latent Geometry Inspired Graph Dissimilarities Enhance Affinity Propagation Community Detection in Complex Networks

Affinity propagation is one of the most effective unsupervised pattern recognition algorithms for data clustering in high-dimensional feature space. However, the numerous attempts to test its performance for community detection in complex networks have been attaining results very far from the state of the art methods such as Infomap and Louvain. Yet, all these studies agreed that the crucial problem is to convert the unweighted network topology in a 'smart-enough' node dissimilarity matrix that is able to properly address the message passing procedure behind affinity propagation clustering. Here we introduce a conceptual innovation and we discuss how to leverage network latent geometry notions in order to design dissimilarity matrices for affinity propagation community detection. Our results demonstrate that the latent geometry inspired dissimilarity measures we design bring affinity propagation to equal or outperform current state of the art methods for community detection. These findings are solidly proven considering both synthetic 'realistic' networks (with known ground-truth communities) and real networks (with community metadata), even when the data structure is corrupted by noise artificially induced by missing or spurious connectivity.

cs.LG↗

arXiv2018

Minimum curvilinear automata with similarity attachment for network embedding and link prediction in the hyperbolic space

The idea of minimum curvilinearity (MC) is that the hidden geometry of complex networks, in particular when they are sufficiently sparse, clustered, small-world and heterogeneous, can be efficiently navigated using the minimum spanning tree (MST), which is a greedy navigator. The local topological information drives the global geometrical navigation and the MST can be interpreted as a growing path that greedily maximizes local similarity between the nodes attached at each step by globally minimizing their overall distances in the network. This is also valid in absence of the network structure and in presence of only the nodes geometrically located over the network generative manifold in a high-dimensional space. We know that random geometric graphs in the hyperbolic space are an adequate model for realistic complex networks: the explanation of this connection is that complex networks exhibit hierarchical, tree-like organization, and in turn the hyperbolic geometry is the geometry of trees. Here we show that, according to a mechanism that we define similarity attachment, the visited node sequence of a network automaton can efficiently approximate the nodes' angular coordinates in the hyperbolic disk, that actually represent an ordering of their similarities. This is a consequence of the fact that the MST, during its greedy growing process, at each step sequentially attaches the node most similar (less distant) to its own cohort. Minimum curvilinear automata (MCA) displays embedding accuracy which seems superior to HyperMap-CN and inferior to coalescent embedding, however its link prediction performance on real networks is without precedent for methods based on the hyperbolic space. Finally, depending on the data structure used to build the MST, the MCA's time complexity can also approach a linear dependence from the number of edges.

physics.soc-ph↗

arXiv2017

A nonuniform popularity-similarity optimization (nPSO) model to efficiently generate realistic complex networks with communities

The hidden metric space behind complex network topologies is a fervid topic in current network science and the hyperbolic space is one of the most studied, because it seems associated to the structural organization of many real complex systems. The Popularity-Similarity-Optimization (PSO) model simulates how random geometric graphs grow in the hyperbolic space, reproducing strong clustering and scale-free degree distribution, however it misses to reproduce an important feature of real complex networks, which is the community organization. The Geometrical-Preferential-Attachment (GPA) model was recently developed to confer to the PSO also a community structure, which is obtained by forcing different angular regions of the hyperbolic disk to have variable level of attractiveness. However, the number and size of the communities cannot be explicitly controlled in the GPA, which is a clear limitation for real applications. Here, we introduce the nonuniform PSO (nPSO) model that, differently from GPA, forces heterogeneous angular node attractiveness by sampling the angular coordinates from a tailored nonuniform probability distribution, for instance a mixture of Gaussians. The nPSO differs from GPA in other three aspects: it allows to explicitly fix the number and size of communities; it allows to tune their mixing property through the network temperature; it is efficient to generate networks with high clustering. After several tests we propose the nPSO as a valid and efficient model to generate networks with communities in the hyperbolic space, which can be adopted as a realistic benchmark for different tasks such as community detection and link prediction.

physics.soc-ph↗

arXiv2017

Coalescent embedding in the hyperbolic space unsupervisedly discloses the hidden geometry of the brain

The human brain displays a complex network topology, whose structural organization is widely studied using diffusion tensor imaging. The original geometry from which emerges the network topology is known, as well as the localization of the network nodes in respect to the brain morphology and anatomy. One of the most challenging problems of current network science is to infer the latent geometry from the mere topology of a complex network. The human brain structural connectome represents the perfect benchmark to test algorithms aimed to solve this problem. Coalescent embedding was recently designed to map a complex network in the hyperbolic space, inferring the node angular coordinates. Here we show that this methodology is able to unsupervisedly reconstruct the latent geometry of the brain with an incredible accuracy and that the intrinsic geometry of the brain networks strongly relates to the lobes organization known in neuroanatomy. Furthermore, coalescent embedding allowed the detection of geometrical pathological changes in the connectomes of Parkinson's Disease patients. The present study represents the first evidence of brain networks' angular coalescence in the hyperbolic space, opening a completely new perspective, possibly towards the realization of latent geometry network markers for evaluation of brain disorders and pathologies.

q-bio.NC↗

arXiv2017

Rich-clubness test: how to determine whether a complex network has or doesn't have a rich-club?

The rich-club concept has been introduced in order to characterize the presence of a cohort of nodes with a large number of links (rich nodes) that tend to be well connected between each other, creating a tight group (club). Rich-clubness defines the extent to which a network displays a topological organization characterized by the presence of a node rich-club. It is crucial for the investigation of internal organization and function of networks arising in systems of disparate fields such as transportation, social, communication and neuroscience. Different methods have been proposed for assessing the rich-clubness and various null-models have been adopted for performing statistical tests. However, a procedure that assigns a unique value of rich-clubness significance to a given network is still missing. Our solution to this problem grows on the basis of three new pillars. We introduce: i) a null-model characterized by a lower rich-club coefficient; ii) a fair strategy to normalize the level of rich-clubness of a network in respect to the null-model; iii) a statistical test that, exploiting the maximum deviation of the normalized rich-club coefficient attributes a unique p-value of rich-clubness to a given network. In conclusion, this study proposes the first attempt to quantify, using a unique measure, whether a network presents a significant rich-club topological organization. The general impact of our study on engineering and science is that simulations investigating how the functional performance of a network is changing in relation to rich-clubness might be more easily tuned controlling one unique value: the proposed rich-clubness measure.

physics.soc-ph↗

arXiv2016

Machine learning meets network science: dimensionality reduction for fast and efficient embedding of networks in the hyperbolic space

Complex network topologies and hyperbolic geometry seem specularly connected, and one of the most fascinating and challenging problems of recent complex network theory is to map a given network to its hyperbolic space. The Popularity Similarity Optimization (PSO) model represents - at the moment - the climax of this theory. It suggests that the trade-off between node popularity and similarity is a mechanism to explain how complex network topologies emerge - as discrete samples - from the continuous world of hyperbolic geometry. The hyperbolic space seems appropriate to represent real complex networks. In fact, it preserves many of their fundamental topological properties, and can be exploited for real applications such as, among others, link prediction and community detection. Here, we observe for the first time that a topological-based machine learning class of algorithms - for nonlinear unsupervised dimensionality reduction - can directly approximate the network's node angular coordinates of the hyperbolic model into a two-dimensional space, according to a similar topological organization that we named angular coalescence. On the basis of this phenomenon, we propose a new class of algorithms that offers fast and accurate coalescent embedding of networks in the hyperbolic space even for graphs with thousands of nodes.

cond-mat.dis-nn↗

arXiv2015

Common neighbours and the local-community-paradigm for link prediction in bipartite networks

Bipartite networks are powerful descriptions of complex systems characterized by two different classes of nodes and connections allowed only across but not within the two classes. Surprisingly, current complex network theory presents a theoretical bottle-neck: a general framework for local-based link prediction directly in the bipartite domain is missing. Here, we overcome this theoretical obstacle and present a formal definition of common neighbour index (CN) and local-community-paradigm (LCP) for bipartite networks. As a consequence, we are able to introduce the first node-neighbourhood-based and LCP-based models for topological link prediction that utilize the bipartite domain. We performed link prediction evaluations in several networks of different size and of disparate origin, including technological, social and biological systems. Our models significantly improve topological prediction in many bipartite networks because they exploit local physical driving-forces that participate in the formation and organization of many real-world bipartite networks. Furthermore, we present a local-based formalism that allows to intuitively implement neighbourhood-based link prediction entirely in the bipartite domain.

cs.SI↗