arXiv subjects

Sebastiano Stramaglia

Publications and source records attributed to Sebastiano Stramaglia.

At least 37 records · Page 2Linked to original sources

arXiv2023

Quantifying High-Order Interdependencies in Entangled Quantum States

Here, we leverage recent advances in information theory to develop a novel method to characterise the dominant character of the high-order dependencies of quantum systems. To this end, we introduce the Q-information: an information-theoretic measure capable of distinguishing quantum states dominated by synergy or redundancy. We illustrate the measure by investigating the properties of paradigmatic entangled Qubit states and find that -- in contrast to classical systems -- quantum systems need at least four variables to exhibit high-order properties. Furthermore, our results reveal that unitary evolution can radically affect the internal information organisation in a way that strongly depends on the corresponding Hamiltonian. Overall, the Q-information sheds light on novel aspects of the internal organisation of quantum systems and their time evolution, opening new avenues for studying several quantum phenomena and related technologies.

quant-ph↗

arXiv2022

Higher-order mutual information reveals synergistic sub-networks for multi-neuron importance

Quantifying which neurons are important with respect to the classification decision of a trained neural network is essential for understanding their inner workings. Previous work primarily attributed importance to individual neurons. In this work, we study which groups of neurons contain synergistic or redundant information using a multivariate mutual information method called the O-information. We observe the first layer is dominated by redundancy suggesting general shared features (i.e. detecting edges) while the last layer is dominated by synergy indicating local class-specific features (i.e. concepts). Finally, we show the O-information can be used for multi-neuron importance. This can be demonstrated by re-training a synergistic sub-network, which results in a minimal change in performance. These results suggest our method can be used for pruning and unsupervised representation learning.

cs.LG↗

arXiv2022

Pairwise and high-order dependencies in the cryptocurrency trading network

In this paper we analyse the effects of information flows in cryptocurrency markets. We first define a cryptocurrency trading network, i.e. the network made using cryptocurrencies as nodes and the Granger causality among their weekly log returns as links, later we analyse its evolution over time. In particular, with reference to years 2020 and 2021, we study the logarithmic US dollar price returns of the cryptocurrency trading network using both pairwise and high-order statistical dependencies, quantified by Granger causality and O-information, respectively. With reference to the former, we find that it shows peaks in correspondence of important events, like e.g., Covid-19 pandemic turbulence or occasional sudden prices rise. The corresponding network structure is rather stable, across weekly time windows in the period considered and the coins are the most influential nodes in the network. In the pairwise description of the network, stable coins seem to play a marginal role whereas, turning high-order dependencies, they appear in the highest number of synergistic information circuits, thus proving that they play a major role for high order effects. With reference to redundancy and synergy with the time evolution of the total transactions in US dollars, we find that their large volume in the first semester of 2021 seems to have triggered a transition in the cryptocurrency network toward a more complex dynamical landscape. Our results show that pairwise and high-order descriptions of complex financial systems provide complementary information for cryptocurrency analysis.

q-fin.TR↗

arXiv2022

Gradients of O-information: low-order descriptors of high-order dependencies

O-information is an information-theoretic metric that captures the overall balance between redundant and synergistic information shared by groups of three or more variables. To complement the global assessment provided by this metric, here we propose the gradients of the O-information as low-order descriptors that can characterise how high-order effects are localised across a system of interest. We illustrate the capabilities of the proposed framework by revealing the role of specific spins in Ising models with frustration, and on practical data analysis on US macroeconomic data. Our theoretical and empirical analyses demonstrate the potential of these gradients to highlight the contribution of variables in forming high-order informational circuits

cs.IT↗

arXiv2022

An information-theoretic approach to hypergraph psychometrics

Psychological network approaches propose to see symptoms or questionnaire items as interconnected nodes, with links between them reflecting pairwise statistical dependencies evaluated cross-sectional, time-series, or panel data. These networks constitute an established methodology to assess the interactions and relative importance of nodes/indicators, providing an important complement to other approaches such as factor analysis. However, focusing the modelling solely on pairwise relationships can neglect potentially critical information shared by groups of three or more variables in the form of higher-order interdependencies. To overcome this important limitation, here we propose an information-theoretic framework based on hypergraphs as psychometric models. As edges in hypergraphs are capable of encompassing several nodes together, this extension can thus provide a richer representation of the interactions that may exist among sets of psychological variables. Our results show how psychometric hypergraphs can highlight meaningful redundant and synergistic interactions on either simulated or state-of-art, re-analyzed psychometric datasets. Overall, our framework extends current network approaches while leading to new ways of assessing the data that differ at their core from other methods, extending the psychometric toolbox and opening promising avenues for future investigation.

stat.ME↗

arXiv2022

Disentangling high-order mechanisms and high-order behaviours in complex systems

Battiston et al. (arXiv:2110.06023) provide a comprehensive overview of how investigations of complex systems should take into account interactions between more than two elements, which can be modelled by hypergraphs and studied via topological data analysis. Following a separate line of enquiry, a broad literature has developed information-theoretic tools to characterize high-order interdependencies from observed data. While these could seem to be competing approaches aiming to address the same question, in this correspondence we clarify that this is not the case, and that a complete account of higher-order phenomena needs to embrace both.

cs.IT↗

arXiv2022

A Framework for the Time- and Frequency-Domain Assessment of High-Order Interactions in Brain and Physiological Networks

While the standard network description of complex systems is based on quantifying links between pairs of system units, higher-order interactions (HOIs) involving three or more units play a major role in governing the collective network behavior. This work introduces an approach to quantify pairwise and HOIs for multivariate rhythmic processes interacting across multiple time scales. We define the so-called O-information rate (OIR) as a new metric to assess HOIs for multivariate time series, and propose a framework to decompose it into measures quantifying Granger-causal and instantaneous influences, as well as to expand it in the frequency domain. The framework exploits the spectral representation of vector autoregressive and state-space models to assess synergistic and redundant interactions among groups of processes, both in specific bands and in the time domain after whole-band integration. Validation on simulated networks illustrates how the spectral OIR can highlight redundant and synergistic HOIs emerging at specific frequencies but not using time-domain measures. The application to physiological networks described by heart period, arterial pressure and respiration measured in healthy subjects during paced breathing, and to brain networks described by ECoG signals acquired in an animal experiment during anesthesia, document the capability of our approach to identify informational circuits relevant to well-defined cardiovascular oscillations and brain rhythms and related to specific physiological mechanisms of autonomic control and altered consciousness. The proposed framework allows a hierarchically-organized evaluation of time- and frequency-domain interactions in networks mapped by multivariate time series, and its high flexibility and scalability make it suitable to investigate networks beyond pairwise interactions in neuroscience, physiology and other fields.

stat.ME↗

arXiv2021

Quantifying high-order interdependencies on individual patterns via the local O-information: theory and applications to music analysis

High-order, beyond-pairwise interdependencies are at the core of biological, economic, and social complex systems, and their adequate analysis is paramount to understand, engineer, and control such systems. This paper presents a framework to measure high-order interdependence that disentangles their effect on each individual pattern exhibited by a multivariate system. The approach is centred on the 'local O-information', a new measure that assesses the balance between synergistic and redundant interdependencies at each pattern. To illustrate the potential of this framework, we present a detailed analysis of music scores from J.S. Bach, which reveals how high-order interdependence is deeply connected with highly non-trivial aspects of the musical discourse. Our results place the local O-information as a promising tool of wide applicability, which opens new perspectives for analysing high-order relationships in the patterns exhibited by complex systems.

physics.data-an↗

arXiv2020

Beyond pairwise network similarity: exploring Mediation and Suppression between networks

Network similarity measures quantify how and when two networks are symmetrically related, including measures of statistical association such as pairwise distance or other correlation measures between networks or between the layers of a multiplex network, but neither can directly unveil whether there are hidden confounding network factors nor can they estimate when such correlation is underpinned by a causal relation. In this work we extend this pairwise conceptual framework to triplets of networks and quantify how and when a network is related to a second network directly or via the indirect mediation or interaction with a third network. Accordingly, we develop a simple and intuitive set-theoretic approach to quantify mediation and suppression between networks. We validate our theory with synthetic models and further apply it to triplets of real-world networks, unveiling mediation and suppression effects which emerge when considering different modes of interaction in online social networks and different routes of information processing in the brain.

cs.SI↗

arXiv2020

Local Granger Causality

Granger causality is a statistical notion of causal influence based on prediction via vector autoregression. For Gaussian variables it is equivalent to transfer entropy, an information-theoretic measure of time-directed information transfer between jointly dependent processes. We exploit such equivalence and calculate exactly the 'local Granger causality', i.e. the profile of the information transfer at each discrete time point in Gaussian processes; in this frame Granger causality is the average of its local version. Our approach offers a robust and computationally fast method to follow the information transfer along the time history of linear stochastic processes, as well as of nonlinear complex systems studied in the Gaussian approximation.

q-bio.QM↗

arXiv2020

Quantifying dynamical high-order interdependencies from the O-information: an application to neural spiking dynamics

We address the problem of efficiently and informatively quantifying how multiplets of variables carry information about the future of the dynamical system they belong to. In particular we want to identify groups of variables carrying redundant or synergistic information, and track how the size and the composition of these multiplets changes as the collective behavior of the system evolves. In order to afford a parsimonious expansion of shared information, and at the same time control for lagged interactions and common effect, we develop a dynamical, conditioned version of the O-information, a framework recently proposed to quantify high-order interdependencies via multivariate extension of the mutual information. We thus obtain an expansion of the transfer entropy in which synergistic and redundant effects are separated. We apply this framework to a dataset of spiking neurons from a monkey performing a perceptual discrimination task. The method identifies synergistic multiplets that include neurons previously categorized as containing little relevant information individually.

q-bio.NC↗

arXiv2020

Synergistic information in a dynamical model implemented on the human structural connectome reveals spatially distinct associations with age

We implement the dynamical Ising model on the large scale architecture of white matter connections of healthy subjects in the age range 4-85 years, and analyze the dynamics in terms of the synergy, a quantity measuring the extent to which the joint state of pairs of variables is projected onto the dynamics of a target one. We find that the amount of synergy in explaining the dynamics of the hubs of the structural connectivity (in terms of degree strength) peaks before the critical temperature, and can thus be considered as a precursor of a critical transition. Conversely the greatest amount of synergy goes into explaining the dynamics of more central nodes. We also find that the aging of the structural connectivity is associated to significant changes in the simulated dynamics: there are brain regions whose synergy decreases with age, in particular the frontal pole, the Subcallosal area and the Supplementary Motor area; these areas could then be more likely to show a decline in terms of the capability to perform higher order computation (if structural connectivity was the sole variable). On the other hand, several regions in the temporal cortex show a positive correlation with age in the first 30 years of life, i.e. during brain maturation.

q-bio.NC↗

arXiv2019

Synergy as a warning sign of transitions: the case of the two-dimensional Ising model

We consider the formalism of information decomposition of target effects from multi-source interactions, i.e. the problem of defining redundant and synergistic components of the information that a set of source variables provides about a target, and apply it to the two-dimensional Ising model as a paradigm of a critically transitioning system. Intuitively, synergy is the information about the target variable that is uniquely obtained taking the sources together, but not considering them alone; redundancy is the information which is shared by the sources. To disentangle the components of the information both at the static level and at the dynamical one, the decomposition is applied respectively to the mutual information and to the transfer entropy between a given spin, the target, and a pair of neighbouring spins (taken as the drivers). We show that a key signature of an impending phase transition (approached from the disordered size) is the fact that the synergy peaks in the disordered phase, both in the static and in the dynamic case: the synergy can thus be considered a precursor of the transition. The redundancy, instead, reaches its maximum at the critical temperature. The peak of the synergy of the transfer entropy is far more pronounced than those of the static mutual information. We show that these results are robust w.r.t. the details of the information decomposition approach, as we find the same results using two different methods; moreover, w.r.t. previous literature rooted on the notion of Global Transfer Entropy, our results demonstrate that considering as few as three variables is sufficient to construct a precursor of the transition, and provide a paradigm for the investigation of a variety of systems prone to crisis, like financial markets, social media, or epileptic seizures.

cond-mat.stat-mech↗

arXiv2017

Multiscale Granger causality

In the study of complex physical and biological systems represented by multivariate stochastic processes, an issue of great relevance is the description of the system dynamics spanning multiple temporal scales. While methods to assess the dynamic complexity of individual processes at different time scales are well-established, multiscale analysis of directed interactions has never been formalized theoretically, and empirical evaluations are complicated by practical issues such as filtering and downsampling. Here we extend the very popular measure of Granger causality (GC), a prominent tool for assessing directed lagged interactions between joint processes, to quantify information transfer across multiple time scales. We show that the multiscale processing of a vector autoregressive (AR) process introduces a moving average (MA) component, and describe how to represent the resulting ARMA process using state space (SS) models and to combine the SS model parameters for computing exact GC values at arbitrarily large time scales. We exploit the theoretical formulation to identify peculiar features of multiscale GC in basic AR processes, and demonstrate with numerical simulations the much larger estimation accuracy of the SS approach compared with pure AR modeling of filtered and downsampled data. The improved computational reliability is exploited to disclose meaningful multiscale patterns of information transfer between global temperature and carbon dioxide concentration time series, both in paleoclimate and in recent years.

stat.ME↗

arXiv2017

On the interpretability and computational reliability of frequency-domain Granger causality

This is a comment to the paper 'A study of problems encountered in Granger causality analysis from a neuroscience perspective'. We agree that interpretation issues of Granger Causality in Neuroscience exist (partially due to the historical unfortunate use of the name 'causality', as nicely described in previous literature). On the other hand we think that the paper uses a formulation of Granger causality which is outdated (albeit still used), and in doing so it dismisses the measure based on a suboptimal use of it. Furthermore, since data from simulated systems are used, the pitfalls that are found with the used formulation are intended to be general, and not limited to neuroscience. It would be a pity if this paper, even written in good faith, became a wildcard against all possible applications of Granger Causality, regardless of the hard work of colleagues aiming to seriously address the methodological and interpretation pitfalls. In order to provide a balanced view, we replicated their simulations used the updated State Space implementation, proposed already some years ago, in which the pitfalls are mitigated or directly solved.

stat.ME↗

arXiv2017

Multiscale Information Decomposition: Exact Computation for Multivariate Gaussian Processes

Exploiting the theory of state space models, we derive the exact expressions of the information transfer, as well as redundant and synergistic transfer, for coupled Gaussian processes observed at multiple temporal scales. All of the terms, constituting the frameworks known as interaction information decomposition and partial information decomposition, can thus be analytically obtained for different time scales from the parameters of the VAR model that fits the processes. We report the application of the proposed methodology firstly to benchmark Gaussian systems, showing that this class of systems may generate patterns of information decomposition characterized by mainly redundant or synergistic information transfer persisting across multiple time scales or even by the alternating prevalence of redundant and synergistic source interaction depending on the time scale. Then, we apply our method to an important topic in neuroscience, i.e., the detection of causal interactions in human epilepsy networks, for which we show the relevance of partial information decomposition to the detection of multiscale information transfer spreading from the seizure onset zone.

stat.ME↗

arXiv2017

Multiscale Granger causality analysis by à trous wavelet transform

Since interactions in neural systems occur across multiple temporal scales, it is likely that information flow will exhibit a multiscale structure, thus requiring a multiscale generalization of classical temporal precedence causality analysis like Granger's approach. However, the computation of multiscale measures of information dynamics is complicated by theoretical and practical issues such as filtering and undersampling: to overcome these problems, we propose a wavelet-based approach for multiscale Granger causality (GC) analysis, which is characterized by the following properties: (i) only the candidate driver variable is wavelet transformed (ii) the decomposition is performed using the à trous wavelet transform with cubic B-spline filter. We measure GC, at a given scale, by including the wavelet coefficients of the driver times series, at that scale, in the regression model of the target. To validate our method, we apply it to publicly available scalp EEG signals, and we find that the condition of closed eyes, at rest, is characterized by an enhanced GC among channels at slow scales w.r.t. eye open condition, whilst the standard Granger causality is not significantly different in the two conditions.

q-bio.QM↗

arXiv2017

Consensus clustering approach to group brain connectivity matrices

A novel approach rooted on the notion of consensus clustering, a strategy developed for community detection in complex networks, is proposed to cope with the heterogeneity that characterizes connectivity matrices in health and disease. The method can be summarized as follows: (i) define, for each node, a distance matrix for the set of subjects by comparing the connectivity pattern of that node in all pairs of subjects; (ii) cluster the distance matrix for each node; (iii) build the consensus network from the corresponding partitions; (iv) extract groups of subjects by finding the communities of the consensus network thus obtained. Differently from the previous implementations of consensus clustering, we thus propose to use the consensus strategy to combine the information arising from the connectivity patterns of each node. The proposed approach may be seen either as an exploratory technique or as an unsupervised pre-training step to help the subsequent construction of a supervised classifier. Applications on a toy model and two real data sets, show the effectiveness of the proposed methodology, which represents heterogeneity of a set of subjects in terms of a weighted network, the consensus matrix.

q-bio.NC↗