arXiv subjects

Fabrizio Lillo

Publications and source records attributed to Fabrizio Lillo.

At least 55 records · Page 3Linked to original sources

arXiv2021

Modelling time-varying interactions in complex systems: the Score Driven Kinetic Ising Model

A common issue when analyzing real-world complex systems is that the interactions between the elements often change over time: this makes it difficult to find optimal models that describe this evolution and that can be estimated from data, particularly when the driving mechanisms are not known. Here we offer a new perspective on the development of models for time-varying interactions introducing a generalization of the well-known Kinetic Ising Model (KIM), a minimalistic pairwise constant interactions model which has found applications in multiple scientific disciplines. Keeping arbitrary choices of dynamics to a minimum and seeking information theoretical optimality, the Score-Driven methodology lets us significantly increase the knowledge that can be extracted from data using the simple KIM. In particular, we first identify a parameter whose value at a given time can be directly associated with the local predictability of the dynamics. Then we introduce a method to dynamically learn the value of such parameter from the data, without the need of specifying parametrically its dynamics. Finally, we extend our framework to disentangle different sources (e.g. endogenous vs exogenous) of predictability in real time. We apply our methodology to several complex systems including financial markets, temporal (social) networks, and neuronal populations. Our results show that the Score-Driven KIM produces insightful descriptions of the systems, allowing to predict forecasting accuracy in real time as well as to separate different components of the dynamics. This provides a significant methodological improvement for data analysis in a wide range of disciplines.

q-fin.ST↗

arXiv2021

Non-Markovian temporal networks with auto- and cross-correlated link dynamics

Many of the biological, social and man-made networks around us are inherently dynamic, with their links switching on and off over time. The evolution of these networks is often non-Markovian, and the dynamics of their links correlated. Hence, to accurately model these networks, predict their evolution, and understand how information and other quantities propagate over them, the inclusion of both memory and dynamical dependencies between links is key. We here introduce a general class of models of temporal networks based on discrete autoregressive processes. As a case study we concentrate on a specific model within this class, generating temporal networks with a specified underlying backbone, and with precise control over the dynamical dependencies between links and the strength and length of their memories. In this network model the presence of each link is influenced by its own past activity and the past activities of other links, as specified by a coupling matrix, which directly controls the causal relations and correlations among links. We propose a method for estimating the models parameters and how to deal with heterogeneity and time-varying patterns, showing how the model allows for a more realistic description of real world temporal networks and also to predict their evolution. We then investigate the role that memory and correlations in link dynamics have on processes occurring over a temporal network by studying the speed of a spreading process, as measured by the time it takes for diffusion to reach equilibrium. Through both numerical simulations and analytical results, we are able to separate the roles of autocorrelations and neighbourhood correlations in link dynamics, showing that the speed of diffusion is non-monotonically dependent on the memory length, and that correlations among neighbouring links can speed up the spreading process, while autocorrelations slow it down.

cond-mat.stat-mech↗

arXiv2021

Tail Granger causalities and where to find them: extreme risk spillovers vs. spurious linkages

Identifying risk spillovers in financial markets is of great importance for assessing systemic risk and portfolio management. Granger causality in tail (or in risk) tests whether past extreme events of a time series help predicting future extreme events of another time series. The topology and connectedness of networks built with Granger causality in tail can be used to measure systemic risk and to identify risk transmitters. Here we introduce a novel test of Granger causality in tail which adopts the likelihood ratio statistic and is based on the multivariate generalization of a discrete autoregressive process for binary time series describing the sequence of extreme events of the underlying price dynamics. The proposed test has very good size and power in finite samples, especially for large sample size, allows inferring the correct time scale at which the causal interaction takes place, and it is flexible enough for multivariate extension when more than two time series are considered in order to decrease false detections as spurious effect of neglected variables. An extensive simulation study shows the performances of the proposed method with a large variety of data generating processes and it introduces also the comparison with the test of Granger causality in tail by [Hong et al., 2009]. We report both advantages and drawbacks of the different approaches, pointing out some crucial aspects related to the false detections of Granger causality for tail events. An empirical application to high frequency data of a portfolio of US stocks highlights the merits of our novel approach.

q-fin.RM↗

arXiv2021

New centrality and causality metrics assessing air traffic network interactions

In ATM systems, the massive number of interacting entities makes it difficult to identify critical elements and paths of disturbance propagation, as well as to predict the system-wide effects that innovations might have. To this end, suitable metrics are required to assess the role of the interconnections between the elements and complex network science provides several network metrics to evaluate the network functioning. Here we focus on centrality and causality metrics measuring, respectively, the importance of a node and the propagation of disturbances along links. By investigating a dataset of US flights, we show that existing centrality and causality metrics are not suited to characterise the effect of delays in the system. We then propose generalisations of such metrics that we prove suited to ATM applications. Specifically, the new centrality is able to account for the temporal and multi-layer structure of ATM network, while the new causality metric focuses on the propagation of extreme events along the system.

physics.soc-ph↗

arXiv2021

Order flow and price formation

I present an overview of some recent advancements on the empirical analysis and theoretical modeling of the process of price formation in financial markets as the result of the arrival of orders in a limit order book exchange. After discussing critically the possible modeling approaches and the observed stylized facts of order flow, I consider in detail market impact and transaction cost of trades executed incrementally over an extended period of time, by comparing model predictions and recent extensive empirical results. I also discuss how the simultaneous presence of many algorithmic trading executions affects the quality and cost of trading.

q-fin.TR↗

arXiv2021

Analysis of bank leverage via dynamical systems and deep neural networks

We consider a model of a simple financial system consisting of a leveraged investor that invests in a risky asset and manages risk by using Value-at-Risk (VaR). The VaR is estimated by using past data via an adaptive expectation scheme. We show that the leverage dynamics can be described by a dynamical system of slow-fast type associated with a unimodal map on [0,1] with an additive heteroscedastic noise whose variance is related to the portfolio rebalancing frequency to target leverage. In absence of noise the model is purely deterministic and the parameter space splits in two regions: (i) a region with a globally attracting fixed point or a 2-cycle; (ii) a dynamical core region, where the map could exhibit chaotic behavior. Whenever the model is randomly perturbed, we prove the existence of a unique stationary density with bounded variation, the stochastic stability of the process and the almost certain existence and continuity of the Lyapunov exponent for the stationary measure. We then use deep neural networks to estimate map parameters from a short time series. Using this method, we estimate the model in a large dataset of US commercial banks over the period 2001-2014. We find that the parameters of a substantial fraction of banks lie in the dynamical core, and their leverage time series are consistent with a chaotic behavior. We also present evidence that the time series of the leverage of large banks tend to exhibit chaoticity more frequently than those of small banks.

q-fin.MF↗

arXiv2021

On the equivalence between the Kinetic Ising Model and discrete autoregressive processes

Binary random variables are the building blocks used to describe a large variety of systems, from magnetic spins to financial time series and neuron activity. In Statistical Physics the Kinetic Ising Model has been introduced to describe the dynamics of the magnetic moments of a spin lattice, while in time series analysis discrete autoregressive processes have been designed to capture the multivariate dependence structure across binary time series. In this article we provide a rigorous proof of the equivalence between the two models in the range of a unique and invertible map unambiguously linking one model parameters set to the other. Our result finds further justification acknowledging that both models provide maximum entropy distributions of binary time series with given means, auto-correlations, and lagged cross-correlations of order one. We further show that the equivalence between the two models permits to exploit the inference methods originally developed for one model in the inference of the other.

cond-mat.stat-mech↗

arXiv2021

Estimating the Total Volume of Queries to a Search Engine

We study the problem of estimating the total number of searches (volume) of queries in a specific domain, which were submitted to a search engine in a given time period. Our statistical model assumes that the distribution of searches follows a Zipf's law, and that the observed sample volumes are biased accordingly to three possible scenarios. These assumptions are consistent with empirical data, with keyword research practices, and with approximate algorithms used to take counts of query frequencies. A few estimators of the parameters of the distribution are devised and experimented, based on the nature of the empirical/simulated data. For continuous data, we recommend using nonlinear least square regression (NLS) on the top-volume queries, where the bound on the volume is obtained from the well-known Clauset, Shalizi and Newman (CSN) estimation of power-law parameters. For binned data, we propose using a Chi-square minimization approach restricted to the top-volume queries, where the bound is obtained by the binned version of the CSN method. Estimations are then derived for the total number of queries and for the total volume of the population, including statistical error bounds. We apply the methods on the domain of recipes and cooking queries searched in Italian in 2017. The observed volumes of sample queries are collected from Google Trends (continuous data) and SearchVolume (binned data). The estimated total number of queries and total volume are computed for the two cases, and the results are compared and discussed.

cs.IR↗

arXiv2020

Temporal mixture ensemble models for intraday volume forecasting in cryptocurrency exchange markets

We study the problem of the intraday short-term volume forecasting in cryptocurrency exchange markets. The predictions are built by using transaction and order book data from different markets where the exchange takes place. Methodologically, we propose a temporal mixture ensemble, capable of adaptively exploiting, for the forecasting, different sources of data and providing a volume point estimate, as well as its uncertainty. We provide evidence of the outperformance of our model by comparing its outcomes with those obtained with different time series and machine learning methods. Finally, we discuss the predictions conditional to volume and we find that also in this case machine learning methods outperform econometric models.

q-fin.TR↗

arXiv2020

Knowledge and Social Relatedness Shape Research Portfolio Diversification

Scientific discovery is shaped by scientists' choices and thus by their career patterns. The increasing knowledge required to work at the frontier of science makes it harder for an individual to embark on unexplored paths. Yet collaborations can reduce learning costs -- albeit at the expense of increased coordination costs. In this article, we use data on the publication histories of a very large sample of physicists to measure the effects of knowledge and social relatedness on their diversification strategies. Using bipartite networks, we compute a measure of topics similarity and a measure of social proximity. We find that scientists' strategies are not random, and that they are significantly affected by both. Knowledge relatedness across topics explains $\approx 10\%$ of logistic regression deviances and social relatedness as much as $\approx 30\%$, suggesting that science is an eminently social enterprise: when scientists move out of their core specialization, they do so through collaborations. Interestingly, we also find a significant negative interaction between knowledge and social relatedness, suggesting that the farther scientists move from their specialization, the more they rely on collaborations. Our results provide a starting point for broader quantitative analyses of scientific diversification strategies, which could also be extended to the domain of technological innovation -- offering insights from a comparative and policy perspective.

physics.soc-ph↗

arXiv2020

A tale of two sentiment scales: Disentangling short-run and long-run components in multivariate sentiment dynamics

We propose a novel approach to sentiment data filtering for a portfolio of assets. In our framework, a dynamic factor model drives the evolution of the observed sentiment and allows to identify two distinct components: a long-term component, modeled as a random walk, and a short-term component driven by a stationary VAR(1) process. Our model encompasses alternative approaches available in literature and can be readily estimated by means of Kalman filtering and expectation maximization. This feature makes it convenient when the cross-sectional dimension of the portfolio increases. By applying the model to a portfolio of Dow Jones stocks, we find that the long term component co-integrates with the market principal factor, while the short term one captures transient swings of the market associated with the idiosyncratic components and captures the correlation structure of returns. Using quantile regressions, we assess the significance of the contemporaneous and lagged explanatory power of sentiment on returns finding strong statistical evidence when extreme returns, especially negative ones, are considered. Finally, the lagged relation is exploited in a portfolio allocation exercise.

q-fin.GN↗

arXiv2020

Unveiling the relation between herding and liquidity with trader lead-lag networks

We propose a method to infer lead-lag networks of traders from the observation of their trade record as well as to reconstruct their state of supply and demand when they do not trade. The method relies on the Kinetic Ising model to describe how information propagates among traders, assigning a positive or negative "opinion" to all agents about whether the traded asset price will go up or down. This opinion is reflected by their trading behavior, but whenever the trader is not active in a given time window, a missing value will arise. Using a recently developed inference algorithm, we are able to reconstruct a lead-lag network and to estimate the unobserved opinions, giving a clearer picture about the state of supply and demand in the market at all times. We apply our method to a dataset of clients of a major dealer in the Foreign Exchange market at the 5 minutes time scale. We identify leading players in the market and define a herding measure based on the observed and inferred opinions. We show the causal link between herding and liquidity in the inter-dealer market used by dealers to rebalance their inventories.

q-fin.TR↗

arXiv2020

Betweenness centrality for temporal multiplexes

Betweenness centrality quantifies the importance of a vertex for the information flow in a network. We propose a flexible definition of betweenness for temporal multiplexes, where geodesics are determined accounting for the topological and temporal structure and the duration of paths. We propose an algorithm to compute the new metric via a mapping to a static graph. We show the importance of considering the temporal multiplex structure and an appropriate distance metric comparing the results with those obtained with static or single-layer metrics on a dataset of $\sim 20$k European flights.

physics.soc-ph↗

arXiv2019

Clusters of investors around Initial Public Offering

The complex networks approach has been gaining popularity in analysing investor behaviour and stock markets, but within this approach, initial public offerings (IPO) have barely been explored. We fill this gap in the literature by analysing investor clusters in the first two years after the IPO filing in the Helsinki Stock Exchange by using a statistically validated network method to infer investor links based on the co-occurrences of investors' trade timing for 69 IPO stocks. Our findings show that a rather large part of statistically similar network structures form in different securities and persist in time for mature and IPO companies. We also find evidence of institutional herding.

q-fin.TR↗

arXiv2019

Inference of the Kinetic Ising Model with Heterogeneous Missing Data

We consider the problem of inferring a causality structure from multiple binary time series by using the Kinetic Ising Model in datasets where a fraction of observations is missing. We take our steps from a recent work on Mean Field methods for the inference of the model with hidden spins and develop a pseudo-Expectation-Maximization algorithm that is able to work even in conditions of severe data sparsity. The methodology relies on the Martin-Siggia-Rose path integral method with second order saddle-point solution to make it possible to calculate the log-likelihood in polynomial time, giving as output a maximum likelihood estimate of the couplings matrix and of the missing observations. We also propose a recursive version of the algorithm, where at every iteration some missing values are substituted by their maximum likelihood estimate, showing that the method can be used together with sparsification schemes like LASSO regularization or decimation. We test the performance of the algorithm on synthetic data and find interesting properties when it comes to the dependency on heterogeneity of the observation frequency of spins and when some of the hypotheses that are necessary to the saddle-point approximation are violated, such as the small couplings limit and the assumption of statistical independence between couplings.

physics.data-an↗

arXiv2019

Trip Centrality: walking on a temporal multiplex with non-instantaneous link travel time

In complex networks, centrality metrics quantify the connectivity of nodes and identify the most important ones in the transmission of signals. In many real world networks, especially in transportation systems, links are dynamic, i.e. their presence depends on time, and travelling between two nodes requires a non-vanishing time. Additionally, many networks are structured on several layers, representing, e.g., different transportation modes or service providers. Temporal generalisations of centrality metrics based on walk-counting, like Katz centrality, exist, however they do not account for non-zero link travel times and for the multiplex structure. We propose a generalisation of Katz centrality, termed Trip Centrality, counting only the paths that can be travelled according to the network temporal structure, i.e. "trips", while also differentiating the contributions of inter- and intra-layer walks to centrality. We show an application to the US air transport system, specifically computing airports' centrality losses due to delays in the flight network.

physics.soc-ph↗

arXiv2019

A Score-Driven Conditional Correlation Model for Noisy and Asynchronous Data: an Application to High-Frequency Covariance Dynamics

The analysis of the intraday dynamics of correlations among high-frequency returns is challenging due to the presence of asynchronous trading and market microstructure noise. Both effects may lead to significant data reduction and may severely underestimate correlations if traditional methods for low-frequency data are employed. We propose to model intraday log-prices through a multivariate local-level model with score-driven covariance matrices and to treat asynchronicity as a missing value problem. The main advantages of this approach are: (i) all available data are used when filtering correlations, (ii) market microstructure noise is taken into account, (iii) estimation is performed through standard maximum likelihood methods. Our empirical analysis, performed on 1-second NYSE data, shows that opening hours are dominated by idiosyncratic risk and that a market factor progressively emerges in the second part of the day. The method can be used as a nowcasting tool for high-frequency data, allowing to study the real-time response of covariances to macro-news announcements and to build intraday portfolios with very short optimization horizons.

q-fin.TR↗

arXiv2019

Detectability of Macroscopic Structures in Directed Asymmetric Stochastic Block Model

We study the problem of identifying macroscopic structures in networks, characterizing the impact of introducing link directions on the detectability phase transition. To this end, building on the stochastic block model, we construct a class of hardly detectable directed networks. We find closed form solutions by using belief propagation method showing how the transition line depends on the assortativity and the asymmetry of the network. Finally, we numerically identify the existence of a hard phase for detection close to the transition point.

physics.soc-ph↗