arXiv subjects

Didier Sornette

Publications and source records attributed to Didier Sornette.

At least 127 records · Page 7Linked to original sources

arXiv2015

Cancer risk is not (just) bad luck

Tomasetti and Vogelstein recently proposed that the majority of variation in cancer risk among tissues is due to "bad luck," that is, random mutations arising during DNA replication in normal noncancerous stem cells. They generalize this finding to cancer overall, claiming that "the stochastic effects of DNA replication appear to be the major contributor to cancer in humans." We show that this conclusion results from a logical fallacy based on ignoring the influence of population heterogeneity in correlations exhibited at the level of the whole population. Because environmental and genetic factors cannot explain the huge differences in cancer rates between different organs, it is wrong to conclude that these factors play a minor role in cancer rates. In contrast, we show that one can indeed measure huge differences in cancer rates between different organs and, at the same time, observe a strong effect of environmental and genetic factors in cancer rates.

q-bio.QM↗

arXiv2015

A generic model of dyadic social relationships

We introduce a model of dyadic social interactions and establish its correspondence with relational models theory (RMT), a theory of human social relationships. RMT posits four elementary models of relationships governing human interactions, singly or in combination: Communal Sharing, Authority Ranking, Equality Matching, and Market Pricing. To these are added the limiting cases of asocial and null interactions, whereby people do not coordinate with reference to any shared principle. Our model is rooted in the observation that each individual in a dyadic interaction can do either the same thing as the other individual, a different thing or nothing at all. To represent these three possibilities, we consider two individuals that can each act in one out of three ways toward the other: perform a social action X or Y, or alternatively do nothing. We demonstrate that the relationships generated by this model aggregate into six exhaustive and disjoint categories. We propose that four of these categories match the four relational models, while the remaining two correspond to the asocial and null interactions defined in RMT. We generalize our results to the presence of N social actions. We infer that the four relational models form an exhaustive set of all possible dyadic relationships based on social coordination. Hence, we contribute to RMT by offering an answer to the question of why there could exist just four relational models. In addition, we discuss how to use our representation to analyze data sets of dyadic social interactions, and how social actions may be valued and matched by the agents.

physics.soc-ph↗

arXiv2015

Of Disasters and Dragon Kings: A Statistical Analysis of Nuclear Power Incidents & Accidents

We provide, and perform a risk theoretic statistical analysis of, a dataset that is 75 percent larger than the previous best dataset on nuclear incidents and accidents, comparing three measures of severity: INES (International Nuclear Event Scale), radiation released, and damage dollar losses. The annual rate of nuclear accidents, with size above 20 Million US$, per plant, decreased from the 1950s until dropping significantly after Chernobyl (April, 1986). The rate is now roughly stable at 0.002 to 0.003, i.e., around 1 event per year across the current fleet. The distribution of damage values changed after Three Mile Island (TMI; March, 1979), where moderate damages were suppressed but the tail became very heavy, being described by a Pareto distribution with tail index 0.55. Further, there is a runaway disaster regime, associated with the "dragon-king" phenomenon, amplifying the risk of extreme damage. In fact, the damage of the largest event (Fukushima; March, 2011) is equal to 60 percent of the total damage of all 174 accidents in our database since 1946. In dollar losses we compute a 50% chance that (i) a Fukushima event (or larger) occurs in the next 50 years, (ii) a Chernobyl event (or larger) occurs in the next 27 years and (iii) a TMI event (or larger) occurs in the next 10 years. Finally, we find that the INES scale is inconsistent. To be consistent with damage, the Fukushima disaster would need to have an INES level of 11, rather than the maximum of 7.

physics.soc-ph↗

arXiv2015

Power law scaling and "Dragon-Kings" in distributions of intraday financial drawdowns

We investigate the distributions of epsilon-drawdowns and epsilon-drawups of the most liquid futures financial contracts of the world at time scales of 30 seconds. The epsilon-drawdowns (resp. epsilon- drawups) generalise the notion of runs of negative (resp. positive) returns so as to capture the risks to which investors are arguably the most concerned with. Similarly to the distribution of returns, we find that the distributions of epsilon-drawdowns and epsilon-drawups exhibit power law tails, albeit with exponents significantly larger than those for the return distributions. This paradoxical result can be attributed to (i) the existence of significant transient dependence between returns and (ii) the presence of large outliers (dragon-kings) characterizing the extreme tail of the drawdown/drawup distributions deviating from the power law. The study of the tail dependence between the sizes, speeds and durations of drawdown/drawup indicates a clear relationship between size and speed but none between size and duration. This implies that the most extreme drawdown/drawup tend to occur fast and are dominated by a few very large returns. We discuss both the endogenous and exogenous origins of these extreme events.

q-fin.ST↗

arXiv2014

Estimation of the Hawkes Process With Renewal Immigration Using the EM Algorithm

We introduce the Hawkes process with renewal immigration and make its statistical estimation possible with two Expectation Maximization (EM) algorithms. The standard Hawkes process introduces immigrant points via a Poisson process, and each immigrant has a subsequent cluster of associated offspring of multiple generations. We generalize the immigration to come from a Renewal process; introducing dependence between neighbouring clusters, and allowing for over/under dispersion in cluster locations. This complicates evaluation of the likelihood since one needs to know which subset of the observed points are immigrants. Two EM algorithms enable estimation here: The first is an extension of an existing algorithm that treats the entire branching structure - which points are immigrants, and which point is the parent of each offspring - as missing data. The second considers only if a point is an immigrant or not as missing data and can be implemented with linear time complexity. Both algorithms are found to be consistent in simulation studies. Further, we show that misspecifying the immigration process introduces signficant bias into model estimation-- especially the branching ratio, which quantifies the strength of self excitation. Thus, this extended model provides a valuable alternative model in practice.

stat.AP↗

arXiv2014

Forecasting future oil production in Norway and the UK: a general improved methodology

We present a new Monte-Carlo methodology to forecast the crude oil production of Norway and the U.K. based on a two-step process, (i) the nonlinear extrapolation of the current/past performances of individual oil fields and (ii) a stochastic model of the frequency of future oil field discoveries. Compared with the standard methodology that tends to underestimate remaining oil reserves, our method gives a better description of future oil production, as validated by our back-tests starting in 2008. Specifically, we predict remaining reserves extractable until 2030 to be 188 +/- 10 million barrels for Norway and 98 +/- 10 million barrels for the UK, which are respectively 45% and 66% above the predictions using the standard methodology.

physics.data-an↗

arXiv2014

Apparent criticality and calibration issues in the Hawkes self-excited point process model: application to high-frequency financial data

We present a careful analysis of possible issues on the application of the self-excited Hawkes process to high-frequency financial data. We carefully analyze a set of effects leading to significant biases in the estimation of the "criticality index" n that quantifies the degree of endogeneity of how much past events trigger future events. We report a number of model biases that are intrinsic to the estimation of brnaching ratio (n) when using power law memory kernels. We demonstrate that the calibration of the Hawkes process on mixtures of pure Poisson process with changes of regime leads to completely spurious apparent critical values for the branching ratio (n~1) while the true value is actually n=0. More generally, regime shifts on the parameters of the Hawkes model and/or on the generating process itself are shown to systematically lead to a significant upward bias in the estimation of the branching ratio. We also demonstrate the importance of the preparation of the high-frequency financial data and give special care to the decrease of quality of the timestamps of tick data due to latency and grouping of messages to packets by the stock exchange. Altogether, our careful exploration of the caveats of the calibration of the Hawkes process stresses the need for considering all the above issues before any conclusion can be sustained. In this respect, because the above effects are plaguing their analyses, the claim by Hardiman, Bercot and Bouchaud (2013) that financial market have been continuously functioning at or close to criticality (n~1) cannot be supported. In contrast, our previous results on E-mini S&P 500 Futures Contracts and on major commodity future contracts are upheld.

q-fin.ST↗

arXiv2014

Using Prediction Markets to Incentivize and Measure Collective Knowledge Production

We present a mechanism design, coupling an online collaboration software and a prediction market, which allows tracking down the very roots of individual incentives, actions and how these behaviors influence collective intelligence in terms of knowledge production as a public good. We show that the incentive mechanism efficiently engages users without further governance structure, and doesn't crowd out intrinsic motivation. Furthermore, it enables a powerful and robust creative destruction process, which helps quickly filter out irrelevant knowledge. While still at an early stage, this mechanism design can not only bring insights for knowledge production organization design, but also has the potential to illuminate the fundamental mechanisms underlying the emergence of collective intelligence.

cs.GT↗

arXiv2014

Effective Measure of Endogeneity for the Autoregressive Conditional Duration Point Processes via Mapping to the Self-Excited Hawkes Process

In order to disentangle the internal dynamics from exogenous factors within the Autoregressive Conditional Duration (ACD) model, we present an effective measure of endogeneity. Inspired from the Hawkes model, this measure is defined as the average fraction of events that are triggered due to internal feedback mechanisms within the total population. We provide a direct comparison of the Hawkes and ACD models based on numerical simulations and show that our effective measure of endogeneity for the ACD can be mapped onto the "branching ratio" of the Hawkes model.

q-fin.ST↗

arXiv2014

How Much is the Whole Really More than the Sum of its Parts? 1 + 1 = 2.5: Superlinear Productivity in Collective Group Actions

In a variety of open source software projects, we document a superlinear growth of production ($R \sim c^β$) as a function of the number of active developers $c$, with $β\simeq 4/3$ with large dispersions. For a typical project in this class, doubling of the group size multiplies typically the output by a factor $2^β=2.5$, explaining the title. This superlinear law is found to hold for group sizes ranging from 5 to a few hundred developers. We propose two classes of mechanisms, {\it interaction-based} and {\it large deviation}, along with a cascade model of productive activity, which unifies them. In this common framework, superlinear productivity requires that the involved social groups function at or close to criticality, in the sense of a subtle balance between order and disorder. We report the first empirical test of the renormalization of the exponent of the distribution of the sizes of first generation events into the renormalized exponent of the distribution of clusters resulting from the cascade of triggering over all generation in a critical branching process in the non-meanfield regime. Finally, we document a size effect in the strength and variability of the superlinear effect, with smaller groups exhibiting widely distributed superlinear exponents, some of them characterizing highly productive teams. In contrast, large groups tend to have a smaller superlinearity and less variability.

physics.soc-ph↗

arXiv2014

Financial bubbles: mechanisms and diagnostics

We define a financial bubble as a period of unsustainable growth, when the price of an asset increases ever more quickly, in a series of accelerating phases of corrections and rebounds. More technically, during a bubble phase, the price follows a faster-than-exponential power law growth process, often accompanied by log-periodic oscillations. This dynamic ends abruptly in a change of regime that may be a crash or a substantial correction. Because they leave such specific traces, bubbles may be recognised in advance, that is, before they burst. In this paper, we will explain the mechanism behind financial bubbles in an intuitive way. We will show how the log-periodic power law emerges spontaneously from the complex system that financial markets are, as a consequence of feedback mechanisms, hierarchical structure and specific trading dynamics and investment styles. We argue that the risk of a major correction, or even a crash, becomes substantial when a bubble develops towards maturity, and that it is therefore very important to find evidence of bubbles and to follow their development from as early a stage as possible. The tools that are explained in this paper actually serve that purpose. They are at the core of the Financial Crisis Observatory at the ETH Zurich, where tens of thousands of assets are monitored on a daily basis. This allow us to have a continuous overview of emerging bubbles in the global financial markets. The companion report available as part of the Notenstein white paper series (2014) with the title ``Financial bubbles: mechanism, diagnostic and state of the World (Feb. 2014)'' presents a practical application of the methodology outlines in this article and describes our view of the status concerning positive and negative bubbles in the financial markets, as of the end of January 2014.

q-fin.RM↗

arXiv2014

Micro-transition cascades to percolation

We report the discovery of a discrete hierarchy of micro-transitions occurring in models of continuous and discontinuous percolation. The precursory micro-transitions allow us to target almost deterministically the location of the transition point to global connectivity. This extends to the class of intrinsically stochastic processes the possibility to use warning signals anticipating phase transitions in complex systems.

cond-mat.dis-nn↗

arXiv2014

Fractal multi-level organisation of human groups in a virtual world

Humans are fundamentally social. They have progressively dominated their environment by the strength and creativity provided by and within their grouping. It is well recognised that human groups are highly structured, and the anthropological literature has loosely classified them according to their size and function, such as support cliques, sympathy groups, bands, cognitive groups, tribes, linguistic groups and so on. Recently, combining data on human grouping patterns in a comprehensive and systematic study, Zhou et al. identified a quantitative discrete hierarchy of group sizes with a preferred scaling ratio close to $3$, which was later confirmed for hunter-gatherer groups and for other mammalian societies. Using high precision large scale Internet-based social network data, we extend these early findings on a very large data set. We analyse the organisational structure of a complete, multi-relational, large social multiplex network of a human society consisting of about 400,000 odd players of a massive multiplayer online game for which we know all about the group memberships of every player. Remarkably, the online players exhibit the same type of structured hierarchical layers as the societies studied by anthropologists, where each of these layers is three to four times the size of the lower layer. Our findings suggest that the hierarchical organisation of human society is deeply nested in human psychology.

physics.soc-ph↗

arXiv2014

Financial Brownian particle in the layered order book fluid and Fluctuation-Dissipation relations

We introduce a novel description of the dynamics of the order book of financial markets as that of an effective colloidal Brownian particle embedded in fluid particles. The analysis of a comprehensive market data enables us to identify all motions of the fluid particles. Correlations between the motions of the Brownian particle and its surrounding fluid particles reflect specific layering interactions; in the inner-layer, the correlation is strong and with short memory while, in the outer-layer, it is weaker and with long memory. By interpreting and estimating the contribution from the outer-layer as a drag resistance, we demonstrate the validity of the fluctuation-dissipation relation (FDR) in this non-material Brownian motion process.

q-fin.TR↗

arXiv2014

A Creepy World

Using the mechanics of creep in material sciences as a metaphor, we present a general framework to understand the evolution of financial, economic and social systems and to construct scenarios for the future. In a nutshell, highly non-linear out-of-equilibrium systems subjected to exogenous perturbations tend to exhibit a long phase of slow apparent stable evolution, which are nothing but slow maturations towards instabilities, failures and changes of regimes. With examples from history where a small event had a cataclysmic consequence, we propose a novel view of the current state of the world via the logical scenarios that derive, avoiding the traps of an illusionary stability and simple linear extrapolation. The endogenous scenarios are "muddling along", "managing through" and "blood red abyss". The exogenous scenarios are "painful adjustment" and "golden east".

physics.soc-ph↗

arXiv2013

Predictability and suppression of extreme events in complex systems

In many complex systems, large events are believed to follow power-law, scale-free probability distributions, so that the extreme, catastrophic events are unpredictable. Here, we study coupled chaotic oscillators that display extreme events. The mechanism responsible for the rare, largest events makes them distinct and their distribution deviates from a power-law. Based on this mechanism identification, we show that it is possible to forecast in real time an impending extreme event. Once forecasted, we also show that extreme events can be suppressed by applying tiny perturbations to the system.

nlin.CD↗

arXiv2013

A Stable and Robust Calibration Scheme of the Log-Periodic Power Law Model

We present a simple transformation of the formulation of the log-periodic power law formula of the Johansen-Ledoit-Sornette model of financial bubbles that reduces it to a function of only three nonlinear parameters. The transformation significantly decreases the complexity of the fitting procedure and improves its stability tremendously because the modified cost function is now characterized by good smooth properties with in general a single minimum in the case where the model is appropriate to the empirical data. We complement the approach with an additional subordination procedure that slaves two of the nonlinear parameters to what can be considered to be the most crucial nonlinear parameter, the critical time $t_c$ defined as the end of the bubble and the most probably time for a crash to occur. This further decreases the complexity of the search and provides an intuitive representation of the results of the calibration. With our proposed methodology, metaheuristic searches are not longer necessary and one can resort solely to rigorous controlled local search algorithms, leading to dramatic increase in efficiency. Empirical tests on the Shanghai Composite index (SSE) from January 2007 to March 2008 illustrate our findings.

q-fin.GN↗

arXiv2013

Clarifications to Questions and Criticisms on the Johansen-Ledoit-Sornette Bubble Model

The Johansen-Ledoit-Sornette (JLS) model of rational expectation bubbles with finite-time singular crash hazard rates has been developed to describe the dynamics of financial bubbles and crashes. It has been applied successfully to a large variety of financial bubbles in many different markets. Having been developed for more than one decade, the JLS model has been studied, analyzed, used and criticized by several researchers. Much of this discussion is helpful for advancing the research. However, several serious misconceptions seem to be present within this collective conversation both on theoretical and empirical aspects. Several of these problems appear to stem from the fast evolution of the literature on the JLS model and related works. In the hope of removing possible misunderstanding and of catalyzing useful future developments, we summarize these common questions and criticisms concerning the JLS model and offer a synthesis of the existing state-of-the-art and best-practice advices.

q-fin.GN↗