Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · physics/0512090

Large dimension forecasting models and random singular value spectra

Jean-Philippe Bouchaud·Laurent Laloux·M. Augusta Miceli·Marc Potters
View original source ↗Read source document ↗Export metadata ↓

Abstract

We present a general method to detect and extract from a finite time sample statistically meaningful correlations between input and output variables of large dimensionality. Our central result is derived from the theory of free random matrices, and gives an explicit expression for the interval where singular values are expected in the absence of any true correlations between the variables under study. Our result can be seen as the natural generalization of the Marcenko-Pastur distribution for the case of rectangular correlation matrices. We illustrate the interest of our method on a set of macroeconomic time series.

Explore related subjects

physics.data-ancond-mat.stat-mechq-fin.ST

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Jean-Philippe Bouchaud, Laurent Laloux, M. Augusta Miceli, Marc Potters. 2005-12-10. Large dimension forecasting models and random singular value spectra. https://arxiv.org/abs/physics/0512090

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

How much data does a memory kernel need? Measurement requirements for fractional models of battery and corrosion degradation

How much data does a memory kernel need before a fit is meaningful? For a distributed-order model, the answer is about 3.1 decades of observation, more than standard battery-degradation tests provide. Battery capacity fade and corrosion-film growth depend on their history, so they are often modelled using general fractional calculus (GFC), where memory is described by an arbitrary Sonine kernel rather than a single power law. We tested this using two mechanistic models without fractional operators and de Levie's porous-electrode model, whose high-frequency order is exactly 0.5; our estimator recovered 0.500. A single aging curve can recover an effective order, alpha, to within 5% noise using a weak formulation, while separating the tempered parameters (alpha, lambda) requires several Arrhenius conditions, with errors improving from (0.25, 0.43) with one condition to (0.04, 0.07) with four. A distributed-order memory needs 3.1 decades of data to be resolved to 90%; below that, a two-order memory can collapse into one intermediate order that fits the data better than the true spectrum. For saturating passivation, however, a general kernel improves the fit substantially, raising R2 from 0.862 to 0.998. Across 20 cells from two public datasets, observation windows span only 1.23 to 2.24 decades, making the 3.1-decade result a practical design rule: it defines what must be measured before fitting a memory kernel, not only what cannot be fitted.

physics.data-an↗
arXiv2026

A Probability Model for Pentagonal Prism Dice Rolls

In the realm of probability theory and game design, the study of dice probabilities plays a crucial role. The featured dice are usually fair, so their probabilities can be predicted without rolling the dice. Pentagonal prismatic dice are not fair. Hence, the probabilities are more complicated. Our mathematical model is naturally derived from both the geometric properties of the dice and empirical evidence. Dice of varying sizes but with constant volume were designed. Using a 3D printer with 100 percent infill, the dice were printed and prepared. A machine that simulates dropping dice through a dice tower onto a hard surface was used to obtain about 41,000 results. From these data, a statistical model was constructed that gives probabilities of dice with varying height-to-radius ratios. The formula for the model is based on the centroid solid angle, but with two adjustment parameters due to varying energy and other requirements to transition between different resting aspects. Within the range of height-to-radius ratios tested, the data are within reasonable statistical error of the model.

physics.data-an↗
arXiv2026

Accelerating fits in high-energy physics and beyond

Data analysis in particle physics often involves optimizing a loss function that quantifies how well a theoretical model describes the observed data, in order to estimate the parameters of the model. This is referred to as fitting. As datasets grow larger and more expensive analysis methods are used, the time required to perform this parameter estimation can be a limiting factor. This work explores techniques for accelerating the fitting process, and measures their performance on a set of models representative of a variety of physics fits. We compare the speed and precision of variants of well-known optimization algorithms, namely Newton's method, BFGS, and the Migrad algorithm (commonly used in particle physics fits). We observe significant speedups when using some algorithms: most notably, some variants of BFGS are about ten times faster than Migrad, while achieving the same precision. For quasi-Newtonian algorithms like BFGS and Migrad, we find that the initial value of the Hessian matrix has a large impact on the runtime. We additionally discuss methods for accelerating the evaluation of the loss function and its derivatives, using algebraic simplifications, automatic differentiation, and modern hardware features. We explore techniques for achieving good precision with 32-bit operations, which can achieve significantly higher throughput on the GPU. Lastly we explore efficient parallelization strategies for the CPU and GPU. All of the techniques and algorithms in this paper are implemented in our new FastFit library, which was used for all of our benchmarks.

physics.data-an↗