Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 0705.2209

Annotated Bibliography of Some Papers on Combining Significances or p-values

Robert D. Cousins
View original source ↗Read source document ↗Export metadata ↓

Abstract

A question that comes up repeatedly is how to combine the results of two experiments if all that is known is that one experiment had a n-sigma effect and another experiment had a m-sigma effect. This question is not well-posed: depending on what additional assumptions are made, the preferred answer is different. The note lists some of the more prominent papers on the topic, with some brief comments and excerpts.

Explore related subjects

physics.data-an

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Robert D. Cousins. 2008-12-20. Annotated Bibliography of Some Papers on Combining Significances or p-values. https://arxiv.org/abs/0705.2209

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Simulation-Based Inference and Unbinned Asimov Construction with Hybrid Neural Density Estimation

High-dimensional, unbinned neural simulation-based inference often relies on neural ratio estimation, which uses expressive supervised models to estimate density ratios, but a learned ratio by itself provides neither an explicit normalized density nor a generative model. Flow-based surrogate models instead enable tractable density evaluation and efficient sampling, but residual density-estimation errors can limit precision for complex implicit distributions. We propose \textit{hybrid neural density estimation}, which uses a flow to define a parameter-independent reference distribution, and classifiers to estimate target-to-reference density ratios. Multiplying a learned ratio by the reference density gives an evaluable target density surrogate. The ratio also provides importance weights for integration and resampling. We show how this representation defines an exact Asimov dataset, whose maximum likelihood fit returns the generating parameters. We also show how the tractable reference supplies renewable samples for pseudo-experiments and how it enables methods to reduce the Monte Carlo variance of expected test statistic calculations. We demonstrate the construction in a toy statistical model motivated by high-energy physics measurements, but for which the exact densities are known analytically.

physics.data-an↗
arXiv2026

Learning to bin: differentiable and Bayesian optimization for multi-dimensional discriminants in high-energy physics

Categorizing events using discriminant observables is central to many high-energy physics analyses. Yet, bin boundaries are often chosen manually. A simple, popular choice in multi-classification tasks is to assign events according to the largest per-class score ("argmax") and to apply equidistant binning to the resulting one-dimensional discriminants. We propose a binning optimization for signal significance directly in multi-dimensional discriminants. We use a Gaussian Mixture Model (GMM) to define flexible regions in the score space, which can be interpreted either as bins or as analysis categories. While this GMM-based strategy is applicable in both one and multiple dimensions, we also study a direct bin-boundary optimization in one dimension as a simpler alternative for binary discriminants. On this binning model, we study two optimization strategies: a differentiable and a Bayesian optimization approach. We study two toy setups: a binary classification and a three-class problem with two signals and backgrounds. In the one-dimensional case, both approaches achieve similar gains in signal sensitivity compared to equidistant binning for a given number of bins, while in the multi-dimensional case the differentiable approach performs best. We show that the GMM-based optimization can outperform argmax classification even after optimized binning is applied to the one-dimensional projections. We further study the performance of our methods on the FAIR Universe $H\rightarrowττ$ dataset, where the GMM-based optimization gives the highest signal significance. Both methods are released as lightweight Python plugins intended for straightforward integration into existing analyses.

physics.data-an↗
arXiv2026

When Should Team KPIs Be Absolute or Relative for Match-Outcome Prediction?

In rugby union and association football, team key performance indicators (KPIs) can be represented in absolute terms or relative to the opponent. Relativisation sometimes improves match-outcome prediction and sometimes harms it. There has been no general account of when each occurs. The answer depends on how much the two teams differ in variability and how strongly their KPI values rise and fall together. We combine these properties into the Paired Efficiency Factor (PEF), which generalises Fisher's paired-efficiency result to the unequal-variance conditions typical of competitive sport. The PEF also connects a KPI's statistical efficiency to how much information its relative form carries about the outcome. Combined KPIs can interact in complex ways, so we analyse each indicator on its own. Across 86 team KPIs from professional rugby union and association football, the PEF places every metric in one of four regimes. Anti-correlation is common, especially for high-volume competitive counts, and absolute measures are then usually preferable. Relativisation can still improve prediction when a noisier difference carries more outcome information. An idealised simulation and one representative KPI from each regime confirm both signs under team-blocked cross-validation. The same pairing geometry appears in healthcare, genomics, finance, and manufacturing. The PEF turns an ad hoc feature-engineering choice into a transparent, data-informed diagnostic for when to relativise performance metrics and when not to.

physics.data-an↗