SEARCH · Searcharxiv

Results for “stat.OT”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,153 records · Page 64Linked to original sources

arXiv2026

Nonparametric Learning Non-Gaussian Quantum States of Continuous Variable Systems

Continuous-variable quantum systems are foundational to quantum computation, communication, and sensing. While traditional representations using wave functions or density matrices are often impractical, the tomographic picture of quantum mechanics provides an accessible alternative by associating quantum states with classical probability distribution functions called tomograms. Despite its advantages, including compatibility with classical statistical methods, tomographic method remain underutilized due to a lack of robust estimation techniques. This work addresses this gap by introducing a non-parametric \emph{kernel quantum state estimation} (KQSE) framework for reconstructing quantum states and their trace characteristics from noisy data, without prior knowledge of the state. In contrast to existing methods, KQSE yields estimates of the density matrix in various bases, as well as trace quantities such as purity, higher moments, overlap, and trace distance, with a near-optimal convergence rate of $\tilde{O}\bigl(T^{-1}\bigr)$, where $T$ is the total number of measurements. KQSE is robust for multimodal, non-Gaussian states, making it particularly well suited for characterizing states essential for quantum science.

quant-ph↗

arXiv2025

The p-value from a fuzzy point of view

The purpose of the paper is to provide a new way of seeing the p-value in terms of a fuzzy membership function. According to the ASAs statement, we aim at removing the arbitrary choice of the significance level and at demonstrating that the p-value can be profitably interpreted from a fuzzy point of view. In particular, we propose a new class of membership functions by viewing the p-value as a function of the null hypothesis and we apply our approach to compare two independent binomial proportions. The proposed membership functions can also be employed to assess the precision of confidence intervals and the power of statistical tests.

stat.OT↗

arXiv2025

Asymptotic Consistency and Generalization in Hybrid Models of Regularized Selection and Nonlinear Learning

This study explores how different types of supervised models perform in the task of predicting and selecting relevant variables in high-dimensional contexts, especially when the data is very noisy. We analyzed three approaches: regularized models (such as Lasso, Ridge, and Elastic Net), black-box models (such as Random Forest, XGBoost, LightGBM, CatBoost, and H2O GBM), and hybrid models that combine both approaches: regularization with nonlinear algorithms. Based on simulations inspired by the Friedman equation, we evaluated 23 models using three complementary metrics: RMSE, Jaccard index, and recall rate. The results reveal that, although black-box models excel in predictive accuracy, they lack interpretability and simplicity, essential factors in many real-world contexts. Regularized models, on the other hand, proved to be more sensitive to an excess of irrelevant variables. In this scenario, hybrid models stood out for their balance: they maintain good predictive performance, identify relevant variables more consistently, and offer greater robustness, especially as the sample size increases. Therefore, we recommend using this hybrid framework in market applications, where it is essential that the results make sense in a practical context and support decisions with confidence.

stat.OT↗

arXiv2025

A Review and Classification of Model Uncertainty

Model uncertainty is a crucial issue in statistics, econometrics and machine learning, yet its definition remains ambiguous and is subject to various interpretations in the literature. So far, there has not been a universally accepted definition of model uncertainty. We review different understandings of model uncertainty and categorize them into three distinct types: uncertainty about the true model, model selection uncertainty, and model selection instability. We further offer interpretations and examples for a better illustration of these definitions. We also discuss the potential consequences of neglecting model uncertainty in the process of conducting statistical inference, and provide effective solutions to these problems. Our aim is to help researchers better understand the concept of model uncertainty and obtain valid statistical inference results on the premise of its existence.

stat.ME↗

arXiv2026

Exploring the Shape of Economics: A Multilayer Network Analysis of Social Communities and Intellectual Similarity Among Journals Before and After the 2008 Financial Crisis

This paper develops a multilayer network approach for exploring the evolution of scientific disciplines, using the case of economics before and after the 2008 global financial crisis as a large-scale empirical testing ground. The units of analysis are journals, linked by social and intellectual relationships. The analysis covers all journals indexed in EconLit across three years (2006, 2012 and 2019). In the most recent year (2019), the dataset includes 909 journals, over 30,000 editorial board members, more than 260,000 authors, 134,000 articles, and nearly 2 million cited references. For each period, we model journals as connected in a four-layer multiplex network: the social relationships are based on shared editors (interlocking editorship) and shared authors (interlocking authorship), while the intellectual ones are based on shared references (bibliographic coupling) and textual similarity between articles. These four layers are integrated using Similarity Network Fusion to produce unified similarity networks from which journal communities are identified. Comparing the field across the three periods reveals a high degree of structural continuity. Although research topics changed after the crisis, the fundamental social and intellectual relationships among journals remained remarkably stable. A major result of the analysis is that editorial networks play the dominant role in shaping hierarchies and legitimize knowledge production within the discipline. Whether this finding holds in other scientific disciplines remains an open question for future research.

econ.GN↗

arXiv2025

Dynamic Survival Prediction using Longitudinal Images based on Transformer

Survival analysis utilizing multiple longitudinal medical images plays a pivotal role in the early detection and prognosis of diseases by providing insight beyond single-image evaluations. However, current methodologies often inadequately utilize censored data, overlook correlations among longitudinal images measured over multiple time points, and lack interpretability. We introduce SurLonFormer, a novel Transformer-based neural network that integrates longitudinal medical imaging with structured data for survival prediction. Our architecture comprises three key components: a Vision Encoder for extracting spatial features, a Sequence Encoder for aggregating temporal information, and a Survival Encoder based on the Cox proportional hazards model. This framework effectively incorporates censored data, addresses scalability issues, and enhances interpretability through occlusion sensitivity analysis and dynamic survival prediction. Extensive simulations and a real-world application in Alzheimer's disease analysis demonstrate that SurLonFormer achieves superior predictive performance and successfully identifies disease-related imaging biomarkers.

eess.IV↗

arXiv2025

Performances and Correlations of Centrality Measures in Complex Networks

Numerous centrality measures have been proposed to evaluate the importance of nodes in networks, yet comparative analyses of these measures remain limited. Based on 80 real-world networks, we conducted an empirical analysis of 16 representative centrality measures. In general, there exists a moderate to high level of correlation between node rankings derived from different measures. We identified two distinct communities: one comprising 4 measures and the other 7 measures. Measures within the same community exhibit exceptionally strong pairwise correlations. In contrast, the remaining five measures display markedly different behaviors, showing weak correlations not only among themselves but also with the other measures. This suggests that each of these five measures likely captures unique properties of node importance. Further analysis reveals that the distribution patterns of the most influential nodes identified by different centrality measures vary significantly: some measures tend to cluster influential nodes closely together, while others disperse them across distant locations within the network. Using the epidemic spreading model, we found that LocalRank, Subgraph Centrality, and Katz Centrality perform best in identifying the most influential single node, whereas Leverage Centrality, Collective Influence, and Cycle Ratio excel in identifying the most influential node sets. Overall, measures that identify influential nodes with larger topological distances between them tend to perform better in detecting influential node sets. Interestingly, despite being applied to the same dynamical process, when using two seemingly similar tasks, identifying influential nodes versus identifying influential node sets, to rank the performances of the 16 centrality measures, the resulting rankings are negatively correlated.

stat.OT↗

arXiv2025

Examining the Association between Estimated Prevalence and Diagnostic Test Accuracy using Directed Acyclic Graphs

There have been reports of correlation between estimates of prevalence and test accuracy across studies included in diagnostic meta-analyses. It has been hypothesized that this unexpected association arises because of certain biases commonly found in diagnostic accuracy studies. A theoretical explanation has not been studied systematically. In this work, we introduce directed acyclic graphs to illustrate common structures of bias in diagnostic test accuracy studies and to define the resulting data-generating mechanism behind a diagnostic meta-analysis. Using simulation studies, we examine how these common biases can produce a correlation between estimates of prevalence and index test accuracy and what factors influence its magnitude and direction. We found that an association arises either in the absence of a perfect reference test or in the presence of a covariate that simultaneously causes spectrum effect and is associated with the prevalence (confounding). We also show that the association between prevalence and accuracy can be removed by appropriate statistical methods. In the risk of bias evaluation in diagnostic meta-analyses, an observed association between estimates of prevalence and accuracy should be explored to understand its source and to adjust for latent or observed variables if possible.

stat.ME↗

arXiv2025

Relationship Between Leisure Activities, Stress Management Methods, Study Methods, and Methods of Learning New Things Among First-Year Statistics Students

The interplay between leisure activities, stress management methods, studying methods, and methods of learning new things is crucial and affects performance in all aspects of life. On the other hand, data science and statistics are rapidly growing fields with high demands across universities. Thus, this study aimed to identify the similarities and dissimilarities between the four dimensions: leisure activities, stress management methods, studying methods and methods of learning new things. The participants of this study were first-year undergraduates studying statistics at one of the universities in Sri Lanka. There were 117 students in the sample (female-65, male-52). A self-reported questionnaire was used to collect data. First, individual responses for each question under each dimension were visualized using tile maps separately for males and females to identify similarities and dissimilarities in responses. Next, individuals were clustered based on the responses for each dimension separately. Finally, all resulting clusters were re-clustered to identify the relationships between the dimensions. In all cluster analyses, we used Jaccard distance with hierarchical clustering using the complete linkage method. The results were visualized using tile maps. Across all four dimensions we considered, the top activities were either listening to music or lectures and watching videos or TV shows, suggesting that individuals are introverts and passive learners. There was no strong relationship between these dimensions. By identifying these clusters and relationships, educators can tailor instructional approaches to enhance engagement and effectiveness in diverse learning environments.

stat.OT↗

arXiv2025

Revisiting Functional Derivatives in Multi-object Tracking

Probability generating functionals (PGFLs) are efficient and powerful tools for tracking independent objects in clutter. It was shown that PGFLs could be used for the elegant derivation of practical multi-object tracking algorithms, e.g., the probability hypothesis density (PHD) filter. However, derivations using PGFLs use the so-called functional derivatives whose definitions usually appear too complicated or heuristic, involving Dirac delta ``functions''. This paper begins by comparing different definitions of functional derivatives and exploring their relationships and implications for practical applications. It then proposes a rigorous definition of the functional derivative, utilizing straightforward yet precise mathematics for clarity. Key properties of the functional derivative are revealed and discussed.

eess.SY↗

arXiv2026

Understanding Pedagogical Content Knowledge of Introductory Data Science Instructors: An Inaugural Framework

As data science emerges as a distinct academic discipline, introductory data science (IDS) courses play a key role in shaping students foundational understanding. Often taught by instructors without formal training in data science or pedagogy, these courses present a unique and globally relevant context for examining pedagogical content knowledge (PCK). Drawing on semi-structured interviews with 14 IDS instructors and their course syllabi, this study explores how IDS instructors describe and make sense of their teaching practices, which are analyzed through the lens of PCK. The findings highlight key components of PCK about IDS and offer insights into supporting instructor development. This work contributes to expanding the scope of PCK research into new interdisciplinary domains and ongoing global efforts to build capacity in data science education. It could serve as a starting point for developing a PCK framework specific to IDS.

stat.OT↗

arXiv2025

Replicability: Terminology, Measuring Success, and Strategy

Empirical science needs to be based on facts and claims that can be reproduced. This calls for replicating the studies that proclaim the claims, but practice in most fields still fails to implement this idea. When such studies emerged in the past decade, the results were generally disappointing. There have been an overwhelming number of papers addressing the ``reproducibility crisis'' in the last 20 years. Nevertheless, terminology is not yet settled, and there is no consensus about when a replication should be called successful. This paper intends to clarify such issues. A fundamental problem in empirical science is that usual claims only state that effects are non-zero, and such statements are scientifically void. An effect must have a \emph{relevant} size to become a reasonable item of knowledge. Therefore, estimation of an effect, with an indication of precision, forms a substantial scientific task, whereas testing it against zero does not. A relevant effect is one that is shown to exceed a relevance threshold. This paradigm has implications for the judgement on replication success. A further issue is the unavoidable variability between studies, called heterogeneity in meta-analysis. Therefore, it is of little value, again, to test for zero difference between an original effect and its replication, but exceedance of a corresponding relevance threshold should be tested. In order to estimate the degree of heterogeneity, more than one replication is needed, and an appropriate indication of the precision of an estimated effect requires such an estimate. These insights, which are discussed in the paper, show the complexity of obtaining solid scientific results, implying the need for a strategy to make replication happen.

stat.OT↗

arXiv2025

Quantile Function-Based Models for Neuroimaging Classification Using Wasserstein Regression

We propose a novel quantile function-based approach for neuroimaging classification using Wasserstein-Fréchet regression, specifically applied to the detection of mild traumatic brain injury (mTBI) based on the MEG and MRI data. Conventional neuroimaging classification methods for mTBI detection typically extract summary statistics from brain signals across the different epochs, which may result in the loss of important distributional information, such as variance, skewness, kurtosis, etc. Our approach treats complete probability density functions of epoch space results as functional response variables within a Wasserstein-Fréchet regression framework, thereby preserving the full distributional characteristics of epoch results from $L_{1}$ minimum norm solutions. The global Wasserstein-Fréchet regression model incorporating covariates (age and gender) allows us to directly compare the distributional patterns between healthy control subjects and mTBI patients. The classification procedure computes Wasserstein distances between estimated quantile functions from control and patient groups, respectively. These distances are then used as the basis for diagnostic decisions. This framework offers a statistically principled approach to improving diagnostic accuracy in mTBI detection. In practical applications, the test accuracy on unseen data from Innovision IP's dataset achieves up to 98\%.

stat.AP↗

arXiv2025

Grid Transmission Evaluation for Solar Deployment and Data Center Growth

The rapid growth of renewable energy deployment and data center demand in the United States has intensified challenges in grid interconnection, with project delays and escalating costs threatening both economic expansion and energy reliability. This study investigates transmission constraints using the IEEE 39-bus New England Power System model to evaluate the simultaneous interconnection of a 1 GW solar facility and a 1 GW data center load. Employing PSSE and Python-based automation (psspy), we conducted 1,560 load flow simulations across varying siting configurations to assess branch overloads and transmission line limits. Results revealed that only 14 configurations avoided overloads, while most scenarios highlighted recurring congestion on specific network branches, particularly between buses 21 and 22. Optimal siting was identified with the load at bus #35 and the generator at bus #39, yielding minimal overloads (maximum 91.1% loading). Conversely, poor siting decisions resulted in severe congestion with maximum branch loading above 220%. The findings underscore the critical importance of optimized siting and modernized, automated interconnection studies to reduce delays and costs in renewable integration. This research demonstrates the potential of advanced modeling tools to accelerate interconnection processes, improve system reliability, and inform future strategies for balancing renewable energy deployment with rising data center demand.

stat.OT↗

arXiv2025

The Actuary's Final Word on Algorithmic Decision Making

Paul Meehl's foundational work "Clinical versus Statistical Prediction," provided early theoretical justification and empirical evidence of the superiority of statistical methods over clinical judgment. Despite a century of empirical evidence supporting Meehl's central thesis, from early parole prediction studies in the 1920s to modern meta-analyses, confusion persists regarding when and why his troubling finding applies. This paper provides a contemporary theoretical justification for Meehl's result. Importantly, Meehl's prediction problems require a small set of possible outcomes and machine-readable data. Second, individual predictions and decisions are evaluated only on average. This formulation leads to a natural analysis from statistical decision theory, which shows that statistical rules are more accurate than clinical intuition almost by definition. Meehl's prediction paradox is an example of metrical determinism, where the rules of evaluation implicitly determine the best procedure. The decision-theoretic analysis of Meehl's problem elucidates the utility of algorithmic systems as decision-support tools, but also reveals their natural shortcomings, inducing expertise erosion, decision fatigue, and the usurpation of discretionary judgment.

stat.OT↗

arXiv2025

Bridge Modal Identification using Single Moving Sensor under Random Traffic Loading

This paper explores the feasibility of utilizing the response recorded by a single moving sensor to identify the modal parameters of a bridge system under different loading conditions, such as known excitation and unknown random traffic-induced vibrations. The sensor traverses the bridge and captures its dynamic response (acceleration). The natural frequencies and damping ratios are identified using the moving sensor data in the frequency domain. In the case of known inputs, these parameters are then used to obtain the mode shapes, expressed as a linear combination of basic orthonormal polynomials (BOPs), with the coefficients of the BOPs in the linear combinations obtained via optimization. A statistical formulation is proposed to estimate the mode shapes in the case of unknown random traffic-induced vibrations, including the effect of road roughness. It is shown that the absolute value of the mode shapes are proportional to the ensemble standard deviation (SD) of the modal responses. This approach requires the sensor to traverse the bridge multiple times, with the mode shapes identified in both the time domain using variances, and in frequency domain through the evolutionary power spectrum of these responses. The random traffic loading is modeled such that vehicle arrival times follow a Poisson distribution, while the mass and velocity of the vehicles are assumed to follow uniform distributions. To incorporate the effect of road roughness, modeled as a homogeneous random field, a vehicle-bridge-interaction (VBI) model is utilized. Numerical validation under the different loading conditions demonstrates that a single moving sensor can be used to identify the modal parameters quite accurately, with high spatial resolution of the identified mode shapes, offering a cost-effective and efficient alternative for bridge health monitoring.

stat.OT↗

arXiv2025

On the Ambiguities of Incompatibility in Frequentist Inference

The interpretation of the P-value and its monotone transform s=-log2(p), or S-value, remains debated despite decades of dedicated literature. Within the neo-Fisherian framework, these values are often described as indices of (in)compatibility between the observed data and a set of ideal assumptions (i.e., the statistical model). In this regard, this paper proposes the distinction between two domains: the model domain, where assumptions are taken as perfectly true and every admissible outcome is, by construction, fully compatible with the model; and the real domain, where assumptions may fail and face empirical scrutiny. I argue that, although interpreted through an objective numerical index, any level of incompatibility can arise only in the latter domain, where the epistemic status of the model under examination is uncertain and a genuine conflict between data and hypotheses can therefore occur. The extent to which P- and S-values are taken as indicating incompatibility is a matter of contextual judgment. Within this framework, descriptive approaches serve to quantify the numerical values of P and S; these can be interpreted as indicative of a certain degree (or amount) of incompatibility between data and hypotheses once causal knowledge of the data-generating process and information about the costs and benefits of related decisions become clearer. Although the distinction between the model domain and the real domain may appear merely theoretical or even philosophical, I argue that this perspective is useful for developing a clear mental representation of how statistical estimates should be evaluated in practical settings and applications.

stat.OT↗

arXiv2025

Chaotic Bayesian Inference: Strange Attractors as Risk Models for Black Swan Events

We introduce a new risk modeling framework where chaotic attractors shape the geometry of Bayesian inference. By combining heavy-tailed priors with Lorenz and Rossler dynamics, the models naturally generate volatility clustering, fat tails, and extreme events. We compare two complementary approaches: Model A, which emphasizes geometric stability, and Model B, which highlights rare bursts using Fibonacci diagnostics. Together, they provide a dual perspective for systemic risk analysis, linking Black Swan theory to practical tools for stress testing and volatility monitoring.

q-fin.RM↗