Searcharxiv⌕ Search

SEARCH · Searcharxiv

Results for “stat.OT”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,027 records · Page 57Linked to original sources

Cyclicity Analysis of the Ornstein-Uhlenbeck Process

In this thesis, we consider an $N$-dimensional Ornstein-Uhlenbeck (OU) process satisfying the linear stochastic differential equation $d\mathbf x(t) = - \mathbf B\mathbf x(t) dt + \boldsymbol Σd \mathbf w(t).$ Here, $\mathbf B$ is a fixed $N \times N$ circulant friction matrix whose eigenvalues have positive real parts, $\boldsymbol Σ$ is a fixed $N \times M$ matrix. We consider a signal propagation model governed by this OU process. In this model, an underlying signal propagates throughout a network consisting of $N$ linked sensors located in space. We interpret the $n$-th component of the OU process as the measurement of the propagating effect made by the $n$-th sensor. The matrix $\mathbf B$ represents the sensor network structure: if $\mathbf B$ has first row $(b_1 \ , \ \dots \ , \ b_N),$ where $b_1>0$ and $b_2 \ , \ \dots \ ,\ b_N \le 0,$ then the magnitude of $b_p$ quantifies how receptive the $n$-th sensor is to activity within the $(n+p-1)$-th sensor. Finally, the $(m,n)$-th entry of the matrix $\mathbf D = \frac{\boldsymbol Σ\boldsymbol Σ^\text T}{2}$ is the covariance of the component noises injected into the $m$-th and $n$-th sensors. For different choices of $\mathbf B$ and $\boldsymbol Σ,$ we investigate whether Cyclicity Analysis enables us to recover the structure of network. Roughly speaking, Cyclicity Analysis studies the lead-lag dynamics pertaining to the components of a multivariate signal. We specifically consider an $N \times N$ skew-symmetric matrix $\mathbf Q,$ known as the lead matrix, in which the sign of its $(m,n)$-th entry captures the lead-lag relationship between the $m$-th and $n$-th component OU processes. We investigate whether the structure of the leading eigenvector of $\mathbf Q,$ the eigenvector corresponding to the largest eigenvalue of $\mathbf Q$ in modulus, reflects the network structure induced by $\mathbf B.$

math.ST↗

Heckman Selection Contaminated Normal Model

The Heckman selection model is one of the most well-renounced econometric models in the analysis of data with sample selection. This model is designed to rectify sample selection biases based on the assumption of bivariate normal error terms. However, real data diverge from this assumption in the presence of heavy tails and/or atypical observations. Recently, this assumption has been relaxed via a more flexible Student's t-distribution, which has appealing statistical properties. This paper introduces a novel Heckman selection model using a bivariate contaminated normal distribution for the error terms. We present an efficient ECM algorithm for parameter estimation with closed-form expressions at the E-step based on truncated multinormal distribution formulas. The identifiability of the proposed model is also discussed, and its properties have been examined. Through simulation studies, we compare our proposed model with the normal and Student's t counterparts and investigate the finite-sample properties and the variation in missing rate. Results obtained from two real data analyses showcase the usefulness and effectiveness of our model. The proposed algorithms are implemented in the R package HeckmanEM.

stat.ME↗

Advancing Employee Behavior Analysis through Synthetic Data: Leveraging ABMs, GANs, and Statistical Models for Enhanced Organizational Efficiency

Success in todays data-driven corporate climate requires a deep understanding of employee behavior. Companies aim to improve employee satisfaction, boost output, and optimize workflow. This research study delves into creating synthetic data, a powerful tool that allows us to comprehensively understand employee performance, flexibility, cooperation, and team dynamics. Synthetic data provides a detailed and accurate picture of employee activities while protecting individual privacy thanks to cutting-edge methods like agent-based models (ABMs), Generative Adversarial Networks (GANs), and statistical models. Through the creation of multiple situations, this method offers insightful viewpoints regarding increasing teamwork, improving adaptability, and accelerating overall productivity. We examine how synthetic data has evolved from a specialized field to an essential resource for researching employee behavior and enhancing management efficiency. Keywords: Agent-Based Model, Generative Adversarial Network, workflow optimization, organizational success

cs.LG↗

Survey Data Integration for Distribution Function Estimation

Estimates of finite population cumulativedistribution functions (CDFs) and quantiles are critical forpolicy-making, resource allocation, and public health planning. For instance, federal finance agencies may require accurate estimates of the proportion of individuals with income below the federal poverty line to determine funding eligibility, while health organizations may rely on precise quantile estimates of key health variables to guide local health interventions. Despite growing interest in survey data integration, research on the integration of probability and nonprobability samples toestimate CDFs and quantiles remains limited. In this study, we propose a novel residual-based CDF estimator that integrates information from a probability sample with data from potentially large nonprobability samples. Our approach leverages shared covariates observed in both datasets, while the response variable is available only in the nonprobability sample. Using a semiparametric approach, we train an outcome model on the nonprobability sample and incorporate model residuals with sampling weights from the probability sample to estimate the CDF of the target variable. Based on this CDF estimator, we define a quantile estimator and introduce linearization and bootstrap methods for variance estimation of both the CDF and quantile estimators. Under certain regularity conditions, we establish the asymptotic properties, including bias and variance, of the CDF estimator. Our empirical findings support the theoretical results and demonstrate the favorable performance of the proposed estimators relative to plug-in mass imputation estimators and the naïve estimators derived from the nonprobability sample only. A real data example is presented to illustrate the proposed estimators.

math.ST↗

A Modified Satterthwaite (1941,1946) Effective Degrees of Freedom Approximation

This study introduces a correction to the approximation of effective degrees of freedom as proposed by Satterthwaite (1941, 1946), specifically addressing scenarios where component degrees of freedom are small. The correction is grounded in analytical results concerning the moments of standard normal random variables. This modification is applicable to complex variance estimates that involve both small and large degrees of freedom, offering an enhanced approximation of the higher moments required by Satterthwaite's framework. Additionally, this correction extends and partially validates the empirically derived adjustment by Johnson & Rust (1992), as it is based on theoretical foundations rather than simulations used to derive empirical transformation constants.

stat.OT↗

Approximation of Smooth Numbers for Harmonic Samples A Stein method Approach

We present a de Bruijn type approximation for quantifying the content of m smooth numbers, derived from samples obtained through a probability measure over the set of integers less than or equal to n, with point mass function at k inversely proportional to k. Our analysis is based on a stochastic representation of the measure of interest, utilizing weighted independent geometric random variables. This representation is analyzed through the lens of Stein method for the Dickman distribution. A pivotal element of our arguments relies on precise estimations concerning the regularity properties of the solution to the Dickman Stein equation for heaviside functions, recently developed by Bhattacharjee and Schulte. Remarkably, our arguments remain mostly in the realm of probability theory, with Mertens first and third theorems standing as the only number theory estimations required.

math.PR↗

Oral exams in introductory statistics class with non-native English speakers

Oral exams are a powerful tool to assess student's learning. This is particularly important in introductory statistics classes where students struggle to grasp various topics like the interpretation of probability, $p$-values and more. The challenge of acquiring conceptual understanding is only heightened when students are learning in a second language. In this paper, I share my experience administering oral exams to an introductory statistics class of non-native English speakers at a Japanese university. I explain the context of the university and course, before detailing the exam. Of particular interest is the relationship between exam performance and English proficiency. The results showed little relationship between the two, meaning the exam seemed to truly test student's statistical knowledge rather than their English ability. I close with encouragements and recommendations for practitioners hoping to implement similar oral exams, focusing on the unique difficulties faced by students not learning in their mother tongue.

stat.OT↗

Uncertainty Quantification for Agent Based Models: A Tutorial

We explore the application of uncertainty quantification methods to agent-based models (ABMs) using a simple sheep and wolf predator-prey model. This work serves as a tutorial on how techniques like emulation can be powerful tools in this context. We also highlight the importance of advanced statistical methods in effectively utilising computationally expensive ABMs. Specifically, we implement stochastic Gaussian processes, Gaussian process classification, sequential design, and history matching to address uncertainties in model input parameters and outputs. Our results show that these methods significantly enhance the robustness, accuracy, and predictive power of ABMs.

stat.OT↗

Bi-Filtration and Stability of TDA Mapper for Point Cloud Data

Carlsson, Singh and Memoli's TDA mapper takes a point cloud dataset and outputs a graph that depends on several parameter choices. Dey, Memoli, and Wang developed Multiscale Mapper for abstract topological spaces so that parameter choices can be analyzed via persistent homology. However, when applied to actual data, one does not always obtain filtrations of mapper graphs. DBSCAN, one of the most common clustering algorithms used in the TDA mapper software, has two parameters, \textbf{$ε$} and \textbf{MinPts}. If \textbf{MinPts = 1} then DBSCAN is equivalent to single linkage clustering with cutting height \textbf{$ε$}. We show that if DBSCAN clustering is used with \textbf{MinPts $>$ 2}, a filtration of mapper graphs may not exist except in the absence of free-border points; but such filtrations exist if DBSCAN clustering is used with \textbf{MinPts = 1} or \textbf{2} as the cover size increases, \textbf{$ε$} increases, and/or \textbf{MinPts} decreases. However, the 1-dimensional filtration is unstable. If one adds noise to a data set so that each data point has been perturbed by a distance at most \textbf{$δ$}, the persistent homology of the mapper graph of the perturbed data set can be significantly different from that of the original data set. We show that we can obtain stability by increasing both the cover size and \textbf{$ε$} at the same time. In particular, we show that the bi-filtrations of the homology groups with respect to cover size and $ε$ between these two datasets are \textbf{2$δ$}-interleaved.

math.AT↗

easylayout: an R package for interactive force-directed layouts within RStudio

Motivation Network visualization is critical for effective communication in various fields of knowledge. Currently, a gap separates network manipulation from network visualization in programming environments. Users often export network data to be laid out in external interactive software, like Cytoscape and Gephi. We argue the current R package ecosystem lacks an interactive layout engine well integrated with common data analysis workflows. Results We present easylayout, an R package that bridges network manipulation and visualization by leveraging interactive force simulations within the IDE itself (e.g., RStudio, VSCode). It is not yet another visualization library, but instead aims to interconnect existing libraries and streamline their usage into the R ecosystem. easylayout takes an igraph object and serializes it into a web application integrated with the IDE's interface through a Shiny server. The app lays out the network by simulating attraction and repulsion forces. Simulation parameters can be adjusted in real-time. An editing mode allows moving and rotating nodes. The implementation aims for performance, so that even lower-end devices are able to work with relatively large networks. Once the user finishes tweaking the layout, it is sent back to the R session to be plotted through popular libraries like ggraph, igraph or even the base package itself. The current implementation focuses on the R ecosystem, but using web technologies makes it easily portable to similar environments, like Python/Jupyter Notebooks. We expect this tool to reduce the time spent searching for suitable network layouts, ultimately allowing researchers to generate more compelling figures. Availability and implementation easylayout is freely available under an MIT license on GitHub (https://github.com/dalmolingroup/easylayout). The package is implemented in R/Shiny and JavaScript/Svelte.

stat.OT↗

Grammatical structures in mathematics: a personal view

The ability to read, write, and speak mathematics is critical to students becoming comfortable with statistical models and skills. Faster development of those skills may act as encouragement to further engage with the discipline. Vocabulary has been the focus of scholarship in existing literature on the linguistics of mathematics and statistics but there are structures such as grammar that go beyond the content of words and symbols. Here I introduce ideas for grammar structures through a sequence of examples.

stat.OT↗

Statistical Methods for Crossover Trials: A Review of Classical and Recent Developments

A comprehensive review of the literature on crossover design is needed to highlight its evolution, applications, and methodological advancements across various fields. Given its widespread use in clinical trials and other research domains, understanding this design's challenges, assumptions, and innovations is essential for optimizing its implementation and ensuring accurate, unbiased results. This article extensively reviews the history and statistical inference methods for crossover designs. A primary focus is given to the AB-BA design as it is the most widely used design in literature. Extension from two periods to higher-order designs is discussed, and a general inference procedure for continuous response is studied. Analysis of multivariate and categorical responses is also reviewed in this context. Recent developments, including causal (potential outcomes) formulations of carryover, efficient use of period-specific baselines, generalized estimating equations for repeated measurements within periods, and estimands for incomplete crossover trials, are also discussed. Several open problems in this area are shortlisted.

stat.ME↗

Convergence to the Truth

This article reviews and develops an epistemological tradition in the philosophy of science, known as convergentism, which holds that inference methods should be assessed based on their ability to converge to the truth across a range of possible scenarios. Emphasis is placed on its historical origins in the work of C. S. Peirce and its recent developments in formal epistemology and data science (including statistics and machine learning). Comparisons are made with three other traditions: (1) explanationism, which holds that theory choice should be guided by a theory's overall balance of explanatory virtues, such as simplicity and fit with data; (2) instrumentalism, which maintains that scientific inference should be driven by the goal of obtaining useful models rather than true theories; and (3) Bayesianism, which shifts the focus from all-or-nothing beliefs to degrees of belief.

stat.OT↗

A geometrical perspective on parametric psychometric models

Psychometrics and quantitative psychology rely strongly on statistical models to measure psychological processes. As a branch of mathematics, geometry is inherently connected to measurement and focuses on properties such as distance and volume. However, despite the common root of measurement, geometry is currently not used a lot in psychological measurement. In this paper, my aim is to illustrate how ideas from non-Euclidean geometry may be relevant for psychometrics.

stat.AP↗

A comparative study of self-starting CUSUM control charts for location shifts

In recent years, self-starting methods have garnered increasing attention in Statistical Process Control and Monitoring (SPC/M), as they offer real-time disorder detection without the need for a calibration phase (Phase I). This study focuses on evaluating parametric self-starting CUSUM-type control charts, specifically the Bayesian Predictive Ratio CUSUM (PRC) developed by Bourazas et al. (2023) and the frequentist alternative self-starting CUSUM proposed by Hawkins and Olwell (1998). The performance of these methods is thoroughly examined through an extensive simulation study under various scenarios involving a change in the mean of Normal data. Additionally, a prior sensitivity analysis for PRC is conducted. The work ands with concluding remarks summarizing the findings.

stat.OT↗

Probabilistic inference when the population space is open

In using observed data to make inferences about a population quantity, it is commonly assumed that the sampling distribution from which the data were drawn belongs to a given parametric family of distributions, or at least, a given finite set of such families, i.e. the population space is assumed to be closed. Here, we address the problem of how to determine an appropriate post-data distribution for a given population quantity when such an assumption about the underlying sampling distribution is not made, i.e. when the population space is open. The strategy used to address this problem is based on the fact that even though, due to an open population space being non-measurable, we are not able to place a post-data distribution over all the sampling distributions contained in such a population space, it is possible to partition this type of space into a finite, countable or uncountable number of subsets such that a distribution can be placed over a variable that simply indicates which of these subsets contains the true sampling distribution. Moreover, it is argued that, by using sampling distributions that belong to a number of parametric families, it is possible to adequately and elegantly represent the sampling distributions that belong to each of the subsets of such a partition. Since a statistical model is conceived as being a model of a population space rather than a model of a sampling distribution, it is also argued that neither the type of models that are put forward nor the expression of pre-data knowledge via such models can be directly brought into question by the data. Finally, the case is made that, as well as not being required in the modelling process that is proposed, the standard practice of using P values to measure the absolute compatibility of an individual or family of sampling distributions with observed data is neither meaningful nor useful.

stat.ME↗

Developing Consistency Among Undergraduate Graders Scoring Open-Ended Statistics Tasks

Undergraduate graders are frequently important contributors to an instructional team in post-secondary education settings. This study set out to investigate agreement for a team of undergraduate graders as they acquired training and experience for scoring responses to open-ended tasks. Results demonstrate evidence that undergraduate graders can develop the ability to establish and sustain substantial agreement with an instructor, especially when equipped with proper training and a high-quality scoring rubric.

stat.OT↗

Adaptive partition Factor Analysis

Factor Analysis has traditionally been utilized across diverse disciplines to extrapolate latent traits that influence the behavior of multivariate observed variables. Historically, the focus has been on analyzing data from a single study, neglecting the potential study-specific variations present in data from multiple studies. Multi-study factor analysis has emerged as a recent methodological advancement that addresses this gap by distinguishing between latent traits shared across studies and study-specific components arising from artifactual or population-specific sources of variation. In this paper, we extend the current methodologies by introducing novel shrinkage priors for the latent factors, thereby accommodating a broader spectrum of scenarios -- from the absence of study-specific latent factors to models in which factors pertain only to small subgroups nested within or shared between the studies. For the proposed construction we provide conditions for identifiability of factor loadings and guidelines to perform straightforward posterior computation via Gibbs sampling. Through comprehensive simulation studies, we demonstrate that our proposed method exhibits competing performance across a variety of scenarios compared to existing methods, yet providing richer insights. The practical benefits of our approach are further illustrated through applications to bird species co-occurrence data and ovarian cancer gene expression data.

stat.ME↗