SEARCH · Searcharxiv

Results for “stat.OT”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23Linked to original sources

arXiv2022

On a generalised form of subjective probability

This paper is motivated by the questions of how to give the concept of probability an adequate real-world meaning, and how to explain a certain type of phenomenon that can be found, for instance, in Ellsberg's paradox. It attempts to answer these questions by constructing an alternative theory to one that was proposed in earlier papers on the basis of various important criticisms that were raised against this earlier theory. The conceptual principles of the corresponding definition of probability are laid out and explained in detail. In particular, what is required to fully specify a probability distribution under this definition is not just the distribution function of the variable concerned, but also an assessment of the internal and/or the external strength of this function relative to other distribution functions of interest. This way of defining probability is applied to various examples and problems including, perhaps most notably, to a long-running controversy concerning the distinction between Bayesian and fiducial inference. The characteristics of this definition of probability are carefully evaluated in terms of the issues that it sets out to address.

stat.OT↗

arXiv2018

Monte Carlo Simulations on robustness of functional location estimator based on several functional depth

Functional data analysis has been a growing field of study in recent decades, and one fundamental task in functional data analysis is estimating the sample location. A notion called statistical depth has been extended from multivariate data to functional data, and it can provide a center-outward order for each observation within a sample of functional curves. Making use of this intuitive nature of depth methods, a depth-based trimmed mean where curves with lower depth values are excluded can be used as a robust location estimator for the sample. In this project, we first introduced several state-of-the-art depth approaches for functional data. These depths were half region depth, functional majority depth, band depth, modified band depth and functional spatial depth. Then we described a robust location estimator based on functional depth, and studied performances of these estimators based on different functional depth approaches via simulation tests. Finally, the test results showed that estimators based on functional spatial depth and modified band depth exhibited superior performances.

stat.AP↗

arXiv2018

Using GitHub Classroom To Teach Statistics

Git and GitHub are common tools for keeping track of multiple versions of data analytic content, which allow for more than one person to simultaneously work on a project. GitHub Classroom aims to provide a way for students to work on and submit their assignments via Git and GitHub, giving teachers an opportunity to teach these version control tools as part of their course. In the Fall 2017 semester, we implemented GitHub Classroom in two educational settings--an introductory computational statistics lab and a more advanced computational statistics course. We found many educational benefits of implementing GitHub Classroom, such as easily providing coding feedback during assignments and making students more confident in their ability to collaborate and use version control tools for future data science work. To encourage and ease the transition into using GitHub Classroom, we provide free and publicly available resources--both for students to begin using Git/GitHub and for teachers to use GitHub Classroom for their own courses.

stat.OT↗

arXiv2019

Iterated Integrals and Population Time Series Analysis

One of the core advantages topological methods for data analysis provide is that the language of (co)chains can be mapped onto the semantics of the data, providing a natural avenue for human understanding of the results. Here, we describe such a semantic structure on Chen's classical iterated integral cochain model for paths in Euclidean space. Specifically, in the context of population time series data, we observe that iterated integrals provide a model-free measure of pairwise influence that can be used for causality inference. Along the way, we survey recent results and applications, review the current standard methods for causality inference, and briefly provide our outlook on generalizations to go beyond time series data.

stat.OT↗

arXiv2019

The ASCCR Frame for Learning Essential Collaboration Skills

Statistics and data science are especially collaborative disciplines that typically require practitioners to interact with many different people or groups. Consequently, interdisciplinary collaboration skills are part of the personal and professional skills essential for success as an applied statistician or data scientist. These skills are learnable and teachable, and learning and improving collaboration skills provides a way to enhance one's practice of statistics and data science. To help individuals learn these skills and organizations to teach them, we have developed a framework covering five essential components of statistical collaboration: Attitude, Structure, Content, Communication, and Relationship. We call this the ASCCR Frame. This framework can be incorporated into formal training programs in the classroom or on the job and can also be used by individuals through self-study. We show how this framework can be applied specifically to statisticians and data scientists to improve their collaboration skills and their interdisciplinary impact. We believe that the ASCCR Frame can help organize and stimulate research and teaching in interdisciplinary collaboration and call on individuals and organizations to begin generating evidence regarding its effectiveness.

stat.OT↗

arXiv2018

Surrogate Modeling of Stochastic Functions - Application to computational Electromagnetic Dosimetry

Metamodeling of complex numerical systems has recently attracted the interest of the mathematical programming community. Despite the progress in high performance computing, simulations remain costly, as a matter of fact, the assessment of the exposure to radio frequency electromagnetic fields is computationally prohibitive since one simulation can require hours. Moreover, in many engineering problems, carrying out deterministic numerical operations without considering uncertainties can lead to unreliable designs. In this paper we focus on the surrogate modeling of a particular type of computational models called stochastic simulators. In contrast to deterministic simulators which yield a unique output for each set of input parameters, stochastic simulators inherently contain some sources of randomness and the output at a given point is a probability density function. Characterizing the stochastic simulators is even more time consuming. This paper represents stochastic simulators as a stochastic process and describes a metamodeling approach based on the Karhunen-Loève spectral decomposition.

stat.OT↗

arXiv2019

Openness and Reproducibility: Insights from a Model-Centric Approach

This paper investigates the conceptual relationship between openness and reproducibility using a model-centric approach, heavily informed by probability theory and statistics. We first clarify the concepts of reliability, auditability, replicability, and reproducibility--each of which denotes a potential scientific objective. Then we advance a conceptual analysis to delineate the relationship between open scientific practices and these objectives. Using the notion of an idealized experiment, we identify which components of an experiment need to be reported and which need to be repeated to achieve the relevant objective. The model-centric framework we propose aims to contribute precision and clarity to the discussions surrounding the so-called reproducibility crisis.

stat.OT↗

arXiv2019

Batch Self Organizing maps for distributional data using adaptive distances

The paper deals with a Batch Self Organizing Map algorithm (DBSOM) for data described by distributional-valued variables. This kind of variables is characterized to take as values one-dimensional probability or frequency distributions on a numeric support. The objective function optimized in the algorithm depends on the choice of the distance measure. According to the nature of the date, the $L_2$ Wasserstein distance is proposed as one of the most suitable metrics to compare distributions. It is widely used in several contexts of analysis of distributional data. Conventional batch SOM algorithms consider that all variables are equally important for the training of the SOM. However, it is well known that some variables are less relevant than others for this task. In order to take into account the different contribution of the variables we propose an adaptive version of the DBSOM algorithm that tackles this problem with an additional step: a relevance weight is automatically learned for each distributional-valued variable. Moreover, since the $L_2$ Wasserstein distance allows a decomposition into two components: one related to the means and one related to the size and shape of the distributions, also relevance weights are automatically learned for each of the measurement components to emphasize the importance of the different estimated parameters of the distributions. Examples of real and synthetic datasets of distributional data illustrate the usefulness of the proposed DBSOM algorithms.

stat.OT↗

arXiv2019

Castor: Contextual IoT Time Series Data and Model Management at Scale

We demonstrate Castor, a cloud-based system for contextual IoT time series data and model management at scale. Castor is designed to assist Data Scientists in (a) exploring and retrieving all relevant time series and contextual information that is required for their predictive modelling tasks; (b) seamlessly storing and deploying their predictive models in a cloud production environment; (c) monitoring the performance of all predictive models in production and (semi-)automatically retraining them in case of performance deterioration. The main features of Castor are: (1) an efficient pipeline for ingesting IoT time series data in real time; (2) a scalable, hybrid data management service for both time series and contextual data; (3) a versatile semantic model for contextual information which can be easily adopted to different application domains; (4) an abstract framework for developing and storing predictive models in R or Python; (5) deployment services which automatically train and/or score predictive models upon user-defined conditions. We demonstrate Castor for a real-world Smart Grid use case and discuss how it can be adopted to other application domains such as Smart Buildings, Telecommunication, Retail or Manufacturing.

stat.CO↗

arXiv2018

Quantifying Privacy in Nuclear Warhead Authentication Protocols

International verification of nuclear warheads is a practical problem in which the protection of secret warhead information is of paramount importance. We propose a measure that would enable a weapon owner to evaluate the privacy of a proposed protocol in a technology-neutral fashion. We show the problem is reducible to `natural' and `corrective' learning. The natural learning can be computed without assumptions about the inspector, while the corrective learning accounts for the inspector's prior knowledge. The natural learning provides the warhead owner a useful lower bound on the information leaked by the proposed protocol. Using numerical examples, we demonstrate that the proposed measure correlates better with the accuracy of a maximum a posteriori probability estimate than alternative measures.

stat.OT↗

arXiv2018

On some properties of the new Sine-skewed Cardioid Distribution

The new Sine Skewed Cardioid (ssc) distribution been just introduced and characterized by Ahsanullah (2018). Here, we study the asymptotic properties of its tails by determining its extreme value domain, the characteristic function, the moments and likelihood estimators of the two parameters, the asymptotic normality of the moments estimators and the random generation of data from the \textit{ssc} distribution. Finally, we proceed to a simulation study to show the performance of the random generation method and the quality of the moments estimation of the parameters.

stat.OT↗

arXiv2019

Calculating CVaR and bPOE for Common Probability Distributions With Application to Portfolio Optimization and Density Estimation

Conditional Value-at-Risk (CVaR) and Value-at-Risk (VaR), also called the superquantile and quantile, are frequently used to characterize the tails of probability distribution's and are popular measures of risk. Buffered Probability of Exceedance (bPOE) is a recently introduced characterization of the tail which is the inverse of CVaR, much like the CDF is the inverse of the quantile. These quantities can prove very useful as the basis for a variety of risk-averse parametric engineering approaches. Their use, however, is often made difficult by the lack of well-known closed-form equations for calculating these quantities for commonly used probability distribution's. In this paper, we derive formulas for the superquantile and bPOE for a variety of common univariate probability distribution's. Besides providing a useful collection within a single reference, we use these formulas to incorporate the superquantile and bPOE into parametric procedures. In particular, we consider two: portfolio optimization and density estimation. First, when portfolio returns are assumed to follow particular distribution families, we show that finding the optimal portfolio via minimization of bPOE has advantages over superquantile minimization. We show that, given a fixed threshold, a single portfolio is the minimal bPOE portfolio for an entire class of distribution's simultaneously. Second, we apply our formulas to parametric density estimation and propose the method of superquantile's (MOS), a simple variation of the method of moment's (MM) where moment's are replaced by superquantile's at different confidence levels. With the freedom to select various combinations of confidence levels, MOS allows the user to focus the fitting procedure on different portions of the distribution, such as the tail when fitting heavy-tailed asymmetric data.

q-fin.RM↗

arXiv2018

Rapid Prototyping Model for Healthcare Alternative Payment Models: Replicating the Federally Qualified Health Center Advanced Primary Care Practice Demonstration

Innovation in healthcare payment and service delivery utilizes high cost, high risk pilots paired with traditional program evaluations. Decision-makers are unable to reliably forecast the impacts of pilot interventions in this complex system, complicating the feasibility assessment of proposed healthcare models. We developed and validated a Discrete Event Simulation (DES) model of primary care for patients with Diabetes to allow rapid prototyping and assessment of models before pilot implementation. We replicated four outcomes from the Centers for Medicare and Medicaid Services Federally Qualified Health Center Advanced Primary Care Practice pilot. The DES model simulates a synthetic population's healthcare experience, including symptom onset, appointment scheduling, screening, and treatment, as well as the impact of physician training. A network of detailed event modules was developed from peer-reviewed literature. Synthetic patients' attributes modify the probability distributions for event outputs and direct them through an episode of care; attributes are in turn modified by patients' experiences. Our model replicates the direction of the effect of physician training on the selected outcomes, and the strength of the effect increases with the number of trainings. The simulated effect strength replicates the pilot results for eye exams and nephropathy screening, but over-estimates results for HbA1c and LDL screening. Our model will improve decision-makers' abilities to assess the feasibility of pilot success, with reproducible, literature-based systems models. Our model identifies intervention and healthcare system components to which outcomes are sensitive, so these aspects can be monitored and controlled during pilot implementation. More work is needed to improve replication of HbA1c and LDL screening, and to elaborate sub-models related to intervention components.

stat.OT↗

arXiv2022

Multiple testing with persistent homology

In this paper we propose a computationally efficient multiple hypothesis testing procedure for persistent homology. The computational efficiency of our procedure is based on the observation that one can empirically simulate a null distribution that is universal across many hypothesis testing applications involving persistence homology. Our observation suggests that one can simulate the null distribution efficiently based on a small number of summaries of the collected data and use this null in the same way that p-value tables were used in classical statistics. To illustrate the efficiency and utility of the null distribution we provide procedures for rejecting acyclicity with both control of the Family-Wise Error Rate (FWER) and the False Discovery Rate (FDR). We will argue that the empirical null we propose is very general conditional on a few summaries of the data based on simulations and limit theorems for persistent homology for point processes.

cs.CG↗

arXiv2018

Gaussian Process Mixtures for Estimating Heterogeneous Treatment Effects

We develop a Gaussian-process mixture model for heterogeneous treatment effect estimation that leverages the use of transformed outcomes. The approach we will present attempts to improve point estimation and uncertainty quantification relative to past work that has used transformed variable related methods as well as traditional outcome modeling. Earlier work on modeling treatment effect heterogeneity using transformed outcomes has relied on tree based methods such as single regression trees and random forests. Under the umbrella of non-parametric models, outcome modeling has been performed using Bayesian additive regression trees and various flavors of weighted single trees. These approaches work well when large samples are available, but suffer in smaller samples where results are more sensitive to model misspecification - our method attempts to garner improvements in inference quality via a correctly specified model rooted in Bayesian non-parametrics. Furthermore, while we begin with a model that assumes that the treatment assignment mechanism is known, an extension where it is learnt from the data is presented for applications to observational studies. Our approach is applied to simulated and real data to demonstrate our theorized improvements in inference with respect to two causal estimands: the conditional average treatment effect and the average treatment effect. By leveraging our correctly specified model, we are able to more accurately estimate the treatment effects while reducing their variance.

stat.ME↗

arXiv2019

On a flexible construction of a negative binomial model

This work presents a construction of stationary Markov models with negative-binomial marginal distributions. A simple closed form expression for the corresponding transition probabilities is given, linking the proposal to well-known classes of birth and death processes and thus revealing interesting characterizations. The advantage of having such closed form expressions is tested on simulated and real data.

stat.ME↗

arXiv2019

Generalized Score Matching for Non-Negative Data

A common challenge in estimating parameters of probability density functions is the intractability of the normalizing constant. While in such cases maximum likelihood estimation may be implemented using numerical integration, the approach becomes computationally intensive. The score matching method of Hyvärinen [2005] avoids direct calculation of the normalizing constant and yields closed-form estimates for exponential families of continuous distributions over $\mathbb{R}^m$. Hyvärinen [2007] extended the approach to distributions supported on the non-negative orthant, $\mathbb{R}_+^m$. In this paper, we give a generalized form of score matching for non-negative data that improves estimation efficiency. As an example, we consider a general class of pairwise interaction models. Addressing an overlooked inexistence problem, we generalize the regularized score matching method of Lin et al. [2016] and improve its theoretical guarantees for non-negative Gaussian graphical models.

stat.ML↗

arXiv2018

Application of Robust Estimators in Shewhart S-Charts

Maintaining the quality of manufactured products at a desired level is known to increase customer satisfaction and profitability. Shewhart control chart is the most widely used in statistical process control (SPC) technique to monitor the quality of products and control process variability. Based on the assumption of independent and normally distributed data sets, sample mean and standard deviation statistics are known to be the most efficient conventional estimators to determine the process location and scale, respectively. On the other hand, there is not guarantee that the real-world process data would be normally distributed: outliers may exist, and/or sampled population may be contaminated. In such cases, efficiency of the conventional estimators is significantly reduced, and power of the Shewhart charts may be undesirably low, e.g. occasional outliers in the rational subgroups (Phase I dataset) may drastically affect the sample mean and standard deviation, resulting a serious delay in detection of inferior products (Phase II procedure). For more efficient analyses, it is required to use robust estimators against contaminations. Consequently, it is determined that robust estimators are more efficient both against diffuse localized and symmetric-asymmetric contaminations, and have higher power in detecting disturbances, compared to conventional methods.

stat.OT↗