SEARCH · Searcharxiv

Results for “stat.OT”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 649 records · Page 36Linked to original sources

arXiv2020

On statistical deficiency: Why the test statistic of the matching method is hopelessly underpowered and uniquely informative

The random variate m is, in combinatorics, a basis for comparing permutations, as well as the solution to a centuries-old riddle involving the mishandling of hats. In statistics, m is the test statistic for a disused null hypothesis statistical test (NHST) of association, the matching method. In this paper, I show that the matching method has an absolute and relatively low limit on its statistical power. I do so first by reinterpreting Rae's theorem, which describes the joint distributions of m with several rank correlation statistics under a true null. I then derive this property solely from m's unconditional sampling distribution, on which basis I develop the concept of a deficient statistic: a statistic that is insufficient and inconsistent and inefficient with respect to its parameter. Finally, I demonstrate an application for m that makes use of its deficiency to qualify the sampling error in a jointly estimated sample correlation.

stat.OT↗

arXiv2020

Methods to Deal with Unknown Populational Minima during Parameter Inference

There is a myriad of phenomena that are better modelled with semi-infinite distribution families, many of which are studied in survival analysis. When performing inference, lack of knowledge of the populational minimum becomes a problem, which can be dealt with by making a good guess thereof, or by handcrafting a grid of initial parameters that will be useful for that particular problem. These solutions are fine when analyzing a single set of samples, but it becomes unfeasible when there are multiple datasets and a case-by-case analysis would be too time consuming. In this paper we propose methods to deal with the populational minimum in algorithmic, efficient and/or simple ways. Six methods are presented and analyzed, two of which have full theoretical support, but lack simplicity. The other four are simple and have some theoretical grounds in non-parametric results such as the law of iterated logarithm, and they exhibited very good results when it comes to maximizing likelihood and being able to recycle the grid of initial parameters among the datasets. With our results, we hope to ease the inference process for practitioners, and expect that these methods will eventually be included in software packages themselves.

stat.CO↗

arXiv2021

A tutorial comparing different covariate balancing methods with an application evaluating the causal effects of substance use treatment programs for adolescents

Randomized controlled trials are the gold standard for measuring causal effects. However, they are often not always feasible, and causal treatment effects must be estimated from observational data. Observational studies do not allow robust conclusions about causal relationships unless statistical techniques account for the imbalance of pretreatment confounders across groups while key assumptions hold. Propensity score and balance weighting (PSBW) are useful techniques that aim to reduce the imbalances between treatment groups by weighting the groups to look alike on the observed confounders. There are many methods available to estimate PSBW. However, it is unclear a priori which will achieve the best trade-off between covariate balance and effective sample size. Moreover, it is critical to assess the validity of key assumptions required for robust estimation of the needed treatment effects, including the overlap and no unmeasured confounding assumptions. We present a step-by-step guide to covariate balancing strategies, including how to evaluate overlap, obtain estimates of PSBW, check for covariate balance, and assess sensitivity to unobserved confounding. We compare the performance of several estimation methods using a case study examining the relative effectiveness of substance use treatment programs and provide a user-friendly web application that can implement the proposed steps.

stat.ME↗

arXiv2021

Covariate-adjusted Fisher randomization tests for the average treatment effect

Fisher's randomization test (FRT) delivers exact $p$-values under the strong null hypothesis of no treatment effect on any units whatsoever and allows for flexible covariate adjustment to improve the power. Of interest is whether the procedure could also be valid for testing the weak null hypothesis of zero average treatment effect. Towards this end, we evaluate two general strategies for FRT with covariate-adjusted test statistics: that based on the residuals from an outcome model with only the covariates, and that based on the output from an outcome model with both the treatment and the covariates. Based on theory and simulation, we recommend using the ordinary least squares (OLS) fit of the observed outcome on the treatment, centered covariates, and their interactions for covariate adjustment, and conducting FRT with the robust $t$-value of the treatment as the test statistic. The resulting FRT is finite-sample exact for the strong null hypothesis, asymptotically valid for the weak null hypothesis, and more powerful than the unadjusted analog under alternatives, all irrespective of whether the linear model is correctly specified or not. We develop the theory for complete randomization, cluster randomization, stratified randomization, and rerandomization, respectively, and give a recommendation for the test procedure and test statistic under each design. We first focus on the finite-population perspective and then extend the result to the super-population perspective, highlighting the difference in standard errors. Motivated by the similarity in procedure, we also evaluate the design-based properties of five existing permutation tests originally for linear models and show the superiority of the proposed FRT for testing the treatment effects.

stat.ME↗

arXiv2020

Evaluation of approaches for accommodating interactions and non-linear terms in multiple imputation of incomplete three-level data

Three-level data structures arising from repeated measures on individuals clustered within larger units are common in health research studies. Missing data are prominent in such studies and are often handled via multiple imputation (MI). Although several MI approaches can be used to account for the three-level structure, including adaptations to single- and two-level approaches, when the substantive analysis model includes interactions or quadratic effects these too need to be accommodated in the imputation model. In such analyses, substantive model compatible (SMC) MI has shown great promise in the context of single-level data. While there have been recent developments in multilevel SMC MI, to date only one approach that explicitly handles incomplete three-level data is available. Alternatively, researchers can use pragmatic adaptations to single- and two-level MI approaches, or two-level SMC-MI approaches. We describe the available approaches and evaluate them via simulation in the context of a three three-level random effects analysis models involving an interaction between the incomplete time-varying exposure and time, an interaction between the time-varying exposure and an incomplete time-fixed confounder, or a quadratic effect of the exposure. Results showed that all approaches considered performed well in terms of bias and precision when the target analysis involved an interaction with time, but the three-level SMC MI approach performed best when the target analysis involved an interaction between the time-varying exposure and an incomplete time-fixed confounder, or a quadratic effect of the exposure. We illustrate the methods using data from the Childhood to Adolescence Transition Study.

stat.ME↗

arXiv2020

New definitions (measures) of skewness, mean and dispersion of fuzzy numbers -- by way of a new representation as parameterized curves

We give a geometrically motivated measure of skewness, define a mean value triangle number, and dispersion (in that order) of a fuzzy number without reference or seeking analogy to the namesake but parallel concepts in probability theory. These measures come about by way of a new representation of fuzzy numbers as parameterized curves respectively their associated tangent bundle. Importantly skewness and dispersion are given as functions of $α$ (the degree of membership) and such may be given separately and pointwise at each $α$-level, as well as overall. This allows for e.g., when a mathematical model is formulated in fuzzy numbers, to run optimization programs level-wise thereby encapsuling with deliberate accuracy the involved membership functions' characteristics while increasing the computational complexity by only a multiplicative factor compared to the same program formulated in real variables and parameters. As an example the work offers a contribution to the recently very popular fuzzy mean-variance-skewness portfolio optimization.

stat.OT↗

arXiv2022

The causal foundations of applied probability and statistics

Statistical science (as opposed to mathematical statistics) involves far more than probability theory, for it requires realistic causal models of data generators - even for purely descriptive goals. Statistical decision theory requires more causality: Rational decisions are actions taken to minimize costs while maximizing benefits, and thus require explication of causes of loss and gain. Competent statistical practice thus integrates logic, context, and probability into scientific inference and decision using narratives filled with causality. This reality was seen and accounted for intuitively by the founders of modern statistics, but was not well recognized in the ensuing statistical theory (which focused instead on the causally inert properties of probability measures). Nonetheless, both statistical foundations and basic statistics can and should be taught using formal causal models. The causal view of statistical science fits within a broader information-processing framework which illuminates and unifies frequentist, Bayesian, and related probability-based foundations of statistics. Causality theory can thus be seen as a key component connecting computation to contextual information, not extra-statistical but instead essential for sound statistical training and applications.

stat.OT↗

arXiv2020

Semiparametric inference for the scale-mixture of normal partial linear regression model with censored data

In the framework of censored data modeling, the classical linear regression model that assumes normally distributed random errors has received increasing attention in recent years, mainly for mathematical and computational convenience. However, practical studies have often criticized this linear regression model due to its sensitivity to departure from the normality and from the partial nonlinearity. This paper proposes to solve these potential issues simultaneously in the context of the partial linear regression model by assuming that the random errors follow a scale-mixture of normal (SMN) family of distributions. The proposed method allows us to model data with great flexibility, accommodating heavy tails, and outliers. By implementing the B-spline function and using the convenient hierarchical representation of the SMN distributions, a computationally analytical EM-type algorithm is developed to perform maximum likelihood inference of the model parameters. Various simulation studies are conducted to investigate the finite sample properties as well as the robustness of the model in dealing with the heavy-tails distributed datasets. Real-word data examples are finally analyzed for illustrating the usefulness of the proposed methodology.

stat.ME↗

arXiv2020

Digital trace data collection through data donation

A potentially powerful method of social-scientific data collection and investigation has been created by an unexpected institution: the law. Article 15 of the EU's 2018 General Data Protection Regulation (GDPR) mandates that individuals have electronic access to a copy of their personal data, and all major digital platforms now comply with this law by providing users with "data download packages" (DDPs). Through voluntary donation of DDPs, all data collected by public and private entities during the course of citizens' digital life can be obtained and analyzed to answer social-scientific questions - with consent. Thus, consented DDPs open the way for vast new research opportunities. However, while this entirely new method of data collection will undoubtedly gain popularity in the coming years, it also comes with its own questions of representativeness and measurement quality, which are often evaluated systematically by means of an error framework. Therefore, in this paper we provide a blueprint for digital trace data collection using DDPs, and devise a "total error framework" for such projects. Our error framework for digital trace data collection through data donation is intended to facilitate high quality social-scientific investigations using DDPs while critically reflecting its unique methodological challenges and sources of error. In addition, we provide a quality control checklist to guide researchers in leveraging the vast opportunities afforded by this new mode of investigation.

cs.CY↗

arXiv2021

CompModels: A suite of computer model test functions for Bayesian optimization

The CompModels package for R provides a suite of computer model test functions that can be used for computer model prediction/emulation, uncertainty quantification, and calibration, but in particular, the sequential optimization of computer models. The package is a mix of real-world physics problems, known mathematical functions, and black-box functions that have been converted into computer models with the goal of Bayesian (i.e., sequential) optimization in mind. Likewise, the package contains computer models that represent either the constrained or unconstrained optimization case, each with varying levels of difficulty. In this paper, we illustrate the use of the package with both real-world examples and black-box functions by solving constrained optimization problems via Bayesian optimization. Ultimately, the package is shown to provide users with a source of computer model test functions that are reproducible, shareable, and that can be used for benchmarking of novel optimization methods.

stat.CO↗

arXiv2020

Gonogo: An R Implementation of Test Methods to Perform, Analyze and Simulate Sensitivity Experiments

This work provides documentation for a suite of R functions contained in gonogo.R. The functions provide sensitivity testing practitioners and researchers with an ability to conduct, analyze and simulate various sensitivity experiments involving binary responses and a single stimulus level (e.g., drug dosage, drop height, velocity, etc.). Included are the modern Neyer and 3pod adaptive procedures, as well as the Bruceton and Langlie. The latter two benchmark procedures are capable of being performed according to generalized up-down transformed-response rules. Each procedure is designated phase-one of a three-phase experiment. The goal of phase-one is to achieve overlapping data. The two additional (and optional) refinement phases utilize the D-optimal criteria and the Robbins-Monro-Joseph procedure. The goals of the two refinement phases are to situate testing in the vicinity of the median and tails of the latent response distribution, respectively.

stat.CO↗

arXiv2020

Comparisons of proportions in k dose groups against a negative control assuming order restriction: Williams-type test vs. closed test procedures

The comparison of proportions is considered in the asymptotic generalized linear model with the odds ratio as effect size. When several doses are compared with a control assuming an order restriction, a Williams-type trend test can be used. As an alternative, two variants of the closed testing approach are considered, one using global Williams-tests in the partition hypotheses, one with pairwise contrasts. Their advantages in terms of power and simplicity are demonstrated. Related R-code is provided.

stat.AP↗

arXiv2020

Including climate system feedbacks in calculations of the social cost of methane

Integrated assessment models (IAMs) are valuable tools that consider the interactions between socioeconomic systems and the climate system. Decision-makers and policy analysts employ IAMs to calculate the marginalized monetary cost of climate damages resulting from an incremental emission of a greenhouse gas. Used within the context of regulating anthropogenic methane emissions, this metric is called the social cost of methane (SC-CH$_4$). Because several key IAMs used for social cost estimation contain a simplified model structure that prevents the endogenous modeling of non-CO$_2$ greenhouse gases, very few estimates of the SC-CH$_4$ exist. For this reason, IAMs should be updated to better represent methane cycle dynamics that are consistent with comprehensive Earth System Models. We include feedbacks of climate change on the methane cycle to estimate the SC-CH$_4$. Our expected value for the SC-CH$_4$ is \$1163/t-CH$_4$ under a constant 3.0% discount rate. This represents a 44% increase relative to a mean estimate without feedbacks on the methane cycle.

physics.ao-ph↗

arXiv2020

Casting Multiple Shadows: High-Dimensional Interactive Data Visualisation with Tours and Embeddings

Non-linear dimensionality reduction (NLDR) methods such as t-distributed stochastic neighbour embedding (t-SNE) are ubiquitous in the natural sciences, however, the appropriate use of these methods is difficult because of their complex parameterisations; analysts must make trade-offs in order to identify structure in the visualisation of an NLDR technique. We present visual diagnostics for the pragmatic usage of NLDR methods by combining them with a technique called the tour. A tour is a sequence of interpolated linear projections of multivariate data onto a lower dimensional space. The sequence is displayed as a dynamic visualisation, allowing a user to see the shadows the high-dimensional data casts in a lower dimensional view. By linking the tour to an NLDR view, we can preserve global structure and through user interactions like linked brushing observe where the NLDR view may be misleading. We display several case studies from both simulations and single cell transcriptomics, that shows our approach is useful for cluster orientation tasks.

stat.OT↗

arXiv2020

Spatial variation in the basic reproduction number of COVID-19: A systematic review

OBJECTIVES: Estimates of the basic reproduction number (R0) of COVID-19 vary across countries. This paper aims to characterise the spatial variability in R0 across the first six months of the global COVID-19 outbreak, and to explore social factors that impact R0 estimates at national and regional level. METHODS: We searched PubMed, LitCOVID and the WHO COVID-19 database from January to June 2020. Peer-reviewed English-language papers were included that provided R0 estimates. For each study, the value of the estimate, country under study and publication month were extracted. The median R0 value was calculated per country, and the median and variance were calculated per region. For each country with an R0 estimate, the Human Development Index (HDI), Sustainable Mobility Index (SMI), median age, population density and development status were obtained from external sources. RESULTS: A total of 81 studies were included in the analysis. These studies provided at least one estimate of R0, along with sufficient methodology to explain how the value was calculated. Values of R0 ranged between 0.48 and 14.8, and between 0.48 and 6.7 when excluding outliers. CONCLUSIONS: This systematic review provides a comprehensive overview of the estimates of the basic reproduction number of COVID-19 globally and highlights the spatial heterogeneity in R0. Higher values were recorded in more developed countries, and countries with an older population or more sustainable mobility. Countries with higher population density had lower R0 estimates. For most regions, variability in R0 spiked initially before reducing and stabilising as more estimates became available.

stat.OT↗

arXiv2020

Ordering results of extreme order statistics from multiple-outlier scale models with dependence

In this paper, we focus on stochastic comparisons of extreme order statistics stemming from multiple-outlier scale models with dependence. Archimedean copula is used to model dependence structure among nonnegative random variables. Sufficient conditions are obtained for comparison of the largest order statistics in the sense of the usual stochastic, reversed hazard rate, star and Lorenz orders. The smallest order statistics are also compared with respect to the usual stochastic, hazard rate, star and Lorenz orders. To illustrate the theoretical establishments, some examples are provided.

math.ST↗

arXiv2020

A Survey on the Visual Perceptions of Gaussian Noise Filtering on Photography

Statisticians, as well as machine learning and computer vision experts, have been studying image reconstitution through denoising different domains of photography, such as textual documentation, tomographic, astronomical, and low-light photography. In this paper, we apply common inferential kernel filters in the R and python languages, as well as Adobe Lightroom's denoise filter, and compare their effectiveness in removing noise from JPEG images. We ran standard benchmark tests to evaluate each method's effectiveness for removing noise. In doing so, we also surveyed students at Elon University about their opinion of a single filtered photo from a collection of photos processed by the various filter methods. Many scientists believe that noise filters cause blurring and image quality loss so we analyzed whether or not people felt as though denoising causes any quality loss as compared to their noiseless images. Individuals assigned scores indicating the image quality of a denoised photo compared to its noiseless counterpart on a 1 to 10 scale. Survey scores are compared across filters to evaluate whether there were significant differences in image quality scores received. Benchmark scores were compared to the visual perception scores. Then, an analysis of covariance test was run to identify whether or not survey training scores explained any unplanned variation in visual scores assigned by students across the filter methods.

cs.CV↗

arXiv2020

Beyond Strictly Proper Scoring Rules: The Importance of Being Local

The evaluation of probabilistic forecasts plays a central role both in the interpretation and in the use of forecast systems and their development. Probabilistic scores (scoring rules) provide statistical measures to assess the quality of probabilistic forecasts. Often, many probabilistic forecast systems are available while evaluations of their performance are not standardized, with different scoring rules being used to measure different aspects of forecast performance. Even when the discussion is restricted to strictly proper scoring rules, there remains considerable variability between them; indeed strictly proper scoring rules need not rank competing forecast systems in the same order when none of these systems are perfect. The locality property is explored to further distinguish scoring rules. The nonlocal strictly proper scoring rules considered are shown to have a property that can produce "unfortunate" evaluations. Particularly the fact that Continuous Rank Probability Score prefers the outcome close to the median of the forecast distribution regardless the probability mass assigned to the value at/near the median raises concern to its use. The only local strictly proper scoring rules, the logarithmic score, has direct interpretations in terms of probabilities and bits of information. The nonlocal strictly proper scoring rules, on the other hand, lack meaningful direct interpretation for decision support. The logarithmic score is also shown to be invariant under smooth transformation of the forecast variable, while the nonlocal strictly proper scoring rules considered may, however, change their preferences due to the transformation. It is therefore suggested that the logarithmic score always be included in the evaluation of probabilistic forecasts.

stat.ME↗