Searcharxiv⌕ Search

SEARCH · Searcharxiv

Results for “stat.OT”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21Linked to original sources
arXiv2018

Multiple factor analysis of distributional data

In the framework of Symbolic Data Analysis (SDA), distribution-variables are a particular case of multi-valued variables: each unit is represented by a set of distributions (e.g. histograms, density functions or quantile functions), one for each variable. Factor analysis (FA) methods are primary exploratory tools for dimension reduction and visualization. In the present work, we use Multiple Factor Analysis (MFA) approach for the analysis of data described by distributional variables. Each distributional variable induces a set new numeric variable related to the quantiles of each distribution. We call these new variables as \textit{quantile variables} and the set of quantile variables related to a distributional one is a block in the MFA approach. Thus, MFA is performed on juxtaposed tables of quantile variables. \\ We show that the criterion decomposed in the analysis is an approximation of the variability based on a suitable metrics between distributions: the squared $L_2$ Wasserstein distance. \\ Applications on simulated and real distributional data corroborate the method. The interpretation of the results on the factorial planes is performed by new interpretative tools that are related to the several characteristics of the distributions (location, scale and shape).

stat.ME↗
arXiv2018

Resolving the Lord's Paradox

An explanation to Lord's paradox using ordinary least square regression models is given. It is not a paradox at all, if the regression parameters are interpreted as predictive or as causal with stricter conditions and be aware of laws of averages. We use derivation of a super-model from a given sub-model, when its residuals can be modelled with other potential predictors as a solution.

stat.OT↗
arXiv2018

Viewing Simpson's Paradox

Well known Simpson's paradox is puzzling and surprising for many, especially for the empirical researchers and users of statistics. However there is no surprise as far as mathematical details are concerned. A lot more is written about the paradox but most of them are beyond the grasp of such users. This short article is about explaining the phenomenon in an easy way to grasp using simple algebra and geometry. The mathematical conditions under which the paradox can occur are made explicit and a simple geometrical illustrations is used to describe it. We consider the reversal of the association between two binary variables, say, $X$ and $Y$ by a third binary variable, say, $Z$. We show that it is always possible to define $Z$ algebraically for non-extreme dependence between $X$ and $Y$, therefore occurrence of the paradox depends on identifying it with a practical meaning for it in a given context of interest, that is up to the subject domain expert. And finally we discuss the paradox in predictive contexts since in literature it is argued that the paradox is resolved using causal reasoning.

stat.OT↗
arXiv2018

BNSP: an R Package for Fitting Bayesian Semiparametric Regression Models and Variable Selection

The R package BNSP provides a unified framework for semiparametric location-scale regression and stochastic search variable selection. The statistical methodology that the package is built upon utilizes basis function expansions to represent semiparametric covariate effects in the mean and variance functions, and spike-slab priors to perform selection and regularization of the estimated effects. In addition to the main function that performs posterior sampling, the package includes functions for assessing convergence of the sampler, summarizing model fits, visualizing covariate effects and obtaining predictions for new responses or their means given feature/covariate vectors.

stat.OT↗
arXiv2018

Benford or not Benford: new results on digits beyond the first

In this paper, we will see that the proportion of d as p th digit, where p > 1 and d $\in$ 0, 9, in data (obtained thanks to the hereunder developed model) is more likely to follow a law whose probability distribution is determined by a specific upper bound, rather than the generalization of Benford's Law to digits beyond the first one. These probability distributions fluctuate around theoretical values determined by Hill in 1995. Knowing beforehand the value of the upper bound can be a way to find a better adjusted law than Hill's one.

stat.OT↗
arXiv2020

Conjectures on Optimal Nested Generalized Group Testing Algorithm

Consider a finite population of $N$ items, where item $i$ has a probability $p_i$ to be defective. The goal is to identify all items by means of group testing. This is the generalized group testing problem (hereafter GGTP). In the case of $\displaystyle p_1=\cdots=p_{N}=p$ \cite{YH1990} proved that the pairwise testing algorithm is the optimal nested algorithm, with respect to the expected number of tests, for all $N$ if and only if $\displaystyle p \in [1-1/\sqrt{2},\,(3-\sqrt{5})/2]$ (R-range hereafter) (an optimal at the boundary values). In this note, we present a result that helps to define the generalized pairwise testing algorithm (hereafter GPTA) for the GGTP. We present two conjectures: (1) when all $p_i, i=1,\ldots,N$ belong to the R-range, GPTA is the optimal procedure among nested procedures applied to $p_i$ of nondecreasing order; (2) if all $p_i, i=1,\ldots,N$ belong to the R-range, GPTA the optimal nested procedure, i.e., minimises the expected total number of tests with respect to all possible testing orders in the class of nested procedures. Although these conjectures are logically reasonable, we were only able to empirically verify the first one up to a particular level of $N$. We also provide a short survey of GGTP.

stat.OT↗
arXiv2018

On estimands and the analysis of adverse events in the presence of varying follow-up times within the benefit assessment of therapies

The analysis of adverse events (AEs) is a key component in the assessment of a drug's safety profile. Inappropriate analysis methods may result in misleading conclusions about a therapy's safety and consequently its benefit-risk ratio. The statistical analysis of AEs is complicated by the fact that the follow-up times can vary between the patients included in a clinical trial. This paper takes as its focus the analysis of AE data in the presence of varying follow-up times within the benefit assessment of therapeutic interventions. Instead of approaching this issue directly and solely from an analysis point of view, we first discuss what should be estimated in the context of safety data, leading to the concept of estimands. Although the current discussion on estimands is mainly related to efficacy evaluation, the concept is applicable to safety endpoints as well. Within the framework of estimands, we present statistical methods for analysing AEs with the focus being on the time to the occurrence of the first AE of a specific type. We give recommendations which estimators should be used for the estimands described. Furthermore, we state practical implications of the analysis of AEs in clinical trials and give an overview of examples across different indications. We also provide a review of current practices of health technology assessment (HTA) agencies with respect to the evaluation of safety data. Finally, we describe problems with meta-analyses of AE data and sketch possible solutions.

stat.OT↗
arXiv2018

To Bayes or Not To Bayes? That's no longer the question!

This paper seeks to provide a thorough account of the ubiquitous nature of the Bayesian paradigm in modern statistics, data science and artificial intelligence. Once maligned, on the one hand by those who philosophically hated the very idea of subjective probability used in prior specification, and on the other hand because of the intractability of the computations needed for Bayesian estimation and inference, the Bayesian school of thought now permeates and pervades virtually all areas of science, applied science, engineering, social science and even liberal arts, often in unsuspected ways. Thanks in part to the availability of powerful computing resources, but also to the literally unavoidable inherent presence of the quintessential building blocks of the Bayesian paradigm in all walks of life, the Bayesian way of handling statistical learning, estimation and inference is not only mainstream but also becoming the most central approach to learning from the data. This paper explores some of the most relevant elements to help to the reader appreciate the pervading power and presence of the Bayesian paradigm in statistics, artificial intelligence and data science, with an emphasis on how the Gospel according to Reverend Thomas Bayes has turned out to be the truly good news, and some cases the amazing saving grace, for all who seek to learn statistically from the data. To further help the reader gain deeper and tangible practical insights into the Bayesian machinery, we point to some computational tools designed for the R Statistical Software Environment to help explore Bayesian statistical learning.

stat.OT↗
arXiv2018

Absolutely Zero Evidence

Statistical analysis is often used to evaluate the evidence for or against scientific hypotheses, and various statistics (e.g., p-values, likelihood ratios, Bayes factors) are interpreted as measures of evidence strength. Here I consider evidence measurement from the point of view of representational measurement theory, and argue that familiar evidence statistics do not conform to any legitimate measurement scale type. I then consider the notion of an absolute scale for evidence measurement, in a sense to be defined, focusing particularly on the notion of absolute 0 evidence, which turns out to be something other than what one might have expected.

stat.OT↗
arXiv2018

A Constructive Algebraic Proof of Student's Theorem

Student's theorem is an important result in statistics which states that for normal population, the sample variance is independent from the sample mean and has a chi-square distribution. The existing proofs of this theorem either overly rely on advanced tools such as moment generating functions, or fail to explicitly construct an orthogonal matrix used in the proof. This paper provides an elegant explicit construction of that matrix, making the algebraic proof complete. The constructive algebraic proof proposed here is thus very suitable for being included in textbooks.

stat.OT↗
arXiv2018

Nonparametric Confidence Regions for Veronese-Whitney Means and Antimeans on Planar Kendall Shape Spaces

In this paper after a brief revision of VW-means, which are extrinsic means on real and complex projective spaces, relative to the Veronese-Whitney embeddings, we give two examples of sample VW means computations on planar Kendall shape spaces. Here we derive large sample and pivotal nonparametric bootstrap confidence regions for VW-antimeans, using VW-anti-covariance matrices, and their sample counterparts

stat.OT↗
arXiv2018

Discussion on Using Stacking to Average Bayesian Predictive Distributions by Yao et al

I begin by summarizing key ideas of the paper under discussion. Then I will talk about a graphical modeling perspective, posterior contraction rates and alternative methods of aggregation. Moreover, I will also discuss possible applications of the stacking method to other problems, in particular, aggregating (sub)posterior distributions in distributed computing.

stat.OT↗
arXiv2019

Applying a Dynamical Systems Model and Network Theory to Major Depressive Disorder

Mental disorders like major depressive disorder can be seen as complex dynamical systems. In this study we investigate the dynamic behaviour of individuals to see whether or not we can expect a transition to another mood state. We introduce a mean field model to a binomial process, where we reduce a dynamic multidimensional system (stochastic cellular automaton) to a one-dimensional system to analyse the dynamics. Using maximum likelihood estimation, we can estimate the parameter of interest which, in combination with a bifurcation diagram, reflects the expectancy that someone has to transition to another mood state. After validating the proposed method with simulated data, we apply this method to two empirical examples, where we show its use in a clinical sample consisting of patients diagnosed with major depressive disorder, and a general population sample. Results showed that the majority of the clinical sample was categorized as having an expectancy for a transition, while the majority of the general population sample did not have this expectancy. We conclude that the mean field model has great potential in assessing the expectancy for a transition between mood states. With some extensions it could, in the future, aid clinical therapists in the treatment of depressed patients.

physics.soc-ph↗
arXiv2018

Spatially varying coefficient modeling for large datasets: Eliminating N from spatial regressions

While spatially varying coefficient (SVC) modeling is popular in applied science, its computational burden is substantial. This is especially true if a multiscale property of SVC is considered. Given this background, this study develops a Moran's eigenvector-based spatially varying coefficients (M-SVC) modeling approach that estimates multiscale SVCs computationally efficiently. This estimation is accelerated through a (i) rank reduction, (ii) pre-compression, and (iii) sequential likelihood maximization. Steps (i) and (ii) eliminate the sample size N from the likelihood function; after these steps, the likelihood maximization cost is independent of N. Step (iii) further accelerates the likelihood maximization so that multiscale SVCs can be estimated even if the number of SVCs, K, is large. The M-SVC approach is compared with geographically weighted regression (GWR) through Monte Carlo simulation experiments. These simulation results show that our approach is far faster than GWR when N is large, despite numerically estimating 2K parameters while GWR numerically estimates only 1 parameter. Then, the proposed approach is applied to a land price analysis as an illustration. The developed SVC estimation approach is implemented in the R package "spmoran."

stat.ME↗
arXiv2018

Sounding Spider: An Efficient Way for Representing Uncertainties in High Dimensions

This article proposes a visualization method for multidimensional data based on: (i) Animated functional Hypothetical Outcome Plots (f-HOPs); (ii) 3-dimensional Kiviat plot; and (iii) data sonification. In an Uncertainty Quantification (UQ) framework, such analysis coupled with standard statistical analysis tools such as Probability Density Functions (PDF) can be used to augment the understanding of how the uncertainties in the numerical code inputs translate into uncertainties in the quantity of interest (QoI). In contrast with static representation of most advanced techniques such as functional Highest Density Region (HDR) boxplot or functional boxplot, f-HOPs is a dynamic visualization that enables the practitioners to infer the dynamics of the physics and enables to see functional correlations that may exist. While this technique only allows to represent the QoI, we propose a 3-dimensional version of the Kiviat plot to encode all input parameters. This new visualization takes advantage of information from f-HOPs through data sonification. All in all, this allows to analyse large datasets within a high-dimensional parameter space and a functional QoI in the same canvas. The proposed method is assessed and showed its benefits on two related environmental datasets.

stat.ME↗
arXiv2019

Allocations of Cold Standbys to Series and Parallel Systems with Dependent Components

In the context of industrial engineering, cold-standby redundancies allocation strategy is usually adopted to improve the reliability of coherent systems. This paper investigates optimal allocation strategies of cold standbys for series and parallel systems comprised of dependent components with left/right tail weakly stochastic arrangement increasing lifetimes. For the case of heterogeneous and independent matched cold standbys, it is proved that better redundancies should be put in the nodes having weaker [better] components for series [parallel] systems. For the case of homogeneous and independent cold standbys, it is shown that more redundancies should be put in standby with weaker [better] components to enhance the reliability of series [parallel] systems. The results developed here generalize and extend those corresponding ones in the literature to the case of series and parallel systems with dependent components. Numerical examples are also presented to provide guidance for the practical use of our theoretical findings.

stat.OT↗
arXiv2020

An optimal design for hierarchical generalized group testing

Choosing an optimal strategy for hierarchical group testing is an important problem for practitioners who are interested in disease screening with limited resources. For example, when screening for infectious diseases in large populations, it is important to use algorithms that minimize the cost of potentially expensive assays. Black et al. (2015) described this as an intractable problem unless the number of individuals to screen is small. They proposed an approximation to an optimal strategy that is difficult to implement for large population sizes. In this article, we develop an optimal design with respect to the expected total number of tests that can be obtained using a novel dynamic programming algorithm. We show that this algorithm is substantially more efficient than the approach proposed by Black et al. (2015). In addition, we compare the two designs for imperfect tests. R code is provided for the practitioner.

stat.ME↗