SEARCH · Searcharxiv

Results for “stat.OT”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30Linked to original sources

arXiv2019

Time-based analysis of the NBA hot hand fallacy

The debate surrounding the hot hand in the NBA has been ongoing for many years. However, many of the previous works on this theme has focused on only the very next sequential shot attempt, often on very select players. This work looks in more detail the effect of a made or missed shot on the next series of shots over a two-year span, with time between shots shown to be a critical factor in the analysis. Also, multi-year streakiness is analyzed, and all indications are that players cannot really sustain their good (or bad) fortune from year to year.

stat.OT↗

arXiv2019

Visualisation of Brain Statistics with R-packages ggseg and ggseg3d

There is an increased emphasis on visualizing neuroimaging results in more intuitive ways. Common statistical tools for dissemination, such as bar charts, lack the spatial dimension that is inherent in neuroimaging data. Here we present two packages for the statistical software R, ggseg and ggseg3d, that integrate this spatial component. The ggseg and ggseg3d packages visualize pre-defined brain segmentations as both 2D polygons and 3D meshes, respectively. Both packages are integrated with other well-established R-packages, allowing great flexibility. In this tutorial, we present the main data and functions in the ggseg and ggseg3d packages for brain atlas visualization. The main highlighted functions are able to display brain segmentation plots in R. Further, the accompanying ggsegExtra-package includes a wider collection of atlases, and is intended for community-based efforts to develop more compatible atlases to ggseg and ggseg3d. Overall, the ggseg-packages facilitate parcellation-based visualizations in R, improve and ease the dissemination of the results, and increase the efficiency of the workflows.

stat.OT↗

arXiv2021

Quantile Diffusions for Risk Analysis

We develop a novel approach for the construction of quantile processes governing the stochastic dynamics of quantiles in continuous time. Two classes of quantile diffusions are identified: the first, which we largely focus on, features a dynamic random quantile level and allows for direct interpretation of the resulting quantile process characteristics such as location, scale, skewness and kurtosis, in terms of the model parameters. The second type are function-valued quantile diffusions and are driven by stochastic parameter processes, which determine the entire quantile function at each point in time. By the proposed innovative and simple -- yet powerful -- construction method, quantile processes are obtained by transforming the marginals of a diffusion process under a composite map consisting of a distribution and a quantile function. Such maps, analogous to rank transmutation maps, produce the marginals of the resulting quantile process. We discuss the relationship and differences between our approach and existing methods and characterisations of quantile processes in discrete and continuous time. As an example of an application of quantile diffusions, we show how probability measure distortions, a form of dynamic tilting, can be induced. Though particularly useful in financial mathematics and actuarial science, examples of which are given in this work, measure distortions feature prominently across multiple research areas. For instance, dynamic distributional approximations (statistics), non-parametric and asymptotic analysis (mathematical statistics), dynamic risk measures (econometrics), behavioural economics, decision making (operations research), signal processing (information theory), and not least in general risk theory including applications thereof, for example in the context of climate change.

math.PR↗

arXiv2019

AVaN Pack: An Analytical/Numerical Solution for Variance-Based Sensitivity Analysis

Sensitivity analysis is an important concept to analyze the influences of parameters in a system, an equation or a collection of data. The methods used for sensitivity analysis are divided into deterministic and statistical techniques. Generally, deterministic techniques analyze fixed points of a model whilst stochastic techniques analyze a range of values. Deterministic methods fail in analyze the entire range of input values and stochastic methods generate outcomes with random errors. In this manuscript, we are interested in stochastic methods, mainly in variance-based techniques such as Variance and Sobol indices, since this class of techniques is largely used on literature. The objective of this manuscript is to present an analytical solution for variance based sensitive analysis. As a result of this research, two small programs were developed in Javascript named as AVaN Pack (Analysis of Variance through Numerical solution). These programs allow users to find the contribution of each individual parameter in any function by means of a mathematical solution, instead of sampling-based ones.

stat.OT↗

arXiv2019

Bayesian estimation of large dimensional time varying VARs using copulas

This paper provides a simple, yet reliable, alternative to the (Bayesian) estimation of large multivariate VARs with time variation in the conditional mean equations and/or in the covariance structure. With our new methodology, the original multivariate, n dimensional model is treated as a set of n univariate estimation problems, and cross-dependence is handled through the use of a copula. Thus, only univariate distribution functions are needed when estimating the individual equations, which are often available in closed form, and easy to handle with MCMC (or other techniques). Estimation is carried out in parallel for the individual equations. Thereafter, the individual posteriors are combined with the copula, so obtaining a joint posterior which can be easily resampled. We illustrate our approach by applying it to a large time-varying parameter VAR with 25 macroeconomic variables.

econ.EM↗

arXiv2019

A generalization of the symmetrical and optimal probability-to-possibility transformations

Possibility and probability theories are alternative and complementary ways to deal with uncertainty, which has motivated over the last years an interest for the study of ways to transform probability distributions into possibility distributions and conversely. This paper studies the advantages and shortcomings of two well-known discrete probability to possibility transformations: the optimal transformation and the symmetrical transformation, and presents a novel parametric family of probability to possibility transformations which generalizes them and alleviate their shortcomings, showing a big potential for practical application. The paper also introduces a novel fuzzy measure of specificity for probability distributions based on the concept of fuzzy subsethood and presents a empirical validation of the generalized transformation usefulness applying it to the text authorship attribution problem.

stat.OT↗

arXiv2020

Selection Induced Contrast Estimate (SICE) Effect: An Attempt to Quantify the Impact of Some Patient Selection Criteria in Randomized Clinical Trials

Defining the Inclusion/Exclusion (I/E) criteria of a trial is one of the most important steps during a trial design. Increasingly complex I/E criteria potentially create information imbalance and transparency issues between the people who design and run the trials and those who consume the information produced by the trials. In order to better understand and quantify the impact of a category of I/E criteria on observed treatment effects, a concept, named the Selection Induced Contrast Estimate (SICE) effect, is introduced and formulated in this paper. The SICE effect can exist in controlled clinical trials when treatment affects the correlation between a marker used for selection and the response of interest. This effect is demonstrated with both simulations and real clinical trial data. Although the statistical elements behind the SICE effect have been well studied, explicitly formulating and studying this effect can benefit several areas, including better transparency in I/E criteria, meta-analysis of multiple clinical trials, treatment effect interpretation in real-world medical practice, etc.

stat.AP↗

arXiv2022

Exact Bayesian inference for discretely observed Markov Jump Processes using finite rate matrices

We present new methodologies for Bayesian inference on the rate parameters of a discretely observed continuous-time Markov jump processes with a countably infinite state space. The usual method of choice for inference, particle Markov chain Monte Carlo (particle MCMC), struggles when the observation noise is small. We consider the most challenging regime of exact observations and provide two new methodologies for inference in this case: the minimal extended state space algorithm (MESA) and the nearly minimal extended state space algorithm (nMESA). By extending the Markov chain Monte Carlo state space, both MESA and nMESA use the exponentiation of finite rate matrices to perform exact Bayesian inference on the Markov jump process even though its state space is countably infinite. Numerical experiments show improvements over particle MCMC of between a factor of three and several orders of magnitude.

stat.CO↗

arXiv2025

Resolving the induction problem: Can we state with complete confidence via induction that the sun rises forever?

Induction is a form of reasoning that starts with a particular example and generalizes to a rule, namely, a hypothesis. However, establishing the truth of a hypothesis is problematic due to the potential occurrence of conflicting events, also known as the induction problem. The sunrise problem, first introduced by Laplace (1814), is a quintessential example of the probability-based induction. In his solution, a zero probability is always assigned to the hypothesis that the sun rises forever, regardless of the number of observations made. This is a symptom of fundamental deficiency of probability-based induction: A hypothesis can never be accepted via the Bayes-Laplace approach. Alternative priors have been proposed to address this issue, but they have failed to fully overcome the deficiency. We investigate why this occurs and demonstrate that the confidence does not exhibit such a deficiency, as it is not a probability and therefore does not adhere to Bayes' rule. The confidence is neither a likelihood to allow not only a reconciliation between epistemic and aleatory interpretations of probability but also a resolution in agreement with the evidence by enabling us to accept a hypothesis with complete confidence as a rational decision.

stat.OT↗

arXiv2020

Distributions of differences of Riemann zeta zeros

We study distributions of differences of unscaled Riemann zeta zeros, $γ-γ'$, at large. We show, that independently of the location of the zeros, their differences have similar statistical properties. The distributions of differences are skewed towards the nearest zeta zero, have local maximum of variance and local minimum of kurtosis at or near each zeta zero. Furthermore, we show that distributions can be fitted with Johnson probability density function, despite the value of skewness or kurtosis of the distribution.

math.NT↗

arXiv2020

MonteCarloMeasurements.jl: Nonlinear Propagation of Arbitrary Multivariate Distributions by means of Method Overloading

This manuscript outlines a software package that facilitates working with probability distributions by means of Monte-Carlo methods, in a way that allows for propagation of multivariate probability distributions through arbitrary functions. We provide a \emph{type} that represents probability distributions by an internal vector of unweighted samples, \texttt{Particles}, which is a subtype of a \texttt{Real} number and behaves just like a regular real number in calculations by means of method overloading. This makes the software easy to work with and presents minimal friction for the user. We highlight how this design facilitates optimal usage of SIMD instructions and showcase the package for uncertainty propagation through an off-the-shelf ODE solver, as well as for robust probabilistic optimization with automatic differentiation.

cs.MS↗

arXiv2023

When black box algorithms are (not) appropriate: a principled prediction-problem ontology

In the 1980s a new, extraordinarily productive way of reasoning about algorithms emerged. In this paper, we introduce the term "outcome reasoning" to refer to this form of reasoning. Though outcome reasoning has come to dominate areas of data science, it has been under-discussed and its impact under-appreciated. For example, outcome reasoning is the primary way we reason about whether ``black box'' algorithms are performing well. In this paper we analyze outcome reasoning's most common form (i.e., as "the common task framework") and its limitations. We discuss why a large class of prediction-problems are inappropriate for outcome reasoning. As an example, we find the common task framework does not provide a foundation for the deployment of an algorithm in a real world situation. Building off of its core features, we identify a class of problems where this new form of reasoning can be used in deployment. We purposefully develop a novel framework so both technical and non-technical people can discuss and identify key features of their prediction problem and whether or not it is suitable for outcome reasoning.

stat.OT↗

arXiv2022

Integrating data science ethics into an undergraduate major: A case study

We present a programmatic approach to incorporating ethics into an undergraduate major in statistical and data sciences. We discuss departmental-level initiatives designed to meet the National Academy of Sciences recommendation for integrating ethics into the curriculum from top-to-bottom as our majors progress from our introductory courses to our senior capstone course, as well as from side-to-side through co-curricular programming. We also provide six examples of data science ethics modules used in five different courses at our liberal arts college, each focusing on a different ethical consideration. The modules are designed to be portable such that they can be flexibly incorporated into existing courses at different levels of instruction with minimal disruption to syllabi. We connect our efforts to a growing body of literature on the teaching of data science ethics, present assessments of our effectiveness, and conclude with next steps and final thoughts.

stat.OT↗

arXiv2025

Statistical Consequences of Fat Tails: Real World Preasymptotics, Epistemology, and Applications

(The third edition corrects minor typos and adds 3 chapters synthesized from published papers plus an appendix on maximum entropy distributions.) The monograph investigates the misapplication of conventional statistical techniques to fat tailed distributions and looks for remedies, when possible. Switching from thin tailed to fat tailed distributions requires more than "changing the color of the dress". Traditional asymptotics deal mainly with either n=1 or $n=\infty$, and the real world is in between, under of the "laws of the medium numbers" --which vary widely across specific distributions. Both the law of large numbers and the generalized central limit mechanisms operate in highly idiosyncratic ways outside the standard Gaussian or Levy-Stable basins of convergence. A few examples: + The sample mean is rarely in line with the population mean, with effect on "naive empiricism", but can be sometimes be estimated via parametric methods. + The "empirical distribution" is rarely empirical. + Parameter uncertainty has compounding effects on statistical metrics. + Dimension reduction (principal components) fails. + Inequality estimators (GINI or quantile contributions) are not additive and produce wrong results. + Many "biases" found in psychology become entirely rational under more sophisticated probability distributions + Most of the failures of financial economics, econometrics, and behavioral economics can be attributed to using the wrong distributions. This book, the first volume of the Technical Incerto, weaves a narrative around published journal articles.

stat.OT↗

arXiv2020

A Case Study of Promoting Informal Inferential Reasoning in Learning Sampling Distribution for High School Students

Drawing inference from data is an important skill for students to understand their everyday life, so that the sampling distribution as a central topic in statistical inference is necessary to be learned by the students. However, little is known about how to teach the topic for high school students, especially in Indonesian context. Therefore, the present study provides a teaching experiment to support the students' informal inferential reasoning in understanding the sampling distribution, as well as the students' perceptions toward the teaching experiment. The subjects in the present study were three 11th-grader of one private school in Yogyakarta majoring in mathematics and natural science. The method of data collection was direct observation of sampling distribution learning process, interviews, and documentation. The present study found that that informal inferential reasoning with problem-based learning using contextual problems and real data could support the students to understand the sampling distribution, and they also gave positive responses about their learning experience.

stat.OT↗

arXiv2020

TDEFSI: Theory Guided Deep Learning Based Epidemic Forecasting with Synthetic Information

Influenza-like illness (ILI) places a heavy social and economic burden on our society. Traditionally, ILI surveillance data is updated weekly and provided at a spatially coarse resolution. Producing timely and reliable high-resolution spatiotemporal forecasts for ILI is crucial for local preparedness and optimal interventions. We present TDEFSI (Theory Guided Deep Learning Based Epidemic Forecasting with Synthetic Information), an epidemic forecasting framework that integrates the strengths of deep neural networks and high-resolution simulations of epidemic processes over networks. TDEFSI yields accurate high-resolution spatiotemporal forecasts using low-resolution time series data. During the training phase, TDEFSI uses high-resolution simulations of epidemics that explicitly model spatial and social heterogeneity inherent in urban regions as one component of training data. We train a two-branch recurrent neural network model to take both within-season and between-season low-resolution observations as features, and output high-resolution detailed forecasts. The resulting forecasts are not just driven by observed data but also capture the intricate social, demographic and geographic attributes of specific urban regions and mathematical theories of disease propagation over networks. We focus on forecasting the incidence of ILI and evaluate TDEFSI's performance using synthetic and real-world testing datasets at the state and county levels in the USA. The results show that, at the state level, our method achieves comparable/better performance than several state-of-the-art methods. At the county level, TDEFSI outperforms the other methods. The proposed method can be applied to other infectious diseases as well.

stat.OT↗

arXiv2020

Inverse Ising techniques to infer underlying mechanisms from data

As a problem in data science the inverse Ising (or Potts) problem is to infer the parameters of a Gibbs-Boltzmann distributions of an Ising (or Potts) model from samples drawn from that distribution. The algorithmic and computational interest stems from the fact that this inference task cannot be done efficiently by the maximum likelihood criterion, since the normalizing constant of the distribution (the partition function) can not be calculated exactly and efficiently. The practical interest on the other hand flows from several outstanding applications, of which the most well known has been predicting spatial contacts in protein structures from tables of homologous protein sequences. Most applications to date have been to data that has been produced by a dynamical process which, as far as it is known, cannot be expected to satisfy detailed balance. There is therefore no a priori reason to expect the distribution to be of the Gibbs-Boltzmann type, and no a priori reason to expect that inverse Ising (or Potts) techniques should yield useful information. In this review we discuss two types of problems where progress nevertheless can be made. We find that depending on model parameters there are phases where, in fact, the distribution is close to Gibbs-Boltzmann distribution, a non-equilibrium nature of the under-lying dynamics notwithstanding. We also discuss the relation between inferred Ising model parameters and parameters of the underlying dynamics.

stat.OT↗

arXiv2021

Integrated organic inference (IOI): A reconciliation of statistical paradigms

It is recognised that the Bayesian approach to inference can not adequately cope with all the types of pre-data beliefs about population quantities of interest that are commonly held in practice. In particular, it generally encounters difficulty when there is a lack of such beliefs over some or all the parameters of a model, or within certain partitions of the parameter space concerned. To address this issue, a fairly comprehensive theory of inference is put forward called integrated organic inference that is based on a fusion of Fisherian and Bayesian reasoning. Depending on the pre-data knowledge that is held about any given model parameter, inferences are made about the parameter conditional on all other parameters using one of three methods of inference, namely organic fiducial inference, bispatial inference and Bayesian inference. The full conditional post-data densities that result from doing this are then combined using a framework that allows a joint post-data density for all the parameters to be sensibly formed without requiring these full conditional densities to be compatible. Various examples of the application of this theory are presented. Finally, the theory is defended against possible criticisms partially in terms of what was previously defined as generalised subjective probability.

stat.ME↗