SEARCH · Searcharxiv

Results for “stat.OT”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35Linked to original sources

arXiv2021

Stick-breaking processes with exchangeable length variables

Our object of study is the general class of stick-breaking processes with exchangeable length variables. These generalize well-known Bayesian non-parametric priors in an unexplored direction. We give conditions to assure the respective species sampling process is proper and the corresponding prior has full support. For a rich sub-class we explain how, by tuning a single $[0,1]$-valued parameter, the stochastic ordering of the weights can be modulated, and Dirichlet and Geometric priors can be recovered. A general formula for the distribution of the latent allocation variables is derived and an MCMC algorithm is proposed for density estimation purposes.

math.ST↗

arXiv2020

A Bayesian Approach to Spherical Factor Analysis for Binary Data

Factor models are widely used across diverse areas of application for purposes that include dimensionality reduction, covariance estimation, and feature engineering. Traditional factor models can be seen as an instance of linear embedding methods that project multivariate observations onto a lower dimensional Euclidean latent space. This paper discusses a new class of geometric embedding models for multivariate binary data in which the embedding space correspond to a spherical manifold, with potentially unknown dimension. The resulting models include traditional factor models as a special case, but provide additional flexibility. Furthermore, unlike other techniques for geometric embedding, the models are easy to interpret, and the uncertainty associated with the latent features can be properly quantified. These advantages are illustrated using both simulation studies and real data on voting records from the U.S. Senate.

stat.ME↗

arXiv2020

Randomization and Fair Judgment in Law and Science

Randomization procedures are used in legal and statistical applications, aiming to shield important decisions from spurious influences. This article gives an intuitive introduction to randomization and examines some intended consequences of its use related to truthful statistical inference and fair legal judgment. This article also presents an open-code Java implementation for a cryptographically secure, statistically reliable, transparent, traceable, and fully auditable randomization tool.

stat.OT↗

arXiv2020

Tackling COVID-19 through Responsible AI Innovation: Five Steps in the Right Direction

Innovations in data science and AI/ML have a central role to play in supporting global efforts to combat COVID-19. The versatility of AI/ML technologies enables scientists and technologists to address an impressively broad range of biomedical, epidemiological, and socioeconomic challenges. This wide-reaching scientific capacity, however, also raises a diverse array of ethical challenges. The need for researchers to act quickly and globally in tackling SARS-CoV-2 demands unprecedented practices of open research and responsible data sharing at a time when innovation ecosystems are hobbled by proprietary protectionism, inequality, and a lack of public trust. Moreover, societally impactful interventions like digital contact tracing are raising fears of surveillance creep and are challenging widely held commitments to privacy, autonomy, and civil liberties. Prepandemic concerns that data-driven innovations may function to reinforce entrenched dynamics of societal inequity have likewise intensified given the disparate impact of the virus on vulnerable social groups and the life-and-death consequences of biased and discriminatory public health outcomes. To address these concerns, I offer five steps that need to be taken to encourage responsible research and innovation. These provide a practice-based path to responsible AI/ML design and discovery centered on open, accountable, equitable, and democratically governed processes and products. When taken from the start, these steps will not only enhance the capacity of innovators to tackle COVID-19 responsibly, they will, more broadly, help to better equip the data science and AI/ML community to cope with future pandemics and to support a more humane, rational, and just society.

cs.CY↗

arXiv2022

Presenting the Probabilities of Different Effect Sizes: Towards a Better Understanding and Communication of Statistical Uncertainty

How should social scientists understand and communicate the uncertainty of statistically estimated causal effects? I propose we utilize the posterior distribution of a causal effect and present the probability of the effect being greater (in absolute terms) than different minimum effect sizes. Probability is an intuitive measure of uncertainty for understanding and communication. In addition, the proposed approach needs no decision threshold for an uncertainty measure or an effect size, unlike the conventional approaches. I apply the proposed approach to a previous social scientific study, showing it enables richer inference than the significance-vs.-insignificance approach taken by the original study. The accompanying R package makes my approach easy to implement.

stat.AP↗

arXiv2020

The use of multiple models within an organisation

Organisations, whether in government, industry or commerce, are required to make decisions in a complex and uncertain environment. The way models are used is intimately connected to the way organisations make decisions and the context in which they make them. Typically, in a complex organisation, multiple related models will often be used in support of a decision. For example, engineering models might be combined with financial models and macro-economic models in order to decide whether to invest in new production capability. Different parts of a complex organisation might operate their own related models which might then be presented to a central decision maker. Yet in practice, there is little awareness of the practical challenges of using models in a robust way to support decision making. There is significant scope to improve decision making though an enhanced understanding of the role and limitations of modelling and through the application of cutting edge methodologies and organisational best practice. This report is in the form of a 'white paper', whose purpose is to identify key issues for consideration whist postulating tentative approaches to these issues that might be worthy of further exploration, focussing on both technical and organisational aspects. It begins with a framework for consideration of how model-based decisions are made in organisations. It then looks more closely at the questions of uncertainty and multiple models. It then postulates some technical statistical and organisational approaches for managing some of these issues. Finally, it considers the way forward, and the possible focus for further work.

stat.OT↗

arXiv2023

Non-asymptotic robustness analysis of regression depth median

The maximum depth estimator (aka depth median) ($\bsβ^*_{RD}$) induced from regression depth (RD) of Rousseeuw and Hubert (1999) (RH99) is one of the most prevailing estimators in regression. It possesses outstanding robustness similar to the univariate location counterpart. Indeed, $\bsβ^*_{RD}$ can, asymptotically, resist up to $33\%$ contamination without breakdown, in contrast to the $0\%$ for the traditional (least squares and least absolute deviations) estimators (see Van Aelst and Rousseeuw, 2000) (VAR00)). The results from VAR00 are pioneering, yet they are limited to regression-symmetric populations (with a strictly positive density) and the $ε$-contamination and maximum-bias model. With a fixed finite-sample size practice, the most prevailing measure of robustness for estimators is the finite-sample breakdown point (FSBP) (Donoho and Huber (1983)). Despite many attempts made in the literature, only sporadic partial results on FSBP for $\bsβ^*_{RD}$ were obtained whereas an exact FSBP for $\bsβ^*_{RD}$ remained open in the last twenty-plus years. Furthermore, is the asymptotic breakdown value $1/3$ (the limit of an increasing sequence of finite-sample breakdown values) relevant in the finite-sample practice? (Or what is the difference between the finite-sample and the limit breakdown values?). Such discussions are yet to be given in the literature. This article addresses the above issues, revealing an intrinsic connection between the regression depth of $\bsβ^*_{RD}$ and the newly obtained exact FSBP. It justifies the employment of $\bsβ^*_{RD}$ as a robust alternative to the traditional estimators and demonstrates the necessity and the merit of using the FSBP in finite-sample real practice.

math.ST↗

arXiv2022

Defending the P-value

Attacks on the P-value are nothing new, but the recent attacks are increasingly more serious. They come from more mainstream sources, with widening targets such as a call to retire the significance testing altogether. While well meaning, I believe these attacks are nevertheless misdirected: Blaming the P-value for the naturally tentative trial-and-error process of scientific discoveries, and presuming that banning the P-value would make the process cleaner and less error-prone. However tentative, the skeptical scientists still have to form unambiguous opinions, proximately to move forward in their investigations and ultimately to present results to the wider community. With obvious reasons, they constantly need to balance between the false-positive and false-negative errors. How would banning the P-value or significance tests help in this balancing act? It seems trite to say that this balance will always depend on the relative costs or the trade-off between the errors. These costs are highly context specific, varying by area of applications or by stage of investigation. A calibrated but tunable knob, such as that given by the P-value, is needed for controlling this balance. This paper presents detailed arguments in support of the P-value.

stat.OT↗

arXiv2020

Variance decompositions for extensive-form games

Quantitative measures of randomness in games are useful for game design and have implications for gambling law. We treat the outcome of a game as a random variable and derive a closed-form expression and estimator for the variance in the outcome attributable to a player of the game. We analyze poker hands to show that randomness in the cards dealt has little influence on the outcomes of each hand. A simple example is given to demonstrate how variance decompositions can be used to measure other interesting properties of games.

stat.OT↗

arXiv2020

A Study on the Possible Effects of the Implementation of the Nordic Model in India on Crime Rates and Sexually Transmitted Diseases

Prostitution is one of the root causes of sex trafficking and the transmission of sexual diseases. The rules and regulations followed by the Indian government to regulate the same, fall under the umbrella of the abolitionism model. Neo-abolitionism (also known as the Nordic model) is a new legislative model that has been introduced by the Nordic countries to regulate prostitution. The purpose of this research paper is to examine the possible effects of the application of the Nordic model on the crime rates and the spread of sexually transmitted diseases in India. Further, we also aim to study the effects of the implementation of Neo-abolitionism in Sweden.

stat.AP↗

arXiv2020

Multilevel regression with poststratification for the national level Viber/Street poll on the 2020 presidential election in Belarus

Independent sociological polls are forbidden in Belarus. Online polls performed without sound scientific rigour do not yield representative results. Yet, both inside and outside Belarus it is of great importance to obtain precise estimates of the ratings of all candidates. These ratings could function as reliable proxies for the election's outcomes. We conduct an independent poll based on the combination of the data collected via Viber and on the streets of Belarus. The Viber and the street data samples consist of almost 45000 and 1150 unique observations respectively. Bayesian regressions with poststratification were build to estimate ratings of the candidates and rates of early voting turnout for the population as a whole and within various focus subgroups. We show that both the officially announced results of the election and early voting rates are highly improbable. With a probability of at least 95%, Sviatlana Tikhanouskaya's rating lies between 75% and 80%, whereas Aliaksandr Lukashenka's rating lies between 13% and 18% and early voting rate predicted by the method ranges from 9% to 13% of those who took part in the election. These results contradict the officially announced outcomes, which are 10.12%, 80.11%, and 49.54% respectively and lie far outside even the 99.9% credible intervals predicted by our model. The only marginal groups of people where the upper bounds of the 99.9% credible intervals of the rating of Lukashenka are above 50% are people older than 60 and uneducated people. For all other marginal subgroups, including rural residents, even the upper bounds of 99.9% credible intervals for Lukashenka are far below 50%. The same is true for the population as a whole. Thus, with a probability of at least 99.9% Lukashenka could not have had enough electoral support to win the 2020 presidential election in Belarus.

stat.AP↗

arXiv2021

Using Known Words to Learn More Words: A Distributional Analysis of Child Vocabulary Development

Why do children learn some words before others? Understanding individual variability across children and also variability across words, may be informative of the learning processes that underlie language learning. We investigated item-based variability in vocabulary development using lexical properties of distributional statistics derived from a large corpus of child-directed speech. Unlike previous analyses, we predicted word trajectories cross-sectionally, shedding light on trends in vocabulary development that may not have been evident at a single time point. We also show that whether one looks at a single age group or across ages as a whole, the best distributional predictor of whether a child knows a word is the number of other known words with which that word tends to co-occur. Keywords: age of acquisition; vocabulary development; lexical diversity; child-directed speech;

cs.CL↗

arXiv2021

A note on the closed-form solution for the longest head run problem of Abraham de Moivre

The problem of the longest head run was introduced and solved by Abraham de Moivre in the second edition of his book Doctrine of Chances (de Moivre, 1738). The closed-form solution as a finite sum involving binomial coefficients was provided in Uspensky (1937). Since then, the problem and its variations and extensions have found broad interest and diverse applications. Surprisingly, a very simple closed form can be obtained, which we present in this note.

math.HO↗

arXiv2020

Risk upper bounds for RKHS ridge group sparse estimator in the regression model with non-Gaussian and non-bounded error

We consider the problem of estimating a meta-model of an unknown regression model with non-Gaussian and non-bounded error. The meta-model belongs to a reproducing kernel Hilbert space constructed as a direct sum of Hilbert spaces leading to an additive decomposition including the variables and interactions between them. The estimator of this meta-model is calculated by minimizing an empirical least-squares criterion penalized by the sum of the Hilbert norm and the empirical $L^2$-norm. In this context, the upper bounds of the empirical $L^2$ risk and the $L^2$ risk of the estimator are established.

math.ST↗

arXiv2020

Dealing with multiple testing: To adjust or not to adjust

Multiple testing problems arise naturally in scientific studies because of the need to capture or convey more information with more variables. The literature is enormous, but the emphasis is primarily methodological, providing numerous methods with their mathematical justification and practical implementation. Our aim is to highlight the logical issues involved in the application of multiple testing adjustment.

stat.OT↗

arXiv2020

Likelihood-based solution to the Monty Hall puzzle and a related 3-prisoner paradox

The Monty Hall puzzle has been solved and dissected in many ways, but always using probabilistic arguments, so it is considered a probability puzzle. In this paper the puzzle is set up as an orthodox statistical problem involving an unknown parameter, a probability model and an observation. This means we can compute a likelihood function, and the decision to switch corresponds to choosing the maximum likelihood solution. One advantage of the likelihood-based solution is that the reasoning applies to a single game, unaffected by the future plan of the host. I also describe an earlier version of the puzzle in terms of three prisoners: two to be executed and one released. Unlike the goats and the car, these prisoners have consciousness, so they can think about exchanging punishments. When two of them do that, however, we have a paradox, where it is advantageous for both to exchange their punishment with each other. Overall, the puzzle and the paradox are useful examples of statistical thinking, so they are excellent teaching topics.

stat.OT↗

arXiv2020

A spatial functional count model for heterogeneity analysis in time

A spatial curve dynamical model framework is adopted for functional prediction of counts in a spatiotemporal log-Gaussian Cox process model. Our spatial functional estimation approach handles both wavelet-based heterogeneity analysis in time, and spectral analysis in space. Specifically, model fitting is achieved by minimising the information divergence or relative entropy between the multiscale model underlying the data and the corresponding candidates in the spatial spectral domain. A simulation study is carried out within the family of log-Gaussian Spatial Autoregressive l2-valued processes (SARl2 processes) to illustrate the asymptotic properties of the proposed spatial functional estimators. We apply our modelling strategy to spatiotemporal prediction of respiratory disease mortality.

stat.ME↗

arXiv2020

Computational Skills by Stealth in Secondary School Data Science

The unprecedented growth in the availability of data of all types and qualities and the emergence of the field of data science has provided an impetus to finally realizing the implementation of the full breadth of the Nolan and Temple Lang proposed integration of computing concepts into statistics curricula at all levels in statistics and new data science programs and courses. Moreover, data science, implemented carefully, opens accessible pathways to stem for students for whom neither mathematics nor computer science are natural affinities, and who would traditionally be excluded. We discuss a proposal for the stealth development of computational skills in students' first exposure to data science through careful, scaffolded exposure to computation and its power. The intent of this approach is to support students, regardless of interest and self-efficacy in coding, in becoming data-driven learners, who are capable of asking complex questions about the world around them, and then answering those questions through the use of data-driven inquiry. This discussion is presented in the context of the International Data Science in Schools Project which recently published computer science and statistics consensus curriculum frameworks for a two-year secondary school data science program, designed to make data science accessible to all.

cs.CY↗