Searcharxiv⌕ Search

SEARCH · Searcharxiv

Results for “stat.OT”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31Linked to original sources
arXiv2020

Descriptive and Predictive Analysis of Euroleague Basketball Games and the Wisdom of Basketball Crowds

In this study we focus on the prediction of basketball games in the Euroleague competition using machine learning modelling. The prediction is a binary classification problem, predicting whether a match finishes 1 (home win) or 2 (away win). Data is collected from the Euroleague's official website for the seasons 2016-2017, 2017-2018 and 2018-2019, i.e. in the new format era. Features are extracted from matches' data and off-the-shelf supervised machine learning techniques are applied. We calibrate and validate our models. We find that simple machine learning models give accuracy not greater than 67% on the test set, worse than some sophisticated benchmark models. Additionally, the importance of this study lies in the "wisdom of the basketball crowd" and we demonstrate how the predicting power of a collective group of basketball enthusiasts can outperform machine learning models discussed in this study. We argue why the accuracy level of this group of "experts" should be set as the benchmark for future studies in the prediction of (European) basketball games using machine learning.

stat.ML↗
arXiv2020

Online Statistics Teaching and Learning

For statistics courses at all levels, teaching and learning online poses challenges in different aspects. Particular online challenges include how to effectively and interactively conduct exploratory data analyses, how to incorporate statistical programming, how to include individual or team projects, and how to present mathematical derivations efficiently and effectively. This article draws from the authors' experience with seven different online statistics courses to address some of the aforementioned challenges. One course is an online exploratory data analysis course taught at Bowling Green State University. A second course is an upper level Bayesian statistics course taught at Vassar College and shared among 10 liberal arts colleges through a hybrid model. We alo describes a five-course MOOC specialization on Coursera, offered by Duke University.

stat.OT↗
arXiv2026

Examples of statistical practice in elementary particle physics for comparison with Mayo's severity concept

The field of elementary particle physics (also known as high energy physics) has strong traditions in frequentist inference, heavily influenced by Fisher and by Neyman and Pearson. While I am not aware of direct use and citation of Deborah Mayo's severity concept in the field, one might expect that ideas similar to severity were independently developed. Here, I describe typical practice that addresses the same issues that severity was developed to address, with some tentative comparisons with severity. I invite experts on severity to explain how they would apply severity in these cases and to discuss any differences.

stat.OT↗
arXiv2020

Liquid Scorecards

Traditional credit scorecards are generalized additive models (GAMs) with step functions as the component functions. The shapes of the step functions may be constrained in order to satisfy the PILE (Palatability, Interpretability, Legal, Explain-ability) constraints. Before 2003, FICO used Linear Programming to find the traditional scorecard that approximately maximizes divergence subject to the PILE constraints. In this paper, I introduce the Liquid Scorecard, that allows the component functions to be, at least partially, smooth curves. I use Quadratic Programming and B-Spline theory to find the Liquid Scorecard that exactly maximizes divergence subject to the PILE constraints. FICO uses aspects of this technology to develop the famous FICO Credit Score.

stat.OT↗
arXiv2020

"Playing the whole game": A data collection and analysis exercise with Google Calendar

We provide a computational exercise suitable for early introduction in an undergraduate statistics or data science course that allows students to 'play the whole game' of data science: performing both data collection and data analysis. While many teaching resources exist for data analysis, such resources are not as abundant for data collection given the inherent difficulty of the task. Our proposed exercise centers around student use of Google Calendar to collect data with the goal of answering the question 'How do I spend my time?' On the one hand, the exercise involves answering a question with near universal appeal, but on the other hand, the data collection mechanism is not beyond the reach of a typical undergraduate student. A further benefit of the exercise is that it provides an opportunity for discussions on ethical questions and considerations that data providers and data analysts face in today's age of large-scale internet-based data collection.

stat.OT↗
arXiv2021

Criteria for the numerical constant recognition

The need for recognition/approximation of functions in terms of elementary functions/operations emerges in many areas of experimental mathematics, numerical analysis, computer algebra systems, model building, machine learning, approximation and data compression. One of the most underestimated methods is the symbolic regression. In the article, reductionist approach is applied, reducing full problem to constant functions, i.e, pure numbers (decimal, floating-point). However, existing solutions are plagued by lack of solid criteria distinguishing between random formula, matching approximately or literally decimal expansion and probable ''exact'' (the best) expression match in the sense of Occam's razor. In particular, convincing STOP criteria for search were never developed. In the article, such a criteria, working in statistical sense, are provided. Recognition process can be viewed as (1) enumeration of all formulas in order of increasing Kolmogorov complexity K (2) random process with appropriate statistical distribution (3) compression of a decimal string. All three approaches are remarkably consistent, and provide essentially the same limit for practical depth of search. Tested unique formulas count must not exceed 1/sigma, where sigma is relative numerical error of the target constant. Beyond that, further search is pointless, because, in the view of approach (1), number of equivalent expressions within error bounds grows exponentially; in view of (2), probability of random match approaches 1; in view of (3) compression ratio much smaller than 1.

cs.DM↗
arXiv2020

What Does the "Mean" Really Mean?

The arithmetic average of a collection of observed values of a homogeneous collection of quantities is often taken to be the most representative observation. There are several arguments supporting this choice the moment of inertia being the most familiar. But what does this mean? In this note, we bring forth the Kolmogorov-Nagumo point of view that the arithmetic average is a special case of a sequence of functions of a special kind, the quadratic and the geometric means being some of the other cases. The median fails to belong to this class of functions. The Kolmogorov-Nagumo interpretation is the most defensible and the most definitive one for the arithmetic average, but its essence boils down to the fact that this average is merely an abstraction which has meaning only within its mathematical set-up.

stat.OT↗
arXiv2020

Exploiting disagreement between high-dimensional variable selectors for uncertainty visualization

We propose Combined Selection and Uncertainty Visualizer (CSUV), which estimates the set of true covariates in high-dimensional linear regression and visualizes selection uncertainties by exploiting the (dis)agreement among different base selectors. Our proposed method selects covariates that get selected the most frequently by the different variable selection methods on subsampled data. The method is generic and can be used with different existing variable selection methods. We demonstrate its variable selection performance using real and simulated data. The variable selection method and its uncertainty illustration tool are publicly available as R package CSUV (https://github.com/christineyuen/CSUV). The graphical tool is also available online via https://csuv.shinyapps.io/csuv

stat.ME↗
arXiv2021

Asymptotic relatively more efficient test with auxiliary information: the case of the $Z$-test and the chi-square test

The main goal of this article is to study how an auxiliary information can be used to improve the efficiency of two famous statistical tests: the $ Z$-test and the chi-square test. Many definitions of auxiliary information can be found in the statistical literature. In this article, the notion of auxiliary information is discussed from a very general point of view and depends on the relevant test. These two statistical tests are modified so that this information is taken into account. It is shown in particular that the efficiency of these new tests is improved in the sense of Pitman's ARE. Some statistical examples illustrate the use of this method.

math.ST↗
arXiv2020

Bernoulli Trials With Skewed Propensities for Certification and Validation

The impetus for writing this paper are the well publicized media reports that software failure was the cause of the two recent mishaps of the Boeing 737 Max aircraft. The problem considered here though, is a specific one, in the sense that it endeavors to address the general matter of conditions under which an item such as a drug, a material specimen, or a complex, system can be certified for use based on a large number of Bernoulli trials, all successful. More broadly, the paper is an attempt to answer the old and honorable philosophical question, namely," when can empirical testing on its own validate a law of nature?" Our message is that the answer depends on what one starts with, namely, what is one's prior distribution, what unknown does this prior distribution endow, and what has been observed as data. The paper is expository in that it begins with a historical overview, and ends with some new ideas and proposals for addressing the question posed. In the sequel, it also articulates on Popper's notion of "propensity" and its role in providing a proper framework for Bayesian inference under Bernoulli trials, as well as the need to engage with posterior distributions that are subjectively specified; that is, without a recourse to the usual Bayesian prior to posterior iteration.

stat.OT↗
arXiv2020

How and Why Did Probability Theory Come About?

This paper is a top down historical perspective on the several phases in the development of probability from its prehistoric origins to its modern day evolution, as one of the key methodologies in artificial intelligence, data science, and machine learning. It is written in honor of Barry Arnold's birthday for his many contributions to statistical theory and methodology. Despite the fact that much of Barry's work is technical, a descriptive document to mark his achievements should not be viewed as being out of line. Barry's dissertation adviser at Stanford (he received a Ph.D. in Statistics there) was a philosopher of Science who dug deep in the foundations and roots of probability, and it is this breadth of perspective is what Barry has inherent. The paper is based on lecture materials compiled by the first author from various published sources, and over a long period of time. The material below gives a limited list of references, because the cast of characters is many, and their contributions are a part of the historical heritage of those of us who are interested in probability, statistics, and the many topics they have spawned.

stat.OT↗
arXiv2020

Flight restrictions from China during the COVID-2019 Coronavirus outbreak

This short note provides estimates of the number of passengers that travel from China to all world airports in the period October 2019 - March 2020 on the basis of historical data. From this baseline we subtract the expected reduction in the number of passengers taking into account the temporary ban of some routes which was put in place since 23 January 2020 following the COVID-2019 Coronavirus outbreak. The results indicate a reduction of the number of passengers in the period January - March 2020 of -2.5%. This calculation considers only the complete closure of routes (not just direct flights) and not the reduction in the number of passengers on still active direct and indirect connections. At the moment of writing, with such partial information it is premature to quantify economic losses on the civil air transport and tourism industry. This note is meant to provide a baseline that be extended to all countries of origin and updated as more recent data will become available.

physics.soc-ph↗
arXiv2020

Bayes' Theorem under Conditional Independence

In this article we provide a substantial discussion on the statistical concept of conditional independence, which is not routinely mentioned in most elementary statistics and mathematical statistics textbooks. Under the assumption of conditional independence, an extended version of Bayes' Theorem is then proposed with illustrations from both hypothetical and real-world examples of disease diagnosis.

stat.OT↗
arXiv2021

Level set and density estimation on manifolds

We tackle the problem of the estimation of the level sets L_f(λ) of the density f of a random vector X supported on a smooth manifold M\subsetR^d , from an iid sample of X. To do that we introduce a kernel-based estimator f^n,h , which is a slightly modified version of the one proposed in [45], and proves its a.s. uniform convergence to f . Then, we propose two estimators of L f (λ), the first one is a plug-in: L f^n,h (λ), which is proven to be a.s. consistent in Hausdorff distance and distance in measure, if L f(λ) does not meet the boundary of M . While the second one assumes that L f(λ) is r-convex, and is estimated by means of the r-convex hull of L f^n,h(λ). The performance of our proposal is illustrated through some simulated examples. In a real data example we analyze the intensity and direction of strong and moderate winds.

math.ST↗
arXiv2020

An R Autograder for PrairieLearn

We describe how we both use and extend the PrarieLearn framework by taking advantage of its built-in support for external auto-graders. By using a custom Docker container, we can match our course requirements perfectly. Moreover, by relying on the flexibility of the interface we can customize our Docker container. A specific extension for unit testing is described which creates context-dependent difference between student answers and reference solution providing a more comprehensive response at test time.

stat.OT↗
arXiv2020

On new data sources for the production of official statistics

In the past years we have witnessed the rise of new data sources for the potential production of official statistics, which, by and large, can be classified as survey, administrative, and digital data. Apart from the differences in their generation and collection, we claim that their lack of statistical metadata, their economic value, and their lack of ownership by data holders pose several entangled challenges lurking the incorporation of new data into the routinely production of official statistics. We argue that every challenge must be duly overcome in the international community to bring new statistical products based on these sources. These challenges can be naturally classified into different entangled issues regarding access to data, statistical methodology, quality, information technologies, and management. We identify the most relevant to be necessarily tackled before new data sources can be definitively considered fully incorporated into the production of official statistics.

stat.OT↗
arXiv2020

Space Filling Split Plot Design using Fast Flexible Filling

In this article, an adaption of an algorithm for the creation of experimental designs by Lekivetz and Jones (2015) is suggested, dealing with constraints around randomization. Split-plot design of experiments is used, when the levels of some factors cannot be modified as easily as others. While most split-plot designs deal in the context of I-optimal or D-optimal designs for continuous response outputs, a space filling design strategy is suggested in here. The proposed designs are evaluated based on different design criteria, as well as an analytical example.

stat.ME↗