SEARCH · Searcharxiv

Results for “stat.OT”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 883 records · Page 49Linked to original sources

arXiv2024

Exploratory analysis of injury severity under different levels of driving automation (SAE Level 2-5) using multi-source data

Vehicles equipped with automated driving capabilities have shown potential to improve safety and operations. Advanced driver assistance systems (ADAS) and automated driving systems (ADS) have been widely developed to support vehicular automation. Although the studies on the injury severity outcomes that involve automated vehicles are ongoing, there is limited research investigating the difference between injury severity outcomes for the ADAS and ADS equipped vehicles. To ensure a comprehensive analysis, a multi-source dataset that includes 1,001 ADAS crashes (SAE Level 2 vehicles) and 548 ADS crashes (SAE Level 4 vehicles) is used. Two random parameters multinomial logit models with heterogeneity in the means of random parameters are considered to gain a better understanding of the variables impacting the crash injury severity outcomes for the ADAS (SAE Level 2) and ADS (SAE Level 4) vehicles. It was found that while 67 percent of crashes involving the ADAS equipped vehicles in the dataset took place on a highway, 94 percent of crashes involving ADS took place in more urban settings. The model estimation results also reveal that the weather indicator, driver type indicator, differences in the system sophistication that are captured by both manufacture year and high/low mileage as well as rear and front contact indicators all play a role in the crash injury severity outcomes. The results offer an exploratory assessment of safety performance of the ADAS and ADS equipped vehicles using the real-world data and can be used by the manufacturers and other stakeholders to dictate the direction of their deployment and usage.

stat.AP↗

arXiv2023

Using online student focus groups in the development of new educational resources

Educational resources, such as web apps and self-directed tutorials, have become popular tools for teaching and active learning. Ideally, students - the intended users of these resources - should be involved in the resource development stage. However, in practice students often only interact with fully developed resources, when it might be too late to incorporate changes. Previous work has addressed this by involving students in the development of new resources via in-person focus groups and interviews. In these, the resource developers observe students interacting with the resource. This allows developers to incorporate their observations and students' direct feedback into further development of the resource. However, as a result of the COVID-19 pandemic, carrying out in-person focus groups became infeasible due to social distancing restrictions. Instead, online meetings and classes became ubiquitous. In this work, we describe a fully-online methodology to evaluate new resources in development. Specifically, our methodology consists of carrying out student focus groups via online video conferencing software. We assessed two educational resources for introductory statistics using our methodology and found that the online setting allowed us to obtain rich, detailed information from the students. We also found online focus groups to be more efficient: students and researchers did not need to travel and scheduling was not restricted by the availability of physical space. Our findings suggest that online focus groups are an attractive alternative to in-person focus groups for student assessment of resources in development, even now that pandemic restrictions are being eased.

stat.OT↗

arXiv2023

Testing for idiosyncratic Treatment Effect Heterogeneity

This paper provides asymptotically valid tests for the null hypothesis of no treatment effect heterogeneity. Importantly, I consider the presence of heterogeneity that is not explained by observed characteristics, or so-called idiosyncratic heterogeneity. When examining this heterogeneity, common statistical tests encounter a nuisance parameter problem in the average treatment effect which renders the asymptotic distribution of the test statistic dependent on that parameter. I propose an asymptotically valid test that circumvents the estimation of that parameter using the empirical characteristic function. A simulation study illustrates not only the test's validity but its higher power in rejecting a false null as compared to current tests. Furthermore, I show the method's usefulness through its application to a microfinance experiment in Bosnia and Herzegovina. In this experiment and for outcomes related to loan take-up and self-employment, the tests suggest that treatment effect heterogeneity does not seem to be completely accounted for by baseline characteristics. For those outcomes, researchers could potentially try to collect more baseline characteristics to inspect the remaining treatment effect heterogeneity, and potentially, improve treatment targeting.

econ.EM↗

arXiv2023

Communication of Statistics and Evidence in Times of Crisis

This review provides an overview of concepts relating to the communication of statistical and empirical evidence in times of crisis, with a special focus on COVID-19. In it, we consider topics relating both to the communication of numbers -- such as the role of format, context, comparisons, and visualization -- and the communication of evidence more broadly -- such as evidence quality, the influence of changes in available evidence, transparency, and repeated decision making. A central focus is on the communication of the inherent uncertainties in statistical analysis, especially in rapidly changing informational environments during crises. We present relevant literature on these topics and draw connections to the communication of statistics and empirical evidence during the COVID-19 pandemic and beyond. We finish by suggesting some considerations for those faced with communicating statistics and evidence in times of crisis.

stat.OT↗

arXiv2023

Convergent estimators of variance of a spatial mean in the presence of missing observations

In the geosciences, a recurring problem is one of estimating spatial means of a physical field using weighted averages of point observations. An important variant is when individual observations are counted with some probability less than one. This can occur in different contexts: from missing data to estimating the statistics across subsamples. In such situations, the spatial mean is a ratio of random variables, whose statistics involve approximate estimators derived through series expansion. The present paper considers truncated estimators of variance of the spatial mean and their general structure in the presence of missing data. To all orders, the variance estimator depends only on the first and second moments of the underlying field, and convergence requires these moments to be finite. Furthermore, convergence occurs if either the probability of counting individual observations is larger than 1/2 or the number of point observations is large. In case the point observations are weighted uniformly, the estimators are easily found using combinatorics and involve Stirling numbers of the second kind.

math.ST↗

arXiv2023

Individualized Conformal

The problem of individualized prediction can be addressed using variants of conformal prediction, obtaining the intervals to which the actual values of the variables of interest belong. Here we present a method based on detecting the observations that may be relevant for a given question and then using simulated controls to yield the intervals for the predicted values. This method is shown to be adaptive and able to detect the presence of latent relevant variables.

stat.ME↗

arXiv2023

Finite mixtures in capture-recapture surveys for modelling residency patterns in marine wildlife populations

In this work, the goal is to estimate the abundance of an animal population using data coming from capture-recapture surveys. We leverage the prior knowledge about the population's structure to specify a parsimonious finite mixture model tailored to its behavioral pattern. Inference is carried out under the Bayesian framework, where we discuss suitable priors' specification that could alleviate label-switching and non-identifiability issues affecting finite mixtures. We conduct simulation experiments to show the competitive advantage of our proposal over less specific alternatives. Finally, the proposed model is used to estimate the common bottlenose dolphins' population size at the Tiber River estuary (Mediterranean Sea), using data collected via photo-identification from 2018 to 2020. Results provide novel insights on the population's size and structure, and shed light on some of the ecological processes governing the population dynamics.

stat.AP↗

arXiv2024

Studying continuous, time-varying, and/or complex exposures using longitudinal modified treatment policies

This tutorial discusses methodology for causal inference using longitudinal modified treatment policies. This method facilitates the mathematical formalization, identification, and estimation of many novel parameters, and mathematically generalizes many commonly used parameters, such as the average treatment effect. Longitudinal modified treatment policies apply to a wide variety of exposures, including binary, multivariate, and continuous, and can accommodate time-varying treatments and confounders, competing risks, loss-to-follow-up, as well as survival, binary, or continuous outcomes. Longitudinal modified treatment policies can be seen as an extension of static and dynamic interventions to involve the natural value of treatment, and, like dynamic interventions, can be used to define alternative estimands with a positivity assumption that is more likely to be satisfied than estimands corresponding to static interventions. This tutorial aims to illustrate several practical uses of the longitudinal modified treatment policy methodology, including describing different estimation strategies and their corresponding advantages and disadvantages. We provide numerous examples of types of research questions which can be answered using longitudinal modified treatment policies. We go into more depth with one of these examples--specifically, estimating the effect of delaying intubation on critically ill COVID-19 patients' mortality. We demonstrate the use of the open-source R package lmtp to estimate the effects, and we provide code on https://github.com/kathoffman/lmtp-tutorial.

stat.ME↗

arXiv2023

Smart Sports Predictions via Hybrid Simulation: NBA Case Study

Increased data availability has stimulated the interest in studying sports prediction problems via analytical approaches; in particular, with machine learning and simulation. We characterize several models that have been proposed in the literature, all of which suffer from the same drawback: they cannot incorporate rational decision-making and strategies from teams/players effectively. We tackle this issue by proposing hybrid simulation logic that incorporates teams as agents, generalizing the models/methodologies that have been proposed in the past. We perform a case study on the NBA with two goals: i) study the quality of predictions when using only one predictive variable, and ii) study how much historical data should be kept to maximize prediction accuracy. Results indicate that there is an optimal range of data quantity and that studying what data and variables to include is of extreme importance.

stat.OT↗

arXiv2023

A new copula regression model for hierarchical data

This paper proposes multivariate copula models for hierarchical data. They account for two types of correlation: one is between variables measured on the same unit and the other is a correlation between units in the same cluster. This model is used to carry out copula regression for hierarchical data that gives cluster specific prediction curves. In the simple case where a cluster contains two units and where two variables are measured on each one, the new model is constructed within a D-vine. Then we focus on situations where two variables are measured on the units of a cluster of arbitrary size. The proposed copula density has an explicit form; it is expressed in terms of three copula families. We study the properties of the model; compare it to the linear mixed model and end with special cases. When the three copula families and the marginal distributions are normal, the model is equivalent to a normal linear mixed model with random, cluster specific, intercepts. The method to select the three copula families and to estimate their parameters are proposed. We perform a Monte Carlo study of the parameter estimators. A data set on the marks of students in several school is used to implement the proposed model and to compare its performance to standard normal mixed linear models.

stat.ME↗

arXiv2023

Pandemic Data Quality Modelling: A Bayesian Approach

When pandemics like COVID-19 spread around the world, the rapidly evolving situation compels officials and executives to take prompt decisions and adapt policies depending on the current state of the disease. In this context, it is crucial for policymakers to have always a firm grasp on what is the current state of the pandemic, and to envision how the number of infections and possible deaths is going to evolve over the next weeks. However, as in many other situations involving compulsory registration of sensitive data from multiple collectors, cases might be reported with errors, often with delays deferring an up-to-date view of the state of things. Errors in collecting new cases affect the overall mortality, resulting in excess deaths reported by official statistics only months later. In this paper, we provide tools for evaluating the quality of pandemic mortality data. We accomplish this through a Bayesian approach accounting for the excess mortality pandemics might bring with respect to the normal level of mortality in the population.

stat.AP↗

arXiv2023

Using Auxiliary Data to Guide the Recruitment of Sites for Randomized Controlled Trials

Sampling methods such as Stratified Random Sampling can be used to select representative samples of schools for randomized controlled trials of educational interventions. However, these methods may still yield external validity bias when participation by schools is voluntary and participation decisions are associated with unobserved variables. This paper offers a new sampling method called Stratified Random Sampling with Quotas. Under this method, quotas are set to avoid including too many schools of a particular type, as defined by auxiliary variables that are unobserved in the sampling frame, but whose population distribution can be estimated from external data. Our simulations find that when the auxiliary variables affect whether or not a school participates in the study, quotas set based on those variables reduce external validity bias. These results suggest that when auxiliary data are available on strong impact moderators for the target population, these data can be used to address non-ignorable self-selection by schools into randomized controlled trials.

stat.OT↗

arXiv2023

Downscaling Epidemiological Time Series Data for Improving Forecasting Accuracy: An Algorithmic Approach

Data scarcity and discontinuity are common occurrences in the healthcare and epidemiological dataset and often need help in forming an educative decision and forecasting the upcoming scenario. Often, these data are stored as monthly/yearly aggregate where the prevalent forecasting tools like Autoregressive Integrated Moving Average (ARIMA), Seasonal Autoregressive Integrated Moving Average (SARIMA), and TBATS often fail to provide satisfactory results. Artificial data synthesis methods have been proven to be a powerful tool for tackling these challenges. The paper aims to propose a downscaling data algorithm based on the underlying distribution. Our findings show that the synthesized data is in agreement with the original data in terms of trend, seasonality, and residuals, and the synthesized data provides a stable foothold for the forecasting tools to generate a much more accurate forecast of the situation.

stat.OT↗

arXiv2025

Information Theory for Complex Systems Scientists

In the 21st century, many of the crucial scientific and technical issues facing humanity can be understood as problems associated with understanding, modelling, and ultimately controlling complex systems: systems comprised of a large number of non-trivially interacting components whose collective behaviour can be difficult to predict. Information theory, a branch of mathematics historically associated with questions about encoding and decoding messages, has emerged as something of a lingua franca for those studying complex systems, far exceeding its original narrow domain of communication systems engineering. In the context of complexity science, information theory provides a set of tools which allow researchers to uncover the statistical and effective dependencies between interacting components; relationships between systems and their environment; mereological whole-part relationships; and is sensitive to non-linearities missed by commonly parametric statistical models. In this review, we aim to provide an accessible introduction to the core of modern information theory, aimed specifically at aspiring (and established) complex systems scientists. This includes standard measures, such as Shannon entropy, relative entropy, and mutual information, before building to more advanced topics, including: information dynamics, measures of statistical complexity, information decomposition, and effective network inference. In addition to detailing the formal definitions, in this review we make an effort to discuss how information theory can be interpreted and develop the intuition behind abstract concepts like "entropy," in the hope that this will enable interested readers to understand what information is, and how it is used, at a more fundamental level.

cs.IT↗

arXiv2023

Optimal Covariance Cleaning for Heavy-Tailed Distributions: Insights from Information Theory

In optimal covariance cleaning theory, minimizing the Frobenius norm between the true population covariance matrix and a rotational invariant estimator is a key step. This estimator can be obtained asymptotically for large covariance matrices, without knowledge of the true covariance matrix. In this study, we demonstrate that this minimization problem is equivalent to minimizing the loss of information between the true population covariance and the rotational invariant estimator for normal multivariate variables. However, for Student's t distributions, the minimal Frobenius norm does not necessarily minimize the information loss in finite-sized matrices. Nevertheless, such deviations vanish in the asymptotic regime of large matrices, which might extend the applicability of random matrix theory results to Student's t distributions. These distributions are characterized by heavy tails and are frequently encountered in real-world applications such as finance, turbulence, or nuclear physics. Therefore, our work establishes a connection between statistical random matrix theory and estimation theory in physics, which is predominantly based on information theory.

cs.IT↗

arXiv2023

Estimation of the Impact of COVID-19 Pandemic Lockdowns on Breast Cancer Deaths and Costs in Poland using Markovian Monte Carlo Simulation

This study examines the effect of COVID-19 pandemic and associated lockdowns on access to crucial diagnostic procedures for breast cancer patients, including screenings and treatments. To quantify the impact of the lockdowns on patient outcomes and cost, the study employs a mathematical model of breast cancer progression. The model includes ten different states that represent various stages of health and disease, along with the four different stages of cancer that can be diagnosed or undiagnosed. The study employs a natural history stochastic model to simulate the progression of breast cancer in patients. The model includes transition probabilities between states, estimated using both literature and empirical data. The study utilized a Markov Chain Monte Carlo simulation to model the natural history of each simulated patient over a seven-year period from 2019 to 2025. The simulation was repeated 100 times to estimate the variance in outcome variables. The study found that the COVID-19 pandemic and associated lockdowns caused a significant increase in breast cancer costs, with an average rise of 172.5 million PLN (95% CI [82.4, 262.6]) and an additional 1005 breast cancer deaths (95% CI [426, 1584]) in Poland during the simulated period. While these results are preliminary, they highlight the potential harmful impact of lockdowns on breast cancer treatment outcomes and costs.

stat.OT↗

arXiv2023

Defining Replicability of Prediction Rules

In this article I propose an approach for defining replicability for prediction rules. Motivated by a recent NAS report, I start from the perspective that replicability is obtaining consistent results across studies suitable to address the same prediction question, each of which has obtained its own data. I then discuss concept and issues in defining key elements of this statement. I focus specifically on the meaning of "consistent results" in typical utilization contexts, and propose a multi-agent framework for defining replicability, in which agents are neither partners nor adversaries. I recover some of the prevalent practical approaches as special cases. I hope to provide guidance for a more systematic assessment of replicability in machine learning.

stat.ME↗

arXiv2023

The Saltbox-Roof Probability Distribution

The saltbox-roof parametric probability distribution is a special case of the triangular distribution, where only one side is truncated. Here it is presented as a single and independent distribution, where the explicit equations are defined for its probability density--, the cumulative distribution--, and the inverse of the cumulative distribution (quantile--) functions as also its random generator. Four parameters are necessary to define it: the lower and the upper limits, the mode, and a shape parameter. Also, the saltbox-roof distribution degenerates into the uniform distribution, into a kind of a trapezoidal distribution and into other special cases of the general triangular distribution, all of them which are related to the domain of the shape parameter within the mode. The mean, median, and the variance are also here expressed by explicit equations. The function equations have been verified with theorems of truncated distributions. Three application examples are exposed.

stat.OT↗