SEARCH · Searcharxiv

Results for “stat.OT”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,009 records · Page 56Linked to original sources

arXiv2024

Active Learning for Fair and Stable Online Allocations

We explore an active learning approach for dynamic fair resource allocation problems. Unlike previous work that assumes full feedback from all agents on their allocations, we consider feedback from a select subset of agents at each epoch of the online resource allocation process. Despite this restriction, our proposed algorithms provide regret bounds that are sub-linear in number of time-periods for various measures that include fairness metrics commonly used in resource allocation problems and stability considerations in matching mechanisms. The key insight of our algorithms lies in adaptively identifying the most informative feedback using dueling upper and lower confidence bounds. With this strategy, we show that efficient decision-making does not require extensive feedback and produces efficient outcomes for a variety of problem classes.

cs.LG↗

arXiv2024

Transfer Learning for High Dimensional Robust Regression

Transfer learning has become an essential technique for utilizing information from source datasets to improve the performance of the target task. However, in the context of high-dimensional data, heterogeneity arises due to heteroscedastic variance or inhomogeneous covariate effects. To solve this problem, this paper proposes a robust transfer learning based on the Huber regression, specifically designed for scenarios where the transferable source data set is known. This method effectively mitigates the impact of data heteroscedasticity, leading to improvements in estimation and prediction accuracy. Moreover, when the transferable source data set is unknown, the paper introduces an efficient detection algorithm to identify informative sources. The effectiveness of the proposed method is proved through numerical simulation and empirical analysis using superconductor data.

stat.ME↗

arXiv2024

Stock Volume Forecasting with Advanced Information by Conditional Variational Auto-Encoder

We demonstrate the use of Conditional Variational Encoder (CVAE) to improve the forecasts of daily stock volume time series in both short and long term forecasting tasks, with the use of advanced information of input variables such as rebalancing dates. CVAE generates non-linear time series as out-of-sample forecasts, which have better accuracy and closer fit of correlation to the actual data, compared to traditional linear models. These generative forecasts can also be used for scenario generation, which aids interpretation. We further discuss correlations in non-stationary time series and other potential extensions from the CVAE forecasts.

q-fin.ST↗

arXiv2024

Fractal dimension, and the problems traps of its estimation

This chapter deals with error and uncertainty in data. Treats their measuring methods and meaning. It shows that uncertainty is a natural property of many data sets. Uncertainty is fundamental for the survival os living species, Uncertainty of the "chaos" type occurs in many systems, is fundamental to understand these systems.

stat.OT↗

arXiv2025

Reducing Total Trip Time and Vehicle Emission through Park-and-Ride -- methods and case-study

This study addresses important issues of traffic congestion and vehicle emissions in urban areas by developing a comprehensive mathematical framework to evaluate Park-and-Ride (PnR) systems. The proposed approach integrates queueing theory and emissions modeling to simultaneously assess waiting times, travel times, and vehicle emissions under various PnR usage scenarios. The methodology employs a novel combination of Monte Carlo simulation and matrix geometric analytic methods to analyze a queueing network representing PnR facilities and road traffic. A case study of Tsukuba, Japan demonstrates the model's applicability, revealing potential reductions in social costs related to total trip time and emissions through optimized PnR policies. Specifically, the study found that implementing optimal bus frequency and capacity policies could reduce total social costs by up to 30\% compared to current conditions. This research contributes to the literature by providing a unified framework for evaluating PnR systems that considers both time and environmental costs, offering valuable insights for urban planners and policymakers seeking to improve transportation sustainability. The proposed model utilizes a single server queue with a deterministic service time and multiple arrival streams to represent traffic flow, incorporating both private cars and public buses. Emissions are calculated using the Methodologies for Estimating Air Pollutant Emissions from Transport (MEET) framework. The social cost of emissions and total trip time (SCETT) is introduced as a comprehensive metric for evaluating PnR system performance.

stat.CO↗

arXiv2024

The Need for a Recurring Large-Scale Benchmarking Survey to Continually Evaluate Sampling Methods and Administration Modes: Lessons from the 2022 Collaborative Midterm Survey

As survey methods adapt to technological and societal changes, a growing body of research seeks to understand the tradeoffs associated with various sampling methods and administration modes. We show how the NSF-funded 2022 Collaborative Midterm Survey (CMS) can be used as a dynamic and transparent framework for evaluating which sampling approaches - or combination of approaches - are best suited for various research goals. The CMS is ideally suited for this purpose because it includes almost 20,000 respondents interviewed using two administration modes (phone and online) and data drawn from random digit dialing, random address-based sampling, a probability-based panel, two nonprobability panels, and two nonprobability marketplaces. The analysis considers three types of population benchmarks (election data, administrative records, and large government surveys) and focuses on the national-level estimates as well as oversamples in three states (California, Florida, and Wisconsin). In addition to documenting how each of the survey strategies performed, we develop a strategy to assess how different combinations of approaches compare to different population benchmarks in order to guide researchers combining sampling methods and sources. We conclude by providing specific recommendations to public opinion and election survey researchers and demonstrating how our approach could be applied to a large government survey conducted at regular intervals to provide ongoing guidance to researchers, government, businesses, and nonprofits regarding the most appropriate survey sampling and administration methods.

stat.OT↗

arXiv2024

More than Formulas -- Integrity, Communication, Computing and Reproducibility in Statistics Education

This paper introduces a novel course design in the Master Program in Biostatistics at the University of Zurich that integrates computing skills, effective communication, reproducibility, and scientific integrity within one course. Utilizing a flipped classroom model, the course aims to equip students with the necessary competencies to handle real-world data analysis challenges and effective statistical practice in general. The curriculum includes practical tools such as version control with Git, dynamic reporting, unit testing and containerization to foster reproducibility, and integrity in statistical practice. Feedback gathered from both staff and students post-implementation indicates that the course significantly enhances student readiness for professional and academic environments, demonstrating the effectiveness of this educational approach.

stat.OT↗

arXiv2024

Toward a Complete Criterion for Value of Information in Insoluble Decision Problems

In a decision problem, observations are said to be material if they must be taken into account to perform optimally. Decision problems have an underlying (graphical) causal structure, which may sometimes be used to evaluate certain observations as immaterial. For soluble graphs - ones where important past observations are remembered - there is a complete graphical criterion; one that rules out materiality whenever this can be done on the basis of the graphical structure alone. In this work, we analyse a proposed criterion for insoluble graphs. In particular, we prove that some of the conditions used to prove immateriality are necessary; when they are not satisfied, materiality is possible. We discuss possible avenues and obstacles to proving necessity of the remaining conditions.

stat.OT↗

arXiv2024

Alternative proof for the bias of the hot hand statistic of streak length one

For a sequence of $n$ random variables taking values $0$ or $1$, the hot hand statistic of streak length $k$ counts what fraction of the streaks of length $k$, that is, $k$ consecutive variables taking the value $1$, among the $n$ variables are followed by another $1$. Since this statistic does not use the expected value of how many streaks of length $k$ are observed, but instead uses the realization of the number of streaks present in the data, it may be a biased estimator of the conditional probability of a fixed random variable taking value $1$ if it is preceded by a streak of length $k$, as was first studied and observed explicitly in [Miller and Sanjurjo, 2018]. In this short note, we suggest an alternative proof for an explicit formula of the expectation of the hot hand statistic for the case of streak length one. This formula was obtained through a different argument in [Miller and Sanjurjo, 2018] and [Rinott and Bar-Hillel, 2015].

stat.OT↗

arXiv2025

The Future of Data Science Education

The definition of Data Science is a hotly debated topic. For many, the definition is a simple shortcut to Artificial Intelligence or Machine Learning. However, there is far more depth and nuance to the field of Data Science than a simple shortcut can provide. The School of Data Science at the University of Virginia has developed a novel model for the definition of Data Science. This model is based on identifying a unified understanding of the data work done across all areas of Data Science. It represents a generational leap forward in how we understand and teach Data Science. In this paper we will present the core features of the model and explain how it unifies various concepts going far beyond the analytics component of AI. From this foundation we will present our Undergraduate Major curriculum in Data Science and demonstrate how it prepares students to be well-rounded Data Science team members and leaders. The paper will conclude with an in-depth overview of the Foundations of Data Science course designed to introduce students to the field while also implementing proven STEM oriented pedagogical methods. These include, for example, specifications grading, active learning lectures, guest lectures from industry experts and weekly gamification labs.

stat.OT↗

arXiv2024

Identifying Research Hotspots and Future Development Trends in Current Psychology: A Bibliometric Analysis of the Past Decade's Publications

By conducting a bibliometric analysis on 4,869 publications in Current Psychology from 2013 to 2022, this paper examined the annual publications and annual citations, as well as the leading institutions, countries, and keywords. CiteSpace, VOSviewer and SCImago Graphica were utilized for visualization analysis. On one hand, this paper analyzed the academic influence of Current Psychology over the past decade. On the other hand, it explored the research hotspots and future development trends within the field of international psychology. The results revealed that the three main research areas covered in the publications of Current Psychology were: the psychological well-being of young people, the negative emotions of adults, and self-awareness and management. The latest research hotspots highlighted in the journal include negative emotions, personality, and mental health. The three main development trends of Current Psychology are: 1) exploring the personality psychology of both adolescents and adults, 2) promoting the interdisciplinary research to study social psychological issues through the use of diversified research methods, and 3) emphasizing the emotional psychology of individuals and their interaction with social reality, from a people-oriented perspective.

cs.DL↗

arXiv2024

FAVis: Visual Analytics of Factor Analysis for Psychological Research

Psychological research often involves understanding psychological constructs through conducting factor analysis on data collected by a questionnaire, which can comprise hundreds of questions. Without interactive systems for interpreting factor models, researchers are frequently exposed to subjectivity, potentially leading to misinterpretations or overlooked crucial information. This paper introduces FAVis, a novel interactive visualization tool designed to aid researchers in interpreting and evaluating factor analysis results. FAVis enhances the understanding of relationships between variables and factors by supporting multiple views for visualizing factor loadings and correlations, allowing users to analyze information from various perspectives. The primary feature of FAVis is to enable users to set optimal thresholds for factor loadings to balance clarity and information retention. FAVis also allows users to assign tags to variables, enhancing the understanding of factors by linking them to their associated psychological constructs. Our user study demonstrates the utility of FAVis in various tasks.

cs.HC↗

arXiv2025

The Analytic Stockwell Transform and its Zeros

A recent original line of research in time--frequency analysis has shifted the interest in energy maxima toward zeros. Initially motivated by the intriguing uniform spread of the zeros of the spectrogram of white noise, it has led to fruitful theoretical developments combining probability theory, complex analysis and signal processing. In this vein, the present work proposes a characterization of the zeros of the Stockwell Transform of white noise, which consists in an hybrid time--frequency multiresolution representation. First of all, an analytic version of the Stockwell Transform is designed. Then, analyticity is leveraged to establish a connection with the hyperbolic Gaussian Analytic Function, whose zero set is invariant under the isometries of the Poincaré disk. Finally, the theoretical spatial statistics of the zeros of the hyperbolic Gaussian Analytic Function and the empirical statistics of the zeros the Analytic Stockwell Transform of white noise are compared through intensive Monte Carlo simulations, supporting the established connection. A publicly available documented Python toolbox accompanies this work.

eess.SP↗

arXiv2026

Bernoulli amputation

A novel, stochastic approach to amputation, the process of introducing missing values to a complete dataset, is presented. It allows one to construct a wide variety of missingness patterns by only having to specify distributions of missingness indicators as opposed to specifying each missingness pattern manually. Missingness indicators are modeled in a principled way via copulas and Bernoulli margins, thus allowing one to incorporate dependence in missingness patterns. Besides more classical missingness mechanisms such as missing completely at random, missing at random, and missing not at random, the approach is able to model structured missingness such as block missingness and, via mixtures, monotone missingness, which are patterns of missing data frequently found in real-life datasets. Properties such as joint missingness probabilities or missingness correlation are derived mathematically. The flexibility of the approach in capturing different missingness patterns while only requiring to specify distributional assumptions on missingness indicators is demonstrated with mathematical examples and empirical illustrations in terms of a well-known example dataset of sufficiently small sample size that allows to identify each missing data point visually. Finally, an example application to multivariate financial time series is provided.

stat.AP↗

arXiv2024

The Impact of Foreign Players in the English Premier League: A Mathematical Analys

We undertake extensive analysis of English Premier League data over the period 2009/10 to 2017/18 to identify and rank key factors affecting the economic and footballing performances of the teams. Alternative end-of-season league tables are generated by re-ranking the teams based on five different descriptors - total expenditure, total funds spent on players, total funds spent on foreign players, the ratio of foreign to British players and the overall profit. The unequal distribution of resources and expenditure between the clubs is analyzed through Lorenz curves. A comparative analysis of the differences between the alternative tables and the conventional end-of-season league table establishes the most likely factors to influence the performances of the teams that we also rank using Principal Component Analysis. We find that the top teams in the league are also those that tend to have the highest expenditure overall, for all players, including foreign players; they also have the highest ratios of foreign to British players. Our statistical and machine learning study also indicates that successful performance on the field may not guarantee healthy profits at the end of the season.

stat.OT↗

arXiv2024

How to survive the Squid Games using probability theory

In this paper, we consider how probability theory can be used to determine the survival strategy in two of the ``Squid Game" and ``Squid Game: The Challenge" challenges: the Hopscotch and the Warships. We show how Hopscotch can be easily tackled with the knowledge of the binomial distribution, taught in introductory statistics courses, while Warships is a much more complex problem, which can be tackled at different levels.

stat.OT↗

arXiv2024

Censored Data Forecasting: Applying Tobit Exponential Smoothing with Time Aggregation

This study introduces a novel approach to forecasting by Tobit Exponential Smoothing with time aggregation constraints. This model, a particular case of the Tobit Innovations State Space system, handles censored observed time series effectively, such as sales data, with known and potentially variable censoring levels over time. The paper provides a comprehensive analysis of the model structure, including its representation in system equations and the optimal recursive estimation of states. It also explores the benefits of time aggregation in state space systems, particularly for inventory management and demand forecasting. Through a series of case studies, the paper demonstrates the effectiveness of the model across various scenarios, including hourly and daily censoring levels. The results highlight the model's ability to produce accurate forecasts and confidence bands comparable to those from uncensored models, even under severe censoring conditions. The study further discusses the implications for inventory policy, emphasizing the importance of avoiding spiral-down effects in demand estimation. The paper concludes by showcasing the superiority of the proposed model over standard methods, particularly in reducing lost sales and excess stock, thereby optimizing inventory costs. This research contributes to the field of forecasting by offering a robust model that effectively addresses the challenges of censored data and time aggregation.

stat.ME↗

arXiv2025

Jackknife Empirical Likelihood Ratio Test for Cauchy Distribution

Heavy-tailed distributions, such as the Cauchy distribution, are acknowledged for providing more accurate models for financial returns, as the normal distribution is deemed insufficient for capturing the significant fluctuations observed in real-world assets. Data sets characterized by outlier sensitivity are critically important in diverse areas, including finance, economics, telecommunications, and signal processing. This article addresses a goodness-of-fit test for the Cauchy distribution. The proposed test utilizes empirical likelihood methods, including the jackknife empirical likelihood (JEL) and adjusted jackknife empirical likelihood (AJEL). Extensive Monte Carlo simulation studies are conducted to evaluate the finite sample performance of the proposed test. The application of the proposed test is illustrated through the analysing two real data sets.

math.ST↗