SEARCH · Searcharxiv

Results for “stat.OT”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 721 records · Page 40Linked to original sources

arXiv2021

Each student with her/his own data: understanding sampling distributions

Sampling distribution, a foundational concept in statistics, is difficult to understand, since we usually have only one realization of the estimator of interest. In this work, we present an innovative method for helping university students understand the variability of an estimator. In our approach, each student uses a different data set, getting diverse estimations. Then, sharing the results, we can empirically study the sampling distribution. After some "handmade" experiences, we have built a web page to deliver a personalized data set for each student. Through this web page, we can also reformulate the role of the student in the classroom, inviting him/her to became an active player, submitting the solution to different problems and checking whether they are correct. In this work we present a recent experience in such direction.

stat.OT↗

arXiv2021

Probabilistic Modeling of Hurricane Wind-Induced Damage in Infrastructure Systems

This paper presents a modeling approach for probabilistic estimation of hurricane wind-induced damage to infrastructural assets. In our approach, we employ a Nonhomogeneous Poisson Process (NHPP) model for estimating spatially-varying probability distributions of damage as a function of hurricane wind field velocities. Specifically, we consider a physically-based, quadratic NHPP model for failures of overhead assets in electricity distribution systems. The wind field velocities are provided by Forecasts of Hurricanes using Large-Ensemble Outputs (FHLO), a framework for generating probabilistic hurricane forecasts. We use FHLO in conjunction with the NHPP model, such that the hurricane forecast uncertainties represented by FHLO are accounted for in estimating the probability distributions of damage. Furthermore, we evaluate the spatial variability and extent of hurricane damage under key wind field parameters (intensity, size, and asymmetries). By applying our approach to prediction of power outages (loss-of-service) in northwestern Florida due to Hurricane Michael (2018), we demonstrate a statistically significant relationship between outage rate and failure rate. Finally, we formulate parametric models that relate total damage and financial losses to the hurricane parameters of intensity and size. Overall, this paper's findings suggest that our approach is well-suited to jointly account for spatial variability and forecast uncertainty in the damage estimates, and is readily applicable to prediction of system loss-of-service due to the damage.

stat.AP↗

arXiv2021

Phase transition in a power-law uniform hypergraph

We propose a power-law $m$-uniform random hypergraph on $n$ vertexes. In this hypergraph, each vertex is independently assigned a random weight from a power-law distribution with exponent $α\in(0,\infty)$ and the hyperedge probabilities are defined as functions of the random weights. We characterize the number of hyperedge and the number of loose 2-cycle. There is a phase transition phenomenon for the number of hyperedge at $α=1$. Interestingly, for the number of loose 2-cycle, phase transition occurs at both $α=1$ and $α=2$.

stat.OT↗

arXiv2021

Perspective on Data Science

The field of data science currently enjoys a broad definition that includes a wide array of activities which borrow from many other established fields of study. Having such a vague characterization of a field in the early stages might be natural, but over time maintaining such a broad definition becomes unwieldy and impedes progress. In particular, the teaching of data science is hampered by the seeming need to cover many different points of interest. Data scientists must ultimately identify the core of the field by determining what makes the field unique and what it means to develop new knowledge in data science. In this review we attempt to distill some core ideas from data science by focusing on the iterative process of data analysis and develop some generalizations from past experience. Generalizations of this nature could form the basis of a theory of data science and would serve to unify and scale the teaching of data science to large audiences.

stat.OT↗

arXiv2021

A Spatio-Temporal Model for Predicting Wind Speeds in Southern California

The share of wind power in fuel mixes worldwide has increased considerably. The main ingredient when deriving wind power predictions are wind speed data; the closer to the wind farms, the better they forecast the power supply. The current paper proposes a hybrid model for predicting wind speeds at convenient locations. It is then applied to Southern California power price area. We build random fields with time series of gridded historical forecasts and actual wind speed observations. We estimate with ordinary kriging the spatial variability of the temporal parameters and derive predictions. The advantages of this work are twofold: (1) an accurate daily wind speed forecast at any location in the area and (2) a general method applicable to other markets.

stat.AP↗

arXiv2021

Comments on "Two Cultures": What have changed over 20 years?

Twenty years ago Breiman (2001) called to our attention a significant cultural division in modeling and data analysis between the stochastic data models and the algorithmic models. Out of his deep concern that the statistical community was so deeply and "almost exclusively" committed to the former, Breiman warned that we were losing our abilities to solve many real-world problems. Breiman was not the first, and certainly not the only statistician, to sound the alarm; we may refer to none other than John Tukey who wrote almost 60 years ago "data analysis is intrinsically an empirical science." However, the bluntness and timeliness of Breiman's article made it uniquely influential. It prepared us for the data science era and encouraged a new generation of statisticians to embrace a more broadly defined discipline. Some might argue that "The cultural division between these two statistical learning frameworks has been growing at a steady pace in recent years", to quote Mukhopadhyay and Wang (2020). In this commentary, we focus on some of the positive changes over the past 20 years and offer an optimistic outlook for our profession.

stat.OT↗

arXiv2021

An Introduction to DoSStoolkit

We describe a series of interactive, student-developed, self-paced, modules for learning R. We detail the components of this resource, and the pedagogical underpinning. We discuss the development of this resource, and avenues for future work. Our resource is available as an R package: DoSStoolkit.

stat.OT↗

arXiv2022

Bayesian Uncertainty Quantification of Local Volatility Model

Local volatility is an important quantity in option pricing, portfolio hedging, and risk management. It is not directly observable from the market; hence calibrations of local volatility models are necessary using observable market data. Unlike most existing point-estimate methods, we cast the large-scale nonlinear inverse problem into the Bayesian framework, yielding a posterior distribution of the local volatility, which naturally quantifies its uncertainty. This extra uncertainty information enables traders and risk managers to make better decisions. To alleviate the computational cost, we apply Karhunen--Lòeve expansion to reduce the dimensionality of the Gaussian Process prior for local volatility. A modified two-stage adaptive Metropolis algorithm is applied to sample the posterior probability distribution, which further reduces computational burdens caused by repetitive numerical forward option pricing model solver and time of heuristic tuning. We demonstrate our methodology with both synthetic and market data.

stat.AP↗

arXiv2021

Path-specific Effects Based on Information Accounts of Causality

Path-specific effects in mediation analysis provide a useful tool for fairness analysis, which is mostly based on nested counterfactuals. However, the dictum ``no causation without manipulation'' implies that path-specific effects might be induced by certain interventions. This paper proposes a new path intervention inspired by information accounts of causality, and develops the corresponding intervention diagrams and $π$-formula. Compared with the interventionist approach of Robins et al.(2020) based on nested counterfactuals, our proposed path intervention method explicitly describes the manipulation in structural causal model with a simple information transferring interpretation, and does not require the non-existence of recanting witness to identify path-specific effects. Hence, it could serve useful communications and theoretical focus for mediation analysis.

cs.AI↗

arXiv2021

Convex Sparse Blind Deconvolution

In the blind deconvolution problem, we observe the convolution of an unknown filter and unknown signal and attempt to reconstruct the filter and signal. The problem seems impossible in general, since there are seemingly many more unknowns than knowns . Nevertheless, this problem arises in many application fields; and empirically, some of these fields have had success using heuristic methods -- even economically very important ones, in wireless communications and oil exploration. Today's fashionable heuristic formulations pose non-convex optimization problems which are then attacked heuristically as well. The fact that blind deconvolution can be solved under some repeatable and naturally-occurring circumstances poses a theoretical puzzle. To bridge the gulf between reported successes and theory's limited understanding, we exhibit a convex optimization problem that -- assuming signal sparsity -- can convert a crude approximation to the true filter into a high-accuracy recovery of the true filter. Our proposed formulation is based on L1 minimization of inverse filter outputs. We give sharp guarantees on performance of the minimizer assuming sparsity of signal, showing that our proposal precisely recovers the true inverse filter, up to shift and rescaling. There is a sparsity/initial accuracy tradeoff: the less accurate the initial approximation, the greater we rely on sparsity to enable exact recovery. To our knowledge this is the first reported tradeoff of this kind. We consider it surprising that this tradeoff is independent of dimension. We also develop finite-$N$ guarantees, for highly accurate reconstruction under $N\geq O(k \log(k) )$ with high probability. We further show stable approximation when the true inverse filter is infinitely long and extend our guarantees to the case where the observations are contaminated by stochastic or adversarial noise.

cs.IT↗

arXiv2021

Limited-Information Maximum Likelihood based Model Selection Procedures for Binary Outcomes

Unmeasured covariates constitute one of the important problems in causal inference. Even if there are some unmeasured covariates, some instrumental variable methods such as a two-stage residual inclusion (2SRI) estimator, or a limited-information maximum likelihood (LIML) estimator can obtain an unbiased estimate for causal effects despite there being nonlinear outcomes such as binary outcomes; however, it requires that we specify not only a correct outcome model but also a correct treatment model. Therefore, detecting correct models is an important process. In this paper, we propose two model selection procedures: AIC-type and BIC-type, and confirm their properties. The proposed model selection procedures are based on a LIML estimator. We prove that a proposed BIC-type model selection procedure has model selection consistency, and confirm their properties of the proposed model selection procedures through simulation datasets.

stat.ME↗

arXiv2021

Extreme event propagation using counterfactual theory and vine copulas

Understanding multivariate extreme events play a crucial role in managing the risks of complex systems since extremes are governed by their own mechanisms. Conditional on a given variable exceeding a high threshold (e.g.\ traffic intensity), knowing which high-impact quantities (e.g\ air pollutant levels) are the most likely to be extreme in the future is key. This article investigates the contribution of marginal extreme events on future extreme events of related quantities. We propose an Extreme Event Propagation framework to maximise counterfactual causation probabilities between a known cause and future high-impact quantities. Extreme value theory provides a tool for modelling upper tails whilst vine copulas are a flexible device for capturing a large variety of joint extremal behaviours. We optimise for the probabilities of causation and apply our framework to a London road traffic and air pollutants dataset. We replicate documented atmospheric mechanisms beyond linear relationships. This provides a new tool for quantifying the propagation of extremes in a large variety of applications.

stat.ME↗

arXiv2021

Evolving-Graph Gaussian Processes

Graph Gaussian Processes (GGPs) provide a data-efficient solution on graph structured domains. Existing approaches have focused on static structures, whereas many real graph data represent a dynamic structure, limiting the applications of GGPs. To overcome this we propose evolving-Graph Gaussian Processes (e-GGPs). The proposed method is capable of learning the transition function of graph vertices over time with a neighbourhood kernel to model the connectivity and interaction changes between vertices. We assess the performance of our method on time-series regression problems where graphs evolve over time. We demonstrate the benefits of e-GGPs over static graph Gaussian Process approaches.

cs.LG↗

arXiv2022

A nonBayesian view of Hempel's paradox of the ravens

In Hempel's paradox of the ravens, seeing a red pencil is considered as supporting evidence that all ravens are black. Also known as the Paradox of Confirmation, the paradox and its many resolutions indicate that we cannot underestimate the logical and statistical elements needed in the assessment of evidence in support of a hypothesis. Most of the previous analyses of the paradox are within the Bayesian framework. These analyses and Hempel himself generally accept the paradoxical conclusion; it feels paradoxical supposedly because the amount of evidence is extremely small. Here I describe a nonBayesian analysis of various statistical models with an accompanying likelihood-based reasoning. The analysis shows that the paradox feels paradoxical because there are natural models where observing a red pencil has no relevance to the color of ravens. In general the value of the evidence depends crucially on the sampling scheme and on the assumption about the underlying parameters of the relevant model.

stat.OT↗

arXiv2021

Approximations to ultimate ruin probabilities with a Wienner process perturbation

In this paper, we adapt the classic Cramér-Lundberg collective risk theory model to a perturbed model by adding a Wiener process to the compound Poisson process, which can be used to incorporate premium income uncertainty, interest rate fluctuations and changes in the number of policyholders. Our study is part of a Master dissertation, our aim is to make a short overview and present additionally some new approximation methods for the infinite time ruin probabilities for the perturbed risk model. We present four different approximation methods for the perturbed risk model. The first method is based on iterative upper and lower approximations to the maximal aggregate loss distribution. The second method relies on a four-moment exponential De Vylder approximation. The third method is based on the first-order Padé approximation of the Renyi and De Vylder approximations. The last method is the second order Padé-Ramsay approximation. These are generated by fitting one, two, three or four moments of the claim amount distribution, which greatly generalizes the approximations. We test the precision of approximations using a combination of light and heavy tailed distributions for the individual claim amount. We assess the ultimate ruin probability and present numerical results for the exponential, gamma, and mixed exponential claim distributions, demonstrating the high accuracy of these four methods. Analytical and numerical methods are used to highlight the practical implications of our findings.

q-fin.RM↗

arXiv2021

Patient-reported outcomes in the context of the benefit assessment in Germany

Since the 2011 Act on the Reform of the Market for Medicinal Products, benefit dossiers are submitted by pharmaceutical companies to facilitate the Health Technology Assessment (HTA) appraisals in Germany. The Institute for Quality and Efficiency in Health Care conducts the added benefit assessment following their General Methods Paper, which was updated November 5, 2020. This White Paper is dedicated to patient-reported outcomes (PRO) to highlight their importance for the added benefit assessment. We focus on methodological aspects but consider also other relevant requirements and challenges, which are demanded by G-BA and IQWiG. The following topics will be presented and discussed: 1. Role of PRO in HTA decision making exemplary to benefit assessment in Germany 2. Guidances of PRO evaluations 3. PRO Estimand framework 4. Perception and requirements for PRO within the German benefit assessment 5. Validity of instrument 6. Response thresholds for assessing clinical relevance of PRO 7. PRO endpoints / outcome measures / operationalization 8. Missing PRO data 9. PRO after treatment discontinuation This White Paper aims to provide deeper insights about new requirements concerning PRO evaluations for HTA decision making in Germany, highlight points to consider that should inform global development in terms of study planning and frame the requirements also in the context of global recommendations and guidelines. We also aim to enhance the understanding of the complexity when preparing the benefit dossier and promote further scientific discussions where appropriate.

stat.AP↗

arXiv2021

Discovery of Bayes' Table at Tunbridge Wells

In 1755 Thomas Bayes expressed an interest in the problem of combining repeated measurements of the location of a star. Bayes described a tandem set-up of a ball thrown on a table, followed by repeated throws of a second ball. Bayes' table has long been taken as a billiard table, for which there is no evidence. We report the discovery of Bayes' table, a bowling green located half a km uphill (SE) from the meeting house where Bayes served as minister for two decades. Bayes' drawing shows a rectangular space marked off in yards, which allows calculation of an interval measurement of uncertainty. The Bayes rule interval from 2.5% to 97.5% is from 0.56 - 0.42 = 0.12 perches equivalent to 0.61 m. The discovery of Bayes' table establishes the physical basis for Bayes' symmetrical probability model, a fixed parameter binomial (θ = 0.5). The discovery establishes Bayes as the founder of statistical science, defined as the application of mathematics to scientific measurement.

stat.OT↗

arXiv2021

Impact of heat waves and cold spells on cause-specific mortality in the city of Sao Paulo, Brazil

The impact of heat waves and cold spells on mortality has become a major public health problem worldwide, especially among older adults living in low- to middle-income countries. This study aimed to investigate the effects of heat waves and cold spells under different definitions on cause-specific mortality among people aged 65 years and over in Sao Paulo from 2006 to 2015. A quasi-Poisson generalized linear model with a distributed lag model was used to investigate the association between cause-specific mortality and extreme air temperature events. To evaluate the effects of the intensity under different durations, we considered 12 heat wave and nine cold spell definitions. Our results showed an increase in cause-specific deaths related to heat waves and cold spells under several definitions. The highest risk of death related to heat waves was identified mostly at higher temperature thresholds with longer events. We verified that men were more vulnerable to die from an ischemic stroke on heat waves and cold spells days than women, while women presented a higher risk of dying from ischemic heart diseases during cold spells and tended to have a higher risk of chronic obstructive pulmonary disease than men. Identification of heat wave- and cold spell-related mortality is important for the development and promotion of public health measures.

stat.AP↗