Searcharxiv⌕ Search

SEARCH · Searcharxiv

Results for “stat.OT”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 739 records · Page 41Linked to original sources

Inference in heavy-tailed non-stationary multivariate time series

We study inference on the common stochastic trends in a non-stationary, $N$-variate time series $y_{t}$, in the possible presence of heavy tails. We propose a novel methodology which does not require any knowledge or estimation of the tail index, or even knowledge as to whether certain moments (such as the variance) exist or not, and develop an estimator of the number of stochastic trends $m$ based on the eigenvalues of the sample second moment matrix of $y_{t}$. We study the rates of such eigenvalues, showing that the first $m$ ones diverge, as the sample size $T$ passes to infinity, at a rate faster by $O\left(T \right)$ than the remaining $N-m$ ones, irrespective of the tail index. We thus exploit this eigen-gap by constructing, for each eigenvalue, a test statistic which diverges to positive infinity or drifts to zero according to whether the relevant eigenvalue belongs to the set of the first $m$ eigenvalues or not. We then construct a randomised statistic based on this, using it as part of a sequential testing procedure, ensuring consistency of the resulting estimator of $m$. We also discuss an estimator of the common trends based on principal components and show that, up to a an invertible linear transformation, such estimator is consistent in the sense that the estimation error is of smaller order than the trend itself. Finally, we also consider the case in which we relax the standard assumption of \textit{i.i.d.} innovations, by allowing for heterogeneity of a very general form in the scale of the innovations. A Monte Carlo study shows that the proposed estimator for $m$ performs particularly well, even in samples of small size. We complete the paper by presenting four illustrative applications covering commodity prices, interest rates data, long run PPP and cryptocurrency markets.

econ.EM↗

Profit and loss manipulations by online trading brokers

Online trading has attracted millions of people around the world. In March 2021, it was reported there were 18 million accounts from just one broker. Historically, manipulation in financial markets is considered to be fraudulently influencing share, currency pairs or any other indices prices. This article introduces the idea that online trading platform technical issues can be considered as brokers manipulation to control traders profit and loss. More importantly it shows these technical issues are the contributing factors of the 82% risk of retail traders losing money. We identify trading platform technical issues of one of the world's leading online trading providers and calculate retail traders losses caused by these issues. To do this, we independently record each trade details using the REST API response provided by the broker. We show traders log activity files is the only way to assess any suspected profit or loss manipulation by the broker. Therefore, it is essential for any retail trader to have access to their log files. We compare our findings with broker's Trustpilot customer reviews. We illustrate how traders' profit and loss can be negatively affected by broker's platform technical issues such as not being able to close profitable trades, closing trades with delays, disappearance of trades, disappearance of profit from clients statements, profit and loss discrepancies, stop loss not being triggered, stop loss or limit order triggered too early. Although regulatory bodies try to ensure that consumers get a fair deal, these attempts are hugely insufficient in protecting retail traders. Therefore, regulatory bodies such as the FCA should take these technical issues seriously and not rely on brokers' internal investigations, because under any other circumstances, these platform manipulations would be considered as crimes and connivingly misappropriating funds.

q-fin.TR↗

Signatures, Lipschitz-free spaces, and paths of persistence diagrams

Paths of persistence diagrams provide a summary of the dynamic topological structure of a one-parameter family of metric spaces. These summaries can be used to study and characterize the dynamic shape of data such as swarming behavior in multi-agent systems, time-varying fMRI scans from neuroscience, and time-dependent scalar fields in hydrodynamics. While persistence diagrams can provide a powerful topological summary of data, the standard space of persistence diagrams lacks the sufficient algebraic and analytic structure required for many theoretical and computational analyses. We enrich the space of persistence diagrams by isometrically embedding it into a Lipschitz-free space, a Banach space built from a universal construction. We utilize the Banach space structure to define bounded variation paths of persistence diagrams, which can be studied using the path signature, a reparametrization-invariant characterization of paths valued in a Banach space. The signature is universal and characteristic which allows us to theoretically characterize measures on the space of paths and motivates its use in the context of kernel methods. However, kernel methods often require a feature map into a Hilbert space, so we introduce the moment map, a stable and injective feature map for static persistence diagrams, and compose it with the discrete path signature, producing a computable feature map into a Hilbert space. Finally, we demonstrate the efficacy of our methods by applying this to a parameter estimation problem for a 3D model of swarming behavior.

math.AT↗

Lottery paradox, DNA evidence and other stories: How to accept uncertain statements

I think we can agree that dealing with uncertainty is not easy. Probability is the main tool for dealing with uncertainty, and we know there are many probability-related puzzles and paradoxes. Here I describe a rather idiosyncratic selection that highlights the problem of accepting uncertain statements. Without going into a formal decision theory, there are simple intuitive rational bases for doing that, for instance based on high probability alone. The lottery paradox shows the logical problem of accepting uncertain statements based on high probability. The DNA evidence story is an example of the use probabilistic reasoning in court, where philosophical differences between the schools of inference -- the frequentist, Bayesian and likelihood schools -- lead to substantial differences in the quantification of evidence.

stat.OT↗

On the role of data, statistics and decisions in a pandemic

A pandemic poses particular challenges to decision-making because of the need to continuously adapt decisions to rapidly changing evidence and available data. For example, which countermeasures are appropriate at a particular stage of the pandemic? How can the severity of the pandemic be measured? What is the effect of vaccination in the population and which groups should be vaccinated first? The process of decision-making starts with data collection and modeling and continues to the dissemination of results and the subsequent decisions taken. The goal of this paper is to give an overview of this process and to provide recommendations for the different steps from a statistical perspective. In particular, we discuss a range of modeling techniques including mathematical, statistical and decision-analytic models along with their applications in the COVID-19 context. With this overview, we aim to foster the understanding of the goals of these modeling approaches and the specific data requirements that are essential for the interpretation of results and for successful interdisciplinary collaborations. A special focus is on the role played by data in these different models, and we incorporate into the discussion the importance of statistical literacy, and of effective dissemination and communication of findings.

stat.OT↗

An examination of the generalised pooled binomial distribution and its information properties

This paper examines the statistical properties of a distributional form that arises from pooled testing for the prevalence of a binary outcome. Our base distribution is a two-parameter distribution using a prevalence and excess intensity parameter; the latter is included to allow for a dilution or intensification effect with larger pools. We also examine a generalised form of the distribution where pools have covariate information that affects the prevalence through a linked linear form. We study the general pooled binomial distribution in its own right and as a special case of broader forms of binomial GLMs using the complementary log-log link function. We examine the information function and show the information content of individual sample items. We demonstrate that pooling reduces information content of sample units and we give simple heuristics for choosing an "optimal" pool size for testing. We derive the form of the log-likelihood function and its derivatives and give results for maximum likelihood estimation. We also discuss diagnostic testing of the positive pool probabilities, including testing for intensification/dilution in the testing mechanism. We illustrate the use of this distribution by applying it to pooled testing data on virus prevalence in a mosquito population.

stat.ME↗

Rank Energy Statistics in the Context of Change Point Detection

In this paper, I propose a general procedure for multivariate distribution-free nonparametric testing derived from the concept of ranks that are based upon measure transportation in the context of multiple change point analysis. I will use this algorithm to estimate both the number of change points and their locations within an observed multivariate time series. In this paper, the change point problem is observed in a general setting in which both the given distribution and number of change points are unknown, rather than assume the observed time series follows a specific distribution or contains only one change point as many works in this area of study assume. The intention of this is to develop a technique for accurately identifying the changes in a distribution while making as few suppositions as possible. The rank energy statistic used here is based on energy statistics and has the potential to detect any change in a distribution. I present the properties of this new algorithm, which can be used to analyze various datasets, including hierarchical clustering, testing multivariate normality, gene selection, and microarray data analysis. This algorithm has also been implemented in the R package recp, which is available on GitHub.

stat.OT↗

Which Type of Statistical Uncertainty Helps Evidence-Based Policymaking? An Insight from a Survey Experiment in Ireland

Which type of statistical uncertainty -- statistical (in)significance with a p-value, or a Bayesian probability -- enables people to see the continuous nature of uncertainty more clearly in a policymaking context? An original survey experiment used a hypothetical scenario, where participants from Ireland were asked whether to introduce a new bus line to reduce traffic jams, given a research report estimating its effectiveness. The treatments were uncertainty information: statistical significance with a p-value of 2%, statistical insignificance with a p-value of 25%, the 95% probability that the estimate is correct, and the 68% probability that the estimate is correct. In the case of lower uncertainty, both significance and Bayesian frameworks resulted in a large proportion of participants adopting the policy (0.82 and 0.91 respectively). In the case of higher uncertainty, the significance framework led a much smaller proportion of participants to adopt the policy (0.39 against 0.83). The findings suggest participants saw the continuous nature of uncertainty more clearly in the Bayesian framework than in the significance framework.

stat.OT↗

The Generalized Gamma distribution as a useful RND under Heston's stochastic volatility model

Following Boukai (2021) we present the Generalized Gamma (GG) distribution as a possible RND for modeling European options prices under Heston's (1993) stochastic volatility (SV) model. This distribution is seen as especially useful in situations in which the spot's price follows a negatively skewed distribution and hence, Black-Scholes based (i.e. the log-normal distribution) modeling is largely inapt. We apply the GG distribution as RND to modeling current market option data on three large market-index ETFs, namely the SPY, IWM and QQQ as well as on the TLT (an ETF that tracks an index of long term US Treasury bonds). The current option chain of each of the three market-index ETFs shows of a pronounced skew of their volatility `smile' which indicates a likely distortion in the Black-Scholes modeling of such option data. Reflective of entirely different market expectations, this distortion appears not to exist in the TLT option data. We provide a thorough modeling of the available option data we have on each ETF (with the October 15, 2021 expiration) based on the GG distribution and compared it to the option pricing and RND modeling obtained directly from a well-calibrated Heston's (1993) SV model (both theoretically and empirically, using Monte-Carlo simulations of the spot's price). All three market-index ETFs exhibit negatively skewed distributions which are well-matched with those derived under the GG distribution as RND. The inadequacy of the Black-Scholes modeling in such instances which involve negatively skewed distribution is further illustrated by its impact on the hedging factor, delta, and the immediate implications to the retail trader. In contrast, for the TLT ETF, which exhibits no such distortion to the volatility `smile', the three pricing models (i.e. Heston's, Black-Scholes and Generalized Gamma) appear to yield similar results.

q-fin.CP↗

Functional Data Representation with Merge Trees

In this paper we face the problem of representation of functional data with the tools of algebraic topology. We represent functions by means of merge trees, which, like the more commonly used persistence diagrams, are invariant under homeomorphic reparametrizations of the functions they represent, thus allowing for a statistical analysis which is indifferent to functional misalignment. We consider a recently defined metric for merge trees and we prove some theoretical results related to its specific implementation when merge trees represent functions, establishing also a class of consistent estimators with convergence rates. To showcase the good properties of our topological approach to functional data analysis, we test it on the Aneurisk65 dataset replicating, from our different perspective, the supervised classification analysis which contributed to make this dataset a benchmark for methods dealing with misaligned functional data. In the Appendix we provide an extensive comparison between merge trees and persistence diagrams, highlighting similarities and differences, which can guide the analyst in choosing between the two representations.

stat.ME↗

Comments on The clinical meaningfulness of a treatment's effect on a time-to-event variable

Some years ago, Snapinn and Jiang[1] considered the interpretation and pitfalls of absolute versus relative treatment effect measures in analyses of time-to-event outcomes. Through specific examples and analytical considerations based solely on the exponential and the Weibull distributions they reach two conclusions: 1) that the commonly used criteria for clinical effectiveness, the ARR (Absolute Risk Reduction) and the median (survival time) difference (MD) directly contradict each other and 2) cost-effectiveness depends only the hazard ratio(HR) and the shape parameter (in the Weibull case) but not the overall baseline risk of the population. Though provocative, the first conclusion does not apply to either the two special cases considered or even more generally, while the second conclusion is strictly correct only for the exponential case. Therefore, the implication inferred by the authors i.e. all measures of absolute treatment effect are of little value compared with the relative measure of the hazard ratio, is not of general validity and hence both absolute and relative measures should continue to be used when appraising clinical evidence.

stat.AP↗

A generalized bootstrap procedure of the standard error and confidence interval estimation for inverse probability of treatment weighting

The inverse probability of treatment weighting (IPTW) approach is commonly used in propensity score analysis to infer causal effects in regression models. Due to oversized IPTW weights and errors associated with propensity score estimation, the IPTW approach can underestimate the standard error of causal effect. To remediate this, bootstrap standard errors have been recommended to replace the IPTW standard error, but the ordinary bootstrap (OB) procedure might still result in underestimation of the standard error because of its inefficient sampling algorithm and un-stabilized weights. In this paper, we develop a generalized bootstrap (GB) procedure for estimating the standard error of the IPTW approach. Compared with the OB procedure, the GB procedure has much lower risk of underestimating the standard error and is more efficient for both point and standard error estimates. The GB procedure also has smaller risk of standard error underestimation than the ordinary bootstrap procedure with trimmed weights, with comparable efficiencies. We demonstrate the effectiveness of the GB procedure via a simulation study and a dataset from the National Educational Longitudinal Study-1988 (NELS-88).

stat.ME↗

A truncated mean-parameterised Conway-Maxwell-Poisson model for the analysis of Test match bowlers

Assessing the relative merits of sportsmen and women whose careers took place far apart in time via a suitable statistical model is a complex task as any comparison is compromised by fundamental changes to the sport and society and often handicapped by the popularity of inappropriate traditional metrics. In this work we focus on cricket and the ranking of Test match bowlers using bowling data from the first Test in 1877 onwards. A truncated, mean-parameterised Conway-Maxwell-Poisson model is developed to handle the under- and overdispersed nature of the data, which are in the form of small counts, and to extract the innate ability of individual bowlers. Inferences are made using a Bayesian approach by deploying a Markov Chain Monte Carlo algorithm to obtain parameter estimates and confidence intervals. The model offers a good fit and indicates that the commonly used bowling average is a flawed measure.

stat.AP↗

SIHR: Statistical Inference in High-Dimensional Linear and Logistic Regression Models

We introduce the R package \CRANpkg{SIHR} for statistical inference in high-dimensional generalized linear models with continuous and binary outcomes. The package provides functionalities for constructing confidence intervals and performing hypothesis tests for low-dimensional objectives in both one-sample and two-sample regression settings. We illustrate the usage of \CRANpkg{SIHR} through numerical examples and present real data applications to demonstrate the package's performance and practicality.

stat.CO↗

Evaluation of imputation techniques with varying percentage of missing data

Missing data is a common problem which has consistently plagued statisticians and applied analytical researchers. While replacement methods like mean-based or hot deck imputation have been well researched, emerging imputation techniques enabled through improved computational resources have had limited formal assessment. This study formally considers five more recently developed imputation methods: Amelia, Mice, mi, Hmisc and missForest - compares their performances using RMSE against actual values and against the well-established mean-based replacement approach. The RMSE measure was consolidated by method using a ranking approach. Our results indicate that the missForest algorithm performed best and the mi algorithm performed worst.

stat.ME↗

RWP+: A New Random Waypoint Model for High-Speed Mobility

In this letter, we emulate real-world statistics for mobility patterns on road systems. We then propose modifications to the assumptions of the random waypoint (RWP) model to better represent high-mobility profiles. We call the model under our new framework as RWP+. Specifically, we show that the lengths of the transitions which constitute a trip, are best represented by a lognormal distribution, and that the velocities are best described by a linear combination of normal distributions with different mean values. Compared to the assumptions used in the literature for mobile cellular networks, our modeling provides mobility metrics, such as handoff rates, that better characterize actual emulated trips from the collected statistics.

cs.IT↗

Bayesian model-based outlier detection in network meta-analysis

In a network meta-analysis, some of the collected studies may deviate markedly from the others, for example having very unusual effect sizes. These deviating studies can be regarded as outlying with respect to the rest of the network and can be influential on the pooled results. Thus, it could be inappropriate to synthesize those studies without further investigation. In this paper, we propose two Bayesian methods to detect outliers in a network meta-analysis via: (a) a mean-shifted outlier model and (b), posterior predictive p-values constructed from ad-hoc discrepancy measures. The former method uses Bayes factors to formally test each study against outliers while the latter provides a score of outlyingness for each study in the network, which allows to numerically quantify the uncertainty associated with being outlier. Furthermore, we present a simple method based on informative priors as part of the network meta-analysis model to down-weight the detected outliers. We conduct extensive simulations to evaluate the effectiveness of the proposed methodology while comparing it to some alternative, available outlier diagnostic tools. Two real networks of interventions are then used to demonstrate our methods in practice.

stat.ME↗

On variable selection in joint modeling of mean and dispersion

The joint modeling of mean and dispersion (JMMD) provides an efficient method to obtain useful models for the mean and dispersion, especially in problems of robust design experiments. However, in the literature on JMMD there are few works dedicated to variable selection and this theme is still a challenge. In this article, we propose a procedure for selecting variables in JMMD, based on hypothesis testing and the quality of the model's fit. A criterion for checking the goodness of fit is used, in each iteration of the selection process, as a filter for choosing the terms that will be evaluated by a hypothesis test. Three types of criteria were considered for checking the quality of the model fit in our variable selection procedure. The criteria used were: the extended Akaike information criterion, the corrected Akaike information criterion and a specific criterion for the JMMD, proposed by us, a type of extended adjusted coefficient of determination. Simulation studies were carried out to verify the efficiency of our variable selection procedure. In all situations considered, the proposed procedure proved to be effective and quite satisfactory. The variable selection process was applied to a real example from an industrial experiment.

stat.ME↗