Searcharxiv⌕ Search

SEARCH · Searcharxiv

Results for “stat.OT”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12Linked to original sources
arXiv2015

Publication Bias in Meta-Analysis: Confidence Intervals for Rosenthal's Fail-Safe Number

The purpose of the present paper is to assess the efficacy of confidence intervals for Rosenthal's fail-safe number. Although Rosenthal's estimator is highly used by researchers, its statistical properties are largely unexplored. First of all, we developed statistical theory which allowed us to produce confidence intervals for Rosenthal's fail-safe number.This was produced by discerning whether the number of studies analysed in a meta-analysis is fixed or random. Each case produces different variance estimators. For a given number of studies and a given distribution, we provided five variance estimators. Confidence intervals are examined with a normal approximation and a nonparametric bootstrap. The accuracy of the different confidence interval estimates was then tested by methods of simulation under different distributional assumptions. The half normal distribution variance estimator has the best probability coverage. Finally, we provide a table of lower confidence intervals for Rosenthal's estimator.

stat.ME↗
arXiv2015

A multi-dimensional stream and its signature representation

The signature of a path is an essential object in the theory of rough paths. The signature representation of the data stream can recover standard statistics, e.g. the moments of the data stream. The classification of random walks indicates the advantages of using the signature of a stream as the feature set for machine learning.

stat.OT↗
arXiv2015

Designed Sampling from Large Databases for Controlled Trials

The increasing prevalence of rich sources of data and the availability of electronic medical record databases and electronic registries opens tremendous opportunities for enhancing medical research. For example, controlled trials are ubiquitously used to investigate the effect of a medical treatment, perhaps dependent on a set of patient covariates, and traditional approaches have relied primarily on randomized patient sampling and allocation to treatment and control group. However, when covariate data for a large cohort group of patients have already been collected and are available in a database, one can potentially design a treatment/control sample and allocation that provides far better estimates of the covariate-dependent effects of the treatment. In this paper, we develop a new approach that uses optimal design of experiments (DOE) concepts to accomplish this objective. The approach selects the patients for the treatment and control samples upfront, based on their covariate values, in a manner that optimizes the information content in the data. For the optimal sample selection, we develop simple guidelines and an optimization algorithm that provides solutions that are substantially better than random sampling. Moreover, our approach causes no sampling bias in the estimated effects, for the same reason that DOE principles do not bias estimated effects. We test our method with a simulation study based on a testbed data set containing information on the effect of statins on low-density lipoprotein (LDL) cholesterol.

stat.ME↗
arXiv2015

From Statistician to Data Scientist

According to a recent report from the European Commission, the world generates every minute 1.7 million of billions of data bytes, the equivalent of 360,000 DVDs, and companies that build their decision-making processes by exploiting these data increase their productivity. The treatment and valorization of massive data has consequences on the employment of graduate students in statistics. Which additional skills do students trained in statistics need to acquire to become data scientists ? How to evolve training so that future graduates can adapt to rapid changes in this area, without neglecting traditional jobs and the fundamental and lasting foundation for the training? After considering the notion of big data and questioning the emergence of a "new" science: Data Science, we present the current developments in the training of engineers in Mathematical and Modeling at INSA Toulouse.

stat.OT↗
arXiv2015

Statistical Engineering: An Idea Whose Time Has Come?

Several authors, including the American Statistician (ASA), have noted the challenges facing statisticians when attacking large, complex, unstructured problems, as opposed to well-defined textbook problems. Clearly, the standard paradigm of selecting the one "correct" statistical method for such problems is not sufficient; a new paradigm is needed. Statistical engineering has been proposed as a discipline that can provide a viable paradigm to attack such problems, used in conjunction with sound statistical science. Of course, in order to develop as a true discipline, statistical engineering needs a well-developed theory, not just a formal definition and successful case studies. This article documents and disseminates the current state of the underlying theory of statistical engineering. Our purpose is to provide a vehicle for applied statisticians to further enhance the practice of statistics, and for academics so interested to continue development of the underlying theory of statistical engineering.

stat.OT↗
arXiv2016

J. B. S. Haldane's Contribution to the Bayes Factor Hypothesis Test

This article brings attention to some historical developments that gave rise to the Bayes factor for testing a point null hypothesis against a composite alternative. In line with current thinking, we find that the conceptual innovation - to assign prior mass to a general law - is due to a series of three articles by Dorothy Wrinch and Sir Harold Jeffreys (1919, 1921, 1923). However, our historical investigation also suggests that in 1932 J. B. S. Haldane made an important contribution to the development of the Bayes factor by proposing the use of a mixture prior comprising a point mass and a continuous probability density. Jeffreys was aware of Haldane's work and it may have inspired him to pursue a more concrete statistical implementation for his conceptual ideas. It thus appears that Haldane may have played a much bigger role in the statistical development of the Bayes factor than has hitherto been assumed.

stat.OT↗
arXiv2015

Modelling Hospital length of stay using convolutive mixtures distributions

Length of hospital stay (LOS) is an important indicator of the hospital activity and management of health care. The skewness in the distribution of LOS poses problems in statistical modelling because it fails to adequately follow the usual traditional distribution such as the log-normal distribution. The aim of this work is to model the variable LOS using the convolution of two distributions; a technique well known in the signal processing community. The specificity of that model is that the variable of interest is considered to be the resulting sum of two random variables with different distributions. One of the variables will feature the patient-related factors in terms their need to recover from their admission condition, while the other models the hospital management process such as the discharging process. Two estimation procedures are proposed. One is the classical maximum likelihood, while the other relates to the expectation maximisation algorithm. We will present some results obtained by applying this model to a set of real data from a group of hospitals in Victoria (Australia).

stat.AP↗
arXiv2016

Solution for the Indefinite Integral of the Standard Normal Probability Density Function

Conventional wisdom assumes that the indefinite integral of the probability density function for the standard normal distribution cannot be expressed in finite elementary terms. While this is true, there is an expression for this anti-derivative in infinite elementary terms that, when being differentiated, directly yields the standard normal density function. We derive this function using infinite partial integration and review its relation to the cumulative distribution function for the standard normal distribution and the error function.

stat.OT↗
arXiv2015

Implicit Regression: Detecting Constants and Inverse Relationships with Bivariate Random Error

In 2011, Wooten introduced Non-Response Analysis the founding theory in Implicit Regression where Implicit Regression treats the variables implicitly as codependent variables and not as an explicit function with dependent or independent variables as in standard regression. The motivation of this paper is to introduce methods of implicit regression to determine the constant nature of a variable or the interactive term, and address inverse relationship among measured variables with random error present in both directions.

stat.AP↗
arXiv2015

Hot Hands, Streaks and Coin-flips: Numerical Nonsense in the New York Times

The existence of "Hot Hands" and "Streaks" in sports and gambling is hotly debated, but there is no uncertainty about the recent batting-average of the New York Times: it is now two-for-two in mangling and misunderstanding elementary concepts in probability and statistics; and mixing up the key points in a recent paper that re-examines earlier work on the statistics of streaks. In so doing, it's high-visibility articles have added to the general-public's confusion about probability, making it seem mysterious and paradoxical when it needn't be. However, those articles make excellent case studies on how to get it wrong, and for discussions in high-school and college classes focusing on quantitative reasoning, data analysis, probability and statistics. What I have written here is intended for that audience.

math.HO↗
arXiv2015

Joint limiting laws for high-dimensional independence tests

Testing independence is of significant interest in many important areas of large-scale inference. Using extreme-value form statistics to test against sparse alternatives and using quadratic form statistics to test against dense alternatives are two important testing procedures for high-dimensional independence. However, quadratic form statistics suffer from low power against sparse alternatives, and extreme-value form statistics suffer from low power against dense alternatives with small disturbances and may have size distortions due to its slow convergence. For real-world applications, it is important to derive powerful testing procedures against more general alternatives. Based on intermediate limiting distributions, we derive (model-free) joint limiting laws of extreme-value form and quadratic form statistics, and surprisingly, we prove that they are asymptotically independent. Given such asymptotic independencies, we propose (model-free) testing procedures to boost the power against general alternatives and also retain the correct asymptotic size. Under the high-dimensional setting, we derive the closed-form limiting null distributions, and obtain their explicit rates of uniform convergence. We prove their consistent statistical powers against general alternatives. We demonstrate the performance of our proposed test statistics in simulation studies. Our work provides very helpful insights to high-dimensional independence tests, and fills an important gap.

math.ST↗
arXiv2015

Impugning Randomness, Convincingly

John organized a state lottery and his wife won the main prize. You may feel that the event of her winning wasn't particularly random, but how would you argue that in a fair court of law? Traditional probability theory does not even have the notion of random events. Algorithmic information theory does, but it is not applicable to real-world scenarios like the lottery one. We attempt to rectify that.

stat.OT↗
arXiv2016

Chaos in Fractionally Integrated Generalized Autoregressive Conditional Heteroskedastic Processes

Fractionally integrated generalized autoregressive conditional heteroskedasticity (FIGARCH) arises in modeling of financial time series. FIGARCH is essentially governed by a system of nonlinear stochastic difference equations ${u_t}$ = ${z_t}$ $(1-\sum\limits_{j=1}^q β_j L^j)σ_{t}^2 = ω+(1-\sum\limits_{j=1}^q β_j L^j - (\sum\limits_{k=1}^p φ_k L^k) (1-L)^d) u_t^2$, where $ω\in$ R, and $β_j\in$ R are constant parameters, $\{u_t\}_{{t\in}^+}$ and $\{σ_t\}_{{t\in}^+}$ are the discrete time real valued stochastic processes which represent FIGARCH (p,d,q) and stochastic volatility, respectively. Moreover, L is the backward shift operator, i.e. $L^d u_t \equiv u_{t-d}$ (d is the fractional differencing parameter 0$<$d$<$1). In this work, we have studied the chaoticity properties of FIGARCH (p,d,q) processes by computing mutual information, correlation dimensions, FNNs (False Nearest Neighbour), the Lyapunov exponents, and for both the stochastic difference equation given above and for the financial time series. We have observed that maximal Lyapunov exponents are negative, therefore, it can be suggested that FIGARCH (p,d,q) is not deterministic chaotic process.

q-fin.MF↗
arXiv2018

Bayesian prediction for physical models with application to the optimization of the synthesis of pharmaceutical products using chemical kinetics

Quality control in industrial processes is increasingly making use of prior scientific knowledge, often encoded in physical models that require numerical approximation. Statistical prediction, and subsequent optimization, is key to ensuring the process output meets a specification target. However, the numerical expense of approximating the models poses computational challenges to the identification of combinations of the process factors where there is confidence in the quality of the response. Recent work in Bayesian computation and statistical approximation (emulation) of expensive computational models is exploited to develop a novel strategy for optimizing the posterior probability of a process meeting specification. The ensuing methodology is motivated by, and demonstrated on, a chemical synthesis process to manufacture a pharmaceutical product, within which an initial set of substances evolve according to chemical reactions, under certain process conditions, into a series of new substances. One of these substances is a target pharmaceutical product and two are unwanted by-products. The aim is to determine the combinations of process conditions and amounts of initial substances that maximize the probability of obtaining sufficient target pharmaceutical product whilst ensuring unwanted by-products do not exceed a given level. The relationship between the factors and amounts of substances of interest is theoretically described by the solution to a system of ordinary differential equations incorporating temperature dependence. Using data from a small experiment, it is shown how the methodology can approximate the multivariate posterior predictive distribution of the pharmaceutical target and by-products, and therefore identify suitable operating values. Materials to replicate the analysis can be found at www.github.com/amo105/chemicalkinetics.

stat.OT↗
arXiv2016

Impact of degree truncation on the spread of a contagious process on networks

Understanding how person-to-person contagious processes spread through a population requires accurate information on connections between population members. However, such connectivity data, when collected via interview, is often incomplete due to partial recall, respondent fatigue or study design, e.g., fixed choice designs (FCD) truncate out-degree by limiting the number of contacts each respondent can report. Past research has shown how FCD truncation affects network properties, but its implications for predicted speed and size of spreading processes remain largely unexplored. To study the impact of degree truncation on spreading processes, we generated collections of synthetic networks containing specific properties (degree distribution, degree-assortativity, clustering), and also used empirical social network data from 75 villages in Karnataka, India. We simulated FCD using various truncation thresholds and ran a susceptible-infectious-recovered (SIR) process on each network. We found that spreading processes propagated on truncated networks resulted in slower and smaller epidemics, with a sudden decrease in prediction accuracy at a level of truncation that varied by network type. Our results have implications beyond FCD to truncation due to any limited sampling from a larger network. We conclude that knowledge of network structure is important for understanding the accuracy of predictions of process spread on degree truncated networks.

physics.soc-ph↗
arXiv2016

Modelling the level of adoption of analytical tools; An implementation of multi-criteria evidential reasoning

In the future, competitive advantages will be given to organisations that can extract valuable information from massive data and make better decisions. In most cases, this data comes from multiple sources. Therefore, the challenge is to aggregate them into a common framework in order to make them meaningful and useful. This paper will first review the most important multi-criteria decision analysis methods (MCDA) existing in current literature. We will offer a novel, practical and consistent methodology based on a type of MCDA, to aggregate data from two different sources into a common framework. Two datasets that are different in nature but related to the same topic are aggregated to a common scale by implementing a set of transformation rules. This allows us to generate appropriate evidence for assessing and finally prioritising the level of adoption of analytical tools in four types of companies. A numerical example is provided to clarify the form for implementing this methodology. A six-step process is offered as a guideline to assist engineers, researchers or practitioners interested in replicating this methodology in any situation where there is a need to aggregate and transform multiple source data.

stat.AP↗
arXiv2016

Interactive graphics for functional data analyses

Although there are established graphics that accompany the most common functional data analyses, generating these graphics for each dataset and analysis can be cumbersome and time consuming. Often, the barriers to visualization inhibit useful exploratory data analyses and prevent the development of intuition for a method and its application to a particular dataset. The refund.shiny package was developed to address these issues for several of the most common functional data analyses. After conducting an analysis, the plot_shiny() function is used to generate an interactive visualization environment that contains several distinct graphics, many of which are updated in response to user input. These visualizations reduce the burden of exploratory analyses and can serve as a useful tool for the communication of results to non-statisticians.

stat.OT↗