Searcharxiv⌕ Search

arXiv subjects

Elena Kulinskaya

Publications and source records attributed to Elena Kulinskaya.

17 recordsLinked to original sources

Point and interval estimators of a changepoint in stochastical dominance between two distributions

For differences between means of continuous data from independent groups, the customary scale-free measure of effect is the standardized mean difference (SMD). To justify use of SMD, one should be reasonably confident that the group-level variances are equal. Empirical evidence often contradicts this assumption. Thus, we have investigated an alternate approach, based on stochastic ordering of the treatment and control distributions, that takes into account means and variances. For applying stochastic ordering, our development yields a key quantity, $\mathsf{A}$, the outcome value at which the direction of the ordering of the treatment and control distributions changes. Using an extensive simulation, we studied relative bias of point estimators of $\mathsf{A}$ and coverage and relative width of bootstrap confidence intervals.

math.ST↗

Simulations for estimation of random effects and overall effect in three-level meta-analysis of standardized mean differences using constant and inverse-variance weights

We consider a three-level meta-analysis of standardized mean differences. The standard method of estimation uses inverse-variance weights and REML/PL estimation of variance components for the random effects. We introduce new moment-based point and interval estimators for the two variance components and related estimators of the overall mean. Similar to traditional analysis of variance, our method is based on two conditional $Q$ statistics with effective-sample-size weights. We study, by simulation, bias and coverage of these new estimators. For comparison, we also study bias and coverage of the REML/PL-based approach as implemented in {\it rma.mv} in {\it metafor}. Our results demonstrate that the new methods are often considerably better and do not have convergence problems, which plague the standard analysis.

stat.ME↗

Simulations for Meta-analysis of Magnitude Measures

Meta-analysis aims to combine effect measures from several studies. For continuous outcomes, the most popular effect measures use simple or standardized differences in sample means. However, a number of applications focus on the absolute values of these effect measures (i.e., unsigned magnitude effects). We provide statistical methods for meta-analysis of magnitude effects based on standardized mean differences. We propose a suitable statistical model for random-effects meta-analysis of absolute standardized mean differences (ASMD), investigate a number of statistical methods for point and interval estimation, and provide practical recommendations for choosing among them.

stat.ME↗

Simulations for estimation of heterogeneity variance and overall effect with constant and inverse-variance weights in meta-analysis of difference in standardized means (DSM)

When the individual studies assembled for a meta-analysis report means ($μ_C$, $μ_T$) for their treatment (T) and control (C) arms, but those data are on different scales or come from different instruments, the customary measure of effect is the standardized mean difference (SMD). The SMD is defined as the difference between the means in the treatment and control arms, standardized by the assumed common standard deviation, $σ$. However, if the variances in the two arms differ, there is no consensus on a definition of SMD. Thus, we propose a new effect measure, the difference of standardized means (DSM), defined as $Δ= μ_T/σ_T - μ_C/σ_C$. The estimated DSM can easily be used as an effect measure in standard meta-analysis. For random-effects meta-analysis of DSM, we introduce new point and interval estimators of the between-studies variance ($τ^2$) based on the $Q$ statistic with effective-sample-size weights, $Q_F$. We study, by simulation, bias and coverage of these new estimators of $τ^2$ and related estimators of $Δ$. For comparison, we also study bias and coverage of well-known estimators based on the $Q$ statistic with inverse-variance weights, $Q_{IV}$, such as the Mandel-Paule, DerSimonian-Laird, and restricted-maximum-likelihood estimators.

stat.ME↗

Simulations for the Q statistic with constant and inverse variance weights for binary effect measures

Cochran's $Q$ statistic is routinely used for testing heterogeneity in meta-analysis. Its expected value (under an incorrect null distribution) is part of several popular estimators of the between-study variance, $τ^2$. Those applications generally do not account for the studies' use of estimated variances in the inverse-variance weights that define $Q$ (more explicitly, $Q_{IV}$). Importantly, those weights make approximating the distribution of $Q_{IV}$ rather complicated. As an alternative, we are investigating a $Q$ statistic, $Q_F$, whose constant weights use only the studies' arm-level sample sizes. For log-odds-ratio, log-relative-risk, and risk difference as the measure of effect, these simulations study approximations to the distributions of $Q_F$ and $Q_{IV}$, as the basis for tests of heterogeneity. We present the results in 132 Figures, 153 pages in total.

stat.ME↗

Simulations for estimation of heterogeneity variance $τ^2$ in constant and inverse variance weights meta-analysis of log-odds-ratios

A number of popular estimators of the between-study variance, $τ^2$, are based on the Cochran's $Q$ statistic for testing heterogeneity in meta analysis. We introduce new point and interval estimators of $τ^2$ for log-odds-ratio. These include new DerSimonian-Kacker-type moment estimators based on the first moment of $Q_F$, the $Q$ statistic with effective-sample-size weights, and novel median-unbiased estimators. We study, by simulation, bias and coverage of these new estimators of $τ^2$ and, for comparative purposes, bias and coverage of a number of well-known estimators based on the $Q$ statistic with inverse-variance weights, $Q_{IV}$, such as the Mandel-Paule, DerSimonian-Laird, and restricted-maximum-likelihood estimators, and an estimator based on the Kulinskaya-Dollinger (2015) improved approximation to $Q_{IV}$.

stat.ME↗

A simulation study of the estimation quality in the double-Cox model with shared frailty for non-proportional hazards survival analysis

The Cox regression, a semi-parametric method of survival analysis, is extremely popular in biomedical applications. The proportional hazards assumption is a key requirement in the Cox model. To accommodate non-proportional hazards, we propose to parameterise the shape parameter of the baseline hazard function using the additional, separate Cox-regression term which depends on the vector of the covariates. We call this model the double-Cox model. The R programs for fitting the double-Cox model are available on Github. We formally introduce the double-Cox model with shared frailty and investigate, by simulation, the estimation bias and the coverage of the proposed point and interval estimation methods for the Gompertz and the Weibull baseline hazards. In applications with low frailty variance and a large number of clusters, the marginal likelihood estimation is almost unbiased and the profile likelihood-based confidence intervals provide good coverage for all model parameters. We also compare the results from the over-fitted double-Cox model to those from the standard Cox model with frailty in the case of the scale-only proportional hazards. Results of our simulations on the bias and coverage of the model parameters are provided in 12 Tables and in 145 A4 Figures, 178 pages in total.

stat.ME↗

Simulation study of Q statistic with constant weights for testing and estimation of heterogeneity of standardized mean differences in meta-analysis

Cochran's $Q$ statistic is routinely used for testing heterogeneity in meta-analysis. Its expected value is also used for estimation of between-study variance $τ^2$. Cochran's $Q$, or $Q_{IV}$, uses estimated inverse-variance weights which makes approximating its distribution rather complicated. As an alternative, we are investigating a new $Q$ statistic, $Q_F$, whose constant weights use only the studies' effective sample sizes. For standardized mean difference as the measure of effect, we study, by simulation, approximations to distributions of $Q_{IV}$ and $Q_F$, as the basis for tests of heterogeneity and for new point and interval estimators of the between-study variance $τ^2$. These include new DerSimonian-Kacker (2007)-type moment estimators based on the first moment of $Q_F$, and novel median-unbiased estimators of $τ^2$.

stat.ME↗

Simulations for a Q statistic with constant weights to assess heterogeneity in meta-analysis of mean difference

A variety of problems in random-effects meta-analysis arise from the conventional $Q$ statistic, which uses estimated inverse-variance (IV) weights. In previous work on standardized mean difference and log-odds-ratio, we found superior performance with an estimator of the overall effect whose weights use only group-level sample sizes. The $Q$ statistic with those weights has the form proposed by DerSimonian and Kacker. The distribution of this $Q$ and the $Q$ with IV weights must generally be approximated. We investigate approximations for those distributions, as a basis for testing and estimating the between-study variance ($τ^2$). Some approximations require the variance and third moment of $Q$, which we derive. We describe the design and results of a simulation study, with mean difference as the effect measure, which provides a framework for assessing accuracy of the approximations, level and power of the tests, and bias in estimating $τ^2$. Use of $Q$ with sample-size-based weights and its exact distribution (available for mean difference and evaluated by Farebrother's algorithm) provides precise levels even for very small and unbalanced sample sizes. The corresponding estimator of $τ^2$ is almost unbiased for 10 or more small studies. Under these circumstances this performance compares favorably with the extremely liberal behavior of the standard tests of heterogeneity and the largely biased estimators based on inverse-variance weights.

stat.ME↗

Exploring Consequences of Simulation Design for Apparent Performance of Statistical Methods. 2: Results from simulations with normally and uniformly distributed sample sizes

This report continues our investigation of effects a simulation design may have on the conclusions on performance of statistical methods. In the context of meta-analysis of log-odds-ratios, we consider five generation mechanisms for control probabilities and log-odds-ratios. Our first report (Kulinskaya et al. 2020) considered constant sample sizes. Here we report on the results for normally and uniformly distributed sample sizes.

stat.ME↗

Exploring Consequences of Simulation Design for Apparent Performance of Statistical Methods. 1: Results from simulations with constant sample sizes

Contemporary statistical publications rely on simulation to evaluate performance of new methods and compare them with established methods. In the context of meta-analysis of log-odds-ratios, we investigate how the ways in which simulations are implemented affect such conclusions. Choices of distributions for sample sizes and/or control probabilities considerably affect conclusions about statistical methods. Here we report on the results for constant sample sizes. Our two subsequent publications will cover normally and uniformly distributed sample sizes.

stat.ME↗

Simulation study of estimating between-study variance and overall effect in meta-analyses of log-response-ratio for lognormal data

Methods for random-effects meta-analysis require an estimate of the between-study variance, $τ^2$. The performance of estimators of $τ^2$ (measured by bias and coverage) affects their usefulness in assessing heterogeneity of study-level effects, and also the performance of related estimators of the overall effect. For the effect measure log-response-ratio (LRR, also known as the logarithm of the ratio of means, RoM), we review four point estimators of $τ^2$ (the popular methods of DerSimonian-Laird (DL), restricted maximum likelihood, and Mandel and Paule (MP), and the less-familiar method of Jackson), four interval estimators for $τ^2$ (profile likelihood, Q-profile, Biggerstaff and Jackson, and Jackson), five point estimators of the overall effect (the four related to the point estimators of $τ^2$ and an estimator whose weights use only study-level sample sizes), and seven interval estimators for the overall effect (four based on the point estimators for $τ^2$, the Hartung-Knapp-Sidik-Jonkman (HKSJ) interval, a modification of HKSJ that uses the MP estimator of $τ^2$ instead of the DL estimator, and an interval based on the sample-size-weighted estimator). We obtain empirical evidence from extensive simulations of data from lognormal distributions.

stat.ME↗

Simulation study of estimating between-study variance and overall effect in meta-analyses of mean difference

Methods for random-effects meta-analysis require an estimate of the between-study variance, $τ^2$. The performance of estimators of $τ^2$ (measured by bias and coverage) affects their usefulness in assessing heterogeneity of study-level effects, and also the performance of related estimators of the overall effect. For the effect measure mean difference (MD), we review five point estimators of $τ^2$ (the popular methods of DerSimonian-Laird, restricted maximum likelihood, and Mandel and Paule (MP); the less-familiar method of Jackson; and a new method (WT) based on the improved approximation to the distribution of the $Q$ statistic by \cite{kulinskaya2004welch}), five interval estimators for $τ^2$ (profile likelihood, Q-profile, Biggerstaff and Jackson, Jackson, and the new WT method), six point estimators of the overall effect (the five related to the point estimators of $τ^2$ and an estimator whose weights use only study-level sample sizes), and eight interval estimators for the overall effect (five based on the point estimators for $τ^2$, the Hartung-Knapp-Sidik-Jonkman (HKSJ) interval, a modification of HKSJ, and an interval based on the sample-size-weighted estimator). We obtain empirical evidence from extensive simulations and an example.

stat.ME↗

Simulation study of estimating between-study variance and overall effect in meta-analysis of standardized mean difference

Methods for random-effects meta-analysis require an estimate of the between-study variance, $τ^2$. The performance of estimators of $τ^2$ (measured by bias and coverage) affects their usefulness in assessing heterogeneity of study-level effects, and also the performance of related estimators of the overall effect. For the effect measure standardized mean difference (SMD), we provide the results from extensive simulations on five point estimators of $τ^2$ (the popular methods of DerSimonian-Laird, restricted maximum likelihood, and Mandel and Paule (MP); the less-familiar method of Jackson; the new method (KDB) based on the improved approximation to the distribution of the Q statistic by Kulinskaya, Dollinger and Bjørkestøl (2011) ), five interval estimators for $τ^2$ (profile likelihood, Q-profile, Biggerstaff and Jackson, Jackson, and the new KDB method), six point estimators of the overall effect (the five related to the point estimators of $τ^2$ and an estimator whose weights use only study-level sample sizes), and eight interval estimators for the overall effect (five based on the point estimators for $τ^2$; the Hartung-Knapp-Sidik-Jonkman (HKSJ) interval; a modification of HKSJ; and an interval based on the sample-size-weighted estimator).

stat.ME↗

Simulation study of estimating between-study variance and overall effect in meta-analysis of odds-ratios

Random-effects meta-analysis requires an estimate of the between-study variance, $τ^2$. We study methods of estimation of $τ^2$ and its confidence interval in meta-analysis of odds ratio, and also the performance of related estimators of the overall effect. We provide results of extensive simulations on five point estimators of $τ^2$ (the popular methods of DerSimonian-Laird, restricted maximum likelihood, and Mandel and Paule; the less-familiar method of Jackson; and the new method (KD) based on the improved approximation to the distribution of the Q statistic by Kulinskaya and Dollinger (2015)); five interval estimators for $τ^2$ (profile likelihood, Q-profile, Biggerstaff and Jackson, Jackson, and KD), six point estimators of the overall effect (the five inverse-variance estimators related to the point estimators of $τ^2$ and an estimator (SSW) whose weights use only study-level sample sizes), and eight interval estimators for the overall effect (five based on the point estimators for $τ^2$; the Hartung-Knapp-Sidik-Jonkman (HKSJ) interval; a KD-based modification of HKSJ; and an interval based on the sample-size-weighted estimator). Results of our simulations show that none of the point estimators of $τ^2$ can be recommended, however the new KD estimator provides a reliable coverage of $τ^2$. Inverse-variance estimators of the overall effect are substantially biased. The SSW estimator of the overall effect and the related confidence interval provide the reliable point and interval estimation of log-odds-ratio.

stat.ME↗

Testing for Homogeneity in Meta-Analysis I. The One Parameter Case: Standardized Mean Difference

Meta-analysis seeks to combine the results of several experiments in order to improve the accuracy of decisions. It is common to use a test for homogeneity to determine if the results of the several experiments are sufficiently similar to warrant their combination into an overall result. Cochran's Q statistic is frequently used for this homogeneity test. It is often assumed that Q follows a chi-square distribution under the null hypothesis of homogeneity, but it has long been known that this asymptotic distribution for Q is not accurate for moderate sample sizes. Here we present formulas for the mean and variance of Q under the null hypothesis which represent O(1/n) corrections to the corresponding chi-square moments in the one parameter case. The formulas are fairly complicated, and so we provide a program (available at http://www.imperial.ac.uk/stathelp/researchprojects/metaanalysis) for making the necessary calculations. We apply the results to the standardized mean difference (Cohen's d-statistic) and consider two approximations: a gamma distribution with estimated shape and scale parameters and the chi-square distribution with fractional degrees of freedom equal to the estimated mean of Q. We recommend the latter distribution as an approximate distribution for Q to use for testing the null hypothesis.

stat.ME↗

On two-sided p-values for non-symmetric distributions

Two-sided statistical tests and p-values are well defined only when the test statistic in question has a symmetric distribution. A new two-sided p-value called conditional p-value $P_C$ is introduced here. It is closely related to the doubled p-value and has an intuitive appeal. Its use is advocated for both continuous and discrete distributions. An important advantage of this p-value is that equivalent 1-sided tests are transformed into $P_C$-equivalent 2-sided tests. It is compared to the widely used doubled and minimum likelihood p-values. Examples include the variance test, the binomial and the Fisher's exact test.

math.ST↗