arXiv subjects

Ravi Varadhan

Publications and source records attributed to Ravi Varadhan.

14 recordsLinked to original sources

arXiv2026

Bayesian Imputation for Unplayed Games in Round-Robin Chess Tournaments: Application to Grand Chess Tour, Bucharest 2026

When a player withdraws mid-tournament from a round-robin chess event, organizers face a fundamental problem: how should scores be assigned for games that were never played? Current FIDE guidelines specify annulment if withdrawal occurs before 50% of games are completed, and forfeit (awarding unplayed opponents a full point) thereafter. This dichotomous rule creates arbitrary discontinuities and can substantially distort final standings. We develop a Bayesian framework based on best linear unbiased prediction (BLUP) that optimally combines pre-tournament ratings with observed performance, producing imputed scores that reflect both the withdrawn player's current form and the strength differentials among unplayed opponents. The estimator is consistent, point-conserving, and minimizes mean squared error among linear unbiased predictors. A Monte Carlo simulation study on 180,000 simulated tournaments demonstrates that Bayesian BLUP imputation reduces prediction error by 26% overall compared to FIDE's current rule, with improvements of 41% over forfeit and 12% over annulment. The largest gains occur when the withdrawn player is underperforming, the most common withdrawal scenario. We further show that annulment achieves 15-45% lower RMSE than forfeit across all scenarios. The methodology is applied to GM Alireza Firouzja's withdrawal at Grand Chess Tour, Bucharest 2026, where Bayesian imputation would have awarded unplayed opponents 0.55-0.70 points rather than the 1.0 awarded under forfeit rules. An open-source R Shiny application is provided for tournament organizers. We recommend that FIDE adopt Bayesian imputation for World Championship cycle events, or at minimum replace the current dichotomous rule with uniform annulment.

stat.ME↗

arXiv2025

SensIAT: An R Package for Conducting Sensitivity Analysis of Randomized Trials with Irregular Assessment Times

This paper introduces an R package SensIAT that implements a sensitivity analysis methodology, based on augmented inverse intensity weighting, for randomized trials with irregular and potentially informative assessment times. Targets of inference involve the population mean outcome in each treatment arm as well as the difference in these means (i.e., treatment effect) at specified times after randomization. This methodology is useful in settings where there is concern that study participants are either more, or less, likely to have assessments at times when their outcomes are worse. In such settings, unadjusted estimates can be biased. The methodology allows researchers to see how inferences are impacted by a range of assumptions about the strength and direction of informative timing in each arm, while incorporating flexible semi-parametric modeling. We describe the functions implemented in SensIAT and illustrate them through an analysis of a synthetic dataset motivated by the HAP2 asthma randomized clinical trial.

stat.ME↗

arXiv2025

Accelerating Proximal Gradient-type Algorithms using Damped Anderson Acceleration with Restarts and Nesterov Initialization

Despite their frequent slow convergence, proximal gradient schemes are widely used in large-scale optimization tasks due to their tremendous stability, scalability, and ease of computation. In this paper, we develop and investigate a general two-phase scheme for accelerating the convergence of proximal gradient algorithms. By using Nesterov's momentum method in an initialization phase, our procedure delivers fast initial descent that is robust to the choice of starting value. Once iterates are much closer to the solution after the first phase, we utilize a variation of Anderson acceleration to deliver more rapid local convergence in the second phase. Drawing upon restarting schemes developed for Nesterov acceleration, we can readily identify points where it is advantageous to switch from the first to the second phase, which enables use of the procedure without requiring one to specify the number of iterations used in each phase. For the second phase, we adapt and extend a version of Anderson acceleration with algorithm restarts, and we introduce a subsetted version of this procedure that improves performance in problems with substantial sparsity. Through simulation studies involving four representative optimization problems, we show that our proposed algorithm can generate substantial improvements over competing acceleration methods.

stat.CO↗

arXiv2025

Quantifying uncertainty of individualized treatment effects in right-censored survival data: A comparison of Bayesian additive regression trees and causal survival forest

Estimation of individualized treatment effects (ITE), also known as conditional average treatment effects (CATE), is an active area of methodology development. However, much less attention has been paid to the quantification of uncertainty of ITE/CATE estimates in right-censored survival data. Here we undertake an extensive simulation study to examine the coverage of interval estimates from two popular estimation algorithms, Bayesian additive regression trees (BART) and causal survival forest (CSF). We conducted simulation designs from 3 different settings: first, in a setting where BART was developed for an accelerated failure time model; second, where CSF was developed; and finally, a ``neutral'' simulation taken from a setting where neither BART nor CSF was developed. BART outperformed CSF in all three simulation settings. Both the BART and CSF algorithms involve multiple hyperparameters, and BART credible intervals had better coverage than the CSF confidence intervals under the default values, as well as under optimized values, of these hyperparameters.

stat.ME↗

arXiv2025

Optimization problems constrained by parameter sums

This article presents a discussion of optimization problems where the objective function f(x) has parameters that are constrained by some scaling, so that q(x) = constant, where this function q() involves a sum of the parameters, their squares, or similar simple function. Our focus is on ways to use standardized optimization programs to solve such problems rather than specialized codes. Examples are presented with R.

math.OC↗

arXiv2025

Various approaches to solving nonlinear equations

Modelling real world systems frequently requires the solution of systems of nonlinear equations. A number of approaches have been suggested and developed for this computational problem. However, it is also possible to attempt solutions using more general nonlinear least squares or function minimization techniques. There are concerns, nonetheless, that we may fail to find solutions, or that the process will be inefficient. Examples are presented with R with the goal of providing guidance on the solution of nonlinear equations problems.

stat.CO↗

arXiv2022

Semi-Parametric Sensitivity Analysis for Trials with Irregular and Informative Assessment Times

Many trials are designed to collect outcomes at or around pre-specified times after randomization. If there is variability in the times when participants are actually assessed, this can pose a challenge to learning the effect of treatment, since not all participants have outcome assessments at the times of interest. Furthermore, observed outcome values may not be representative of all participants' outcomes at a given time. Methods have been developed that account for some types of such irregular and informative assessment times; however, since these methods rely on untestable assumptions, sensitivity analyses are needed. We develop a methodology that is benchmarked at the explainable assessmen (EA) assumption, under which assessment and outcomes at each time are related only through data collected prior to that time. Our method uses an exponential tilting assumption, governed by a sensitivity analysis parameter, that posits deviations from the EA assumption. Our inferential strategy is based on a new influence function-based, augmented inverse intensity-weighted estimator. Our approach allows for flexible semiparametric modeling of the observed data, which is separated from specification of the sensitivity parameter. We apply our method to a randomized trial of low-income individuals with uncontrolled asthma, and we illustrate implementation of our estimation procedure in detail.

stat.ME↗

arXiv2022

Improved Small Domain Estimation via Compromise Regression Weights

Shrinkage estimates of small domain parameters typically utilize a combination of a noisy "direct" estimate that only uses data from a specific small domain and a more stable regression estimate. When the regression model is misspecified, estimation performance for the noisier domains can suffer due to substantial shrinkage towards a poorly estimated regression surface. In this paper, we introduce a new class of robust, empirically-driven regression weights that target estimation of the small domain means under potential misspecification of the global regression model. Our regression weights are a convex combination of the model-based weights associated with the best linear unbiased predictor (BLUP) and those associated with the observed best predictor (OBP). The compromise parameter in this convex combination is found by minimizing a novel, unbiased estimate of the mean-squared prediction error for the small domain means, and we label the associated small domain estimates the "compromise best predictor", or CBP. Using a data-adaptive mixture for the regression weights enables the CBP to possess the robustness of the OBP while retaining the main advantages of the EBLUP whenever the regression model is correct. We demonstrate the use of the CBP in an application estimating gait speed in older adults.

stat.ME↗

arXiv2021

Network Security Modeling using NetFlow Data: Detecting Botnet attacks in IP Traffic

Cybersecurity, security monitoring of malicious events in IP traffic, is an important field largely unexplored by statisticians. Computer scientists have made significant contributions in this area using statistical anomaly detection and other supervised learning methods to detect specific malicious events. In this research, we investigate the detection of botnet command and control (C&C) hosts in massive IP traffic. We use the NetFlow data, the industry standard for monitoring of IP traffic for exploratory analysis and extracting new features. Using statistical as well as deep learning models, we develop a statistical intrusion detection system (SIDS) to predict traffic traces identified with malicious attacks. Employing interpretative machine learning techniques, botnet traffic signatures are derived. These models successfully detected botnet C&C hosts and compromised devices. The results were validated by matching predictions to existing blacklists of published malicious IP addresses.

cs.CR↗

arXiv2019

Brand vs. Generic: Addressing Non-Adherence, Secular Trends, and Non-Overlap

While generic drugs offer a cost-effective alternative to brand name drugs, regulators need a method to assess therapeutic equivalence in a post market setting. We develop such a method in the context of assessing the therapeutic equivalence of immediate release (IM) venlafaxine, based on a large insurance claims dataset provided by OptumLabs\textsuperscript{\textregistered}. To properly address this question, our methodology must deal with issues of non-adherence, secular trends in health outcomes, and lack of treatment overlap due to sharp uptake of the generic once it becomes available. We define, identify (under assumptions) and estimate (using G-computation) a causal effect for a time-to-event outcome by extending regression discontinuity to survival curves. We do not find evidence for a lack of therapeutic equivalence of brand and generic IM venlafaxine.

stat.AP↗

arXiv2018

SQUAREM: An R Package for Off-the-Shelf Acceleration of EM, MM and Other EM-like Monotone Algorithms

We discuss R package SQUAREM for accelerating iterative algorithms which exhibit slow, monotone convergence. These include the well-known expectation-maximization algorithm, majorize-minimize (MM), and other EM-like algorithms such as expectation conditional maximization, and generalized EM algorithms. We demonstrate the simplicity, generality, and power of SQUAREM through a wide array of applications of EM/MM problems, including binary Poisson mixture, factor analysis, interval censoring, genetics admixture, and logistic regression maximum likelihood estimation (an MM problem). We show that SQUAREM is easy to apply, and can accelerate any fixed-point, smooth, contraction mapping with linear convergence rate. Squared iterative scheme (Squarem) algorithm provides significant speed-up of EM-like algorithms. The margin of the advantage for Squarem is especially huge for high-dimensional problems or when EM step is relatively time-consuming to evaluate. Squarem can be used off-the-shelf since there is no need for the user to tweak any control parameters to optimize performance. Given its remarkable ease of use, Squarem may be considered as a default accelerator for slowly converging EM-like algorithms. All the comparisons of CPU computing time in the paper are made on a quad-core 2.3 GHz Intel Core i7 Mac computer. R Package SQUAREM can be downloaded at https://cran.r-project.org/web/packages/SQUAREM/index.html.

stat.CO↗

arXiv2018

Damped Anderson acceleration with restarts and monotonicity control for accelerating EM and EM-like algorithms

The expectation-maximization (EM) algorithm is a well-known iterative method for computing maximum likelihood estimates from incomplete data. Despite its numerous advantages, a main drawback of the EM algorithm is its frequently observed slow convergence which often hinders the application of EM algorithms in high-dimensional problems or in other complex settings.To address the need for more rapidly convergent EM algorithms, we describe a new class of acceleration schemes that build on the Anderson acceleration technique for speeding fixed-point iterations. Our approach is effective at greatly accelerating the convergence of EM algorithms and is automatically scalable to high dimensional settings. Through the introduction of periodic algorithm restarts and a damping factor, our acceleration scheme provides faster and more robust convergence when compared to un-modified Anderson acceleration while also improving global convergence. Crucially, our method works as an "off-the-shelf" method in that it may be directly used to accelerate any EM algorithm without relying on the use of any model-specific features or insights. Through a series of simulation studies involving five representative problems, we show that our algorithm is substantially faster than the existing state-of-art acceleration schemes.

stat.CO↗

arXiv2018

Bayesian Bivariate Subgroup Analysis for Risk-Benefit Evaluation

Subgroup analysis is a frequently used tool for evaluating heterogeneity of treatment effect and heterogeneity in treatment harm across observed baseline patient characteristics. While treatment efficacy and adverse event measures are often reported separately for each subgroup, analyzing their within-subgroup joint distribution is critical for better informed patient decision-making. In this paper, we describe Bayesian models for performing a subgroup analysis to compare the joint occurrence of a primary endpoint and an adverse event between two treatment arms. Our approaches emphasize estimation of heterogeneity in this joint distribution across subgroups, and our approaches directly accommodate subgroups with small numbers of observed primary and adverse event combinations. In addition, we describe several ways in which our models may be used to generate interpretable summary measures of benefit-risk tradeoffs for each subgroup. The methods described here are illustrated throughout using a large cardiovascular trial (N = 9,361) investigating the efficacy of an intervention for reducing systolic blood pressure to a lower-than-usual target.

stat.AP↗

arXiv2017

Individualized Treatment Effects with Censored Data via Fully Nonparametric Bayesian Accelerated Failure Time Models

Individuals often respond differently to identical treatments, and characterizing such variability in treatment response is an important aim in the practice of personalized medicine. In this article, we describe a non-parametric accelerated failure time model that can be used to analyze heterogeneous treatment effects (HTE) when patient outcomes are time-to-event. By utilizing Bayesian additive regression trees and a mean-constrained Dirichlet process mixture model, our approach offers a flexible model for the regression function while placing few restrictions on the baseline hazard. Our non-parametric method leads to natural estimates of individual treatment effect and has the flexibility to address many major goals of HTE assessment. Moreover, our method requires little user input in terms of tuning parameter selection or subgroup specification. We illustrate the merits of our proposed approach with a detailed analysis of two large clinical trials for the prevention and treatment of congestive heart failure using an angiotensin-converting enzyme inhibitor. The analysis revealed considerable evidence for the presence of HTE in both trials as demonstrated by substantial estimated variation in treatment effect and by high proportions of patients exhibiting strong evidence of having treatment effects which differ from the overall treatment effect.

stat.ME↗