arXiv subjects

Lu Tian

Publications and source records attributed to Lu Tian.

At least 37 records · Page 2Linked to original sources

arXiv2025

Comment on "Average Hazard as Harmonic Mean" by Chiba

In a recent article published in Pharmaceutical Statistics, Chiba proposed a reinterpretation of the average hazard as a harmonic mean of the hazard function and questioned the validity of the Kaplan-Meier plug-in estimator when the truncation time does not coincide with an observed event time. In this commentary, we examine the arguments presented and highlight several points that warrant clarification. Through simulation studies, we further show that the plug-in estimator provides reliable estimates across a range of truncation times, even in small samples. These support the continued utilization of the Kaplan-Meier plug-in estimator for the average hazard and help clarify its proper interpretation and implementation.

stat.ME↗

arXiv2025

Latent Noise Injection for Private and Statistically Aligned Synthetic Data Generation

Synthetic Data Generation has become essential for scalable, privacy-preserving statistical analysis. While standard approaches based on generative models, such as Normalizing Flows, have been widely used, they often suffer from slow convergence in high-dimensional settings, frequently converging more slowly than the canonical $1/\sqrt{n}$ rate when approximating the true data distribution. To overcome these limitations, we propose a Latent Noise Injection method using Masked Autoregressive Flows (MAF). Instead of directly sampling from the trained model, our method perturbs each data point in the latent space and maps it back to the data domain. This construction preserves a one to one correspondence between observed and synthetic data, enabling synthetic outputs that closely reflect the underlying distribution, particularly in challenging high-dimensional regimes where traditional sampling struggles. Our procedure satisfies local $(ε, δ)$-differential privacy and introduces a single perturbation parameter to control the privacy-utility trade-off. Although estimators based on individual synthetic datasets may converge slowly, we show both theoretically and empirically that aggregating across $K$ studies in a meta analysis framework restores classical efficiency and yields consistent, reliable inference. We demonstrate that with a well-calibrated perturbation parameter, Latent Noise Injection achieves strong statistical alignment with the original data and robustness against membership inference attacks. These results position our method as a compelling alternative to conventional flow-based sampling for synthetic data sharing in decentralized and privacy-sensitive domains, such as biomedical research.

stat.ML↗

arXiv2025

Resilience Measures for the Surrogate Paradox

Surrogate markers are often used in clinical trials to evaluate treatment effects when primary outcomes are costly, invasive, or take a long time to observe. However, reliance on surrogates can lead to the surrogate paradox, where a treatment appears beneficial based on the surrogate but is actually harmful with respect to the primary outcome. In this paper, we propose formal measures to assess resilience against the surrogate paradox. Our setting assumes an existing study in which the surrogate marker and primary outcome have been measured (Study A) and a new study (Study B) in which only the surrogate is measured. Rather than assuming transportability of the conditional mean functions across studies, we consider a class of functions for Study B that deviate from those in Study A. Using these, we estimate the distribution of potential treatment effects on the unmeasured primary outcome and define resilience measures including a resilience probability, resilience bound, and resilience set. Our approach complements traditional surrogate validation methods by quantifying the plausibility of the surrogate paradox under controlled deviations from what is known from Study A. We investigate the performance of our proposed measures via a simulation study and application to two distinct HIV clinical trials.

stat.ME↗

arXiv2025

An IPCW Adjusted Win Statistics Approach in Clinical Trials Incorporating Equivalence Margins to Define Ties

In clinical trials, multiple outcomes of different priorities commonly occur as the patient's response may not be adequately characterized by a single outcome. Win statistics are appealing summary measures for between-group difference at more than one endpoint. When defining the result of pairwise comparisons of a time-to-event endpoint, it is desirable to allow ties to account for incomplete follow-up and not clinically meaningful difference in endpoints of interest. In this paper, we propose a class of win statistics for time-to-event endpoints with a user-specified equivalence margin. These win statistics are identifiable in the presence of right-censoring and do not depend on the censoring distribution. We then develop estimation and inference procedures for the proposed win statistics based on inverse-probability-of-censoring {weighting} (IPCW) adjustment to handle right-censoring. We conduct extensive simulations to investigate the operational characteristics of the proposed procedure in the finite sample setting. A real oncology trial is used to illustrate the proposed approach.

stat.ME↗

arXiv2025

Sampling-based federated inference for M-estimators with non-smooth objective functions

We propose a novel sampling-based federated learning framework for statistical inference on M-estimators with non-smooth objective functions, which frequently arise in modern statistical applications such as quantile regression and AUC maximization. Classical inference methods for such estimators are often computationally intensive or require nonparametric estimation of nuisance quantities. Our approach circumvents these challenges by leveraging Markov Chain Monte Carlo (MCMC) sampling and a second-stage perturbation scheme to efficiently estimate both the parameter of interest and its variance. In the presence of multiple sites with data-sharing constraints, we introduce an adaptive strategy to borrow information from potentially heterogeneous source sites without transferring individual-level data. This strategy selects source sites based on a dissimilarity measure and constructs an optimally weighted estimator using a lasso regularization. The resulting estimator has an oracle property, i.e., it achieves the optimal asymptotical efficiency by borrowing information from eligible sites while guarding against negative transfer. We establish consistency and asymptotic normality of our proposed estimators and validate the method through extensive simulations and a real-data application on type 2 diabetes. Our results demonstrate substantial gains in inference precision and underscore the importance of inclusive, data-adaptive analysis frameworks in federated learning settings.

stat.ME↗

arXiv2025

DL-QAT: Weight-Decomposed Low-Rank Quantization-Aware Training for Large Language Models

Improving the efficiency of inference in Large Language Models (LLMs) is a critical area of research. Post-training Quantization (PTQ) is a popular technique, but it often faces challenges at low-bit levels, particularly in downstream tasks. Quantization-aware Training (QAT) can alleviate this problem, but it requires significantly more computational resources. To tackle this, we introduced Weight-Decomposed Low-Rank Quantization-Aware Training (DL-QAT), which merges the advantages of QAT while training only less than 1% of the total parameters. Specifically, we introduce a group-specific quantization magnitude to adjust the overall scale of each quantization group. Within each quantization group, we use LoRA matrices to update the weight size and direction in the quantization space. We validated the effectiveness of our method on the LLaMA and LLaMA2 model families. The results show significant improvements over our baseline method across different quantization granularities. For instance, for LLaMA-7B, our approach outperforms the previous state-of-the-art method by 4.2% in MMLU on 3-bit LLaMA-7B model. Additionally, our quantization results on pre-trained models also surpass previous QAT methods, demonstrating the superior performance and efficiency of our approach.

cs.CV↗

arXiv2025

Conditional Inference for Secondary Outcomes Based on the Testing Result for the Primary Outcome in Clinical Trials

In clinical trials, inferences on clinical outcomes are often made conditional on specific selective processes. For instance, only when a treatment demonstrates a significant effect on the primary outcome, further analysis is conducted to investigate its efficacy on selected secondary outcomes. Similarly, inferences may also depend on whether a trial is terminated early at interim stage. While conventional approaches primarily aim to control the family-wise error rate through multiplicity adjustments, they do not necessarily ensure the desired statistical property of the inference result, when the analysis is conducted according to a selective process. For example, the conditional coverage level of a regular confidence interval under a selective processes can be very different from its nominal level even after adjustment for multiple testing. In this paper, we argue that the validity of the inference procedure conditional on selective process is important in many applications. In addition, we propose to construct confidence intervals with correct conditional coverage probability by accounting for related selective process. Specifically, our approach involves a pivotal quantity constructed by inversing the cumulative distribution function of a truncated normal distribution induced by the selective process. Theoretical justification and comprehensive simulations illustrate the effectiveness of this method in realistic settings. We also apply our method to analyze data from the SPRINT, resulting in more conservative but conditionally valid confidence intervals for the average treatment effect than those originally published.

stat.ME↗

arXiv2025

MonoGS++: Fast and Accurate Monocular RGB Gaussian SLAM

We present MonoGS++, a novel fast and accurate Simultaneous Localization and Mapping (SLAM) method that leverages 3D Gaussian representations and operates solely on RGB inputs. While previous 3D Gaussian Splatting (GS)-based methods largely depended on depth sensors, our approach reduces the hardware dependency and only requires RGB input, leveraging online visual odometry (VO) to generate sparse point clouds in real-time. To reduce redundancy and enhance the quality of 3D scene reconstruction, we implemented a series of methodological enhancements in 3D Gaussian mapping. Firstly, we introduced dynamic 3D Gaussian insertion to avoid adding redundant Gaussians in previously well-reconstructed areas. Secondly, we introduced clarity-enhancing Gaussian densification module and planar regularization to handle texture-less areas and flat surfaces better. We achieved precise camera tracking results both on the synthetic Replica and real-world TUM-RGBD datasets, comparable to those of the state-of-the-art. Additionally, our method realized a significant 5.57x improvement in frames per second (fps) over the previous state-of-the-art, MonoGS.

cs.CV↗

arXiv2025

Partial Convolution Meets Visual Attention

Designing an efficient and effective neural network has remained a prominent topic in computer vision research. Depthwise onvolution (DWConv) is widely used in efficient CNNs or ViTs, but it needs frequent memory access during inference, which leads to low throughput. FasterNet attempts to introduce partial convolution (PConv) as an alternative to DWConv but compromises the accuracy due to underutilized channels. To remedy this shortcoming and consider the redundancy between feature map channels, we introduce a novel Partial visual ATtention mechanism (PAT) that can efficiently combine PConv with visual attention. Our exploration indicates that the partial attention mechanism can completely replace the full attention mechanism and reduce model parameters and FLOPs. Our PAT can derive three types of blocks: Partial Channel-Attention block (PAT_ch), Partial Spatial-Attention block (PAT_sp) and Partial Self-Attention block (PAT_sf). First, PAT_ch integrates the enhanced Gaussian channel attention mechanism to infuse global distribution information into the untouched channels of PConv. Second, we introduce the spatial-wise attention to the MLP layer to further improve model accuracy. Finally, we replace PAT_ch in the last stage with the self-attention mechanism to extend the global receptive field. Building upon PAT, we propose a novel hybrid network family, named PATNet, which achieves superior top-1 accuracy and inference speed compared to FasterNet on ImageNet-1K classification and excel in both detection and segmentation on the COCO dataset. Particularly, our PATNet-T2 achieves 1.3% higher accuracy than FasterNet-T2, while exhibiting 25% higher GPU throughput and 24% lower CPU latency.

cs.CV↗

arXiv2025

On the two-step hybrid design for augmenting randomized trials using real-world data

Hybrid clinical trials, that borrow real-world data (RWD), are gaining interest, especially for rare diseases. They assume RWD and randomized control arm be exchangeable, but violations can bias results, inflate type I error, or reduce power. A two-step hybrid design first tests exchangeability, reducing inappropriate borrowing but potentially inflating type I error (Yuan et al., 2019). We propose four methods to better control type I error. Approach 1 estimates the variance of test statistics, rejecting the null hypothesis based on large sample normal approximation. Approach 2 uses a numerical approach for exact critical value determination. Approach 3 splits type I error rates by equivalence test outcome. Approach 4 adjusts the critical value only when equivalence is established. Simulation studies using a hypothetical ALS scenario, evaluate type I error and power under various conditions, compared to the Bayesian power prior approach (Ibrahim et al., 2015). Our methods and the Bayesian power prior control type I error, whereas Yuan et al. (2019) increases it under exchangeability. If exchangeability doesn't hold, all methods fail to control type I error. Our methods show type I error inflation of 6%-8%, compared to 10% for Yuan et al. (2019) and 16% for the Bayesian power prior.

stat.ME↗

arXiv2025

MSWA: Refining Local Attention with Multi-ScaleWindow Attention

Transformer-based LLMs have achieved exceptional performance across a wide range of NLP tasks. However, the standard self-attention mechanism suffers from quadratic time complexity and linearly increased cache size. Sliding window attention (SWA) solves this problem by restricting the attention range to a fixed-size local context window. Nevertheless, SWA employs a uniform window size for each head in each layer, making it inefficient in capturing context of varying scales. To mitigate this limitation, we propose Multi-Scale Window Attention (MSWA) which applies diverse window sizes across heads and layers in the Transformer. It not only allows for different window sizes among heads within the same layer but also progressively increases window size allocation from shallow to deep layers, thus enabling the model to capture contextual information with different lengths and distances. Experimental results on language modeling and common-sense reasoning tasks substantiate that MSWA outperforms traditional local attention in both effectiveness and efficiency.

cs.CL↗

arXiv2024

EGSRAL: An Enhanced 3D Gaussian Splatting based Renderer with Automated Labeling for Large-Scale Driving Scene

3D Gaussian Splatting (3D GS) has gained popularity due to its faster rendering speed and high-quality novel view synthesis. Some researchers have explored using 3D GS for reconstructing driving scenes. However, these methods often rely on various data types, such as depth maps, 3D boxes, and trajectories of moving objects. Additionally, the lack of annotations for synthesized images limits their direct application in downstream tasks. To address these issues, we propose EGSRAL, a 3D GS-based method that relies solely on training images without extra annotations. EGSRAL enhances 3D GS's capability to model both dynamic objects and static backgrounds and introduces a novel adaptor for auto labeling, generating corresponding annotations based on existing annotations. We also propose a grouping strategy for vanilla 3D GS to address perspective issues in rendering large-scale, complex scenes. Our method achieves state-of-the-art performance on multiple datasets without any extra annotation. For example, the PSNR metric reaches 29.04 on the nuScenes dataset. Moreover, our automated labeling can significantly improve the performance of 2D/3D detection tasks. Code is available at https://github.com/jiangxb98/EGSRAL.

cs.CV↗

arXiv2024

Model-free Approach to Evaluate a Censored Intermediate Outcome as a Surrogate for Overall Survival

Clinical trials or studies oftentimes require long-term and/or costly follow-up of participants to evaluate a novel treatment/drug/vaccine. There has been increasing interest in the past few decades in using short-term surrogate outcomes as a replacement of the primary outcome i.e., in using the surrogate outcome, which can potentially be observed sooner, to make inference about the treatment effect on the long-term primary outcome. Very few of the available statistical methods to evaluate a surrogate are applicable to settings where both the surrogate and the primary outcome are time-to-event outcomes subject to censoring. Methods that can handle this setting tend to require parametric assumptions or be limited to assessing only the restricted mean survival time. In this paper, we propose a non-parametric approach to evaluate a censored surrogate outcome, such as time to progression, when the primary outcome is also a censored time-to-event outcome, such as time to death, and the treatment effect of interest is the difference in overall survival. Specifically, we define the proportion of the treatment effect on the primary outcome that is explained (PTE) by the censored surrogate outcome in this context, and estimate this proportion by defining and deriving an optimal transformation of the surrogate information. Our approach provides the added advantage of relaxed assumptions to guarantee that the true PTE is within (0,1), along with being model-free. Finite sample performance of our estimators are illustrated via extensive simulation studies and a real data application examining progression-free survival as a surrogate for overall survival for patients with metastatic colorectal cancer.

stat.ME↗

arXiv2024

FTP: A Fine-grained Token-wise Pruner for Large Language Models via Token Routing

Recently, large language models (LLMs) have demonstrated superior performance across various tasks by adhering to scaling laws, which significantly increase model size. However, the huge computation overhead during inference hinders the deployment in industrial applications. Many works leverage traditional compression approaches to boost model inference, but these always introduce additional training costs to restore the performance and the pruning results typically show noticeable performance drops compared to the original model when aiming for a specific level of acceleration. To address these issues, we propose a fine-grained token-wise pruning approach for the LLMs, which presents a learnable router to adaptively identify the less important tokens and skip them across model blocks to reduce computational cost during inference. To construct the router efficiently, we present a search-based sparsity scheduler for pruning sparsity allocation, a trainable router combined with our proposed four low-dimensional factors as input and three proposed losses. We conduct extensive experiments across different benchmarks on different LLMs to demonstrate the superiority of our method. Our approach achieves state-of-the-art (SOTA) pruning results, surpassing other existing pruning methods. For instance, our method outperforms BlockPruner and ShortGPT by approximately 10 points on both LLaMA2-7B and Qwen1.5-7B in accuracy retention at comparable token sparsity levels.

cs.CL↗

arXiv2024

Fast Occupancy Network

Occupancy Network has recently attracted much attention in autonomous driving. Instead of monocular 3D detection and recent bird's eye view(BEV) models predicting 3D bounding box of obstacles, Occupancy Network predicts the category of voxel in specified 3D space around the ego vehicle via transforming 3D detection task into 3D voxel segmentation task, which has much superiority in tackling category outlier obstacles and providing fine-grained 3D representation. However, existing methods usually require huge computation resources than previous methods, which hinder the Occupancy Network solution applying in intelligent driving systems. To address this problem, we make an analysis of the bottleneck of Occupancy Network inference cost, and present a simple and fast Occupancy Network model, which adopts a deformable 2D convolutional layer to lift BEV feature to 3D voxel feature and presents an efficient voxel feature pyramid network (FPN) module to improve performance with few computational cost. Further, we present a cost-free 2D segmentation branch in perspective view after feature extractors for Occupancy Network during inference phase to improve accuracy. Experimental results demonstrate that our method consistently outperforms existing methods in both accuracy and inference speed, which surpasses recent state-of-the-art (SOTA) OCCNet by 1.7% with ResNet50 backbone with about 3X inference speedup. Furthermore, our method can be easily applied to existing BEV models to transform them into Occupancy Network models.

cs.CV↗

arXiv2024

Using Principal Progression Rate to Quantify and Compare Disease Progression in Comparative Studies

In comparative studies of progressive diseases, such as randomized controlled trials (RCTs), the mean Change From Baseline (CFB) of a continuous outcome at a pre-specified follow-up time across subjects in the target population is a standard estimand used to summarize the overall disease progression. Despite its simplicity in interpretation, the mean CFB may not efficiently capture important features of the trajectory of the mean outcome relevant to the evaluation of the treatment effect of an intervention. Additionally, the estimation of the mean CFB does not use all longitudinal data points. To address these limitations, we propose a class of estimands called Principal Progression Rate (PPR). The PPR is a weighted average of local or instantaneous slope of the trajectory of the population mean during the follow-up. The flexibility of the weight function allows the PPR to cover a broad class of intuitive estimands, including the mean CFB, the slope of ordinary least-square fit to the trajectory, and the area under the curve. We showed that properly chosen PPRs can enhance statistical power over the mean CFB by amplifying the signal of treatment effect and/or improving estimation precision. We evaluated different versions of PPRs and the performance of their estimators through numerical studies. A real dataset was analyzed to demonstrate the advantage of using alternative PPR over the mean CFB.

stat.ME↗

arXiv2024

DiP-GO: A Diffusion Pruner via Few-step Gradient Optimization

Diffusion models have achieved remarkable progress in the field of image generation due to their outstanding capabilities. However, these models require substantial computing resources because of the multi-step denoising process during inference. While traditional pruning methods have been employed to optimize these models, the retraining process necessitates large-scale training datasets and extensive computational costs to maintain generalization ability, making it neither convenient nor efficient. Recent studies attempt to utilize the similarity of features across adjacent denoising stages to reduce computational costs through simple and static strategies. However, these strategies cannot fully harness the potential of the similar feature patterns across adjacent timesteps. In this work, we propose a novel pruning method that derives an efficient diffusion model via a more intelligent and differentiable pruner. At the core of our approach is casting the model pruning process into a SubNet search process. Specifically, we first introduce a SuperNet based on standard diffusion via adding some backup connections built upon the similar features. We then construct a plugin pruner network and design optimization losses to identify redundant computation. Finally, our method can identify an optimal SubNet through few-step gradient optimization and a simple post-processing procedure. We conduct extensive experiments on various diffusion models including Stable Diffusion series and DiTs. Our DiP-GO approach achieves 4.4 x speedup for SD-1.5 without any loss of accuracy, significantly outperforming the previous state-of-the-art methods.

cs.CV↗

arXiv2024

Amphista: Bi-directional Multi-head Decoding for Accelerating LLM Inference

Large Language Models (LLMs) inherently use autoregressive decoding, which lacks parallelism in inference and results in significantly slow inference speed. While methods such as Medusa constructs parallelized heads, they lack adequate information interaction across different prediction positions. To overcome this limitation, we introduce Amphista, an enhanced speculative decoding framework that builds upon Medusa. Specifically, Amphista models an Auto-embedding Block capable of parallel inference, incorporating bi-directional attention to enable interaction between different drafting heads. Additionally, Amphista integrates Staged Adaptation Layers, which ensure a seamless transition of semantic information from the target model's autoregressive inference to the drafting heads' non-autoregressive inference, effectively achieving paradigm shift and feature fusion. Experimental results on Vicuna models using MT-Bench and Spec-Bench demonstrate that Amphista achieves substantial acceleration while maintaining generation quality. On MT-Bench, Amphista delivers up to 2.75$\times$ speedup over vanilla autoregressive decoding and 1.40$\times$ over Medusa on Vicuna 33B in wall-clock time.

cs.AI↗