arXiv subjects

Lingzhou Xue

Publications and source records attributed to Lingzhou Xue.

At least 37 records · Page 2Linked to original sources

arXiv2026

Bayesian Global Fréchet Regression via Weak Conditional Expectations

Fréchet regression provides a versatile framework for modeling responses in metric spaces with Euclidean predictors, yet current methodologies rely almost exclusively on frequentist approaches. We propose a Bayesian framework for Fréchet regression that offers a principled way of incorporating prior information into nonlinear global Fréchet regression. By targeting a novel Fréchet Bayes rule, we reduce the object-valued regression problem to a collection of tractable scalar regression tasks. Our approach allows for a controlled interpolation between the prior and the data-driven frequentist estimate, facilitating effective shrinkage toward informed values. While initially derived under Gaussian assumptions, we demonstrate that our framework is robust to model misspecification by establishing its validity under moment conditions via weak conditional expectations. The numerical properties of the proposed methodology are demonstrated in simulation studies and an application to microbiome compositional data, where we show that leveraging an auxiliary cohort to inform the prior significantly enhances predictive performance in a targeted, small-scale study

stat.ME↗

arXiv2026

Mitigating the Curse of Dimensionality in Uniform Convergence of Deep Neural Networks via Smooth Activations

This paper establishes a theoretical framework for the uniform convergence of smoothly activated deep neural network (DNN) estimators. While standard ReLU networks achieve minimax-optimal rates in the $L^2(P)$ norm for various nonparametric regression tasks, we establish a theoretical lower bound demonstrating that least-squares ReLU estimators can suffer from the curse of dimensionality in their uniform convergence behavior. Motivated by the need for reliable uniform guarantees in downstream tasks requiring worst-case reliability, we address this limitation by analyzing smoothly activated DNNs (smooth DNNs), encompassing both feedforward and residual structures. We establish novel pseudo-dimension bounds, non-asymptotic approximation guarantees, and Hölder-norm bounds for the approximators of these models. Leveraging these results, we derive non-asymptotic uniform convergence rates for smooth DNN estimators across multiple statistical contexts, including Huber, least-squares, quantile, and logistic regression. We prove that smooth DNNs can mitigate the {curse of dimensionality} in uniform convergence by adaptively exploiting the low-dimensional hierarchical composition structure of the target function. Supported by both simulation studies and a real-world application, our results position smooth DNNs as a theoretically grounded and practically viable alternative to ReLU networks for statistical learning tasks requiring uniform guarantees.

cs.LG↗

arXiv2026

New Empirical Process Tools and Their Applications to Robust Deep ReLU Networks and Phase Transitions for Nonparametric Regression

This paper introduces new empirical process tools for analyzing a broad class of statistical learning models under heavy-tailed noise and complex function classes. Our primary contribution is the derivation of two Dudley-type maximal inequalities for expected empirical processes that remove restrictive assumptions such as light tails and uniform boundedness of the function class. These inequalities enlarge the scope of empirical process theory available for statistical learning and nonparametric estimation. Exploiting the new bounds, we establish robustness guarantees for deep ReLU network estimators in Huber and quantile regression. In particular, we prove a unified non-asymptotic sub-Gaussian concentration bound that remains valid even under infinite-variance noise and provide a comprehensive analysis of non-asymptotic robustness for deep Huber estimators across all noise regimes. For deep quantile regression, we provide the first non-asymptotic sub-Gaussian bounds without requiring moment assumptions. As an additional application, our framework yields estimation error bounds for nonparametric least-squares estimators that simultaneously accommodate infinite-variance noise, non-Donsker function classes, and approximation error. Moreover, unlike prior approaches based on specialized multiplier processes, our framework extends to broader empirical risk minimization problems, including the nonparametric generalized linear models and the ``set-structured'' models.

math.ST↗

arXiv2026

A Unified Framework for Nonlinear Mediation Analysis of Random Objects

Mediation analysis for complex, non-Euclidean data, such as probability distributions, compositions, images, and networks, presents significant methodological challenges due to the inherent nonlinearity and geometric constraints of such spaces. Existing approaches are often restricted to Euclidean settings or specific data types. We propose Random Object Mediation Analysis (ROMA), a unified framework that simultaneously accommodates object-valued exposures, mediators, and outcomes, enabling the analysis of nonlinear causal pathways in general metric spaces. ROMA leverages an additive Reproducing Kernel Hilbert Space (RKHS) operator model to rigorously disentangle direct and indirect causal pathways, which is a significant advancement over existing single-predictor or purely predictive additive frameworks. Theoretically, we establish the nonparametric identification of causal effects and derive global asymptotic normality for our estimators. Crucially, this theoretical foundation enables the construction of simultaneous confidence bands and global test statistics without the need for computationally intensive resampling. We demonstrate the practical utility of ROMA through simulations and real-world applications involving compositional mediators and distributional outcomes, extending the scope of mediation analysis.

stat.ME↗

arXiv2026

Regret-Optimal Q-Learning with Low Cost for Single-Agent and Federated Reinforcement Learning

Motivated by real-world settings where data collection and policy deployment -- whether for a single agent or across multiple agents -- are costly, we study the problem of on-policy single-agent reinforcement learning (RL) and federated RL (FRL) with a focus on minimizing burn-in costs (the sample sizes needed to reach near-optimal regret) and policy switching or communication costs. In parallel finite-horizon episodic Markov Decision Processes (MDPs) with $S$ states and $A$ actions, existing methods either require superlinear burn-in costs in $S$ and $A$ or fail to achieve logarithmic switching or communication costs. We propose two novel model-free RL algorithms -- Q-EarlySettled-LowCost and FedQ-EarlySettled-LowCost -- that are the first in the literature to simultaneously achieve: (i) the best near-optimal regret among all known model-free RL or FRL algorithms, (ii) low burn-in cost that scales linearly with $S$ and $A$, and (iii) logarithmic policy switching cost for single-agent RL or communication cost for FRL. Additionally, we establish gap-dependent theoretical guarantees for both regret and switching/communication costs, improving or matching the best-known gap-dependent bounds.

stat.ML↗

arXiv2026

EXACT: Explicit Attribute-Guided Decoding-Time Personalization

Achieving personalized alignment requires adapting large language models to each user's evolving context. While decoding-time personalization offers a scalable alternative to training-time methods, existing methods largely rely on implicit, less interpretable preference representations and impose a rigid, context-agnostic user representation, failing to account for how preferences shift across prompts. We introduce EXACT, a new decoding-time personalization that aligns generation with limited pairwise preference feedback using a predefined set of interpretable attributes. EXACT first identifies user-specific attribute subsets by maximizing the likelihood of preferred responses in the offline stage. Then, for online inference, EXACT retrieves the most semantically relevant attributes for an incoming prompt and injects them into the context to steer generation. We establish theoretical approximation guarantees for the proposed algorithm under mild assumptions, and provably show that our similarity-based retrieval mechanism effectively mitigates contextual preference shifts, adapting to disparate tasks without pooling conflicting preferences. Extensive experiments on human-annotated preference datasets demonstrate that EXACT consistently outperforms strong baselines, including preference modeling accuracy and personalized generation quality.

cs.LG↗

arXiv2025

Structure-Preserving Nonlinear Sufficient Dimension Reduction for Tensors

We introduce two nonlinear sufficient dimension reduction methods for regressions with tensor-valued predictors. Our goal is two-fold: the first is to preserve the tensor structure when performing dimension reduction, particularly the meaning of the tensor modes, for improved interpretation; the second is to substantially reduce the number of parameters in dimension reduction, thereby achieving model parsimony and enhancing estimation accuracy. Our two tensor dimension reduction methods echo the two commonly used tensor decomposition mechanisms: one is the Tucker decomposition, which reduces a larger tensor to a smaller one; the other is the CP-decomposition, which represents an arbitrary tensor as a sequence of rank-one tensors. We developed the Fisher consistency of our methods at the population level and established their consistency and convergence rates. Both methods are easy to implement numerically: the Tucker-form can be implemented through a sequence of least-squares steps, and the CP-form can be implemented through a sequence of singular value decompositions. We investigated the finite-sample performance of our methods and showed substantial improvement in accuracy over existing methods in simulations and two data applications.

math.ST↗

arXiv2025

Collapsing Categories for Regression with Mixed Predictors

Categorical predictors are omnipresent in everyday regression practice: in fact, most regression data involve some categorical predictors, and this tendency is increasing in modern applications with more complex structures and larger data sizes. However, including too many categories in a regression model would seriously hamper accuracy, as the information in the data is fragmented by the multitude of categories. In this paper, we introduce a systematic method to reduce the complexity of categorical predictors by adaptively collapsing categories in regressions, so as to enhance the performance of regression estimation. Our method is based on the {\em pairwise vector fused LASSO}, which automatically fuses the categories that bear a similar regression relation with the response. We develop our method under a wide class of regression models defined by a general loss function, which includes linear models and generalized linear models as special cases. We rigorously established the category collapsing consistency of our method, developed an Inexact Proximal Gradient Descent algorithm to implement it, and proved the feasibility and convergence of our algorithm. Through simulations and an application to Spotify music data, we demonstrate that our method can effectively reduce categorical complexity while improving prediction performance, making it a powerful tool for regression with mixed predictors.

stat.ME↗

arXiv2025

AltLoRA: Towards Better Gradient Approximation in Low-Rank Adaptation with Alternating Projections

Low-Rank Adaptation (LoRA) has emerged as an effective technique for reducing memory overhead in fine-tuning large language models. However, it often suffers from sub-optimal performance compared with full fine-tuning since the update is constrained in the low-rank space. Recent variants such as LoRA-Pro attempt to mitigate this by adjusting the gradients of the low-rank matrices to approximate the full gradient. However, LoRA-Pro's solution is not unique, and different solutions can lead to significantly varying performance in ablation studies. Besides, to incorporate momentum or adaptive optimization design, approaches like LoRA-Pro must first compute the equivalent gradient, causing a higher memory cost close to full fine-tuning. A key challenge remains in integrating momentum properly into the low-rank space with lower memory cost. In this work, we propose AltLoRA, an alternating projection method that avoids the difficulties in gradient approximation brought by the joint update design, meanwhile integrating momentum without higher memory complexity. Our theoretical analysis provides convergence guarantees and further shows that AltLoRA enables stable feature learning and robustness to transformation invariance. Extensive experiments across multiple tasks demonstrate that AltLoRA outperforms LoRA and its variants, narrowing the gap toward full fine-tuning while preserving superior memory efficiency.

cs.LG↗

arXiv2025

Gap-Dependent Bounds for Federated $Q$-learning

We present the first gap-dependent analysis of regret and communication cost for on-policy federated $Q$-Learning in tabular episodic finite-horizon Markov decision processes (MDPs). Existing FRL methods focus on worst-case scenarios, leading to $\sqrt{T}$-type regret bounds and communication cost bounds with a $\log T$ term scaling with the number of agents $M$, states $S$, and actions $A$, where $T$ is the average total number of steps per agent. In contrast, our novel framework leverages the benign structures of MDPs, such as a strictly positive suboptimality gap, to achieve a $\log T$-type regret bound and a refined communication cost bound that disentangles exploration and exploitation. Our gap-dependent regret bound reveals a distinct multi-agent speedup pattern, and our gap-dependent communication cost bound removes the dependence on $MSA$ from the $\log T$ term. Notably, our gap-dependent communication cost bound also yields a better global switching cost when $M=1$, removing $SA$ from the $\log T$ term.

stat.ML↗

arXiv2025

Variable Selection for Additive Global Fréchet Regression

We present a novel framework for variable selection in Fréchet regression with responses in general metric spaces, a setting increasingly relevant for analyzing non-Euclidean data such as probability distributions and covariance matrices. Building on the concept of (weak) Fréchet conditional means, we develop an additive regression model that represents the metric-based discrepancy of the response as a sum of covariate-specific nonlinear functions in reproducing kernel Hilbert spaces (RKHS). To address the absence of linear structure in the response space, we transform the response via squared distances, enabling an interpretable and tractable additive decomposition. Variable selection is performed using Elastic Net regularization, extended to the RKHS setting, and further refined through a local linear approximation scheme that incorporates folded concave penalties such as the SCAD. We establish theoretical guarantees, including variable selection consistency and the strong oracle property, under minimal assumptions tailored to metric-space-valued responses. Simulations and applications to distributional and matrix-valued data demonstrate the scalability, interpretability, and practical effectiveness of the proposed approach. This work provides a principled foundation for statistical learning with random object data.

math.ST↗

arXiv2025

Doubly robust estimation of causal effects for random object outcomes with continuous treatments

Causal inference is central to statistics and scientific discovery, enabling researchers to identify cause-and-effect relationships beyond associations. While traditionally studied within Euclidean spaces, contemporary applications increasingly involve complex, non-Euclidean data structures that reside in abstract metric spaces, known as random objects, such as images, shapes, networks, and distributions. This paper introduces a novel framework for causal inference with continuous treatments applied to non-Euclidean data. To address the challenges posed by the lack of linear structures, we leverage Hilbert space embeddings of the metric spaces to facilitate Fréchet mean estimation and causal effect mapping. Motivated by a study on the impact of exposure to fine particulate matter on age-at-death distributions across U.S. counties, we propose a nonparametric, doubly-debiased causal inference approach for outcomes as random objects with continuous treatments. Our framework can accommodate moderately high-dimensional vector-valued confounders and derive efficient influence functions for estimation to ensure both robustness and interpretability. We establish rigorous asymptotic properties of the cross-fitted estimators and employ conformal inference techniques for counterfactual outcome prediction. Validated through numerical experiments and applied to real-world environmental data, our framework extends causal inference methodologies to complex data structures, broadening its applicability across scientific disciplines.

stat.ME↗

arXiv2025

Mediation Analysis for Sparse and Irregularly Spaced Longitudinal Outcomes with Application to the MrOS Sleep Study

Mediation analysis has become a widely used method for identifying the pathways through which an independent variable influences a dependent variable via intermediate mediators. However, limited research addresses the case where mediators are high-dimensional and the outcome is represented by sparse, irregularly spaced longitudinal data. To address these challenges, we propose a mediation analysis approach for scalar exposures, high-dimensional mediators, and sparse longitudinal outcomes. This approach effectively identifies significant mediators by addressing two key issues: (i) the underlying correlation structure within the sparse and irregular cognitive measurements, and (ii) adjusting mediation effects to handle the high-dimensional set of candidate mediators. In the MrOS Sleep study, our primary objective is to explore lipid pathways that may mediate the relationship between rest-activity rhythms and longitudinal cognitive decline in older men. Our findings suggest a potential mechanism involving rest-activity rhythms, lipid metabolites, and cognitive decline, and highlight significant mediators identified through multiple testing procedures.

stat.ME↗

arXiv2025

Strongly Consistent Community Detection in Popularity Adjusted Block Models

The Popularity Adjusted Block Model (PABM) provides a flexible framework for community detection in network data by allowing heterogeneous node popularity across communities. However, this flexibility increases model complexity and raises key unresolved challenges, particularly in effectively adapting spectral clustering techniques and efficiently achieving strong consistency in label recovery. To address these challenges, we first propose the Thresholded Cosine Spectral Clustering (TCSC) algorithm and establish its weak consistency under the PABM. We then introduce the one-step Refined TCSC algorithm and prove that it achieves strong consistency under the PABM, correctly recovering all community labels with high probability. We further show that the two-step Refined TCSC accelerates clustering error convergence, especially with small sample sizes. Additionally, we propose a data-driven approach for selecting the number of communities, which outperforms existing methods under the PABM. The effectiveness and robustness of our methods are validated through extensive simulations and real-world applications.

stat.ME↗

arXiv2025

Understanding the Statistical Accuracy-Communication Trade-off in Personalized Federated Learning with Minimax Guarantees

Personalized federated learning (PFL) offers a flexible framework for aggregating information across distributed clients with heterogeneous data. This work considers a personalized federated learning setting that simultaneously learns global and local models. While purely local training has no communication cost, collaborative learning among the clients can leverage shared knowledge to improve statistical accuracy, presenting an accuracy-communication trade-off in personalized federated learning. However, the theoretical analysis of how personalization quantitatively influences sample and algorithmic efficiency and their inherent trade-off is largely unexplored. This paper makes a contribution towards filling this gap, by providing a quantitative characterization of the personalization degree on the tradeoff. The results further offers theoretical insights for choosing the personalization degree. As a side contribution, we establish the minimax optimality in terms of statistical accuracy for a widely studied PFL formulation. The theoretical result is validated on both synthetic and real-world datasets and its generalizability is verified in a non-convex setting.

stat.ML↗

arXiv2025

Statistical Inference for High-Dimensional Robust Linear Regression Models via Recursive Online-Score Estimation

This paper introduces a novel framework for estimation and inference in penalized M-estimators applied to robust high-dimensional linear regression models. Traditional methods for high-dimensional statistical inference, which predominantly rely on convex likelihood-based approaches, struggle to address the nonconvexity inherent in penalized M-estimation with nonconvex objective functions. Our proposed method extends the recursive online score estimation (ROSE) framework of Shi et al. (2021) to robust high-dimensional settings by developing a recursive score equation based on penalized M-estimation, explicitly addressing nonconvexity. We establish the statistical consistency and asymptotic normality of the resulting estimator, providing a rigorous foundation for valid inference in robust high-dimensional regression. The effectiveness of our method is demonstrated through simulation studies and a real-world application, showcasing its superior performance compared to existing approaches.

stat.ME↗

arXiv2025

Federated Q-Learning with Reference-Advantage Decomposition: Almost Optimal Regret and Logarithmic Communication Cost

In this paper, we consider model-free federated reinforcement learning for tabular episodic Markov decision processes. Under the coordination of a central server, multiple agents collaboratively explore the environment and learn an optimal policy without sharing their raw data. Despite recent advances in federated Q-learning algorithms achieving near-linear regret speedup with low communication cost, existing algorithms only attain suboptimal regrets compared to the information bound. We propose a novel model-free federated Q-learning algorithm, termed FedQ-Advantage. Our algorithm leverages reference-advantage decomposition for variance reduction and operates under two distinct mechanisms: synchronization between the agents and the server, and policy update, both triggered by events. We prove that our algorithm not only requires a lower logarithmic communication cost but also achieves an almost optimal regret, reaching the information bound up to a logarithmic factor and near-linear regret speedup compared to its single-agent counterpart when the time horizon is sufficiently large.

stat.ML↗

arXiv2025

Gap-Dependent Bounds for Q-Learning using Reference-Advantage Decomposition

We study the gap-dependent bounds of two important algorithms for on-policy Q-learning for finite-horizon episodic tabular Markov Decision Processes (MDPs): UCB-Advantage (Zhang et al. 2020) and Q-EarlySettled-Advantage (Li et al. 2021). UCB-Advantage and Q-EarlySettled-Advantage improve upon the results based on Hoeffding-type bonuses and achieve the almost optimal $\sqrt{T}$-type regret bound in the worst-case scenario, where $T$ is the total number of steps. However, the benign structures of the MDPs such as a strictly positive suboptimality gap can significantly improve the regret. While gap-dependent regret bounds have been obtained for Q-learning with Hoeffding-type bonuses, it remains an open question to establish gap-dependent regret bounds for Q-learning using variance estimators in their bonuses and reference-advantage decomposition for variance reduction. We develop a novel error decomposition framework to prove gap-dependent regret bounds of UCB-Advantage and Q-EarlySettled-Advantage that are logarithmic in $T$ and improve upon existing ones for Q-learning algorithms. Moreover, we establish the gap-dependent bound for the policy switching cost of UCB-Advantage and improve that under the worst-case MDPs. To our knowledge, this paper presents the first gap-dependent regret analysis for Q-learning using variance estimators and reference-advantage decomposition and also provides the first gap-dependent analysis on policy switching cost for Q-learning.

stat.ML↗