arXiv subjects

Qiao Hu

Publications and source records attributed to Qiao Hu.

9 recordsLinked to original sources

arXiv2026

UniVer: A Unified Perspective for Multi-step and Multi-draft Speculative Decoding

Speculative decoding accelerates Large Language Models via draft-then-verify, where verification can be framed as an Optimal Transport (OT) problem. Existing approaches typically handle multi-draft and multi-step aspects in isolation, applying either flat OT to single-step drafts or per-token rejection sampling to tree-structured candidates. This separation leaves the joint regime (where multi-step dependencies meet multi-draft branching) poorly optimized, as local verification rules fail to exploit the coupling between horizontal and vertical dimensions of candidate trees. In this paper, we propose a unified perspective that casts tree-based verification as a conditional OT problem. Our key insight is that vertical dependencies can be abstracted through prefix acceptance probabilities, which act as dynamic scaling factors to actively guide horizontal draft selection. Based on this principle, we introduce UniVer, a verification algorithm that jointly optimizes across tree levels by composing local optimal transport plans under prefix constraints. We prove that UniVer remains lossless and achieves the optimal acceptance rate under the proposed conditional framework. Extensive experiments across different tasks and models demonstrate that UniVer improves acceptance length by 4.2% to 8.5% over standard recursive rejection sampling without replacement, while maintaining exact distributional alignment with the target model.

cs.CL↗

arXiv2026

A Decomposition Framework for Certifiably Optimal Orthogonal Sparse PCA

Sparse Principal Component Analysis (SPCA) is an important technique for high-dimensional data analysis, improving interpretability by imposing sparsity on principal components. However, existing methods often fail to simultaneously guarantee sparsity, orthogonality, and optimality of the principal components. To address this challenge, this work introduces a novel Sparse Principal Component Analysis (SPCA) algorithm called \textsc{GS-SPCA} (SPCA with Gram-Schmidt Orthogonalization), which simultaneously enforces sparsity, orthogonality, and optimality. However, the original GS-SPCA algorithm is computationally expensive due to the inherent $\ell_0$-norm constraint. To address this issue, we propose two acceleration strategies: First, we combine \textbf{Branch-and-Bound} with the GS-SPCA algorithm. By incorporating this strategy, we are able to obtain $\varepsilon$-optimal solutions with a trade-off between precision and efficiency, significantly improving computational speed. Second, we propose a \textbf{decomposition framework} for efficiently solving \textbf{multiple} principal components. This framework approximates the covariance matrix using a block-diagonal matrix through a thresholding method, reducing the original SPCA problem to a set of block-wise subproblems on approximately block-diagonal matrices.

cs.LG↗

arXiv2025

Convergent and efficient iteratively regularized contrast source inversion-type methods for inverse medium scattering problems

The contrast source inversion (CSI) method and the subspace-based optimization method (SOM) are first proposed in 1997 and 2009, respectively, and subsequently modified. The two methods and their variants share several properties and thus are called the CSI-type methods. The CSI-type methods are efficient and popular methods for solving inverse medium scattering problems, but their rigorous convergence remains an open problem. In this paper, we propose two iteratively regularized CSI-type (IRCSI-type) methods with a novel $\ell_1$ proximal term as the iteratively regularized term: the iteratively regularized CSI (IRCSI) method and the iteratively regularized SOM (IRSOM) method, which have a similar computation complexity to the original CSI and SOM methods, respectively, and prove their global convergence under natural and weak conditions on the original objective function. To the best of our knowledge, this is the first convergence result for iterative methods of solving nonlinear inverse scattering problems with a fixed frequency. The convergence and performance of the two IRCSI-type algorithms are illustrated by numerical experiments.

math.NA↗

arXiv2025

Traversal Verification for Speculative Tree Decoding

Speculative decoding is a promising approach for accelerating large language models. The primary idea is to use a lightweight draft model to speculate the output of the target model for multiple subsequent timesteps, and then verify them in parallel to determine whether the drafted tokens should be accepted or rejected. To enhance acceptance rates, existing frameworks typically construct token trees containing multiple candidates in each timestep. However, their reliance on token-level verification mechanisms introduces two critical limitations: First, the probability distribution of a sequence differs from that of individual tokens, leading to suboptimal acceptance length. Second, current verification schemes begin from the root node and proceed layer by layer in a top-down manner. Once a parent node is rejected, all its child nodes should be discarded, resulting in inefficient utilization of speculative candidates. This paper introduces Traversal Verification, a novel speculative decoding algorithm that fundamentally rethinks the verification paradigm through leaf-to-root traversal. Our approach considers the acceptance of the entire token sequence from the current node to the root, and preserves potentially valid subsequences that would be prematurely discarded by existing methods. We theoretically prove that the probability distribution obtained through Traversal Verification is identical to that of the target model, guaranteeing lossless inference while achieving substantial acceleration gains. Experimental results across different large language models and multiple tasks show that our method consistently improves acceptance length and throughput over existing methods.

cs.CL↗

arXiv2025

Empirical likelihood approach for high-dimensional moment restrictions with dependent data

Economic and financial models -- such as vector autoregressions, local projections, and multivariate volatility models -- feature complex dynamic interactions and spillovers across many time series. These models can be integrated into a unified framework, with high-dimensional parameters identified by moment conditions. As the number of parameters and moment conditions may surpass the sample size, we propose adding a double penalty to the empirical likelihood criterion to induce sparsity and facilitate dimension reduction. Notably, we utilize a marginal empirical likelihood approach despite temporal dependence in the data. Under regularity conditions, we provide asymptotic guarantees for our method, making it an attractive option for estimating large-scale multivariate time series models. We demonstrate the versatility of our procedure through extensive Monte Carlo simulations and three empirical applications, including analyses of US sectoral inflation rates, fiscal multipliers, and volatility spillover in China's banking sector.

econ.EM↗

arXiv2024

Edge differentially private estimation in the $β$-model via jittering and method of moments

A standing challenge in data privacy is the trade-off between the level of privacy and the efficiency of statistical inference. Here we conduct an in-depth study of this trade-off for parameter estimation in the $β$-model (Chatterjee, Diaconis and Sly, 2011) for edge differentially private network data released via jittering (Karwa, Krivitsky and Slavković, 2017). Unlike most previous approaches based on maximum likelihood estimation for this network model, we proceed via method-of-moments. This choice facilitates our exploration of a substantially broader range of privacy levels - corresponding to stricter privacy - than has been to date. Over this new range we discover our proposed estimator for the parameters exhibits an interesting phase transition, with both its convergence rate and asymptotic variance following one of three different regimes of behavior depending on the level of privacy. Because identification of the operable regime is difficult if not impossible in practice, we devise a novel adaptive bootstrap procedure to construct uniform inference across different phases. In fact, leveraging this bootstrap we are able to provide for simultaneous inference of all parameters in the $β$-model (i.e., equal to the number of nodes), which, to our best knowledge, is the first result of its kind. Numerical experiments confirm the competitive and reliable finite sample performance of the proposed inference methods, next to a comparable maximum likelihood method, as well as significant advantages in terms of computational speed and memory.

math.ST↗

arXiv2022

Optimal covariance matrix estimation for high-dimensional noise in high-frequency data

We consider high-dimensional measurement errors with high-frequency data. Our objective is on recovering the high-dimensional cross-sectional covariance matrix of the random errors with optimality. In this problem, not all components of the random vector are observed at the same time and the measurement errors are latent variables, leading to major challenges besides high data dimensionality. We propose a new covariance matrix estimator in this context with appropriate localization and thresholding, and then conduct a series of comprehensive theoretical investigations of the proposed estimator. By developing a new technical device integrating the high-frequency data feature with the conventional notion of $α$-mixing, our analysis successfully accommodates the challenging serial dependence in the measurement errors. Our theoretical analysis establishes the minimax optimal convergence rates associated with two commonly used loss functions; and we demonstrate with concrete cases when the proposed localized estimator with thresholding achieves the minimax optimal convergence rates. Considering that the variances and covariances can be small in reality, we conduct a second-order theoretical analysis that further disentangles the dominating bias in the estimator. A bias-corrected estimator is then proposed to ensure its practical finite sample performance. We also extensively analyze our estimator in the setting with jumps, and show that its performance is reasonably robust. We illustrate the promising empirical performance of the proposed estimator with extensive simulation studies and a real data analysis.

math.ST↗

arXiv2021

Cryptanalyzing two image encryption algorithms based on a first-order time-delay system

Security is a key problem for the transmission, interchange and storage process of multimedia systems and applications. In 2018, M. Li et al. proposed in-depth security analysis on an image encryption algorithm based on a first-order time-delay system (IEATD) and gave a specific chosen-plaintext attack on it. Moreover, an enhanced version called as IEACD was designed to fix the reported security defects. This paper analyzes the essential structures of the two algorithms and evaluates their real security performances: 1) no efficient nonlinear operations are adopted to assure the sensibility of keystream; 2) the equivalent secret key of IEATD can be efficiently recovered from one known plain-image and the corresponding cipher-image; 3) IEACD can still be efficiently cracked with a chosen-plaintext attack. Both rigorous theoretical analyses and detailed experimental results are provided to demonstrate effectiveness of the advanced cryptanalytic methods.

cs.CR↗

arXiv2020

Inference for Network Structure and Dynamics from Time Series Data via Graph Neural Network

Network structures in various backgrounds play important roles in social, technological, and biological systems. However, the observable network structures in real cases are often incomplete or unavailable due to measurement errors or private protection issues. Therefore, inferring the complete network structure is useful for understanding complex systems. The existing studies have not fully solved the problem of inferring network structure with partial or no information about connections or nodes. In this paper, we tackle the problem by utilizing time series data generated by network dynamics. We regard the network inference problem based on dynamical time series data as a problem of minimizing errors for predicting future states and proposed a novel data-driven deep learning model called Gumbel Graph Network (GGN) to solve the two kinds of network inference problems: Network Reconstruction and Network Completion. For the network reconstruction problem, the GGN framework includes two modules: the dynamics learner and the network generator. For the network completion problem, GGN adds a new module called the States Learner to infer missing parts of the network. We carried out experiments on discrete and continuous time series data. The experiments show that our method can reconstruct up to 100% network structure on the network reconstruction task. While the model can also infer the unknown parts of the structure with up to 90% accuracy when some nodes are missing. And the accuracy decays with the increase of the fractions of missing nodes. Our framework may have wide application areas where the network structure is hard to obtained and the time series data is rich.

cs.LG↗