arXiv subjects

Yizhe Zhu

Publications and source records attributed to Yizhe Zhu.

At least 37 records · Page 2Linked to original sources

arXiv2025

Residual Rotation Correction using Tactile Equivariance

Visuotactile policy learning augments vision-only policies with tactile input, facilitating contact-rich manipulation. However, the high cost of tactile data collection makes sample efficiency the key requirement for developing visuotactile policies. We present EquiTac, a framework that exploits the inherent SO(2) symmetry of in-hand object rotation to improve sample efficiency and generalization for visuotactile policy learning. EquiTac first reconstructs surface normals from raw RGB inputs of vision-based tactile sensors, so rotations of the normal vector field correspond to in-hand object rotations. An SO(2)-equivariant network then predicts a residual rotation action that augments a base visuomotor policy at test time, enabling real-time rotation correction without additional reorientation demonstrations. On a real robot, EquiTac accurately achieves robust zero-shot generalization to unseen in-hand orientations with very few training samples, where baselines fail even with more training data. To our knowledge, this is the first tactile learning method to explicitly encode tactile equivariance for policy learning, yielding a lightweight, symmetry-aware module that improves reliability in contact-rich tasks.

cs.RO↗

arXiv2025

Generalizable Hierarchical Skill Learning via Object-Centric Representation

We present Generalizable Hierarchical Skill Learning (GSL), a novel framework for hierarchical policy learning that significantly improves policy generalization and sample efficiency in robot manipulation. One core idea of GSL is to use object-centric skills as an interface that bridges the high-level vision-language model and the low-level visual-motor policy. Specifically, GSL decomposes demonstrations into transferable and object-canonicalized skill primitives using foundation models, ensuring efficient low-level skill learning in the object frame. At test time, the skill-object pairs predicted by the high-level agent are fed to the low-level module, where the inferred canonical actions are mapped back to the world frame for execution. This structured yet flexible design leads to substantial improvements in sample efficiency and generalization of our method across unseen spatial arrangements, object appearances, and task compositions. In simulation, GSL trained with only 3 demonstrations per task outperforms baselines trained with 30 times more data by 15.5 percent on unseen tasks. In real-world experiments, GSL also surpasses the baseline trained with 10 times more data.

cs.RO↗

arXiv2025

Universality of Kernel Random Matrices and Kernel Regression in the Quadratic Regime

Kernel ridge regression (KRR) is a popular class of machine learning models that has become an important tool for understanding deep learning. Much of the focus thus far has been on studying the proportional asymptotic regime, $n \asymp d$, where $n$ is the number of training samples and $d$ is the dimension of the dataset. In the proportional regime, under certain conditions on the data distribution, the kernel random matrix involved in KRR exhibits behavior akin to that of a linear kernel. In this work, we extend the study of kernel regression to the quadratic asymptotic regime, where $n \asymp d^2$. In this regime, we demonstrate that a broad class of inner-product kernels exhibits behavior similar to a quadratic kernel. Specifically, we establish an operator norm approximation bound for the difference between the original kernel random matrix and a quadratic kernel random matrix with additional correction terms compared to the Taylor expansion of the kernel functions. The approximation works for general data distributions under a Gaussian-moment-matching assumption with a covariance structure. This new approximation is utilized to obtain a limiting spectral distribution of the original kernel matrix and characterize the precise asymptotic training and test errors for KRR in the quadratic regime when $n/d^2$ converges to a non-zero constant. The generalization errors are obtained for (i) a random teacher model, (ii) a deterministic teacher model where the weights are perfectly aligned with the covariance of the data. Under the random teacher model setting, we also verify that the generalized cross-validation (GCV) estimator can consistently estimate the generalization error in the quadratic regime for anisotropic data. Our proof techniques combine moment methods, Wick's formula, orthogonal polynomials, and resolvent analysis of random matrices with correlated entries.

stat.ML↗

arXiv2025

A note on the improved sparse Hanson-Wright inequalities

We establish sparse Hanson-Wright inequalities for quadratic forms of sparse $α$-sub-exponential random vectors with exponent parameter $α\in(0, 2]$. In the regime $0< α\le 1$ we derive a refined inequality that is optimal in several canonical models. These results extend the classical Hanson-Wright bound to the sparse setting. Illustrative applications include covariance matrix estimation with incomplete observations, low-rank matrix approximation under the maximum norm with sparsified sketches, and concentration inequalities for sparse $α$-sub-exponential random vectors.

math.PR↗

arXiv2025

Singular values of sparse random rectangular matrices: Emergence of outliers at criticality

Consider the random bipartite Erdős-Rényi graph $\mathbb{G}(n, m, p)$, where each edge with one vertex in $V_{1}=[n]$ and the other vertex in $V_{2} =[m]$ is connected with probability $p$, and $n=\lfloor γm\rfloor$ for a constant aspect ratio $γ\geq 1$. It is well known that the empirical spectral measure of its centered and normalized adjacency matrix converges to the Marčenko-Pastur (MP) distribution. However, largest and smallest singular values may not converge to the right and left edges, respectively, especially when $p = o(1)$. Notably, it was proved by Dumitriu and Zhu (2024) that there are almost surely no singular value outside the compact support of the MP law when $np = ω(\log(n))$. In this paper, we consider the critical sparsity regime where $p = b\log(n)/\sqrt{mn}$ for some constant $b>0$. We quantitatively characterize the emergence of outlier singular values as follows. For explicit $b_{*}$ and $b^{*}$ as functions of $γ$, we prove that when $b > b_{*}$, there is no outlier outside the bulk; when $b^{*}< b < b_{*}$, outliers are present only outside the right edge of the MP law; and when $b < b^{*}$, outliers are present on both sides, all with high probability. Moreover, the locations of those outliers are precisely characterized by a function depending on the largest and smallest degree vertices of the random graph. We estimate the number of outliers as well. Our results follow the path forged by Alt, Ducatez and Knowles (2021), and can be extended to sparse random rectangular matrices with bounded entries.

math.PR↗

arXiv2025

Federated Adversarial Domain Adaptation

Federated learning improves data privacy and efficiency in machine learning performed over networks of distributed devices, such as mobile phones, IoT and wearable devices, etc. Yet models trained with federated learning can still fail to generalize to new devices due to the problem of domain shift. Domain shift occurs when the labeled data collected by source nodes statistically differs from the target node's unlabeled data. In this work, we present a principled approach to the problem of federated domain adaptation, which aims to align the representations learned among the different nodes with the data distribution of the target node. Our approach extends adversarial adaptation techniques to the constraints of the federated setting. In addition, we devise a dynamic attention mechanism and leverage feature disentanglement to enhance knowledge transfer. Empirically, we perform extensive experiments on several image and text classification tasks and show promising results under unsupervised federated domain adaptation setting.

cs.CV↗

arXiv2025

Uniqueness of diffeomorphic minimizers of $L^p$-mean distortion

We study the $L^p$-mean distortion functionals, \[{\cal E}_p[f] = \int_\mathbb Y K^p_f(z) \; dz, \] for Sobolev homeomorphisms $f: \overline{\mathbb Y}\xrightarrow{\rm onto} \overline{\mathbb X}$ where $\mathbb X$ and $\mathbb Y$ are bounded simply connected Lipschitz domains, and $f$ coincides with a given boundary map $f_0 \colon \partial \mathbb Y \to \partial \mathbb X$. Here, $K_f(z)$ denotes the pointwise distortion function of $f$. It is conjectured that for every $1 < p < \infty$, the functional $\mathcal{E}_p$ admits a minimizer that is a diffeomorphism. We prove that if such a diffeomorphic minimizer exists, then it is unique.

math.FA↗

arXiv2025

Non-convex matrix sensing: Breaking the quadratic rank barrier in the sample complexity

For the problem of reconstructing a low-rank matrix from a few linear measurements, two classes of algorithms have been widely studied in the literature: convex approaches based on nuclear norm minimization, and non-convex approaches that use factorized gradient descent. Under certain statistical model assumptions, it is known that nuclear norm minimization recovers the ground truth as soon as the number of samples scales linearly with the number of degrees of freedom of the ground-truth. In contrast, while non-convex approaches are computationally less expensive, existing recovery guarantees assume that the number of samples scales at least quadratically with the rank $r$ of the ground-truth matrix. In this paper, we close this gap by showing that the non-convex approaches can be as efficient as nuclear norm minimization in terms of sample complexity. Namely, we consider the problem of reconstructing a positive semidefinite matrix from a few Gaussian measurements. We show that factorized gradient descent with spectral initialization converges to the ground truth at a linear rate as soon as the number of samples scales with $ Ω(rdκ^2)$, where $d$ is the dimension, and $κ$ is the condition number of the ground truth matrix. This improves the previous rank-dependence in the sample complexity of non-convex matrix factorization from quadratic to linear. Furthermore, we extend our theory to the noisy setting, where we show that with noisy measurements, factorized gradient descent with spectral initialization converges to the minimax optimal error up to a factor linear in $κ$. Our proof relies on a probabilistic decoupling argument, where we show that the gradient descent iterates are only weakly dependent on the individual entries of the measurement matrices. We expect that our proof technique is of independent interest for other non-convex problems.

stat.ML↗

arXiv2025

Community detection with the Bethe-Hessian

The Bethe-Hessian matrix, introduced by Saade, Krzakala, and Zdeborová (2014), is a Hermitian matrix designed for applying spectral clustering algorithms to sparse networks. Rather than employing a non-symmetric and high-dimensional non-backtracking operator, a spectral method based on the Bethe-Hessian matrix is conjectured to also reach the Kesten-Stigum detection threshold in the sparse stochastic block model (SBM). We provide the first rigorous analysis of the Bethe-Hessian spectral method in the SBM under both the bounded expected degree and the growing degree regimes. Specifically, we demonstrate that: (i) When the expected degree $d\geq 2$, the number of negative outliers of the Bethe-Hessian matrix can consistently estimate the number of blocks above the Kesten-Stigum threshold, thus confirming a conjecture from Saade, Krzakala, and Zdeborová (2014) for $d\geq 2$. (ii) For sufficiently large $d$, its eigenvectors can be used to achieve weak recovery. (iii) As $d\to\infty$, we establish the concentration of the locations of its negative outlier eigenvalues, and weak consistency can be achieved via a spectral method based on the Bethe-Hessian matrix.

math.ST↗

arXiv2025

EquAct: An SE(3)-Equivariant Multi-Task Transformer for Open-Loop Robotic Manipulation

Transformer architectures can effectively learn language-conditioned, multi-task 3D open-loop manipulation policies from demonstrations by jointly processing natural language instructions and 3D observations. However, although both the robot policy and language instructions inherently encode rich 3D geometric structures, standard transformers lack built-in guarantees of geometric consistency, often resulting in unpredictable behavior under SE(3) transformations of the scene. In this paper, we leverage SE(3) equivariance as a key structural property shared by both policy and language, and propose EquAct-a novel SE(3)-equivariant multi-task transformer. EquAct is theoretically guaranteed to be SE(3) equivariant and consists of two key components: (1) an efficient SE(3)-equivariant point cloud-based U-net with spherical Fourier features for policy reasoning, and (2) SE(3)-invariant Feature-wise Linear Modulation (iFiLM) layers for language conditioning. To evaluate its spatial generalization ability, we benchmark EquAct on 18 RLBench simulation tasks with both SE(3) and SE(2) scene perturbations, and on 4 physical tasks. EquAct performs state-of-the-art across these simulation and physical tasks.

cs.RO↗

arXiv2025

Central limit theorems for linear spectral statistics of inhomogeneous random graphs with graphon limits

We establish central limit theorems (CLTs) for the linear spectral statistics of the adjacency matrix of inhomogeneous random graphs across all sparsity regimes, providing explicit covariance formulas under the assumption that the variance profile of the random graphs converges to a graphon limit. Two types of CLTs are derived for the (non-centered) adjacency matrix and the centered adjacency matrix, with different scaling factors when the sparsity parameter $p$ satisfies $np = n^{Ω(1)}$, and with the same scaling factor when $np = n^{o(1)}$. In both cases, the limiting covariance is expressed in terms of homomorphism densities from certain types of finite graphs to a graphon. These results highlight a phase transition in the centering effect for global eigenvalue fluctuations. For the non-centered adjacency matrix, we also identify new phase transitions for the CLTs in the sparse regime when $n^{1/m} \ll np \ll n^{1/(m-1)}$ for $m \geq 2$. Furthermore, weaker conditions for the graphon convergence of the variance profile are sufficient as $p$ decreases from being constant to $np \to c\in (0,\infty)$. These findings reveal a novel connection between graphon limits and linear spectral statistics in random matrix theory.

math.PR↗

arXiv2025

Hierarchical Equivariant Policy via Frame Transfer

Recent advances in hierarchical policy learning highlight the advantages of decomposing systems into high-level and low-level agents, enabling efficient long-horizon reasoning and precise fine-grained control. However, the interface between these hierarchy levels remains underexplored, and existing hierarchical methods often ignore domain symmetry, resulting in the need for extensive demonstrations to achieve robust performance. To address these issues, we propose Hierarchical Equivariant Policy (HEP), a novel hierarchical policy framework. We propose a frame transfer interface for hierarchical policy learning, which uses the high-level agent's output as a coordinate frame for the low-level agent, providing a strong inductive bias while retaining flexibility. Additionally, we integrate domain symmetries into both levels and theoretically demonstrate the system's overall equivariance. HEP achieves state-of-the-art performance in complex robotic manipulation tasks, demonstrating significant improvements in both simulation and real-world settings.

cs.RO↗

arXiv2024

Extreme singular values of inhomogeneous sparse random rectangular matrices

We develop a unified approach to bounding the largest and smallest singular values of an inhomogeneous random rectangular matrix, based on the non-backtracking operator and the Ihara-Bass formula for general random Hermitian matrices with a bipartite block structure. We obtain probabilistic upper (respectively, lower) bounds for the largest (respectively, smallest) singular values of a large rectangular random matrix $X$. These bounds are given in terms of the maximal and minimal $\ell_2$-norms of the rows and columns of the variance profile of $X$. The proofs involve finding probabilistic upper bounds on the spectral radius of an associated non-backtracking matrix $B$. The two-sided bounds can be applied to the centered adjacency matrix of sparse inhomogeneous Erdős-Rényi bipartite graphs for a wide range of sparsity, down to criticality. In particular, for Erdős-Rényi bipartite graphs $G(n,m,p)$ with $p=ω(\log n)/n$, and $m/n\to y \in (0,1)$, our sharp bounds imply that there are no outliers outside the support of the Marčenko-Pastur law almost surely. This result extends the Bai-Yin theorem to sparse rectangular random matrices.

math.PR↗

arXiv2024

Differentially Private Low-dimensional Synthetic Data from High-dimensional Datasets

Differentially private synthetic data provide a powerful mechanism to enable data analysis while protecting sensitive information about individuals. However, when the data lie in a high-dimensional space, the accuracy of the synthetic data suffers from the curse of dimensionality. In this paper, we propose a differentially private algorithm to generate low-dimensional synthetic data efficiently from a high-dimensional dataset with a utility guarantee with respect to the Wasserstein distance. A key step of our algorithm is a private principal component analysis (PCA) procedure with a near-optimal accuracy bound that circumvents the curse of dimensionality. Unlike the standard perturbation analysis, our analysis of private PCA works without assuming the spectral gap for the covariance matrix.

cs.LG↗

arXiv2024

Online Differentially Private Synthetic Data Generation

We present a polynomial-time algorithm for online differentially private synthetic data generation. For a data stream within the hypercube $[0,1]^d$ and an infinite time horizon, we develop an online algorithm that generates a differentially private synthetic dataset at each time $t$. This algorithm achieves a near-optimal accuracy bound of $O(\log(t)t^{-1/d})$ for $d\geq 2$ and $O(\log^{4.5}(t)t^{-1})$ for $d=1$ in the 1-Wasserstein distance. This result extends the previous work on the continual release model for counting queries to Lipschitz queries. Compared to the offline case, where the entire dataset is available at once, our approach requires only an extra polylog factor in the accuracy bound.

math.ST↗

arXiv2024

Regular bipartite decompositions of pseudorandom graphs

In 1972, Kotzig proved that for every even $n$, the complete graph $K_n$ can be decomposed into $\lceil\log_2n\rceil$ edge-disjoint regular bipartite spanning subgraphs, which is best possible. In this paper, we study regular bipartite decompositions of $(n,d,λ)$-graphs, where $n$ is an even integer and $d_0\leq d\leq n-1$ for some absolute constant $d_0$. With a randomized algorithm, we prove that such an $(n,d,λ)$-graph with $λ\leq d/12$ can be decomposed into at most $\log_2 d + 36$ regular bipartite spanning subgraphs. This is best possible up to the additive constant term. As a consequence, we also improve the best known bounds on $λ= λ(d)$ by Ferber and Jain (2020) to guarantee that an $(n,d,λ)$-graph on an even number of vertices admits a $1$-factorization, showing that $λ\leq cd$ is sufficient for some absolute constant $c > 0$.

math.CO↗

arXiv2024

A non-backtracking method for long matrix and tensor completion

We consider the problem of low-rank rectangular matrix completion in the regime where the matrix $M$ of size $n\times m$ is ``long", i.e., the aspect ratio $m/n$ diverges to infinity. Such matrices are of particular interest in the study of tensor completion, where they arise from the unfolding of a low-rank tensor. In the case where the sampling probability is $\frac{d}{\sqrt{mn}}$, we propose a new spectral algorithm for recovering the singular values and left singular vectors of the original matrix $M$ based on a variant of the standard non-backtracking operator of a suitably defined bipartite weighted random graph, which we call a \textit{non-backtracking wedge operator}. When $d$ is above a Kesten-Stigum-type sampling threshold, our algorithm recovers a correlated version of the singular value decomposition of $M$ with quantifiable error bounds. This is the first result in the regime of bounded $d$ for weak recovery and the first for weak consistency when $d\to\infty$ arbitrarily slowly without any polylog factors. As an application, for low-CP-rank orthogonal $k$-tensor completion, we efficiently achieve weak recovery with sample size $O(n^{k/2})$ and weak consistency with sample size $ω(n^{k/2})$. A similar result is obtained for low-multilinear-rank tensor completion with $O(n^{k/2})$ many samples.

math.ST↗

arXiv2024

MagicPose: Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusion

In this work, we propose MagicPose, a diffusion-based model for 2D human pose and facial expression retargeting. Specifically, given a reference image, we aim to generate a person's new images by controlling the poses and facial expressions while keeping the identity unchanged. To this end, we propose a two-stage training strategy to disentangle human motions and appearance (e.g., facial expressions, skin tone and dressing), consisting of (1) the pre-training of an appearance-control block and (2) learning appearance-disentangled pose control. Our novel design enables robust appearance control over generated human images, including body, facial attributes, and even background. By leveraging the prior knowledge of image diffusion models, MagicPose generalizes well to unseen human identities and complex poses without the need for additional fine-tuning. Moreover, the proposed model is easy to use and can be considered as a plug-in module/extension to Stable Diffusion. The code is available at: https://github.com/Boese0601/MagicDance

cs.CV↗