arXiv subjects

Zhiqiang Xu

Publications and source records attributed to Zhiqiang Xu.

At least 73 records · Page 4Linked to original sources

arXiv2024

Stability in Phase Retrieval: Characterizing Condition Numbers and the Optimal Vector Set

In this paper, we primarily focus on analyzing the stability property of phase retrieval by examining the bi-Lipschitz property of the map $Φ_{\boldsymbol{A}}(\boldsymbol{x})=|\boldsymbol{A}\boldsymbol{x}|\in \mathbb{R}_+^m$, where $\boldsymbol{x}\in \mathbb{H}^d$ and $\boldsymbol{A}\in \mathbb{H}^{m\times d}$ is the measurement matrix for $\mathbb{H}\in\{\mathbb{R},\mathbb{C}\}$. We define the condition number $β_{\boldsymbol{A}}=\frac{U_{\boldsymbol{A}}}{L_{\boldsymbol{A}}}$, where $L_{\boldsymbol{A}}$ and $U_{\boldsymbol{A}}$ represent the optimal lower and upper Lipschitz constants, respectively. We establish the first universal lower bound on $β_{\boldsymbol{A}}$ by demonstrating that for any ${\boldsymbol{A}}\in\mathbb{H}^{m\times d}$, \begin{equation*} β_{\boldsymbol{A}}\geq β_0^{\mathbb{H}}=\begin{cases} \sqrt{\fracπ{π-2}}\,\,\approx\,\, 1.659 & \text{if $\mathbb{H}=\mathbb{R}$,}\\ \sqrt{\frac{4}{4-π}}\,\,\approx\,\, 2.159 & \text{if $\mathbb{H}=\mathbb{C}$.} \end{cases} \end{equation*} We prove that the condition number of a standard Gaussian matrix in $\mathbb{H}^{m\times d}$ asymptotically matches the lower bound $β_0^{\mathbb{H}}$ for both real and complex cases. This result indicates that the constant lower bound $β_0^{\mathbb{H}}$ is asymptotically tight, holding true for both the real and complex scenarios. As an application of this result, we utilize it to investigate the performance of quadratic models for phase retrieval. Lastly, we establish that for any odd integer $m\geq 3$, the harmonic frame $\boldsymbol{A}\in \mathbb{R}^{m\times 2}$ possesses the minimum condition number among all $\boldsymbol{A}\in \mathbb{R}^{m\times 2}$. We are confident that these findings carry substantial implications for enhancing our understanding of phase retrieval.

cs.IT↗

arXiv2024

Interlacing Polynomial Method for the Column Subset Selection Problem

This paper investigates the spectral norm version of the column subset selection problem. Given a matrix $\mathbf{A}\in\mathbb{R}^{n\times d}$ and a positive integer $k\leq\text{rank}(\mathbf{A})$, the objective is to select exactly $k$ columns of $\mathbf{A}$ that minimize the spectral norm of the residual matrix after projecting $\mathbf{A}$ onto the space spanned by the selected columns. We use the method of interlacing polynomials introduced by Marcus-Spielman-Srivastava to derive a new upper bound on the minimal approximation error. This new bound is asymptotically sharp when the matrix $\mathbf{A}\in\mathbb{R}^{n\times d}$ obeys a spectral power-law decay. The relevant expected characteristic polynomials can be written as an extension of the expected polynomial for the restricted invertibility problem, incorporating two extra variable substitution operators. Finally, we propose a deterministic polynomial-time algorithm that achieves this error bound up to a computational error.

cs.DS↗

arXiv2023

Learning Time-aware Graph Structures for Spatially Correlated Time Series Forecasting

Spatio-temporal forecasting of future values of spatially correlated time series is important across many cyber-physical systems (CPS). Recent studies offer evidence that the use of graph neural networks to capture latent correlations between time series holds a potential for enhanced forecasting. However, most existing methods rely on pre-defined or self-learning graphs, which are either static or unintentionally dynamic, and thus cannot model the time-varying correlations that exhibit trends and periodicities caused by the regularity of the underlying processes in CPS. To tackle such limitation, we propose Time-aware Graph Structure Learning (TagSL), which extracts time-aware correlations among time series by measuring the interaction of node and time representations in high-dimensional spaces. Notably, we introduce time discrepancy learning that utilizes contrastive learning with distance-based regularization terms to constrain learned spatial correlations to a trend sequence. Additionally, we propose a periodic discriminant function to enable the capture of periodic changes from the state of nodes. Next, we present a Graph Convolution-based Gated Recurrent Unit (GCGRU) that jointly captures spatial and temporal dependencies while learning time-aware and node-specific patterns. Finally, we introduce a unified framework named Time-aware Graph Convolutional Recurrent Network (TGCRN), combining TagSL, and GCGRU in an encoder-decoder architecture for multi-step spatio-temporal forecasting. We report on experiments with TGCRN and popular existing approaches on five real-world datasets, thus providing evidence that TGCRN is capable of advancing the state-of-the-art. We also cover a detailed ablation study and visualization analysis, offering detailed insight into the effectiveness of time-aware structure learning.

cs.LG↗

arXiv2023

Label-Retrieval-Augmented Diffusion Models for Learning from Noisy Labels

Learning from noisy labels is an important and long-standing problem in machine learning for real applications. One of the main research lines focuses on learning a label corrector to purify potential noisy labels. However, these methods typically rely on strict assumptions and are limited to certain types of label noise. In this paper, we reformulate the label-noise problem from a generative-model perspective, $\textit{i.e.}$, labels are generated by gradually refining an initial random guess. This new perspective immediately enables existing powerful diffusion models to seamlessly learn the stochastic generative process. Once the generative uncertainty is modeled, we can perform classification inference using maximum likelihood estimation of labels. To mitigate the impact of noisy labels, we propose the $\textbf{L}$abel-$\textbf{R}$etrieval-$\textbf{A}$ugmented (LRA) diffusion model, which leverages neighbor consistency to effectively construct pseudo-clean labels for diffusion training. Our model is flexible and general, allowing easy incorporation of different types of conditional information, $\textit{e.g.}$, use of pre-trained models, to further boost model performance. Extensive experiments are conducted for evaluation. Our model achieves new state-of-the-art (SOTA) results on all the standard real-world benchmark datasets. Remarkably, by incorporating conditional information from the powerful CLIP model, our method can boost the current SOTA accuracy by 10-20 absolute points in many cases.

cs.LG↗

arXiv2023

Behavior Optimized Image Generation

The last few years have witnessed great success on image generation, which has crossed the acceptance thresholds of aesthetics, making it directly applicable to personal and commercial applications. However, images, especially in marketing and advertising applications, are often created as a means to an end as opposed to just aesthetic concerns. The goal can be increasing sales, getting more clicks, likes, or image sales (in the case of stock businesses). Therefore, the generated images need to perform well on these key performance indicators (KPIs), in addition to being aesthetically good. In this paper, we make the first endeavor to answer the question of "How can one infuse the knowledge of the end-goal within the image generation process itself to create not just better-looking images but also "better-performing'' images?''. We propose BoigLLM, an LLM that understands both image content and user behavior. BoigLLM knows how an image should look to get a certain required KPI. We show that BoigLLM outperforms 13x larger models such as GPT-3.5 and GPT-4 in this task, demonstrating that while these state-of-the-art models can understand images, they lack information on how these images perform in the real world. To generate actual pixels of behavior-conditioned images, we train a diffusion-based model (BoigSD) to align with a proposed BoigLLM-defined reward. We show the performance of the overall pipeline on two datasets covering two different behaviors: a stock dataset with the number of forward actions as the KPI and a dataset containing tweets with the total likes as the KPI, denoted as BoigBench. To advance research in the direction of utility-driven image generation and understanding, we release BoigBench, a benchmark dataset containing 168 million enterprise tweets with their media, brand account names, time of post, and total likes.

cs.CV↗

arXiv2023

Unsupervised Video Domain Adaptation for Action Recognition: A Disentanglement Perspective

Unsupervised video domain adaptation is a practical yet challenging task. In this work, for the first time, we tackle it from a disentanglement view. Our key idea is to handle the spatial and temporal domain divergence separately through disentanglement. Specifically, we consider the generation of cross-domain videos from two sets of latent factors, one encoding the static information and another encoding the dynamic information. A Transfer Sequential VAE (TranSVAE) framework is then developed to model such generation. To better serve for adaptation, we propose several objectives to constrain the latent factors. With these constraints, the spatial divergence can be readily removed by disentangling the static domain-specific information out, and the temporal divergence is further reduced from both frame- and video-levels through adversarial learning. Extensive experiments on the UCF-HMDB, Jester, and Epic-Kitchens datasets verify the effectiveness and superiority of TranSVAE compared with several state-of-the-art approaches. Code is publicly available.

cs.CV↗

arXiv2023

The performance of the amplitude-based model for complex phase retrieval

The paper aims to study the performance of the amplitude-based model \newline $\widehat{\mathbf x} \in {\rm argmin}_{{\mathbf x}\in \mathbb{C}^d}\sum_{j=1}^m\left(|\langle {\mathbf a}_j,{\mathbf x}\rangle|-b_j\right)^2$, where $b_j:=|\langle {\mathbf a}_j,{\mathbf x}_0\rangle|+η_j$ and ${\mathbf x}_0\in \mathbb{C}^d$ is a target signal. The model is raised in phase retrieval as well as in absolute value rectification neural networks. Many efficient algorithms have been developed to solve it in the past decades. {However, there are very few results available regarding the estimation performance in the complex case under noisy conditions.} In this paper, {we present a theoretical guarantee on the amplitude-based model for the noisy complex phase retrieval problem}. Specifically, we show that $\min_{θ\in[0,2π)}\|\widehat{\mathbf x}-\exp(\mathrm{i}θ)\cdot{\mathbf x}_0\|_2 \lesssim \frac{\|{\mathbf η}\|_2}{\sqrt{m}}$ holds with high probability provided the measurement vectors ${\mathbf a}_j\in \mathbb{C}^d,$ $j=1,\ldots,m,$ are {i.i.d.} complex sub-Gaussian random vectors and $m\gtrsim d$. Here ${\mathbf η}=(η_1,\ldots,η_m)\in \mathbb{R}^m$ is the noise vector without any assumption on the distribution. Furthermore, we prove that the reconstruction error is sharp. For the case where the target signal ${\mathbf x}_0\in \mathbb{C}^{d}$ is sparse, we establish a similar result for the nonlinear constrained $\ell_1$ minimization model. { To accomplish this, we leverage a strong version of restricted isometry property for an operator on the space of simultaneous low-rank and sparse matrices.}

math.NA↗

arXiv2023

A Cover Time Study of a non-Markovian Algorithm

Given a traversal algorithm, cover time is the expected number of steps needed to visit all nodes in a given graph. A smaller cover time means a higher exploration efficiency of traversal algorithm. Although random walk algorithms have been studied extensively in the existing literature, there has been no cover time result for any non-Markovian method. In this work, we stand on a theoretical perspective and show that the negative feedback strategy (a count-based exploration method) is better than the naive random walk search. In particular, the former strategy can locally improve the search efficiency for an arbitrary graph. It also achieves smaller cover times for special but important graphs, including clique graphs, tree graphs, etc. Moreover, we make connections between our results and reinforcement learning literature to give new insights on why classical UCB and MCTS algorithms are so useful. Various numerical results corroborate our theoretical findings.

cs.DS↗

arXiv2023

SR-R$^2$KAC: Improving Single Image Defocus Deblurring

We propose an efficient deep learning method for single image defocus deblurring (SIDD) by further exploring inverse kernel properties. Although the current inverse kernel method, i.e., kernel-sharing parallel atrous convolution (KPAC), can address spatially varying defocus blurs, it has difficulty in handling large blurs of this kind. To tackle this issue, we propose a Residual and Recursive Kernel-sharing Atrous Convolution (R$^2$KAC). R$^2$KAC builds on a significant observation of inverse kernels, that is, successive use of inverse-kernel-based deconvolutions with fixed size helps remove unexpected large blurs but produces ringing artifacts. Specifically, on top of kernel-sharing atrous convolutions used to simulate multi-scale inverse kernels, R$^2$KAC applies atrous convolutions recursively to simulate a large inverse kernel. Specifically, on top of kernel-sharing atrous convolutions, R$^2$KAC stacks atrous convolutions recursively to simulate a large inverse kernel. To further alleviate the contingent effect of recursive stacking, i.e., ringing artifacts, we add identity shortcuts between atrous convolutions to simulate residual deconvolutions. Lastly, a scale recurrent module is embedded in the R$^2$KAC network, leading to SR-R$^2$KAC, so that multi-scale information from coarse to fine is exploited to progressively remove the spatially varying defocus blurs. Extensive experimental results show that our method achieves the state-of-the-art performance.

cs.CV↗

arXiv2022

Robust Offline Reinforcement Learning with Gradient Penalty and Constraint Relaxation

A promising paradigm for offline reinforcement learning (RL) is to constrain the learned policy to stay close to the dataset behaviors, known as policy constraint offline RL. However, existing works heavily rely on the purity of the data, exhibiting performance degradation or even catastrophic failure when learning from contaminated datasets containing impure trajectories of diverse levels. e.g., expert level, medium level, etc., while offline contaminated data logs exist commonly in the real world. To mitigate this, we first introduce gradient penalty over the learned value function to tackle the exploding Q-functions. We then relax the closeness constraints towards non-optimal actions with critic weighted constraint relaxation. Experimental results show that the proposed techniques effectively tame the non-optimal trajectories for policy constraint offline RL methods, evaluated on a set of contaminated D4RL Mujoco and Adroit datasets.

cs.LG↗

arXiv2022

Affine phase retrieval for sparse signals via $\ell_1$ minimization

Affine phase retrieval is the problem of recovering signals from the magnitude-only measurements with a priori information. In this paper, we use the $\ell_1$ minimization to exploit the sparsity of signals for affine phase retrieval, showing that $O(k\log(en/k))$ Gaussian random measurements are sufficient to recover all $k$-sparse signals by solving a natural $\ell_1$ minimization program, where $n$ is the dimension of signals. For the case where measurements are corrupted by noises, the reconstruction error bounds are given for both real-valued and complex-valued signals. Our results demonstrate that the natural $\ell_1$ minimization program for affine phase retrieval is stable.

cs.IT↗

arXiv2022

No existence of linear algorithm for Fourier phase retrieval

Fourier phase retrieval, which seeks to reconstruct a signal from its Fourier magnitude, is of fundamental importance in fields of engineering and science. In this paper, we give a theoretical understanding of algorithms for Fourier phase retrieval. Particularly, we show if there exists an algorithm which could reconstruct an arbitrary signal ${\mathbf x}\in {\mathbb C}^N$ in $ \mbox{Poly}(N) \log(1/ε)$ time to reach $ε$-precision from its magnitude of discrete Fourier transform and its initial value $x(0)$, then $\mathcal{ P}=\mathcal{NP}$. This demystifies the phenomenon that, although almost all signals are determined uniquely by their Fourier magnitude with a prior conditions, there is no algorithm with theoretical guarantees being proposed over the past few decades. Our proofs employ the result in computational complexity theory that Product Partition problem is NP-complete in the strong sense.

cs.IT↗

arXiv2022

Strong convexity of affine phase retrieval

The recovery of a signal from the intensity measurements with some entries being known in advance is termed as {\em affine phase retrieval}. In this paper, we prove that a natural least squares formulation for the affine phase retrieval is strongly convex on the entire space under some mild conditions, provided the measurements are complex Gaussian random vecotrs and the measurement number $m \gtrsim d \log d$ where $d$ is the dimension of signals. Based on the result, we prove that the simple gradient descent method for the affine phase retrieval converges linearly to the target solution with high probability from an arbitrary initial point. These results show an essential difference between the affine phase retrieval and the classical phase retrieval, where the least squares formulations for the classical phase retrieval are non-convex.

cs.IT↗

arXiv2022

Local Differential Privacy for Belief Functions

In this paper, we propose two new definitions of local differential privacy for belief functions. One is based on Shafer's semantics of randomly coded messages and the other from the perspective of imprecise probabilities. We show that such basic properties as composition and post-processing also hold for our new definitions. Moreover, we provide a hypothesis testing framework for these definitions and study the effect of "don't know" in the trade-off between privacy and utility in discrete distribution estimation.

cs.CR↗

arXiv2021

Performance bound of the intensity-based model for noisy phase retrieval

The aim of noisy phase retrieval is to estimate a signal $\mathbf{x}_0\in \mathbb{C}^d$ from $m$ noisy intensity measurements $b_j=\left\lvert \langle \mathbf{a}_j,\mathbf{x}_0 \rangle \right\rvert^2+η_j, \; j=1,\ldots,m$, where $\mathbf{a}_j \in \mathbb{C}^d$ are known measurement vectors and $η=(η_1,\ldots,η_m)^\top \in \mathbb{R}^m$ is a noise vector. A commonly used model for estimating $\mathbf{x}_0$ is the intensity-based model $\widehat{\mathbf{x}}:=\mbox{argmin}_{\mathbf{x} \in \mathbb{C}^d} \sum_{j=1}^m \big(\left\lvert \langle \mathbf{a}_j,\mathbf{x} \rangle \right\rvert^2-b_j \big)^2$. Although one has already developed many efficient algorithms to solve the intensity-based model, there are very few results about its estimation performance. In this paper, we focus on the estimation performance of the intensity-based model and prove that the error bound satisfies $\min_{θ\in \mathbb{R}}\|\widehat{\mathbf{x}}-e^{iθ}\mathbf{x}_0\|_2 \lesssim \min\Big\{\frac{\sqrt{\|η\|_2}}{{m}^{1/4}}, \frac{\|η\|_2}{\| \mathbf{x}_0\|_2 \cdot \sqrt{m}}\Big\}$ under the assumption of $m \gtrsim d$ and $\mathbf{a}_j, j=1,\ldots,m,$ being Gaussian random vectors. We also show that the error bound is sharp. For the case where $\mathbf{x}_0$ is a $s$-sparse signal, we present a similar result under the assumption of $m \gtrsim s \log (ed/s)$. To the best of our knowledge, our results are the first theoretical guarantees for the intensity-based model and its sparse version. Our proofs employ Mendelson's small ball method which can deliver an effective lower bound on a nonnegative empirical process.

cs.IT↗

arXiv2021

Improved bounds in Weaver's ${\rm KS}_r$ conjecture for high rank positive semidefinite matrices

Recently Marcus, Spielman and Srivastava proved Weaver's ${\rm{KS}}_r$ conjecture, which gives a positive solution to the Kadison-Singer problem. Cohen and Brändén independently extended this result to obtain the arbitrary-rank version of Weaver's ${\rm{KS}}_r$ conjecture. In this paper, we present a new bound in Weaver's ${\rm{KS}}_r$ conjecture for the arbitrary-rank case. To do that, we introduce the definition of $(k,m)$-characteristic polynomials and employ it to improve the previous estimate on the largest root of the mixed characteristic polynomials. For the rank-one case, our bound agrees with the Bownik-Casazza-Marcus-Speegle's bound when $r=2$ and with the Ravichandran-Leake's bound when $r>2$. For the higher-rank case, we sharpen the previous bounds from Cohen and from Brändén .

math.CO↗

arXiv2021

Upper and Lower bounds for matrix discrepancy

The aim of this paper is to study the matrix discrepancy problem. Assume that $ξ_1,\ldots,ξ_n$ are independent scalar random variables with finite support and $\mathbf{u}_1,\ldots,\mathbf{u}_n\in \mathbb{C}^d$. Let $\mathcal{C}_0$ be the minimal constant for which the following holds: \[ {\rm Disc}(\mathbf{u}_1\mathbf{u}_1^*,\ldots,\mathbf{u}_n\mathbf{u}_n^*; ξ_1,\ldots,ξ_n)\,\,:=\,\,\min_{\varepsilon_1\in \mathcal{S}_1,\ldots,\varepsilon_n\in \mathcal{S}_n}\bigg\|\sum_{i=1}^n\mathbb{E}[ξ_i]\mathbf{u}_i\mathbf{u}_i^*-\sum_{i=1}^n\varepsilon_i\mathbf{u}_i\mathbf{u}_i^*\bigg\|\leq \mathcal{C}_0\cdotσ, \] where $σ^2 = \big\|\sum_{i=1}^n \mathbf{Var}[ξ_i](\mathbf{u}_i\mathbf{u}_i^*)^2\big\|$ and $\mathcal{S}_j$ denotes the support of $ξ_j, j=1,\ldots,n$. Motivated by the technology developed by Bownik, Casazza, Marcus, and Speegle, we prove $\mathcal{C}_0\leq 3$. This improves Kyng, Luh and Song's method with which $\mathcal{C}_0\leq 4$. For the case where $\{\mathbf{u}_i\}_{i=1}^n\subset \mathbb{C}^d$ is a unit-norm tight frame with $ n\leq 2d-1$ and $ξ_1,\ldots,ξ_n$ are independent Rademacher random variables, we present the exact value of ${\rm Disc}(\mathbf{u}_1\mathbf{u}_1^*,\ldots,\mathbf{u}_n\mathbf{u}_n^*; ξ_1,\ldots,ξ_n)=\sqrt{\frac{n}{d}}\cdotσ$, which implies $\mathcal{C}_0\geq \sqrt{2}$.

math.CO↗

arXiv2021

Uniqueness and stability for the solution of a nonlinear least squares problem

In this paper, we focus on the nonlinear least squares: $\mbox{min}_{\mathbf{x} \in \mathbb{H}^d}\| |A\mathbf{x}|-\mathbf{b}\|$ where $A\in \mathbb{H}^{m\times d}$, $\mathbf{b} \in \mathbb{R}^m$ with $\mathbb{H} \in \{\mathbb{R},\mathbb{C} \}$ and consider the uniqueness and stability of solutions. Such problem arises, for instance, in phase retrieval and absolute value rectification neural networks. For the case where $\mathbf{b}=|A\mathbf{x}_0|$ for some $\mathbf{x}_0\in \mathbb{H}^d$, many results have been developed to characterize the uniqueness and stability of solutions. However, for the case where $\mathbf{b} \neq |A\mathbf{x}_0| $ for any $\mathbf{x}_0\in \mathbb{H}^d$, there is no existing result for it to the best of our knowledge. In this paper, we first focus on the uniqueness of solutions and show for any matrix $A\in \mathbb{H}^{m \times d}$ there always exists a vector $\mathbf{b} \in \mathbb{R}^m$ such that the solution is not unique. But, in real case, such ``bad'' vectors $\mathbf{b}$ are negligible, namely, if $\mathbf{b} \in \mathbb{R}_{+}^m$ does not lie in some measure zero set, then the solution is unique. We also present some conditions under which the solution is unique. For the stability of solutions, we prove that the solution is never uniformly stable. But if we restrict the vectors $\mathbf{b}$ to any convex set then it is stable.

math.NA↗