arXiv subjects

Xiaoqun Zhang

Publications and source records attributed to Xiaoqun Zhang.

At least 37 records · Page 2Linked to original sources

arXiv2024

Compressing MIMO Channel Submatrices with Tucker Decomposition: Enabling Efficient Storage and Reducing SINR Computation Overhead

Massive multiple-input multiple-output (MIMO) systems employ a large number of antennas to achieve gains in capacity, spectral efficiency, and energy efficiency. However, the large antenna array also incurs substantial storage and computational costs. This paper proposes a novel data compression framework for massive MIMO channel matrices based on tensor Tucker decomposition. To address the substantial storage and computational burdens of massive MIMO systems, we formulate the high-dimensional channel matrices as tensors and propose a novel groupwise Tucker decomposition model. This model efficiently compresses the tensorial channel representations while reducing SINR estimation overhead. We develop an alternating update algorithm and HOSVD-based initialization to compute the core tensors and factor matrices. Extensive simulations demonstrate significant channel storage savings with minimal SINR approximation errors. By exploiting tensor techniques, our approach balances channel compression against SINR computation complexity, providing an efficient means to simultaneously address the storage and computational challenges of massive MIMO.

math.OC↗

arXiv2023

Stochastic Three-Operator Splitting Algorithms for Nonconvex and Nonsmooth Optimization Arising from FLASH Radiotherapy

Radiation therapy (RT) aims to deliver tumoricidal doses with minimal radiation-induced normal-tissue toxicity. Compared to conventional RT (of conventional dose rate), FLASH-RT (of ultra-high dose rate) can provide additional normal tissue sparing, which however has created a new nonconvex and nonsmooth optimization problem that is highly challenging to solve. In this paper, we propose a stochastic three-operator splitting (STOS) algorithm to address the FLASH optimization problem. We establish the convergence and convergence rates of the STOS algorithm under the nonconvex framework for both unbiased gradient estimators and variance-reduced gradient estimators. These stochastic gradient estimators include the most popular ones, such as SGD, SAGA, SARAH, and SVRG, among others. The effectiveness of the STOS algorithm is validated using FLASH radiotherapy planning for patients.

math.OC↗

arXiv2023

NF-ULA: Langevin Monte Carlo with Normalizing Flow Prior for Imaging Inverse Problems

Bayesian methods for solving inverse problems are a powerful alternative to classical methods since the Bayesian approach offers the ability to quantify the uncertainty in the solution. In recent years, data-driven techniques for solving inverse problems have also been remarkably successful, due to their superior representation ability. In this work, we incorporate data-based models into a class of Langevin-based sampling algorithms for Bayesian inference in imaging inverse problems. In particular, we introduce NF-ULA (Normalizing Flow-based Unadjusted Langevin algorithm), which involves learning a normalizing flow (NF) as the image prior. We use NF to learn the prior because a tractable closed-form expression for the log prior enables the differentiation of it using autograd libraries. Our algorithm only requires a normalizing flow-based generative network, which can be pre-trained independently of the considered inverse problem and the forward operator. We perform theoretical analysis by investigating the well-posedness and non-asymptotic convergence of the resulting NF-ULA algorithm. The efficacy of the proposed NF-ULA algorithm is demonstrated in various image restoration problems such as image deblurring, image inpainting, and limited-angle X-ray computed tomography (CT) reconstruction. NF-ULA is found to perform better than competing methods for severely ill-posed inverse problems.

math.NA↗

arXiv2023

Enhancing SAEAs with Unevaluated Solutions: A Case Study of Relation Model for Expensive Optimization

Surrogate-assisted evolutionary algorithms (SAEAs) hold significant importance in resolving expensive optimization problems~(EOPs). Extensive efforts have been devoted to improving the efficacy of SAEAs through the development of proficient model-assisted selection methods. However, generating high-quality solutions is a prerequisite for selection. The fundamental paradigm of evaluating a limited number of solutions in each generation within SAEAs reduces the variance of adjacent populations, thus impacting the quality of offspring solutions. This is a frequently encountered issue, yet it has not gained widespread attention. This paper presents a framework using unevaluated solutions to enhance the efficiency of SAEAs. The surrogate model is employed to identify high-quality solutions for direct generation of new solutions without evaluation. To ensure dependable selection, we have introduced two tailored relation models for the selection of the optimal solution and the unevaluated population. A comprehensive experimental analysis is performed on two test suites, which showcases the superiority of the relation model over regression and classification models in the selection phase. Furthermore, the surrogate-selected unevaluated solutions with high potential have been shown to significantly enhance the efficiency of the algorithm.

cs.NE↗

arXiv2023

Robust Graph Representation Learning for Local Corruption Recovery

The performance of graph representation learning is affected by the quality of graph input. While existing research usually pursues a globally smoothed graph embedding, we believe the rarely observed anomalies are as well harmful to an accurate prediction. This work establishes a graph learning scheme that automatically detects (locally) corrupted feature attributes and recovers robust embedding for prediction tasks. The detection operation leverages a graph autoencoder, which does not make any assumptions about the distribution of the local corruptions. It pinpoints the positions of the anomalous node attributes in an unbiased mask matrix, where robust estimations are recovered with sparsity promoting regularizer. The optimizer approaches a new embedding that is sparse in the framelet domain and conditionally close to input observations. Extensive experiments are provided to validate our proposed model can recover a robust graph representation from black-box poisoning and achieve excellent performance.

cs.LG↗

arXiv2023

Federated Primal Dual Fixed Point Algorithm

Federated learning (FL) is a distributed learning paradigm that allows several clients to learn a global model without sharing their private data. In this paper, we generalize a primal dual fixed point (PDFP) \cite{PDFP} method to federated learning setting and propose an algorithm called Federated PDFP (FPDFP) for solving composite optimization problems. In addition, a quantization scheme is applied to reduce the communication overhead during the learning process. An $O(\frac{1}{k})$ convergence rate (where $k$ is the communication round) of the proposed FPDFP is provided. Numerical experiments, including graph-guided logistic regression, 3D Computed Tomography (CT) reconstruction are considered to evaluate the proposed algorithm.

math.OC↗

arXiv2023

Approximate Primal-Dual Fixed-Point based Langevin Algorithms for Non-smooth Convex Potentials

The Langevin algorithms are frequently used to sample the posterior distributions in Bayesian inference. In many practical problems, however, the posterior distributions often consist of non-differentiable components, posing challenges for the standard Langevin algorithms, as they require to evaluate the gradient of the energy function in each iteration. To this end, a popular remedy is to utilize the proximity operator, and as a result one needs to solve a proximity subproblem in each iteration. The conventional practice is to solve the subproblems accurately, which can be exceedingly expensive, as the subproblem needs to be solved in each iteration. We propose an approximate primal-dual fixed-point algorithm for solving the subproblem, which only seeks an approximate solution of the subproblem and therefore reduces the computational cost considerably. We provide theoretical analysis of the proposed method and also demonstrate its performance with numerical examples.

math.NA↗

arXiv2023

On the convergence of continuous and discrete unbalanced optimal transport models

We consider a Beckmann formulation of an unbalanced optimal transport (UOT) problem. The $Γ$-convergence of this formulation of UOT to the corresponding optimal transport (OT) problem is established as the balancing parameter $α$ goes to infinity. The discretization of the problem is further shown to be asymptotic preserving regarding the same limit, which ensures that a numerical method can be applied uniformly and the solutions converge to the one of the OT problem automatically. Particularly, there exists a critical value, which is independent of the mesh size, such that the discrete problem reduces to the discrete OT problem for $α$ being larger than this critical value. The discrete problem is solved by a convergent primal-dual hybrid algorithm and the iterates for UOT are also shown to converge to that for OT. Finally, numerical experiments on shape deformation and partial color transfer are implemented to validate the theoretical convergence and the proposed numerical algorithm.

math.NA↗

arXiv2022

A Dataset-free Deep learning Method for Low-Dose CT Image Reconstruction

Low-dose CT (LDCT) imaging attracted a considerable interest for the reduction of the object's exposure to X-ray radiation. In recent years, supervised deep learning (DL) has been extensively studied for LDCT image reconstruction, which trains a network over a dataset containing many pairs of normal-dose and low-dose images. However, the challenge on collecting many such pairs in the clinical setup limits the application of such supervised-learning-based methods for LDCT image reconstruction in practice. Aiming at addressing the challenges raised by the collection of training dataset, this paper proposed a unsupervised deep learning method for LDCT image reconstruction, which does not require any external training data. The proposed method is built on a re-parametrization technique for Bayesian inference via deep network with random weights, combined with additional total variational~(TV) regularization. The experiments show that the proposed method noticeably outperforms existing dataset-free image reconstruction methods on the test data.

eess.IV↗

arXiv2022

A stochastic three-block splitting algorithm and its application to quantized deep neural networks

Deep neural networks (DNNs) have made great progress in various fields. In particular, the quantized neural network is a promising technique making DNNs compatible on resource-limited devices for memory and computation saving. In this paper, we mainly consider a non-convex minimization model with three blocks to train quantized DNNs and propose a new stochastic three-block alternating minimization (STAM) algorithm to solve it. We develop a convergence theory for the STAM algorithm and obtain an $ε$-stationary point with optimal convergence rate $\mathcal{O}(ε^{-4})$. Furthermore, we apply our STAM algorithm to train DNNs with relaxed binary weights. The experiments are carried out on three different network structures, namely VGG-11, VGG-16 and ResNet-18. These DNNs are trained using two different data sets, CIFAR-10 and CIFAR-100, respectively. We compare our STAM algorithm with some classical efficient algorithms for training quantized neural networks. The test accuracy indicates the effectiveness of STAM algorithm for training relaxed binary quantization DNNs.

math.OC↗

arXiv2021

Variable metric extrapolation proximal iterative hard thresholding method for $\ell_0$ minimization problem

In this paper, we consider the $\ell_0$ minimization problem whose objective function is the sum of $\ell_0$-norm and convex differentiable function. A variable metric type method which combines the PIHT method and the skill in quasi-newton method, named variable metric extrapolation proximal iterative hard thresholding (VMEPIHT) method, is proposed. Then we analyze its convergence, linear convergence rate and superlinear convergence rate under appropriate assumptions. Finally, we conduct numerical experiments on compressive sensing problem and CT image reconstruction problem to confirm VMPIHT method's efficiency, compared with other state-of-art methods.

math.OC↗

arXiv2021

Low-Rank and Framelet Based Sparsity Decomposition for Interventional MRI Reconstruction

Objective: Interventional MRI (i-MRI) is crucial for MR image-guided therapy. Current image reconstruction methods for dynamic MR imaging are mostly retrospective that may not be suitable for i-MRI in real-time. Therefore, an algorithm to reconstruct images without a temporal pattern as in dynamic imaging is needed for i-MRI. Methods: We proposed a low-rank and sparsity (LS) decomposition algorithm with framelet transform to reconstruct the interventional feature with a high temporal resolution. Different from the existing LS based algorithm, we utilized the spatial sparsity of both the low-rank and sparsity components. We also used a primal dual fixed point (PDFP) method for optimization of the objective function to avoid solving sub-problems. Intervention experiments with gelatin and brain phantoms were carried out for validation. Results: The LS decomposition with framelet transform and PDFP could provide the best reconstruction performance compared with those without. Satisfying reconstruction results were obtained with only 10 radial spokes for a temporal resolution of 60 ms. Conclusion and Significance: The proposed method has the potential for i-MRI in many different application scenarios.

physics.med-ph↗

arXiv2020

A Stochastic Alternating Direction Method of Multipliers for Non-smooth and Non-convex Optimization

Alternating direction method of multipliers (ADMM) is a popular first-order method owing to its simplicity and efficiency. However, similar to other proximal splitting methods, the performance of ADMM degrades significantly when the scale of the optimization problems to solve becomes large. In this paper, we consider combining ADMM with a class of stochastic gradient with variance reduction for solving large-scale non-convex and non-smooth optimization problems. Global convergence of the generated sequence is established under the extra additional assumption that the object function satisfies Kurdyka-Lojasiewicz (KL) property. Numerical experiments on graph-guided fused Lasso and computed tomography are presented to demonstrate the performance of the proposed methods.

math.OC↗

arXiv2020

A Stochastic Variance Reduced Primal Dual Fixed Point Method For Linearly Constrained Separable Optimization

In this paper we combine the stochastic variance reduced gradient (SVRG) method [17] with the primal dual fixed point method (PDFP) proposed in [7] to solve a sum of two convex functions and one of which is linearly composite. This type of problems are typically arisen in sparse signal and image reconstruction. The proposed SVRG-PDFP can be seen as a generalization of Prox-SVRG [37] originally designed for the minimization of a sum of two convex functions. Based on some standard assumptions, we propose two variants, one is for strongly convex objective function and the other is for general convex cases. Convergence analysis shows that the convergence rate of SVRG-PDFP is O(1/k) (here k is the iteration number) for general convex objective function and linear for k strongly convex case. Numerical examples on machine learning and CT image reconstruction are provided to show the effectiveness of the algorithms.

math.OC↗

arXiv2020

A parameterized Douglas-Rachford Splitting algorithm for nonconvex optimization

In this paper, we study a parameterized Douglas-Rachford splitting method for a class of nonconvex optimization problem. A new merit function is constructed to establish the convergence of the whole sequence generated by the parameterized Douglas-Rachford splitting method. We then apply the parameterized Douglas-Rachford splitting method to three important classes of nonconvex optimization problems arising in data science: sparsity constrained least squares problem, feasibility problem and low rank matrix completion. Numerical results validate the effectiveness of the parameterized Douglas-Rachford splitting method compared with some other classical methods.

math.OC↗

arXiv2020

A three-operator splitting algorithm for nonconvex sparsity regularization

Sparsity regularization has been largely applied in many fields, such as signal and image processing and machine learning. In this paper, we mainly consider nonconvex minimization problems involving three terms, for the applications such as: sparse signal recovery and low rank matrix recovery. We employ a three-operator splitting proposed by Davis and Yin (called DYS) to solve the resulting possibly nonconvex problems and develop the convergence theory for this three-operator splitting algorithm in the nonconvex case. We show that if the step size is chosen less than a computable threshold, then the whole sequence converges to a stationary point. By defining a new decreasing energy function associated with the DYS method, we establish the global convergence of the whole sequence and a local convergence rate under an additional assumption that this energy function is a Kurdyka-$Ł$ojasiewicz function. We also provide sufficient conditions for the boundedness of the generated sequence. Finally, some numerical experiments are conducted to compare the DYS algorithm with some classical efficient algorithms for sparse signal recovery and low rank matrix completion. The numerical results indicate that DYS method outperforms the exsiting methods for these specific applications.

math.OC↗

arXiv2020

Semi-Implicit Back Propagation

Neural network has attracted great attention for a long time and many researchers are devoted to improve the effectiveness of neural network training algorithms. Though stochastic gradient descent (SGD) and other explicit gradient-based methods are widely adopted, there are still many challenges such as gradient vanishing and small step sizes, which leads to slow convergence and instability of SGD algorithms. Motivated by error back propagation (BP) and proximal methods, we propose a semi-implicit back propagation method for neural network training. Similar to BP, the difference on the neurons are propagated in a backward fashion and the parameters are updated with proximal mapping. The implicit update for both hidden neurons and parameters allows to choose large step size in the training algorithm. Finally, we also show that any fixed point of convergent sequences produced by this algorithm is a stationary point of the objective loss function. The experiments on both MNIST and CIFAR-10 demonstrate that the proposed semi-implicit BP algorithm leads to better performance in terms of both loss decreasing and training/validation accuracy, compared to SGD and a similar algorithm ProxBP.

math.OC↗

arXiv2019

Bayesian inference and uncertainty quantification for image reconstruction with Poisson data

We provide a complete framework for performing infinite-dimensional Bayesian inference and uncertainty quantification for image reconstruction with Poisson data. In particular, we address the following issues to make the Bayesian framework applicable in practice. We first introduce a positivity-preserving reparametrization, and we prove that under the reparametrization and a hybrid prior, the posterior distribution is well-posed in the infinite dimensional setting. Second we provide a dimension-independent MCMC algorithm, based on the preconditioned Crank-Nicolson Langevin method, in which we use a primal-dual scheme to compute the offset direction. Third we give a method combining the model discrepancy method and maximum likelihood estimation to determine the regularization parameter in the hybrid prior. Finally we propose to use the obtained posterior distribution to detect artifacts in a recovered image. We provide an example to demonstrate the effectiveness of the proposed method.

math.NA↗