arXiv subjects

Zhi-Quan Luo

Publications and source records attributed to Zhi-Quan Luo.

At least 91 records · Page 5Linked to original sources

arXiv2021

An efficient linear programming rounding-and-refinement algorithm for large-scale network slicing problem

In this paper, we consider the network slicing problem which attempts to map multiple customized virtual network requests (also called services) to a common shared network infrastructure and allocate network resources to meet diverse service requirements, and propose an efficient two-stage algorithm for solving this NP-hard problem. In the first stage, the proposed algorithm uses an iterative linear programming (LP) rounding procedure to place the virtual network functions of all services into cloud nodes while taking traffic routing of all services into consideration; in the second stage, the proposed algorithm uses an iterative LP refinement procedure to obtain a solution for traffic routing of all services with their end-to-end delay constraints being satisfied. Compared with the existing algorithms which either have an exponential complexity or return a low-quality solution, our proposed algorithm achieves a better trade-off between solution quality and computational complexity. In particular, the worst-case complexity of our proposed algorithm is polynomial, which makes it suitable for solving large-scale problems. Numerical results demonstrate the effectiveness and efficiency of our proposed algorithm.

cs.NI↗

arXiv2020

Analysis of Optimal Thresholding Algorithms for Compressed Sensing

The optimal $k$-thresholding (OT) and optimal $k$-thresholding pursuit (OTP) are newly introduced frameworks of thresholding techniques for compressed sensing and signal approximation. Such frameworks motivate the practical and efficient algorithms called relaxed optimal $k$-thresholding ($\textrm{ROT}ω$) and relaxed optimal $k$-thresholding pursuit ($\textrm{ROTP}ω$) which are developed through the tightest convex relaxations of OT and OTP, where $ω$ is a prescribed integer number. The preliminary numerical results demonstrated in \cite{Z19} indicate that these approaches can stably reconstruct signals with a wide range of sparsity levels. However, the guaranteed performance of these algorithms with parameter $ ω\geq 2 $ has not yet established in \cite{Z19}. The purpose of this paper is to show the guaranteed performance of OT and OTP in terms of the restricted isometry property (RIP) of nearly optimal order for the sensing matrix governing the $k$-sparse signal recovery, and to establish the first guaranteed performance result for $\textrm{ROT}ω$ and $\textrm{ROTP}ω$ with $ ω\geq 2.$ In the meantime, we provide a numerical comparison between ROTP$ω$ and several existing thresholding methods.

math.OC↗

arXiv2020

Distributed Stochastic Consensus Optimization with Momentum for Nonconvex Nonsmooth Problems

While many distributed optimization algorithms have been proposed for solving smooth or convex problems over the networks, few of them can handle non-convex and non-smooth problems. Based on a proximal primal-dual approach, this paper presents a new (stochastic) distributed algorithm with Nesterov momentum for accelerated optimization of non-convex and non-smooth problems. Theoretically, we show that the proposed algorithm can achieve an $ε$-stationary solution under a constant step size with $\mathcal{O}(1/ε^2)$ computation complexity and $\mathcal{O}(1/ε)$ communication complexity. When compared to the existing gradient tracking based methods, the proposed algorithm has the same order of computation complexity but lower order of communication complexity. To the best of our knowledge, the presented result is the first stochastic algorithm with the $\mathcal{O}(1/ε)$ communication complexity for non-convex and non-smooth problems. Numerical experiments for a distributed non-convex regression problem and a deep neural network based classification problem are presented to illustrate the effectiveness of the proposed algorithms.

math.OC↗

arXiv2020

Transfer Learning and Meta Learning Based Fast Downlink Beamforming Adaptation

This paper studies fast adaptive beamforming optimization for the signal-to-interference-plus-noise ratio balancing problem in a multiuser multiple-input single-output downlink system. Existing deep learning based approaches to predict beamforming rely on the assumption that the training and testing channels follow the same distribution which may not hold in practice. As a result, a trained model may lead to performance deterioration when the testing network environment changes. To deal with this task mismatch issue, we propose two offline adaptive algorithms based on deep transfer learning and meta-learning, which are able to achieve fast adaptation with the limited new labelled data when the testing wireless environment changes. Furthermore, we propose an online algorithm to enhance the adaptation capability of the offline meta algorithm in realistic non-stationary environments. Simulation results demonstrate that the proposed adaptive algorithms achieve much better performance than the direct deep learning algorithm without adaptation in new environments. The meta-learning algorithm outperforms the deep transfer learning algorithm and achieves near optimal performance. In addition, compared to the offline meta-learning algorithm, the proposed online meta-learning algorithm shows superior adaption performance in changing environments.

cs.IT↗

arXiv2020

Pushing The Limit of Type I Codebook For FDD Massive MIMO Beamforming: A Channel Covariance Reconstruction Approach

There is a fundamental trade-off between the channel representation resolution of codebooks and the overheads of feedback communications in the fifth generation new radio (5G NR) frequency division duplex (FDD) massive multiple-input and multiple-output (MIMO) systems. In particular, two types of codebooks (namely Type I and Type II codebooks) are introduced with different resolution and overhead requirements. Although the Type I codebook based scheme requires lower feedback overhead, its channel state information (CSI) reconstruction and beamforming performance are not as good as those from the Type II codebook based scheme. However, since the Type I codebook based scheme has been widely used in 4G systems for many years, replacing it by the Type II codebook based scheme overnight is too costly to be an option. Therefore, in this paper, using Type I codebook, we leverage advances in cutting plane method to optimize the CSI reconstruction at the base station (BS), in order to close the gap between these two codebook based beamforming schemes. Numerical results based on channel samples from QUAsi Deterministic RadIo channel GenerAtor (QuaDRiGa) are presented to show the excellent performance of the proposed algorithm in terms of beamforming vector acquisition.

eess.SP↗

arXiv2020

Improved RIP-Based Bounds for Guaranteed Performance of two Compressed Sensing Algorithms

Iterative hard thresholding (IHT) and compressive sampling matching pursuit (CoSaMP) are two types of mainstream compressed sensing algorithms using hard thresholding operators for signal recovery and approximation. The guaranteed performance for signal recovery via these algorithms has mainly been analyzed under the condition that the restricted isometry constant of a sensing matrix, denoted by $ δ_K$ (where $K$ is an integer number), is smaller than a certain threshold value in the interval $(0,1).$ The condition $ δ_{K}< δ^*$ for some constant $ δ^* \leq 1 $ ensuring the success of signal recovery with a specific algorithm is called the restricted-isometry-property-based (RIP-based) bound for guaranteed performance of the algorithm. At the moment, the best known RIP-based bound for the guaranteed recovery of $k$-sparse signals via IHT is $δ_{3k}< 1/\sqrt{3}\approx 0.5774,$ and the bound for guaranteed recovery via CoSaMP is $δ_{4k} < 0.4782. $ A fundamental question in this area is whether such theoretical results can be further improved. The purpose of this paper is to affirmatively answer this question and rigorously show that the RIP-based bounds for guaranteed performance of IHT can be significantly improved to $ δ_{3k} < (\sqrt{5}-1)/2 \approx 0.618, $ and the bound for CoSaMP can be improved and pushed to $ δ_{4k}< 0.5102. $ These improvements are achieved through a deep property of the hard thresholding operator.

eess.SP↗

arXiv2020

A Proximal Alternating Direction Method of Multiplier for Linearly Constrained Nonconvex Minimization

Consider the minimization of a nonconvex differentiable function over a polyhedron. A popular primal-dual first-order method for this problem is to perform a gradient projection iteration for the augmented Lagrangian function and then update the dual multiplier vector using the constraint residual. However, numerical examples show that this approach can exhibit "oscillation" and may not converge. In this paper, we propose a proximal alternating direction method of multipliers for the multi-block version of this problem. A distinctive feature of this method is the introduction of a "smoothed" (i.e., exponentially weighted) sequence of primal iterates, and the inclusion, at each iteration, to the augmented Lagrangian function a quadratic proximal term centered at the current smoothed primal iterate. The resulting proximal augmented Lagrangian function is inexactly minimized (via a gradient projection step) at each iteration while the dual multiplier vector is updated using the residual of the linear constraints. When the primal and dual stepsizes are chosen sufficiently small, we show that suitable "smoothing" can stabilize the "oscillation", and the iterates of the new proximal ADMM algorithm converge to a stationary point under some mild regularity conditions. Furthermore, when the objective function is quadratic, we establish the linear convergence of the algorithm. Our proof is based on a new potential function and a novel use of error bounds.

math.OC↗

arXiv2020

Optimally Combining Classifiers for Semi-Supervised Learning

This paper considers semi-supervised learning for tabular data. It is widely known that Xgboost based on tree model works well on the heterogeneous features while transductive support vector machine can exploit the low density separation assumption. However, little work has been done to combine them together for the end-to-end semi-supervised learning. In this paper, we find these two methods have complementary properties and larger diversity, which motivates us to propose a new semi-supervised learning method that is able to adaptively combine the strengths of Xgboost and transductive support vector machine. Instead of the majority vote rule, an optimization problem in terms of ensemble weight is established, which helps to obtain more accurate pseudo labels for unlabeled data. The experimental results on the UCI data sets and real commercial data set demonstrate the superior classification performance of our method over the five state-of-the-art algorithms improving test accuracy by about $3\%-4\%$. The partial code can be found at https://github.com/hav-cam-mit/CTO.

cs.LG↗

arXiv2020

Network Slicing for Service-Oriented Networks with Flexible Routing and Guaranteed E2E Latency

Network function virtualization is a promising technology to simultaneously support multiple services with diverse characteristics and requirements in the fifth generation and beyond networks. In practice, each service consists of a predetermined sequence of functions, called service function chain (SFC), running on a cloud environment. To make different service slices work properly in harmony, it is crucial to select the cloud nodes to deploy the functions in the SFC and flexibly route the flow of the services such that these functions are processed in sequence, the end-to-end (E2E) latency constraints of all services are guaranteed, and all resource constraints are respected. In this paper, we propose a new (mixed binary linear program) formulation of the above network slicing problem that optimizes the system energy efficiency while jointly considers the resource budget, functional instantiation, flow routing, and E2E latency requirement. Numerical results show the advantage of the proposed formulation compared to the existing ones.

cs.NI↗

arXiv2019

Inexact Block Coordinate Descent Algorithms for Nonsmooth Nonconvex Optimization

In this paper, we propose an inexact block coordinate descent algorithm for large-scale nonsmooth nonconvex optimization problems. At each iteration, a particular block variable is selected and updated by inexactly solving the original optimization problem with respect to that block variable. More precisely, a local approximation of the original optimization problem is solved. The proposed algorithm has several attractive features, namely, i) high flexibility, as the approximation function only needs to be strictly convex and it does not have to be a global upper bound of the original function; ii) fast convergence, as the approximation function can be designed to exploit the problem structure at hand and the stepsize is calculated by the line search; iii) low complexity, as the approximation subproblems are much easier to solve and the line search scheme is carried out over a properly constructed differentiable function; iv) guaranteed convergence of a subsequence to a stationary point, even when the objective function does not have a Lipschitz continuous gradient. Interestingly, when the approximation subproblem is solved by a descent algorithm, convergence of a subsequence to a stationary point is still guaranteed even if the approximation subproblem is solved inexactly by terminating the descent algorithm after a finite number of iterations. These features make the proposed algorithm suitable for large-scale problems where the dimension exceeds the memory and/or the processing capability of the existing hardware. These features are also illustrated by several applications in signal processing and machine learning, for instance, network anomaly detection and phase retrieval.

math.OC↗

arXiv2019

A General $\mathcal{O}(n^2)$ Hyper-Parameter Optimization for Gaussian Process Regression with Cross-Validation and Non-linearly Constrained ADMM

Hyper-parameter optimization remains as the core issue of Gaussian process (GP) for machine learning nowadays. The benchmark method using maximum likelihood (ML) estimation and gradient descent (GD) is impractical for processing big data due to its $O(n^3)$ complexity. Many sophisticated global or local approximation models, for instance, sparse GP, distributed GP, have been proposed to address such complexity issue. In this paper, we propose two novel and general-purpose GP hyper-parameter training schemes (GPCV-ADMM) by replacing ML with cross-validation (CV) as the fitting criterion and replacing GD with a non-linearly constrained alternating direction method of multipliers (ADMM) as the optimization method. The proposed schemes are of $O(n^2)$ complexity for any covariance matrix without special structure. We conduct various experiments based on both synthetic and real data sets, wherein the proposed schemes show excellent performance in terms of convergence, hyper-parameter estimation accuracy, and computational time in comparison with the traditional ML based routines given in the GPML toolbox.

stat.ML↗

arXiv2019

Quadratic Matrix Inequality Approach to Robust Adaptive Beamforming for General-Rank Signal Model

The worst-case robust adaptive beamforming problem for general-rank signal model is considered. This is a nonconvex problem, and an approximate version of it (obtained by introducing a matrix decomposition on the presumed covariance matrix of the desired signal) has been well studied in the literature. Different from the existing literature, herein however the original beamforming problem is tackled. Resorting to the strong duality of linear conic programming, the robust adaptive beamforming problem for general-rank signal model is reformulated into an equivalent quadratic matrix inequality (QMI) problem. By employing a linear matrix inequality (LMI) relaxation technique, the QMI problem is turned into a convex semidefinite programming problem. Using the fact that there is often a positive gap between the QMI problem and its LMI relaxation, an approximate algorithm is proposed to solve the robust adaptive beamforming in the QMI form. Besides, several sufficient optimality conditions for the nonconvex QMI problem are built. To validate our results, simulation examples are presented, which also demonstrate the improved performance of the new robust beamformer in terms of the output signal-to-interference-plus-noise ratio.

eess.SP↗

arXiv2019

A Linearly Convergent Doubly Stochastic Gauss-Seidel Algorithm for Solving Linear Equations and A Certain Class of Over-Parameterized Optimization Problems

Consider the classical problem of solving a general linear system of equations $Ax=b$. It is well known that the (successively over relaxed) Gauss-Seidel scheme and many of its variants may not converge when $A$ is neither diagonally dominant nor symmetric positive definite. Can we have a linearly convergent G-S type algorithm that works for {\it any} $A$? In this paper we answer this question affirmatively by proposing a doubly stochastic G-S algorithm that is provably linearly convergent (in the mean square error sense) for any feasible linear system of equations. The key in the algorithm design is to introduce a {\it nonuniform double stochastic} scheme for picking the equation and the variable in each update step as well as a stepsize rule. These techniques also generalize to certain iterative alternating projection algorithms for solving the linear feasibility problem $A x\le b$ with an arbitrary $A$, as well as high-dimensional minimization problems for training over-parameterized models in machine learning. Our results demonstrate that a carefully designed randomization scheme can make an otherwise divergent G-S algorithm converge.

math.OC↗

arXiv2019

On the Efficiency of Random Permutation for ADMM and Coordinate Descent

Random permutation is observed to be powerful for optimization algorithms: for multi-block ADMM (alternating direction method of multipliers), while the classical cyclic version divergence, the randomly permuted version converges in practice; for BCD (block coordinate descent), the randomly permuted version is typically faster than other versions. In this paper, we provide strong theoretical evidence that random permutation has positive effects on ADMM and BCD, by analyzing randomly permuted ADMM (RP-ADMM) for solving linear systems of equations, and randomly permuted BCD (RP-BCD) for solving unconstrained quadratic problems. First, we prove that RP-ADMM converges in expectation for solving systems of linear equations. The key technical result is that the spectrum of the expected update matrix of RP-BCD lies in $(-1/3, 1)$, instead of the typical range $(-1, 1)$. Second, we establish expected convergence rates of RP-ADMM for solving linear sytems and RP-BCD for solving unconstrained quadratic problems. This expected rate of RP-BCD is $O(n)$ times better than the worst-case rate of cyclic BCD, thus establishing a gap of at least $O(n)$ between RP-BCD and cyclic BCD. To analyze RP-BCD, we propose a conjecture of a new matrix AM-GM (algebraic mean-geometric mean) inequality, and prove a weaker version of it.

math.OC↗

arXiv2018

Bilinear Factor Matrix Norm Minimization for Robust PCA: Algorithms and Applications

The heavy-tailed distributions of corrupted outliers and singular values of all channels in low-level vision have proven effective priors for many applications such as background modeling, photometric stereo and image alignment. And they can be well modeled by a hyper-Laplacian. However, the use of such distributions generally leads to challenging non-convex, non-smooth and non-Lipschitz problems, and makes existing algorithms very slow for large-scale applications. Together with the analytic solutions to lp-norm minimization with two specific values of p, i.e., p=1/2 and p=2/3, we propose two novel bilinear factor matrix norm minimization models for robust principal component analysis. We first define the double nuclear norm and Frobenius/nuclear hybrid norm penalties, and then prove that they are in essence the Schatten-1/2 and 2/3 quasi-norms, respectively, which lead to much more tractable and scalable Lipschitz optimization problems. Our experimental analysis shows that both our methods yield more accurate solutions than original Schatten quasi-norm minimization, even when the number of observations is very limited. Finally, we apply our penalties to various low-level vision problems, e.g., text removal, moving object detection, image alignment and inpainting, and show that our methods usually outperform the state-of-the-art methods.

cs.LG↗

arXiv2018

Decision Level Fusion: An Event Driven Approach

This paper presents a technique that combines the occurrence of certain events, as observed by different sensors, in order to detect and classify objects. This technique explores the extent of dependence between features being observed by the sensors, and generates more informed probability distributions over the events. Provided some additional information about the features of the object, this fusion technique can outperform other existing decision level fusion approaches that may not take into account the relationship between different features.

eess.SP↗

arXiv2018

Optimal Estimation of Sensor Biases for Asynchronous Multi-Sensor Data Fusion

An important step in a multi-sensor surveillance system is to estimate sensor biases from their noisy asynchronous measurements. This estimation problem is computationally challenging due to the highly nonlinear transformation between the global and local coordinate systems as well as the measurement asynchrony from different sensors. In this paper, we propose a novel nonlinear least squares (LS) formulation for the problem by assuming the existence of a reference target moving with an (unknown) constant velocity. We also propose an efficient block coordinate decent (BCD) optimization algorithm, with a judicious initialization, to solve the problem. The proposed BCD algorithm alternately updates the range and azimuth bias estimates by solving linear least squares problems and semidefinite programs (SDPs). In the absence of measurement noise, the proposed algorithm is guaranteed to find the global solution of the problem and the true biases. Simulation results show that the proposed algorithm significantly outperforms the existing approaches in terms of the root mean square error (RMSE).

cs.IT↗

arXiv2017

Weak Stability of $\ell_1$-minimization Methods in Sparse Data Reconstruction

As one of the most plausible convex optimization methods for sparse data reconstruction, $\ell_1$-minimization plays a fundamental role in the development of sparse optimization theory. The stability of this method has been addressed in the literature under various assumptions such as restricted isometry property (RIP), null space property (NSP), and mutual coherence. In this paper, we propose a unified means to develop the so-called weak stability theory for $\ell_1$-minimization methods under the condition called weak range space property of a transposed design matrix, which turns out to be a necessary and sufficient condition for the standard $\ell_1$-minimization method to be weakly stable in sparse data reconstruction. The reconstruction error bounds established in this paper are measured by the so-called Robinson's constant. We also provide a unified weak stability result for standard $\ell_1$-minimization under several existing compressed-sensing matrix properties. In particular, the weak stability of $\ell_1$-minimization under the constant-free range space property of order $k$ of the transposed design matrix is established for the first time in this paper. Different from the existing analysis, we utilize the classic Hoffman's Lemma concerning the error bound of linear systems as well as the Dudley's theorem concerning the polytope approximation of the unit $\ell_2$-ball to show that $\ell_1$-minimization is robustly and weakly stable in recovering sparse data from inaccurate measurements.

cs.IT↗