arXiv subjects

Anthony Man-Cho So

Publications and source records attributed to Anthony Man-Cho So.

At least 109 records · Page 6Linked to original sources

arXiv2017

Vector Transport-Free SVRG with General Retraction for Riemannian Optimization: Complexity Analysis and Practical Implementation

In this paper, we propose a vector transport-free stochastic variance reduced gradient (SVRG) method with general retraction for empirical risk minimization over Riemannian manifold. Existing SVRG methods on manifold usually consider a specific retraction operation, and involve additional computational costs such as parallel transport or vector transport. The vector transport-free SVRG with general retraction we propose in this paper handles general retraction operations, and do not need additional computational costs mentioned above. As a result, we name our algorithm S-SVRG, where the first "S" means simple. We analyze the iteration complexity of S-SVRG for obtaining an $ε$-stationary point and its local linear convergence by assuming the Łojasiewicz inequality, which naturally holds for PCA and holds with high probability for matrix completion problem. We also incorporate the Barzilai-Borwein step size and design a very practical S-SVRG-BB method. Numerical results on PCA and matrix completion problems are reported to demonstrate the efficiency of our methods.

math.OC↗

arXiv2017

Scalable and Flexible Multiview MAX-VAR Canonical Correlation Analysis

Generalized canonical correlation analysis (GCCA) aims at finding latent low-dimensional common structure from multiple views (feature vectors in different domains) of the same entities. Unlike principal component analysis (PCA) that handles a single view, (G)CCA is able to integrate information from different feature spaces. Here we focus on MAX-VAR GCCA, a popular formulation which has recently gained renewed interest in multilingual processing and speech modeling. The classic MAX-VAR GCCA problem can be solved optimally via eigen-decomposition of a matrix that compounds the (whitened) correlation matrices of the views; but this solution has serious scalability issues, and is not directly amenable to incorporating pertinent structural constraints such as non-negativity and sparsity on the canonical components. We posit regularized MAX-VAR GCCA as a non-convex optimization problem and propose an alternating optimization (AO)-based algorithm to handle it. Our algorithm alternates between {\em inexact} solutions of a regularized least squares subproblem and a manifold-constrained non-convex subproblem, thereby achieving substantial memory and computational savings. An important benefit of our design is that it can easily handle structure-promoting regularization. We show that the algorithm globally converges to a critical point at a sublinear rate, and approaches a global optimal solution at a linear rate when no regularization is considered. Judiciously designed simulations and large-scale word embedding tasks are employed to showcase the effectiveness of the proposed algorithm.

stat.ML↗

arXiv2016

Unraveling the Rank-One Solution Mystery of Robust MISO Downlink Transmit Optimization: A Verifiable Sufficient Condition via a New Duality Result

This paper concentrates on a robust transmit optimization problem for the multiuser multi-input single-output (MISO) downlink scenario and under inaccurate channel state information (CSI). This robust problem deals with a general-rank transmit covariance design, and it follows a safe rate-constrained formulation under spherically bounded CSI uncertainties. Curiously, simulation results in previous works suggested that the robust problem admits rank-one optimal transmit covariances in most cases. Such a numerical finding is appealing because transmission with rank-one covariances can be easily realized by single-stream transmit beamforming. This gives rise to a fundamentally important question, namely, whether we can theoretically identify conditions under which the robust problem admits a rank-one solution. In this paper, we identify one such condition. Simply speaking, we show that the robust problem is guaranteed to admit a rank-one solution if the CSI uncertainties are not too large and the multiuser channel is not too poorly conditioned. To establish the aforementioned condition, we develop a novel duality framework, through which an intimate relationship between the robust problem and a related maximin problem is revealed. Our condition involves only a simple expression with respect to the multiuser channel and other system parameters. In particular, unlike other sufficient rank-one conditions that have appeared in the literature, ours is verifiable. The application of our analysis framework to several other CSI uncertainty models is also discussed.

cs.IT↗

arXiv2016

On the Estimation Performance and Convergence Rate of the Generalized Power Method for Phase Synchronization

An estimation problem of fundamental interest is that of phase synchronization, in which the goal is to recover a collection of phases using noisy measurements of relative phases. It is known that in the Gaussian noise setting, the maximum likelihood estimator (MLE) has an expected squared $\ell_2$-estimation error that is on the same order as the Cramér-Rao lower bound. Moreover, even though the MLE is an optimal solution to a non-convex quadratic optimization problem, it can be found with high probability using semidefinite programming (SDP), provided that the noise power is not too large. In this paper, we study the estimation and convergence performance of a recently-proposed low-complexity alternative to the SDP-based approach, namely, the generalized power method (GPM). Our contribution is twofold. First, we bound the rate at which the estimation error decreases in each iteration of the GPM and use this bound to show that all iterates---not just the MLE---achieve an estimation error that is on the same order as the Cramér-Rao bound. Our result holds under the least restrictive assumption on the noise power and gives the best provable bound on the estimation error known to date. It also implies that one can terminate the GPM at any iteration and still obtain an estimator that has a theoretical guarantee on its estimation error. Second, we show that under the same assumption on the noise power as that for the SDP-based method, the GPM will converge to the MLE at a linear rate with high probability. This answers a question raised in [3] and shows that the GPM is competitive in terms of both theoretical guarantees and numerical efficiency with the SDP-based method. At the heart of our convergence rate analysis is a new error bound for the non-convex quadratic optimization formulation of the phase synchronization problem, which could be of independent interest.

math.OC↗

arXiv2016

A Stochastic Beamformed Amplify-and-Forward Scheme in a Multigroup Multicast MIMO Relay Network with Per-Antenna Power Constraints

In this paper, we consider a two-hop one-way relay network for multigroup multicast transmission between long-distance users, in which the relay is equipped with multiple antennas, while the transmitters and receivers are all with a single antenna. Assuming that perfect channel state information is available, we study amplify-and-forward (AF) schemes that aim at optimizing the max-min-fair (MMF) rate. We begin by considering the classic beamformed AF (BF-AF) scheme, whose corresponding MMF design problem can be formulated as a rank-constrained fractional semidefinite program (SDP). We show that the gap between the BF-AF rate and the SDR rate associated with an optimal SDP solution is sensitive to the number of users as well as the number of power constraints in the relay system. This reveals that the BF-AF scheme may not be well suited for large-scale systems. We therefore propose the stochastic beamformed AF (SBF-AF) schemes, which differ from the BF-AF scheme in that time-varying AF weights are used. We prove that the MMF rates of the proposed SBF-AF schemes are at most $0.8317$ bits/s/Hz less than the SDR rate, irrespective of the number of users or power constraints. Thus, SBF-AF can outperform BF-AF especially in large-scale systems. Finally, we present numerical results to demonstrate the viability of our proposed schemes.

cs.IT↗

arXiv2016

Some Proof Derivations and Further Simulation Results for "Semidefinite Relaxation and Approximation Analysis of a Beamformed Alamouti Scheme for Relay Beamforming Networks"

This is a companion technical report of the main manuscript "Semidefinite Relaxation and Approximation Analysis of a Beamformed Alamouti Scheme for Relay Beamforming Networks". The report serves to give detailed derivations of Lemma 1-2 in the main manuscript, which are too long to be included in the latter. In addition, more simulation results are presented to verify the viability of the BF Alamouti AF schemes developed in the main manuscript.

cs.IT↗

arXiv2016

Semidefinite Relaxation and Approximation Analysis of a Beamformed Alamouti Scheme for Relay Beamforming Networks

In this paper, we study the amplify-and-forward (AF) schemes in two-hop one-way relay networks. In particular, we consider the multigroup multicast transmission between long-distance users. Given that perfect channel state information is perceived, our goal is to design the AF process so that the max-min-fair (MMF) signal-to-interference-plus-noise ratio (SINR) is optimized subject to generalized power constraints. We propose a rank-two beamformed Alamouti (BFA) AF scheme and formulate the corresponding AF design problem as a \emph{two-variable} fractional quadratically-constrained quadratic program (QCQP), which is further tackled by the semidefinite relaxation (SDR) technique. We analyze the approximation quality of two-variable fractional SDRs under the Gaussian randomization algorithm. These results are fundamentally new and reveal that the proposed BFA AF scheme can outperform the traditional BF AF scheme, especially when there are many users in the system or many generalized power constraints in the problem formulation. From a practical perspective, the BFA AF scheme offers two degrees of freedom (DoFs) in beamformer design, as opposed to the one DoF offered by the BF AF scheme, to improve the receivers' SINR. In the latter part of this paper, we demonstrate how this extra DoF leads to provable performance gains by considering two special cases of multicasting, where the AF process is shown to employ a special structure. The numerical simulations further validate that the proposed BFA AF scheme outperforms the BF AF scheme and works well for large-scale relay systems.

cs.IT↗

arXiv2015

A Unified Approach to Error Bounds for Structured Convex Optimization Problems

Error bounds, which refer to inequalities that bound the distance of vectors in a test set to a given set by a residual function, have proven to be extremely useful in analyzing the convergence rates of a host of iterative methods for solving optimization problems. In this paper, we present a new framework for establishing error bounds for a class of structured convex optimization problems, in which the objective function is the sum of a smooth convex function and a general closed proper convex function. Such a class encapsulates not only fairly general constrained minimization problems but also various regularized loss minimization formulations in machine learning, signal processing, and statistics. Using our framework, we show that a number of existing error bound results can be recovered in a unified and transparent manner. To further demonstrate the power of our framework, we apply it to a class of nuclear-norm regularized loss minimization problems and establish a new error bound for this class under a strict complementarity-type regularity condition. We then complement this result by constructing an example to show that the said error bound could fail to hold without the regularity condition. Consequently, we obtain a rather complete answer to a question raised by Tseng. We believe that our approach will find further applications in the study of error bounds for structured convex optimization problems.

math.OC↗

arXiv2015

A Robust Design for MISO Physical-Layer Multicasting over Line-of-Sight Channels

This paper studies a robust design problem for far-field line-of-sight (LOS) channels where phase errors are present. Compared with the commonly used additive error model, the phase error model is more suitable for capturing the uncertainty in an LOS channel, as the dominant source of uncertainty lies in the phase. We consider a multiple-input single-output (MISO) multicast scenario, in which our goal is to design a beamformer that minimizes the transmit power while satisfying probabilistic signal-to-noise ratio (SNR) constraints. The probabilistic constraints give rise to a new computational challenge, as they involve random trigonometric forms. In this work, we propose to first approximate the random trigonometric form by its second-order Taylor expansion and then tackle the resulting random quadratic form using a Bernstein-type inequality. The advantage of such an approach is that an approximately optimal beamformer can be obtained using the standard semidefinite relaxation technique. In the simulations, we first show that if a non-robust design (i.e., one that does not take phase errors into account) is used, then the whole system may collapse. We then show that our proposed method is less conservative than the existing robust design based on Gaussian approximation and thus requires a lower power budget.

cs.IT↗

arXiv2015

Quadratic Optimization with Orthogonality Constraints: Explicit Lojasiewicz Exponent and Linear Convergence of Line-Search Methods

A fundamental class of matrix optimization problems that arise in many areas of science and engineering is that of quadratic optimization with orthogonality constraints. Such problems can be solved using line-search methods on the Stiefel manifold, which are known to converge globally under mild conditions. To determine the convergence rate of these methods, we give an explicit estimate of the exponent in a Lojasiewicz inequality for the (non-convex) set of critical points of the aforementioned class of problems. By combining such an estimate with known arguments, we are able to establish the linear convergence of a large class of line-search methods. A key step in our proof is to establish a local error bound for the set of critical points, which may be of independent interest.

math.OC↗

arXiv2014

A Perturbation Inequality for the Schatten-$p$ Quasi-Norm and Its Applications to Low-Rank Matrix Recovery

In this paper, we establish the following perturbation result concerning the singular values of a matrix: Let $A,B \in \mathbb{R}^{m\times n}$ be given matrices, and let $f:\mathbb{R}_+\rightarrow\mathbb{R}_+$ be a concave function satisfying $f(0)=0$. Then, we have $$ \sum_{i=1}^{\min\{m,n\}} \big| f(σ_i(A)) - f(σ_i(B)) \big| \le \sum_{i=1}^{\min\{m,n\}} f(σ_i(A-B)), $$ where $σ_i(\cdot)$ denotes the $i$--th largest singular value of a matrix. This answers an open question that is of interest to both the compressive sensing and linear algebra communities. In particular, by taking $f(\cdot)=(\cdot)^p$ for any $p \in (0,1]$, we obtain a perturbation inequality for the so--called Schatten $p$--quasi--norm, which allows us to confirm the validity of a number of previously conjectured conditions for the recovery of low--rank matrices via the popular Schatten $p$--quasi--norm heuristic. We believe that our result will find further applications, especially in the study of low--rank matrix recovery.

math.OC↗

arXiv2013

Non-Asymptotic Convergence Analysis of Inexact Gradient Methods for Machine Learning Without Strong Convexity

Many recent applications in machine learning and data fitting call for the algorithmic solution of structured smooth convex optimization problems. Although the gradient descent method is a natural choice for this task, it requires exact gradient computations and hence can be inefficient when the problem size is large or the gradient is difficult to evaluate. Therefore, there has been much interest in inexact gradient methods (IGMs), in which an efficiently computable approximate gradient is used to perform the update in each iteration. Currently, non-asymptotic linear convergence results for IGMs are typically established under the assumption that the objective function is strongly convex, which is not satisfied in many applications of interest; while linear convergence results that do not require the strong convexity assumption are usually asymptotic in nature. In this paper, we combine the best of these two types of results and establish---under the standard assumption that the gradient approximation errors decrease linearly to zero---the non-asymptotic linear convergence of IGMs when applied to a class of structured convex optimization problems. Such a class covers settings where the objective function is not necessarily strongly convex and includes the least squares and logistic regression problems. We believe that our techniques will find further applications in the non-asymptotic convergence analysis of other first-order methods.

math.OC↗

arXiv2013

Physical-Layer Multicasting by Stochastic Transmit Beamforming and Alamouti Space-Time Coding

Consider transceiver designs in a multiuser multi-input single-output (MISO) downlink channel, where the users are to receive the same data stream simultaneously. This problem, known as physical-layer multicasting, has drawn much interest. Presently, a popularized approach is transmit beamforming, in which the beamforming optimization is handled by a rank-one approximation method called semidefinite relaxation (SDR). SDR-based beamforming has been shown to be promising for a small or moderate number of users. This paper describes two new transceiver strategies for physical-layer multicasting. The first strategy, called stochastic beamforming (SBF), randomizes the beamformer in a per-symbol time-varying manner, so that the rank-one approximation in SDR can be bypassed. We propose several efficiently realizable SBF schemes, and prove that their multicast achievable rate gaps with respect to the MISO multicast capacity must be no worse than 0.8314 bits/s/Hz, irrespective of any other factors such as the number of users. The use of channel coding and the assumption of sufficiently long code lengths play a crucial role in achieving the above result. The second strategy combines transmit beamforming and the Alamouti space-time code. The result is a rank-two generalization of SDR-based beamforming. We show by analysis that this SDR-based beamformed Alamouti scheme has a better worst-case effective signal-to-noise ratio (SNR) scaling, and hence a better multicast rate scaling, than SDR-based beamforming. We further the work by combining SBF and the beamformed Alamouti scheme, wherein an improved constant rate gap of 0.39 bits/s/Hz is proven. Simulation results show that under a channel-coded, many-user setting, the proposed multicast transceiver schemes yield significant SNR gains over SDR-based beamforming at the same bit error rate level.

cs.IT↗

arXiv2013

Achievable Rate Derivations and Further Simulation Results for "Physical-Layer Multicasting by Stochastic Transmit Beamforming and Alamouti Space-Time Coding"

This is a companion technical report of the main manuscript "Physical-Layer Multicasting by Stochastic Transmit Beamforming and Alamouti Space-Time Coding". The report serves to give detailed derivations of the achievable rate functions encountered in the main manuscript, which are too long to be included in the latter. In addition, more simulation results are presented to verify the viability of the multicast schemes developed in the main manuscript.

cs.IT↗

arXiv2012

Hardness and Approximation Results for $L_p$-Ball Constrained Homogeneous Polynomial Optimization Problems

In this paper, we establish hardness and approximation results for various $L_p$-ball constrained homogeneous polynomial optimization problems, where $p \in [2,\infty]$. Specifically, we prove that for any given $d \ge 3$ and $p \in [2,\infty]$, both the problem of optimizing a degree-$d$ homogeneous polynomial over the $L_p$-ball and the problem of optimizing a degree-$d$ multilinear form (regardless of its super-symmetry) over $L_p$-balls are NP-hard. On the other hand, we show that these problems can be approximated to within a factor of $Ω((\log n)^{(d-2)/p} \big/ n^{d/2-1})$ in deterministic polynomial time, where $n$ is the number of variables. We further show that with the help of randomization, the approximation guarantee can be improved to $Ω((\log n/n)^{d/2-1})$, which is independent of $p$ and is currently the best for the aforementioned problems. Our results unify and generalize those in the literature, which focus either on the quadratic case or the case where $p \in {2,\infty}$. We believe that the wide array of tools used in this paper will have further applications in the study of polynomial optimization problems.

math.OC↗

arXiv2011

Outage Constrained Robust Transmit Optimization for Multiuser MISO Downlinks: Tractable Approximations by Conic Optimization

In this paper we consider a probabilistic signal-to-interference and-noise ratio (SINR) constrained problem for transmit beamforming design in the presence of imperfect channel state information (CSI), under a multiuser multiple-input single-output (MISO) downlink scenario. In particular, we deal with outage-based quality-of-service constraints, where the probability of each user's SINR not satisfying a service requirement must not fall below a given outage probability specification. The study of solution approaches to the probabilistic SINR constrained problem is important because CSI errors are often present in practical systems and they may cause substantial SINR outages if not handled properly. However, a major technical challenge is how to process the probabilistic SINR constraints. To tackle this, we propose a novel relaxation- restriction (RAR) approach, which consists of two key ingredients-semidefinite relaxation (SDR), and analytic tools for conservatively approximating probabilistic constraints. The underlying goal is to establish approximate probabilistic SINR constrained formulations in the form of convex conic optimization problems, so that they can be readily implemented by available solvers. Using either an intuitive worst-case argument or specialized probabilistic results, we develop various conservative approximation schemes for processing probabilistic constraints with quadratic uncertainties. Consequently, we obtain several RAR alternatives for handling the probabilistic SINR constrained problem. Our techniques apply to both complex Gaussian CSI errors and i.i.d. bounded CSI errors with unknown distribution. Moreover, results obtained from our extensive simulations show that the proposed RAR methods significantly improve upon existing ones, both in terms of solution quality and computational complexity.

cs.IT↗

arXiv2010

Probabilistic Sinr Constrained Robust Transmit Beamforming: A Bernstein-Type Inequality Based Conservative Approach

Recently, robust transmit beamforming has drawn considerable attention because it can provide guaranteed receiver performance in the presence of channel state information (CSI) errors. Assuming complex Gaussian distributed CSI errors, this paper investigates the robust beamforming design problem that minimizes the transmission power subject to probabilistic signal-to-interference-plus-noise ratio (SINR) constraints. The probabilistic SINR constraints in general have no closed-form expression and are difficult to handle. Based on a Bernstein-type inequality of complex Gaussian random variables, we propose a conservative formulation to the robust beamforming design problem. The semidefinite relaxation technique can be applied to efficiently handle the proposed conservative formulation. Simulation results show that, in comparison with the existing methods, the proposed method is more power efficient and is able to support higher target SINR values for receivers.

cs.IT↗

arXiv2010

Optimal Spectrum Sharing in MIMO Cognitive Radio Networks via Semidefinite Programming

In this paper, we study the optimal secondary-link beamforming pattern that balances between the SU's throughput and the interference it causes to PUs in MIMO cognitive radio networks. In particular, we aim to maximize the throughput of the SU, while keeping the interference temperature at the primary receivers below a certain threshold. Unlike traditional MIMO systems, SUs may not have the luxury of knowing the channel state information (CSI) on the links to PUs. This presents a key challenge for a secondary transmitter to steer interference away from primary receivers. In this paper, we consider three scenarios, namely when the secondary transmitter has complete, partial, or no knowledge about the channels to the primary receivers. In particular, when complete CSI is not available, the interference-temperature constraints are to be satisfied with high probability, thus resulting in chance constraints that are typically hard to deal with. Our contribution is fourfold. First, by analyzing the distributional characteristics of MIMO channels, we propose a unified homogeneous QCQP formulation that can be applied to all three scenarios. The homogeneous QCQP formulation, though non-convex, is amenable to semidefinite programming (SDP) relaxation methods. Secondly, we show that the SDP relaxation admits no gap when the number of primary links is no larger than two. Thirdly, we propose a randomized polynomial-time algorithm for constructing a near-optimal solution to the QCQP problem when there are more than two primary links. Finally, we show that when the secondary transmitter has no CSI on the links to primary receivers, the optimal solution to the QCQP problem can be found by a simple matrix eigenvalue-eigenvector computation, which can be done much more efficiently than solving the QCQP directly.

cs.NI↗