arXiv subjects

Xiao Fu

Publications and source records attributed to Xiao Fu.

At least 109 records · Page 6Linked to original sources

arXiv2019

Data-Driven Learning-Based Optimization for Distribution System State Estimation

Distribution system state estimation (DSSE) is a core task for monitoring and control of distribution networks. Widely used algorithms such as Gauss-Netwon perform poorly with the limited number of measurements typically available for DSSE, often require many iterations to obtain reasonable results, and sometimes fail to converge. DSSE is a non-convex problem, and working with a limited number of measurements further aggravate the situation, as indeterminacy induces multiple global (in addition to local) minima. Gauss-Newton is also known to be sensitive to initialization. Hence, the situation is far from ideal. It is therefore natural to ask if there is a smart way of initializing Gauss-Newton that will avoid these DSSE-specific pitfalls. This paper proposes using historical or simulation-derived data to train a shallow neural network to `learn to initialize' -- that is, map the available measurements to a point in the neighborhood of the true latent states (network voltages), which is used to initialize Gauss-Newton. It is shown that this hybrid machine learning / optimization approach yields superior performance in terms of stability, accuracy, and runtime efficiency, compared to conventional optimization-only approaches. It is also shown that judicious design of the neural network training cost function helps to improve the overall DSSE performance.

eess.SP↗

arXiv2019

Tensor Completion from Regular Sub-Nyquist Samples

Signal sampling and reconstruction is a fundamental engineering task at the heart of signal processing. The celebrated Shannon-Nyquist theorem guarantees perfect signal reconstruction from uniform samples, obtained at a rate twice the maximum frequency present in the signal. Unfortunately a large number of signals of interest are far from being band-limited. This motivated research on reconstruction from sub-Nyquist samples, which mainly hinges on the use of random / incoherent sampling procedures. However, uniform or regular sampling is more appealing in practice and from the system design point of view, as it is far simpler to implement, and often necessary due to system constraints. In this work, we study regular sampling and reconstruction of three- or higher-dimensional signals (tensors). We show that reconstructing a tensor signal from regular samples is feasible. Under the proposed framework, the sample complexity is determined by the tensor rank---rather than the signal bandwidth. This result offers new perspectives for designing practical regular sampling patterns and systems for signals that are naturally tensors, e.g., images and video. For a concrete application, we show that functional magnetic resonance imaging (fMRI) acceleration is a tensor sampling problem, and design practical sampling schemes and an algorithmic framework to handle it. Numerical results show that our tensor sampling strategy accelerates the fMRI sampling process significantly without sacrificing reconstruction accuracy.

eess.SP↗

arXiv2019

Multiuser Video Streaming Rate Adaptation: A Physical Layer Resource-Aware Deep Reinforcement Learning Approach

We consider a multi-user video streaming service optimization problem over a time-varying and mutually interfering multi-cell wireless network. The key research challenge is to appropriately adapt each user's video streaming rate according to the radio frequency environment (e.g., channel fading and interference level) and service demands (e.g., play request), so that the users' long-term experience for watching videos can be optimized. To address the above challenge, we propose a novel two-level cross-layer optimization framework for multiuser adaptive video streaming over wireless networks. The key idea is to jointly design the physical layer optimization-based beamforming scheme (performed at the base stations) and the application layer Deep Reinforcement Learning (DRL)-based scheme (performed at the user terminals), so that a highly complex multi-user, cross-layer, time-varying video streaming problem can be decomposed into relatively simple problems and solved effectively. Our strategy represents a significant departure for the existing schemes where either short-term user experience optimization is considered, or only single-user point-to-point long-term optimization is considered. Extensive simulations based on real-data sets show that the proposed cross-layer design is effective and promising.

cs.MM↗

arXiv2019

Learning Nonlinear Mixtures: Identifiability and Algorithm

Linear mixture models have proven very useful in a plethora of applications, e.g., topic modeling, clustering, and source separation. As a critical aspect of the linear mixture models, identifiability of the model parameters is well-studied, under frameworks such as independent component analysis and constrained matrix factorization. Nevertheless, when the linear mixtures are distorted by an unknown nonlinear functions -- which is well-motivated and more realistic in many cases -- the identifiability issues are much less studied. This work proposes an identification criterion for a nonlinear mixture model that is well grounded in many real-world applications, and offers identifiability guarantees. A practical implementation based on a judiciously designed neural network is proposed to realize the criterion, and an effective learning algorithm is proposed. Numerical results on synthetic and real-data corroborate effectiveness of the proposed method.

cs.LG↗

arXiv2018

Nonnegative Matrix Factorization for Signal and Data Analytics: Identifiability, Algorithms, and Applications

Nonnegative matrix factorization (NMF) has become a workhorse for signal and data analytics, triggered by its model parsimony and interpretability. Perhaps a bit surprisingly, the understanding to its model identifiability---the major reason behind the interpretability in many applications such as topic mining and hyperspectral imaging---had been rather limited until recent years. Beginning from the 2010s, the identifiability research of NMF has progressed considerably: Many interesting and important results have been discovered by the signal processing (SP) and machine learning (ML) communities. NMF identifiability has a great impact on many aspects in practice, such as ill-posed formulation avoidance and performance-guaranteed algorithm design. On the other hand, there is no tutorial paper that introduces NMF from an identifiability viewpoint. In this paper, we aim at filling this gap by offering a comprehensive and deep tutorial on model identifiability of NMF as well as the connections to algorithms and applications. This tutorial will help researchers and graduate students grasp the essence and insights of NMF, thereby avoiding typical `pitfalls' that are often times due to unidentifiable NMF formulations. This paper will also help practitioners pick/design suitable factorization tools for their own problems.

eess.SP↗

arXiv2018

Tensor-Based Channel Estimation for Dual-Polarized Massive MIMO Systems

The 3GPP suggests to combine dual polarized (DP) antenna arrays with the double directional (DD) channel model for downlink channel estimation. This combination strikes a good balance between high-capacity communications and parsimonious channel modeling, and also brings limited feedback schemes for downlink channel state information within reach---since such channel can be fully characterized by several key parameters. However, most existing channel estimation work under the DD model has not yet considered DP arrays, perhaps because of the complex array manifold and the resulting difficulty in algorithm design. In this paper, we first reveal that the DD channel with DP arrays at the transmitter and receiver can be naturally modeled as a low-rank tensor, and thus the key parameters of the channel can be effectively estimated via tensor decomposition algorithms. On the theory side, we show that the DD-DP parameters are identifiable under very mild conditions, by leveraging identifiability of low-rank tensors. Furthermore, a compressed tensor decomposition algorithm is developed for alleviating the downlink training overhead. We show that, by using judiciously designed pilot structure, the channel parameters are still guaranteed to be identified via the compressed tensor decomposition formulation even when the size of the pilot sequence is much smaller than what is needed for conventional channel identification methods, such as linear least squares and matched filtering. Numerical simulations are presented to showcase the effectiveness of the proposed methods.

eess.SP↗

arXiv2018

Limited Feedback Channel Estimation in Massive MIMO with Non-uniform Directional Dictionaries

Channel state information (CSI) at the base station (BS) is crucial to achieve beamforming and multiplexing gains in multiple-input multiple-output (MIMO) systems. State-of-the-art limited feedback schemes require feedback overhead that scales linearly with the number of BS antennas, which is prohibitive for $5$G massive MIMO. This work proposes novel limited feedback algorithms that lift this burden by exploiting the inherent sparsity in double directional (DD) MIMO channel representation using overcomplete dictionaries. These dictionaries are associated with angle of arrival (AoA) and angle of departure (AoD) that specifically account for antenna directivity patterns at both ends of the link. The proposed algorithms achieve satisfactory channel estimation accuracy using a small number of feedback bits, even when the number of transmit antennas at the BS is large -- making them ideal for $5$G massive MIMO. Judicious simulations reveal that they outperform a number of popular feedback schemes, and underscore the importance of using angle dictionaries matching the given antenna directivity patterns, as opposed to uniform dictionaries. The proposed algorithms are lightweight in terms of computation, especially on the user equipment side, making them ideal for actual deployment in $5$G systems.

cs.IT↗

arXiv2018

Tensors, Learning, and 'Kolmogorov Extension' for Finite-alphabet Random Vectors

Estimating the joint probability mass function (PMF) of a set of random variables lies at the heart of statistical learning and signal processing. Without structural assumptions, such as modeling the variables as a Markov chain, tree, or other graphical model, joint PMF estimation is often considered mission impossible - the number of unknowns grows exponentially with the number of variables. But who gives us the structural model? Is there a generic, `non-parametric' way to control joint PMF complexity without relying on a priori structural assumptions regarding the underlying probability model? Is it possible to discover the operational structure without biasing the analysis up front? What if we only observe random subsets of the variables, can we still reliably estimate the joint PMF of all? This paper shows, perhaps surprisingly, that if the joint PMF of any three variables can be estimated, then the joint PMF of all the variables can be provably recovered under relatively mild conditions. The result is reminiscent of Kolmogorov's extension theorem - consistent specification of lower-dimensional distributions induces a unique probability measure for the entire process. The difference is that for processes of limited complexity (rank of the high-dimensional PMF) it is possible to obtain complete characterization from only three-dimensional distributions. In fact not all three-dimensional PMFs are needed; and under more stringent conditions even two-dimensional will do. Exploiting multilinear algebra, this paper proves that such higher-dimensional PMF completion can be guaranteed - several pertinent identifiability results are derived. It also provides a practical and efficient algorithm to carry out the recovery task. Judiciously designed simulations and real-data experiments on movie recommendation and data classification are presented to showcase the effectiveness of the approach.

eess.SP↗

arXiv2018

Learning Hidden Markov Models from Pairwise Co-occurrences with Application to Topic Modeling

We present a new algorithm for identifying the transition and emission probabilities of a hidden Markov model (HMM) from the emitted data. Expectation-maximization becomes computationally prohibitive for long observation records, which are often required for identification. The new algorithm is particularly suitable for cases where the available sample size is large enough to accurately estimate second-order output probabilities, but not higher-order ones. We show that if one is only able to obtain a reliable estimate of the pairwise co-occurrence probabilities of the emissions, it is still possible to uniquely identify the HMM if the emission probability is \emph{sufficiently scattered}. We apply our method to hidden topic Markov modeling, and demonstrate that we can learn topics with higher quality if documents are modeled as observations of HMMs sharing the same emission (topic) probability, compared to the simple but widely used bag-of-words model.

cs.CL↗

arXiv2018

Structured SUMCOR Multiview Canonical Correlation Analysis for Large-Scale Data

The sum-of-correlations (SUMCOR) formulation of generalized canonical correlation analysis (GCCA) seeks highly correlated low-dimensional representations of different views via maximizing pairwise latent similarity of the views. SUMCOR is considered arguably the most natural extension of classical two-view CCA to the multiview case, and thus has numerous applications in signal processing and data analytics. Recent work has proposed effective algorithms for handling the SUMCOR problem at very large scale. However, the existing scalable algorithms cannot incorporate structural regularization and prior information -- which are critical for good performance in real-world applications. In this work, we propose a new computational framework for large-scale SUMCOR GCCA that can easily incorporate a suite of structural regularizers which are frequently used in data analytics. The updates of the proposed algorithm are lightweight and the memory complexity is also low. In addition, the proposed algorithm can be readily implemented in a parallel fashion. We show that the proposed algorithm converges to a Karush-Kuhn-Tucker (KKT) point of the regularized SUMCOR problem. Judiciously designed simulations and real-data experiments are employed to demonstrate the effectiveness of the proposed algorithm.

cs.LG↗

arXiv2018

Hyperspectral Super-Resolution: A Coupled Tensor Factorization Approach

Hyperspectral super-resolution refers to the problem of fusing a hyperspectral image (HSI) and a multispectral image (MSI) to produce a super-resolution image (SRI) that has fine spatial and spectral resolution. State-of-the-art methods approach the problem via low-rank matrix approximations to the matricized HSI and MSI. These methods are effective to some extent, but a number of challenges remain. First, HSIs and MSIs are naturally third-order tensors (data "cubes") and thus matricization is prone to loss of structural information--which could degrade performance. Second, it is unclear whether or not these low-rank matrix-based fusion strategies can guarantee identifiability or exact recovery of the SRI. However, identifiability plays a pivotal role in estimation problems and usually has a significant impact on performance in practice. Third, the majority of the existing methods assume that there are known (or easily estimated) degradation operators applied to the SRI to form the corresponding HSI and MSI--which is hardly the case in practice. In this work, we propose to tackle the super-resolution problem from a tensor perspective. Specifically, we utilize the multidimensional structure of the HSI and MSI to propose a coupled tensor factorization framework that can effectively overcome the aforementioned issues. The proposed approach guarantees the identifiability of the SRI under mild and realistic conditions. Furthermore, it works with little knowledge of the degradation operators, which is clearly an advantage over the existing methods. Semi-real numerical experiments are included to show the effectiveness of the proposed approach.

eess.SP↗

arXiv2018

Timing Channel in IaaS: How to Identify and Investigate

Recently, the IaaS (Infrastructure as a Service) Cloud (e.g., Amazon EC2) has been widely used by many organizations. However, some IaaS security issues create serious threats to its users. A typical issue is the timing channel. This kind of channel can be a cross-VM information channel, as proven by many researchers. Because it is covert and traceless, the traditional identification methods cannot build an accurate analysis model and obtain a compromised result. We investigated the underlying behavior of the timing channel from the perspective of the memory activity records and summarized the signature of the timing channel in the underlying memory activities. An identification method based on long-term behavior signatures was proposed. We proposed a complete set of forensics steps including evidence extraction, identification, record reserve, and evidence reports. We studied four typical timing channels, and the experiments showed that these channels can be detected and investigated, even with disturbances from normal processes.

cs.CR↗

arXiv2018

Tensor-Based Parameter Estimation of Double Directional Massive MIMO Channel with Dual-Polarized Antennas

The 3GPP suggests to combine dual polarized (DP) antenna arrays with the double directional (DD) channel model for downlink channel estimation. This combination strikes a good balance between high-capacity communications and parsimonious channel modeling, and also brings limited feedback schemes for downlink channel estimation within reach. However, most existing channel estimation work under the DD model has not considered DP arrays, perhaps because of the complex array manifold and the resulting difficulty in algorithm design. In this paper, we first reveal that the DD channel with DP arrays at the transmitter and receiver can be naturally modeled as a low-rank four-way tensor, and thus the parameters can be effectively estimated via tensor decomposition algorithms. To reduce computational complexity, we show that the problem can be recast as a four-snapshot three-dimensional harmonic retrieval problem, which can be solved using computationally efficient subspace methods. On the theory side, we show that the DD channel with DP arrays is identifiable under very mild conditions, leveraging identifiability of low-rank tensors. Numerical simulations are employed to showcase the effectiveness of our methods.

eess.SP↗

arXiv2017

Penalty Dual Decomposition Method For Nonsmooth Nonconvex Optimization

Many contemporary signal processing, machine learning and wireless communication applications can be formulated as nonconvex nonsmooth optimization problems. Often there is a lack of efficient algorithms for these problems, especially when the optimization variables are nonlinearly coupled in some nonconvex constraints. In this work, we propose an algorithm named penalty dual decomposition (PDD) for these difficult problems and discuss its various applications. The PDD is a double-loop iterative algorithm. Its inner iterations is used to inexactly solve a nonconvex nonsmooth augmented Lagrangian problem via block-coordinate-descenttype methods, while its outer iteration updates the dual variables and/or a penalty parameter. In Part I of this work, we describe the PDD algorithm and rigorously establish its convergence to KKT solutions. In Part II we evaluate the performance of PDD by customizing it to three applications arising from signal processing and wireless communications.

cs.IT↗

arXiv2017

On Convergence of Epanechnikov Mean Shift

Epanechnikov Mean Shift is a simple yet empirically very effective algorithm for clustering. It localizes the centroids of data clusters via estimating modes of the probability distribution that generates the data points, using the `optimal' Epanechnikov kernel density estimator. However, since the procedure involves non-smooth kernel density functions, the convergence behavior of Epanechnikov mean shift lacks theoretical support as of this writing---most of the existing analyses are based on smooth functions and thus cannot be applied to Epanechnikov Mean Shift. In this work, we first show that the original Epanechnikov Mean Shift may indeed terminate at a non-critical point, due to the non-smoothness nature. Based on our analysis, we propose a simple remedy to fix it. The modified Epanechnikov Mean Shift is guaranteed to terminate at a local maximum of the estimated density, which corresponds to a cluster centroid, within a finite number of iterations. We also propose a way to avoid running the Mean Shift iterates from every data point, while maintaining good clustering accuracies under non-overlapping spherical Gaussian mixture models. This further pushes Epanechnikov Mean Shift to handle very large and high-dimensional data sets. Experiments show surprisingly good performance compared to the Lloyd's K-means algorithm and the EM algorithm.

stat.ML↗

arXiv2017

Learning to Optimize: Training Deep Neural Networks for Wireless Resource Management

For the past couple of decades, numerical optimization has played a central role in addressing wireless resource management problems such as power control and beamformer design. However, optimization algorithms often entail considerable complexity, which creates a serious gap between theoretical design/analysis and real-time processing. To address this challenge, we propose a new learning-based approach. The key idea is to treat the input and output of a resource allocation algorithm as an unknown non-linear mapping and use a deep neural network (DNN) to approximate it. If the non-linear mapping can be learned accurately by a DNN of moderate size, then resource allocation can be done in almost real time -- since passing the input through a DNN only requires a small number of simple operations. In this work, we address both the thereotical and practical aspects of DNN-based algorithm approximation with applications to wireless resource management. We first pin down a class of optimization algorithms that are `learnable' in theory by a fully connected DNN. Then, we focus on DNN-based approximation to a popular power allocation algorithm named WMMSE (Shi {\it et al} 2011). We show that using a DNN to approximate WMMSE can be fairly accurate -- the approximation error $ε$ depends mildly [in the order of $\log(1/ε)$] on the numbers of neurons and layers of the DNN. On the implementation side, we use extensive numerical simulations to demonstrate that DNNs can achieve orders of magnitude speedup in computational time compared to state-of-the-art power allocation algorithms based on optimization.

cs.IT↗

arXiv2017

On Identifiability of Nonnegative Matrix Factorization

In this letter, we propose a new identification criterion that guarantees the recovery of the low-rank latent factors in the nonnegative matrix factorization (NMF) model, under mild conditions. Specifically, using the proposed criterion, it suffices to identify the latent factors if the rows of one factor are \emph{sufficiently scattered} over the nonnegative orthant, while no structural assumption is imposed on the other factor except being full-rank. This is by far the mildest condition under which the latent factors are provably identifiable from the NMF model.

cs.LG↗

arXiv2017

Inexact Alternating Optimization for Phase Retrieval In the Presence of Outliers

Phase retrieval has been mainly considered in the presence of Gaussian noise. However, the performance of the algorithms proposed under the Gaussian noise model severely degrades when grossly corrupted data, i.e., outliers, exist. This paper investigates techniques for phase retrieval in the presence of heavy-tailed noise -- which is considered a better model for situations where outliers exist. An $\ell_p$-norm ($0<p<2$) based estimator is proposed for fending against such noise, and two-block inexact alternating optimization is proposed as the algorithmic framework to tackle the resulting optimization problem. Two specific algorithms are devised by exploring different local approximations within this framework. Interestingly, the core conditional minimization steps can be interpreted as iteratively reweighted least squares and gradient descent. Convergence properties of the algorithms are discussed, and the Cramér-Rao bound (CRB) is derived. Simulations demonstrate that the proposed algorithms approach the CRB and outperform state-of-the-art algorithms in heavy-tailed noise.

cs.IT↗