arXiv subjects

Lars K. Rasmussen

Publications and source records attributed to Lars K. Rasmussen.

At least 19 recordsLinked to original sources

arXiv2020

$α$ Belief Propagation for Approximate Inference

Belief propagation (BP) algorithm is a widely used message-passing method for inference in graphical models. BP on loop-free graphs converges in linear time. But for graphs with loops, BP's performance is uncertain, and the understanding of its solution is limited. To gain a better understanding of BP in general graphs, we derive an interpretable belief propagation algorithm that is motivated by minimization of a localized $α$-divergence. We term this algorithm as $α$ belief propagation ($α$-BP). It turns out that $α$-BP generalizes standard BP. In addition, this work studies the convergence properties of $α$-BP. We prove and offer the convergence conditions for $α$-BP. Experimental simulations on random graphs validate our theoretical results. The application of $α$-BP to practical problems is also demonstrated.

stat.ML↗

arXiv2020

Region-based Energy Neural Network for Approximate Inference

Region-based free energy was originally proposed for generalized belief propagation (GBP) to improve loopy belief propagation (loopy BP). In this paper, we propose a neural network based energy model for inference in general Markov random fields (MRFs), which directly minimizes the region-based free energy defined on region graphs. We term our model Region-based Energy Neural Network (RENN). Unlike message-passing algorithms, RENN avoids iterative message propagation and is faster. Also different from recent deep neural network based models, inference by RENN does not require sampling, and RENN works on general MRFs. RENN can also be employed for MRF learning. Our experiments on marginal distribution estimation, partition function estimation, and learning of MRFs show that RENN outperforms the mean field method, loopy BP, GBP, and the state-of-the-art neural network based model.

cs.LG↗

arXiv2020

Neural Network based Explicit Mixture Models and Expectation-maximization based Learning

We propose two neural network based mixture models in this article. The proposed mixture models are explicit in nature. The explicit models have analytical forms with the advantages of computing likelihood and efficiency of generating samples. Computation of likelihood is an important aspect of our models. Expectation-maximization based algorithms are developed for learning parameters of the proposed models. We provide sufficient conditions to realize the expectation-maximization based learning. The main requirements are invertibility of neural networks that are used as generators and Jacobian computation of functional form of the neural networks. The requirements are practically realized using a flow-based neural network. In our first mixture model, we use multiple flow-based neural networks as generators. Naturally the model is complex. A single latent variable is used as the common input to all the neural networks. The second mixture model uses a single flow-based neural network as a generator to reduce complexity. The single generator has a latent variable input that follows a Gaussian mixture distribution. We demonstrate efficiency of proposed mixture models through extensive experiments for generating samples and maximum likelihood based classification.

cs.LG↗

arXiv2020

Powering Hidden Markov Model by Neural Network based Generative Models

Hidden Markov model (HMM) has been successfully used for sequential data modeling problems. In this work, we propose to power the modeling capacity of HMM by bringing in neural network based generative models. The proposed model is termed as GenHMM. In the proposed GenHMM, each HMM hidden state is associated with a neural network based generative model that has tractability of exact likelihood and provides efficient likelihood computation. A generative model in GenHMM consists of mixture of generators that are realized by flow models. A learning algorithm for GenHMM is proposed in expectation-maximization framework. The convergence of the learning GenHMM is analyzed. We demonstrate the efficiency of GenHMM by classification tasks on practical sequential data. Code available at https://github.com/FirstHandScientist/genhmm.

cs.LG↗

arXiv2020

On Dominant Interference in Random Networks and Communication Reliability

In this paper, we study the characteristics of dominant interference power with directional reception in a random network modelled by a Poisson Point Process. Additionally, the Laplace functional of cumulative interference excluding the $n$ dominant interferers is also derived, which turns out to be a generalization of omni-directional reception and complete accumulative interference. As an application of these results, we study the impact of directional receivers in random networks in terms of outage probability and error probability with queue length constraint.

cs.IT↗

arXiv2020

Will Scale-free Popularity Develop Scale-free Geo-social Networks?

Empirical results show that spatial factors such as distance, population density and communication range affect our social activities, also reflected by the development of ties in social networks. This motivates the need for social network models that take these spatial factors into account. Therefore, in this paper we propose a gravity-low-based geo-social network model, where connections develop according to the popularity of the individuals, but are constrained through their geographic distance and the surrounding population density. Specifically, we consider a power-law distributed popularity, and random node positions governed by a Poisson point process. We evaluate the characteristics of the emerging networks, considering the degree distribution, the average degree of neighbors and the local clustering coefficient. These local metrics reflect the robustness of the network, the information dissemination speed and the communication locality. We show that unless the communication range is strictly limited, the emerging networks are scale-free, with a rank exponent affected by the spatial factors. Even the average neighbor degree and the local clustering coefficient show tendencies known in non-geographic scale-free networks, at least when considering individuals with low popularity. At high-popularity values, however, the spatial constraints lead to popularity-independent average neighbor degrees and clustering coefficients.

physics.soc-ph↗

arXiv2019

$α$ Belief Propagation as Fully Factorized Approximation

Belief propagation (BP) can do exact inference in loop-free graphs, but its performance could be poor in graphs with loops, and the understanding of its solution is limited. This work gives an interpretable belief propagation rule that is actually minimization of a localized $α$-divergence. We term this algorithm as $α$ belief propagation ($α$-BP). The performance of $α$-BP is tested in MAP (maximum a posterior) inference problems, where $α$-BP can outperform (loopy) BP by a significant margin even in fully-connected graphs.

cs.LG↗

arXiv2018

Entropy-regularized Optimal Transport Generative Models

We investigate the use of entropy-regularized optimal transport (EOT) cost in developing generative models to learn implicit distributions. Two generative models are proposed. One uses EOT cost directly in an one-shot optimization problem and the other uses EOT cost iteratively in an adversarial game. The proposed generative models show improved performance over contemporary models for image generation on MNSIT.

cs.LG↗

arXiv2018

Locally Convex Sparse Learning over Networks

We consider a distributed learning setup where a sparse signal is estimated over a network. Our main interest is to save communication resource for information exchange over the network and reduce processing time. Each node of the network uses a convex optimization based algorithm that provides a locally optimum solution for that node. The nodes exchange their signal estimates over the network in order to refine their local estimates. At a node, the optimization algorithm is based on an $\ell_1$-norm minimization with appropriate modifications to promote sparsity as well as to include influence of estimates from neighboring nodes. Our expectation is that local estimates in each node improve fast and converge, resulting in a limited demand for communication of estimates between nodes and reducing the processing time. We provide restricted-isometry-property (RIP)-based theoretical analysis on estimation quality. In the scenario of clean observation, it is shown that the local estimates converge to the exact sparse signal under certain technical conditions. Simulation results show that the proposed algorithms show competitive performance compared to a globally optimum distributed LASSO algorithm in the sense of convergence speed and estimation error.

stat.ML↗

arXiv2018

Golden Angle Modulation: Approaching the AWGN Capacity

In this work, targeting, e.g., future generation cellular, microwave-links, or optical fiber systems, we propose a new geometric shaping design for golden angle modulation (GAM) based on a (double) truncated Gaussian input distribution. The design improves the mutual information (MI), and the peak-to-average power ratio, over the full signal-to-noise ratio (SNR) range relative to two key GAM schemes introduced in [1],[2]. Inspired by the proposed geometric shaping, a simpler, SNR-dependent, design is also suggested. The performance is numerically evaluated with respect to MI and compared with classical modulation schemes. With the proposed design, the SNR can be decreased relative to classical quadrature amplitude modulation, even for relatively modest target spectral efficiencies. As the GAM design can approach the Gaussian channel capacity, the power/energy efficiency is expected to improve.

cs.IT↗

arXiv2017

Estimate Exchange over Network is Good for Distributed Hard Thresholding Pursuit

We investigate an existing distributed algorithm for learning sparse signals or data over networks. The algorithm is iterative and exchanges intermediate estimates of a sparse signal over a network. This learning strategy using exchange of intermediate estimates over the network requires a limited communication overhead for information transmission. Our objective in this article is to show that the strategy is good for learning in spite of limited communication. In pursuit of this objective, we first provide a restricted isometry property (RIP)-based theoretical analysis on convergence of the iterative algorithm. Then, using simulations, we show that the algorithm provides competitive performance in learning sparse signals vis-a-vis an existing alternate distributed algorithm. The alternate distributed algorithm exchanges more information including observations and system parameters.

stat.ML↗

arXiv2017

The Golden Quantizer: The Complex Gaussian Random Variable Case

The problem of quantizing a circularly-symmetric complex Gaussian random variable is considered. For this purpose, we design two non-uniform quantizers, a high-rate-, and a Lloyd-Max-, quantizer that are both based on the (golden angle) spiral-phyllotaxis packing principle. We find that the proposed schemes have lower mean-square error distortion compared to (non)-uniform polar/rectangular-quantizers, and near-identical to the best performing trained vector quantizers. The proposed quantizer scheme offers a structured design, a simple natural index ordering, and allow for any number of centroids.

cs.IT↗

arXiv2016

The Matrix Exponential Distribution - A Tool for Wireless System Performance Analysis

In [1], we introduced a new, matrix algebraic, performance analysis framework for wireless systems with fading channels based on the matrix exponential distribution. The main idea was to use the compact, powerful, and easy-to-use, matrix exponential (ME)-distribution for i) modeling the unprocessed channel signal to noise ratio (SNR), ii) exploiting the closure property of the ME-distribution for SNR processing operations to give the effective channel random variable (r.v.) on ME-distribution form, and then to iii) express the performance measure in a closed-form based on ME-distribution matrix/vector parameters only. In this work, we aim to more clearly present, formalize, refine and develop this unified bottom-up analysis framework, show its versatility to handle important communication cases, performance evaluation levels, and performance metrics. The bivariate ME-distribution is introduced here as yet another useful ME-tool, e.g. to account for dependency among two r.v.s. We propose that the ME-distribution may, in addition to fading, also characterize the pdf of discrete-time signal r.v.s, thus extending the ME-distribution matrix form to new generalized 1D/2D-Gaussian-, and Rayleigh-, distribution-like matrix forms. Our findings here, strengthen the observation from [1], [2], and indicates that the ME-distribution can be a promising tool for wireless system modeling and performance analysis.

cs.IT↗

arXiv2016

Effective Capacity of Retransmission Schemes - A Recurrence Relation Approach

We consider the effective capacity performance measure of persistent- and truncated-retransmission schemes that can involve any combination of multiple transmissions per packet, multiple communication modes, or multiple packet communication. We present a structured unified analytical approach, based on a random walk model and recurrence relation formulation, and give exact effective capacity expressions for persistent hybrid automatic repeat request (HARQ) and for truncated-retransmission schemes. For the latter, effective capacity expressions are given for systems with finite (infinite) time horizon on an algebraic (spectral radius-based) form of a special block companion matrix. In contrast to prior HARQ models, assuming infinite time horizon, the proposed method does not involve a non-trivial per case modeling step. We give effective capacity expressions for several important cases that have not been addressed before, e.g. persistent-HARQ, truncated-HARQ, network-coded ARQ (NC-ARQ), two-mode-ARQ, and multilayer-ARQ. We propose an alternative QoS parameter (instead of the commonly used moment generating function parameter) that represents explicitly the target delay and the delay violation probability. This also enables closed-form expressions for many of the studied systems. Moreover, we use the recently proposed matrix-exponential distributed (MED) modeling of wireless fading channels to provide the basis for numerous new effective capacity results for HARQ.

cs.IT↗

arXiv2016

Asymptotic Performance Analysis of a K-Hop Amplify-and-Forward Relay MIMO Channel

The present paper studies the asymptotic performance of multi-hop amplify-and-forward relay multiple-antenna communication channels. Each multi-antenna terminal in the network amplifies the received signal, sent by a source, and retransmits it upstream towards a destination. Achievable ergodic rates under both jointly optimal detection and decoding and practical separate decoding schemes for arbitrary signaling schemes, along with the average bit error rate for various receiver structures are derived in the regime where the number of antennas at each terminal grows large without a bound. To overcome the difficulty of averaging over channel realizations we apply large-system analysis based on the replica method from statistical physics. The validity of the large-system analysis is further verified through Monte Carlo simulations of realistic finite-sized systems.

cs.IT↗

arXiv2015

Cooperative Communication Using Network Coding

We consider a cognitive radio network scenario where a primary transmitter and a secondary transmitter, respectively, communicate a message to their respective primary receiver and secondary receiver over a packet-based wireless link, using a joint automatic-repeat-request (ARQ) error control scheme. The secondary transmitter assists in the retransmission of the primary message, which improves the primary performance, and is granted limited access to the transmission resources. Conventional ARQ, as well as two network-coding schemes are investigated for application in the retransmission phase; namely the static network-coding (SNC) scheme and the adaptive network-coding (ANC) scheme. For each scheme we analyze the transmission process by investigating the distribution of the number of transmission attempts and approximate it by normal distributions. Considering both the cases of an adaptive frame size and a truncated frame size, we derive analytical results on packet throughput and infer that the ANC scheme outperforms the SNC scheme.

cs.IT↗

arXiv2014

On the Optimal Precoding for MIMO Gaussian Wire-Tap Channels

We consider the problem of finding secrecy rate of a multiple-input multiple-output (MIMO) wire-tap channel. A transmitter, a legitimate receiver, and an eavesdropper are all equipped with multiple antennas. The channel states from the transmitter to the legitimate user and to the eavesdropper are assumed to be known at the transmitter. In this contribution, we address the problem of finding the optimal precoder/transmit covariance matrix maximizing the secrecy rate of the given wiretap channel. The problem formulation is shown to be equivalent to a difference of convex functions programming problem and an efficient algorithm for addressing this problem is developed.

cs.IT↗

arXiv2014

On the Transmit Beamforming for MIMO Wiretap Channels: Large-System Analysis

With the growth of wireless networks, security has become a fundamental issue in wireless communications due to the broadcast nature of these networks. In this work, we consider MIMO wiretap channels in a fast fading environment, for which the overall performance is characterized by the ergodic MIMO secrecy rate. Unfortunately, the direct solution to finding ergodic secrecy rates is prohibitive due to the expectations in the rates expressions in this setting. To overcome this difficulty, we invoke the large-system assumption, which allows a deterministic approximation to the ergodic mutual information. Leveraging results from random matrix theory, we are able to characterize the achievable ergodic secrecy rates. Based on this characterization, we address the problem of covariance optimization at the transmitter. Our numerical results demonstrate a good match between the large-system approximation and the actual simulated secrecy rates, as well as some interesting features of the precoder optimization.

cs.IT↗