arXiv subjects

Mikael Skoglund

Publications and source records attributed to Mikael Skoglund.

At least 181 records · Page 10Linked to original sources

arXiv2020

The Convex Information Bottleneck Lagrangian

The information bottleneck (IB) problem tackles the issue of obtaining relevant compressed representations $T$ of some random variable $X$ for the task of predicting $Y$. It is defined as a constrained optimization problem which maximizes the information the representation has about the task, $I(T;Y)$, while ensuring that a certain level of compression $r$ is achieved (i.e., $ I(X;T) \leq r$). For practical reasons, the problem is usually solved by maximizing the IB Lagrangian (i.e., $\mathcal{L}_{\text{IB}}(T;β) = I(T;Y) - βI(X;T)$) for many values of $β\in [0,1]$. Then, the curve of maximal $I(T;Y)$ for a given $I(X;T)$ is drawn and a representation with the desired predictability and compression is selected. It is known when $Y$ is a deterministic function of $X$, the IB curve cannot be explored and another Lagrangian has been proposed to tackle this problem: the squared IB Lagrangian: $\mathcal{L}_{\text{sq-IB}}(T;β_{\text{sq}})=I(T;Y)-β_{\text{sq}}I(X;T)^2$. In this paper, we (i) present a general family of Lagrangians which allow for the exploration of the IB curve in all scenarios; (ii) provide the exact one-to-one mapping between the Lagrange multiplier and the desired compression rate $r$ for known IB curve shapes; and (iii) show we can approximately obtain a specific compression level with the convex IB Lagrangian for both known and unknown IB curve shapes. This eliminates the burden of solving the optimization problem for many values of the Lagrange multiplier. That is, we prove that we can solve the original constrained problem with a single optimization.

stat.ML↗

arXiv2019

Hidden Markov Models for sepsis detection in preterm infants

We explore the use of traditional and contemporary hidden Markov models (HMMs) for sequential physiological data analysis and sepsis prediction in preterm infants. We investigate the use of classical Gaussian mixture model based HMM, and a recently proposed neural network based HMM. To improve the neural network based HMM, we propose a discriminative training approach. Experimental results show the potential of HMMs over logistic regression, support vector machine and extreme learning machine.

cs.LG↗

arXiv2019

Fixed-Length Strong Coordination

We consider the problem of synthesizing joint distributions of signals and actions over noisy channels in the finite-length regime. For a fixed blocklength $n$ and an upper bound on the distance $\varepsilon$, a coding scheme is proposed such that the induced joint distribution is $\varepsilon$-close in $L^1$ distance to a target i.i.d. distribution. The set of achievable target distributions and rate for asymptotic strong coordination can be recovered from the main result of this paper by having $n$ that tends to infinity.

cs.IT↗

arXiv2019

Low-Latency Communication with Computational Complexity Constraints

Low-latency communication is one of the most important application scenarios in next-generation wireless networks. Often in communication-theoretic studies latency is defined as the time required for the transmission of a packet over a channel. However, with very stringent latency requirements and complexity constrained receivers, the time required for the decoding of the packet cannot be ignored and must be included in the total latency analysis through accurate modeling. In this paper, we first present a way to calculate decoding time using \textit{per bit} complexity metric and introduce an empirical model that accurately describes the trade-off between the decoding complexity versus the performance of state-of-the-art codes. By considering various communication parameters, we show that including the decoding time in latency analyses has a significant effect on the optimum selection of parameters.

eess.SP↗

arXiv2019

Two-Way Coding and Attack Decoupling in Control Systems Under Injection Attacks

In this paper, we introduce the concept of two-way coding, which originates in communication theory characterizing coding schemes for two-way channels, into control theory, particularly to facilitate the analysis and design of feedback control systems under injection attacks. Moreover, we propose the notion of attack decoupling, and show how the controller and the two-way coding can be co-designed to nullify the transfer function from attack to plant, rendering the attack effect zero both in transient phase and in steady state.

eess.SY↗

arXiv2019

Mobility-aware Content Preference Learning in Decentralized Caching Networks

Due to the drastic increase of mobile traffic, wireless caching is proposed to serve repeated requests for content download. To determine the caching scheme for decentralized caching networks, the content preference learning problem based on mobility prediction is studied. We first formulate preference prediction as a decentralized regularized multi-task learning (DRMTL) problem without considering the mobility of mobile terminals (MTs). The problem is solved by a hybrid Jacobian and Gauss-Seidel proximal multi-block alternating direction method (ADMM) based algorithm, which is proven to conditionally converge to the optimal solution with a rate $O(1/k)$. Then we use the tool of \textit{Markov renewal process} to predict the moving path and sojourn time for MTs, and integrate the mobility pattern with the DRMTL model by reweighting the training samples and introducing a transfer penalty in the objective. We solve the problem and prove that the developed algorithm has the same convergence property but with different conditions. Through simulation we show the convergence analysis on proposed algorithms. Our real trace driven experiments illustrate that the mobility-aware DRMTL model can provide a more accurate prediction on geography preference than DRMTL model. Besides, the hit ratio achieved by most popular proactive caching (MPC) policy with preference predicted by mobility-aware DRMTL outperforms the MPC with preference from DRMTL and random caching (RC) schemes.

cs.LG↗

arXiv2019

Non-Asymptotic Behavior of the Maximum Likelihood Estimate of a Discrete Distribution

In this paper, we study the maximum likelihood estimate of the probability mass function (pmf) of $n$ independent and identically distributed (i.i.d.) random variables, in the non-asymptotic regime. We are interested in characterizing the Neyman--Pearson criterion, i.e., the log-likelihood ratio for testing a true hypothesis within a larger hypothesis. Wilks' theorem states that this ratio behaves like a $χ^2$ random variable in the asymptotic case; however, less is known about the precise behavior of the ratio when the number of samples is finite. In this work, we find an explicit bound for the difference between the cumulative distribution function (cdf) of the log-likelihood ratio and the cdf of a $χ^2$ random variable. Furthermore, we show that this difference vanishes with a rate of order $1/\sqrt{n}$ in accordance with Wilks' theorem.

math.ST↗

arXiv2019

Empirical Coordination Subject to a Fidelity Criterion

We study the problem of empirical coordination subject to a fidelity criterion for a general set-up. We prove a result which indicates a strong connection between our framework and the framework of empirical coordination developed in [1]. It turns out that when we design codes that achieve empirical coordination according to a given distribution and subject to the fidelity criterion, it is sufficient to consider codes that produce actions of the same joint type for a class of types which is close enough to our desired distribution is some sense.

cs.IT↗

arXiv2019

An Entropy Power Inequality for Discrete Random Variables

Let $\mathsf{N}_{\rm d}\left[X\right]=\frac{1}{2π{\rm e}}{\rm e}^{2\mathsf{H}\left[X\right]}$ denote the entropy power of the discrete random variable $X$ where $\mathsf{H}\left[X\right]$ denotes the discrete entropy of $X$. In this paper, we show that for two independent discrete random variables $X$ and $Y$, the entropy power inequality $\mathsf{N}_{\rm d}\left[X\right]+\mathsf{N}_{\rm d}\left[Y\right]\leq 2 \mathsf{N}_{\rm d}\left[X+Y\right]$ holds and it can be tight. The basic idea behind the proof is to perturb the discrete random variables using suitably designed continuous random variables. Then, the continuous entropy power inequality is applied to the sum of the perturbed random variables and the resulting lower bound is optimized.

cs.IT↗

arXiv2019

Decentralized Multi-Task Learning Based on Extreme Learning Machines

In multi-task learning (MTL), related tasks learn jointly to improve generalization performance. To exploit the high learning speed of extreme learning machines (ELMs), we apply the ELM framework to the MTL problem, where the output weights of ELMs for all the tasks are learned collaboratively. We first present the ELM based MTL problem in the centralized setting, which is solved by the proposed MTL-ELM algorithm. Due to the fact that many data sets of different tasks are geo-distributed, decentralized machine learning is studied. We formulate the decentralized MTL problem based on ELM as majorized multi-block optimization with coupled bi-convex objective functions. To solve the problem, we propose the DMTL-ELM algorithm, which is a hybrid Jacobian and Gauss-Seidel Proximal multi-block alternating direction method of multipliers (ADMM). Further, to reduce the computation load of DMTL-ELM, DMTL-ELM with first-order approximation (FO-DMTL-ELM) is presented. Theoretical analysis shows that the convergence to the stationary point of DMTL-ELM and FO-DMTL-ELM can be guaranteed conditionally. Through simulations, we demonstrate the convergence of proposed MTL-ELM, DMTL-ELM, and FO-DMTL-ELM algorithms, and also show that they can outperform existing MTL methods. Moreover, by adjusting the dimension of hidden feature space, there exists a trade-off between communication load and learning accuracy for DMTL-ELM.

cs.LG↗

arXiv2019

NOMA in the Uplink: Delay Analysis with Imperfect CSI and Finite-Length Coding

We study whether using non-orthogonal multiple access (NOMA) in the uplink of a mobile network can improve the performance over orthogonal multiple access (OMA) when the system requires ultra-reliable low-latency communications (URLLC). To answer this question, we first consider an ideal system model with perfect channel state information (CSI) at the transmitter and long codewords, where we determine the optimal decoding orders when the decoder uses successive interference cancellation (SIC) and derive closed-form expressions for the optimal rate when joint decoding is used. While joint decoding performs well even under tight delay constraints, NOMA with SIC decoding often performs worse than OMA. For low-latency systems, we must also consider the impact of finite-length channel coding, as well as rate adaptation based imperfect CSI. We derive closed-form approximations for the corresponding outage or error probabilities and find that those effects create a larger performance penalty for NOMA than for OMA. Thus, NOMA with SIC decoding may often be unsuitable for URLLC.

cs.IT↗

arXiv2019

Two-Way Coding in Control Systems Under Injection Attacks: From Attack Detection to Attack Correction

In this paper, we introduce the method of two-way coding, a concept originating in communication theory characterizing coding schemes for two-way channels, into (networked) feedback control systems under injection attacks. We first show that the presence of two-way coding can distort the perspective of the attacker on the control system. In general, the distorted viewpoint on the attacker side as a consequence of two-way coding will facilitate detecting the attacks, or restricting what the attacker can do, or even correcting the attack effect. In the particular case of zero-dynamics attacks, if the attacks are to be designed according to the original plant, then they will be easily detected; while if the attacks are designed with respect to the equivalent plant as viewed by the attacker, then under the additional assumption that the plant is stabilizable by static output feedback, the attack effect may be corrected in steady state.

eess.SY↗

arXiv2018

The Capacity of Private Information Retrieval with Eavesdroppers

We consider the problem of private information retrieval (PIR) with colluding servers and eavesdroppers (abbreviated as ETPIR). The ETPIR problem is comprised of $K$ messages, $N$ servers where each server stores all $K$ messages, a user who wants to retrieve one of the $K$ messages without revealing the desired message index to any set of $T$ colluding servers, and an eavesdropper who can listen to the queries and answers of any $E$ servers but is prevented from learning any information about the messages. The information theoretic capacity of ETPIR is defined to be the maximum number of desired message symbols retrieved privately per information symbol downloaded. We show that the capacity of ETPIR is $C = \left( 1- \frac{E}{N} \right) \left(1 + \frac{T-E}{N-E} + \cdots + \left( \frac{T-E}{N-E} \right)^{K-1} \right)^{-1}$ when $E < T$, and $C = \left( 1 - \frac{E}{N} \right)$ when $E \geq T$. To achieve the capacity, the servers need to share a common random variable (independent of the messages), and its size must be at least $\frac{E}{N} \cdot \frac{1}{C}$ symbols per message symbol. Otherwise, with less amount of shared common randomness, ETPIR is not feasible and the capacity reduces to zero. An interesting observation is that the ETPIR capacity expression takes different forms in two regimes. When $E < T$, the capacity equals the inverse of a sum of a geometric series with $K$ terms and decreases with $K$; this form is typical for capacity expressions of PIR. When $E \geq T$, the capacity does not depend on $K$, a typical form for capacity expressions of SPIR (symmetric PIR, which further requires data-privacy, {\it i.e.,} the user learns no information about other undesired messages); the capacity does not depend on $T$ either. In addition, the ETPIR capacity result includes multiple previous PIR and SPIR capacity results as special cases.

cs.IT↗

arXiv2018

Distributed Coordinated Transmission with Forward-Backward Training for 5G Radio Access

Coordinated multipoint (CoMP) transmission and reception have been considered in cellular networks for enabling larger coverage, improved rates, and interference mitigation. To harness the gains of coordinated beamforming, fast information exchange over a backhaul connecting the cooperating base stations (BSs) is required. In practice, the bandwidth and delay limitations of the backhaul may not be able to meet such stringent demands. These impairments motivate the study of cooperative approaches based only on local channel state information (CSI) and which require minimal or no information exchange between the BSs. To this end, several distributed approaches are introduced for coordinated beamforming (CB)-CoMP. The proposed methods rely on the channel reciprocity and iterative spatially precoded over-the-air pilot signaling. We elaborate how forward-backward (F-B) training facilitates distributed CB by allowing BSs and user equipments (UEs) to iteratively optimize their respective transmitters/receivers based on only locally measured CSI. The trade-off due to the overhead from the F-B iterations is discussed. We also consider the challenge of dynamic TDD where the UE-UE channel knowledge cannot be acquired at the BSs by exploiting channel reciprocity. Finally, standardization activities and practical requirements for enabling the proposed F-B training schemes in 5G radio access are discussed.

cs.IT↗

arXiv2018

The $ε$-error Capacity of Symmetric PIR with Byzantine Adversaries

The capacity of symmetric private information retrieval with $K$ messages, $N$ servers (out of which any $T$ may collude), and an omniscient Byzantine adversary (who can corrupt any $B$ answers) is shown to be $1 - \frac{T+2B}{N}$ [1], under the requirement of zero probability of error. In this work, we show that by weakening the adversary slightly (either providing secret low rate channels between the servers and the user, or limiting the observation of the adversary), and allowing vanishing probability of error, the capacity increases to $1 - \frac{T+B}{N}$.

cs.IT↗

arXiv2018

Privacy-aware Minimum Error Probability Estimation: An Entropy Constrained Approach

This paper studies the design of an optimal privacyaware estimator of a public random variable based on noisy measurements which contain private information. The public random variable carries non-private information, however, its estimate will be correlated with the private information due to the estimation process. It is assumed that the estimate of the public random variable is revealed to an untrusted party. The objective is to design an estimator for the public random variable such that the leakage of the private information, via the estimation process, is kept below a certain level. The privacy metric is defined as the discrete conditional entropy of the private random variable, which carries the private information, given the output of the estimator. A binary loss function is considered for the estimation of the public random variable. It is shown that the optimal privacy-aware estimator is the solution of a (possibly infinite-dimensional) convex optimization problem. We next study the optimal perfect privacy estimator which ensures that the estimate of the public random variable is independent of the private information. A necessary and sufficient condition is derived which guarantees that an estimator satisfies the perfect privacy requirement. It is shown that the optimal perfect privacy estimator is the solution of a linear optimization problem. A sufficient condition for the existence of the optimal perfect privacy estimator is derived when the estimator has access to discretized measurements of sensors.

math.OC↗

arXiv2018

Delay Performance of the Multiuser MISO Downlink under Imperfect CSI and Finite Length Coding

We use stochastic network calculus to investigate the delay performance of a multiuser MISO system with zero-forcing beamforming. First, we consider ideal assumptions with long codewords and perfect CSI at the transmitter, where we observe a strong channel hardening effect that results in very high reliability with respect to the maximum delay of the application. We then study the system under more realistic assumptions with imperfect CSI and finite blocklength channel coding. These effects lead to interference and to transmission errors, and we derive closed-form lower and upper bounds on the resulting error probability. Compared to the ideal case, imperfect CSI and finite length coding cause massive degradations in the average transmission rate. Surprisingly, the system nevertheless maintains the same qualitative behavior as in the ideal case: as long as the average transmission rate is higher than the arrival rate, the system can still achieve very high reliability with respect to the maximum delay.

cs.IT↗

arXiv2018

Learning Kolmogorov Models for Binary Random Variables

We summarize our recent findings, where we proposed a framework for learning a Kolmogorov model, for a collection of binary random variables. More specifically, we derive conditions that link outcomes of specific random variables, and extract valuable relations from the data. We also propose an algorithm for computing the model and show its first-order optimality, despite the combinatorial nature of the learning problem. We apply the proposed algorithm to recommendation systems, although it is applicable to other scenarios. We believe that the work is a significant step toward interpretable machine learning.

cs.LG↗