arXiv subjects

Deniz Gündüz

Publications and source records attributed to Deniz Gündüz.

At least 145 records · Page 8Linked to original sources

arXiv2021

Dynamic Scheduling for Over-the-Air Federated Edge Learning with Energy Constraints

Machine learning and wireless communication technologies are jointly facilitating an intelligent edge, where federated edge learning (FEEL) is a promising training framework. As wireless devices involved in FEEL are resource limited in terms of communication bandwidth, computing power and battery capacity, it is important to carefully schedule them to optimize the training performance. In this work, we consider an over-the-air FEEL system with analog gradient aggregation, and propose an energy-aware dynamic device scheduling algorithm to optimize the training performance under energy constraints of devices, where both communication energy for gradient aggregation and computation energy for local training are included. The consideration of computation energy makes dynamic scheduling challenging, as devices are scheduled before local training, but the communication energy for over-the-air aggregation depends on the l2-norm of local gradient, which is known after local training. We thus incorporate estimation methods into scheduling to predict the gradient norm. Taking the estimation error into account, we characterize the performance gap between the proposed algorithm and its offline counterpart. Experimental results show that, under a highly unbalanced local data distribution, the proposed algorithm can increase the accuracy by 4.9% on CIFAR-10 dataset compared with the myopic benchmark, while satisfying the energy constraints.

cs.LG↗

arXiv2021

Distributed Learning in Wireless Networks: Recent Progress and Future Challenges

The next-generation of wireless networks will enable many machine learning (ML) tools and applications to efficiently analyze various types of data collected by edge devices for inference, autonomy, and decision making purposes. However, due to resource constraints, delay limitations, and privacy challenges, edge devices cannot offload their entire collected datasets to a cloud server for centrally training their ML models or inference purposes. To overcome these challenges, distributed learning and inference techniques have been proposed as a means to enable edge devices to collaboratively train ML models without raw data exchanges, thus reducing the communication overhead and latency as well as improving data privacy. However, deploying distributed learning over wireless networks faces several challenges including the uncertain wireless environment, limited wireless resources (e.g., transmit power and radio spectrum), and hardware resources. This paper provides a comprehensive study of how distributed learning can be efficiently and effectively deployed over wireless edge networks. We present a detailed overview of several emerging distributed learning paradigms, including federated learning, federated distillation, distributed inference, and multi-agent reinforcement learning. For each learning framework, we first introduce the motivation for deploying it over wireless networks. Then, we present a detailed literature review on the use of communication techniques for its efficient deployment. We then introduce an illustrative example to show how to optimize wireless networks to improve its performance. Finally, we introduce future research opportunities. In a nutshell, this paper provides a holistic set of guidelines on how to deploy a broad range of distributed learning frameworks over real-world wireless communication networks.

cs.LG↗

arXiv2021

Dopamine: Differentially Private Federated Learning on Medical Data

While rich medical datasets are hosted in hospitals distributed across the world, concerns on patients' privacy is a barrier against using such data to train deep neural networks (DNNs) for medical diagnostics. We propose Dopamine, a system to train DNNs on distributed datasets, which employs federated learning (FL) with differentially-private stochastic gradient descent (DPSGD), and, in combination with secure aggregation, can establish a better trade-off between differential privacy (DP) guarantee and DNN's accuracy than other approaches. Results on a diabetic retinopathy~(DR) task show that Dopamine provides a DP guarantee close to the centralized training counterpart, while achieving a better classification accuracy than FL with parallel DP where DPSGD is applied without coordination. Code is available at https://github.com/ipc-lab/private-ml-for-health.

cs.LG↗

arXiv2020

Distributed Deep Reinforcement Learning for Functional Split Control in Energy Harvesting Virtualized Small Cells

To meet the growing quest for enhanced network capacity, mobile network operators (MNOs) are deploying dense infrastructures of small cells. This, in turn, increases the power consumption of mobile networks, thus impacting the environment. As a result, we have seen a recent trend of powering mobile networks with harvested ambient energy to achieve both environmental and cost benefits. In this paper, we consider a network of virtualized small cells (vSCs) powered by energy harvesters and equipped with rechargeable batteries, which can opportunistically offload baseband (BB) functions to a grid-connected edge server depending on their energy availability. We formulate the corresponding grid energy and traffic drop rate minimization problem, and propose a distributed deep reinforcement learning (DDRL) solution. Coordination among vSCs is enabled via the exchange of battery state information. The evaluation of the network performance in terms of grid energy consumption and traffic drop rate confirms that enabling coordination among the vSCs via knowledge exchange achieves a performance close to the optimal. Numerical results also confirm that the proposed DDRL solution provides higher network performance, better adaptation to the changing environment, and higher cost savings with respect to a tabular multi-agent reinforcement learning (MRL) solution used as a benchmark.

cs.NI↗

arXiv2020

Multi-Antenna Coded Content Delivery with Caching: A Low-Complexity Solution

We study downlink beamforming in a single-cell network with a multi-antenna base station serving cache-enabled users. Assuming a library of files with a common rate, we formulate the minimum transmit power with proactive caching and coded delivery as a non-convex optimization problem. While this multiple multicast problem can be efficiently solved by successive convex approximation (SCA), the complexity of the problem grows exponentially with the number of subfiles delivered to each user in each time slot, which itself grows exponentially with the number of users. We introduce a low-complexity alternative through time-sharing that limits the number of subfiles received by a user in each time slot. We then consider the joint design of beamforming and content delivery with sparsity constraints to limit the number of subfiles received by a user in each time slot. Numerical simulations show that the low-complexity scheme has only a small performance gap to that obtained by solving the joint problem with sparsity constraints, and outperforms state-of-the-art results at all signal-to-noise ratio (SNR) and rate values with a sufficient number of transmit antennas. A lower bound on the achievable degrees-of-freedom (DoF) of the low-complexity scheme is derived to characterize its performance in the high SNR regime.

cs.IT↗

arXiv2020

Strong Converse for Testing Against Independence over a Noisy channel

A distributed binary hypothesis testing (HT) problem over a noisy (discrete and memoryless) channel studied previously by the authors is investigated from the perspective of the strong converse property. It was shown by Ahlswede and Csiszár that a strong converse holds in the above setting when the channel is rate-limited and noiseless. Motivated by this observation, we show that the strong converse continues to hold in the noisy channel setting for a special case of HT known as testing against independence (TAI). The proof utilizes the blowing up lemma and the recent change of measure technique of Tyagi and Watanabe as the key tools.

cs.IT↗

arXiv2020

Privacy-aware Distributed Hypothesis Testing

A distributed binary hypothesis testing (HT) problem involving two parties, a remote observer and a detector, is studied. The remote observer has access to a discrete memoryless source, and communicates its observations to the detector via a rate-limited noiseless channel. The detector observes another discrete memoryless source, and performs a binary hypothesis test on the joint distribution of its own observations with those of the observer. While the goal of the observer is to maximize the type II error exponent of the test for a given type I error probability constraint, it also wants to keep a private part of its observations as oblivious to the detector as possible. Considering both equivocation and average distortion under a causal disclosure assumption as possible measures of privacy, the trade-off between the communication rate from the observer to the detector, the type II error exponent, and privacy is studied. For the general HT problem, we establish single-letter inner bounds on both the rate-error exponent-equivocation and rate-error exponent-distortion trade-offs. Subsequently, single-letter characterizations for both trade-offs are obtained (i) for testing against conditional independence of the observer's observations from those of the detector, given some additional side-information at the detector; and (ii) when the communication rate constraint over the channel is zero. Finally, we show by providing a counterexample that, the strong converse which holds for distributed HT without a privacy constraint, does not hold when a privacy constraint is imposed. This implies that, in general, the rate-error exponent-equivocation and rate-error exponent-distortion trade-offs are not independent of the type I error probability constraint.

cs.IT↗

arXiv2020

Deep Learning for Massive MIMO Channel State Acquisition and Feedback

Massive multiple-input multiple-output (MIMO) systems are a main enabler of the excessive throughput requirements in 5G and future generation wireless networks as they can serve many users simultaneously with high spectral and energy efficiency. To achieve this, massive MIMO systems require accurate and timely channel state information (CSI), which is acquired by a training process that involves pilot transmission, CSI estimation and feedback. This training process incurs a training overhead, which scales with the number of antennas, users and subcarriers. Reducing this training overhead in massive MIMO systems has been a major topic of research since the emergence of the concept. Recently, deep learning (DL)-based approaches for massive MIMO training have been proposed and showed significant improvements compared to traditional techniques. This paper provides an overview of how neural networks (NNs) can be used in the training process of massive MIMO systems to improve the performance by reducing the CSI acquisition overhead and to reduce complexity.

cs.IT↗

arXiv2020

DeepJSCC-f: Deep Joint Source-Channel Coding of Images with Feedback

We consider wireless transmission of images in the presence of channel output feedback. From a Shannon theoretic perspective feedback does not improve the asymptotic end-to-end performance, and separate source coding followed by capacity-achieving channel coding, which ignores the feedback signal, achieves the optimal performance. It is well known that separation is not optimal in the practical finite blocklength regime; however, there are no known practical joint source-channel coding (JSCC) schemes that can exploit the feedback signal and surpass the performance of separation-based schemes. Inspired by the recent success of deep learning methods for JSCC, we investigate how noiseless or noisy channel output feedback can be incorporated into the transmission system to improve the reconstruction quality at the receiver. We introduce an autoencoder-based JSCC scheme, which we call DeepJSCC-f, that exploits the channel output feedback, and provides considerable improvements in terms of the end-to-end reconstruction quality for fixed-length transmission, or in terms of the average delay for variable-length transmission. To the best of our knowledge, this is the first practical JSCC scheme that can fully exploit channel output feedback, demonstrating yet another setting in which modern machine learning techniques can enable the design of new and efficient communication methods that surpass the performance of traditional structured coding-based designs.

cs.IT↗

arXiv2020

Information transmission bounds between moving terminals

In networks of mobile autonomous agents, e.g. for data acquisition, we may wish to maximize data transfer or to reliably transfer a minimum amount of data, subject to quality of service or energy constraints. These requirements can be guaranteed through both offline node design/specifications and online trajectory/communications design. Regardless of the distance between them, for a stationary point-to-point transmitter-receiver pair communicating across a single link under average power constraints, the total data transfer is unbounded as time tends to infinity. In contrast, we show that if the transmitter/receiver is moving at any constant speed away from each other, then the maximum transmittable data is bounded. Although general closed-form expressions as a function of communication and mobility profile parameters do not yet exist, we provide closed-form expressions for particular cases, such as ideal free space path loss. Under more general scenarios we instead give lower bounds on the total transmittable information across a single link between mobile nodes.

cs.IT↗

arXiv2019

Distributed Hypothesis Testing Over Discrete Memoryless Channels

A distributed binary hypothesis testing (HT) problem involving two parties, one referred to as the observer and the other as the detector is studied. The observer observes a discrete memoryless source (DMS) and communicates its observations to the detector over a discrete memoryless channel (DMC). The detector observes another DMS correlated with that at the observer, and performs a binary HT on the joint distribution of the two DMS's using its own observed data and the information received from the observer. The trade-off between the type I error probability and the type II error-exponent of the HT is explored. Single-letter lower bounds on the optimal type II error-exponent are obtained by using two different coding schemes, a separate HT and channel coding scheme and a joint HT and channel coding scheme based on hybrid coding for the matched bandwidth case. Exact single-letter characterization of the same is established for the special case of testing against conditional independence, and it is shown to be achieved by the separate HT and channel coding scheme. An example is provided where the joint scheme achieves a strictly better performance than the separation based scheme.

cs.IT↗

arXiv2019

Average Age-of-Information with a Backup Information Source

Data collected and transmitted by Internet of things (IoT) devices are typically used for control and monitoring purposes; and hence, their timely delivery is of utmost importance for the underlying applications. However, IoT devices operate with very limited energy sources, severely reducing their ability for timely collection and processing of status updates. IoT systems make up for these limitations by employing multiple low-power low-complexity devices that can monitor the same signal, possibly with different quality observations and different energy costs, to create diversity against the limitations of individual nodes. We investigate policies to minimize the average age of information (AoI) in a monitoring system that collects data from two sources of information denoted as primary and backup sources, respectively. We assume that each source offers a different trade-off between the AoI and the energy cost. The monitoring node is equipped with a finite size battery and harvests ambient energy. For this setup, we formulate the scheduling of status updates from the two sources as a Markov decision process (MDP), and obtain a policy that decides on the optimal action to take (i.e., which source to query or remain idle) depending on the current energy level and AoI. The performance of the obtained policy is compared with an aggressive policy for different system parameters. We identify few types of optimal solution structures and discuss the benefits of having a backup source of information in the system.

cs.IT↗

arXiv2019

Energy-Aware Analog Aggregation for Federated Learning with Redundant Data

Federated learning (FL) enables workers to learn a model collaboratively by using their local data, with the help of a parameter server (PS) for global model aggregation. The high communication cost for periodic model updates and the non-independent and identically distributed (i.i.d.) data become major bottlenecks for FL. In this work, we consider analog aggregation to scale down the communication cost with respect to the number of workers, and introduce data redundancy to the system to deal with non-i.i.d. data. We propose an online energy-aware dynamic worker scheduling policy, which maximizes the average number of workers scheduled for gradient update at each iteration under a long-term energy constraint, and analyze its performance based on Lyapunov optimization. Experiments using MNIST dataset show that, for non-i.i.d. data, doubling data storage can improve the accuracy by 9.8% under a stringent energy budget, while the proposed policy can achieve close-to-optimal accuracy without violating the energy constraint.

cs.IT↗

arXiv2019

Management and Orchestration of Virtual Network Functions via Deep Reinforcement Learning

Management and orchestration (MANO) of resources by virtual network functions (VNFs) represents one of the key challenges towards a fully virtualized network architecture as envisaged by 5G standards. Current threshold-based policies inefficiently over-provision network resources and under-utilize available hardware, incurring high cost for network operators, and consequently, the users. In this work, we present a MANO algorithm for VNFs allowing a central unit (CU) to learn to autonomously re-configure resources (processing power and storage), deploy new VNF instances, or offload them to the cloud, depending on the network conditions, available pool of resources, and the VNF requirements, with the goal of minimizing a cost function that takes into account the economical cost as well as latency and the quality-of-service (QoS) experienced by the users. First, we formulate the stochastic resource optimization problem as a parameterized action Markov decision process (PAMDP). Then, we propose a solution based on deep reinforcement learning (DRL). More precisely, we present a novel RL approach called, parameterized action twin (PAT) deterministic policy gradient, which leverages an actor-critic architecture to learn to provision resources to the VNFs in an online manner. Finally, we present numerical performance results, and map them to 5G key performance indicators (KPIs). To the best of our knowledge, this is the first work that considers DRL for MANO of VNFs' physical resources.

cs.IT↗

arXiv2019

Deep Convolutional Compression for Massive MIMO CSI Feedback

Coded caching provides significant gains over conventional uncoded caching by creating multicasting opportunities among distinct requests. Massive multiple-input multiple-output (MIMO) systems require downlink channel state information (CSI) at the base station (BS) to better utilize the available spatial diversity and multiplexing gains. However, in a frequency division duplex (FDD) massive MIMO system, the huge CSI feedback overhead becomes restrictive and degrades the overall spectral efficiency. In this paper, we propose a deep learning based channel state matrix compression scheme, called DeepCMC, composed of convolutional layers followed by quantization and entropy coding blocks. In comparison with previous works, the main contributions of DeepCMC are two-fold: i) DeepCMC is fully convolutional, and it can be used in a wide range of scenarios with various numbers of sub-channels and transmit antennas; ii) DeepCMC includes quantization and entropy coding blocks and minimizes a cost function that accounts for both the rate of compression and the reconstruction quality of the channel matrix at the BS. Simulation results demonstrate that DeepCMC significantly outperforms the state of the art compression schemes in terms of the reconstruction quality of the channel state matrix for the same compression rate, measured in bits per channel dimension.

cs.IT↗

arXiv2019

Centralized Caching and Delivery of Correlated Contents over Gaussian Broadcast Channels

Content delivery in a multi-user cache-aided broadcast network is studied, where a server holding a database of correlated contents communicates with the users over a Gaussian broadcast channel (BC). The minimum transmission power required to satisfy all possible demand combinations is studied, when the users are equipped with caches of equal size. Assuming uncoded cache placement, a lower bound on the required transmit power as a function of the cache capacity is derived. An achievable centralized caching scheme is proposed, which not only utilizes the user's local caches, but also exploits the correlation among the contents in the database. The performance of the scheme, which provides an upper bound on the required transmit power for a given cache capacity, is characterized. Our results indicate that exploiting the correlations among the contents in a cache-aided Gaussain BC can provide significant energy savings.

cs.IT↗

arXiv2019

Joint Optimization of Transmission and Propulsion in UAV-Assisted Communication Networks

The communication energy in a wireless network of mobile autonomous agents should be defined to include the propulsion energy as well as the transmission energy used to facilitate information transfer. We therefore develop communicationtheoretic and Newtonian dynamic models of the communication and locomotion expenditures of an unmanned aerial vehicle (UAV). These models are used to formulate a novel nonlinear optimal control problem (OCP) for arbitrary networks of autonomous agents. This is the first work to consider mobility as a decision variable in UAV networks with multiple access channels. Where possible, we compare our results with known analytic solutions for particular single-hop network configurations. The OCP is then applied to a multiple-node UAV network for which previous results cannot be readily extended. Numerical results demonstrate increased network capacity and communication energy savings upwards of 70% when compared to more naive communication policies.

eess.SY↗

arXiv2019

Heterogeneous Coded Computation across Heterogeneous Workers

Coded distributed computing framework enables large-scale machine learning (ML) models to be trained efficiently in a distributed manner, while mitigating the straggler effect. In this work, we consider a multi-task assignment problem in a coded distributed computing system, where multiple masters, each with a different matrix multiplication task, assign computation tasks to workers with heterogeneous computing capabilities. Both dedicated and probabilistic worker assignment models are considered, with the objective of minimizing the average completion time of all computations. For dedicated worker assignment, greedy algorithms are proposed and the corresponding optimal load allocation is derived based on the Lagrange multiplier method. For probabilistic assignment, successive convex approximation method is used to solve the non-convex optimization problem. Simulation results show that the proposed algorithms reduce the completion time by 80% over uncoded scheme, and 49% over an unbalanced coded scheme.

cs.IT↗