arXiv subjects

Osvaldo Simeone

Publications and source records attributed to Osvaldo Simeone.

At least 271 records · Page 15Linked to original sources

arXiv2019

An Introduction to Probabilistic Spiking Neural Networks: Probabilistic Models, Learning Rules, and Applications

Spiking neural networks (SNNs) are distributed trainable systems whose computing elements, or neurons, are characterized by internal analog dynamics and by digital and sparse synaptic communications. The sparsity of the synaptic spiking inputs and the corresponding event-driven nature of neural processing can be leveraged by energy-efficient hardware implementations, which can offer significant energy reductions as compared to conventional artificial neural networks (ANNs). The design of training algorithms lags behind the hardware implementations. Most existing training algorithms for SNNs have been designed either for biological plausibility or through conversion from pretrained ANNs via rate encoding. This article provides an introduction to SNNs by focusing on a probabilistic signal processing methodology that enables the direct derivation of learning rules by leveraging the unique time-encoding capabilities of SNNs. We adopt discrete-time probabilistic models for networked spiking neurons and derive supervised and unsupervised learning rules from first principles via variational inference. Examples and open research problems are also provided.

cs.LG↗

arXiv2019

Beyond Max-SNR: Joint Encoding for Reconfigurable Intelligent Surfaces

A communication link aided by a Reconfigurable Intelligent Surface (RIS) is studied, in which the transmitter can control the state of the RIS via a finite-rate control link. Prior work mostly assumed a fixed RIS configuration irrespective of the transmitted information. In contrast, this work derives information-theoretic limits, and demonstrates that the capacity is achieved by a scheme that jointly encodes information in the transmitted signal as well as in the RIS configuration. In addition, a novel signaling strategy based on layered encoding is proposed that enables practical successive cancellation-type decoding at the receiver. Numerical experiments demonstrate that the standard max-SNR scheme that fixes the configuration of the RIS as to maximize the Signal-to-Noise Ratio (SNR) at the receiver is strictly suboptimal, and is outperformed by the proposed strategies at all practical SNR levels.

cs.IT↗

arXiv2019

Joint Source-Channel Coding and Bayesian Message Passing Detection for Grant-Free Radio Access in IoT

Consider an Internet-of-Things (IoT) system that monitors a number of multi-valued events through multiple sensors sharing the same bandwidth. Each sensor measures data correlated to one or more events, and communicates to the fusion center at a base station using grant-free random access whenever the corresponding event is active. The base station aims at detecting the active events, and, for each active event, to determine a scalar value describing each active event's state. A conventional solution based on Separate Source-Channel (SSC) coding would use a separate codebook for each sensor and decode the sensors' transmitted packets at the base station in order to subsequently carry out events' detection. In contrast, this paper considers a potentially more efficient solution based on Joint Source-Channel (JSC) coding via a non-orthogonal generalization of Type-Based Multiple Access (TBMA). Accordingly, all sensors measuring the same event share the same codebook (with non-orthogonal codewords), and the base station directly detects the events' values without first performing individual decoding for each sensor. A novel Bayesian message-passing detection scheme is developed for the proposed TBMA-based protocol, and its performance is compared to conventional solutions.

eess.SP↗

arXiv2019

Meta-Learning to Communicate: Fast End-to-End Training for Fading Channels

When a channel model is available, learning how to communicate on fading noisy channels can be formulated as the (unsupervised) training of an autoencoder consisting of the cascade of encoder, channel, and decoder. An important limitation of the approach is that training should be generally carried out from scratch for each new channel. To cope with this problem, prior works considered joint training over multiple channels with the aim of finding a single pair of encoder and decoder that works well on a class of channels. As a result, joint training ideally mimics the operation of non-coherent transmission schemes. In this paper, we propose to obviate the limitations of joint training via meta-learning: Rather than training a common model for all channels, meta-learning finds a common initialization vector that enables fast training on any channel. The approach is validated via numerical results, demonstrating significant training speed-ups, with effective encoders and decoders obtained with as little as one iteration of Stochastic Gradient Descent.

eess.SP↗

arXiv2019

Federated Neuromorphic Learning of Spiking Neural Networks for Low-Power Edge Intelligence

Spiking Neural Networks (SNNs) offer a promising alternative to conventional Artificial Neural Networks (ANNs) for the implementation of on-device low-power online learning and inference. On-device training is, however, constrained by the limited amount of data available at each device. In this paper, we propose to mitigate this problem via cooperative training through Federated Learning (FL). To this end, we introduce an online FL-based learning rule for networked on-device SNNs, which we refer to as FL-SNN. FL-SNN leverages local feedback signals within each SNN, in lieu of backpropagation, and global feedback through communication via a base station. The scheme demonstrates significant advantages over separate training and features a flexible trade-off between communication load and accuracy via the selective exchange of synaptic weights.

cs.LG↗

arXiv2019

An Introduction to Spiking Neural Networks: Probabilistic Models, Learning Rules, and Applications

Spiking Neural Networks (SNNs) are distributed trainable systems whose computing elements, or neurons, are characterized by internal analog dynamics and by digital and sparse synaptic communications. The sparsity of the synaptic spiking inputs and the corresponding event-driven nature of neural processing can be leveraged by hardware implementations that have demonstrated significant energy reductions as compared to conventional Artificial Neural Networks (ANNs). Most existing training algorithms for SNNs have been designed either for biological plausibility or through conversion from pre-trained ANNs via rate encoding. This paper aims at providing an introduction to SNNs by focusing on a probabilistic signal processing methodology that enables the direct derivation of learning rules leveraging the unique time encoding capabilities of SNNs. To this end, the paper adopts discrete-time probabilistic models for networked spiking neurons, and it derives supervised and unsupervised learning rules from first principles by using variational inference. Examples and open research problems are also provided.

eess.SP↗

arXiv2019

Space Diversity-Based Grant-Free Random Access for Critical and Non-Critical IoT Services

In this paper, we study the coexistence of critical and non-critical Internet of Things (IoT) services on a grant-free channel consisting of radio access and backhaul segments. On the radio access segment, IoT devices send packets to access points (APs) over an erasure collision channel using the slotted ALOHA protocol. Then, the APs forward correctly received messages to a base station (BS) over a shared wireless backhaul segment, modeled as an erasure collision channel. The APs hence play the role of uncoordinated relays that provide space diversity and may reduce performance losses caused by collisions. Both non-orthogonal and inter-service orthogonal resource sharing are considered and compared. Throughput and reliability metrics are analyzed, and numerical results are provided to assess the performance trade-offs between critical and non-critical IoT services.

cs.NI↗

arXiv2019

Blind Sparse Estimation of Intermittent Sources over Unknown Fading Channels

Radio frequency sources are observed at a fusion center via sensor measurements made over slow flat-fading channels. The number of sources may be larger than the number of sensors, but their activity is sparse and intermittent with bursty transmission patterns. To account for this, sources are modeled as hidden Markov models with known or unknown parameters. The problem of blind source estimation in the absence of channel state information is tackled via a novel algorithm, consisting of a dictionary learning (DL) stage and a per-source stochastic filtering (PSF) stage. The two stages work in tandem, with the latter operating on the output produced by the former. Both stages are designed so as to account for the sparsity and memory of the sources. To this end, smooth LASSO is integrated with DL, while the forward-backward algorithm and Expectation Maximization (EM) algorithm are leveraged for PSF. It is shown that the proposed algorithm can enhance the detection and the estimation performance of the sources, and that it is robust to the sparsity level.

eess.SP↗

arXiv2019

Online Supervised Learning for Traffic Load Prediction in Framed-ALOHA Networks

Predicting the current backlog, or traffic load, in framed-ALOHA networks enables the optimization of resource allocation, e.g., of the frame size. However, this prediction is made difficult by the lack of information about the cardinality of collisions and by possibly complex packet generation statistics. Assuming no prior information about the traffic model, apart from a bound on its temporal memory, this paper develops an online learning-based adaptive traffic load prediction method that is based on Recurrent Neural Networks (RNN) and specifically on the Long Short-Term Memory (LSTM) architecture. In order to enable online training in the absence of feedback on the exact cardinality of collisions, the proposed strategy leverages a novel approximate labeling technique that is inspired by Method of Moments (MOM) estimators. Numerical results show that the proposed online predictor considerably outperforms conventional methods and is able to adapt to changing traffic statistics.

cs.NI↗

arXiv2019

Wireless Federated Distillation for Distributed Edge Learning with Heterogeneous Data

Cooperative training methods for distributed machine learning typically assume noiseless and ideal communication channels. This work studies some of the opportunities and challenges arising from the presence of wireless communication links. We specifically consider wireless implementations of Federated Learning (FL) and Federated Distillation (FD), as well as of a novel Hybrid Federated Distillation (HFD) scheme. Both digital implementations based on separate source-channel coding and over-the-air computing implementations based on joint source-channel coding are proposed and evaluated over Gaussian multiple-access channels.

cs.IT↗

arXiv2019

Semantic-Effectiveness Filtering and Control for Post-5G Wireless Connectivity

The traditional role of a communication engineer is to address the technical problem of transporting bits reliably over a noisy channel. With the emergence of 5G, and the availability of a variety of competing and coexisting wireless systems, wireless connectivity is becoming a commodity. This article argues that communication engineers in the post-5G era should extend the scope of their activity in terms of design objectives and constraints beyond connectivity to encompass the semantics of the transferred bits within the given applications and use cases. To provide a platform for semantic-aware connectivity solutions, this paper introduces the concept of a semantic-effectiveness (SE) plane as a core part of future communication architectures. The SE plane augments the protocol stack by providing standardized interfaces that enable information filtering and direct control of functionalities at all layers of the protocol stack. The advantages of the SE plane are described in the perspective of recent developments in 5G, and illustrated through a number of example applications. The introduction of a SE plane may help replacing the current "next-G paradigm" in wireless evolution with a framework based on continuous improvements and extensions of the systems and standards.

cs.NI↗

arXiv2019

Optimizing Pipelined Computation and Communication for Latency-Constrained Edge Learning

Consider a device that is connected to an edge processor via a communication channel. The device holds local data that is to be offloaded to the edge processor so as to train a machine learning model, e.g., for regression or classification. Transmission of the data to the learning processor, as well as training based on Stochastic Gradient Descent (SGD), must be both completed within a time limit. Assuming that communication and computation can be pipelined, this letter investigates the optimal choice for the packet payload size, given the overhead of each data packet transmission and the ratio between the computation and the communication rates. This amounts to a tradeoff between bias and variance, since communicating the entire data set first reduces the bias of the training process but it may not leave sufficient time for learning. Analytical bounds on the expected optimality gap are derived so as to enable an effective optimization, which is validated in numerical results.

cs.LG↗

arXiv2019

On Model Coding for Distributed Inference and Transmission in Mobile Edge Computing Systems

Consider a mobile edge computing system in which users wish to obtain the result of a linear inference operation on locally measured input data. Unlike the offloaded input data, the model weight matrix is distributed across wireless Edge Nodes (ENs). ENs have non-deterministic computing times, and they can transmit any shared computed output back to the users cooperatively. This letter investigates the potential advantages obtained by coding model information prior to ENs' storage. Through an information-theoretic analysis, it is concluded that, while generally limiting cooperation opportunities, coding is instrumental in reducing the overall computation-plus-communication latency.

cs.IT↗

arXiv2019

Non-Orthogonal Unicast and Broadcast Transmission via Joint Beamforming and LDM in Cellular Networks

Limited bandwidth resources and higher energy efficiency requirements motivate incorporating multicast and broadcast transmission into the next-generation cellular network architectures, particularly for multimedia streaming applications. Layered division multiplexing (LDM), a form of NOMA, can potentially improve unicast throughput and broadcast coverage with respect to traditional orthogonal frequency division multiplexing (FDM) or time division multiplexing (TDM), by simultaneously using the same frequency and time resources for multiple unicast or broadcast transmissions. In this paper, the performance of LDM-based unicast and broadcast transmission in a cellular network is studied by assuming a single frequency network (SFN) operation for the broadcast layer, while allowing arbitrarily clustered cooperation among the base stations (BSs) for the transmission of unicast data streams. Beamforming and power allocation between unicast and broadcast layers, the so-called injection level in the LDM literature, are optimized with the aim of minimizing the sum-power under constraints on the user-specific unicast rates and on the common broadcast rate. The effects of imperfect channel coding and imperfect CSI are also studied to gain insights into robust implementation in practical systems. The non-convex optimization problem is tackled by means of successive convex approximation (SCA) techniques. Performance upper bounds are also presented by means of the $\rm{S}$-procedure followed by semidefinite relaxation (SDR). Finally, a dual decomposition-based solution is proposed to facilitate an efficient distributed implementation of LDM where the optimal unicast beamforming vectors can be obtained locally by the cooperating BSs. Numerical results are presented, which show the tightness of the proposed bounds and hence the near-optimality of the proposed solutions.

cs.IT↗

arXiv2019

How Much Can D2D Communication Reduce Content Delivery Latency in Fog Networks with Edge Caching?

A Fog-Radio Access Network (F-RAN) is studied in which cache-enabled Edge Nodes (ENs) with dedicated fronthaul connections to the cloud aim at delivering contents to mobile users. Using an information-theoretic approach, this work tackles the problem of quantifying the potential latency reduction that can be obtained by enabling Device-to-Device (D2D) communication over out-of-band broadcast links. Following prior work, the Normalized Delivery Time (NDT) --- a metric that captures the high signal-to-noise ratio worst-case latency --- is adopted as the performance criterion of interest. Joint edge caching, downlink transmission, and D2D communication policies based on compress-and-forward are proposed that are shown to be information-theoretically optimal to within a constant multiplicative factor of two for all values of the problem parameters, and to achieve the minimum NDT for a number of special cases. The analysis provides insights on the role of D2D cooperation in improving the delivery latency.

cs.IT↗

arXiv2019

Joint Design of Fronthauling and Hybrid Beamforming for Downlink C-RAN Systems

Hybrid beamforming is known to be a cost-effective and wide-spread solution for a system with large-scale antenna arrays. This work studies the optimization of the analog and digital components of the hybrid beamforming solution for remote radio heads (RRHs) in a downlink cloud radio access network (C-RAN) architecture. Digital processing is carried out at a baseband processing unit (BBU) in the "cloud" and the precoded baseband signals are quantized prior to transmission to the RRHs via finite-capacity fronthaul links. In this system, we consider two different channel state information (CSI) scenarios: 1) ideal CSI at the BBU 2) imperfect effective CSI. Optimization of digital beamforming and fronthaul quantization strategies at the BBU as well as analog radio frequency (RF) beamforming at the RRHs is a coupled problem, since the effect of the quantization noise at the receiver depends on the precoding matrices. The resulting joint optimization problem is examined with the goal of maximizing the weighted downlink sum-rate and the network energy efficiency. Fronthaul capacity and per-RRH power constraints are enforced along with constant modulus constraint on the RF beamforming matrices. For the case of perfect CSI, a block coordinate descent scheme is proposed based on the weighted minimum-mean-square-error approach by relaxing the constant modulus constraint of the analog beamformer. Also, we present the impact of imperfect CSI on the weighted sum-rate and network energy efficiency performance, and the algorithm is extended by applying the sample average approximation. Numerical results confirm the effectiveness of the proposed scheme and show that the proposed algorithm is robust to estimation errors.

cs.IT↗

arXiv2019

Learning How to Demodulate from Few Pilots via Meta-Learning

Consider an Internet-of-Things (IoT) scenario in which devices transmit sporadically using short packets with few pilot symbols. Each device transmits over a fading channel and is characterized by an amplifier with a unique non-linear transfer function. The number of pilots is generally insufficient to obtain an accurate estimate of the end-to-end channel, which includes the effects of fading and of the amplifier's distortion. This paper proposes to tackle this problem using meta-learning. Accordingly, pilots from previous IoT transmissions are used as meta-training in order to learn a demodulator that is able to quickly adapt to new end-to-end channel conditions from few pilots. Numerical results validate the advantages of the approach as compared to training schemes that either do not leverage prior transmissions or apply a standard learning algorithm on previously received data.

eess.SP↗

arXiv2019

Learning-Based Physical Layer Communications for Multi-agent Collaboration

Consider a collaborative task carried out by two autonomous agents that are able to communicate over a noisy channel. Each agent is only aware of its own state, while the accomplishment of the task depends on the value of the joint state of both agents. As an example, both agents must simultaneously reach a certain location of the environment, while only being aware of their respective positions. Assuming the presence of feedback in the form of a common reward to the agents, a conventional approach would apply separately: (i) an off-the-shelf coding and decoding scheme in order to enhance the reliability of the communication of the state of one agent to the other; and (ii) a standard multi-agent reinforcement learning strategy to learn how to act in the resulting environment. In this work, it is demonstrated that the performance of the collaborative task can be improved if the agents learn jointly how to communicate and to act, even in the presence of a delay in the communication channel.

cs.IT↗