arXiv subjects

Rongpeng Li

Publications and source records attributed to Rongpeng Li.

At least 73 records · Page 4Linked to original sources

arXiv2022

Rethinking Modern Communication from Semantic Coding to Semantic Communication

Modern communications are usually designed to pursue a higher bit-level precision and fewer bits while transmitting a message. This article rethinks these two major features and introduces the concept and advantage of semantics that characterizes a new kind of semantics-aware communication framework, incorporating both the semantic encoding and the semantic communication problem. After analyzing the underlying defects of existing semantics-aware techniques, we establish a confidence-based distillation mechanism for the joint semantics-noise coding (JSNC) problem and a reinforcement learning (RL)-powered semantic communication paradigm that endows a system the ability to convey the semantics instead of pursuing the bit level accuracy. On top of these technical contributions, this work provides a new insight to understand how the semantics are processed and represented in a semantics-aware coding and communication system, and verifies the significant benefits of doing so. Targeted on the next generation's semantics-aware communication, some critical concerns and open challenges such as the information overhead, semantic security and implementation cost are also discussed and envisioned.

eess.SP↗

arXiv2022

Reinforcement Learning-powered Semantic Communication via Semantic Similarity

We introduce a new semantic communication mechanism - SemanticRL, whose key idea is to preserve the semantic information instead of strictly securing the bit-level precision. Unlike previous methods that mainly concentrate on the network or structure design, we revisit the learning process and point out the semantic blindness of commonly used objective functions. To address this semantic gap, we introduce a schematic shift that learns from semantic similarity, instead of relying on conventional paired bit-level supervisions like cross entropy and bit error rate. However, developing such a semantic communication system is indeed a nontrivial task considering the non-differentiability of most semantic metrics as well as the instability from noisy channels. To further resolve these issues, we put forward a self-critic reinforcement learning (RL) solution which allows an efficient and stable learning on any user-defined semantic measurement, and take a step further to simultaneously tackle the non-differentiable semantic channel optimization problem via self-critic stochastic iterative updating (SCSIU) training on the decoupled semantic transceiver. We have firstly tested the proposed method in the challenging European-parliament dataset, which confirms the superiority of our method in revealing the semantic meanings, and better handling the semantic noise. Apart from the experimental results, we further provide an in-depth look at how the semantic model behaves, along with its superb generalization ability in real-life examples. An RL-based image transmission extension is also exemplified, so as to prove the generalization ability and motivate future discussion.

cs.LG↗

arXiv2022

Communication-Efficient Consensus Mechanism for Federated Reinforcement Learning

The paper considers independent reinforcement learning (IRL) for multi-agent decision-making process in the paradigm of federated learning (FL). We show that FL can clearly improve the policy performance of IRL in terms of training efficiency and stability. However, since the policy parameters are trained locally and aggregated iteratively through a central server in FL, frequent information exchange incurs a large amount of communication overheads. To reach a good balance between improving the model's convergence performance and reducing the required communication and computation overheads, this paper proposes a system utility function and develops a consensus-based optimization scheme on top of the periodic averaging method, which introduces the consensus algorithm into FL for the exchange of a model's local gradients. This paper also provides novel convergence guarantees for the developed method, and demonstrates its superior effectiveness and efficiency in improving the system utility value through theoretical analyses and numerical simulation results.

cs.LG↗

arXiv2021

Semantic Communication with Adaptive Universal Transformer

With the development of deep learning (DL), natural language processing (NLP) makes it possible for us to analyze and understand a large amount of language texts. Accordingly, we can achieve a semantic communication in terms of joint semantic source and channel coding over a noisy channel with the help of NLP. However, the existing method to realize this goal is to use a fixed transformer of NLP while ignoring the difference of semantic information contained in each sentence. To solve this problem, we propose a new semantic communication system based on Universal Transformer. Compared with the traditional transformer, an adaptive circulation mechanism is introduced in the Universal Transformer. Through the introduction of the circulation mechanism, the new semantic communication system can be more flexible to transmit sentences with different semantic information, and achieve better end-to-end performance under various channel conditions.

cs.CL↗

arXiv2021

Graph Attention Network-based Multi-agent Reinforcement Learning for Slicing Resource Management in Dense Cellular Network

Network slicing (NS) management devotes to providing various services to meet distinct requirements over the same physical communication infrastructure and allocating resources on demands. Considering a dense cellular network scenario that contains several NS over multiple base stations (BSs), it remains challenging to design a proper real-time inter-slice resource management strategy, so as to cope with frequent BS handover and satisfy the fluctuations of distinct service requirements. In this paper, we propose to formulate this challenge as a multi-agent reinforcement learning (MARL) problem in which each BS represents an agent. Then, we leverage graph attention network (GAT) to strengthen the temporal and spatial cooperation between agents. Furthermore, we incorporate GAT into deep reinforcement learning (DRL) and correspondingly design an intelligent real-time inter-slice resource management strategy. More specially, we testify the universal effectiveness of GAT for advancing DRL in the multi-agent system, by applying GAT on the top of both the value-based method deep Q-network (DQN) and a combination of policy-based and value-based method advantage actor-critic (A2C). Finally, we verify the superiority of the GAT-based MARL algorithms through extensive simulations.

cs.MA↗

arXiv2021

Fundamental Limits of Reinforcement Learning in Environment with Endogeneous and Exogeneous Uncertainty

Online reinforcement learning (RL) has been widely applied in information processing scenarios, which usually exhibit much uncertainty due to the intrinsic randomness of channels and service demands. In this paper, we consider an un-discounted RL in general Markov decision processes (MDPs) with both endogeneous and exogeneous uncertainty, where both the rewards and state transition probability are unknown to the RL agent and evolve with the time as long as their respective variations do not exceed certain dynamic budget (i.e., upper bound). We first develop a variation-aware Bernstein-based upper confidence reinforcement learning (VB-UCRL), which we allow to restart according to a schedule dependent on the variations. We successfully overcome the challenges due to the exogeneous uncertainty and establish a regret bound of saving at most $\sqrt{S}$ or $S^{\frac{1}{6}}T^{\frac{1}{12}}$ compared with the latest results in the literature, where $S$ denotes the state size of the MDP and $T$ indicates the iteration index of learning steps.

cs.LG↗

arXiv2021

KGTK: A Toolkit for Large Knowledge Graph Manipulation and Analysis

Knowledge graphs (KGs) have become the preferred technology for representing, sharing and adding knowledge to modern AI applications. While KGs have become a mainstream technology, the RDF/SPARQL-centric toolset for operating with them at scale is heterogeneous, difficult to integrate and only covers a subset of the operations that are commonly needed in data science applications. In this paper we present KGTK, a data science-centric toolkit designed to represent, create, transform, enhance and analyze KGs. KGTK represents graphs in tables and leverages popular libraries developed for data science applications, enabling a wide audience of developers to easily construct knowledge graph pipelines for their applications. We illustrate the framework with real-world scenarios where we have used KGTK to integrate and manipulate large KGs, such as Wikidata, DBpedia and ConceptNet.

cs.AI↗

arXiv2021

Toward Native Artificial Intelligence in 6G Networks: System Design, Architectures, and Paradigms

The mobile communication system has transformed to be the fundamental infrastructure to support digital demands from all industry sectors, and 6G is envisioned to go far beyond the communication-only purpose. There is coming to a consensus that 6G will treat Artificial Intelligence (AI) as the cornerstone and has a potential capability to provide "intelligence inclusion", which implies to enable the access of AI services at anytime and anywhere by anyone. Apparently, the intelligent inclusion vision produces far-reaching influence on the corresponding network architecture design in 6G and deserves a clean-slate rethink. In this article, we propose an end-to-end system architecture design scope for 6G, and talk about the necessity to incorporate an independent data plane and a novel intelligent plane with particular emphasis on end-to-end AI workflow orchestration, management and operation. We also highlight the advantages to provision converged connectivity and computing services at the network function plane. Benefiting from these approaches, we believe that 6G will turn to an "everything as a service" (XaaS) platform with significantly enhanced business merits.

cs.NI↗

arXiv2021

Stigmergic Independent Reinforcement Learning for Multi-Agent Collaboration

With the rapid evolution of wireless mobile devices, there emerges an increased need to design effective collaboration mechanisms between intelligent agents, so as to gradually approach the final collective objective through continuously learning from the environment based on their individual observations. In this regard, independent reinforcement learning (IRL) is often deployed in multi-agent collaboration to alleviate the problem of a non-stationary learning environment. However, behavioral strategies of intelligent agents in IRL can only be formulated upon their local individual observations of the global environment, and appropriate communication mechanisms must be introduced to reduce their behavioral localities. In this paper, we address the problem of communication between intelligent agents in IRL by jointly adopting mechanisms with two different scales. For the large scale, we introduce the stigmergy mechanism as an indirect communication bridge between independent learning agents, and carefully design a mathematical method to indicate the impact of digital pheromone. For the small scale, we propose a conflict-avoidance mechanism between adjacent agents by implementing an additionally embedded neural network to provide more opportunities for participants with higher action priorities. In addition, we present a federal training method to effectively optimize the neural network of each agent in a decentralized manner. Finally, we establish a simulation scenario in which a number of mobile agents in a certain area move automatically to form a specified target shape. Extensive simulations demonstrate the effectiveness of our proposed method.

cs.AI↗

arXiv2020

Effect of Spatial and Temporal Traffic Statistics on the Performance of Wireless Networks

The traffic in wireless networks has become diverse and fluctuating both spatially and temporally due to the emergence of new wireless applications and the complexity of scenarios. The purpose of this paper is to quantitatively analyze the impact of the wireless traffic, which fluctuates both spatially and temporally, on the performance of the wireless networks. Specially, we propose to combine the tools from stochastic geometry and queueing theory to model the spatial and temporal fluctuation of traffic, which to our best knowledge has seldom been evaluated analytically. We derive the spatial and temporal statistics, the total arrival rate, the stability of queues and the delay of users by considering two different spatial properties of traffic, i.e., the uniformly and non-uniformly distributed cases. The numerical results indicate that although the fluctuation of traffic (reflected by the variance of total arrival rate) when the users are clustered is much fiercer than that when the users are uniformly distributed, the unstable probability is smaller. Our work provides a useful reference for the design of wireless networks when the complex spatio-temporal fluctuation of the traffic is considered.

cs.NI↗

arXiv2020

Learning to Prune in Training via Dynamic Channel Propagation

In this paper, we propose a novel network training mechanism called "dynamic channel propagation" to prune the neural networks during the training period. In particular, we pick up a specific group of channels in each convolutional layer to participate in the forward propagation in training time according to the significance level of channel, which is defined as channel utility. The utility values with respect to all selected channels are updated simultaneously with the error back-propagation process and will adaptively change. Furthermore, when the training ends, channels with high utility values are retained whereas those with low utility values are discarded. Hence, our proposed scheme trains and prunes neural networks simultaneously. We empirically evaluate our novel training scheme on various representative benchmark datasets and advanced convolutional neural network (CNN) architectures, including VGGNet and ResNet. The experiment results verify the superior performance and robust effectiveness of our approach.

cs.CV↗

arXiv2020

Internet of Intelligence: The Collective Advantage for Advancing Communications and Intelligence

The fifth-generation cellular networks (5G) has boosted the unprecedented convergence between the information world and physical world. On the other hand, empowered with the enormous amount of data and information, artificial intelligence (AI) has been universally applied and pervasive AI is believed to be an integral part of the six-generation cellular networks (6G). Consequently, benefiting from the advancement in communication technology and AI, we boldly argue that the conditions for collective intelligence (CI) will be mature in the 6G era and CI will emerge among the widely connected beings and things. Afterwards, we highlight the potential huge impact of CI on both communications and intelligence. In particular, we introduce a regular language (i.e., the information economy metalanguage) supporting the future collective communications to augment human intelligence and explain its potential applications in naming Internet information and pushing information centric networks forward. Meanwhile, we propose a stigmergy-based federated collective intelligence and demonstrate its achievement in a simulated scenario where the agents collectively work together to form a pattern through simple indirect communications. In a word, CI could advance both communications and intelligence.

cs.AI↗

arXiv2020

Domain-Specific Compilers for Dynamic Simulations of Quantum Materials on Quantum Computers

Simulation of the dynamics of quantum materials is emerging as a promising scientific application for noisy intermediate-scale quantum (NISQ) computers. Due to their high gate-error rates and short decoherence times, however, NISQ computers can only produce high-fidelity results for those quantum circuits smaller than some given circuit size. Dynamic simulations, therefore, pose a challenge as current algorithms produce circuits that grow in size with each subsequent time-step of the simulation. This underscores the crucial role of quantum circuit compilers to produce executable quantum circuits of minimal size, thereby maximizing the range of physical phenomena that can be studied within the NISQ fidelity budget. Here, we present two domain-specific quantum circuit compilers for the Rigetti and IBM quantum computers, specifically designed to compile circuits simulating dynamics under a special class of time-dependent Hamiltonians. The compilers outperform state-of-the-art general-purpose compilers in terms of circuit size reduction by around 25-30% as well as wall-clock compilation time by around 40% (dependent on system size and simulation time-step). Drawing on heuristic techniques commonly used in artificial intelligence, both compilers scale well with simulation time-step and system size. Code for both compilers is included to enhance the results of dynamic simulations for future researchers. We anticipate that our domain-specific compilers will enable dynamic simulations of quantum materials on near-future NISQ computers that would not otherwise be possible with general-purpose compilers.

quant-ph↗

arXiv2020

Intelligent Slicing of Radio Resource Control Layer for Cellular IoT: Design and Implementation

The cellular internet of things (CIoT) has become an important branch to cater various applications of IoT devices. Within CIoT, the radio resource control (RRC) layer is responsible for fundamental functionalities such as connection control and bearer establishment in radio access network (RAN). The emergence of various IoT scenarios and diversified service requirements have made both RAN slicing and intelligent control imperative requirement in RRC layer. This paper focuses on enhancing standardized capabilities of CIoT RRC layer, by designing and implementing a new architecture which accommodate RRC slicing and intelligent controller. The architecture aims to realize functionalities of creating, modifying, and deleting slices in RRC layer, while the intelligent controller is added to satisfy various and dynamic service requirements of different IoT devices smartly. The proposed architecture is further implemented on an open-source software platform OpenAirInterface (OAI), on top of which the effectiveness of RRC slicing is validated and one proof-of-concept case to adopt reinforcement learning to dynamically tune discontinuous reception parameters therein is presented. Simulation results have demonstrated the effectiveness of the proposed intelligent RRC slicing architecture.

cs.IT↗

arXiv2020

On the Capacity of Fractal D2D Social Networks with Hierarchical Communications

The maximum capacity of fractal D2D (device-to-device) social networks with both direct and hierarchical communications is studied in this paper. Specifically, the fractal networks are characterized by the direct social connection and the self-similarity. Firstly, for a fractal D2D social network with direct social communications, it is proved that the maximum capacity is $ Θ\left(\frac{1}{\sqrt{n\log n}}\right) $ if a user communicates with one of his/her direct contacts randomly, where $ n $ denotes the total number of users in the network, and it can reach up to $ Θ\left(\frac{1}{\log n}\right) $ if any pair of social contacts with distance $ d $ communicate according to the probability in proportion to $ d^{-β} $. Secondly, since users might get in touch with others without direct social connections through the inter-connected multiple users, the fractal D2D social network with these hierarchical communications is studied as well, and the related capacity is further derived. Our results show that this capacity is mainly affected by the correlation exponent $ε$ of the fractal structure. The capacity is reduced in proportional to $ \frac{1}{\log n} $ if $ 2<ε<3 $, while the reduction coefficient is $ \frac{1}{n} $ if $ ε>3 $.

cs.IT↗

arXiv2019

GAN-powered Deep Distributional Reinforcement Learning for Resource Management in Network Slicing

Network slicing is a key technology in 5G communications system. Its purpose is to dynamically and efficiently allocate resources for diversified services with distinct requirements over a common underlying physical infrastructure. Therein, demand-aware resource allocation is of significant importance to network slicing. In this paper, we consider a scenario that contains several slices in a radio access network with base stations that share the same physical resources (e.g., bandwidth or slots). We leverage deep reinforcement learning (DRL) to solve this problem by considering the varying service demands as the environment state and the allocated resources as the environment action. In order to reduce the effects of the annoying randomness and noise embedded in the received service level agreement (SLA) satisfaction ratio (SSR) and spectrum efficiency (SE), we primarily propose generative adversarial network-powered deep distributional Q network (GAN-DDQN) to learn the action-value distribution driven by minimizing the discrepancy between the estimated action-value distribution and the target action-value distribution. We put forward a reward-clipping mechanism to stabilize GAN-DDQN training against the effects of widely-spanning utility values. Moreover, we further develop Dueling GAN-DDQN, which uses a specially designed dueling generator, to learn the action-value distribution by estimating the state-value distribution and the action advantage function. Finally, we verify the performance of the proposed GAN-DDQN and Dueling GAN-DDQN algorithms through extensive simulations.

cs.LG↗

arXiv2019

Deep Reinforcement Learning with Discrete Normalized Advantage Functions for Resource Management in Network Slicing

Network slicing promises to provision diversified services with distinct requirements in one infrastructure. Deep reinforcement learning (e.g., deep $\mathcal{Q}$-learning, DQL) is assumed to be an appropriate algorithm to solve the demand-aware inter-slice resource management issue in network slicing by regarding the varying demands and the allocated bandwidth as the environment state and the action, respectively. However, allocating bandwidth in a finer resolution usually implies larger action space, and unfortunately DQL fails to quickly converge in this case. In this paper, we introduce discrete normalized advantage functions (DNAF) into DQL, by separating the $\mathcal{Q}$-value function as a state-value function term and an advantage term and exploiting a deterministic policy gradient descent (DPGD) algorithm to avoid the unnecessary calculation of $\mathcal{Q}$-value for every state-action pair. Furthermore, as DPGD only works in continuous action space, we embed a k-nearest neighbor algorithm into DQL to quickly find a valid action in the discrete space nearest to the DPGD output. Finally, we verify the faster convergence of the DNAF-based DQL through extensive simulations.

cs.LG↗

arXiv2019

Evaluation Mechanism of Collective Intelligence for Heterogeneous Agents Group

Collective intelligence is manifested when multiple agents coherently work in observation, interaction, decision-making and action. In this paper, we define and quantify the intelligence level of heterogeneous agents group with the improved Anytime Universal Intelligence Test(AUIT), based on an extension of the existing evaluation of homogeneous agents group. The relationship of intelligence level with agents composition, group size, spatial complexity and testing time is analyzed. The intelligence level of heterogeneous agents groups is compared with the homogeneous ones to analyze the effects of heterogeneity on collective intelligence. Our work will help to understand the essence of collective intelligence more deeply and reveal the effect of various key factors on group intelligence level.

cs.AI↗