arXiv subjects

Zhisheng Niu

Publications and source records attributed to Zhisheng Niu.

At least 55 records · Page 3Linked to original sources

arXiv2022

Time-Correlated Sparsification for Efficient Over-the-Air Model Aggregation in Wireless Federated Learning

Federated edge learning (FEEL) is a promising distributed machine learning (ML) framework to drive edge intelligence applications. However, due to the dynamic wireless environments and the resource limitations of edge devices, communication becomes a major bottleneck. In this work, we propose time-correlated sparsification with hybrid aggregation (TCS-H) for communication-efficient FEEL, which exploits jointly the power of model compression and over-the-air computation. By exploiting the temporal correlations among model parameters, we construct a global sparsification mask, which is identical across devices, and thus enables efficient model aggregation over-the-air. Each device further constructs a local sparse vector to explore its own important parameters, which are aggregated via digital communication with orthogonal multiple access. We further design device scheduling and power allocation algorithms for TCS-H. Experiment results show that, under limited communication resources, TCS-H can achieve significantly higher accuracy compared to the conventional top-K sparsification with orthogonal model aggregation, with both i.i.d. and non-i.i.d. data distributions.

cs.IT↗

arXiv2022

Online V2X Scheduling for Raw-Level Cooperative Perception

Cooperative perception of connected vehicles comes to the rescue when the field of view restricts stand-alone intelligence. While raw-level cooperative perception preserves most information to guarantee accuracy, it is demanding in communication bandwidth and computation power. Therefore, it is important to schedule the most beneficial vehicle to share its sensor in terms of supplementary view and stable network connection. In this paper, we present a model of raw-level cooperative perception and formulate the energy minimization problem of sensor sharing scheduling as a variant of the Multi-Armed Bandit (MAB) problem. Specifically, volatility of the neighboring vehicles, heterogeneity of V2X channels, and the time-varying traffic context are taken into consideration. Then we propose an online learning-based algorithm with logarithmic performance loss, achieving a decent trade-off between exploration and exploitation. Simulation results under different scenarios indicate that the proposed algorithm quickly learns to schedule the optimal cooperative vehicle and saves more energy as compared to baseline algorithms.

cs.IT↗

arXiv2021

Coded Computation across Shared Heterogeneous Workers with Communication Delay

Distributed computing enables large-scale computation tasks to be processed over multiple workers in parallel. However, the randomness of communication and computation delays across workers causes the straggler effect, which may degrade the performance. Coded computation helps to mitigate the straggler effect, but the amount of redundant load and their assignment to the workers should be carefully optimized. In this work, we consider a multi-master heterogeneous-worker distributed computing scenario, where multiple matrix multiplication tasks are encoded and allocated to workers for parallel computation. The goal is to minimize the communication plus computation delay of the slowest task. We propose worker assignment, resource allocation and load allocation algorithms under both dedicated and fractional worker assignment policies, where each worker can process the encoded tasks of either a single master or multiple masters, respectively. Then, the non-convex delay minimization problem is solved by employing the Markov's inequality-based approximation, Karush-Kuhn-Tucker conditions, and successive convex approximation methods. Through extensive simulations, we show that the proposed algorithms can reduce the task completion delay compared to the benchmarks, and observe that dedicated and fractional worker assignment policies have different scopes of applications.

cs.DC↗

arXiv2021

Dynamic Scheduling for Over-the-Air Federated Edge Learning with Energy Constraints

Machine learning and wireless communication technologies are jointly facilitating an intelligent edge, where federated edge learning (FEEL) is a promising training framework. As wireless devices involved in FEEL are resource limited in terms of communication bandwidth, computing power and battery capacity, it is important to carefully schedule them to optimize the training performance. In this work, we consider an over-the-air FEEL system with analog gradient aggregation, and propose an energy-aware dynamic device scheduling algorithm to optimize the training performance under energy constraints of devices, where both communication energy for gradient aggregation and computation energy for local training are included. The consideration of computation energy makes dynamic scheduling challenging, as devices are scheduled before local training, but the communication energy for over-the-air aggregation depends on the l2-norm of local gradient, which is known after local training. We thus incorporate estimation methods into scheduling to predict the gradient norm. Taking the estimation error into account, we characterize the performance gap between the proposed algorithm and its offline counterpart. Experimental results show that, under a highly unbalanced local data distribution, the proposed algorithm can increase the accuracy by 4.9% on CIFAR-10 dataset compared with the myopic benchmark, while satisfying the energy constraints.

cs.LG↗

arXiv2020

Edge Learning with Timeliness Constraints: Challenges and Solutions

Future machine learning (ML) powered applications, such as autonomous driving and augmented reality, involve training and inference tasks with timeliness requirements and are communication and computation intensive, which demands for the edge learning framework. The real-time requirements drive us to go beyond accuracy for ML. In this article, we introduce the concept of timely edge learning, aiming to achieve accurate training and inference while minimizing the communication and computation delay. We discuss key challenges and propose corresponding solutions from data, model and resource management perspectives to meet the timeliness requirements. Particularly, for edge training, we argue that the total training delay rather than rounds should be considered, and propose data or model compression, and joint device scheduling and resource management schemes for both centralized training and federated learning systems. For edge inference, we explore the dependency between accuracy and delay for communication and computation, and propose dynamic data compression and flexible pruning schemes. Two case studies show that the timeliness performances, including the training accuracy under a given delay budget and the completion ratio of inference tasks within deadline, are highly improved with the proposed solutions.

cs.NI↗

arXiv2020

Joint Device Scheduling and Resource Allocation for Latency Constrained Wireless Federated Learning

In federated learning (FL), devices contribute to the global training by uploading their local model updates via wireless channels. Due to limited computation and communication resources, device scheduling is crucial to the convergence rate of FL. In this paper, we propose a joint device scheduling and resource allocation policy to maximize the model accuracy within a given total training time budget for latency constrained wireless FL. A lower bound on the reciprocal of the training performance loss, in terms of the number of training rounds and the number of scheduled devices per round, is derived. Based on the bound, the accuracy maximization problem is solved by decoupling it into two sub-problems. First, given the scheduled devices, the optimal bandwidth allocation suggests allocating more bandwidth to the devices with worse channel conditions or weaker computation capabilities. Then, a greedy device scheduling algorithm is introduced, which in each step selects the device consuming the least updating time obtained by the optimal bandwidth allocation, until the lower bound begins to increase, meaning that scheduling more devices will degrade the model accuracy. Experiments show that the proposed policy outperforms state-of-the-art scheduling policies under extensive settings of data distributions and cell radius.

cs.IT↗

arXiv2020

Distributed Task Replication for Vehicular Edge Computing: Performance Analysis and Learning-based Algorithm

In a vehicular edge computing (VEC) system, vehicles can share their surplus computation resources to provide cloud computing services. The highly dynamic environment of the vehicular network makes it challenging to guarantee the task offloading delay. To this end, we introduce task replication to the VEC system, where the replicas of a task are offloaded to multiple vehicles at the same time, and the task is completed upon the first response among replicas. First, the impact of the number of task replicas on the offloading delay is characterized, and the optimal number of task replicas is approximated in closed-form. Based on the analytical result, we design a learning-based task replication algorithm (LTRA) with combinatorial multi-armed bandit theory, which works in a distributed manner and can automatically adapt itself to the dynamics of the VEC system. A realistic traffic scenario is used to evaluate the delay performance of the proposed algorithm. Results show that, under our simulation settings, LTRA with an optimized number of task replicas can reduce the average offloading delay by over 30% compared to the benchmark without task replication, and at the same time can improve the task completion ratio from 97% to 99.6%.

cs.NI↗

arXiv2020

Urgency of Information for Context-Aware Timely Status Updates in Remote Control Systems

As 5G and Internet-of-Things (IoT) are deeply integrated into vertical industries such as autonomous driving and industrial robotics, timely status update is crucial for remote monitoring and control. In this regard, Age of Information (AoI) has been proposed to measure the freshness of status updates. However, it is just a metric changing linearly with time and irrelevant of context-awareness. We propose a context-based metric, named as Urgency of Information (UoI), to measure the nonlinear time-varying importance and the non-uniform context-dependence of the status information. This paper first establishes a theoretical framework for UoI characterization and then provides UoI-optimal status updating and user scheduling schemes in both single-terminal and multi-terminal cases. Specifically, an update-index-based scheme is proposed for a single-terminal system, where the terminal always updates and transmits when its update index is larger than a threshold. For the multi-terminal case, the UoI of the proposed scheduling scheme is proven to be upper-bounded and its decentralized implementation by Carrier Sensing Multiple Access with Collision Avoidance (CSMA/CA) is also provided. In the simulations, the proposed updating and scheduling schemes notably outperform the existing ones such as round robin and AoI-optimal schemes in terms of UoI, error-bound violation and control system stability.

cs.IT↗

arXiv2020

Beyond Age: Urgency of Information for Timeliness Guarantee in Status Update Systems

Timely status updating is crucial for future applications that involve remote monitoring and control, such as autonomous driving and Industrial Internet of Things (IIoT). Age of Information (AoI) has been proposed to measure the freshness of status updates. However, it is incapable of capturing critical systematic context information that indicates the time-varying importance of status information, and the dynamic evolution of status. In this paper, we propose a context-based metric, namely the Urgency of Information (UoI), to evaluate the timeliness of status updates. Compared to AoI, the new metric incorporates both time-varying context information and dynamic status evolution, which enables the analysis on context-based adaptive status update schemes, as well as more effective remote monitoring and control. The minimization of average UoI for a status update terminal with an updating frequency constraint is investigated, and an update-index-based adaptive scheme is proposed. Simulation results show that the proposed scheme achieves a near-optimal performance with a low computational complexity.

cs.IT↗

arXiv2019

Channel Fingerprint Based Beam Tracking for Millimeter Wave Communications

Beamforming structures with fixed beam codebooks provide economical solutions for millimeter wave (mmWave) communications due to the low hardware cost. However, the training overhead to search for the optimal beamforming configuration is proportional to the codebook size. To improve the efficiency of beam tracking, we propose a beam tracking scheme based on the channel fingerprint database, which comprises mappings between statistical beamforming gains and user locations. The scheme tracks user movement by utilizing the trained beam configurations and estimating the gains of beam configurations that are not trained. Simulations show that the proposed scheme achieves significant beamforming performance gains over existing beam tracking schemes.

cs.IT↗

arXiv2019

AI-Assisted Low Information Latency Wireless Networking

The 5G Phase-2 and beyond wireless systems will focus more on vertical applications such as autonomous driving and industrial Internet-of-things, many of which are categorized as ultra-Reliable Low-Latency Communications (uRLLC). In this article, an alternative view on uRLLC is presented, that information latency, which measures the distortion of information resulted from time lag of its acquisition process, is more relevant than conventional communication latency of uRLLC in wireless networked control systems. An AI-assisted Situationally-aware Multi-Agent Reinforcement learning framework for wireless neTworks (SMART) is presented to address the information latency optimization challenge. Case studies of typical applications in Autonomous Driving (AD) are demonstrated, i.e., dense platooning and intersection management, which show that SMART can effectively optimize information latency, and more importantly, information latency-optimized systems outperform conventional uRLLC-oriented systems significantly in terms of AD performance such as traffic efficiency, thus pointing out a new research and system design paradigm.

cs.IT↗

arXiv2019

Device Scheduling with Fast Convergence for Wireless Federated Learning

Owing to the increasing need for massive data analysis and model training at the network edge, as well as the rising concerns about the data privacy, a new distributed training framework called federated learning (FL) has emerged. In each iteration of FL (called round), the edge devices update local models based on their own data and contribute to the global training by uploading the model updates via wireless channels. Due to the limited spectrum resources, only a portion of the devices can be scheduled in each round. While most of the existing work on scheduling focuses on the convergence of FL w.r.t. rounds, the convergence performance under a total training time budget is not yet explored. In this paper, a joint bandwidth allocation and scheduling problem is formulated to capture the long-term convergence performance of FL, and is solved by being decoupled into two sub-problems. For the bandwidth allocation sub-problem, the derived optimal solution suggests to allocate more bandwidth to the devices with worse channel conditions or weaker computation capabilities. For the device scheduling sub-problem, by revealing the trade-off between the number of rounds required to attain a certain model accuracy and the latency per round, a greedy policy is inspired, that continuously selects the device that consumes the least time in model updating until achieving a good trade-off between the learning efficiency and latency per round. The experiments show that the proposed policy outperforms other state-of-the-art scheduling policies, with the best achievable model accuracy under training time budgets.

cs.NI↗

arXiv2019

Closed-Form Whittle's Index-Enabled Random Access for Timely Status Update

We consider a star-topology wireless network for status update where a central node collects status data from a large number of distributed machine-type terminals that share a wireless medium. The Age of Information (AoI) minimization scheduling problem is formulated by the restless multi-armed bandit. A widely-proven near-optimal solution, i.e., the Whittle's index, is derived in closed-form and the corresponding indexability is established. The index is then generalized to incorporate stochastic, periodic packet arrivals and unreliable channels. Inspired by the index scheduling policies which achieve near-optimal AoI but require heavy signaling overhead, a contention-based random access scheme, namely Index-Prioritized Random Access (IPRA), is further proposed. Based on IPRA, terminals that are not urgent to update, indicated by their indices, are barred access to the wireless medium, thus improving the access timeliness. A computer-based simulation shows that IPRA's performance is close to the optimal AoI in this setting and outperforms standard random access schemes. Also, for applications with hard AoI deadlines, we provide reliable deadline guarantee analysis. Closed-form achievable AoI stationary distributions under Bernoulli packet arrivals are derived such that AoI deadline with high reliability can be ensured by calculating the maximum number of supportable terminals and allocating system resources proportionally.

cs.NI↗

arXiv2019

Context-Aware Information Lapse for Timely Status Updates in Remote Control Systems

Emerging applications in Internet of Things (IoT), such as remote monitoring and control, extensively rely on timely status updates. Age of Information (AoI) has been proposed to characterize the freshness of information in status update systems. However, it only considers the time elapsed since the generation of the latest packet, and is incapable of capturing other critical information in remote control systems, such as the stochastic evolution and the importance of the source status. In order to evaluate the timeliness of status updates in remote control systems, we propose a context-aware metric, namely the context-aware information lapse. The context-aware information lapse characterizes both the stochastic evolution of the source status and the context-aware importance of the status. In this paper, the minimization of average context-aware lapse in a multi-user system is considered, and a corresponding user scheduling policy is proposed based on Lyapunov optimization. Numerical results show that compared to AoI-based policy, the context-aware-lapse-based policy can achieve a substantial improvement in terms of error-threshold violation probability and control performance.

cs.IT↗

arXiv2019

Closed-Form Analysis of Non-Linear Age-of-Information in Status Updates with an Energy Harvesting Transmitter

Timely status updates are crucial to enabling applications in massive Internet of Things (IoT). This paper measures the data-freshness performance of a status update system with an energy harvesting transmitter, considering the randomness in information generation, transmission and energy harvesting. The performance is evaluated by a non-linear function of age of information (AoI) that is defined as the time elapsed since the generation of the most up-to-date status information at the receiver. The system is formulated as two queues with status packet generation and energy arrivals both assumed to be Poisson processes. With negligible service time, both First-Come-First-Served (FCFS) and Last-Come-First-Served (LCFS) disciplines for arbitrary buffer and battery capacities are considered, and a method for calculating the average penalty with non-linear penalty functions is proposed. The average AoI, the average penalty under exponential penalty function, and AoI's threshold violation probability are obtained in closed form. When the service time is assumed to follow exponential distribution, matrix geometric method is used to obtain the average peak AoI. The results illustrate that under the FCFS discipline, the status update frequency needs to be carefully chosen according to the service rate and energy arrival rate in order to minimize the average penalty.

cs.IT↗

arXiv2019

Optimizing Information Freshness in Broadcast Network with Unreliable Links and Random Arrivals: An Approximate Index Policy

With the rapid growth of real-time Internet of Things (IoT) applications, the need for fresh information has surged. Age of Information (AoI) is a tailor-made metric to characterize the information freshness perceived by the devices. In this paper, we investigate the problem of scheduling updates to minimize AoI in broadcast network. In this case, a central controller, e.g. a base station, collects status updates from different sources and schedules them to corresponding clients. Different from previous work, we consider both stochastic status updates and unreliable links. The problem is first modeled as an infinite horizon average constrained cost Markov decision problem (CMDP). With Lagrangian relaxation, an approximation of Whittle's index is derived and a scheduling policy is designed based on the approximate index. The results in previous work can be view as degenerate cases of the approximation index policy either with reliable links or periodic arrival constraints. Simulation results demonstrate the near-optimal performance of the proposed policy.

cs.IT↗

arXiv2019

Improving Device-Edge Cooperative Inference of Deep Learning via 2-Step Pruning

Deep neural networks (DNNs) are state-of-the-art solutions for many machine learning applications, and have been widely used on mobile devices. Running DNNs on resource-constrained mobile devices often requires the help from edge servers via computation offloading. However, offloading through a bandwidth-limited wireless link is non-trivial due to the tight interplay between the computation resources on mobile devices and wireless resources. Existing studies have focused on cooperative inference where DNN models are partitioned at different neural network layers, and the two parts are executed at the mobile device and the edge server, respectively. Since the output data size of a DNN layer can be larger than that of the raw data, offloading intermediate data between layers can suffer from high transmission latency under limited wireless bandwidth. In this paper, we propose an efficient and flexible 2-step pruning framework for DNN partition between mobile devices and edge servers. In our framework, the DNN model only needs to be pruned once in the training phase where unimportant convolutional filters are removed iteratively. By limiting the pruning region, our framework can greatly reduce either the wireless transmission workload of the device or the total computation workload. A series of pruned models are generated in the training phase, from which the framework can automatically select to satisfy varying latency and accuracy requirements. Furthermore, coding for the intermediate data is added to provide extra transmission workload reduction. Our experiments show that the proposed framework can achieve up to 25.6$\times$ reduction on transmission workload, 6.01$\times$ acceleration on total computation and 4.81$\times$ reduction on end-to-end latency as compared to partitioning the original DNN model without pruning.

cs.NI↗

arXiv2019

Distributed Policy Learning Based Random Access for Diversified QoS Requirements

Future wireless access networks need to support diversified quality of service (QoS) metrics required by various types of Internet-of-Things (IoT) devices, e.g., age of information (AoI) for status generating sources and ultra low latency for safety information in vehicular networks. In this paper, a novel inner-state driven random access (ISDA) framework is proposed based on distributed policy learning, in particular a cross-entropy method. Conventional random access schemes, e.g., $p$-CSMA, assume state-less terminals, and thus assigning equal priorities to all. In ISDA, the inner-states of terminals are described by a time-varying state vector, and the transmission probabilities of terminals in the contention period are determined by their respective inner-states. Neural networks are leveraged to approximate the function mappings from inner-states to transmission probabilities, and an iterative approach is adopted to improve these mappings in a distributed manner. Experiment results show that ISDA can improve the QoS of heterogeneous terminals simultaneously compared to conventional CSMA schemes.

cs.IT↗