arXiv subjects

Xiaoyang Li

Publications and source records attributed to Xiaoyang Li.

At least 73 records · Page 4Linked to original sources

arXiv2024

Integrated Sensing-Communication-Computation for Edge Artificial Intelligence

Edge artificial intelligence (AI) has been a promising solution towards 6G to empower a series of advanced techniques such as digital twins, holographic projection, semantic communications, and auto-driving, for achieving intelligence of everything. The performance of edge AI tasks, including edge learning and edge AI inference, depends on the quality of three highly coupled processes, i.e., sensing for data acquisition, computation for information extraction, and communication for information transmission. However, these three modules need to compete for network resources for enhancing their own quality-of-services. To this end, integrated sensing-communication-computation (ISCC) is of paramount significance for improving resource utilization as well as achieving the customized goals of edge AI tasks. By investigating the interplay among the three modules, this article presents various kinds of ISCC schemes for federated edge learning tasks and edge AI inference tasks in both application and physical layers.

cs.IT↗

arXiv2024

Finding the Missing Data: A BERT-inspired Approach Against Package Loss in Wireless Sensing

Despite the development of various deep learning methods for Wi-Fi sensing, package loss often results in noncontinuous estimation of the Channel State Information (CSI), which negatively impacts the performance of the learning models. To overcome this challenge, we propose a deep learning model based on Bidirectional Encoder Representations from Transformers (BERT) for CSI recovery, named CSI-BERT. CSI-BERT can be trained in an self-supervised manner on the target dataset without the need for additional data. Furthermore, unlike traditional interpolation methods that focus on one subcarrier at a time, CSI-BERT captures the sequential relationships across different subcarriers. Experimental results demonstrate that CSI-BERT achieves lower error rates and faster speed compared to traditional interpolation methods, even when facing with high loss rates. Moreover, by harnessing the recovered CSI obtained from CSI-BERT, other deep learning models like Residual Network and Recurrent Neural Network can achieve an average increase in accuracy of approximately 15\% in Wi-Fi sensing tasks. The collected dataset WiGesture and code for our model are publicly available at https://github.com/RS2002/CSI-BERT.

cs.LG↗

arXiv2024

Integrating Sensing, Communication, and Power Transfer: From Theory to Practice

To support the development of internet-of-things applications, an enormous population of low-power devices are expected to be incorporated in wireless networks performing sensing and communication tasks. As a key technology for improving the data collection efficiency, integrated sensing and communication (ISAC) enables simultaneous data transmission and radar sensing by reusing the same radio signals. In addition to information carriers, wireless signals can also serve as energy delivers, which enables simultaneous wireless information and power transfer (SWIPT). To improve the energy and spectrum efficiency, the advantages of ISAC and SWIPT are expected to be exploited, leading to the emerging technology of integrating sensing, communication, and power transfer (ISCPT). In this article, a timely overview of ISCPT is provided with the description of the fundamentals, the characterization of the theoretical boundary, the discussion on the key technologies, and the demonstration of the implementation platform.

cs.IT↗

arXiv2023

RdimKD: Generic Distillation Paradigm by Dimensionality Reduction

Knowledge Distillation (KD) emerges as one of the most promising compression technologies to run advanced deep neural networks on resource-limited devices. In order to train a small network (student) under the guidance of a large network (teacher), the intuitive method is regularizing the feature maps or logits of the student using the teacher's information. However, existing methods either over-restrict the student to learn all information from the teacher, which lead to some bad local minimum, or use various fancy and elaborate modules to process and align features, which are complex and lack generality. In this work, we proposed an abstract and general paradigm for the KD task, referred to as DIMensionality Reduction KD (RdimKD), which solely relies on dimensionality reduction, with a very minor modification to naive L2 loss. RdimKD straightforwardly utilizes a projection matrix to project both the teacher's and student's feature maps onto a low-dimensional subspace, which are then optimized during training. RdimKD achieves the goal in the simplest way that not only does the student get valuable information from the teacher, but it also ensures sufficient flexibility to adapt to the student's low-capacity reality. Our extensive empirical findings indicate the effectiveness of RdimKD across various learning tasks and diverse network architectures.

cs.LG↗

arXiv2023

Integrating Sensing, Communication, and Power Transfer: Multiuser Beamforming Design

In the sixth-generation (6G) networks, massive low-power devices are expected to sense environment and deliver tremendous data. To enhance the radio resource efficiency, the integrated sensing and communication (ISAC) technique exploits the sensing and communication functionalities of signals, while the simultaneous wireless information and power transfer (SWIPT) techniques utilizes the same signals as the carriers for both information and power delivery. The further combination of ISAC and SWIPT leads to the advanced technology namely integrated sensing, communication, and power transfer (ISCPT). In this paper, a multi-user multiple-input multiple-output (MIMO) ISCPT system is considered, where a base station equipped with multiple antennas transmits messages to multiple information receivers (IRs), transfers power to multiple energy receivers (ERs), and senses a target simultaneously. The sensing target can be regarded as a point or an extended surface. When the locations of IRs and ERs are separated, the MIMO beamforming designs are optimized to improve the sensing performance while meeting the communication and power transfer requirements. The resultant non-convex optimization problems are solved based on a series of techniques including Schur complement transformation and rank reduction. Moreover, when the IRs and ERs are co-located, the power splitting factors are jointly optimized together with the beamformers to balance the performance of communication and power transfer. To better understand the performance of ISCPT, the target positioning problem is further investigated. Simulations are conducted to verify the effectiveness of our proposed designs, which also reveal a performance tradeoff among sensing, communication, and power transfer.

cs.IT↗

arXiv2023

Beamforming Design for RIS-Aided THz Wideband Communication Systems

Benefiting from tens of GHz of bandwidth, terahertz (THz) communications has become a promising technology for future 6G networks. However, the conventional hybrid beamforming architecture based on frequency-independent phase-shifters is not able to cope with the beam split effect (BSE) in THz massive multiple-input multiple-output (MIMO) systems. Despite some work introducing the frequency-dependent phase shifts via the time delay network to mitigate the beam splitting in THz wideband communications, the corresponding issue in reconfigurable intelligent surface (RIS)-aided communications has not been well investigated. In this paper, the BSE in THz massive MIMO is quantified by analyzing the array gain loss. A new beamforming architecture has been proposed to mitigate this effect under RIS-aided communications scenarios. Simulations are performed to evaluate the effectiveness of the proposed system architecture in combating the array gain loss.

cs.IT↗

arXiv2023

Joint Beamforming and Power Allocation for RIS Aided Full-Duplex Integrated Sensing and Uplink Communication System

Integrated sensing and communication (ISAC) capability is envisioned as one key feature for future cellular networks. Classical half-duplex (HD) radar sensing is conducted in a "first-emit-then-listen" manner. One challenge to realize HD ISAC lies in the discrepancy of the two systems' time scheduling for transmitting and receiving. This difficulty can be overcome by full-duplex (FD) transceivers. Besides, ISAC generally has to comprise its communication rate due to realizing sensing functionality. This loss can be compensated by the emerging reconfigurable intelligent surface (RIS) technology. This paper considers the joint design of beamforming, power allocation and signal processing in a FD uplink communication system aided by RIS, which is a highly nonconvex problem. To resolve this challenge, via leveraging the cutting-the-edge majorization-minimization (MM) and penalty-dual-decomposition (PDD) methods, we develop an iterative solution that optimizes all variables via using convex optimization techniques. Besides, by wisely exploiting alternative direction method of multipliers (ADMM) and optimality analysis, we further develop a low complexity solution that updates all variables analytically and runs highly efficiently. Numerical results are provided to verify the effectiveness and efficiency of our proposed algorithms and demonstrate the significant performance boosting by employing RIS in the FD ISAC system.

cs.IT↗

arXiv2023

BiFSMNv2: Pushing Binary Neural Networks for Keyword Spotting to Real-Network Performance

Deep neural networks, such as the Deep-FSMN, have been widely studied for keyword spotting (KWS) applications while suffering expensive computation and storage. Therefore, network compression technologies like binarization are studied to deploy KWS models on edge. In this paper, we present a strong yet efficient binary neural network for KWS, namely BiFSMNv2, pushing it to the real-network accuracy performance. First, we present a Dual-scale Thinnable 1-bit-Architecture to recover the representation capability of the binarized computation units by dual-scale activation binarization and liberate the speedup potential from an overall architecture perspective. Second, we also construct a Frequency Independent Distillation scheme for KWS binarization-aware training, which distills the high and low-frequency components independently to mitigate the information mismatch between full-precision and binarized representations. Moreover, we propose the Learning Propagation Binarizer, a general and efficient binarizer that enables the forward and backward propagation of binary KWS networks to be continuously improved through learning. We implement and deploy the BiFSMNv2 on ARMv8 real-world hardware with a novel Fast Bitwise Computation Kernel, which is proposed to fully utilize registers and increase instruction throughput. Comprehensive experiments show our BiFSMNv2 outperforms existing binary networks for KWS by convincing margins across different datasets and achieves comparable accuracy with the full-precision networks (only a tiny 1.51% drop on Speech Commands V1-12). We highlight that benefiting from the compact architecture and optimized hardware kernel, BiFSMNv2 can achieve an impressive 25.1x speedup and 20.2x storage-saving on edge hardware.

cs.CL↗

arXiv2022

Over-the-Air Integrated Sensing, Communication, and Computation in IoT Networks

To facilitate the development of Internet of Things (IoT) services, tremendous IoT devices are deployed in the wireless network to collect and pass data to the server for further processing. Aiming at improving the data sensing and delivering efficiency, the integrated sensing and communication (ISAC) technique has been proposed to design dual-functional signals for both radar sensing and data communication. To accelerate the data processing, the function computation via signal transmission is enabled by over-the-air computation (AirComp), which is based on the analog-wave addition property in a multi-access channel. As a natural combination, the emerging technology namely over-the-air integrated sensing, communication, and computation (Air-ISCC) adopts both the promising performances of ISAC and AirComp to improve the spectrum efficiency and reduce latency by enabling simultaneous sensing, communication, and computation. In this article, we provide a promptly overview of Air-ISCC by introducing the fundamentals, discussing the advanced techniques, and identifying the applications.

cs.IT↗

arXiv2022

BiFSMN: Binary Neural Network for Keyword Spotting

The deep neural networks, such as the Deep-FSMN, have been widely studied for keyword spotting (KWS) applications. However, computational resources for these networks are significantly constrained since they usually run on-call on edge devices. In this paper, we present BiFSMN, an accurate and extreme-efficient binary neural network for KWS. We first construct a High-frequency Enhancement Distillation scheme for the binarization-aware training, which emphasizes the high-frequency information from the full-precision network's representation that is more crucial for the optimization of the binarized network. Then, to allow the instant and adaptive accuracy-efficiency trade-offs at runtime, we also propose a Thinnable Binarization Architecture to further liberate the acceleration potential of the binarized network from the topology perspective. Moreover, we implement a Fast Bitwise Computation Kernel for BiFSMN on ARMv8 devices which fully utilizes registers and increases instruction throughput to push the limit of deployment efficiency. Extensive experiments show that BiFSMN outperforms existing binarization methods by convincing margins on various datasets and is even comparable with the full-precision counterpart (e.g., less than 3% drop on Speech Commands V1-12). We highlight that benefiting from the thinnable architecture and the optimized 1-bit implementation, BiFSMN can achieve an impressive 22.3x speedup and 15.5x storage-saving on real-world edge hardware. Our code is released at https://github.com/htqin/BiFSMN.

cs.CL↗

arXiv2022

Joint Sensing and Communication-Rate Control for Energy Efficient Mobile Crowd Sensing

Driven by the rapid growth of Internet of Things applications, tremendous data need to be collected by sensors and uploaded to the servers for further process. As a promising solution, mobile crowd sensing enables controllable sensing and transmission processes for multiple types of data in a single device. In this paper, a typical user is considered that is required to sense and transmit data to a server, while it is assumed to remain busy and incapable of sensing data during an interval. An optimization problem is formulated to minimize the energy consumption of data sensing and transmission by controlling the sensing and transmission rates over time, subject to the constraints on the sensing data sizes, transmission data sizes, data casualty, and sensing busy time. This problem is highly challenging, due to the coupling between the rates as well as the existence of the busy time. To deal with this problem, we first show that it can be equivalently decomposed into two subproblems, corresponding to a search for the amount of data size that needs to be sensed before the busy time (referred to as the height), as well as the sensing and transmission rate control given the height. Next, we show that the latter problem can be efficiently solved by using the classical string-pulling method, while an efficient algorithm is proposed to progressively find the optimal height without the exhaustive search. Moreover, the solution approach is extended to a more complex scenario where there is a finite-size buffer at the server for receiving data. Last, simulations are conducted to evaluate the performance of the proposed design.

cs.IT↗

arXiv2022

Integrated Sensing, Communication, and Computation Over-the-Air: MIMO Beamforming Design

To support the unprecedented growth of the Internet of Things (IoT) applications, tremendous data need to be collected by the IoT devices and delivered to the server for further computation. By utilizing the same signals for both radar sensing and data communication, the integrated sensing and communication (ISAC) technique has broken the barriers between data collection and delivery in the physical layer. By exploiting the analog-wave addition in a multi-access channel, over-the-air computation (AirComp) enables function computation via transmissions in the physical layer. The promising performance of ISAC and AirComp motivates the current work on developing a framework called integrated sensing, communication, and computation over-the-air (ISCCO). The performance metrics of radar sensing and AirComp are evaluated by the mean squared errors of the estimated target response matrix and the received computation results, respectively. The design challenge of MIMO ISCCO lies in the joint optimization of beamformers for sensing, communication, and computation at both the IoT devices and the server, which results in a non-convex problem. To solve this problem, an algorithmic solution based on the technique of semidefinite relaxation is proposed. The use case of target location estimation based on ISCCO is demonstrated in simulation to show the performance superiority.

cs.IT↗

arXiv2021

Data Partition and Rate Control for Learning and Energy Efficient Edge Intelligence

The rapid development of artificial intelligence together with the powerful computation capabilities of the advanced edge servers make it possible to deploy learning tasks at the wireless network edge, which is dubbed as edge intelligence (EI). The communication bottleneck between the data resource and the server results in deteriorated learning performance as well as tremendous energy consumption. To tackle this challenge, we explore a new paradigm called learning-and-energy-efficient (LEE) EI, which simultaneously maximizes the learning accuracies and energy efficiencies of multiple tasks via data partition and rate control. Mathematically, this results in a multi-objective optimization problem. Moreover, the continuous varying rates over the whole transmission duration introduce infinite variables. To solve this complex problem, we consider the case with infinite server buffer capacity and one-shot data arrival at sensor. First, the number of variables are reduced to a finite level by exploiting the optimality of constant-rate transmission in each epoch. Second, the optimal solution is found by applying stratified sequencing or objectives merging. By assuming higher priority of learning efficiency in stratified sequencing, the closed form of optimal data partition is derived by the Lagrange method, while the optimal rate control is proved to have the structure of directional water filling (DWF), based on which a string-pulling (SP) algorithm is proposed to obtain the numerical values. The DWF structure of rate control is also proved to be optimal in objectives merging via weighted summation. By exploiting the optimal rate changing properties, the SP algorithm is further extended to account for the cases with limited server buffer capacity or bursty data arrival at sensor. The performance of the proposed design is examined by extensive experiments based on public datasets.

cs.IT↗

arXiv2021

Deploying Federated Learning in Large-Scale Cellular Networks: Spatial Convergence Analysis

The deployment of federated learning in a wireless network, called federated edge learning (FEEL), exploits low-latency access to distributed mobile data to efficiently train an AI model while preserving data privacy. In this work, we study the spatial (i.e., spatially averaged) learning performance of FEEL deployed in a large-scale cellular network with spatially random distributed devices. Both the schemes of digital and analog transmission are considered, providing support of error-free uploading and over-the-air aggregation of local model updates by devices. The derived spatial convergence rate for digital transmission is found to be constrained by a limited number of active devices regardless of device density and converges to the ground-true rate exponentially fast as the number grows. The population of active devices depends on network parameters such as processing gain and signal-to-interference threshold for decoding. On the other hand, the limit does not exist for uncoded analog transmission. In this case, the spatial convergence rate is slowed down due to the direct exposure of signals to the perturbation of inter-cell interference. Nevertheless, the effect diminishes when devices are dense as interference is averaged out by aggressive over-the-air aggregation. In terms of learning latency (in second), analog transmission is preferred to the digital scheme as the former dramatically reduces multi-access latency by enabling simultaneous access.

cs.IT↗

arXiv2020

Few Is Enough: Task-Augmented Active Meta-Learning for Brain Cell Classification

Deep Neural Networks (or DNNs) must constantly cope with distribution changes in the input data when the task of interest or the data collection protocol changes. Retraining a network from scratch to combat this issue poses a significant cost. Meta-learning aims to deliver an adaptive model that is sensitive to these underlying distribution changes, but requires many tasks during the meta-training process. In this paper, we propose a tAsk-auGmented actIve meta-LEarning (AGILE) method to efficiently adapt DNNs to new tasks by using a small number of training examples. AGILE combines a meta-learning algorithm with a novel task augmentation technique which we use to generate an initial adaptive model. It then uses Bayesian dropout uncertainty estimates to actively select the most difficult samples when updating the model to a new task. This allows AGILE to learn with fewer tasks and a few informative samples, achieving high performance with a limited dataset. We perform our experiments using the brain cell classification task and compare the results to a plain meta-learning model trained from scratch. We show that the proposed task-augmented meta-learning framework can learn to classify new cell types after a single gradient step with a limited number of training samples. We show that active learning with Bayesian uncertainty can further improve the performance when the number of training samples is extremely small. Using only 1% of the training data and a single update step, we achieved 90% accuracy on the new cell type classification task, a 50% points improvement over a state-of-the-art meta-learning algorithm.

cs.CV↗

arXiv2019

Joint Annotator-and-Spectrum Allocation in Wireless Networks for Crowd Labelling

The massive sensing data generated by Internet-of-Things will provide fuel for ubiquitous artificial intelligence (AI), automating the operations of our society ranging from transportation to healthcare. The realistic adoption of this technique however entails labelling of the enormous data prior to the training of AI models via supervised learning. To tackle this challenge, we explore a new perspective of wireless crowd labelling that is capable of downloading data to many imperfect mobile annotators for repetition labelling by exploiting multicasting in wireless networks. In this cross-disciplinary area, the integration of the rate-distortion theory and the principle of repetition labelling for accuracy improvement gives rise to a new tradeoff between radio-and-annotator resources under a constraint on labelling accuracy. Building on the tradeoff and aiming at maximizing the labelling throughput, this work focuses on the joint optimization of encoding rate, annotator clustering, and sub-channel allocation, which results in an NP-hard integer programming problem. To devise an efficient solution approach, we establish an optimal sequential annotator-clustering scheme based on the order of decreasing signal-to-noise ratios. Thereby, the optimal solution can be found by an efficient tree search. Next, the solution is simplified by applying truncated channel inversion. Alternatively, the optimization problem can be recognized as a knapsack problem, which can be efficiently solved in pseudo-polynomial time by means of dynamic programming. In addition, exact polices are derived for the annotators constrained and spectrum constrained cases. Last, simulation results demonstrate the significant throughput gains based on the optimal solution compared with decoupled allocation of the two types of resources.

cs.IT↗

arXiv2019

An Overview of Data-Importance Aware Radio Resource Management for Edge Machine Learning

The 5G network connecting billions of Internet-of-Things (IoT) devices will make it possible to harvest an enormous amount of real-time mobile data. Furthermore, the 5G virtualization architecture will enable cloud computing at the (network) edge. The availability of both rich data and computation power at the edge has motivated Internet companies to deploy artificial intelligence (AI) there, creating the hot area of edge-AI. Edge learning, the theme of this project, concerns training edge-AI models, which endow on IoT devices intelligence for responding to real-time events. However, the transmission of high-dimensional data from many edge devices to servers can result in excessive communication latency, creating a bottleneck for edge learning. Traditional wireless techniques deigned for only radio access are ineffective in tackling the challenge. Attempts to overcome the communication bottleneck has led to the development of a new class of techniques for intelligent radio resource management (RRM), called data-importance aware RRM. Their designs feature the interplay of active machine learning and wireless communication. Specifically, the metrics that measure data importance in active learning (e.g., classification uncertainty and data diversity) are applied to RRM for efficient acquisition of distributed data in wireless networks to train AI models at servers. This article aims at providing an introduction to the emerging area of importance-aware RRM. To this end, we will introduce the design principles, survey recent advancements in the area, discuss some design examples, and suggest some promising research opportunities.

cs.IT↗

arXiv2019

Attenuating Random Noise in Seismic Data by a Deep Learning Approach

In the geophysical field, seismic noise attenuation has been considered as a critical and long-standing problem, especially for the pre-stack data processing. Here, we propose a model to leverage the deep-learning model for this task. Rather than directly applying an existing de-noising model from ordinary images to the seismic data, we have designed a particular deep-learning model, based on residual neural networks. It is named as N2N-Seismic, which has a strong ability to recover the seismic signals back to intact condition with the preservation of primary signals. The proposed model, achieving with great success in attenuating noise, has been tested on two different seismic datasets. Several metrics show that our method outperforms conventional approaches in terms of Signal-to-Noise-Ratio, Mean-Squared-Error, Phase Spectrum, etc. Moreover, robust tests in terms of effectively removing random noise from any dataset with strong and weak noises have been extensively scrutinized in making sure that the proposed model is able to maintain a good level of adaptation while dealing with large variations of noise characteristics and intensities.

cs.LG↗