arXiv subjects

Shugong Xu

Publications and source records attributed to Shugong Xu.

At least 73 records · Page 4Linked to original sources

arXiv2024

Quality of Experience Oriented Cross-layer Optimization for Real-time XR Video Transmission

Extended reality (XR) is one of the most important applications of beyond 5G and 6G networks. Real-time XR video transmission presents challenges in terms of data rate and delay. In particular, the frame-by-frame transmission mode of XR video makes real-time XR video very sensitive to dynamic network environments. To improve the users' quality of experience (QoE), we design a cross-layer transmission framework for real-time XR video. The proposed framework allows the simple information exchange between the base station (BS) and the XR server, which assists in adaptive bitrate and wireless resource scheduling. We utilize the cross-layer information to formulate the problem of maximizing user QoE by finding the optimal scheduling and bitrate adjustment strategies. To address the issue of mismatched time scales between two strategies, we decouple the original problem and solve them individually using a multi-agent-based approach. Specifically, we propose the multi-step Deep Q-network (MS-DQN) algorithm to obtain a frame-priority-based wireless resource scheduling strategy and then propose the Transformer-based Proximal Policy Optimization (TPPO) algorithm for video bitrate adaptation. The experimental results show that the TPPO+MS-DQN algorithm proposed in this study can improve the QoE by 3.6% to 37.8%. More specifically, the proposed MS-DQN algorithm enhances the transmission quality by 49.9%-80.2%.

cs.NI↗

arXiv2024

Real-time Extended Reality Video Transmission Optimization Based on Frame-priority Scheduling

Extended reality (XR) is one of the most important applications of 5G. For real-time XR video transmission in 5G networks, a low latency and high data rate are required. In this paper, we propose a resource allocation scheme based on frame-priority scheduling to meet these requirements. The optimization problem is modelled as a frame-priority-based radio resource scheduling problem to improve transmission quality. We propose a scheduling framework based on multi-step Deep Q-network (MS-DQN) and design a neural network model based on convolutional neural network (CNN). Simulation results show that the scheduling framework based on frame-priority and MS-DQN can improve transmission quality by 49.9%-80.2%.

cs.NI↗

arXiv2023

On the performance of an integrated communication and localization system: an analytical framework

Quantifying the performance bound of an integrated localization and communication (ILAC) system and the trade-off between communication and localization performance is critical. In this letter, we consider an ILAC system that can perform communication and localization via time-domain or frequency-domain resource allocation. We develop an analytical framework to derive the closed-form expression of the capacity loss versus localization Cramer-Rao lower bound (CRB) loss via time-domain and frequency-domain resource allocation. Simulation results validate the analytical model and demonstrate that frequency-domain resource allocation is preferable in scenarios with a smaller number of antennas at the next generation nodeB (gNB) and a larger distance between user equipment (UE) and gNB, while time-domain resource allocation is preferable in scenarios with a larger number of antennas and smaller distance between UE and the gNB.

cs.IT↗

arXiv2023

Joint Optimization of Video-based AI Inference Tasks in MEC-assisted Augmented Reality Systems

The high computational complexity and energy consumption of artificial intelligence (AI) algorithms hinder their application in augmented reality (AR) systems. However, mobile edge computing (MEC) makes it possible to solve this problem. This paper considers the scene of completing video-based AI inference tasks in the MEC system. We formulate a mixed-integer nonlinear programming problem (MINLP) to reduce inference delays, energy consumption and to improve recognition accuracy. We give a simplified expression of the inference complexity model and accuracy model through derivation and experimentation. The problem is then solved iteratively by using alternating optimization. Specifically, by assuming that the offloading decision is given, the problem is decoupled into two sub-problems, i.e., the resource allocation problem for the devices set that completes the inference tasks locally, and that for the devices set that offloads tasks. For the problem of offloading decision optimization, we propose a Channel-Aware heuristic algorithm. To further reduce the complexity, we propose an alternating direction method of multipliers (ADMM) based distributed algorithm. The ADMM-based algorithm has a low computational complexity that grows linearly with the number of devices. Numerical experiments show the effectiveness of proposed algorithms. The trade-off relationship between delay, energy consumption, and accuracy is also analyzed.

cs.NI↗

arXiv2022

High-Performance Transmission Mechanism Design of Multi-Stream Carrier Aggregation for 5G Non-Standalone Network

Multi-stream carrier aggregation is a key technology to expand bandwidth and improve the throughput of the fifth-generation wireless communication systems. However, due to the diversified propagation properties of different frequency bands, the traffic migration task is much more challenging, especially in hybrid sub-6 GHz and millimeter wave bands scenario. Existing schemes either neglected to consider the transmission rate difference between multi-stream carrier, or only consider simple low mobility scenario. In this paper, we propose a low-complexity traffic splitting algorithm based on fuzzy proportional integral derivative control mechanism. The proposed algorithm only relies on the local radio link control buffer information of sub-6 GHz and mmWave bands, while frequent feedback from user equipment (UE) side is minimized. As shown in the numerical examples, the proposed traffic splitting mechanism can achieve more than 90% link resource utilization ratio for different UE transmission requirements with different mobilities, which corresponds to 10% improvement if compared with conventional baselines.

eess.SY↗

arXiv2022

Prediction-based Hybrid Slicing Framework for Service Level Agreement Guarantee in Mobility Scenarios: A Deep Learning Approach

Network slicing is a critical driver for guaranteeing the diverse service level agreements (SLA) in 5G and future networks. Inter-slice radio resource allocation (IS-RRA) in the radio access network (RAN) is very important. However, user mobility brings new challenges for optimal IS-RRA. This paper first proposes a soft and hard hybrid slicing framework where a common slice is introduced to realize a trade-off between isolation and spectrum efficiency (SE). To address the challenges posed by user mobility, we propose a two-step deep learning-based algorithm: joint long short-term memory (LSTM)-based network state prediction and deep Q network (DQN)-based slicing strategy. In the proposal, LSTM networks are employed to predict traffic demand and the location of each user in a slicing window level. Moreover, channel gain is mapped by location and a radio map. Then, the predicted channel gain and traffic demand are input to the DQN to output the precise slicing adjustment. Finally, experiment results confirm the effectiveness of our proposed slicing framework: the slices' SLA can be guaranteed well, and the proposed algorithm can achieve near-optimal performance in terms of the SLA satisfaction ratio, isolation degree and SE.

eess.SY↗

arXiv2022

Joint Optimization of DNN Inference Delay and Energy under Accuracy Constraints for AR Applications

The high computational complexity and high energy consumption of artificial intelligence (AI) algorithms hinder their application in augmented reality (AR) systems. This paper considers the scene of completing video-based AI inference tasks in the mobile edge computing (MEC) system. We use multiply-and-accumulate operations (MACs) for problem analysis and optimize delay and energy consumption under accuracy constraints. To solve this problem, we first assume that offloading policy is known and decouple the problem into two subproblems. After solving these two subproblems, we propose an iterative-based scheduling algorithm to obtain the optimal offloading policy. We also experimentally discuss the relationship between delay, energy consumption, and inference accuracy.

eess.SY↗

arXiv2022

A Hard and Soft Hybrid Slicing Framework for Service Level Agreement Guarantee via Deep Reinforcement Learning

Network slicing is a critical driver for guaranteeing the diverse service level agreements (SLA) in 5G and future networks. Recently, deep reinforcement learning (DRL) has been widely utilized for resource allocation in network slicing. However, existing related works do not consider the performance loss associated with the initial exploration phase of DRL. This paper proposes a new performance-guaranteed slicing strategy with a soft and hard hybrid slicing setting. Mainly, a common slice setting is applied to guarantee slices' SLA when training the neural network. Moreover, the resource of the common slice tends to precisely redistribute to slices with the training of DRL until it converges. Furthermore, experiment results confirm the effectiveness of our proposed slicing framework: the slices' SLA of the training phase can be guaranteed, and the proposed algorithm can achieve the near-optimal performance in terms of the SLA satisfaction ratio, isolation degree and spectrum maximization after convergence.

cs.NI↗

arXiv2021

TRIG: Transformer-Based Text Recognizer with Initial Embedding Guidance

Scene text recognition (STR) is an important bridge between images and text, attracting abundant research attention. While convolutional neural networks (CNNS) have achieved remarkable progress in this task, most of the existing works need an extra module (context modeling module) to help CNN to capture global dependencies to solve the inductive bias and strengthen the relationship between text features. Recently, the transformer has been proposed as a promising network for global context modeling by self-attention mechanism, but one of the main shortcomings, when applied to recognition, is the efficiency. We propose a 1-D split to address the challenges of complexity and replace the CNN with the transformer encoder to reduce the need for a context modeling module. Furthermore, recent methods use a frozen initial embedding to guide the decoder to decode the features to text, leading to a loss of accuracy. We propose to use a learnable initial embedding learned from the transformer encoder to make it adaptive to different input images. Above all, we introduce a novel architecture for text recognition, named TRansformer-based text recognizer with Initial embedding Guidance (TRIG), composed of three stages (transformation, feature extraction, and prediction). Extensive experiments show that our approach can achieve state-of-the-art on text recognition benchmarks.

cs.CV↗

arXiv2021

Multi-Task and Multi-Modal Learning for RGB Dynamic Gesture Recognition

Gesture recognition is getting more and more popular due to various application possibilities in human-machine interaction. Existing multi-modal gesture recognition systems take multi-modal data as input to improve accuracy, but such methods require more modality sensors, which will greatly limit their application scenarios. Therefore we propose an end-to-end multi-task learning framework in training 2D convolutional neural networks. The framework can use the depth modality to improve accuracy during training and save costs by using only RGB modality during inference. Our framework is trained to learn a representation for multi-task learning: gesture segmentation and gesture recognition. Depth modality contains the prior information for the location of the gesture. Therefore it can be used as the supervision for gesture segmentation. A plug-and-play module named Multi-Scale-Decoder is designed to realize gesture segmentation, which contains two sub-decoder. It is used in the lower stage and higher stage respectively, and can help the network pay attention to key target areas, ignore irrelevant information, and extract more discriminant features. Additionally, the MSD module and depth modality are only used in the training stage to improve gesture recognition performance. Only RGB modality and network without MSD are required during inference. Experimental results on three public gesture recognition datasets show that our proposed method provides superior performance compared with existing gesture recognition frameworks. Moreover, using the proposed plug-and-play MSD in other 2D CNN-based frameworks also get an excellent accuracy improvement.

cs.CV↗

arXiv2021

HENet: Forcing a Network to Think More for Font Recognition

Although lots of progress were made in Text Recognition/OCR in recent years, the task of font recognition is remaining challenging. The main challenge lies in the subtle difference between these similar fonts, which is hard to distinguish. This paper proposes a novel font recognizer with a pluggable module solving the font recognition task. The pluggable module hides the most discriminative accessible features and forces the network to consider other complicated features to solve the hard examples of similar fonts, called HE Block. Compared with the available public font recognition systems, our proposed method does not require any interactions at the inference stage. Extensive experiments demonstrate that HENet achieves encouraging performance, including on character-level dataset Explor_all and word-level dataset AdobeVFR

cs.CV↗

arXiv2021

Deep Image Matting with Flexible Guidance Input

Image matting is an important computer vision problem. Many existing matting methods require a hand-made trimap to provide auxiliary information, which is very expensive and limits the real world usage. Recently, some trimap-free methods have been proposed, which completely get rid of any user input. However, their performance lag far behind trimap-based methods due to the lack of guidance information. In this paper, we propose a matting method that use Flexible Guidance Input as user hint, which means our method can use trimap, scribblemap or clickmap as guidance information or even work without any guidance input. To achieve this, we propose Progressive Trimap Deformation(PTD) scheme that gradually shrink the area of the foreground and background of the trimap with the training step increases and finally become a scribblemap. To make our network robust to any user scribble and click, we randomly sample points on foreground and background and perform curve fitting. Moreover, we propose Semantic Fusion Module(SFM) which utilize the Feature Pyramid Enhancement Module(FPEM) and Joint Pyramid Upsampling(JPU) in matting task for the first time. The experiments show that our method can achieve state-of-the-art results comparing with existing trimap-based and trimap-free methods.

cs.CV↗

arXiv2021

High Precision Indoor Localization with Dummy Antennas -- An Experimental Study

With the rising demand for indoor localization, high precision technique-based fingerprints became increasingly important nowadays. The newest advanced localization system makes effort to improve localization accuracy in the time or frequency domain, for example, the UWB localization technique can achieve centimeter-level accuracy but have a high cost. Therefore, we present a spatial domain extension-based scheme with low cost and verify the effectiveness of antennas extension in localization accuracy. In this paper, we achieve sub-meter level localization accuracy using a single AP by extending three radio links of the modified laptops to more antennas. Moreover, the experimental results show that the localization performance is superior as the number of antennas increases with the help of spatial domain extension and angular domain assisted.

eess.SP↗

arXiv2021

RW-Resnet: A Novel Speech Anti-Spoofing Model Using Raw Waveform

In recent years, synthetic speech generated by advanced text-to-speech (TTS) and voice conversion (VC) systems has caused great harms to automatic speaker verification (ASV) systems, urging us to design a synthetic speech detection system to protect ASV systems. In this paper, we propose a new speech anti-spoofing model named ResWavegram-Resnet (RW-Resnet). The model contains two parts, Conv1D Resblocks and backbone Resnet34. The Conv1D Resblock is based on the Conv1D block with a residual connection. For the first part, we use the raw waveform as input and feed it to the stacked Conv1D Resblocks to get the ResWavegram. Compared with traditional methods, ResWavegram keeps all the information from the audio signal and has a stronger ability in extracting features. For the second part, the extracted features are fed to the backbone Resnet34 for the spoofed or bonafide decision. The ASVspoof2019 logical access (LA) corpus is used to evaluate our proposed RW-Resnet. Experimental results show that the RW-Resnet achieves better performance than other state-of-the-art anti-spoofing models, which illustrates its effectiveness in detecting synthetic speech attacks.

cs.SD↗

arXiv2021

IFR: Iterative Fusion Based Recognizer For Low Quality Scene Text Recognition

Although recent works based on deep learning have made progress in improving recognition accuracy on scene text recognition, how to handle low-quality text images in end-to-end deep networks remains a research challenge. In this paper, we propose an Iterative Fusion based Recognizer (IFR) for low quality scene text recognition, taking advantage of refined text images input and robust feature representation. IFR contains two branches which focus on scene text recognition and low quality scene text image recovery respectively. We utilize an iterative collaboration between two branches, which can effectively alleviate the impact of low quality input. A feature fusion module is proposed to strengthen the feature representation of the two branches, where the features from the Recognizer are Fused with image Restoration branch, referred to as RRF. Without changing the recognition network structure, extensive quantitative and qualitative experimental results show that the proposed method significantly outperforms the baseline methods in boosting the recognition accuracy of benchmark datasets and low resolution images in TextZoom dataset.

cs.CV↗

arXiv2021

SGTBN: Generating Dense Depth Maps from Single-Line LiDAR

Depth completion aims to generate a dense depth map from the sparse depth map and aligned RGB image. However, current depth completion methods use extremely expensive 64-line LiDAR(about $100,000) to obtain sparse depth maps, which will limit their application scenarios. Compared with the 64-line LiDAR, the single-line LiDAR is much less expensive and much more robust. Therefore, we propose a method to tackle the problem of single-line depth completion, in which we aim to generate a dense depth map from the single-line LiDAR info and the aligned RGB image. A single-line depth completion dataset is proposed based on the existing 64-line depth completion dataset(KITTI). A network called Semantic Guided Two-Branch Network(SGTBN) which contains global and local branches to extract and fuse global and local info is proposed for this task. A Semantic guided depth upsampling module is used in our network to make full use of the semantic info in RGB images. Except for the usual MSE loss, we add the virtual normal loss to increase the constraint of high-order 3D geometry in our network. Our network outperforms the state-of-the-art in the single-line depth completion task. Besides, compared with the monocular depth estimation, our method also has significant advantages in precision and model size.

cs.CV↗

arXiv2021

A Novel GCN based Indoor Localization System with Multiple Access Points

With the rapid development of indoor location-based services (LBSs), the demand for accurate localization keeps growing as well. To meet this demand, we propose an indoor localization algorithm based on graph convolutional network (GCN). We first model access points (APs) and the relationships between them as a graph, and utilize received signal strength indication (RSSI) to make up fingerprints. Then the graph and the fingerprint will be put into GCN for feature extraction, and get classification by multilayer perceptron (MLP).In the end, experiments are performed under a 2D scenario and 3D scenario with floor prediction. In the 2D scenario, the mean distance error of GCN-based method is 11m, which improves by 7m and 13m compare with DNN-based and CNN-based schemes respectively. In the 3D scenario, the accuracy of predicting buildings and floors are up to 99.73% and 93.43% respectively. Moreover, in the case of predicting floors and buildings correctly, the mean distance error is 13m, which outperforms DNN-based and CNN-based schemes, whose mean distance errors are 34m and 26m respectively.

eess.SP↗

arXiv2021

Arbitrary-Shaped Text Detection withAdaptive Text Region Representation

Text detection/localization, as an important task in computer vision, has witnessed substantialadvancements in methodology and performance with convolutional neural networks. However, the vastmajority of popular methods use rectangles or quadrangles to describe text regions. These representationshave inherent drawbacks, especially relating to dense adjacent text and loose regional text boundaries,which usually cause difficulty detecting arbitrarily shaped text. In this paper, we propose a novel text regionrepresentation method, with a robust pipeline, which can precisely detect dense adjacent text instances witharbitrary shapes. We consider a text instance to be composed of an adaptive central text region mask anda corresponding expanding ratio between the central text region and the full text region. More specifically,our pipeline generates adaptive central text regions and corresponding expanding ratios with a proposedtraining strategy, followed by a new proposed post-processing algorithm which expands central text regionsto the complete text instance with the corresponding expanding ratios. We demonstrated that our new textregion representation is effective, and that the pipeline can precisely detect closely adjacent text instances ofarbitrary shapes. Experimental results on common datasets demonstrate superior performance o

cs.CV↗