arXiv subjects

Shaohua Wu

Publications and source records attributed to Shaohua Wu.

At least 37 records · Page 2Linked to original sources

arXiv2024

SimpleLLM4AD: An End-to-End Vision-Language Model with Graph Visual Question Answering for Autonomous Driving

Many fields could benefit from the rapid development of the large language models (LLMs). The end-to-end autonomous driving (e2eAD) is one of the typically fields facing new opportunities as the LLMs have supported more and more modalities. Here, by utilizing vision-language model (VLM), we proposed an e2eAD method called SimpleLLM4AD. In our method, the e2eAD task are divided into four stages, which are perception, prediction, planning, and behavior. Each stage consists of several visual question answering (VQA) pairs and VQA pairs interconnect with each other constructing a graph called Graph VQA (GVQA). By reasoning each VQA pair in the GVQA through VLM stage by stage, our method could achieve e2e driving with language. In our method, vision transformers (ViT) models are employed to process nuScenes visual data, while VLM are utilized to interpret and reason about the information extracted from the visual inputs. In the perception stage, the system identifies and classifies objects from the driving environment. The prediction stage involves forecasting the potential movements of these objects. The planning stage utilizes the gathered information to develop a driving strategy, ensuring the safety and efficiency of the autonomous vehicle. Finally, the behavior stage translates the planned actions into executable commands for the vehicle. Our experiments demonstrate that SimpleLLM4AD achieves competitive performance in complex driving scenarios.

cs.CV↗

arXiv2024

Semantics-Empowered Space-Air-Ground-Sea Integrated Network: New Paradigm, Frameworks, and Challenges

In the coming sixth generation (6G) communication era, to provide seamless and ubiquitous connections, the space-air-ground-sea integrated network (SAGSIN) is envisioned to address the challenges of communication coverage in areas with difficult conditions, such as the forest, desert, and sea. Considering the fundamental limitations of the SAGSIN including large-scale scenarios, highly dynamic channels, and limited device capabilities, traditional communications based on Shannon information theory cannot satisfy the communication demands. Moreover, bit-level reconstruction is usually redundant for many human-to-machine or machine-to-machine applications in the SAGSIN. Therefore, it is imperative to consider high-level communications towards semantics exchange, called semantic communications. In this survey, according to the interpretations of the term "semantics", including "significance", "meaning", and "effectiveness-related information", we review state-of-the-art works on semantic communications from three perspectives, which are 1) significance representation and protection, 2) meaning similarity measurement and meaning enhancement, and 3) ultimate effectiveness and effectiveness yielding. Sequentially, three types of semantic communication systems can be correspondingly introduced, namely the significance-oriented, meaning-oriented, and effectiveness/task-oriented semantic communication systems. Implementation of the above three types of systems in the SAGSIN necessitates a new perception-communication-computing-actuation-integrated paradigm (PCCAIP), where all the available perception, computing, and actuation techniques jointly facilitates significance-oriented sampling & transmission, semantic extraction & reconstruction, and task decision. Finally, we point out some future challenges on semantic communications in the SAGSIN. ...

cs.IT↗

arXiv2024

Yuan 2.0-M32: Mixture of Experts with Attention Router

Yuan 2.0-M32, with a similar base architecture as Yuan-2.0 2B, uses a mixture-of-experts architecture with 32 experts of which 2 experts are active. A new router network, Attention Router, is proposed and adopted for a more efficient selection of experts, which improves the accuracy compared to the model with classical router network. Yuan 2.0-M32 is trained with 2000B tokens from scratch, and the training computation consumption is only 9.25% of a dense model at the same parameter scale. Yuan 2.0-M32 demonstrates competitive capability on coding, math, and various domains of expertise, with only 3.7B active parameters of 40B in total, and 7.4 GFlops forward computation per token, both of which are only 1/19 of Llama3-70B. Yuan 2.0-M32 surpass Llama3-70B on MATH and ARC-Challenge benchmark, with accuracy of 55.89 and 95.8 respectively. The models and source codes of Yuan 2.0-M32 are released at Github1.

cs.AI↗

arXiv2024

Optimal Sampling for Uncertainty-of-Information Minimization in a Remote Monitoring System

In this paper, we study a remote monitoring system where a receiver observes a remote binary Markov source and decides whether to sample and transmit the state through a randomly delayed channel. We adopt uncertainty of information (UoI), defined as the entropy conditional on past observations at the receiver, as a metric of value of information, in contrast to the traditional state-agnostic nonlinear age of information (AoI) penalty functions. To address the limitations of prior UoI research that assumes one-time-slot delays, we extend our analysis to scenarios with random delays. We model the problem as a partially observable Markov decision process (POMDP) problem and simplify it to a semi-Markov decision process (SMDP) by introducing the belief state. We propose two algorithms: A globally optimal bisection relative value iteration (bisec-RVI) algorithm and a computationally efficient sub-optimal index-based threshold algorithm to solve the long-term average UoI minimization problem. Numerical simulations demonstrate that our sampling policies surpass traditional zero wait and AoI-optimal policies, particularly under conditions of large delay, with the sub-optimal policy nearly matching the performance of the optimal one.

cs.IT↗

arXiv2023

YUAN 2.0: A Large Language Model with Localized Filtering-based Attention

In this work, we develop and release Yuan 2.0, a series of large language models with parameters ranging from 2.1 billion to 102.6 billion. The Localized Filtering-based Attention (LFA) is introduced to incorporate prior knowledge of local dependencies of natural language into Attention. A data filtering and generating system is presented to build pre-training and fine-tuning dataset in high quality. A distributed training method with non-uniform pipeline parallel, data parallel, and optimizer parallel is proposed, which greatly reduces the bandwidth requirements of intra-node communication, and achieves good performance in large-scale distributed training. Yuan 2.0 models display impressive ability in code generation, math problem-solving, and chatting compared with existing models. The latest version of YUAN 2.0, including model weights and source code, is accessible at Github.

cs.CL↗

arXiv2023

Evaluate PAC codes via Efficient Estimation on Weight Distribution

In this letter, we introduce an efficient method for estimating weight distributions of polar codes and polarization-adjusted convolutional (PAC) codes. Based on a recursive algorithm of computing the weight enumerating functions of polar cosets, this method focuses on two key objectives: accurately determining the number of low-weight codewords and quickly approximating the distribution of high-weight codewords. Simulation results demonstrate that this hybrid method maintains competitively low complexity while effectively achieving the objectives.

cs.IT↗

arXiv2023

Goal-oriented Tensor: Beyond Age of Information Towards Semantics-Empowered Goal-Oriented Communications

Optimizations premised on open-loop metrics such as Age of Information (AoI) indirectly enhance the system's decision-making utility. We therefore propose a novel closed-loop metric named Goal-oriented Tensor (GoT) to directly quantify the impact of semantic mismatches on goal-oriented decision-making utility. Leveraging the GoT, we consider a sampler & decision-maker pair that works collaboratively and distributively to achieve a shared goal of communications. We formulate a two-agent infinite-horizon Decentralized Partially Observable Markov Decision Process (Dec-POMDP) to conjointly deduce the optimal deterministic sampling policy and decision-making policy. To circumvent the curse of dimensionality in obtaining an optimal deterministic joint policy through Brute-Force-Search, a sub-optimal yet computationally efficient algorithm is developed. This algorithm is predicated on the search for a Nash Equilibrium between the sampler and the decision-maker. Simulation results reveal that the proposed sampler & decision-maker co-design surpasses the current literature on AoI and its variants in terms of both goal achievement utility and sparse sampling rate, signifying progress in the semantics-conscious, goal-driven sparse sampling design.

cs.IT↗

arXiv2023

Goal-oriented Tensor: Beyond AoI Towards Semantics-Empowered Goal-oriented Communications

The intricate interplay of source dynamics, unreliable channels, and staleness of information has long been recognized as a significant impediment for the receiver to achieve accurate, timely, and most importantly, goal-oriented decision making. Thus, a plethora of promising metrics, such as Age of Information, Value of Information, and Mean Square Error, have emerged to quantify these underlying adverse factors. Following this avenue, optimizing these metrics has indirectly improved the utility of goal-oriented decision making. Nevertheless, no metric has hitherto been expressly devised to evaluate the utility of a goal-oriented decision-making process. To this end, this paper investigates a novel performance metric, the Goal-oriented Tensor (GoT), to directly quantify the impact of semantic mismatches on the goal-oriented decision making. Based on the GoT, we consider a sampler-decision maker pair that work collaboratively and distributively to achieve a shared goal of communications. We formulate an infinite-horizon Decentralized Partially Observable Markov Decision Process (Dec-POMDP) to conjointly deduce the optimal deterministic sampling policy and decision-making policy. The simulation results reveal that the sampler-decision maker co-design surpasses the current literature on AoI and its variants in terms of both goal achievement utility and sparse sampling rate, signifying a notable accomplishment for a sparse sampler and goal-oriented decision maker co-design.

cs.IT↗

arXiv2023

Tight Upper Bounds on the Error Probability of Spinal Codes over Fading Channels

Spinal codes, a family of rateless codes introduced in 2011, have been proved to achieve Shannon capacity over both the additive white Gaussian noise (AWGN) channel and the binary symmetric channel (BSC). In this paper, we derive explicit tight upper bounds on the error probability of Spinal codes under maximum-likelihood (ML) decoding and perfect channel state information (CSI) over three typical fading channels, including the Rayleigh channel, the Nakagami-m channel and the Rician channel. Simulation results verify the tightness of the derived upper bounds.

cs.IT↗

arXiv2023

Deep Reinforcement Learning-Assisted Age-optimal Transmission Policy for HARQ-aided NOMA Networks

The recent interweaving of AI-6G technologies has sparked extensive research interest in further enhancing reliable and timely communications. \emph{Age of Information} (AoI), as a novel and integrated metric implying the intricate trade-offs among reliability, latency, and update frequency, has been well-researched since its conception. This paper contributes new results in this area by employing a Deep Reinforcement Learning (DRL) approach to intelligently decide how to allocate power resources and when to retransmit in a \emph{freshness-sensitive} downlink multi-user Hybrid Automatic Repeat reQuest with Chase Combining (HARQ-CC) aided Non-Orthogonal Multiple Access (NOMA) network. Specifically, an AoI minimization problem is formulated as a Markov Decision Process (MDP) problem. Then, to achieve deterministic, age-optimal, and intelligent power allocations and retransmission decisions, the Double-Dueling-Deep Q Network (DQN) is adopted. Furthermore, a more flexible retransmission scheme, referred to as Retransmit-At-Will scheme, is proposed to further facilitate the timeliness of the HARQ-aided NOMA network. Simulation results verify the superiority of the proposed intelligent scheme and demonstrate the threshold structure of the retransmission policy. Also, answers to whether user pairing is necessary are discussed by extensive simulation results.

cs.IT↗

arXiv2022

New Upper Bounds on the Error Probability under ML Decoding for Spinal Codes and the Joint Transmission-Decoding System Design

Spinal codes are a type of capacity-achieving rateless codes that have been proved to approach the Shannon capacity over the additive white Gaussian noise (AWGN) channel and the binary symmetric channel (BSC). In this paper, we aim to analyze the bounds on the error probability of Spinal codes and design a joint transmission-decoding system. First, in the finite block-length regime, we derive new upper bounds on the Maximum Likelihood (ML) decoding error probability for Spinal codes over both the AWGN channel and the BSC. Then, based on the derived bounds, we formulate a rate maximization problem. As the solution exhibits an incremental-tail-transmission pattern, we propose an improved transmission scheme, referred to as the thresholded incremental tail transmission (TITT) scheme. Moreover, we also develop a dynamic TITT-matching decoding algorithm, called the bubble decoding with memory (BD-M) algorithm, to reduce the decoding time complexity. The TITT scheme at the transmitter and the BD-M algorithm at the receiver jointly constitute a dynamic transmission-decoding system for Spinal code, improving its rate performance and decoding throughput. Theoretical analysis and simulation results are provided to verify the superiority of the derived bounds and the proposed joint transmission-decoding system design.

cs.IT↗

arXiv2022

Age of Information with Hybrid-ARQ: A Unified Explicit Result

Delivering timely status updates in a timeliness-critical communication system is of paramount importance to assist accurate and efficient decision making. Therefore, the topic of analyzing Age of Information has aroused new research interest. This paper contributes to new results in this area by systematically analyzing the AoI of two types of Hybrid Automatic Repeat reQuest (HARQ) techniques that have been newly standardized in the Release-16 5G New Radio (NR) specifications, namely reactive HARQ and proactive HARQ. Under a code-based status update system with non-trivial coding delay, transmission delay, propagation delay, decoding delay, and feedback delay, we derive unified closed-form average AoI and average Peak AoI expressions for reactive HARQ and proactive HARQ, respectively. Based on the obtained explicit expressions, we formulate an AoI minimization problem to investigate the age-optimal codeblock assignment strategy in the finite block-length (FBL) regime. Through case studies and analytical results, we provide comparative insights between reactive HARQ and proactive HARQ from a perspective of freshness of information. The numerical results and optimization solutions show that proactive HARQ draws its strength from both age performance and system robustness, thus enabling the potential to provide new system advancement of a freshness-critical status update system.

cs.IT↗

arXiv2021

Yuan 1.0: Large-Scale Pre-trained Language Model in Zero-Shot and Few-Shot Learning

Recent work like GPT-3 has demonstrated excellent performance of Zero-Shot and Few-Shot learning on many natural language processing (NLP) tasks by scaling up model size, dataset size and the amount of computation. However, training a model like GPT-3 requires huge amount of computational resources which makes it challengeable to researchers. In this work, we propose a method that incorporates large-scale distributed training performance into model architecture design. With this method, Yuan 1.0, the current largest singleton language model with 245B parameters, achieves excellent performance on thousands GPUs during training, and the state-of-the-art results on NLP tasks. A data processing method is designed to efficiently filter massive amount of raw data. The current largest high-quality Chinese corpus with 5TB high quality texts is built based on this method. In addition, a calibration and label expansion method is proposed to improve the Zero-Shot and Few-Shot performance, and steady improvement is observed on the accuracy of various tasks. Yuan 1.0 presents strong capacity of natural language generation, and the generated articles are difficult to distinguish from the human-written ones.

cs.CL↗

arXiv2021

On the Scheduling Policy for Multi-process WNCS under Edge Computing

This paper considers a multi-process and multi-controller wireless networked control system (WNCS). There are $N$ independent linear time-invariant processes in the system plant which represent different kinds of physical processes. By considering the edge computing, the controllers are played by edge server and cloud server. Each process is measured by a sensor, and the status updates is sent to controller to generate the control command. The link delay of cloud server is longer than that of edge server. The processing time of status update depends on the characteristic of servers and processes. By taking into account such conditions, we mainly investigate how to choose the destination of status updates to minimize the system's average Mean Square Error (MSE), edge server or cloud server? To address this issue, we formulate an infinite horizon average cost Markov Decision Process (MDP) problem and obtain the optimal scheduling policy. The monotonicity of the value function in MDP is characterized and then used to show the threshold structure properties of the optimal scheduling policy. To overcome the curse of dimensionality, we propose a low-complexity suboptimal policy by using additive separable structure of value function. Furthermore, the processing preemption mechanism is considered to handle the status updates more flexible, and the consistency property is proved. On this basis, a numerical example is provided. The simulation results illustrate that the controller selection is related to the timeliness of process and show the performance of the suboptimal policy. We also find that the optimal policy will become a static policy in which the destination of status update is fixed when the wireless channel is error free.

eess.SY↗

arXiv2021

Delay-aware and Energy-Efficient Computation Offloading in Mobile Edge Computing Using Deep Reinforcement Learning

Internet of Things (IoT) is considered as the enabling platform for a variety of promising applications, such as smart transportation and smart city, where massive devices are interconnected for data collection and processing. These IoT applications pose a high demand on storage and computing capacity, while the IoT devices are usually resource-constrained. As a potential solution, mobile edge computing (MEC) deploys cloud resources in the proximity of IoT devices so that their requests can be better served locally. In this work, we investigate computation offloading in a dynamic MEC system with multiple edge servers, where computational tasks with various requirements are dynamically generated by IoT devices and offloaded to MEC servers in a time-varying operating environment (e.g., channel condition changes over time). The objective of this work is to maximize the completed tasks before their respective deadlines and minimize energy consumption. To this end, we propose an end-to-end Deep Reinforcement Learning (DRL) approach to select the best edge server for offloading and allocate the optimal computational resource such that the expected long-term utility is maximized. The simulation results are provided to demonstrate that the proposed approach outperforms the existing methods.

cs.NI↗

arXiv2019

Energy-Efficient Mobile-Edge Computation Offloading over Multiple Fading Blocks

By allowing a mobile device to offload computation-intensive tasks to a base station, mobile edge computing (MEC) is a promising solution for saving the mobile device's energy. In real applications, the offloading may span multiple fading blocks. In this paper, we investigate energy-efficient offloading over multiple fading blocks with random channel gains. An optimization problem is formulated, which optimizes the amount of data for offloading to minimize the total expected energy consumption of the mobile device. Although the formulated optimization problem is non-convex, we prove that the objective function of the problem is piecewise convex, and accordingly develop an optimal solution for the problem. Numerical results verify the correctness of our findings and the effectiveness of our proposed method.

cs.IT↗

arXiv2018

A free boundary problem for a class of parabolic-elliptic type chemotaxis model

In this paper, we study a free boundary problem for a class of parabolic-elliptic type chemotaxis model in high dimensional symmetry domain Ω. By using the contraction mapping principle and operator semigroup approach, we establish the existence of the solution for such kind of chemotaxis system in the domain Ωwith free boundary condition. Besides, we get the explicit formula for the free boundary and show the chemotactic collapse for the solution of the system.

math.AP↗

arXiv2018

A Novel High-Rate Polar-Staircase Coding Scheme

The long-haul communication systems can offer ultra high-speed data transfer rates but suffer from burst errors. The high-rate and high-performance staircase codes provide an efficient way for long-haul transmission. The staircase coding scheme is a concatenation structure, which provides the opportunity to improve the performance of high-rate polar codes. At the same time, the polar codes make the staircase structure more reliable. Thus, a high-rate polar-staircase coding scheme is proposed, where the systematic polar codes are applied as the component codes. The soft cancellation decoding of the systematic polar codes is proposed as a basic ingredient. The encoding of the polar-staircase codes is designed with the help of density evolution, where the unreliable parts of the polar codes are enhanced. The corresponding decoding is proposed with low complexity, and is also optimized for burst error channels. With the well designed encoding and decoding algorithms, the polar-staircase codes perform well on both AWGN channels and burst error channels.

cs.IT↗