arXiv subjects

Yuxiang Lu

Publications and source records attributed to Yuxiang Lu.

39 records · Page 3Linked to original sources

arXiv2021

ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation

Pre-trained models have achieved state-of-the-art results in various Natural Language Processing (NLP) tasks. Recent works such as T5 and GPT-3 have shown that scaling up pre-trained language models can improve their generalization abilities. Particularly, the GPT-3 model with 175 billion parameters shows its strong task-agnostic zero-shot/few-shot learning capabilities. Despite their success, these large-scale models are trained on plain texts without introducing knowledge such as linguistic knowledge and world knowledge. In addition, most large-scale models are trained in an auto-regressive way. As a result, this kind of traditional fine-tuning approach demonstrates relatively weak performance when solving downstream language understanding tasks. In order to solve the above problems, we propose a unified framework named ERNIE 3.0 for pre-training large-scale knowledge enhanced models. It fuses auto-regressive network and auto-encoding network, so that the trained model can be easily tailored for both natural language understanding and generation tasks with zero-shot learning, few-shot learning or fine-tuning. We trained the model with 10 billion parameters on a 4TB corpus consisting of plain texts and a large-scale knowledge graph. Empirical results show that the model outperforms the state-of-the-art models on 54 Chinese NLP tasks, and its English version achieves the first place on the SuperGLUE benchmark (July 3, 2021), surpassing the human performance by +0.8% (90.6% vs. 89.8%).

cs.CL↗

arXiv2021

Double Blind $T$-Private Information Retrieval

Double blind $T$-private information retrieval (DB-TPIR) enables two users, each of whom specifies an index ($θ_1, θ_2$, resp.), to efficiently retrieve a message $W(θ_1,θ_2)$ labeled by the two indices, from a set of $N$ servers that store all messages $W(k_1,k_2), k_1\in\{1,2,\cdots,K_1\}, k_2\in\{1,2,\cdots,K_2\}$, such that the two users' indices are kept private from any set of up to $T_1,T_2$ colluding servers, respectively, as well as from each other. A DB-TPIR scheme based on cross-subspace alignment is proposed in this paper, and shown to be capacity-achieving in the asymptotic setting of large number of messages and bounded latency. The scheme is then extended to $M$-way blind $X$-secure $T$-private information retrieval (MB-XS-TPIR) with multiple ($M$) indices, each belonging to a different user, arbitrary privacy levels for each index ($T_1, T_2,\cdots, T_M$), and arbitrary level of security ($X$) of data storage, so that the message $W(θ_1,θ_2,\cdots, θ_M)$ can be efficiently retrieved while the stored data is held secure against collusion among up to $X$ colluding servers, the $m^{th}$ user's index is private against collusion among up to $T_m$ servers, and each user's index $θ_m$ is private from all other users. The general scheme relies on a tensor-product based extension of cross-subspace alignment and retrieves $1-(X+T_1+\cdots+T_M)/N$ bits of desired message per bit of download.

cs.IT↗

arXiv2020

Cognitive Radar Using Reinforcement Learning in Automotive Applications

The concept of cognitive radar (CR) enables radar systems to achieve intelligent adaption to a changeable environment with feedback facility from receiver to transmitter. However, the implementation of CR in a fast-changing environment usually requires a well-known environmental model. In our work, we stress the learning ability of CR in an unknown environment using a combination of CR and reinforcement learning (RL), called RL-CR. Less or no model of the environment is required. We also apply the general RL-CR to a specific problem of automotive radar spectrum allocation to mitigate mutual interference. Using RL-CR, each vehicle can autonomously choose a frequency subband according to its own observation of the environment. Since radar's single observation is quite limited compared to the overall information of the environment, a long short-term memory (LSTM) network is utilized so that radar can decide the next transmitted subband by aggregating its observations over time. Compared with centralized spectrum allocation approaches, our approach has the advantage of reducing communication between vehicles and the control center. It also outperforms some other distributive frequency subband selecting policies in reducing interference under certain circumstances.

eess.SP↗