arXiv subjects

He Bai

Publications and source records attributed to He Bai.

54 records · Page 3Linked to original sources

arXiv2021

Semantics of the Unwritten: The Effect of End of Paragraph and Sequence Tokens on Text Generation with GPT2

The semantics of a text is manifested not only by what is read, but also by what is not read. In this article, we will study how the implicit "not read" information such as end-of-paragraph (\eop) and end-of-sequence (\eos) affect the quality of text generation. Specifically, we find that the pre-trained language model GPT2 can generate better continuations by learning to generate the \eop in the fine-tuning stage. Experimental results on English story generation show that \eop can lead to higher BLEU score and lower \eos perplexity. We also conduct experiments on a self-collected Chinese essay dataset with Chinese-GPT2, a character level LM without \eop or \eos during pre-training. Experimental results show that the Chinese GPT2 can generate better essay endings with \eop.

cs.CL↗

arXiv2021

Emulating UAV Motion by Utilizing Robotic Arm for mmWave Wireless Channel Characterization

In this paper, millimeter wave (mmWave) wireless channel characteristics (Doppler spread and path loss modeling) for Unmanned Aerial Vehicles (UAVs) assisted communication is analyzed and studied by emulating the real UAV motion using a robotic arm. The motion considers the actual turbulence caused by the wind gusts to the UAV in the atmosphere, which is statistically modeled by the widely used Dryden wind model. The frequency under consideration is 28 GHz in an anechoic chamber setting. A total of 11 distance points from 3.5 feet to 23.5 feet in increments of 2 feet were considered in this experiment. At each distance point, 3 samples of data were collected for better inference purposes. In this emulated environment, it was found out that the average Doppler spread at these different distances was around -20 Hz and +20 Hz at the noise floor of -60 dB. On the other hand, the path loss exponent was found to be 1.843. This study presents and lays out a novel framework of emulating UAV motion for mmWave communication systems, which will pave the way out for future design and implementation of next generation UAV-assisted wireless communication systems.

eess.SP↗

arXiv2021

Model-Free Optimal Control of Linear Multi-Agent Systems via Decomposition and Hierarchical Approximation

Designing the optimal linear quadratic regulator (LQR) for a large-scale multi-agent system (MAS) is time-consuming since it involves solving a large-size matrix Riccati equation. The situation is further exasperated when the design needs to be done in a model-free way using schemes such as reinforcement learning (RL). To reduce this computational complexity, we decompose the large-scale LQR design problem into multiple smaller-size LQR design problems. We consider the objective function to be specified over an undirected graph, and cast the decomposition as a graph clustering problem. The graph is decomposed into two parts, one consisting of independent clusters of connected components, and the other containing edges that connect different clusters. Accordingly, the resulting controller has a hierarchical structure, consisting of two components. The first component optimizes the performance of each independent cluster by solving the smaller-size LQR design problem in a model-free way using an RL algorithm. The second component accounts for the objective coupling different clusters, which is achieved by solving a least squares problem in one shot. Although suboptimal, the hierarchical controller adheres to a particular structure as specified by inter-agent couplings in the objective function and by the decomposition strategy. Mathematical formulations are established to find a decomposition that minimizes the number of required communication links or reduces the optimality gap. Numerical simulations are provided to highlight the pros and cons of the proposed designs.

eess.SY↗

arXiv2021

Dynamic Control Allocation between Onboard and Delayed Remote Control for Unmanned Aircraft System Detect-and-Avoid

This paper develops and evaluates the performance of an allocation agent to be potentially integrated into the onboard Detect and Avoid (DAA) computer of an Unmanned Aircraft System (UAS). We consider a UAS that can be fully controlled by the onboard DAA system and by a remote human pilot. With a communication channel prone to latency, we consider a mixed initiative interaction environment, where the control authority of the UAS is dynamically allocated by the allocation agent. In an encounter with a dynamic intruder, the probability of collision may increase in the absence of pilot commands in the presence of latency. Moreover, a delayed pilot command may not result in safe resolution of the current scenario and need to be improvised. We design an optimization algorithm to reduce collision risk and refine delayed pilot commands. Towards this end, a Markov Decision Process (MDP)and its solution are employed to create a wait time map. The map consists of estimated times that the UAS can wait for the remote pilot commands at each state. A command blending algorithm is designed to select an avoidance maneuver that prioritizes the pilot intention extracted from the pilot commands. The wait time map and the command blending algorithm are implemented and integrated into a closed-loop simulator. We conduct ten thousands fast-time Monte Carlo simulations and compare the performance of the integrated setup with a standalone DAA setup. The simulation results show that the allocation agent enables the UAS to wait without inducing any near mid air collision (NMAC) and severe loss of well clear (LoWC) while positively improve pilot involvement in the encounter resolution.

eess.SY↗

arXiv2021

Decomposability and Parallel Computation of Multi-Agent LQR

Individual agents in a multi-agent system (MAS) may have decoupled open-loop dynamics, but a cooperative control objective usually results in coupled closed-loop dynamics thereby making the control design computationally expensive. The computation time becomes even higher when a learning strategy such as reinforcement learning (RL) needs to be applied to deal with the situation when the agents dynamics are not known. To resolve this problem, we propose a parallel RL scheme for a linear quadratic regulator (LQR) design in a continuous-time linear MAS. The idea is to exploit the structural properties of two graphs embedded in the $Q$ and $R$ weighting matrices in the LQR objective to define an orthogonal transformation that can convert the original LQR design to multiple decoupled smaller-sized LQR designs. We show that if the MAS is homogeneous then this decomposition retains closed-loop optimality. Conditions for decomposability, an algorithm for constructing the transformation matrix, a parallel RL algorithm, and robustness analysis when the design is applied to non-homogeneous MAS are presented. Simulations show that the proposed approach can guarantee significant speed-up in learning without any loss in the cumulative value of the LQR cost.

eess.SY↗

arXiv2021

Learning Distributed Stabilizing Controllers for Multi-Agent Systems

We address the problem of model-free distributed stabilization of heterogeneous multi-agent systems using reinforcement learning (RL). Two algorithms are developed. The first algorithm solves a centralized linear quadratic regulator (LQR) problem without knowing any initial stabilizing gain in advance. The second algorithm builds upon the results of the first algorithm, and extends it to distributed stabilization of multi-agent systems with predefined interaction graphs. Rigorous proofs are provided to show that the proposed algorithms achieve guaranteed convergence if specific conditions hold. A simulation example is presented to demonstrate the theoretical results.

eess.SY↗

arXiv2021

A Decentralized Approach to Bayesian Learning

Motivated by decentralized approaches to machine learning, we propose a collaborative Bayesian learning algorithm taking the form of decentralized Langevin dynamics in a non-convex setting. Our analysis show that the initial KL-divergence between the Markov Chain and the target posterior distribution is exponentially decreasing while the error contributions to the overall KL-divergence from the additive noise is decreasing in polynomial time. We further show that the polynomial-term experiences speed-up with number of agents and provide sufficient conditions on the time-varying step-sizes to guarantee convergence to the desired distribution. The performance of the proposed algorithm is evaluated on a wide variety of machine learning tasks. The empirical results show that the performance of individual agents with locally available data is on par with the centralized setting with considerable improvement in the convergence rate.

stat.ML↗

arXiv2020

Segatron: Segment-Aware Transformer for Language Modeling and Understanding

Transformers are powerful for sequence modeling. Nearly all state-of-the-art language models and pre-trained language models are based on the Transformer architecture. However, it distinguishes sequential tokens only with the token position index. We hypothesize that better contextual representations can be generated from the Transformer with richer positional information. To verify this, we propose a segment-aware Transformer (Segatron), by replacing the original token position encoding with a combined position encoding of paragraph, sentence, and token. We first introduce the segment-aware mechanism to Transformer-XL, which is a popular Transformer-based language model with memory extension and relative position encoding. We find that our method can further improve the Transformer-XL base model and large model, achieving 17.1 perplexity on the WikiText-103 dataset. We further investigate the pre-training masked language modeling task with Segatron. Experimental results show that BERT pre-trained with Segatron (SegaBERT) can outperform BERT with vanilla Transformer on various NLP tasks, and outperforms RoBERTa on zero-shot sentence representation learning.

cs.CL↗

arXiv2020

Hierarchical Control of Multi-Agent Systems using Online Reinforcement Learning

We propose a new reinforcement learning based approach to designing hierarchical linear quadratic regulator (LQR) controllers for heterogeneous linear multi-agent systems with unknown state-space models and separated control objectives. The separation arises from grouping the agents into multiple non-overlapping groups, and defining the control goal as two distinct objectives. The first objective aims to minimize a group-wise block-decentralized LQR function that models group-level mission. The second objective, on the other hand, tries to minimize an LQR function between the average states (centroids) of the groups. Exploiting this separation, we redefine the weighting matrices of the LQR functions in a way that they allow us to decouple their respective algebraic Riccati equations. Thereafter, we develop a reinforcement learning strategy that uses online measurements of the agent states and the average states to learn the respective controllers based on the approximate Riccati equations. Since the first controller is block-decentralized and, therefore, can be learned in parallel, while the second controller is reduced-dimensional due to averaging, the overall design enjoys a significantly reduced learning time compared to centralized reinforcement learning.

eess.SY↗

arXiv2020

Reduced-Dimensional Reinforcement Learning Control using Singular Perturbation Approximations

We present a set of model-free, reduced-dimensional reinforcement learning (RL) based optimal control designs for linear time-invariant singularly perturbed (SP) systems. We first present a state-feedback and output-feedback based RL control design for a generic SP system with unknown state and input matrices. We take advantage of the underlying time-scale separation property of the plant to learn a linear quadratic regulator (LQR) for only its slow dynamics, thereby saving a significant amount of learning time compared to the conventional full-dimensional RL controller. We analyze the sub-optimality of the design using SP approximation theorems and provide sufficient conditions for closed-loop stability. Thereafter, we extend both designs to clustered multi-agent consensus networks, where the SP property reflects through clustering. We develop both centralized and cluster-wise block-decentralized RL controllers for such networks, in reduced dimensions. We demonstrate the details of the implementation of these controllers using simulations of relevant numerical examples and compare them with conventional RL designs to show the computational benefits of our approach.

eess.SY↗

arXiv2019

Multi-Agent Coordination for Distributed Transmit Beamforming

This paper presents the formulation and analysis of a two time-scale optimization algorithm for multi-agent coordination for the purpose of distributed beamforming. Each agent is assumed to be randomly positioned with respect to each other with random phase offsets and amplitudes. Agents are tasked with coordinate among themselves to position themselves and adjust their phase offset and amplitude such that they can construct a desired directed beam. Here we propose a two time-scale optimization algorithm that consists of a fast time-scale algorithm to solve for the amplitude and phase while a slow time-scale algorithm to solve for the control required to re-position the agents. The numerical results given here indicate that the proposed two time-scale approach is able to reconstruct a desired beam pattern.

eess.SP↗

arXiv2019

Distributed Stochastic Gradient Method for Non-Convex Problems with Applications in Supervised Learning

We develop a distributed stochastic gradient descent algorithm for solving non-convex optimization problems under the assumption that the local objective functions are twice continuously differentiable with Lipschitz continuous gradients and Hessians. We provide sufficient conditions on step-sizes that guarantee the asymptotic mean-square convergence of the proposed algorithm. We apply the developed algorithm to a distributed supervised-learning problem, in which a set of networked agents collaboratively train their individual neural nets to recognize handwritten digits in images. Results indicate that all agents report similar performance that is also comparable to the performance of a centrally trained neural net. Numerical results also show that the proposed distributed algorithm allows the individual agents to recognize the digits even though the training data corresponding to all the digits is not locally available to each agent.

math.OC↗

arXiv2019

Wind Estimation Using Quadcopter Motion: A Machine Learning Approach

In this article, we study the well known problem of wind estimation in atmospheric turbulence using small unmanned aerial systems (sUAS). We present a machine learning approach to wind velocity estimation based on quadcopter state measurements without a wind sensor. We accomplish this by training a long short-term memory (LSTM) neural network (NN) on roll and pitch angles and quadcopter position inputs with forcing wind velocities as the targets. The datasets are generated using a simulated quadcopter in turbulent wind fields. The trained neural network is deployed to estimate the turbulent winds as generated by the Dryden gust model as well as a realistic large eddy simulation (LES) of a near-neutral atmospheric boundary layer (ABL) over flat terrain. The resulting NN predictions are compared to a wind triangle approach that uses tilt angle as an approximation of airspeed. Results from this study indicate that the LSTM-NN based approach predicts lower errors in both the mean and variance of the local wind field as compared to the wind triangle approach. The work reported in this article demonstrates the potential of machine learning for sensor-less wind estimation and has strong implications to large-scale low-altitude atmospheric sensing using sUAS for environmental and autonomous navigation applications.

eess.SP↗

arXiv2019

Memory Consolidation for Contextual Spoken Language Understanding with Dialogue Logistic Inference

Dialogue contexts are proven helpful in the spoken language understanding (SLU) system and they are typically encoded with explicit memory representations. However, most of the previous models learn the context memory with only one objective to maximizing the SLU performance, leaving the context memory under-exploited. In this paper, we propose a new dialogue logistic inference (DLI) task to consolidate the context memory jointly with SLU in the multi-task framework. DLI is defined as sorting a shuffled dialogue session into its original logical order and shares the same memory encoder and retrieval mechanism as the SLU model. Our experimental results show that various popular contextual SLU models can benefit from our approach, and improvements are quite impressive, especially in slot filling.

cs.CL↗

arXiv2018

Source-Critical Reinforcement Learning for Transferring Spoken Language Understanding to a New Language

To deploy a spoken language understanding (SLU) model to a new language, language transferring is desired to avoid the trouble of acquiring and labeling a new big SLU corpus. Translating the original SLU corpus into the target language is an attractive strategy. However, SLU corpora consist of plenty of semantic labels (slots), which general-purpose translators cannot handle well, not to mention additional culture differences. This paper focuses on the language transferring task given a tiny in-domain parallel SLU corpus. The in-domain parallel corpus can be used as the first adaptation on the general translator. But more importantly, we show how to use reinforcement learning (RL) to further finetune the adapted translator, where translated sentences with more proper slot tags receive higher rewards. We evaluate our approach on Chinese to English language transferring for SLU systems. The experimental results show that the generated English SLU corpus via adaptation and reinforcement learning gives us over 97% in the slot F1 score and over 84% accuracy in domain classification. It demonstrates the effectiveness of the proposed language transferring method. Compared with naive translation, our proposed method improves domain classification accuracy by relatively 22%, and the slot filling F1 score by relatively more than 71%.

cs.CL↗

arXiv2018

On the Trajectory Dependence of Atmospheric Boundary Layer Turbulence Sensing using Small Unmanned Vehicles

Atmospheric turbulence, especially in the near-surface boundary layer is known to be under-sampled due to the need to capture a wide separation in length and time-scales and limitation in the number of sensors. Over the past decade, the use if Unmanned Aircraft Systems (UAS) technology is approaching ubiquitous proportions for a wide variety of applications, especially with the recent FAA relaxation of flying restrictions. From a geophysical sciences perspective, such technology would allow for sensing of large-scale atmospheric flows, particularly, atmospheric boundary layer (ABL) turbulence, air quality monitoring in urban settings where multitude of small, minimally-invasive and mobile sensors can drastically alter our ability to study such complex phenomena. Currently available observational data of atmospheric boundary layer physics is so sparse and infrequent which significantly limits analysis. With the quantity and resolution of the data that can be measured using a swarm of UAS, three-dimensional reconstruction and deduction of coherent structures in ABL turbulence may be feasible. However, key challenges remain in the form of identifying optimal trajectories to fly the UAS to obtain the relevant quantifications of the turbulence, interpretation of the sensor data from mobile sensors and understanding how representative are the sparse measurements of the overall turbulent boundary layer. This leads to many fundamentally interesting questions if which a few are explored in this article.

physics.flu-dyn↗

arXiv2015

GPS-Denied Relative Motion Estimation For Fixed-Wing UAV Using the Variational Pose Estimator

Relative pose estimation between fixed-wing unmanned aerial vehicles (UAVs) is treated using a stable and robust estimation scheme. The motivating application of this scheme is that of "handoff" of an object being tracked from one fixed-wing UAV to another in a team of UAVs, using onboard sensors in a GPS-denied environment. This estimation scheme uses optical measurements from cameras onboard a vehicle, to estimate both the relative pose and relative velocities of another vehicle or target object. It is obtained by applying the Lagrange-d'Alembert principle to a Lagrangian constructed from measurement residuals using only the optical measurements. This nonlinear pose estimation scheme is discretized for computer implementation using the discrete Lagrange-d'Alembert principle, with a discrete-time linear filter for obtaining relative velocity estimates from optical measurements. Computer simulations depict the stability and robustness of this estimator to noisy measurements and uncertainties in initial relative pose and velocities.

math.OC↗

arXiv2013

Output Synchronization of Nonlinear Systems under Input Disturbances

We study synchronization of nonlinear systems that satisfy an incremental passivity property. We consider the case where the control input is subject to a class of disturbances, including constant and sinusoidal disturbances with unknown phases and magnitudes and known frequencies. We design a distributed control law that recovers the synchronization of the nonlinear systems in the presence of the disturbances. Simulation results of Goodwin oscillators illustrate the effectiveness of the control law. Finally, we highlight the connection of the proposed control law to the dynamic average consensus estimator developed in [1].

eess.SY↗