arXiv subjects

Hao Zhou

Publications and source records attributed to Hao Zhou.

At least 397 records · Page 22Linked to original sources

arXiv2021

Learning Logic Rules for Document-level Relation Extraction

Document-level relation extraction aims to identify relations between entities in a whole document. Prior efforts to capture long-range dependencies have relied heavily on implicitly powerful representations learned through (graph) neural networks, which makes the model less transparent. To tackle this challenge, in this paper, we propose LogiRE, a novel probabilistic model for document-level relation extraction by learning logic rules. LogiRE treats logic rules as latent variables and consists of two modules: a rule generator and a relation extractor. The rule generator is to generate logic rules potentially contributing to final predictions, and the relation extractor outputs final predictions based on the generated logic rules. Those two modules can be efficiently optimized with the expectation-maximization (EM) algorithm. By introducing logic rules into neural networks, LogiRE can explicitly capture long-range dependencies as well as enjoy better interpretation. Empirical results show that LogiRE significantly outperforms several strong baselines in terms of relation performance (1.8 F1 score) and logical consistency (over 3.3 logic score). Our code is available at https://github.com/rudongyu/LogiRE.

cs.CL↗

arXiv2021

Fundamental Diagrams of Commercial Adaptive Cruise Control: Worldwide Experimental Evidence

Experimental measurements on commercial adaptive cruise control (ACC) vehicles \RoundTwo{are} becoming increasingly available from around the world, providing an unprecedented opportunity to study the traffic flow characteristics that arise from this technology. This paper adds new experimental evidence to this knowledge base and presents a comprehensive empirical study on the ACC equilibrium behaviors via the resulting fundamental diagrams. We find that like human-driven vehicles, ACC systems display a linear equilibrium spacing-speed relationship (within the range of available data) but the key parameters of these relationships can differ significantly from human-driven traffic depending on input settings: At the minimum headway setting, equilibrium capacities in excess of 3500 vehicles per hour are observed, together with an extremely fast equilibrium wave speed of 100 kilometers per hour on average. These fast waves are unfamiliar to human drivers, and may pose a safety risk. The results also suggest that ACC jam spacing can be much larger than in human traffic, which reduces the network storage capacity.

eess.SY↗

arXiv2021

Sliding Sequential CVAE with Time Variant Socially-aware Rethinking for Trajectory Prediction

Pedestrian trajectory prediction is a key technology in many applications such as video surveillance, social robot navigation, and autonomous driving, and significant progress has been made in this research topic. However, there remain two limitations of previous studies. First, with the continuation of time, the prediction error at each time step increases significantly, causing the final displacement error to be impossible to ignore. Second, the prediction results of multiple pedestrians might be impractical in the prediction horizon, i.e., the predicted trajectories might collide with each other. To overcome these limitations, this work proposes a novel trajectory prediction method called CSR, which consists of a cascaded conditional variational autoencoder (CVAE) module and a socially-aware regression module. The cascaded CVAE module first estimates the future trajectories in a sequential pattern. Specifically, each CVAE concatenates the past trajectories and the predicted points so far as the input and predicts the location at the following time step. Then, the socially-aware regression module generates offsets from the estimated future trajectories to produce the socially compliant final predictions, which are more reasonable and accurate results than the estimated trajectories. Moreover, considering the large model parameters of the cascaded CVAE module, a slide CVAE module is further exploited to improve the model efficiency using one shared CVAE, in a slidable manner. Experiments results demonstrate that the proposed method exhibits improvements over state-of-the-art method on the Stanford Drone Dataset (SDD) and ETH/UCY of approximately 38.0% and 22.2%, respectively.

cs.CV↗

arXiv2021

Knowledge Transfer based Radio and Computation Resource Allocation for 5G RAN Slicing

To implement network slicing in 5G, resource allocation is a key function to allocate limited network resources such as radio and computation resources to multiple slices. However, the joint resource allocation also leads to a higher complexity in the network management. In this work, we propose a knowledge transfer based resource allocation (KTRA) method to jointly allocate radio and computation resources for 5G RAN slicing. Compared with existing works, the main difference is that the proposed KTRA method has a knowledge transfer capability. It is designed to use the prior knowledge of similar tasks to improve performance of the target task, e.g., faster convergence speed or higher average reward. The proposed KTRA is compared with Qlearning based resource allocation (QLRA), and KTRA method presents a 18.4% lower URLLC delay and a 30.1% higher eMBB throughput as well as a faster convergence speed.

cs.NI↗

arXiv2021

CNewSum: A Large-scale Chinese News Summarization Dataset with Human-annotated Adequacy and Deducibility Level

Automatic text summarization aims to produce a brief but crucial summary for the input documents. Both extractive and abstractive methods have witnessed great success in English datasets in recent years. However, there has been a minimal exploration of text summarization in Chinese, limited by the lack of large-scale datasets. In this paper, we present a large-scale Chinese news summarization dataset CNewSum, which consists of 304,307 documents and human-written summaries for the news feed. It has long documents with high-abstractive summaries, which can encourage document-level understanding and generation for current summarization models. An additional distinguishing feature of CNewSum is that its test set contains adequacy and deducibility annotations for the summaries. The adequacy level measures the degree of summary information covered by the document, and the deducibility indicates the reasoning ability the model needs to generate the summary. These annotations can help researchers analyze and target their model performance bottleneck. We examine recent methods on CNewSum and release our dataset to provide a solid testbed for automatic Chinese summarization research.

cs.CL↗

arXiv2021

Non-Autoregressive Translation with Layer-Wise Prediction and Deep Supervision

How do we perform efficient inference while retaining high translation quality? Existing neural machine translation models, such as Transformer, achieve high performance, but they decode words one by one, which is inefficient. Recent non-autoregressive translation models speed up the inference, but their quality is still inferior. In this work, we propose DSLP, a highly efficient and high-performance model for machine translation. The key insight is to train a non-autoregressive Transformer with Deep Supervision and feed additional Layer-wise Predictions. We conducted extensive experiments on four translation tasks (both directions of WMT'14 EN-DE and WMT'16 EN-RO). Results show that our approach consistently improves the BLEU scores compared with respective base models. Specifically, our best variant outperforms the autoregressive model on three translation tasks, while being 14.8 times more efficient in inference.

cs.CL↗

arXiv2021

Simulated annealing for optimization of graphs and sequences

Optimization of discrete structures aims at generating a new structure with the better property given an existing one, which is a fundamental problem in machine learning. Different from the continuous optimization, the realistic applications of discrete optimization (e.g., text generation) are very challenging due to the complex and long-range constraints, including both syntax and semantics, in discrete structures. In this work, we present SAGS, a novel Simulated Annealing framework for Graph and Sequence optimization. The key idea is to integrate powerful neural networks into metaheuristics (e.g., simulated annealing, SA) to restrict the search space in discrete optimization. We start by defining a sophisticated objective function, involving the property of interest and pre-defined constraints (e.g., grammar validity). SAGS searches from the discrete space towards this objective by performing a sequence of local edits, where deep generative neural networks propose the editing content and thus can control the quality of editing. We evaluate SAGS on paraphrase generation and molecule generation for sequence optimization and graph optimization, respectively. Extensive results show that our approach achieves state-of-the-art performance compared with existing paraphrase generation methods in terms of both automatic and human evaluations. Further, SAGS also significantly outperforms all the previous methods in molecule generation.

cs.LG↗

arXiv2021

Smart Home Energy Management: Sequence-to-Sequence Load Forecasting and Q-Learning

A smart home energy management system (HEMS) can contribute towards reducing the energy costs of customers; however, HEMS suffers from uncertainty in both energy generation and consumption patterns. In this paper, we propose a sequence to sequence (Seq2Seq) learning-based supply and load prediction along with reinforcement learning-based HEMS control. We investigate how the prediction method affects the HEMS operation. First, we use Seq2Seq learning to predict photovoltaic (PV) power and home devices' load. We then apply Q-learning for offline optimization of HEMS based on the prediction results. Finally, we test the online performance of the trained Q-learning scheme with actual PV and load data. The Seq2Seq learning is compared with VARMA, SVR, and LSTM in both prediction and operation levels. The simulation results show that Seq2Seq performs better with a lower prediction error and online operation performance.

eess.SY↗

arXiv2021

The Volctrans GLAT System: Non-autoregressive Translation Meets WMT21

This paper describes the Volctrans' submission to the WMT21 news translation shared task for German->English translation. We build a parallel (i.e., non-autoregressive) translation system using the Glancing Transformer, which enables fast and accurate parallel decoding in contrast to the currently prevailing autoregressive models. To the best of our knowledge, this is the first parallel translation system that can be scaled to such a practical scenario like WMT competition. More importantly, our parallel translation system achieves the best BLEU score (35.0) on German->English translation task, outperforming all strong autoregressive counterparts.

cs.CL↗

arXiv2021

A kilonova from an ultra-quick merger of a neutron star binary

GRB 060505 was the first well-known nearby (at redshift 0.089) "hybrid" gamma-ray burst that has a duration longer than 2 seconds but without the association of a supernova down to very stringent limits. The prompt $γ-$ray flash lasting $\sim 4$ sec could consist of an intrinsic short burst and its tail emission, but the sizable temporal lag ($\sim 0.35$ sec) as well as the environment properties led to the widely-accepted classification of a long duration gamma-ray burst originated from the collapse of a massive star. Here for the $ first$ time we report the convincing evidence for a thermal-like optical radiation component in the spectral energy distribution of the early afterglow emission. In comparison to AT2017gfo, the thermal radiation is $\sim 2$ times brighter and the temperature is comparable at similar epochs. The optical decline is much quicker than that in X-rays, which is also at odds with the fireball afterglow model but quite natural for the presence of a blue kilonova. Our finding reveals a neutron star merger origin of the hybrid GRB 060505 and strongly supports the theoretical speculation that some binary neutron stars can merge ultra-quickly (within $\sim 1$ Myr) after their formation when the surrounding region is still highly star-forming and the metallicity remains low. Gravitational wave and electromagnetic jointed observations are expected to confirm such scenarios in the near future.

astro-ph.HE↗

arXiv2021

NUANCED: Natural Utterance Annotation for Nuanced Conversation with Estimated Distributions

Existing conversational systems are mostly agent-centric, which assumes the user utterances would closely follow the system ontology (for NLU or dialogue state tracking). However, in real-world scenarios, it is highly desirable that the users can speak freely in their own way. It is extremely hard, if not impossible, for the users to adapt to the unknown system ontology. In this work, we attempt to build a user-centric dialogue system. As there is no clean mapping for a user's free form utterance to an ontology, we first model the user preferences as estimated distributions over the system ontology and map the users' utterances to such distributions. Learning such a mapping poses new challenges on reasoning over existing knowledge, ranging from factoid knowledge, commonsense knowledge to the users' own situations. To this end, we build a new dataset named NUANCED that focuses on such realistic settings for conversational recommendation. Collected via dialogue simulation and paraphrasing, NUANCED contains 5.1k dialogues, 26k turns of high-quality user responses. We conduct experiments, showing both the usefulness and challenges of our problem setting. We believe NUANCED can serve as a valuable resource to push existing research from the agent-centric system to the user-centric system. The code and data is publicly available at \url{https://github.com/facebookresearch/nuanced}.

cs.CL↗

arXiv2021

Meta-fences: blocking waves utilizing a layer of identical tiny scatters

Wave steering by artificial materials (for example, phononic crystals and acoustic metamaterials) is a fascinating frontier in modern physics and engineering, but suffers from bulky sizes and intractable challenges in fabrication. Here, a sparse layer of identical tiny scatters, which we call meta-fences, is presented with a non-destructive way to omnidirectionally block flexural waves in plates. The underlying mechanism is that the restraining force and moment of the scatter are tuned simultaneously to counter-balance the incident wave. Both our experimental results and numerical analysis have demonstrated that broadband wave sources ranging from 3 to 7 kHz can be segregated from the protected area by the meta-fence. In addition, the meta-fence is further assembled into a waveguide routing with an arbitrary configuration. Compared with previous isolators and waveguides, our meta-fences exhibit absolute advantages in compact size, flexible configuration, and high structural strength. The current scenario sheds light on the design of lightweight-and-strong architectures for vibration control and energy harvesting with a high efficiency, and can be extended to microfluidics, acoustics, seismology and other fields.

physics.app-ph↗

arXiv2021

Physiological-Physical Feature Fusion for Automatic Voice Spoofing Detection

Speaker verification systems have been used in many production scenarios in recent years. Unfortunately, they are still highly prone to different kinds of spoofing attacks such as voice conversion and speech synthesis, etc. In this paper, we propose a new method base on physiological-physical feature fusion to deal with voice spoofing attacks. This method involves feature extraction, a densely connected convolutional neural network with squeeze and excitation block (SE-DenseNet), multi-scale residual neural network with squeeze and excitation block (SE-Res2Net) and feature fusion strategies. We first pre-trained a convolutional neural network using the speaker's voice and face in the video as surveillance signals. It can extract physiological features from speech. Then we use SE-DenseNet and SE-Res2Net to extract physical features. Such a densely connection pattern has high parameter efficiency and squeeze and excitation block can enhance the transmission of the feature. Finally, we integrate the two features into the SE-Densenet to identify the spoofing attacks. Experimental results on the ASVspoof 2019 data set show that our model is effective for voice spoofing detection. In the logical access scenario, our model improves the tandem decision cost function (t-DCF) and equal error rate (EER) scores by 4% and 7%, respectively, compared with other methods. In the physical access scenario, our model improved t-DCF and EER scores by 8% and 10%, respectively.

eess.AS↗

arXiv2021

Impact of Acceleration/deceleration Limits on the String Stability of Adaptive Cruise Control

This paper demonstrates that the acceleration/deceleration limits in ACC systems can make a string stable ACC amplify the speed perturbation in natural driving. It is shown that the constrained acceleration/deceleration of the following ACCs are likely to cause speed overshoot to compensate for an extra large/small spacing. Additionally, we find that the constrained deceleration limits can also jeopardize safety, as the limited braking power produces extra small spacing or even crashes. The findings are validated through experiments on real cars. The paper suggests that the ACC parameter space should be extended to include the acceleration/deceleration limits considering their significant role exposed here. Through numerical simulations of ACC platoons, we show i) a marginal string stable ACC is preferable due to the smaller total queue length and the shorter duration in congestion; ii) congestion waves in a mixed ACC platoon largely depend on the sequence of vehicles provided different acceleration/deceleration limits, and iii) the safety hazard caused by the constrained deceleration limits is more significant in mixed ACC platoons when string unstable ACCs exist.

eess.SY↗

arXiv2021

EVA: An Open-Domain Chinese Dialogue System with Large-Scale Generative Pre-Training

Although pre-trained language models have remarkably enhanced the generation ability of dialogue systems, open-domain Chinese dialogue systems are still limited by the dialogue data and the model size compared with English ones. In this paper, we propose EVA, a Chinese dialogue system that contains the largest Chinese pre-trained dialogue model with 2.8B parameters. To build this model, we collect the largest Chinese dialogue dataset named WDC-Dialogue from various public social media. This dataset contains 1.4B context-response pairs and is used as the pre-training corpus of EVA. Extensive experiments on automatic and human evaluation show that EVA outperforms other Chinese pre-trained dialogue models especially in the multi-turn interaction of human-bot conversations.

cs.CL↗

arXiv2021

Follow Your Path: a Progressive Method for Knowledge Distillation

Deep neural networks often have a huge number of parameters, which posts challenges in deployment in application scenarios with limited memory and computation capacity. Knowledge distillation is one approach to derive compact models from bigger ones. However, it has been observed that a converged heavy teacher model is strongly constrained for learning a compact student network and could make the optimization subject to poor local optima. In this paper, we propose ProKT, a new model-agnostic method by projecting the supervision signals of a teacher model into the student's parameter space. Such projection is implemented by decomposing the training objective into local intermediate targets with an approximate mirror descent technique. The proposed method could be less sensitive with the quirks during optimization which could result in a better local optimum. Experiments on both image and text datasets show that our proposed ProKT consistently achieves superior performance compared to other existing knowledge distillation methods.

cs.LG↗

arXiv2021

Review of Learning-based Longitudinal Motion Planning for Autonomous Vehicles: Research Gaps between Self-driving and Traffic Congestion

Self-driving technology companies and the research community are accelerating their pace to use machine learning longitudinal motion planning (mMP) for autonomous vehicles (AVs). This paper reviews the current state of the art in mMP, with an exclusive focus on its impact on traffic congestion. We identify the availability of congestion scenarios in current datasets, and summarize the required features for training mMP. For learning methods, we survey the major methods in both imitation learning and non-imitation learning. We also highlight the emerging technologies adopted by some leading AV companies, e.g. Tesla, Waymo, and Comma.ai. We find that: i) the AV industry has been mostly focusing on the long tail problem related to safety and overlooked the impact on traffic congestion, ii) the current public self-driving datasets have not included enough congestion scenarios, and mostly lack the necessary input features/output labels to train mMP, and iii) albeit reinforcement learning (RL) approach can integrate congestion mitigation into the learning goal, the major mMP method adopted by industry is still behavior cloning (BC), whose capability to learn a congestion-mitigating mMP remains to be seen. Based on the review, the study identifies the research gaps in current mMP development. Some suggestions towards congestion mitigation for future mMP studies are proposed: i) enrich data collection to facilitate the congestion learning, ii) incorporate non-imitation learning methods to combine traffic efficiency into a safety-oriented technical route, and iii) integrate domain knowledge from the traditional car following (CF) theory to improve the string stability of mMP.

eess.SP↗

arXiv2021

Significance of Low-level Controller for String Stability under Adaptive Cruise Control

Current commercial adaptive cruise control (ACC) systems consist of an upper-level planner controller that decides the optimal trajectory that should be followed, and a low-level controller in charge of sending the gas/brake signals to the mechanical system to actually move the vehicle. We find that the low-level controller has a significant impact on the string stability (SS) even if the planner is string stable: (i) a slow controller deteriorates the SS, (ii) slow controllers are common as they arise from insufficient control gains, from a "weak" gas/brake system or both, and (iii) the integral term in a slow controller causes undesired overshooting which affects the SS. Accordingly, we suggest tuning up the proportional/feedforward gain and ensuring the gas/brake is not "weak". The study results are validated both numerically and empirically with data from commercial cars.

eess.SY↗