Searcharxiv⌕ Search

arXiv subjects

Yi Huang

Publications and source records attributed to Yi Huang.

At least 163 records · Page 9Linked to original sources
arXiv2022

A Generative User Simulator with GPT-based Architecture and Goal State Tracking for Reinforced Multi-Domain Dialog Systems

Building user simulators (USs) for reinforcement learning (RL) of task-oriented dialog systems (DSs) has gained more and more attention, which, however, still faces several fundamental challenges. First, it is unclear whether we can leverage pretrained language models to design, for example, GPT-2 based USs, to catch up and interact with the recently advanced GPT-2 based DSs. Second, an important ingredient in a US is that the user goal can be effectively incorporated and tracked; but how to flexibly integrate goal state tracking and develop an end-to-end trainable US for multi-domains has remained to be a challenge. In this work, we propose a generative user simulator (GUS) with GPT-2 based architecture and goal state tracking towards addressing the above two challenges. Extensive experiments are conducted on MultiWOZ2.1. Different DSs are trained via RL with GUS, the classic agenda-based user simulator (ABUS) and other ablation simulators respectively, and are compared for cross-model evaluation, corpus-based evaluation and human evaluation. The GUS achieves superior results in all three evaluation tasks.

cs.CL↗
arXiv2022

Temporal-Spatial dependencies ENhanced deep learning model (TSEN) for household leverage series forecasting

Analyzing both temporal and spatial patterns for an accurate forecasting model for financial time series forecasting is a challenge due to the complex nature of temporal-spatial dynamics: time series from different locations often have distinct patterns; and for the same time series, patterns may vary as time goes by. Inspired by the successful applications of deep learning, we propose a new model to resolve the issues of forecasting household leverage in China. Our solution consists of multiple RNN-based layers and an attention layer: each RNN-based layer automatically learns the temporal pattern of a specific series with multivariate exogenous series, and then the attention layer learns the spatial correlative weight and obtains the global representations simultaneously. The results show that the new approach can capture the temporal-spatial dynamics of household leverage well and get more accurate and solid predictive results. More, the simulation also studies show that clustering and choosing correlative series are necessary to obtain accurate forecasting results.

cs.LG↗
arXiv2022

Building Markovian Generative Architectures over Pretrained LM Backbones for Efficient Task-Oriented Dialog Systems

Recently, Transformer based pretrained language models (PLMs), such as GPT2 and T5, have been leveraged to build generative task-oriented dialog (TOD) systems. A drawback of existing PLM-based models is their non-Markov architectures across turns, i.e., the whole history is used as the conditioning input at each turn. First, this brings inefficiencies in memory and computation. Furthermore, using the whole history increases model complexity and may hurt the training efficiency, especially when facing small amounts of labeled training data (the low-resource setting). In this paper, motivated by the observation that dialog states could be viewed as Markov states, we propose to build Markovian Generative Architectures (MGA) over PLM backbones for efficient TOD systems. Experiments on MultiWOZ2.1 show that in the rich-resource setting, the proposed Markov models reduce memory and time costs without performance degradation; in the low-resource setting, the training efficiency of the Markov models is more significant.

cs.CL↗
arXiv2022

Jointly Reinforced User Simulator and Task-oriented Dialog System with Simplified Generative Architecture

Recently, there has been progress in supervised funetuning pretrained GPT-2 to build end-to-end task-oriented dialog (TOD) systems. However, online reinforcement learning of a GPT-2 based dialog system (DS), together with a end-to-end user simulator (US), has not ever been explored. Moreover, a drawback with existing GPT-2 based TOD systems is that they mostly employ the whole dialog history as input, which brings inefficiencies in memory and compute. In this paper, we first propose Simplified Generative Architectures (SGA) for DS and US respectively, both based on GPT-2 but using shortened history. Then, we successfully develop Jointly Reinforced US and DS, called SGA-JRUD. Our DS with the proposed SGA, when only supervised trained, achieves state-of-the-art performance on MultiWOZ2.1 and is more compute-efficient in both training and generation. Extensive experiments on MultiWOZ2.1 further show the superiority of SGA-JRUD in both offline and online evaluations.

cs.CL↗
arXiv2022

PSSAT: A Perturbed Semantic Structure Awareness Transferring Method for Perturbation-Robust Slot Filling

Most existing slot filling models tend to memorize inherent patterns of entities and corresponding contexts from training data. However, these models can lead to system failure or undesirable outputs when being exposed to spoken language perturbation or variation in practice. We propose a perturbed semantic structure awareness transferring method for training perturbation-robust slot filling models. Specifically, we introduce two MLM-based training strategies to respectively learn contextual semantic structure and word distribution from unsupervised language perturbation corpus. Then, we transfer semantic knowledge learned from upstream training procedure into the original samples and filter generated data by consistency processing. These procedures aim to enhance the robustness of slot filling models. Experimental results show that our method consistently outperforms the previous basic methods and gains strong generalization while preventing the model from memorizing inherent patterns of entities and contexts.

cs.CL↗
arXiv2022

Generalized Intent Discovery: Learning from Open World Dialogue System

Traditional intent classification models are based on a pre-defined intent set and only recognize limited in-domain (IND) intent classes. But users may input out-of-domain (OOD) queries in a practical dialogue system. Such OOD queries can provide directions for future improvement. In this paper, we define a new task, Generalized Intent Discovery (GID), which aims to extend an IND intent classifier to an open-world intent set including IND and OOD intents. We hope to simultaneously classify a set of labeled IND intent classes while discovering and recognizing new unlabeled OOD types incrementally. We construct three public datasets for different application scenarios and propose two kinds of frameworks, pipeline-based and end-to-end for future work. Further, we conduct exhaustive experiments and qualitative analysis to comprehend key challenges and provide new guidance for future GID research.

cs.CL↗
arXiv2022

Software-Hardware Codesign for Efficient In-Memory Regular Pattern Matching

Regular pattern matching is used in numerous application domains, including text processing, bioinformatics, and network security. Patterns are typically expressed with an extended syntax of regular expressions that include the computationally challenging construct of bounded iteration or counting, which describes the repetition of a pattern a fixed number of times. We develop a design for a specialized in-memory hardware architecture for NFA execution that integrates counter and bit vector elements. The design is inspired by the theoretical model of nondeterministic counter automata (NCA). A key feature of our approach is that we statically analyze regular expressions to determine bounds on the amount of memory needed for the occurrences of counting. The results of this analysis are used by a regex-to-hardware compiler in order to make an appropriate selection of counter or bit vector elements. We evaluate the performance of our hardware implementation on a simulator based on circuit parameters collected by SPICE simulation using a TSMC 28nm process. We find the usage of counter and bit vector quickly outperforms unfolding solutions by orders of magnitude with small counting quantifiers. Experiments concerning realistic workloads show up to 76% energy reduction and 58% area reduction in comparison to traditional in-memory NFA processors.

cs.FL↗
arXiv2022

Biexciton crystal in a two-dimensional semiconductor heteropentalayer

This paper is written for the Special Issue in Honor of Emmanuel Rashba. We study the gas of indirect dipolar excitons created by an interband illumination of a pentalayer WSe$_2$/MoSe$_2$/WSe$_2$/MoSe$_2$/WSe$_2$. We show that two colinear indirect excitons bind into a linear biexciton with twice larger dipole moment. Two biexcitons with opposite dipole directions attract each other at large distances and repel each other at short distances. Therefore, biexcitons form a staggered crystal with anti-ferroelectric square lattice. The electrostatic energy of this crystal per biexciton has a minimum at the biexciton concentration $n =n_c=0.14d^{-2}$, where $d \sim 0.7$ nm is a single layer thickness. At small illumination intensity, biexcitons condense into sparse crystallites with $n = n_c$, where photoluminescence frequency is red shifted and independent on the light intensity. We also study a capacitor made of five identical semiconductor monolayers separated by hBN spacers where a critical voltage applied between layers 1, 3, 5, and 2, 4 abruptly creates a similar biexciton crystal. At this voltage, the differential capacitance diverges.

cond-mat.mes-hall↗
arXiv2022

Conductivity of two-dimensional small gap semiconductors and topological insulators in strong Coulomb disorder

We are honored to dedicate this article to Emmanuel Rashba on the occasion of his 95 birthday. In the ideal disorder-free situation, a two-dimensional band gap insulator has an activation energy for conductivity equal to half the band gap, $Δ$. But transport experiments usually exhibit a much smaller activation energy at low temperature, and the relation between this activation energy and $Δ$ is unclear. Here we consider the temperature-dependent conductivity of a two-dimensional narrow gap semiconductor on a substrate containing Coulomb impurities, mostly focusing on the case when amplitude of the random potential $Γ\gg Δ$. We show that the conductivity generically exhibits three regimes and only the highest temperature regime exhibits an activation energy that reflects the band gap. At lower temperatures, the conduction proceeds through nearest-neighbor or variable-range hopping between electron and hole puddles created by the disorder. We show that the activation energy and characteristic temperature associated with these processes steeply collapse near a critical impurity concentration. Larger concentrations lead to an exponentially small activation energy and exponentially long localization length, which in mesoscopic samples can appear as a disorder-induced insulator-to-metal transition. We arrive at a similar disorder driven steep insulator-metal transition in thin films of three-dimensional topological insulators with very large dielectric constant, where due to confinement of electric field internal Coulomb impurities create larger disorder potential. Away from neutrality point this unconventional insulator-to-metal transition is augmented by conventional metal-insulator transition at small impurity concentrations, so that we arrive at disorder-driven re-entrant metal-insulator-metal transition.

cond-mat.mes-hall↗
arXiv2022

AggPose: Deep Aggregation Vision Transformer for Infant Pose Estimation

Movement and pose assessment of newborns lets experienced pediatricians predict neurodevelopmental disorders, allowing early intervention for related diseases. However, most of the newest AI approaches for human pose estimation methods focus on adults, lacking publicly benchmark for infant pose estimation. In this paper, we fill this gap by proposing infant pose dataset and Deep Aggregation Vision Transformer for human pose estimation, which introduces a fast trained full transformer framework without using convolution operations to extract features in the early stages. It generalizes Transformer + MLP to high-resolution deep layer aggregation within feature maps, thus enabling information fusion between different vision levels. We pre-train AggPose on COCO pose dataset and apply it on our newly released large-scale infant pose estimation dataset. The results show that AggPose could effectively learn the multi-scale features among different resolutions and significantly improve the performance of infant pose estimation. We show that AggPose outperforms hybrid model HRFormer and TokenPose in the infant pose estimation dataset. Moreover, our AggPose outperforms HRFormer by 0.8 AP on COCO val pose estimation on average. Our code is available at github.com/SZAR-LAB/AggPose.

cs.CV↗
arXiv2022

Advancing Semi-Supervised Task Oriented Dialog Systems by JSA Learning of Discrete Latent Variable Models

Developing semi-supervised task-oriented dialog (TOD) systems by leveraging unlabeled dialog data has attracted increasing interests. For semi-supervised learning of latent state TOD models, variational learning is often used, but suffers from the annoying high-variance of the gradients propagated through discrete latent variables and the drawback of indirectly optimizing the target log-likelihood. Recently, an alternative algorithm, called joint stochastic approximation (JSA), has emerged for learning discrete latent variable models with impressive performances. In this paper, we propose to apply JSA to semi-supervised learning of the latent state TOD models, which is referred to as JSA-TOD. To our knowledge, JSA-TOD represents the first work in developing JSA based semi-supervised learning of discrete latent variable conditional models for such long sequential generation problems like in TOD systems. Extensive experiments show that JSA-TOD significantly outperforms its variational learning counterpart. Remarkably, semi-supervised JSA-TOD using 20% labels performs close to the full-supervised baseline on MultiWOZ2.1.

cs.CL↗
arXiv2022

Energy-Efficient Transmit Beamforming and Antenna Selection with Non-Linear PA Efficiency

This letter studies the energy-efficient design in a downlink multi-antenna multi-user system consisting of a multi-antenna base station (BS) and multiple single-antenna users, by considering the practical non-linear power amplifier (PA) efficiency and the on-off power consumption of radio frequency (RF) chain at each transmit antenna. Under this setup, we jointly optimize the transmit beamforming and antenna on/off selection at the BS to minimize its total power consumption while ensuring the individual signal-to-interference-plus-noise ratio (SINR) constraints at the users. However, due to the non-linear PA efficiency and the on-off RF chain power consumption, the formulated SINR-constrained power minimization problem is highly non-convex and difficult to solve. To tackle this issue, we propose an efficient algorithm to obtain a high-quality solution based on the technique of sequential convex approximation (SCA). We provide numerical results to validate the performance of our proposed design. It is shown that at the optimized solution, the BS tends to activate fewer antennas and use higher power transmission at each antenna to exploit the non-linear PA efficiency.

cs.IT↗
arXiv2022

Stratospheric Clouds Do Not Impede JWST Transit Spectroscopy for Exoplanets with Earth-Like Atmospheres

The James Webb Space Telescope (JWST) will provide an opportunity to investigate the atmospheres of potentially habitable planets. Aerosols, significantly mute molecular features in transit spectra because they prevent light from probing the deeper layers of the atmosphere. Earth occasionally has stratospheric/high tropospheric clouds at 15-20 km that could substantially limit the observable depth of the underlying atmosphere. We use solar occultations of Earth's atmosphere to create synthetic JWST transit spectra of Earth analogs orbiting dwarf stars. Unlike previous investigations, we consider both clear and cloudy sightlines from the SCISAT satellite. We find that the maximum difference in effective thickness of the atmosphere between a clear and globally cloudy atmosphere is 8.5 km at 2.28 microns with a resolution of 0.02 microns. After incorporating the effects of refraction and Pandexo's noise modeling, we find that JWST would not be able to detect Earth like stratospheric clouds if an exo-Earth was present in the TRAPPIST-1 system, as the cloud spectrum differs from the clear spectrum by a maximum of 10 ppm. These stratospheric clouds are also not robustly detected by TauREx when performing spectral retrieval for a cloudy TRAPPIST-1 planet. However, if an Earth size planet were to orbit in a white dwarf's habitable zone, then, we predict that JWST's NIRSpec would be able to detect its stratospheric clouds after only 4 transits. We conclude that stratospheric clouds would not impede JWST transit spectroscopy or the detection of biosignatures for Earth-like atmospheres.

astro-ph.EP↗
arXiv2022

Scattering mechanisms in state-of-the-art GaAs/AlGaAs quantum wells

Motivated by recent breakthrough in molecular beam epitaxy of GaAs/AlGaAs quantum wells [Y. J. Chung \textit{et al.}, Nature Materials \textbf{20}, 632 (2021)], we examine contributions to mobility and quantum mobility from various scattering mechanisms and their dependencies on the electron density. We find that at lower electron densities, $n_e \lesssim 1 \times 10^{11}$ cm$^{-2}$, both transport and quantum mobility are limited by unintentional background impurities and follow a power law dependence, $\propto n_e^α$, with $α\approx 0.85$. Our predictions for quantum mobility are in reasonable agreement with an estimate obtained from the resistivity at filling factor $ν= 1/2$ in a sample of Y. J. Chung \textit{et al.} with $n_e = 1 \times 10^{11}$ cm$^{-2}$. Consideration of other scattering mechanisms indicates that interface roughness (remote donors) is a likely limiting factor of transport (quantum) mobility at higher electron densities. Future measurements of quantum mobility should yield information on the distribution of background impurities in GaAs and AlGaAs.

cond-mat.mes-hall↗
arXiv2022

Curriculum-Based Self-Training Makes Better Few-Shot Learners for Data-to-Text Generation

Despite the success of text-to-text pre-trained models in various natural language generation (NLG) tasks, the generation performance is largely restricted by the number of labeled data in downstream tasks, particularly in data-to-text generation tasks. Existing works mostly utilize abundant unlabeled structured data to conduct unsupervised pre-training for task adaption, which fail to model the complex relationship between source structured data and target texts. Thus, we introduce self-training as a better few-shot learner than task-adaptive pre-training, which explicitly captures this relationship via pseudo-labeled data generated by the pre-trained model. To alleviate the side-effect of low-quality pseudo-labeled data during self-training, we propose a novel method called Curriculum-Based Self-Training (CBST) to effectively leverage unlabeled data in a rearranged order determined by the difficulty of text generation. Experimental results show that our method can outperform fine-tuning and task-adaptive pre-training methods, and achieve state-of-the-art performance in the few-shot setting of data-to-text generation.

cs.CL↗
arXiv2022

Plasmons in semiconductor and topological insulator wires with large dielectric constant

The dispersion law of plasmons running along thin wires with radius $a$ is known to be practically linear. We show that in a wire with a dielectric constant $κ$ much larger than that of its environment $κ_e$, such dispersion law crosses over to a dispersionless three-dimensional-like law when the plasmon wavelength becomes shorter than the length $(a/2) \sqrt{(κ/κ_e)\ln(κ/2κ_e)}$ at which the electric field lines of a point charge exit from the wire to the environment. This happens both in trivial semiconductor wires and wires of three-dimensional topological insulators.

cond-mat.mes-hall↗
arXiv2022

Transferable Adversarial Attack based on Integrated Gradients

The vulnerability of deep neural networks to adversarial examples has drawn tremendous attention from the community. Three approaches, optimizing standard objective functions, exploiting attention maps, and smoothing decision surfaces, are commonly used to craft adversarial examples. By tightly integrating the three approaches, we propose a new and simple algorithm named Transferable Attack based on Integrated Gradients (TAIG) in this paper, which can find highly transferable adversarial examples for black-box attacks. Unlike previous methods using multiple computational terms or combining with other methods, TAIG integrates the three approaches into one single term. Two versions of TAIG that compute their integrated gradients on a straight-line path and a random piecewise linear path are studied. Both versions offer strong transferability and can seamlessly work together with the previous methods. Experimental results demonstrate that TAIG outperforms the state-of-the-art methods. The code will available at https://github.com/yihuang2016/TAIG

cs.LG↗