arXiv subjects

Huiqiang Jiang

Publications and source records attributed to Huiqiang Jiang.

46 records · Page 3Linked to original sources

arXiv2023

CoLaDa: A Collaborative Label Denoising Framework for Cross-lingual Named Entity Recognition

Cross-lingual named entity recognition (NER) aims to train an NER system that generalizes well to a target language by leveraging labeled data in a given source language. Previous work alleviates the data scarcity problem by translating source-language labeled data or performing knowledge distillation on target-language unlabeled data. However, these methods may suffer from label noise due to the automatic labeling process. In this paper, we propose CoLaDa, a Collaborative Label Denoising Framework, to address this problem. Specifically, we first explore a model-collaboration-based denoising scheme that enables models trained on different data sources to collaboratively denoise pseudo labels used by each other. We then present an instance-collaboration-based strategy that considers the label consistency of each token's neighborhood in the representation space for denoising. Experiments on different benchmark datasets show that the proposed CoLaDa achieves superior results compared to previous methods, especially when generalizing to distant languages.

cs.CL↗

arXiv2023

ElasticViT: Conflict-aware Supernet Training for Deploying Fast Vision Transformer on Diverse Mobile Devices

Neural Architecture Search (NAS) has shown promising performance in the automatic design of vision transformers (ViT) exceeding 1G FLOPs. However, designing lightweight and low-latency ViT models for diverse mobile devices remains a big challenge. In this work, we propose ElasticViT, a two-stage NAS approach that trains a high-quality ViT supernet over a very large search space that supports a wide range of mobile devices, and then searches an optimal sub-network (subnet) for direct deployment. However, prior supernet training methods that rely on uniform sampling suffer from the gradient conflict issue: the sampled subnets can have vastly different model sizes (e.g., 50M vs. 2G FLOPs), leading to different optimization directions and inferior performance. To address this challenge, we propose two novel sampling techniques: complexity-aware sampling and performance-aware sampling. Complexity-aware sampling limits the FLOPs difference among the subnets sampled across adjacent training steps, while covering different-sized subnets in the search space. Performance-aware sampling further selects subnets that have good accuracy, which can reduce gradient conflicts and improve supernet quality. Our discovered models, ElasticViT models, achieve top-1 accuracy from 67.2% to 80.0% on ImageNet from 60M to 800M FLOPs without extra retraining, outperforming all prior CNNs and ViTs in terms of accuracy and latency. Our tiny and small models are also the first ViT models that surpass state-of-the-art CNNs with significantly lower latency on mobile devices. For instance, ElasticViT-S1 runs 2.62x faster than EfficientNet-B0 with 0.1% higher accuracy.

cs.CV↗

arXiv2022

Improving Hypernasality Estimation with Automatic Speech Recognition in Cleft Palate Speech

Hypernasality is an abnormal resonance in human speech production, especially in patients with craniofacial anomalies such as cleft palate. In clinical application, hypernasality estimation is crucial in cleft palate diagnosis, as its results determine the subsequent surgery and additional speech therapy. Therefore, designing an automatic hypernasality assessment method will facilitate speech-language pathologists to make precise diagnoses. Existing methods for hypernasality estimation only conduct acoustic analysis based on low-resource cleft palate dataset, by using statistical or neural network-based features. In this paper, we propose a novel approach that uses automatic speech recognition model to improve hypernasality estimation. Specifically, we first pre-train an encoder-decoder framework in an automatic speech recognition (ASR) objective by using speech-to-text dataset, and then fine-tune ASR encoder on the cleft palate dataset for hypernasality estimation. Benefiting from such design, our model for hypernasality estimation can enjoy the advantages of ASR model: 1) compared with low-resource cleft palate dataset, the ASR task usually includes large-scale speech data in the general domain, which enables better model generalization; 2) the text annotations in ASR dataset guide model to extract better acoustic features. Experimental results on two cleft palate datasets demonstrate that our method achieves superior performance compared with previous approaches.

eess.AS↗

arXiv2022

Decomposed Meta-Learning for Few-Shot Named Entity Recognition

Few-shot named entity recognition (NER) systems aim at recognizing novel-class named entities based on only a few labeled examples. In this paper, we present a decomposed meta-learning approach which addresses the problem of few-shot NER by sequentially tackling few-shot span detection and few-shot entity typing using meta-learning. In particular, we take the few-shot span detection as a sequence labeling problem and train the span detector by introducing the model-agnostic meta-learning (MAML) algorithm to find a good model parameter initialization that could fast adapt to new entity classes. For few-shot entity typing, we propose MAML-ProtoNet, i.e., MAML-enhanced prototypical networks to find a good embedding space that can better distinguish text span representations from different entity classes. Extensive experiments on various benchmarks show that our approach achieves superior performance over prior methods.

cs.CL↗

arXiv2021

BoningKnife: Joint Entity Mention Detection and Typing for Nested NER via prior Boundary Knowledge

While named entity recognition (NER) is a key task in natural language processing, most approaches only target flat entities, ignoring nested structures which are common in many scenarios. Most existing nested NER methods traverse all sub-sequences which is both expensive and inefficient, and also don't well consider boundary knowledge which is significant for nested entities. In this paper, we propose a joint entity mention detection and typing model via prior boundary knowledge (BoningKnife) to better handle nested NER extraction and recognition tasks. BoningKnife consists of two modules, MentionTagger and TypeClassifier. MentionTagger better leverages boundary knowledge beyond just entity start/end to improve the handling of nesting levels and longer spans, while generating high quality mention candidates. TypeClassifier utilizes a two-level attention mechanism to decouple different nested level representations and better distinguish entity types. We jointly train both modules sharing a common representation and a new dual-info attention layer, which leads to improved representation focus on entity-related information. Experiments over different datasets show that our approach outperforms previous state of the art methods and achieves 86.41, 85.46, and 94.2 F1 scores on ACE2004, ACE2005, and NNE, respectively.

cs.CL↗

arXiv2021

AdvPicker: Effectively Leveraging Unlabeled Data via Adversarial Discriminator for Cross-Lingual NER

Neural methods have been shown to achieve high performance in Named Entity Recognition (NER), but rely on costly high-quality labeled data for training, which is not always available across languages. While previous works have shown that unlabeled data in a target language can be used to improve cross-lingual model performance, we propose a novel adversarial approach (AdvPicker) to better leverage such data and further improve results. We design an adversarial learning framework in which an encoder learns entity domain knowledge from labeled source-language data and better shared features are captured via adversarial training - where a discriminator selects less language-dependent target-language data via similarity to the source language. Experimental results on standard benchmark datasets well demonstrate that the proposed method benefits strongly from this data selection process and outperforms existing state-of-the-art methods; without requiring any additional external resources (e.g., gazetteers or via machine translation). The code is available at https://aka.ms/AdvPicker

cs.CL↗

arXiv2006

On steady states of van der Waals force driven thin film equations

Let $Ω\subset\mathbb{R}^{N}$, $N\geq2$ be a bounded smooth domain and $α>1$. We are interested in the singular elliptic equation% \[ \triangle h=\frac{1}αh^{-α}-p\quad\text{in}Ω\] with Neumann boundary conditions. In this paper, we gave a complete description of all continuous radially symmetric solutions. In particular, we constructed nontrivial smooth solutions as well as rupture solutions. Here a continuous solution is said to be a rupture solution if its zero set is nonempty. When N=2 and $α=3$, the equation has been used to model steady states of van der Waals force driven thin films of viscous fluids. We also considered the physical problem when total volume of the fluid is prescribed.

math.AP↗

arXiv2006

A remark on the existence of suitable vector fields related to the dynamics of scalar semi-linear parabolic equations

In 1992, P. Poláčik\cite{P2} showed that one could linearly imbed any vector fields into a scalar semi-linear parabolic equation on $Ω$ with Neumann boundary condition provided that there exists a smooth vector field $Φ=(ϕ_{1},...,ϕ_{n}) $ on $\barΩ$ such that \[ \left\{\begin{array} [c]{l}% \operatorname*{rank}(Φ(x) ,\partial_{1}Φ(x) ,...,\partial_{n}Φ(x)) =n\text{for all}x\in\barΩ, \frac{\partialΦ}{\partialν}=0\text{on}\partialΩ\text{.}% \end{array} \right. \] In this short note, we give a classification of all the domains on which one may find such type of vector fields.

math.AP↗

arXiv2006

A priori estimates of stationary solutions of an activator-inhibitor system

We consider positive solutions of the stationary Gierer-Meinhardt system. Under suitable conditions on the exponents $p,q,r$ and $s$, different types of a priori estimates are obtained, existence and non-existence results of nontrivial solutions are derived, for both positive and zero source production cases.

math.AP↗

arXiv2006

Analytic regularity of a free boundary problem

In this paper, we consider a free boundary problem with volume constraint. We show that positive minimizer is locally Lipschitz and the free boundary is analytic away from a singular set with Hausdorff dimension at most $n-8$.

math.AP↗