arXiv subjects

Jing Huang

Publications and source records attributed to Jing Huang.

At least 199 records · Page 11Linked to original sources

arXiv2021

Speaker-invariant Affective Representation Learning via Adversarial Training

Representation learning for speech emotion recognition is challenging due to labeled data sparsity issue and lack of gold standard references. In addition, there is much variability from input speech signals, human subjective perception of the signals and emotion label ambiguity. In this paper, we propose a machine learning framework to obtain speech emotion representations by limiting the effect of speaker variability in the speech signals. Specifically, we propose to disentangle the speaker characteristics from emotion through an adversarial training network in order to better represent emotion. Our method combines the gradient reversal technique with an entropy loss function to remove such speaker information. Our approach is evaluated on both IEMOCAP and CMU-MOSEI datasets. We show that our method improves speech emotion classification and increases generalization to unseen speakers.

eess.AS↗

arXiv2021

Energy and spectral analysis of confined solar flares from radio and X-ray observations

The energy and spectral shape of radio bursts may help us understand the generation mechanism of solar eruptions, including solar flares, CMEs, eruptive filaments, and various scales of jets. The different kinds of flares may have different characteristics of energy and spectral distribution. In this work, we selected 10 mostly confined flare events during October 2014 to investigate their overall spectral behavior and the energy emitted in microwaves by using radio observations from microwaves to interplanetary radio waves, and X-ray observations of GOES, RHESSI, and Fermi/GBM. We found that: All the confined flare events were associated with a microwave continuum burst extending to frequencies of 9.4 - 15.4 GHz, and the peak frequencies of all confined flare events are higher than 4.995 GHz and lower than or equal to 17 GHz. The median value is around 9 GHz. The microwave burst energy (or fluence) as well as the peak frequency are found to provide useful criteria to estimate the power of solar flares. The observations imply that the magnetic field in confined flares tends to be stronger than that in 412 flares studied by Nita et al. 2004. All 10 events studied did not produce detectable hard X-rays with energies above 300 keV indicating the lack of efficient acceleration of electrons to high energies in the confined flares.

astro-ph.SR↗

arXiv2021

Joint System-Wise Optimization for Pipeline Goal-Oriented Dialog System

Recent work (Takanobu et al., 2020) proposed the system-wise evaluation on dialog systems and found that improvement on individual components (e.g., NLU, policy) in prior work may not necessarily bring benefit to pipeline systems in system-wise evaluation. To improve the system-wise performance, in this paper, we propose new joint system-wise optimization techniques for the pipeline dialog system. First, we propose a new data augmentation approach which automates the labeling process for NLU training. Second, we propose a novel stochastic policy parameterization with Poisson distribution that enables better exploration and offers a principled way to compute policy gradient. Third, we propose a reward bonus to help policy explore successful dialogs. Our approaches outperform the competitive pipeline systems from Takanobu et al. (2020) by big margins of 12% success rate in automatic system-wise evaluation and of 16% success rate in human evaluation on the standard multi-domain benchmark dataset MultiWOZ 2.1, and also outperform the recent state-of-the-art end-to-end trained model from DSTC9.

cs.CL↗

arXiv2021

TextOCR: Towards large-scale end-to-end reasoning for arbitrary-shaped scene text

A crucial component for the scene text based reasoning required for TextVQA and TextCaps datasets involve detecting and recognizing text present in the images using an optical character recognition (OCR) system. The current systems are crippled by the unavailability of ground truth text annotations for these datasets as well as lack of scene text detection and recognition datasets on real images disallowing the progress in the field of OCR and evaluation of scene text based reasoning in isolation from OCR systems. In this work, we propose TextOCR, an arbitrary-shaped scene text detection and recognition with 900k annotated words collected on real images from TextVQA dataset. We show that current state-of-the-art text-recognition (OCR) models fail to perform well on TextOCR and that training on TextOCR helps achieve state-of-the-art performance on multiple other OCR datasets as well. We use a TextOCR trained OCR model to create PixelM4C model which can do scene text based reasoning on an image in an end-to-end fashion, allowing us to revisit several design choices to achieve new state-of-the-art performance on TextVQA dataset.

cs.CV↗

arXiv2021

UniGNN: a Unified Framework for Graph and Hypergraph Neural Networks

Hypergraph, an expressive structure with flexibility to model the higher-order correlations among entities, has recently attracted increasing attention from various research domains. Despite the success of Graph Neural Networks (GNNs) for graph representation learning, how to adapt the powerful GNN-variants directly into hypergraphs remains a challenging problem. In this paper, we propose UniGNN, a unified framework for interpreting the message passing process in graph and hypergraph neural networks, which can generalize general GNN models into hypergraphs. In this framework, meticulously-designed architectures aiming to deepen GNNs can also be incorporated into hypergraphs with the least effort. Extensive experiments have been conducted to demonstrate the effectiveness of UniGNN on multiple real-world datasets, which outperform the state-of-the-art approaches with a large margin. Especially for the DBLP dataset, we increase the accuracy from 77.4\% to 88.8\% in the semi-supervised hypernode classification task. We further prove that the proposed message-passing based UniGNN models are at most as powerful as the 1-dimensional Generalized Weisfeiler-Leman (1-GWL) algorithm in terms of distinguishing non-isomorphic hypergraphs. Our code is available at \url{https://github.com/OneForward/UniGNN}.

cs.LG↗

arXiv2021

Residual Enhanced Multi-Hypergraph Neural Network

Hypergraphs are a generalized data structure of graphs to model higher-order correlations among entities, which have been successfully adopted into various research domains. Meanwhile, HyperGraph Neural Network (HGNN) is currently the de-facto method for hypergraph representation learning. However, HGNN aims at single hypergraph learning and uses a pre-concatenation approach when confronting multi-modal datasets, which leads to sub-optimal exploitation of the inter-correlations of multi-modal hypergraphs. HGNN also suffers the over-smoothing issue, that is, its performance drops significantly when layers are stacked up. To resolve these issues, we propose the Residual enhanced Multi-Hypergraph Neural Network, which can not only fuse multi-modal information from each hypergraph effectively, but also circumvent the over-smoothing issue associated with HGNN. We conduct experiments on two 3D benchmarks, the NTU and the ModelNet40 datasets, and compare against multiple state-of-the-art methods. Experimental results demonstrate that both the residual hypergraph convolutions and the multi-fusion architecture can improve the performance of the base model and the combined model achieves a new state-of-the-art. Code is available at \url{https://github.com/OneForward/ResMHGNN}.

cs.CV↗

arXiv2021

Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation

Weakly supervised phrase grounding aims at learning region-phrase correspondences using only image-sentence pairs. A major challenge thus lies in the missing links between image regions and sentence phrases during training. To address this challenge, we leverage a generic object detector at training time, and propose a contrastive learning framework that accounts for both region-phrase and image-sentence matching. Our core innovation is the learning of a region-phrase score function, based on which an image-sentence score function is further constructed. Importantly, our region-phrase score function is learned by distilling from soft matching scores between the detected object names and candidate phrases within an image-sentence pair, while the image-sentence score function is supervised by ground-truth image-sentence pairs. The design of such score functions removes the need of object detection at test time, thereby significantly reducing the inference cost. Without bells and whistles, our approach achieves state-of-the-art results on visual phrase grounding, surpassing previous methods that require expensive object detectors at test time.

cs.CV↗

arXiv2021

A Multiplexed Network for End-to-End, Multilingual OCR

Recent advances in OCR have shown that an end-to-end (E2E) training pipeline that includes both detection and recognition leads to the best results. However, many existing methods focus primarily on Latin-alphabet languages, often even only case-insensitive English characters. In this paper, we propose an E2E approach, Multiplexed Multilingual Mask TextSpotter, that performs script identification at the word level and handles different scripts with different recognition heads, all while maintaining a unified loss that simultaneously optimizes script identification and multiple recognition heads. Experiments show that our method outperforms the single-head model with similar number of parameters in end-to-end recognition tasks, and achieves state-of-the-art results on MLT17 and MLT19 joint text detection and script identification benchmarks. We believe that our work is a step towards the end-to-end trainable and scalable multilingual multi-purpose OCR system. Our code and model will be released.

cs.CV↗

arXiv2021

Graph Ensemble Learning over Multiple Dependency Trees for Aspect-level Sentiment Classification

Recent work on aspect-level sentiment classification has demonstrated the efficacy of incorporating syntactic structures such as dependency trees with graph neural networks(GNN), but these approaches are usually vulnerable to parsing errors. To better leverage syntactic information in the face of unavoidable errors, we propose a simple yet effective graph ensemble technique, GraphMerge, to make use of the predictions from differ-ent parsers. Instead of assigning one set of model parameters to each dependency tree, we first combine the dependency relations from different parses before applying GNNs over the resulting graph. This allows GNN mod-els to be robust to parse errors at no additional computational cost, and helps avoid overparameterization and overfitting from GNN layer stacking by introducing more connectivity into the ensemble graph. Our experiments on the SemEval 2014 Task 4 and ACL 14 Twitter datasets show that our GraphMerge model not only outperforms models with single dependency tree, but also beats other ensemble mod-els without adding model parameters.

cs.CL↗

arXiv2021

Chordality of locally semicomplete and weakly quasi-transitive digraphs

Chordal graphs are important in the structural and algorithmic graph theory. A digraph analogue of chordal graphs was introduced by Haskin and Rose in 1973 but has not been a subject of active studies until recently when a characterization of semicomplete chordal digraphs in terms of forbidden subdigraphs was found by Meister and Telle. Locally semicomplete digraphs, quasi-transitive digraphs, and extended semi-complete digraphs are amongst the most popular generalizations of semicomplete digraphs. We extend the forbidden subdigraph characterization of semicomplete chordal digraphs to locally semicomplete chordal digraphs. We introduce a new class of digraphs, called weakly quasi-transitive digraphs, which contains quasi-transitive digraphs, symmetric digraphs, and extended semicomplete digraphs, but is incomparable to the class of locally semicomplete digraphs. We show that weakly quasi-transitive digraphs can be recursively constructed by simple substitutions from transitive oriented graphs, semicomplete digraphs, and symmetric digraphs. This recursive construction of weakly quasi-transitive digraphs, similar to the one for quasi-transitive digraphs, demonstrates the naturalness of the new digraph class. As a by-product, we prove that the forbidden subdigraphs for semicomplete chordal digraphs are the same for weakly quasi-transitive chordal digraphs.

math.CO↗

arXiv2021

Inductive Learning on Commonsense Knowledge Graph Completion

Commonsense knowledge graph (CKG) is a special type of knowledge graph (KG), where entities are composed of free-form text. However, most existing CKG completion methods focus on the setting where all the entities are presented at training time. Although this setting is standard for conventional KG completion, it has limitations for CKG completion. At test time, entities in CKGs can be unseen because they may have unseen text/names and entities may be disconnected from the training graph, since CKGs are generally very sparse. Here, we propose to study the inductive learning setting for CKG completion where unseen entities may present at test time. We develop a novel learning framework named InductivE. Different from previous approaches, InductiveE ensures the inductive learning capability by directly computing entity embeddings from raw entity attributes/text. InductiveE consists of a free-text encoder, a graph encoder, and a KG completion decoder. Specifically, the free-text encoder first extracts the textual representation of each entity based on the pre-trained language model and word embedding. The graph encoder is a gated relational graph convolutional neural network that learns from a densified graph for more informative entity representation learning. We develop a method that densifies CKGs by adding edges among semantic-related entities and provide more supportive information for unseen entities, leading to better generalization ability of entity embedding for unseen entities. Finally, inductiveE employs Conv-TransE as the CKG completion decoder. Experimental results show that InductiveE significantly outperforms state-of-the-art baselines in both standard and inductive settings on ATOMIC and ConceptNet benchmarks. InductivE performs especially well on inductive scenarios where it achieves above 48% improvement over present methods.

cs.AI↗

arXiv2021

Selective Attention Based Graph Convolutional Networks for Aspect-Level Sentiment Classification

Aspect-level sentiment classification aims to identify the sentiment polarity towards a specific aspect term in a sentence. Most current approaches mainly consider the semantic information by utilizing attention mechanisms to capture the interactions between the context and the aspect term. In this paper, we propose to employ graph convolutional networks (GCNs) on the dependency tree to learn syntax-aware representations of aspect terms. GCNs often show the best performance with two layers, and deeper GCNs do not bring additional gain due to over-smoothing problem. However, in some cases, important context words cannot be reached within two hops on the dependency tree. Therefore we design a selective attention based GCN block (SA-GCN) to find the most important context words, and directly aggregate these information into the aspect-term representation. We conduct experiments on the SemEval 2014 Task 4 datasets. Our experimental results show that our model outperforms the current state-of-the-art.

cs.CL↗

arXiv2020

Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling

Document-level relation extraction (RE) poses new challenges compared to its sentence-level counterpart. One document commonly contains multiple entity pairs, and one entity pair occurs multiple times in the document associated with multiple possible relations. In this paper, we propose two novel techniques, adaptive thresholding and localized context pooling, to solve the multi-label and multi-entity problems. The adaptive thresholding replaces the global threshold for multi-label classification in the prior work with a learnable entities-dependent threshold. The localized context pooling directly transfers attention from pre-trained language models to locate relevant context that is useful to decide the relation. We experiment on three document-level RE benchmark datasets: DocRED, a recently released large-scale RE dataset, and two datasets CDRand GDA in the biomedical domain. Our ATLOP (Adaptive Thresholding and Localized cOntext Pooling) model achieves an F1 score of 63.4, and also significantly outperforms existing models on both CDR and GDA.

cs.CL↗

arXiv2020

Entity and Evidence Guided Relation Extraction for DocRED

Document-level relation extraction is a challenging task which requires reasoning over multiple sentences in order to predict relations in a document. In this paper, we pro-pose a joint training frameworkE2GRE(Entity and Evidence Guided Relation Extraction)for this task. First, we introduce entity-guided sequences as inputs to a pre-trained language model (e.g. BERT, RoBERTa). These entity-guided sequences help a pre-trained language model (LM) to focus on areas of the document related to the entity. Secondly, we guide the fine-tuning of the pre-trained language model by using its internal attention probabilities as additional features for evidence prediction.Our new approach encourages the pre-trained language model to focus on the entities and supporting/evidence sentences. We evaluate our E2GRE approach on DocRED, a recently released large-scale dataset for relation extraction. Our approach is able to achieve state-of-the-art results on the public leaderboard across all metrics, showing that our E2GRE is both effective and synergistic on relation extraction and evidence prediction.

cs.CL↗

arXiv2020

Obstructions for acyclic local tournament orientation completions

The orientation completion problem for a fixed class of oriented graphs asks whether a given partially oriented graph can be completed to an oriented graph in the class. Orientation completion problems have been studied recently for several classes of oriented graphs, yielding both polynomial time solutions and NP-completeness results. Local tournaments are a well-structured class of oriented graphs that generalize tournaments and their underlying graphs are intimately related to proper circular-arc graphs. Proper interval graphs are precisely those which can be oriented as acyclic local tournaments. It has been proved that the orientation completion problems for local tournaments and acyclic local tournaments are both polynomial time solvable. In this paper we identify the obstructions for acyclic local tournament orientation completions. These are in a sense minimal partially oriented graphs that cannot be completed to acyclic local tournaments. Our description of the obstructions imply that they can be recognized in polynomial time. In a companion paper we will determine all obstructions for local tournament orientation completions.

math.CO↗

arXiv2020

(k; l)-Colourings and Ferrers Diagram Representations of Cographs

For a pair of natural numbers $k, l$, a $(k,l)$-colouring of a graph $G$ is a partition of the vertex set of $G$ into (possibly empty) sets $S_1, S_2, \dots, S_k$, $C_1, C_2, \dots, C_l$ such that each set $S_i$ is an independent set and each set $C_j$ induces a clique in $G$. The $(k,l)$-colouring problem, which is NP-complete in general, has been studied for special graph classes such as chordal graphs, cographs and line graphs. Let $\hatκ(G) = (κ_0(G),κ_1(G),\dots,κ_{θ(G)-1}(G))$ and $\hatλ(G) = (λ_0(G),λ_1(G),\dots,λ_{χ(G)-1}(G))$ where $κ_l(G)$ (respectively, $λ_k(G)$) is the minimum $k$ (respectively, $l$) such that $G$ has a $(k,l)$-colouring. We prove that $\hatκ(G)$ and $\hatλ(G)$ are a pair of conjugate sequences for every graph $G$ and when $G$ is a cograph, the number of vertices in $G$ is equal to the sum of the entries in $\hatκ(G)$ or in $\hatλ(G)$. Using the decomposition property of cographs we show that every cograph can be represented by Ferrers diagram. We devise algorithms which compute $\hatκ(G)$ for cographs $G$ and find an induced subgraph in $G$ that can be used to certify the non-$(k,l)$-colourability of $G$.

math.CO↗

arXiv2020

Modeling the broadest spectral band of the Crab nebula and constraining the ions acceleration efficiency

Although it is widely accepted that the electromagnetic spectrum from radio to very-high-energy $γ$-rays of pulsar wind nebulae (PWNe) originates from leptons, there is still an open question that protons (or more generally, ions) may exist in pulsar wind and are further accelerated in PWN. The broadband spectrum of the prototype PWN Crab, extended recently by the detection of the Tibet AS$γ$ and HAWC experiments above 100 TeV, may be helpful in constraining the acceleration efficiency of ions. Here, we model the broadest energy spectrum of Crab and find that the broadband spectrum can be explained by the one-zone leptonic model in which the electrons/positrons produce the emission from radio to soft $γ$-rays via the synchrotron process, and simultaneously generate the GeV-TeV $γ$-rays through inverse Compton scattering including the synchrotron self-Compton process. In the framework of this leptonic model, the fraction of energy converted into the energetic protons is constrained to be below $0.5\ (n_{\rm t}/10\ {\rm cm}^{-3})^{-1}$ per cent, where $n_{\rm t}$ is the target gas density in the Crab. However, this fraction can be up to $7\ (n_{\rm t}/10\ {\rm cm}^{-3})^{-1}$ per cent if only the $γ$-rays are used.

astro-ph.HE↗

arXiv2020

Mask TextSpotter v3: Segmentation Proposal Network for Robust Scene Text Spotting

Recent end-to-end trainable methods for scene text spotting, integrating detection and recognition, showed much progress. However, most of the current arbitrary-shape scene text spotters use region proposal networks (RPN) to produce proposals. RPN relies heavily on manually designed anchors and its proposals are represented with axis-aligned rectangles. The former presents difficulties in handling text instances of extreme aspect ratios or irregular shapes, and the latter often includes multiple neighboring instances into a single proposal, in cases of densely oriented text. To tackle these problems, we propose Mask TextSpotter v3, an end-to-end trainable scene text spotter that adopts a Segmentation Proposal Network (SPN) instead of an RPN. Our SPN is anchor-free and gives accurate representations of arbitrary-shape proposals. It is therefore superior to RPN in detecting text instances of extreme aspect ratios or irregular shapes. Furthermore, the accurate proposals produced by SPN allow masked RoI features to be used for decoupling neighboring text instances. As a result, our Mask TextSpotter v3 can handle text instances of extreme aspect ratios or irregular shapes, and its recognition accuracy won't be affected by nearby text or background noise. Specifically, we outperform state-of-the-art methods by 21.9 percent on the Rotated ICDAR 2013 dataset (rotation robustness), 5.9 percent on the Total-Text dataset (shape robustness), and achieve state-of-the-art performance on the MSRA-TD500 dataset (aspect ratio robustness). Code is available at: https://github.com/MhLiao/MaskTextSpotterV3

cs.CV↗