arXiv subjects

Hao Zhang

Publications and source records attributed to Hao Zhang.

At least 973 records · Page 54Linked to original sources

arXiv2022

Contextual Networks and Unsupervised Ranking of Sentences

We construct a contextual network to represent a document with syntactic and semantic relations between word-sentence pairs, based on which we devise an unsupervised algorithm called CNATAR (Contextual Network And Text Analysis Rank) to score sentences, and rank them through a bi-objective 0-1 knapsack maximization problem over topic analysis and sentence scores. We show that CNATAR outperforms the combined ranking of the three human judges provided on the SummBank dataset under both ROUGE and BLEU metrics, which in term significantly outperforms each individual judge's ranking. Moreover, CNATAR produces so far the highest ROUGE scores over DUC-02, and outperforms previous supervised algorithms on the CNN/DailyMail and NYT datasets. We also compare the performance of CNATAR and the latest supervised neural-network summarization models and compute oracle results.

cs.CL↗

arXiv2022

Boilerplate Detection via Semantic Classification of TextBlocks

We present a hierarchical neural network model called SemText to detect HTML boilerplate based on a novel semantic representation of HTML tags, class names, and text blocks. We train SemText on three published datasets of news webpages and fine-tune it using a small number of development data in CleanEval and GoogleTrends-2017. We show that SemText achieves the state-of-the-art accuracy on these datasets. We then demonstrate the robustness of SemText by showing that it also detects boilerplate effectively on out-of-domain community-based question-answer webpages.

cs.CL↗

arXiv2022

Vision-Language Intelligence: Tasks, Representation Learning, and Large Models

This paper presents a comprehensive survey of vision-language (VL) intelligence from the perspective of time. This survey is inspired by the remarkable progress in both computer vision and natural language processing, and recent trends shifting from single modality processing to multiple modality comprehension. We summarize the development in this field into three time periods, namely task-specific methods, vision-language pre-training (VLP) methods, and larger models empowered by large-scale weakly-labeled data. We first take some common VL tasks as examples to introduce the development of task-specific methods. Then we focus on VLP methods and comprehensively review key components of the model structures and training methods. After that, we show how recent work utilizes large-scale raw image-text data to learn language-aligned visual representations that generalize better on zero or few shot learning tasks. Finally, we discuss some potential future trends towards modality cooperation, unified representation, and knowledge incorporation. We believe that this review will be of help for researchers and practitioners of AI and ML, especially those interested in computer vision and natural language processing.

cs.CV↗

arXiv2022

Numerical study of PbTe-Pb hybrid nanowires for engineering Majorana zero modes

Epitaxial semiconductor-superconductor (SM-SC) hybrid nanowires are potential candidates for implementing Majorana qubits. Recent experimental and theoretical works show that charged impurities in SM remain a major problem in all existing hybrid nanowires, in which the SM is either InAs or InSb while the SC is mainly Al. Here, we theoretically validate the recently proposed PbTe-Pb hybrid nanowire as a potential candidate for Majorana devices. By studying the electrostatic and electronic properties of PbTe nanowires, we demonstrate that the huge dielectric constant of PbTe endows itself a high tolerance of charged impurity, which is a potential advantage over InAs and InSb nanowires. Moreover, we find that the effective axial Landé $g$ factor and Rashba spin-orbit coupling strength of PbTe nanowires are comparable to those of InAs nanowires. The conceivable merits of using Pb as the SC are (i) Pb has a larger superconducting gap, higher critical temperature, and higher parallel critical magnetic field than those of Al; (ii) a superconducting gap comparable with those of InAs-Al and InSb-Al can be induced in PbTe-Pb even by a weak coupling between Pb and PbTe, which simultaneously relieves the adverse renormalization and induced disorder effects on SM from SC; and (iii) Pb film can be grown on PbTe with a thin buffer CdTe layer in between, solving the lattice mismatch problem as an important source of disorder. In the presence of a parallel magnetic field, we show that the typical BdG energy spectrum and tunneling spectroscopy of PbTe-Pb resemble those of InAs and InSb based hybrid nanowires exposed to a tilting magnetic field, as a result of the highly anisotropic Landé $g$ factors of PbTe nanowires. The calculated topological phase diagrams of PbTe-Pb indicate that the multivalley character of PbTe makes it easier than InAs and InSb to access topological superconducting phases.

cond-mat.mes-hall↗

arXiv2022

Neighborhood Region Smoothing Regularization for Finding Flat Minima In Deep Neural Networks

Due to diverse architectures in deep neural networks (DNNs) with severe overparameterization, regularization techniques are critical for finding optimal solutions in the huge hypothesis space. In this paper, we propose an effective regularization technique, called Neighborhood Region Smoothing (NRS). NRS leverages the finding that models would benefit from converging to flat minima, and tries to regularize the neighborhood region in weight space to yield approximate outputs. Specifically, gap between outputs of models in the neighborhood region is gauged by a defined metric based on Kullback-Leibler divergence. This metric provides similar insights with the minimum description length principle on interpreting flat minima. By minimizing both this divergence and empirical loss, NRS could explicitly drive the optimizer towards converging to flat minima. We confirm the effectiveness of NRS by performing image classification tasks across a wide range of model architectures on commonly-used datasets such as CIFAR and ImageNet, where generalization ability could be universally improved. Also, we empirically show that the minima found by NRS would have relatively smaller Hessian eigenvalues compared to the conventional method, which is considered as the evidence of flat minima.

cs.LG↗

arXiv2022

Meromorphic quasi-modular forms and their $L$-functions

We investigate the meromorphic quasi-modular forms and their $L$-functions. We study the space of meromorphic quasi-modular forms. Then we define their $L$-functions by using the technique of regularized integral. Moreover, we give an explicit formula for the $L$-functions. As an application, we obtain some vanishing results of special $L$-values of meromorphic quasi-modular forms.

math.NT↗

arXiv2022

Large Andreev bound state zero bias peaks in a weakly dissipative environment

We study Andreev bound states in hybrid InAs-Al nanowire devices. The energy of these states can be tuned to zero by gate voltage or magnetic field, revealing large zero bias peaks (ZBPs) near 2e^2/h in tunneling conductance. Probing these large ZBPs using a weakly dissipative lead reveals non-Fermi liquid temperature (T) dependence due to environmental Coulomb blockade (ECB), an interaction effect from the lead acting on the nanowire junction. By increasing T, these large ZBPs either show a height increase or a transition from split peaks to a ZBP, both deviate significantly from non-dissipative devices where a Fermi-liquid T dependence is revealed. Our result demonstrates the competing effect between ECB and thermal broadening on Andreev bound states.

cond-mat.mes-hall↗

arXiv2022

Capitalization Normalization for Language Modeling with an Accurate and Efficient Hierarchical RNN Model

Capitalization normalization (truecasing) is the task of restoring the correct case (uppercase or lowercase) of noisy text. We propose a fast, accurate and compact two-level hierarchical word-and-character-based recurrent neural network model. We use the truecaser to normalize user-generated text in a Federated Learning framework for language modeling. A case-aware language model trained on this normalized text achieves the same perplexity as a model trained on text with gold capitalization. In a real user A/B experiment, we demonstrate that the improvement translates to reduced prediction error rates in a virtual keyboard application. Similarly, in an ASR language model fusion experiment, we show reduction in uppercase character error rate and word error rate.

cs.CL↗

arXiv2022

Hierarchical Point Cloud Encoding and Decoding with Lightweight Self-Attention based Model

In this paper we present SA-CNN, a hierarchical and lightweight self-attention based encoding and decoding architecture for representation learning of point cloud data. The proposed SA-CNN introduces convolution and transposed convolution stacks to capture and generate contextual information among unordered 3D points. Following conventional hierarchical pipeline, the encoding process extracts feature in local-to-global manner, while the decoding process generates feature and point cloud in coarse-to-fine, multi-resolution stages. We demonstrate that SA-CNN is capable of a wide range of applications, namely classification, part segmentation, reconstruction, shape retrieval, and unsupervised classification. While achieving the state-of-the-art or comparable performance in the benchmarks, SA-CNN maintains its model complexity several order of magnitude lower than the others. In term of qualitative results, we visualize the multi-stage point cloud reconstructions and latent walks on rigid objects as well as deformable non-rigid human and robot models.

cs.CV↗

arXiv2022

Multi-relation Message Passing for Multi-label Text Classification

A well-known challenge associated with the multi-label classification problem is modelling dependencies between labels. Most attempts at modelling label dependencies focus on co-occurrences, ignoring the valuable information that can be extracted by detecting label subsets that rarely occur together. For example, consider customer product reviews; a product probably would not simultaneously be tagged by both "recommended" (i.e., reviewer is happy and recommends the product) and "urgent" (i.e., the review suggests immediate action to remedy an unsatisfactory experience). Aside from the consideration of positive and negative dependencies, the direction of a relationship should also be considered. For a multi-label image classification problem, the "ship" and "sea" labels have an obvious dependency, but the presence of the former implies the latter much more strongly than the other way around. These examples motivate the modelling of multiple types of bi-directional relationships between labels. In this paper, we propose a novel method, entitled Multi-relation Message Passing (MrMP), for the multi-label classification problem. Experiments on benchmark multi-label text classification datasets show that the MrMP module yields similar or superior performance compared to state-of-the-art methods. The approach imposes only minor additional computational and memory overheads.

cs.LG↗

arXiv2022

An Embarrassingly Simple Model for Dialogue Relation Extraction

Dialogue relation extraction (RE) is to predict the relation type of two entities mentioned in a dialogue. In this paper, we propose a simple yet effective model named SimpleRE for the RE task. SimpleRE captures the interrelations among multiple relations in a dialogue through a novel input format named BERT Relation Token Sequence (BRS). In BRS, multiple [CLS] tokens are used to capture possible relations between different pairs of entities mentioned in the dialogue. A Relation Refinement Gate (RRG) is then designed to extract relation-specific semantic representation in an adaptive manner. Experiments on the DialogRE dataset show that SimpleRE achieves the best performance, with much shorter training time. Further, SimpleRE outperforms all direct baselines on sentence-level RE without using external resources.

cs.CL↗

arXiv2022

A Privacy-Preserving Unsupervised Domain Adaptation Framework for Clinical Text Analysis

Unsupervised domain adaptation (UDA) generally aligns the unlabeled target domain data to the distribution of the source domain to mitigate the distribution shift problem. The standard UDA requires sharing the source data with the target, having potential data privacy leaking risks. To protect the source data's privacy, we first propose to share the source feature distribution instead of the source data. However, sharing only the source feature distribution may still suffer from the membership inference attack who can infer an individual's membership by the black-box access to the source model. To resolve this privacy issue, we further study the under-explored problem of privacy-preserving domain adaptation and propose a method with a novel differential privacy training strategy to protect the source data privacy. We model the source feature distribution by Gaussian Mixture Models (GMMs) under the differential privacy setting and send it to the target client for adaptation. The target client resamples differentially private source features from GMMs and adapts on target data with several state-of-art UDA backbones. With our proposed method, the source data provider could avoid leaking source data privacy during domain adaptation as well as reserve the utility. To evaluate our proposed method's utility and privacy loss, we apply our model on a medical report disease label classification task using two noisy challenging clinical text datasets. The results show that our proposed method can preserve source data's privacy with a minor performance influence on the text classification task.

cs.CL↗

arXiv2022

Semi-Supervised Co-Analysis of 3D Shape Styles from Projected Lines

We present a semi-supervised co-analysis method for learning 3D shape styles from projected feature lines, achieving style patch localization with only weak supervision. Given a collection of 3D shapes spanning multiple object categories and styles, we perform style co-analysis over projected feature lines of each 3D shape and then backproject the learned style features onto the 3D shapes. Our core analysis pipeline starts with mid-level patch sampling and pre-selection of candidate style patches. Projective features are then encoded via patch convolution. Multi-view feature integration and style clustering are carried out under the framework of partially shared latent factor (PSLF) learning, a multi-view feature learning scheme. PSLF achieves effective multi-view feature fusion by distilling and exploiting consistent and complementary feature information from multiple views, while also selecting style patches from the candidates. Our style analysis approach supports both unsupervised and semi-supervised analysis. For the latter, our method accepts both user-specified shape labels and style-ranked triplets as clustering constraints.We demonstrate results from 3D shape style analysis and patch localization as well as improvements over state-of-the-art methods. We also present several applications enabled by our style analysis.

cs.GR↗

arXiv2022

A Novel Self-Packaged DBBPF With Multiple TZs for 5G Applications

A self-packaged dual-band bandpass filter (DBBPF) with high isolation and low insertion loss (IL) for 5G applications is proposed in this paper. To get high stopband suppression, multiple and controllable transmission zeros (TZs) are produced. This novel DBBPF is designed with a pair of quarter-wavelength stepped-impedance resonators (QSIRs) and a half-wavelength hairpin resonator (HWHR). This DBBPF is excited by a pair of U-shape feed lines, which are designed on G6 to fully excite the resonators and to introduce source/load TZs at the same time. In this letter, the generation of two passbands and TZs will be discussed by separate electric and magnetic coupling paths (SEMCP) and mixed EM coupling analysis. This DBBPF achieves a low IL of 0.85/1.15 dB with the fractional bandwidths (FBW) of 11.0% and 6.9% at the center frequencies of 3.45 GHz and 4.9 GHz for 5G application, respectively. The total size is 0.32λg*0.45λg. Especially, three controllable TZs are introduced between two passbands.

eess.SP↗

arXiv2021

Hydrogen induced electronic transition within correlated perovskite nickelates with heavy rare-earth composition

Although discovery in hydrogen induced electronic transition within perovskite family of rare-earth nickelate (ReNiO3) opens up a new paradigm in exploring both the new materials functionality and device applications, the existing research stays at ReNiO3 with light rare-earth compositions. To further extend the cognition towards heavier rare-earth, herein we demonstrate the hydrogen induced electronic transitions for quasi-single crystalline ReNiO3/LaAlO3 (001) heterostructures, covering a large variety of the rare-earth composition from Nd to Er. The hydrogen induced elevations in the resistivity of ReNiO3 (RH/R0) show an unexpected non-monotonic tendency with the atomic number of the rare-earth composition, e.g., firstly increase from Nd to Dy and afterwards decreases from Dy to Er. Although ReNiO3 with heavy rare-earth composition (e.g. DyNiO3) exhibits large RH/R0 up to 107, their hydrogen induced electronic transition is not reversible. Further probing the electronic structures via near edge X-ray absorption fine structure analysis clearly demonstrates the respective transition in electronic structures of ReNiO3 from Ni3+ based electron itinerant orbital configurations towards the Ni2+ based electron localized state. Balancing the hydrogen induced transition reversibility with the abruption in the variations of material resistivity, we emphasize that the ReNiO3 with middle rare-earth compositions (e.g. Sm) to be most suitable that caters for the potential applications in correlated electronic devices.

cond-mat.mtrl-sci↗

arXiv2021

Observation of Aharonov-Bohm effect in PbTe nanowire networks

We report phase coherent electron transport in PbTe nanowire networks with a loop geometry. Magneto-conductance shows Aharonov-Bohm (AB) oscillations with periods of $h/e$ and $h/2e$ in flux. The amplitude of $h/2e$ oscillations is enhanced near zero magnetic field, possibly due to interference between time-reversal paths. Temperature dependence of the AB amplitudes suggests a phase coherence length $\sim$ 8 - 12 $μ$m at 50 mK. This length scale is larger than the typical geometry of PbTe-based hybrid semiconductor-superconductor nanowire devices.

cond-mat.mes-hall↗

arXiv2021

Face Deblurring Based on Separable Normalization and Adaptive Denormalization

Face deblurring aims to restore a clear face image from a blurred input image with more explicit structure and facial details. However, most conventional image and face deblurring methods focus on the whole generated image resolution without consideration of special face part texture and generally produce unsufficient details. Considering that faces and backgrounds have different distribution information, in this study, we designed an effective face deblurring network based on separable normalization and adaptive denormalization (SNADNet). First, We fine-tuned the face parsing network to obtain an accurate face structure. Then, we divided the face parsing feature into face foreground and background. Moreover, we constructed a new feature adaptive denormalization to regularize fafcial structures as a condition of the auxiliary to generate more harmonious and undistorted face structure. In addition, we proposed a texture extractor and multi-patch discriminator to enhance the generated facial texture information. Experimental results on both CelebA and CelebA-HQ datasets demonstrate that the proposed face deblurring network restores face structure with more facial details and performs favorably against state-of-the-art methods in terms of structured similarity indexing method (SSIM), peak signal-to-noise ratio (PSNR), Frechet inception distance (FID) and L1, and qualitative comparisons.

cs.CV↗

arXiv2021

Hybrid Neural Networks for On-device Directional Hearing

On-device directional hearing requires audio source separation from a given direction while achieving stringent human-imperceptible latency requirements. While neural nets can achieve significantly better performance than traditional beamformers, all existing models fall short of supporting low-latency causal inference on computationally-constrained wearables. We present DeepBeam, a hybrid model that combines traditional beamformers with a custom lightweight neural net. The former reduces the computational burden of the latter and also improves its generalizability, while the latter is designed to further reduce the memory and computational overhead to enable real-time and low-latency operations. Our evaluation shows comparable performance to state-of-the-art causal inference models on synthetic data while achieving a 5x reduction of model size, 4x reduction of computation per second, 5x reduction in processing time and generalizing better to real hardware data. Further, our real-time hybrid model runs in 8 ms on mobile CPUs designed for low-power wearable devices and achieves an end-to-end latency of 17.5 ms.

cs.SD↗