Searcharxiv⌕ Search

arXiv subjects

Chong Zhang

Publications and source records attributed to Chong Zhang.

At least 127 records · Page 7Linked to original sources
arXiv2023

Reading Order Matters: Information Extraction from Visually-rich Documents by Token Path Prediction

Recent advances in multimodal pre-trained models have significantly improved information extraction from visually-rich documents (VrDs), in which named entity recognition (NER) is treated as a sequence-labeling task of predicting the BIO entity tags for tokens, following the typical setting of NLP. However, BIO-tagging scheme relies on the correct order of model inputs, which is not guaranteed in real-world NER on scanned VrDs where text are recognized and arranged by OCR systems. Such reading order issue hinders the accurate marking of entities by BIO-tagging scheme, making it impossible for sequence-labeling methods to predict correct named entities. To address the reading order issue, we introduce Token Path Prediction (TPP), a simple prediction head to predict entity mentions as token sequences within documents. Alternative to token classification, TPP models the document layout as a complete directed graph of tokens, and predicts token paths within the graph as entities. For better evaluation of VrD-NER systems, we also propose two revised benchmark datasets of NER on scanned documents which can reflect real-world scenarios. Experiment results demonstrate the effectiveness of our method, and suggest its potential to be a universal solution to various information extraction tasks on documents.

cs.CL↗
arXiv2023

Resilient Legged Local Navigation: Learning to Traverse with Compromised Perception End-to-End

Autonomous robots must navigate reliably in unknown environments even under compromised exteroceptive perception, or perception failures. Such failures often occur when harsh environments lead to degraded sensing, or when the perception algorithm misinterprets the scene due to limited generalization. In this paper, we model perception failures as invisible obstacles and pits, and train a reinforcement learning (RL) based local navigation policy to guide our legged robot. Unlike previous works relying on heuristics and anomaly detection to update navigational information, we train our navigation policy to reconstruct the environment information in the latent space from corrupted perception and react to perception failures end-to-end. To this end, we incorporate both proprioception and exteroception into our policy inputs, thereby enabling the policy to sense collisions on different body parts and pits, prompting corresponding reactions. We validate our approach in simulation and on the real quadruped robot ANYmal running in real-time (<10 ms CPU inference). In a quantitative comparison with existing heuristic-based locally reactive planners, our policy increases the success rate over 30% when facing perception failures. Project Page: https://bit.ly/45NBTuh.

cs.RO↗
arXiv2023

Are Soft Prompts Good Zero-shot Learners for Speech Recognition?

Large self-supervised pre-trained speech models require computationally expensive fine-tuning for downstream tasks. Soft prompt tuning offers a simple parameter-efficient alternative by utilizing minimal soft prompt guidance, enhancing portability while also maintaining competitive performance. However, not many people understand how and why this is so. In this study, we aim to deepen our understanding of this emerging method by investigating the role of soft prompts in automatic speech recognition (ASR). Our findings highlight their role as zero-shot learners in improving ASR performance but also make them vulnerable to malicious modifications. Soft prompts aid generalization but are not obligatory for inference. We also identify two primary roles of soft prompts: content refinement and noise information enhancement, which enhances robustness against background noise. Additionally, we propose an effective modification on noise prompts to show that they are capable of zero-shot learning on adapting to out-of-distribution noise environments.

cs.SD↗
arXiv2023

Biocompatible wearable touch panel based on ionically conductive organic hydrogels with anti-freezing, anti-dehydration, self-healing, and underwater adhesion properties

Next-generation touch panels are poised to benefit from the use of stretchable and transparent soft ionic conductors, but these materials face several challenges in practical application, including structural damage, loss of functionality, and device stratification, particularly in extreme environments. To address these challenges, in this work, a biocompatible, transparent, self-adhesive gelatin-PAA-based organic hydrogel (PC-OH) was developed, the gel can adhere to the skin in both air and underwater conditions and also anti-freezing, anti-drying, fast self-healable (with a self-healing time of less than 4s in air and underwater), long-term stable for up to 7 days at a wide range of temperatures, highly stretchable, and conductive over a wide temperature range. Using this organic hydrogel, an organic hydrogel-based surface capacitive touch system has been developed that can detect finger touch position in wet environments and over a wide temperature range, demonstrating its ability to sense finger touch position. The wearable touch panel has been successfully demonstrated through the ability to write text, draw figures, and play electronic games, showcasing its potential use in various applications. This breakthrough has significant implications for the development of next-generation touch panels, particularly in the healthcare, sports, and entertainment industries, where reliable and versatile human-machine interfaces are essential.

cond-mat.soft↗
arXiv2023

Learning Terrain-Adaptive Locomotion with Agile Behaviors by Imitating Animals

In this paper, we present a general learning framework for controlling a quadruped robot that can mimic the behavior of real animals and traverse challenging terrains. Our method consists of two steps: an imitation learning step to learn from motions of real animals, and a terrain adaptation step to enable generalization to unseen terrains. We capture motions from a Labrador on various terrains to facilitate terrain adaptive locomotion. Our experiments demonstrate that our policy can traverse various terrains and produce a natural-looking behavior. We deployed our method on the real quadruped robot Max via zero-shot simulation-to-reality transfer, achieving a speed of 1.1 m/s on stairs climbing.

cs.RO↗
arXiv2023

Single Diamond Structured Titania Scaffold

The single diamond (SD) network, discovered in beetle and weevil skeletons, is the 'holy grail' of photonic materials with the widest complete bandgap known to date. However, the thermodynamic instability of SD has made its self-assembly long been a formidable challenge. By imitating the simultaneous co-folding process of nonequilibrium skeleton formation in natural organisms, we devised an unprecedented bottom-up approach to fabricate SD networks via the synergistic self-assembly of diblock copolymer and inorganic precursors and successfully obtained tetrahedral connected polycrystalline anatase SD frameworks. A photonic bandstructure calculation showed that the resulting SD structure has a wide and complete photonic bandgap. This work provides an ingenious design solution to the complex synthetic puzzle and offers new opportunities for biorelevant materials, next-generation optical devices, etc.

physics.app-ph↗
arXiv2023

A Simple and Effective Baseline for Attentional Generative Adversarial Networks

Synthesising a text-to-image model of high-quality images by guiding the generative model through the Text description is an innovative and challenging task. In recent years, AttnGAN based on the Attention mechanism to guide GAN training has been proposed, SD-GAN, which adopts a self-distillation technique to improve the performance of the generator and the quality of image generation, and Stack-GAN++, which gradually improves the details and quality of the image by stacking multiple generators and discriminators. However, this series of improvements to GAN all have redundancy to a certain extent, which affects the generation performance and complexity to a certain extent. We use the popular simple and effective idea (1) to remove redundancy structure and improve the backbone network of AttnGAN. (2) to integrate and reconstruct multiple losses of DAMSM. Our improvements have significantly improved the model size and training efficiency while ensuring that the model's performance is unchanged and finally proposed our SEAttnGAN. Code is avalilable at https://github.com/jmyissb/SEAttnGAN.

cs.CV↗
arXiv2023

Joint Downlink-Uplink Beamforming for Wireless Multi-Antenna Federated Learning

We study joint downlink-uplink beamforming design for wireless federated learning (FL) with a multi-antenna base station. Considering analog transmission over noisy channels and uplink over-the-air aggregation, we derive the global model update expression over communication rounds. We then obtain an upper bound on the expected global loss function, capturing the downlink and uplink beamforming and receiver noise effect. We propose a low-complexity joint beamforming algorithm to minimize this upper bound, which employs alternating optimization to breakdown the problem into three subproblems, each solved via closed-form gradient updates. Simulation under practical wireless system setup shows that our proposed joint beamforming design solution substantially outperforms the conventional separate-link design approach and nearly attains the performance of ideal FL with error-free communication links.

cs.IT↗
arXiv2023

HiTIN: Hierarchy-aware Tree Isomorphism Network for Hierarchical Text Classification

Hierarchical text classification (HTC) is a challenging subtask of multi-label classification as the labels form a complex hierarchical structure. Existing dual-encoder methods in HTC achieve weak performance gains with huge memory overheads and their structure encoders heavily rely on domain knowledge. Under such observation, we tend to investigate the feasibility of a memory-friendly model with strong generalization capability that could boost the performance of HTC without prior statistics or label semantics. In this paper, we propose Hierarchy-aware Tree Isomorphism Network (HiTIN) to enhance the text representations with only syntactic information of the label hierarchy. Specifically, we convert the label hierarchy into an unweighted tree structure, termed coding tree, with the guidance of structural entropy. Then we design a structure encoder to incorporate hierarchy-aware information in the coding tree into text representations. Besides the text encoder, HiTIN only contains a few multi-layer perceptions and linear transformations, which greatly saves memory. We conduct experiments on three commonly used datasets and the results demonstrate that HiTIN could achieve better test performance and less memory consumption than state-of-the-art (SOTA) methods.

cs.CL↗
arXiv2023

ACA-Net: Towards Lightweight Speaker Verification using Asymmetric Cross Attention

In this paper, we propose ACA-Net, a lightweight, global context-aware speaker embedding extractor for Speaker Verification (SV) that improves upon existing work by using Asymmetric Cross Attention (ACA) to replace temporal pooling. ACA is able to distill large, variable-length sequences into small, fixed-sized latents by attending a small query to large key and value matrices. In ACA-Net, we build a Multi-Layer Aggregation (MLA) block using ACA to generate fixed-sized identity vectors from variable-length inputs. Through global attention, ACA-Net acts as an efficient global feature extractor that adapts to temporal variability unlike existing SV models that apply a fixed function for pooling over the temporal dimension which may obscure information about the signal's non-stationary temporal variability. Our experiments on the WSJ0-1talker show ACA-Net outperforms a strong baseline by 5\% relative improvement in EER using only 1/5 of the parameters.

cs.SD↗
arXiv2023

ET-WB: water balance-based estimations of terrestrial evaporation over global land and major global basins

The prevailing approaches for ET retrievals are either limited in spatiotemporal coverage or largely influenced by choice of input data or simplified model physics, or a combination thereof. Here, using an independent mass conservation approach, we develop water balance-based ET datasets (ET-WB) for the global land and the selected 168 major river basins. We generate 4669 probabilistic unique combinations of the ET-WB leveraging multi-source datasets (23 precipitation, 29 runoff, and 7 storage change datasets) from satellite products, in-situ measurements, reanalysis, and hydrological simulations. We compare our results with the four auxiliary global ET datasets and previous regional studies, followed by a rigorous discussion of the uncertainties, their possible sources, and potential ways to constrain them. The seasonal cycle of global ET-WB possesses a unimodal distribution with the highest (median value: 65.61 mm/month) and lowest (median value: 36.11 mm/month) values in July and January, respectively, with the spread range of roughly +/-10 mm/month from different subsets of the ensemble. Auxiliary ET products illustrate similar intra-annual characteristics with some over/under-estimation, which are completely within the range of the ET-WB ensemble. We found a gradual increase in global ET-WB from 2003 to 2010 and a subsequent decrease during 2010-2015, followed by a sharper reduction in the remaining years primarily attributed to the varying precipitation. Multiple statistical metrics show reasonably good accuracy of monthly ET-WB (e.g., a relative bias of +/-20%) in most river basins, which ameliorates at annual scales. The long-term mean annual ET-WB varies within 500-600 mm/yr and is consistent with the for auxiliary ET products (543-569 mm/yr). Observed trend estimates, though regionally divergent, are evidence of the increasing ET in a warming climate.

physics.ao-ph↗
arXiv2023

Contrastive Speech Mixup for Low-resource Keyword Spotting

Most of the existing neural-based models for keyword spotting (KWS) in smart devices require thousands of training samples to learn a decent audio representation. However, with the rising demand for smart devices to become more personalized, KWS models need to adapt quickly to smaller user samples. To tackle this challenge, we propose a contrastive speech mixup (CosMix) learning algorithm for low-resource KWS. CosMix introduces an auxiliary contrastive loss to the existing mixup augmentation technique to maximize the relative similarity between the original pre-mixed samples and the augmented samples. The goal is to inject enhancing constraints to guide the model towards simpler but richer content-based speech representations from two augmented views (i.e. noisy mixed and clean pre-mixed utterances). We conduct our experiments on the Google Speech Command dataset, where we trim the size of the training set to as small as 2.5 mins per keyword to simulate a low-resource condition. Our experimental results show a consistent improvement in the performance of multiple models, which exhibits the effectiveness of our method.

cs.SD↗
arXiv2023

Detecting Political Opinions in Tweets through Bipartite Graph Analysis: A Skip Aggregation Graph Convolution Approach

Public opinion is a crucial factor in shaping political decision-making. Nowadays, social media has become an essential platform for individuals to engage in political discussions and express their political views, presenting researchers with an invaluable resource for analyzing public opinion. In this paper, we focus on the 2020 US presidential election and create a large-scale dataset from Twitter. To detect political opinions in tweets, we build a user-tweet bipartite graph based on users' posting and retweeting behaviors and convert the task into a Graph Neural Network (GNN)-based node classification problem. Then, we introduce a novel skip aggregation mechanism that makes tweet nodes aggregate information from second-order neighbors, which are also tweet nodes due to the graph's bipartite nature, effectively leveraging user behavioral information. The experimental results show that our proposed model significantly outperforms several competitive baselines. Further analyses demonstrate the significance of user behavioral information and the effectiveness of skip aggregation.

cs.SI↗
arXiv2023

Fast First-Order Algorithm for Large-Scale Max-Min Fair Multi-Group Multicast Beamforming

We propose a first-order fast algorithm for the weighted max-min fair (MMF) multi-group multicast beamforming problem in large-scale systems. Utilizing the optimal multicast beamforming structure obtained recently, we convert the nonconvex MMF problem into a min-max weight minimization problem and show that it is a weakly convex problem. We propose using the projected subgradient algorithm (PSA) to solve the problem directly, instead of the conventional method that requires iteratively solving its inverse problem. We show that PSA for our problem has closed-form updates and thus is computationally cheap. Furthermore, PSA converges to a near-stationary point of our problem within finite time. Simulation results show that our PSA-based algorithm offers near-optimal performance with considerably lower computational complexity than existing methods for large-scale systems.

cs.IT↗
arXiv2023

Ultra-Low-Complexity Algorithms with Structurally Optimal Multi-Group Multicast Beamforming in Large-Scale Systems

In this work, we propose ultra-low-complexity design solutions for multi-group multicast beamforming in large-scale systems. For the quality-of-service (QoS) problem, by utilizing the optimal multicast beamforming structure obtained recently in [2], we convert the original problem into a non-convex weight optimization problem of a lower dimension and propose two fast first-order algorithms to solve it. Both algorithms are based on successive convex approximation (SCA) and provide fast iterative updates to solve each SCA subproblem. The first algorithm uses a saddle point reformulation in the dual domain and applies the extragradient method with an adaptive step-size procedure to find the saddle point with simple closed-form updates. The second algorithm adopts the alternating direction method of multipliers (ADMM) method by converting each SCA subproblem into a favorable ADMM structure. The structure leads to simple closed-form ADMM updates, where the problem in each update block can be further decomposed into parallel subproblems of small sizes, for which closed-form solutions are obtained. We also propose efficient initialization methods to obtain favorable initial points that facilitate fast convergence. Furthermore, taking advantage of the proposed fast algorithms, for the max-min fair (MMF) problem, we propose a simple closed-form scaling scheme that directly uses the solution obtained from the QoS problem, avoiding the conventional computationally expensive method that iteratively solves the inverse QoS problem. We further develop lower and upper bounds on the performance of this scaling scheme. Simulation results show that the proposed algorithms offer near-optimal performance with substantially lower computational complexity than the state-of-the-art algorithms for large-scale systems.

cs.IT↗
arXiv2023

Adaptive Knowledge Distillation between Text and Speech Pre-trained Models

Learning on a massive amount of speech corpus leads to the recent success of many self-supervised speech models. With knowledge distillation, these models may also benefit from the knowledge encoded by language models that are pre-trained on rich sources of texts. The distillation process, however, is challenging due to the modal disparity between textual and speech embedding spaces. This paper studies metric-based distillation to align the embedding space of text and speech with only a small amount of data without modifying the model structure. Since the semantic and granularity gap between text and speech has been omitted in literature, which impairs the distillation, we propose the Prior-informed Adaptive knowledge Distillation (PAD) that adaptively leverages text/speech units of variable granularity and prior distributions to achieve better global and local alignments between text and speech pre-trained models. We evaluate on three spoken language understanding benchmarks to show that PAD is more effective in transferring linguistic knowledge than other metric-based distillation approaches.

cs.CL↗
arXiv2023

Generating a Terrain-Robustness Benchmark for Legged Locomotion: A Prototype via Terrain Authoring and Active Learning

Terrain-aware locomotion has become an emerging topic in legged robotics. However, it is hard to generate diverse, challenging, and realistic unstructured terrains in simulation, which limits the way researchers evaluate their locomotion policies. In this paper, we prototype the generation of a terrain dataset via terrain authoring and active learning, and the learned samplers can stably generate diverse high-quality terrains. We expect the generated dataset to make a terrain-robustness benchmark for legged locomotion. The dataset, the code implementation, and some policy evaluations are released at https://bit.ly/3bn4j7f.

cs.RO↗
arXiv2023

deHuBERT: Disentangling Noise in a Self-supervised Model for Robust Speech Recognition

Existing self-supervised pre-trained speech models have offered an effective way to leverage massive unannotated corpora to build good automatic speech recognition (ASR). However, many current models are trained on a clean corpus from a single source, which tends to do poorly when noise is present during testing. Nonetheless, it is crucial to overcome the adverse influence of noise for real-world applications. In this work, we propose a novel training framework, called deHuBERT, for noise reduction encoding inspired by H. Barlow's redundancy-reduction principle. The new framework improves the HuBERT training algorithm by introducing auxiliary losses that drive the self- and cross-correlation matrix between pairwise noise-distorted embeddings towards identity matrix. This encourages the model to produce noise-agnostic speech representations. With this method, we report improved robustness in noisy environments, including unseen noises, without impairing the performance on the clean set.

cs.SD↗