arXiv subjects

Cheng Chen

Publications and source records attributed to Cheng Chen.

At least 325 records · Page 18Linked to original sources

arXiv2021

GW Ori: circumtriple rings and planets

GW Ori is a hierarchical triple star system with a misaligned circumtriple protoplanetary disc. Recent ALMA observations have identified three dust rings with a prominent gap at $100\, \rm au$ and misalignments between each of the rings. A break in the gas disc may be driven either by the torque from the triple star system or a planet that is massive enough to carve a gap in the disc. Once the disc is broken, the rings nodally precess on different timescales and become misaligned. We investigate the origins of the dust rings by means of $N$-body integrations and 3-dimensional hydrodynamic simulations. We find that for observationally-motivated parameters of protoplanetary discs, the disc does not break due to the torque from the star system. We suggest that the presence of a massive planet (or planets) in the disc separates the inner and outer disc. We conclude that the disc breaking in GW Ori is likely caused by undetected planets -- the first planet(s) in a circumtriple orbit.

astro-ph.EP↗

arXiv2021

Source-Free Domain Adaptive Fundus Image Segmentation with Denoised Pseudo-Labeling

Domain adaptation typically requires to access source domain data to utilize their distribution information for domain alignment with the target data. However, in many real-world scenarios, the source data may not be accessible during the model adaptation in the target domain due to privacy issue. This paper studies the practical yet challenging source-free unsupervised domain adaptation problem, in which only an existing source model and the unlabeled target data are available for model adaptation. We present a novel denoised pseudo-labeling method for this problem, which effectively makes use of the source model and unlabeled target data to promote model self-adaptation from pseudo labels. Importantly, considering that the pseudo labels generated from source model are inevitably noisy due to domain shift, we further introduce two complementary pixel-level and class-level denoising schemes with uncertainty estimation and prototype estimation to reduce noisy pseudo labels and select reliable ones to enhance the pseudo-labeling efficacy. Experimental results on cross-domain fundus image segmentation show that without using any source images or altering source training, our approach achieves comparable or even higher performance than state-of-the-art source-dependent unsupervised domain adaptation methods.

eess.IV↗

arXiv2021

AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language Models

Pre-trained language models (PLMs) have achieved great success in natural language processing. Most of PLMs follow the default setting of architecture hyper-parameters (e.g., the hidden dimension is a quarter of the intermediate dimension in feed-forward sub-networks) in BERT (Devlin et al., 2019). Few studies have been conducted to explore the design of architecture hyper-parameters in BERT, especially for the more efficient PLMs with tiny sizes, which are essential for practical deployment on resource-constrained devices. In this paper, we adopt the one-shot Neural Architecture Search (NAS) to automatically search architecture hyper-parameters. Specifically, we carefully design the techniques of one-shot learning and the search space to provide an adaptive and efficient development way of tiny PLMs for various latency constraints. We name our method AutoTinyBERT and evaluate its effectiveness on the GLUE and SQuAD benchmarks. The extensive experiments show that our method outperforms both the SOTA search-based baseline (NAS-BERT) and the SOTA distillation-based methods (such as DistilBERT, TinyBERT, MiniLM and MobileBERT). In addition, based on the obtained architectures, we propose a more efficient development method that is even faster than the development of a single PLM.

cs.CL↗

arXiv2021

Learning to Bridge Metric Spaces: Few-shot Joint Learning of Intent Detection and Slot Filling

In this paper, we investigate few-shot joint learning for dialogue language understanding. Most existing few-shot models learn a single task each time with only a few examples. However, dialogue language understanding contains two closely related tasks, i.e., intent detection and slot filling, and often benefits from jointly learning the two tasks. This calls for new few-shot learning techniques that are able to capture task relations from only a few examples and jointly learn multiple tasks. To achieve this, we propose a similarity-based few-shot learning scheme, named Contrastive Prototype Merging network (ConProm), that learns to bridge metric spaces of intent and slot on data-rich domains, and then adapt the bridged metric space to the specific few-shot domain. Experiments on two public datasets, Snips and FewJoint, show that our model significantly outperforms the strong baselines in one and five shots settings.

cs.CL↗

arXiv2021

Nonlinear Hall Effect in Antiferromagnetic Half-Heusler Materials

It has recently been demonstrated that various topological states, including Dirac, Weyl, nodal-line, and triple-point semimetal phases, can emerge in antiferromagnetic (AFM) half-Heusler compounds. However, how to determine the AFM structure and to distinguish different topological phases from transport behaviors remains unknown. We show that, due to the presence of combined time-reversal and fractional translation symmetry, the recently proposed second-order nonlinear Hall effect can be used to characterize different topological phases with various AFM configurations. Guided by the symmetry analysis, we obtain the expressions of the Berry curvature dipole for different AFM configurations. Based on the effective model, we explicitly calculate the Berry curvature dipole, which is found to be vanishingly small for the triple-point semimetal phase, and large in the Weyl semimetal phase. Our results not only put forward an effective method for the identification of magnetic orders and topological phases in AFM half-Heusler materials, but also suggest these materials as a versatile platform for engineering the non-linear Hall effect.

cond-mat.mes-hall↗

arXiv2021

Extract then Distill: Efficient and Effective Task-Agnostic BERT Distillation

Task-agnostic knowledge distillation, a teacher-student framework, has been proved effective for BERT compression. Although achieving promising results on NLP tasks, it requires enormous computational resources. In this paper, we propose Extract Then Distill (ETD), a generic and flexible strategy to reuse the teacher's parameters for efficient and effective task-agnostic distillation, which can be applied to students of any size. Specifically, we introduce two variants of ETD, ETD-Rand and ETD-Impt, which extract the teacher's parameters in a random manner and by following an importance metric respectively. In this way, the student has already acquired some knowledge at the beginning of the distillation process, which makes the distillation process converge faster. We demonstrate the effectiveness of ETD on the GLUE benchmark and SQuAD. The experimental results show that: (1) compared with the baseline without an ETD strategy, ETD can save 70\% of computation cost. Moreover, it achieves better results than the baseline when using the same computing resource. (2) ETD is generic and has been proven effective for different distillation methods (e.g., TinyBERT and MiniLM) and students of different sizes. The source code will be publicly available upon publication.

cs.CL↗

arXiv2021

Two-Color Optical Nonlinearity in an Ultracold Rydberg Atom Gas Mixture

We report the experimental observation of strong two-color optical nonlinearity in an ultracold gas of $^{85}\mathrm{Rb}$-$^{87}\mathrm{Rb}$ atom mixture. By simultaneously coupling two probe transitions of $^{85}$Rb and $^{87}$Rb atoms to Rydberg states in electromagnetically induced transparency (EIT) configurations, we observe significant suppression of the transparency resonance for one probe field when the second probe field is detuned at $\sim1~\mathrm{GHz}$ and hitting the EIT resonance of the other isotope. Such a cross-absorption modulation to the beam propagation dynamics can be described by two coupled nonlinear wave equations we develope. We further demonstrate that the two-color optical nonlinearity can be tuned by varying the density ratio of different atomic isotopes, which highlights its potential for exploring strongly interacting multi-component fluids of light.

physics.atom-ph↗

arXiv2021

Temporal Memory Relation Network for Workflow Recognition from Surgical Video

Automatic surgical workflow recognition is a key component for developing context-aware computer-assisted systems in the operating theatre. Previous works either jointly modeled the spatial features with short fixed-range temporal information, or separately learned visual and long temporal cues. In this paper, we propose a novel end-to-end temporal memory relation network (TMRNet) for relating long-range and multi-scale temporal patterns to augment the present features. We establish a long-range memory bank to serve as a memory cell storing the rich supportive information. Through our designed temporal variation layer, the supportive cues are further enhanced by multi-scale temporal-only convolutions. To effectively incorporate the two types of cues without disturbing the joint learning of spatio-temporal features, we introduce a non-local bank operator to attentively relate the past to the present. In this regard, our TMRNet enables the current feature to view the long-range temporal dependency, as well as tolerate complex temporal extents. We have extensively validated our approach on two benchmark surgical video datasets, M2CAI challenge dataset and Cholec80 dataset. Experimental results demonstrate the outstanding performance of our method, consistently exceeding the state-of-the-art methods by a large margin (e.g., 67.0% v.s. 78.9% Jaccard on Cholec80 dataset).

cs.CV↗

arXiv2021

Certifiably-Robust Federated Adversarial Learning via Randomized Smoothing

Federated learning is an emerging data-private distributed learning framework, which, however, is vulnerable to adversarial attacks. Although several heuristic defenses are proposed to enhance the robustness of federated learning, they do not provide certifiable robustness guarantees. In this paper, we incorporate randomized smoothing techniques into federated adversarial training to enable data-private distributed learning with certifiable robustness to test-time adversarial perturbations. Our experiments show that such an advanced federated adversarial learning framework can deliver models as robust as those trained by the centralized training. Further, this enables provably-robust classifiers to $\ell_2$-bounded adversarial perturbations in a distributed setup. We also show that one-point gradient estimation based training approach is $2-3\times$ faster than popular stochastic estimator based approach without any noticeable certified robustness differences.

cs.LG↗

arXiv2021

FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space

Federated learning allows distributed medical institutions to collaboratively learn a shared prediction model with privacy protection. While at clinical deployment, the models trained in federated learning can still suffer from performance drop when applied to completely unseen hospitals outside the federation. In this paper, we point out and solve a novel problem setting of federated domain generalization (FedDG), which aims to learn a federated model from multiple distributed source domains such that it can directly generalize to unseen target domains. We present a novel approach, named as Episodic Learning in Continuous Frequency Space (ELCFS), for this problem by enabling each client to exploit multi-source data distributions under the challenging constraint of data decentralization. Our approach transmits the distribution information across clients in a privacy-protecting way through an effective continuous frequency space interpolation mechanism. With the transferred multi-source distributions, we further carefully design a boundary-oriented episodic learning paradigm to expose the local learning to domain distribution shifts and particularly meet the challenges of model generalization in medical image segmentation scenario. The effectiveness of our method is demonstrated with superior performance over state-of-the-arts and in-depth ablation experiments on two medical image segmentation tasks. The code is available at "https://github.com/liuquande/FedDG-ELCFS".

cs.CV↗

arXiv2021

Computation Resource Allocation Solution in Recommender Systems

Recommender systems rely heavily on increasing computation resources to improve their business goal. By deploying computation-intensive models and algorithms, these systems are able to inference user interests and exhibit certain ads or commodities from the candidate set to maximize their business goals. However, such systems are facing two challenges in achieving their goals. On the one hand, facing massive online requests, computation-intensive models and algorithms are pushing their computation resources to the limit. On the other hand, the response time of these systems is strictly limited to a short period, e.g. 300 milliseconds in our real system, which is also being exhausted by the increasingly complex models and algorithms. In this paper, we propose the computation resource allocation solution (CRAS) that maximizes the business goal with limited computation resources and response time. We comprehensively illustrate the problem and formulate such a problem as an optimization problem with multiple constraints, which could be broken down into independent sub-problems. To solve the sub-problems, we propose the revenue function to facilitate the theoretical analysis, and obtain the optimal computation resource allocation strategy. To address the applicability issues, we devise the feedback control system to help our strategy constantly adapt to the changing online environment. The effectiveness of our method is verified by extensive experiments based on the real dataset from Taobao.com. We also deploy our method in the display advertising system of Alibaba. The online results show that our computation resource allocation solution achieves significant business goal improvement without any increment of computation cost, which demonstrates the efficacy of our method in real industrial practice.

eess.SY↗

arXiv2021

MITNet: GAN Enhanced Magnetic Induction Tomography Based on Complex CNN

Magnetic induction tomography (MIT) is an efficient solution for long-term brain disease monitoring, which focuses on reconstructing bio-impedance distribution inside the human brain using non-intrusive electromagnetic fields. However, high-quality brain image reconstruction remains challenging since reconstructing images from the measured weak signals is a highly non-linear and ill-conditioned problem. In this work, we propose a generative adversarial network (GAN) enhanced MIT technique, named MITNet, based on a complex convolutional neural network (CNN). The experimental results on the real-world dataset validate the performance of our technique, which outperforms the state-of-art method by 25.27%.

cs.CV↗

arXiv2021

A Technical Overview of AV1

The AV1 video compression format is developed by the Alliance for Open Media consortium. It achieves more than 30% reduction in bit-rate compared to its predecessor VP9 for the same decoded video quality. This paper provides a technical overview of the AV1 codec design that enables the compression performance gains with considerations for hardware feasibility.

eess.IV↗

arXiv2021

A concise review of Rydberg atom based quantum computation and quantum simulation

Quantum information processing based on Rydberg atoms emerged as a promising direction two decades ago. Recent experimental and theoretical progresses have shined exciting light on this avenue. In this concise review, we will briefly introduce the basics of Rydberg atoms and their recent applications in associated areas of neutral atom quantum computation and simulation. We shall also include related discussions on quantum optics with Rydberg atomic ensembles, which are increasingly used to explore quantum computation and quantum simulation with photons.

quant-ph↗

arXiv2020

2D-Material-Assisted Bistable Switching of Gap Plasmons Disclosed By Femtosecond Pulse Scattering Spectra

Nanosphere-on-mirror plasmonic antennas, each having a monolayer graphene or MoS2 sheet in the gap, were pumped with a femtosecond laser. Abrupt turnings in the scattering linewidth and peak intensity trends as the laser power changed were experimentally observed. Theoretical modelling of dynamic plasmon evolvement attributes the turning to transitioning between two plasmon states, with a universal switching threshold of four-wave mixing efficiency around 0.14%. This bistability is rendered by a strong feedback from the nonlinear gap current to the gap plasmons, and involvement of both four and high-order wave mixing. This work reveals a pathway to making energy efficient nonlinear plasmonic elements.

physics.optics↗

arXiv2020

Revisiting Co-Occurring Directions: Sharper Analysis and Efficient Algorithm for Sparse Matrices

We study the streaming model for approximate matrix multiplication (AMM). We are interested in the scenario that the algorithm can only take one pass over the data with limited memory. The state-of-the-art deterministic sketching algorithm for streaming AMM is the co-occurring directions (COD), which has much smaller approximation errors than randomized algorithms and outperforms other deterministic sketching methods empirically. In this paper, we provide a tighter error bound for COD whose leading term considers the potential approximate low-rank structure and the correlation of input matrices. We prove COD is space optimal with respect to our improved error bound. We also propose a variant of COD for sparse matrices with theoretical guarantees. The experiments on real-world sparse datasets show that the proposed algorithm is more efficient than baseline methods.

cs.LG↗

arXiv2020

FewJoint: A Few-shot Learning Benchmark for Joint Language Understanding

Few-shot learning (FSL) is one of the key future steps in machine learning and has raised a lot of attention. However, in contrast to the rapid development in other domains, such as Computer Vision, the progress of FSL in Nature Language Processing (NLP) is much slower. One of the key reasons for this is the lacking of public benchmarks. NLP FSL researches always report new results on their own constructed few-shot datasets, which is pretty inefficient in results comparison and thus impedes cumulative progress. In this paper, we present FewJoint, a novel Few-Shot Learning benchmark for NLP. Different from most NLP FSL research that only focus on simple N-classification problems, our benchmark introduces few-shot joint dialogue language understanding, which additionally covers the structure prediction and multi-task reliance problems. This allows our benchmark to reflect the real-word NLP complexity beyond simple N-classification. Our benchmark is used in the few-shot learning contest of SMP2020-ECDT task-1. We also provide a compatible FSL platform to ease experiment set-up.

cs.CL↗

arXiv2020

C2C-GenDA: Cluster-to-Cluster Generation for Data Augmentation of Slot Filling

Slot filling, a fundamental module of spoken language understanding, often suffers from insufficient quantity and diversity of training data. To remedy this, we propose a novel Cluster-to-Cluster generation framework for Data Augmentation (DA), named C2C-GenDA. It enlarges the training set by reconstructing existing utterances into alternative expressions while keeping semantic. Different from previous DA works that reconstruct utterances one by one independently, C2C-GenDA jointly encodes multiple existing utterances of the same semantics and simultaneously decodes multiple unseen expressions. Jointly generating multiple new utterances allows to consider the relations between generated instances and encourages diversity. Besides, encoding multiple existing utterances endows C2C with a wider view of existing expressions, helping to reduce generation that duplicates existing data. Experiments on ATIS and Snips datasets show that instances augmented by C2C-GenDA improve slot filling by 7.99 (11.9%) and 5.76 (13.6%) F-scores respectively, when there are only hundreds of training utterances.

cs.CL↗