arXiv subjects

Yong Cheng

Publications and source records attributed to Yong Cheng.

At least 55 records · Page 3Linked to original sources

arXiv2021

Micro/nanoliter droplet extraction by controlling acoustic vortex with miniwatt

Micro/nanoliter droplet is capable of achieving versatile applications with tiny volume and substantial surface energy, which is a big plus over bulk liquid. Yet, the contradiction of elaborate manipulation and enough power is still a challenge. Here, we unleash the potential of our miniwatt aspirators pumping up liquid and creating droplets with the help of acoustic vortex beams, inspired by the power mechanism that spirals are significant for most mollusks that live in water. These droplet aspirators produce very large interface deformations by small radiation pressures with orbit angular momentum from spiral-electrode transducers. The precisely contactless manipulation of physical, chemical and biological objects at micrometric down to nanometric scales, promises tremendous development in fields as diverse as microrobotics, nanoreactors, or nanoassemblies.

physics.app-ph↗

arXiv2021

Self-supervised and Supervised Joint Training for Resource-rich Machine Translation

Self-supervised pre-training of text representations has been successfully applied to low-resource Neural Machine Translation (NMT). However, it usually fails to achieve notable gains on resource-rich NMT. In this paper, we propose a joint training approach, $F_2$-XEnDec, to combine self-supervised and supervised learning to optimize NMT models. To exploit complementary self-supervised signals for supervised learning, NMT models are trained on examples that are interbred from monolingual and parallel sentences through a new process called crossover encoder-decoder. Experiments on two resource-rich translation benchmarks, WMT'14 English-German and WMT'14 English-French, demonstrate that our approach achieves substantial improvements over several strong baseline methods and obtains a new state of the art of 46.19 BLEU on English-French when incorporating back translation. Results also show that our approach is capable of improving model robustness to input perturbations such as code-switching noise which frequently appears on social media.

cs.CL↗

arXiv2021

On the depth of Gödel's incompleteness theorem

In this paper, we use Gödel's incompleteness theorem as a case study for investigating mathematical depth. We take for granted the widespread judgment by mathematical logicians that Gödel's incompleteness theorem is deep, and focus on the philosophical question of what its depth consists in. We focus on the methodological study of the depth of Gödel's incompleteness theorem, and propose three criteria to account for its depth: influence, fruitfulness, and unity. Finally, we give some explanations for our account of the depth of Gödel's incompleteness theorem.

math.LO↗

arXiv2020

Current research on Gödel's incompleteness theorems

We give a survey of current research on Gödel's incompleteness theorems from the following three aspects: classifications of different proofs of Gödel's incompleteness theorems, the limit of the applicability of Gödel's first incompleteness theorem, and the limit of the applicability of Gödel's second incompleteness theorem.

math.LO↗

arXiv2020

Gödel's incompleteness theorem and the Anti-Mechanist Argument: revisited

This is a paper for a special issue of the journal "Studia Semiotyczne" devoted to Stanislaw Krajewski's paper [30]. This paper gives some supplementary notes to Krajewski's [30] on the Anti-Mechanist Arguments based on Gödel's incompleteness theorem. In Section 3, we give some additional explanations to Section 4-6 in Krajewski's [30] and classify some misunderstandings of Gödel's incompleteness theorem related to Anti-Mechanist Arguments. In Section 4 and 5, we give a more detailed discussion of Gödel's Disjunctive Thesis, Gödel's Undemonstrability of Consistency Thesis and the definability of natural numbers as in Section 7-8 in Krajewski's [30], describing how recent advances bear on these issues.

math.LO↗

arXiv2020

A Communication Efficient Collaborative Learning Framework for Distributed Features

We introduce a collaborative learning framework allowing multiple parties having different sets of attributes about the same user to jointly build models without exposing their raw data or model parameters. In particular, we propose a Federated Stochastic Block Coordinate Descent (FedBCD) algorithm, in which each party conducts multiple local updates before each communication to effectively reduce the number of communication rounds among parties, a principal bottleneck for collaborative learning problems. We analyze theoretically the impact of the number of local updates and show that when the batch size, sample size, and the local iterations are selected appropriately, within $T$ iterations, the algorithm performs $\mathcal{O}(\sqrt{T})$ communication rounds and achieves some $\mathcal{O}(1/\sqrt{T})$ accuracy (measured by the average of the gradient norm squared). The approach is supported by our empirical evaluations on a variety of tasks and datasets, demonstrating advantages over stochastic gradient descent (SGD) approaches.

cs.LG↗

arXiv2020

AdvAug: Robust Adversarial Augmentation for Neural Machine Translation

In this paper, we propose a new adversarial augmentation method for Neural Machine Translation (NMT). The main idea is to minimize the vicinal risk over virtual sentences sampled from two vicinity distributions, of which the crucial one is a novel vicinity distribution for adversarial sentences that describes a smooth interpolated embedding space centered around observed training sentence pairs. We then discuss our approach, AdvAug, to train NMT models using the embeddings of virtual sentences in sequence-to-sequence learning. Experiments on Chinese-English, English-French, and English-German translation benchmarks show that AdvAug achieves significant improvements over the Transformer (up to 4.9 BLEU points), and substantially outperforms other data augmentation techniques (e.g. back-translation) without using extra corpora.

cs.CL↗

arXiv2020

Finding the limit of incompleteness I

In this paper, we examine the limit of applicability of Gödel's first incompleteness theorem ($\sf G1$ for short). We first define the notion "$\sf G1$ holds for the theory $T$". This paper is motivated by the following question: can we find a theory with a minimal degree of interpretation for which $\sf G1$ holds. To approach this question, we first examine the following question: is there a theory $T$ such that Robinson's $\mathbf{R}$ interprets $T$ but $T$ does not interpret $\mathbf{R}$ (i.e. $T$ is weaker than $\mathbf{R}$ w.r.t. interpretation) and $\sf G1$ holds for $T$? In this paper, we show that there are many such theories based on Jeřábek's work using some model theory. We prove that for each recursively inseparable pair $\langle A,B\rangle$, we can construct a r.e. theory $U_{\langle A,B\rangle}$ such that $U_{\langle A,B\rangle}$ is weaker than $\mathbf{R}$ w.r.t. interpretation and $\sf G1$ holds for $U_{\langle A,B\rangle}$. As a corollary, we answer a question from Albert Visser. Moreover, we prove that for any Turing degree $\mathbf{0}< \mathbf{d}<\mathbf{0}^{\prime}$, there is a theory $T$ with Turing degree $\mathbf{d}$ such that $\sf G1$ holds for $T$ and $T$ is weaker than $\mathbf{R}$ w.r.t. Turing reducibility. As a corollary, based on Shoenfield's work using some recursion theory, we show that there is no theory with a minimal degree of Turing reducibility for which $\sf G1$ holds.

math.LO↗

arXiv2020

Learning to Detect Malicious Clients for Robust Federated Learning

Federated learning systems are vulnerable to attacks from malicious clients. As the central server in the system cannot govern the behaviors of the clients, a rogue client may initiate an attack by sending malicious model updates to the server, so as to degrade the learning performance or enforce targeted model poisoning attacks (a.k.a. backdoor attacks). Therefore, timely detecting these malicious model updates and the underlying attackers becomes critically important. In this work, we propose a new framework for robust federated learning where the central server learns to detect and remove the malicious model updates using a powerful detection model, leading to targeted defense. We evaluate our solution in both image classification and sentiment analysis tasks with a variety of machine learning models. Experimental results show that our solution ensures robust federated learning that is resilient to both the Byzantine attacks and the targeted model poisoning attacks.

cs.LG↗

arXiv2019

Abnormal Client Behavior Detection in Federated Learning

In federated learning systems, clients are autonomous in that their behaviors are not fully governed by the server. Consequently, a client may intentionally or unintentionally deviate from the prescribed course of federated model training, resulting in abnormal behaviors, such as turning into a malicious attacker or a malfunctioning client. Timely detecting those anomalous clients is therefore critical to minimize their adverse impacts. In this work, we propose to detect anomalous clients at the server side. In particular, we generate low-dimensional surrogates of model weight vectors and use them to perform anomaly detection. We evaluate our solution through experiments on image classification model training over the FEMNIST dataset. Experimental results show that the proposed detection-based approach significantly outperforms the conventional defense-based methods.

cs.LG↗

arXiv2019

Robust Neural Machine Translation with Doubly Adversarial Inputs

Neural machine translation (NMT) often suffers from the vulnerability to noisy perturbations in the input. We propose an approach to improving the robustness of NMT models, which consists of two parts: (1) attack the translation model with adversarial source examples; (2) defend the translation model with adversarial target inputs to improve its robustness against the adversarial source inputs.For the generation of adversarial inputs, we propose a gradient-based method to craft adversarial examples informed by the translation loss over the clean inputs.Experimental results on Chinese-English and English-German translation tasks demonstrate that our approach achieves significant improvements ($2.8$ and $1.6$ BLEU points) over Transformer on standard clean benchmarks as well as exhibiting higher robustness on noisy data.

cs.CL↗

arXiv2018

Neural Machine Translation with Key-Value Memory-Augmented Attention

Although attention-based Neural Machine Translation (NMT) has achieved remarkable progress in recent years, it still suffers from issues of repeating and dropping translations. To alleviate these issues, we propose a novel key-value memory-augmented attention model for NMT, called KVMEMATT. Specifically, we maintain a timely updated keymemory to keep track of attention history and a fixed value-memory to store the representation of source sentence throughout the whole translation process. Via nontrivial transformations and iterative interactions between the two memories, the decoder focuses on more appropriate source word(s) for predicting the next target word at each decoding step, therefore can improve the adequacy of translations. Experimental results on Chinese=>English and WMT17 German<=>English translation tasks demonstrate the superiority of the proposed model.

cs.CL↗

arXiv2018

Towards Robust Neural Machine Translation

Small perturbations in the input can severely distort intermediate representations and thus impact translation quality of neural machine translation (NMT) models. In this paper, we propose to improve the robustness of NMT models with adversarial stability training. The basic idea is to make both the encoder and decoder in NMT models robust against input perturbations by enabling them to behave similarly for the original input and its perturbed counterpart. Experimental results on Chinese-English, English-German and English-French translation tasks show that our approaches can not only achieve significant improvements over strong NMT systems but also improve the robustness of NMT models.

cs.CL↗

arXiv2018

The $\mathsf{HOD}$ Hypothesis and a supercompact cardinal

In this paper, we prove that: if $κ$ is supercompact and the $\mathsf{HOD}$ Hypothesis holds, then there is a proper class of regular cardinals in $V_κ$ which are measurable in $\mathsf{HOD}$. Woodin also proved this result. As a corollary, we prove Woodin's Local Universality Theorem. This work shows that under the assumption of the $\mathsf{HOD}$ Hypothesis and supercompact cardinals, large cardinals in $\mathsf{V}$ are reflected to be large cardinals in $\mathsf{HOD}$ in a local way, and reveals the huge difference between $\mathsf{HOD}$-supercompact cardinals and supercompact cardinals under the $\mathsf{HOD}$ Hypothesis.

math.LO↗

arXiv2017

THUMT: An Open Source Toolkit for Neural Machine Translation

This paper introduces THUMT, an open-source toolkit for neural machine translation (NMT) developed by the Natural Language Processing Group at Tsinghua University. THUMT implements the standard attention-based encoder-decoder framework on top of Theano and supports three training criteria: maximum likelihood estimation, minimum risk training, and semi-supervised training. It features a visualization tool for displaying the relevance between hidden states in neural networks and contextual words, which helps to analyze the internal workings of NMT. Experiments on Chinese-English datasets show that THUMT using minimum risk training significantly outperforms GroundHog, a state-of-the-art toolkit for NMT.

cs.CL↗

arXiv2017

A Teacher-Student Framework for Zero-Resource Neural Machine Translation

While end-to-end neural machine translation (NMT) has made remarkable progress recently, it still suffers from the data scarcity problem for low-resource language pairs and domains. In this paper, we propose a method for zero-resource NMT by assuming that parallel sentences have close probabilities of generating a sentence in a third language. Based on this assumption, our method is able to train a source-to-target NMT model ("student") without parallel corpora available, guided by an existing pivot-to-target NMT model ("teacher") on a source-pivot parallel corpus. Experimental results show that the proposed method significantly improves over a baseline pivot-based model by +3.0 BLEU points across various language pairs.

cs.CL↗

arXiv2017

Neural Machine Translation with Pivot Languages

While recent neural machine translation approaches have delivered state-of-the-art performance for resource-rich language pairs, they suffer from the data scarcity problem for resource-scarce language pairs. Although this problem can be alleviated by exploiting a pivot language to bridge the source and target languages, the source-to-pivot and pivot-to-target translation models are usually independently trained. In this work, we introduce a joint training algorithm for pivot-based neural machine translation. We propose three methods to connect the two models and enable them to interact with each other during training. Experiments on Europarl and WMT corpora show that joint training of source-to-pivot and pivot-to-target models leads to significant improvements over independent training across various languages.

cs.CL↗

arXiv2016

Semi-Supervised Learning for Neural Machine Translation

While end-to-end neural machine translation (NMT) has made remarkable progress recently, NMT systems only rely on parallel corpora for parameter estimation. Since parallel corpora are usually limited in quantity, quality, and coverage, especially for low-resource languages, it is appealing to exploit monolingual corpora to improve NMT. We propose a semi-supervised approach for training NMT models on the concatenation of labeled (parallel corpora) and unlabeled (monolingual corpora) data. The central idea is to reconstruct the monolingual corpora using an autoencoder, in which the source-to-target and target-to-source translation models serve as the encoder and decoder, respectively. Our approach can not only exploit the monolingual corpora of the target language, but also of the source language. Experiments on the Chinese-English dataset show that our approach achieves significant improvements over state-of-the-art SMT and NMT systems.

cs.CL↗