arXiv subjects

Yong Cheng

Publications and source records attributed to Yong Cheng.

64 records · Page 4Linked to original sources

arXiv2016

Minimum Risk Training for Neural Machine Translation

We propose minimum risk training for end-to-end neural machine translation. Unlike conventional maximum likelihood estimation, minimum risk training is capable of optimizing model parameters directly with respect to arbitrary evaluation metrics, which are not necessarily differentiable. Experiments show that our approach achieves significant improvements over maximum likelihood estimation on a state-of-the-art neural machine translation system across various languages pairs. Transparent to architectures, our approach can be applied to more neural networks and potentially benefit more NLP tasks.

cs.CL↗

arXiv2016

Agreement-based Joint Training for Bidirectional Attention-based Neural Machine Translation

The attentional mechanism has proven to be effective in improving end-to-end neural machine translation. However, due to the intricate structural divergence between natural languages, unidirectional attention-based models might only capture partial aspects of attentional regularities. We propose agreement-based joint training for bidirectional attention-based end-to-end neural machine translation. Instead of training source-to-target and target-to-source translation models independently,our approach encourages the two complementary models to agree on word alignment matrices on the same training data. Experiments on Chinese-English and English-French translation tasks show that agreement-based joint training significantly improves both alignment and translation quality over independent training.

cs.CL↗

arXiv2015

Large cardinals need not be large in HOD

We prove that large cardinals need not generally exhibit their large cardinal nature in HOD. For example, a supercompact cardinal $κ$ need not be weakly compact in HOD, and there can be a proper class of supercompact cardinals in $V$, none of them weakly compact in HOD, with no supercompact cardinals in HOD. Similar results hold for many other types of large cardinals, such as measurable and strong cardinals.

math.LO↗

arXiv2015

Indestructibility properties of remarkable cardinals

Remarkable cardinals were introduced by Schindler, who showed that the existence of a remarkable cardinal is equiconsistent with the assertion that the theory of $L(\mathbb R)$ is absolute for proper forcing. Here, we study the indestructibility properties of remarkable cardinals. We show that if $κ$ is remarkable, then there is a forcing extension in which the remarkability of $κ$ becomes indestructible by all $\ltκ$-closed $\leqκ$-distributive forcing and all two-step iterations of the form ${\rm Add}(κ,θ)*\dot{\mathbb R}$, where $\dot{\mathbb R}$ is forced to be $\ltκ$-closed and $\leqκ$-distributive. In the process, we introduce the notion of a remarkable Laver function and show that every remarkable cardinal carries such a function. We also show that remarkability is preserved by the canonical forcing of the ${\rm GCH}$.

math.LO↗

arXiv2015

The strong reflecting property and Harrington's Principle

In this paper we characterize the strong reflecting property for $L$-cardinals for all $ω_n$, characterize Harrington's Principle $HP(L)$ and its generalization and discuss the relationship between the strong reflecting property for $L$-cardinals and Harrington's Principle $HP(L)$.

math.LO↗

arXiv2015

Harrington's principle over higher order arithmetic

Let $Z_2$, $Z_3$, and $Z_4$ denote $2^{\rm nd}$, $3^{\rm rd}$, and $4^{\rm th}$ order arithmetic, respectively. We let Harrington's Principle, {\sf HP}, denote the statement that there is a real $x$ such that every $x$--admissible ordinal is a cardinal in $L$. The known proofs of Harrington's theorem "$Det(Σ_1^1)$ implies $0^{\sharp}$ exists" are done in two steps: first show that $Det(Σ_1^1)$ implies {\sf HP}, and then show that {\sf HP} implies $0^{\sharp}$ exists. The first step is provable in $Z_2$. In this paper we show that $Z_2 \, + \, {\sf HP}$ is equiconsistent with ${\sf ZFC}$ and that $Z_3\, + \, {\sf HP}$ is equiconsistent with ${\sf ZFC} \, +$ there exists a remarkable cardinal. As a corollary, $Z_3\, + \, {\sf HP}$ does not imply $0^{\sharp}$ exists, whereas $Z_4\, + \, {\sf HP}$ does. We also study strengthenings of Harrington's Principle over $2^{\rm nd}$ and $3^{\rm rd}$ order arithmetic.

math.LO↗

arXiv2014

Force a set model of $Z_3$ + Harrington's Principle

Let $Z_3$ denote $3^{rd}$ order arithmetic. Let Harrington's Principle, HP, denote the statement that there is a real $x$ such that every $x$--admissible ordinal is a cardinal in $L$. In this paper, assuming there exists a remarkable cardinal with a weakly inaccessible cardinal above it, we force a set model of $Z_3\, + \, {\sf HP}$ via set forcing without reshaping.

math.LO↗

arXiv2011

Distributive Network Utility Maximization (NUM) over Time-Varying Fading Channels

Distributed network utility maximization (NUM) has received an increasing intensity of interest over the past few years. Distributed solutions (e.g., the primal-dual gradient method) have been intensively investigated under fading channels. As such distributed solutions involve iterative updating and explicit message passing, it is unrealistic to assume that the wireless channel remains unchanged during the iterations. Unfortunately, the behavior of those distributed solutions under time-varying channels is in general unknown. In this paper, we shall investigate the convergence behavior and tracking errors of the iterative primal-dual scaled gradient algorithm (PDSGA) with dynamic scaling matrices (DSC) for solving distributive NUM problems under time-varying fading channels. We shall also study a specific application example, namely the multi-commodity flow control and multi-carrier power allocation problem in multi-hop ad hoc networks. Our analysis shows that the PDSGA converges to a limit region rather than a single point under the finite state Markov chain (FSMC) fading channels. We also show that the order of growth of the tracking errors is given by O(T/N), where T and N are the update interval and the average sojourn time of the FSMC, respectively. Based on this analysis, we derive a low complexity distributive adaptation algorithm for determining the adaptive scaling matrices, which can be implemented distributively at each transmitter. The numerical results show the superior performance of the proposed dynamic scaling matrix algorithm over several baseline schemes, such as the regular primal-dual gradient algorithm.

cs.IT↗

arXiv2010

A Scalable Limited Feedback Design for Network MIMO using Per-Cell Product Codebook

In network MIMO systems, channel state information is required at the transmitter side to multiplex users in the spatial domain. Since perfect channel knowledge is difficult to obtain in practice, \emph{limited feedback} is a widely accepted solution. The {\em dynamic number of cooperating BSs} and {\em heterogeneous path loss effects} of network MIMO systems pose new challenges on limited feedback design. In this paper, we propose a scalable limited feedback design for network MIMO systems with multiple base stations, multiple users and multiple data streams for each user. We propose a {\em limited feedback framework using per-cell product codebooks}, along with a {\em low-complexity feedback indices selection algorithm}. We show that the proposed per-cell product codebook limited feedback design can asymptotically achieve the same performance as the joint-cell codebook approach. We also derive an asymptotic \emph{per-user throughput loss} due to limited feedback with per-cell product codebooks. Based on that, we show that when the number of per-user feedback-bits $B_{k}$ is $\mathcal{O}\big( Nn_{T}n_{R}\log_{2}(ρg_{k}^{sum})\big)$, the system operates in the \emph{noise-limited} regime in which the per-user throughput is $\mathcal{O} \left( n_{R} \log_{2} \big( \frac{n_{R}ρg_{k}^{sum}}{Nn_{T}} \big) \right)$. On the other hand, when the number of per-user feedback-bits $B_{k}$ does not scale with the \emph{system SNR} $ρ$, the system operates in the \emph{interference-limited} regime where the per-user throughput is $\mathcal{O}\left( \frac{n_{R}B_{k}}{(Nn_{T})^{2}} \right)$. Numerical results show that the proposed design is very flexible to accommodate dynamic number of cooperating BSs and achieves much better performance compared with other baselines (such as the Givens rotation approach).

cs.IT↗

arXiv2010

Distributive Power Control Algorithm for Multicarrier Interference Network over Time-Varying Fading Channels - Tracking Performance Analysis and Optimization

Distributed power control over interference limited network has received an increasing intensity of interest over the past few years. Distributed solutions (like the iterative water-filling, gradient projection, etc.) have been intensively investigated under \emph{quasi-static} channels. However, as such distributed solutions involve iterative updating and explicit message passing, it is unrealistic to assume that the wireless channel remains unchanged during the iterations. Unfortunately, the behavior of those distributed solutions under \emph{time-varying} channels is in general unknown. In this paper, we shall investigate the distributed scaled gradient projection algorithm (DSGPA) in a $K$ pairs multicarrier interference network under a finite-state Markov channel (FSMC) model. We shall analyze the \emph{convergence property} as well as \emph{tracking performance} of the proposed DSGPA. Our analysis shows that the proposed DSGPA converges to a limit region rather than a single point under the FSMC model. We also show that the order of growth of the tracking errors is given by $\mathcal{O}$1 \big/ \bar{N}$$, where $\bar{N}$ is the \emph{average sojourn time} of the FSMC. Based on the analysis, we shall derive the \emph{tracking error optimal scaling matrices} via Markov decision process modeling. We shall show that the tracking error optimal scaling matrices can be implemented distributively at each transmitter. The numerical results show the superior performance of the proposed DSGPA over three baseline schemes, such as the gradient projection algorithm with a constant stepsize.

cs.IT↗