Searcharxiv⌕ Search

arXiv subjects

Liang Ding

Publications and source records attributed to Liang Ding.

192 records · Page 11Linked to original sources
arXiv2020

Generalization Guarantees for Sparse Kernel Approximation with Entropic Optimal Features

Despite their success, kernel methods suffer from a massive computational cost in practice. In this paper, in lieu of commonly used kernel expansion with respect to $N$ inputs, we develop a novel optimal design maximizing the entropy among kernel features. This procedure results in a kernel expansion with respect to entropic optimal features (EOF), improving the data representation dramatically due to features dissimilarity. Under mild technical assumptions, our generalization bound shows that with only $O(N^{\frac{1}{4}})$ features (disregarding logarithmic factors), we can achieve the optimal statistical accuracy (i.e., $O(1/\sqrt{N})$). The salient feature of our design is its sparsity that significantly reduces the time and space cost. Our numerical experiments on benchmark datasets verify the superiority of EOF over the state-of-the-art in kernel approximation.

cs.LG↗
arXiv2019

BdryGP: a new Gaussian process model for incorporating boundary information

Gaussian processes (GPs) are widely used as surrogate models for emulating computer code, which simulate complex physical phenomena. In many problems, additional boundary information (i.e., the behavior of the phenomena along input boundaries) is known beforehand, either from governing physics or scientific knowledge. While there has been recent work on incorporating boundary information within GPs, such models do not provide theoretical insights on improved convergence rates. To this end, we propose a new GP model, called BdryGP, for incorporating boundary information. We show that BdryGP not only has improved convergence rates over existing GP models (which do not incorporate boundaries), but is also more resistant to the "curse-of-dimensionality" in nonparametric regression. Our proofs make use of a novel connection between GP interpolation and finite-element modeling.

stat.ME↗
arXiv2019

Recurrent Graph Syntax Encoder for Neural Machine Translation

Syntax-incorporated machine translation models have been proven successful in improving the model's reasoning and meaning preservation ability. In this paper, we propose a simple yet effective graph-structured encoder, the Recurrent Graph Syntax Encoder, dubbed \textbf{RGSE}, which enhances the ability to capture useful syntactic information. The RGSE is done over a standard encoder (recurrent or self-attention encoder), regarding recurrent network units as graph nodes and injects syntactic dependencies as edges, such that RGSE models syntactic dependencies and sequential information (\textit{i.e.}, word order) simultaneously. Our approach achieves considerable improvements over several syntax-aware NMT models in English$\Rightarrow$German and English$\Rightarrow$Czech translation tasks. And RGSE-equipped big model obtains competitive result compared with the state-of-the-art model in WMT14 En-De task. Extensive analysis further verifies that RGSE could benefit long sentence modeling, and produces better translations.

cs.CL↗
arXiv2019

The University of Sydney's Machine Translation System for WMT19

This paper describes the University of Sydney's submission of the WMT 2019 shared news translation task. We participated in the Finnish$\rightarrow$English direction and got the best BLEU(33.0) score among all the participants. Our system is based on the self-attentional Transformer networks, into which we integrated the most recent effective strategies from academic research (e.g., BPE, back translation, multi-features data selection, data augmentation, greedy model ensemble, reranking, ConMBR system combination, and post-processing). Furthermore, we propose a novel augmentation method $Cycle Translation$ and a data mixture strategy $Big$/$Small$ parallel construction to entirely exploit the synthetic corpus. Extensive experiments show that adding the above techniques can make continuous improvements of the BLEU scores, and the best result outperforms the baseline (Transformer ensemble model trained with the original parallel corpus) by approximately 5.3 BLEU score, achieving the state-of-the-art performance.

cs.CL↗
arXiv2018

Scalable Stochastic Kriging with Markovian Covariances

Stochastic kriging is a popular technique for simulation metamodeling due to its exibility and analytical tractability. Its computational bottleneck is the inversion of a covariance matrix, which takes $O(n^3)$ time in general and becomes prohibitive for large n, where n is the number of design points. Moreover, the covariance matrix is often ill-conditioned for large n, and thus the inversion is prone to numerical instability, resulting in erroneous parameter estimation and prediction. These two numerical issues preclude the use of stochastic kriging at a large scale. This paper presents a novel approach to address them. We construct a class of covariance functions, called Markovian covariance functions (MCFs), which have two properties: (i) the associated covariance matrices can be inverted analytically, and (ii) the inverse matrices are sparse. With the use of MCFs, the inversion-related computational time is reduced to $O(n^2)$ in general, and can be further reduced by orders of magnitude with additional assumptions on the simulation errors and design points. The analytical invertibility also enhance the numerical stability dramatically. The key in our approach is that we identify a general functional form of covariance functions that can induce sparsity in the corresponding inverse matrices. We also establish a connection between MCFs and linear ordinary differential equations. Such a connection provides a flexible, principled approach to constructing a wide class of MCFs. Extensive numerical experiments demonstrate that stochastic kriging with MCFs can handle large-scale problems in an both computationally efficient and numerically stable manner.

stat.ME↗
arXiv2018

Efficient Learning of Optimal Markov Network Topology with k-Tree Modeling

The seminal work of Chow and Liu (1968) shows that approximation of a finite probabilistic system by Markov trees can achieve the minimum information loss with the topology of a maximum spanning tree. Our current paper generalizes the result to Markov networks of tree width $\leq k$, for every fixed $k\geq 2$. In particular, we prove that approximation of a finite probabilistic system with such Markov networks has the minimum information loss when the network topology is achieved with a maximum spanning $k$-tree. While constructing a maximum spanning $k$-tree is intractable for even $k=2$, we show that polynomial algorithms can be ensured by a sufficient condition accommodated by many meaningful applications. In particular, we prove an efficient algorithm for learning the optimal topology of higher order correlations among random variables that belong to an underlying linear structure.

cs.DS↗
arXiv2018

On mountain pass theorem and its application to periodic solutions of some nonlinear discrete systems

We obtain a new quantitative deformation lemma, and then gain a new mountain pass theorem. More precisely, the new mountain pass theorem is independent of the functional value on the boundary of the mountain, which improves the well known results (\cite{AR,PS1,PS2,Qi,Wil}). Moreover, by our new mountain pass theorem, new existence of nontrivial periodic solutions for some nonlinear second-order discrete systems is obtained, which greatly improves the result in \cite{Z04}.

math.DS↗
arXiv2016

RGBD-based Parameter Extraction for Door Opening Tasks with Human Assists in Nuclear Rescue

The ability to open a door is essential for robots to perform home-serving and rescuing tasks. A substantial problem is to obtain the necessary parameters such as the width of the door and the length of the handle. Many researchers utilize computer vision techniques to extract the parameters automatically which lead to fine but not very stable results because of the complexity of the environment. We propose a method that utilizes an RGBD sensor and a GUI for users to 'point' at the target region with a mouse to acquire 3D information. Algorithms that can extract important parameters from the selected points are designed. To avoid large internal force induced by the misalignment of the robot orientation and the normal of the door plane, we design a module that can compute the normal of the plane by pointing at three non-collinear points and then drive the robot to the desired orientation. We carried out experiments on real robot. The result shows that the designed GUI and algorithms can help find the necessary parameters stably and get the robot prepared for further operations.

cs.RO↗
arXiv2014

New Quantitative Deformation Lemma and New Mountain Pass Theorem

In this paper, we obtain a new quantitative deformation Lemma so that we can obtain more critical points, especially for supinf critical value $c_1$, $x=φ^{-1}(c_1)$ is a new critical point. For $infmax$ critical value $c_2$, we can obtain two new critical points $x = 0$ (valley point) and $x = e$(peak point) ,comparing with Willem's variant of the mountain pass theorem of Ambrosetti-Rabinowitz,in which $φ(e)\leqφ(0)<c_2$, but in our new mountain pass theorem, $ φ(e)=c_2$.

math.FA↗
arXiv2014

Ab initio Prediction of RNA Nucleotide Interactions with Backbone k-Tree Model

Given the importance of non-coding RNAs to cellular regulatory functions and rapid growth of RNA transcripts, computational prediction of RNA tertiary structure remains highly demanded yet significantly challenging. Even for a short RNA sequence, the space of tertiary conformations is immense; existing methods to identify native-like conformations mostly resort to random sampling of conformations to gain computational feasibility. However native conformations may not be examined and prediction accuracy may be compromised due to sampling. In particular, the state-of-the-art methods have yet to deliver the desired prediction performance for RNAs of length beyond 50. This paper presents the work to tackle a key step in the RNA tertiary structure prediction problem, the prediction of the nucleotide interactions that constitute the desired tertiary structure. The research is established upon a novel graph model, called backbone k-tree, to markably constrain nucleotide interaction relationships in RNA tertiary structure. It is shown that the new model makes it possible to efficiently predict the optimal set of nucleotide interactions from the query sequence, including the interactions in all recently revealed families. Evident by the preliminary results, the new method can predict with a high accuracy the nucleotide interactions that constitute the tertiary structure of the query sequence, thus providing a viable solution towards ab initio prediction of RNA tertiary structure.

q-bio.BM↗
arXiv2013

Polynomial kernels collapse the W-hierarchy

We prove that, for many parameterized problems in the class FPT, the existence of polynomial kernels implies the collapse of the W-hierarchy (i.e., W[P] = FPT). The collapsing results are also extended to assumed exponential kernels for problems in the class FPT. In particular, we establish a close relationship between polynomial (and exponential) kernelizability and the existence of sub-exponential time algorithms for a spectrum of circuit satisfiability problems in FPT. To the best of our knowledge, this is the first work that connects hardness for polynomial kernelizability of FPT problems to parameterized intractability. Our work also offers some new insights into the class FPT.

cs.CC↗