arXiv subjects

Lei Li

Publications and source records attributed to Lei Li.

At least 703 records · Page 39Linked to original sources

arXiv2020

SOLO: Segmenting Objects by Locations

We present a new, embarrassingly simple approach to instance segmentation in images. Compared to many other dense prediction tasks, e.g., semantic segmentation, it is the arbitrary number of instances that have made instance segmentation much more challenging. In order to predict a mask for each instance, mainstream approaches either follow the 'detect-thensegment' strategy as used by Mask R-CNN, or predict category masks first then use clustering techniques to group pixels into individual instances. We view the task of instance segmentation from a completely new perspective by introducing the notion of "instance categories", which assigns categories to each pixel within an instance according to the instance's location and size, thus nicely converting instance mask segmentation into a classification-solvable problem. Now instance segmentation is decomposed into two classification tasks. We demonstrate a much simpler and flexible instance segmentation framework with strong performance, achieving on par accuracy with Mask R-CNN and outperforming recent singleshot instance segmenters in accuracy. We hope that this very simple and strong framework can serve as a baseline for many instance-level recognition tasks besides instance segmentation.

cs.CV↗

arXiv2020

FoveaBox: Beyond Anchor-based Object Detector

We present FoveaBox, an accurate, flexible, and completely anchor-free framework for object detection. While almost all state-of-the-art object detectors utilize predefined anchors to enumerate possible locations, scales and aspect ratios for the search of the objects, their performance and generalization ability are also limited to the design of anchors. Instead, FoveaBox directly learns the object existing possibility and the bounding box coordinates without anchor reference. This is achieved by: (a) predicting category-sensitive semantic maps for the object existing possibility, and (b) producing category-agnostic bounding box for each position that potentially contains an object. The scales of target boxes are naturally associated with feature pyramid representations. In FoveaBox, an instance is assigned to adjacent feature levels to make the model more accurate.We demonstrate its effectiveness on standard benchmarks and report extensive experimental analysis. Without bells and whistles, FoveaBox achieves state-of-the-art single model performance on the standard COCO and Pascal VOC object detection benchmark. More importantly, FoveaBox avoids all computation and hyper-parameters related to anchor boxes, which are often sensitive to the final detection performance. We believe the simple and effective approach will serve as a solid baseline and help ease future research for object detection. The code has been made publicly available at https://github.com/taokong/FoveaBox .

cs.CV↗

arXiv2020

Do You Have the Right Scissors? Tailoring Pre-trained Language Models via Monte-Carlo Methods

It has been a common approach to pre-train a language model on a large corpus and fine-tune it on task-specific data. In practice, we observe that fine-tuning a pre-trained model on a small dataset may lead to over- and/or under-estimation problem. In this paper, we propose MC-Tailor, a novel method to alleviate the above issue in text generation tasks by truncating and transferring the probability mass from over-estimated regions to under-estimated ones. Experiments on a variety of text generation datasets show that MC-Tailor consistently and significantly outperforms the fine-tuning approach. Our code is available at this url.

cs.CL↗

arXiv2020

Generating Fluent Adversarial Examples for Natural Languages

Efficiently building an adversarial attacker for natural language processing (NLP) tasks is a real challenge. Firstly, as the sentence space is discrete, it is difficult to make small perturbations along the direction of gradients. Secondly, the fluency of the generated examples cannot be guaranteed. In this paper, we propose MHA, which addresses both problems by performing Metropolis-Hastings sampling, whose proposal is designed with the guidance of gradients. Experiments on IMDB and SNLI show that our proposed MHA outperforms the baseline model on attacking capability. Adversarial training with MAH also leads to better robustness and performance.

cs.CL↗

arXiv2020

Energy and quadratic invariants preserving methods for Hamiltonian systems with holonomic constraints

We introduce a new class of parametricization structure-preserving partitioned Runge-Kutta ($α$-PRK) methods for Hamiltonian systems with holonomic constraints. When the scalar parameter $α=0$, the methods are reduced to the usual symplectic PRK methods like Shake-Rattle method or PRK schemes based on Lobatto IIIA-IIIB pairs, which can preserve all the quadratic invariants and the constraints. When $α\neq 0$, the methods are also shown to preserve all the quadratic invariants and the constraints manifold exactly. At the same time, for any given consistent initial values $(p_{0}, q_0)$ and small step size $h>0$, it is proved that there exists $α^*=α(h, p_0, q_0)$ such that the Hamiltonian energy can also be exactly preserved at each step. We provide a new variational formulation for symplectic PRK schemes and use it to prove that the parametrized PRK methods can preserve the quadratic invariants for Hamiltonian systems subject to holonomic constraints. The parametric $α$-PRK methods are shown to have the same convergence rate as the usual PRK methods and perform very well in various numerical experiments.

math.NA↗

arXiv2020

Improving Maximum Likelihood Training for Text Generation with Density Ratio Estimation

Auto-regressive sequence generative models trained by Maximum Likelihood Estimation suffer the exposure bias problem in practical finite sample scenarios. The crux is that the number of training samples for Maximum Likelihood Estimation is usually limited and the input data distributions are different at training and inference stages. Many method shave been proposed to solve the above problem (Yu et al., 2017; Lu et al., 2018), which relies on sampling from the non-stationary model distribution and suffers from high variance or biased estimations. In this paper, we proposeψ-MLE, a new training scheme for auto-regressive sequence generative models, which is effective and stable when operating at large sample space encountered in text generation. We derive our algorithm from a new perspective of self-augmentation and introduce bias correction with density ratio estimation. Extensive experimental results on synthetic data and real-world text generation tasks demonstrate that our method stably outperforms Maximum Likelihood Estimation and other state-of-the-art sequence generative models in terms of both quality and diversity.

stat.ML↗

arXiv2020

Xiaomingbot: A Multilingual Robot News Reporter

This paper proposes the building of Xiaomingbot, an intelligent, multilingual and multimodal software robot equipped with four integral capabilities: news generation, news translation, news reading and avatar animation. Its system summarizes Chinese news that it automatically generates from data tables. Next, it translates the summary or the full article into multiple languages, and reads the multilingual rendition through synthesized speech. Notably, Xiaomingbot utilizes a voice cloning technology to synthesize the speech trained from a real person's voice data in one input language. The proposed system enjoys several merits: it has an animated avatar, and is able to generate and read multilingual news. Since it was put into practice, Xiaomingbot has written over 600,000 articles, and gained over 150,000 followers on social media platforms.

eess.AS↗

arXiv2020

XREF: Entity Linking for Chinese News Comments with Supplementary Article Reference

Automatic identification of mentioned entities in social media posts facilitates quick digestion of trending topics and popular opinions. Nonetheless, this remains a challenging task due to limited context and diverse name variations. In this paper, we study the problem of entity linking for Chinese news comments given mentions' spans. We hypothesize that comments often refer to entities in the corresponding news article, as well as topics involving the entities. We therefore propose a novel model, XREF, that leverages attention mechanisms to (1) pinpoint relevant context within comments, and (2) detect supporting entities from the news article. To improve training, we make two contributions: (a) we propose a supervised attention loss in addition to the standard cross entropy, and (b) we develop a weakly supervised training scheme to utilize the large-scale unlabeled corpus. Two new datasets in entertainment and product domains are collected and annotated for experiments. Our proposed method outperforms previous methods on both datasets.

cs.CL↗

arXiv2020

Joint Left Atrial Segmentation and Scar Quantification Based on a DNN with Spatial Encoding and Shape Attention

We propose an end-to-end deep neural network (DNN) which can simultaneously segment the left atrial (LA) cavity and quantify LA scars. The framework incorporates the continuous spatial information of the target by introducing a spatially encoded (SE) loss based on the distance transform map. Compared to conventional binary label based loss, the proposed SE loss can reduce noisy patches in the resulting segmentation, which is commonly seen for deep learning-based methods. To fully utilize the inherent spatial relationship between LA and LA scars, we further propose a shape attention (SA) mechanism through an explicit surface projection to build an end-to-end-trainable model. Specifically, the SA scheme is embedded into a two-task network to perform the joint LA segmentation and scar quantification. Moreover, the proposed method can alleviate the severe class-imbalance problem when detecting small and discrete targets like scars. We evaluated the proposed framework on 60 LGE MRI data from the MICCAI2018 LA challenge. For LA segmentation, the proposed method reduced the mean Hausdorff distance from 36.4 mm to 20.0 mm compared to the 3D basic U-Net using the binary cross-entropy loss. For scar quantification, the method was compared with the results or algorithms reported in the literature and demonstrated better performance.

eess.IV↗

arXiv2020

VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research

We present a new large-scale multilingual video description dataset, VATEX, which contains over 41,250 videos and 825,000 captions in both English and Chinese. Among the captions, there are over 206,000 English-Chinese parallel translation pairs. Compared to the widely-used MSR-VTT dataset, VATEX is multilingual, larger, linguistically complex, and more diverse in terms of both video and natural language descriptions. We also introduce two tasks for video-and-language research based on VATEX: (1) Multilingual Video Captioning, aimed at describing a video in various languages with a compact unified captioning model, and (2) Video-guided Machine Translation, to translate a source language description into the target language using the video information as additional spatiotemporal context. Extensive experiments on the VATEX dataset show that, first, the unified multilingual model can not only produce both English and Chinese descriptions for a video more efficiently, but also offer improved performance over the monolingual models. Furthermore, we demonstrate that the spatiotemporal video context can be effectively utilized to align source and target languages and thus assist machine translation. In the end, we discuss the potentials of using VATEX for other video-and-language research.

cs.CV↗

arXiv2020

Adaptive Gradient Methods Can Be Provably Faster than SGD after Finite Epochs

Adaptive gradient methods have attracted much attention of machine learning communities due to the high efficiency. However their acceleration effect in practice, especially in neural network training, is hard to analyze, theoretically. The huge gap between theoretical convergence results and practical performances prevents further understanding of existing optimizers and the development of more advanced optimization methods. In this paper, we provide adaptive gradient methods a novel analysis with an additional mild assumption, and revise AdaGrad to \radagrad for matching a better provable convergence rate. To find an $ε$-approximate first-order stationary point in non-convex objectives, we prove random shuffling \radagrad achieves a $\tilde{O}(T^{-1/2})$ convergence rate, which is significantly improved by factors $\tilde{O}(T^{-1/4})$ and $\tilde{O}(T^{-1/6})$ compared with existing adaptive gradient methods and random shuffling SGD, respectively. To the best of our knowledge, it is the first time to demonstrate that adaptive gradient methods can deterministically be faster than SGD after finite epochs. Furthermore, we conduct comprehensive experiments to validate the additional mild assumption and the acceleration effect benefited from second moments and random shuffling.

math.OC↗

arXiv2020

ACMo: Angle-Calibrated Moment Methods for Stochastic Optimization

Due to its simplicity and outstanding ability to generalize, stochastic gradient descent (SGD) is still the most widely used optimization method despite its slow convergence. Meanwhile, adaptive methods have attracted rising attention of optimization and machine learning communities, both for the leverage of life-long information and for the profound and fundamental mathematical theory. Taking the best of both worlds is the most exciting and challenging question in the field of optimization for machine learning. Along this line, we revisited existing adaptive gradient methods from a novel perspective, refreshing understanding of second moments. Our new perspective empowers us to attach the properties of second moments to the first moment iteration, and to propose a novel first moment optimizer, \emph{Angle-Calibrated Moment method} (\method). Our theoretical results show that \method is able to achieve the same convergence rate as mainstream adaptive methods. Furthermore, extensive experiments on CV and NLP tasks demonstrate that \method has a comparable convergence to SOTA Adam-type optimizers, and gains a better generalization performance in most cases.

math.OC↗

arXiv2020

Coherent control of dissipative dynamics in a periodically driven lattice array

We find a different mechanism for suppression of decay in an open one-dimensional lattice system, which originates from a dark Floquet state, a sink state to which the system is asymptotically driven, whose overall probability is determined only by the parameters of the periodic driving field. The zero-quasienergy of dark Floquet state has been shown to be not a real zero, but a vanishingly small negative imaginary number which will cause undesirable physical effect in long-time evolution of quantum states, which is extremely different from the conservative counterpart. Another important finding is that the value of the system's effective decay, determined by the size of the non-zero imaginary part of the dark-Floquet-state-related quasienergy, depends not on how many localized lossy sites there are but on which one of the lossy sites is nearest to the driven site. Thus, for specially designed local dissipation, by controlling the driving parameters, it is possible for us to drive the system to a dark Floquet state with a much lower level of overall probability loss as compared to the undriven case and with good stability over enough longer evolution time. These results are applicable to the multisite lattice system with an odd number of sites and may be significant for long-time control of decay in a vast family of multistate physical systems with localized dissipation.

quant-ph↗

arXiv2020

The reduction of the number of incoherent Kraus operations for qutrit systems

Quantum coherence is a fundamental property that can emerge within any quantum system. Incoherent operations, defined in terms of the Kraus decomposition, take an important role in state transformation. The maximum number of incoherent Kraus operators has been presented in [A. Streltsov, S. Rana, P. Boes, J. Eisert, Phys. Rev. Lett. 119. 140402 (2017)]. In this work, we show that the number of incoherent Kraus operators for a single qubit can be reduced from 5 to 4 by constructing a proper unitary matrix. For qutrit systems we further obtain 32 incoherent Kraus operators, while the upper bound in the research of Sterltsov gives 39 Kraus operators. Besides, we reduce the number of strictly incoherent Kraus operators from more than 15 to 13. And we consider the state transformation problem for these two types of operations in single qutrit systems.

quant-ph↗

arXiv2020

A viral propagation model with nonlinear infection rate and free boundaries

In this paper we put forward a viral propagation model with nonlinear infection rate and free boundaries and investigate the dynamical properties. This model is composed of two ordinary differential equations and one partial differential equation, in which the spatial range of the first equation is the whole space $\mathbb{R}$, and the last two equations have free boundaries. As a new mathematical model, we prove the existence, uniqueness and uniform estimates of global solution, and provide the criteria for spreading and vanishing, and long time behavior of the solution components $u,v,w$. Comparing with the corresponding ordinary differential systems, the {\it Basic Reproduction Number} ${\cal R}_0$ plays a different role. We find that when ${\cal R}_0\le 1$, the virus cannot spread successfully; when ${\cal R}_0>1$, the successful spread of virus depends on the initial value and varying parameters.

math.AP↗

arXiv2020

Nucleons pair shell model in M-scheme

The nucleon pair shell model (NPSM) is casted into the so-called M-scheme for the cases with isospin symmetry and without isospin symmetry. The odd system and even system are treated on the same foot. The uncoupled commutators for nucleon-pairs, which are suitable for M-scheme, are given. Explicit formula of matrix elements in M-scheme for overlap, one-body operators, two-body operators are obtained. It is found that the $cpu$ time used in calculating the matrix elements in M-scheme is much shorter than that in the J-scheme of NPSM.

nucl-th↗

arXiv2020

Convergence of Random Batch Method for interacting particles with disparate species and weights

We consider in this work the convergence of Random Batch Method proposed in our previous work [Jin et al., J. Comput. Phys., 400(1), 2020] for interacting particles to the case of disparate species and weights. We show that the strong error is of $O(\sqrtτ)$ while the weak error is of $O(τ)$ where $τ$ is the time step between two random divisions of batches. Both types of convergence are uniform in $N$, the number of particles. The proof of strong convergence follows closely the proof in [Jin et al., J. Comput. Phys., 400(1), 2020] for indistinguishable particles, but there are still some differences: since there is no exchangeability now, we have to use a certain weighted average of the errors; some refined auxiliary lemmas have to be proved compared with our previous work. To show that the weak convergence of empirical measure is uniform in $N$, certain sharp estimates for the derivatives of the backward equations have been used. The weak convergence analysis is also illustrating for the convergence of Random Batch Method for $N$-body Liouville equations.

math.NA↗

arXiv2020

Variational Quantum Algorithms for Dimensionality Reduction and Classification

In this work, we present a quantum neighborhood preserving embedding and a quantum local discriminant embedding for dimensionality reduction and classification. We demonstrate that these two algorithms have an exponential speedup over their respectively classical counterparts. Along the way, we propose a variational quantum generalized eigenvalue solver that finds the generalized eigenvalues and eigenstates of a matrix pencil $(\mathcal{G},\mathcal{S})$. As a proof-of-principle, we implement our algorithm to solve $2^5\times2^5$ generalized eigenvalue problems. Finally, our results offer two optional outputs with quantum or classical form, which can be directly applied in another quantum or classical machine learning process.

quant-ph↗