arXiv subjects

Lei Li

Publications and source records attributed to Lei Li.

At least 739 records · Page 41Linked to original sources

arXiv2019

Rethinking Text Attribute Transfer: A Lexical Analysis

Text attribute transfer is modifying certain linguistic attributes (e.g. sentiment, style, authorship, etc.) of a sentence and transforming them from one type to another. In this paper, we aim to analyze and interpret what is changed during the transfer process. We start from the observation that in many existing models and datasets, certain words within a sentence play important roles in determining the sentence attribute class. These words are referred to as \textit{the Pivot Words}. Based on these pivot words, we propose a lexical analysis framework, \textit{the Pivot Analysis}, to quantitatively analyze the effects of these words in text attribute classification and transfer. We apply this framework to existing datasets and models and show that: (1) the pivot words are strong features for the classification of sentence attributes; (2) to change the attribute of a sentence, many datasets only requires to change certain pivot words; (3) consequently, many transfer models only perform the lexical-level modification, while leaving higher-level sentence structures unchanged. Our work provides an in-depth understanding of linguistic attribute transfer and further identifies the future requirements and challenges of this task\footnote{Our code can be found at https://github.com/FranxYao/pivot_analysis}.

cs.CL↗

arXiv2019

Random batch methods (RBM) for interacting particle systems

We develop Random Batch Methods for interacting particle systems with large number of particles. These methods use small but random batches for particle interactions, thus the computational cost is reduced from $O(N^2)$ per time step to $O(N)$, for a system with $N$ particles with binary interactions. On one hand, these methods are efficient Asymptotic-Preserving schemes for the underlying particle systems, allowing $N$-independent time steps and also capture, in the $N \to \infty$ limit, the solution of the mean field limit which are nonlinear Fokker-Planck equations; on the other hand, the stochastic processes generated by the algorithms can also be regarded as new models for the underlying problems. For one of the methods, we give a particle number independent error estimate under some special interactions. Then, we apply these methods to some representative problems in mathematics, physics, social and data sciences, including the Dyson Brownian motion from random matrix theory, Thomson's problem, distribution of wealth, opinion dynamics and clustering. Numerical results show that the methods can capture both the transient solutions and the global equilibrium in these problems.

math.NA↗

arXiv2019

Effect of gallium doping on bubbling and helium retention in aluminum exposed to low-energy helium plasma

Surface bubbling and helium retention of pure aluminum and a solid solution of aluminum-gallium(Ga) alloy exposed to low-energy He plasma has been investigated with the fluence of 1.8E24 He/m2 at room temperature . Surface morphology observations show that after irradiation, the average size of He bubbles on Al-1.8 at.% Ga alloy is around 1.8 micro metre, smaller than that on pure Al. Ga doping noticeably increases the area density of the bubble. The thermal desorption spectroscopyshows that the release amount of He in Al-1.8 at.% Ga alloy is 1.77E21 He/m2, nearly three orders of magnitude higher than that in pure Al, whlie the He desorption peak of Al-1.8 at.% Ga alloy is 480 K, much lower than 580 K of pure Al. The results of slow positron annihilation spectroscopy (SPAS) indicate that the vacancy type defects were introduced by He plasma irradiation; and lattice distortion caused by Ga doping plays an important role in determining surface bubbling and He retention characteristics of Al-1.8 at.% Ga.

cond-mat.mtrl-sci↗

arXiv2019

Effective Domain Knowledge Transfer with Soft Fine-tuning

Convolutional neural networks require numerous data for training. Considering the difficulties in data collection and labeling in some specific tasks, existing approaches generally use models pre-trained on a large source domain (e.g. ImageNet), and then fine-tune them on these tasks. However, the datasets from source domain are simply discarded in the fine-tuning process. We argue that the source datasets could be better utilized and benefit fine-tuning. This paper firstly introduces the concept of general discrimination to describe ability of a network to distinguish untrained patterns, and then experimentally demonstrates that general discrimination could potentially enhance the total discrimination ability on target domain. Furthermore, we propose a novel and light-weighted method, namely soft fine-tuning. Unlike traditional fine-tuning which directly replaces optimization objective by a loss function on the target domain, soft fine-tuning effectively keeps general discrimination by holding the previous loss and removes it softly. By doing so, soft fine-tuning improves the robustness of the network to data bias, and meanwhile accelerates the convergence. We evaluate our approach on several visual recognition tasks. Extensive experimental results support that soft fine-tuning provides consistent improvement on all evaluated tasks, and outperforms the state-of-the-art significantly. Codes will be made available to the public.

cs.CV↗

arXiv2019

Uniform-in-Time Weak Error Analysis for Stochastic Gradient Descent Algorithms via Diffusion Approximation

Diffusion approximation provides weak approximation for stochastic gradient descent algorithms in a finite time horizon. In this paper, we introduce new tools motivated by the backward error analysis of numerical stochastic differential equations into the theoretical framework of diffusion approximation, extending the validity of the weak approximation from finite to infinite time horizon. The new techniques developed in this paper enable us to characterize the asymptotic behavior of constant-step-size SGD algorithms for strongly convex objective functions, a goal previously unreachable within the diffusion approximation framework. Our analysis builds upon a truncated formal power expansion of the solution of a stochastic modified equation arising from diffusion approximation, where the main technical ingredient is a uniform-in-time weak error bound controlling the long-term behavior of the expansion coefficient functions near the global minimum. We expect these new techniques to greatly expand the range of applicability of diffusion approximation to cover wider and deeper aspects of stochastic optimization algorithms in data science.

cs.LG↗

arXiv2019

Self-reinforcing Unsupervised Matching

Remarkable gains in deep learning usually rely on tremendous supervised data. Ensuring the modality diversity for one object in training set is critical for the generalization of cutting-edge deep models, but it burdens human with heavy manual labor on data collection and annotation. In addition, some rare or unexpected modalities are new for the current model, causing reduced performance under such emerging modalities. Inspired by the achievements in speech recognition, psychology and behavioristics, we present a practical solution, self-reinforcing unsupervised matching (SUM), to annotate the images with 2D structure-preserving property in an emerging modality by cross-modality matching. This approach requires no any supervision in emerging modality and only one template in seen modality, providing a possible route towards continual learning.

cs.CV↗

arXiv2019

A discretization of Caputo derivatives with application to time fractional SDEs and gradient flows

We consider a discretization of Caputo derivatives resulted from deconvolving a scheme for the corresponding Volterra integral. Properties of this discretization, including signs of the coefficients, comparison principles, and stability of the corresponding implicit schemes, are proved by its linkage to Volterra integrals with completely monotone kernels. We then apply the backward scheme corresponding to this discretization to two time fractional dissipative problems, and these implicit schemes are helpful for the analysis of the corresponding problems. In particular, we show that the overdamped generalized Langevin equation with fractional noise has a unique limiting measure for strongly convex potentials and establish the convergence of numerical solutions to the strong solutions of time fractional gradient flows. The proposed scheme and schemes derived using the same philosophy can be useful for many other applications as well.

math.NA↗

arXiv2019

Correct-and-Memorize: Learning to Translate from Interactive Revisions

State-of-the-art machine translation models are still not on par with human translators. Previous work takes human interactions into the neural machine translation process to obtain improved results in target languages. However, not all model-translation errors are equal -- some are critical while others are minor. In the meanwhile, the same translation mistakes occur repeatedly in a similar context. To solve both issues, we propose CAMIT, a novel method for translating in an interactive environment. Our proposed method works with critical revision instructions, therefore allows human to correct arbitrary words in model-translated sentences. In addition, CAMIT learns from and softly memorizes revision actions based on the context, alleviating the issue of repeating mistakes. Experiments in both ideal and real interactive translation settings demonstrate that our proposed \method enhances machine translation results significantly while requires fewer revision instructions from human compared to previous methods.

cs.CL↗

arXiv2019

Automatic Generation of Personalized Comment Based on User Profile

Comments on social media are very diverse, in terms of content, style and vocabulary, which make generating comments much more challenging than other existing natural language generation~(NLG) tasks. Besides, since different user has different expression habits, it is necessary to take the user's profile into consideration when generating comments. In this paper, we introduce the task of automatic generation of personalized comment~(AGPC) for social media. Based on tens of thousands of users' real comments and corresponding user profiles on weibo, we propose Personalized Comment Generation Network~(PCGN) for AGPC. The model utilizes user feature embedding with a gated memory and attends to user description to model personality of users. In addition, external user representation is taken into consideration during the decoding to enhance the comments generation. Experimental results show that our model can generate natural, human-like and personalized comments.

cs.CL↗

arXiv2019

Hour-scale persistent negative anomaly of atmospheric electrostatic field near the epicenter before earthquake

Although earthquake prediction is a big challenge in the world, some simple observational tools can capture many physical signals and demonstrate that an earthquake (EQ) may be forthcoming in short period. Many researchers have studied the significant variation of atmospheric electrostatic field related to the forthcoming earthquake. However, until now, there is not a compelling physical mechanism which can explain why atmospheric electrostatic abnormal signal could appear just before an earthquake. Here we present a precursor signal and propose a brief physical interpretation. Under fair air conditions, if the near-surface atmospheric electrostatic field Ez (oriented down when it is positive) presents a very stable negative anomaly (from -100 V/m to -5000 V/m), it will forebode that an earthquake (seismicity from 3-8) would take place in the next several to tens of hours within a distance less than 100 km. We name this prediction technique as "DC Ez Determination"(DED). In addition, the mechanism of such abnormal quasi-static electric field before a forthcoming earthquake has been proposed here: (1) Radon gas releases from the rock clefts near the epicenter during seismogenic process. (2) α particles are produced due to the radioactive decay of Radon gas in the air. (3) α particle ionizing radiation creates more positive particles in the air, which is much more effective than that β and γ particles produced during Radon radioactive decay. (4) The new positive particles change formal positive atmospheric electric field (Ez) into stably negative near the earth-surface. (5) The closer the instrument is to the epicenter, the more likely it is to observe the negative Ez signal related to the earthquake. It is recommended to establish an instrument network to capture the reliable precursor and make some warnings before the earthquake disaster.

physics.geo-ph↗

arXiv2019

Generating Sentences from Disentangled Syntactic and Semantic Spaces

Variational auto-encoders (VAEs) are widely used in natural language generation due to the regularization of the latent space. However, generating sentences from the continuous latent space does not explicitly model the syntactic information. In this paper, we propose to generate sentences from disentangled syntactic and semantic spaces. Our proposed method explicitly models syntactic information in the VAE's latent space by using the linearized tree sequence, leading to better performance of language generation. Additionally, the advantage of sampling in the disentangled syntactic and semantic latent spaces enables us to perform novel applications, such as the unsupervised paraphrase generation and syntax-transfer generation. Experimental results show that our proposed model achieves similar or better performance in various tasks, compared with state-of-the-art related work.

cs.CL↗

arXiv2019

Quantum Anomaly Detection with Density Estimation and Multivariate Gaussian Distribution

We study quantum anomaly detection with density estimation and multivariate Gaussian distribution. Both algorithms are constructed using the standard gate-based model of quantum computing. Compared with the corresponding classical algorithms, the resource complexities of our quantum algorithm are logarithmic in the dimensionality of quantum states and the number of training quantum states. We also present a quantum procedure for efficiently estimating the determinant of any Hermitian operators $\mathcal{A}\in\mathcal{R}^{N\times N}$ with time complexity $O(poly\log N)$ which forms an important subroutine in our quantum anomaly detection with multivariate Gaussian distribution. Finally, our results also include the modified quantum kernel principal component analysis (PCA) and the quantum one-class support vector machine (SVM) for detecting classical data.

quant-ph↗

arXiv2019

Dynamically Fused Graph Network for Multi-hop Reasoning

Text-based question answering (TBQA) has been studied extensively in recent years. Most existing approaches focus on finding the answer to a question within a single paragraph. However, many difficult questions require multiple supporting evidence from scattered text among two or more documents. In this paper, we propose Dynamically Fused Graph Network(DFGN), a novel method to answer those questions requiring multiple scattered evidence and reasoning over them. Inspired by human's step-by-step reasoning behavior, DFGN includes a dynamic fusion layer that starts from the entities mentioned in the given query, explores along the entity graph dynamically built from the text, and gradually finds relevant supporting entities from the given documents. We evaluate DFGN on HotpotQA, a public TBQA dataset requiring multi-hop reasoning. DFGN achieves competitive results on the public board. Furthermore, our analysis shows DFGN produces interpretable reasoning chains.

cs.CL↗

arXiv2019

Enhancing test precision for local Lorentz symmetry violation with entanglement

A recent proposal for testing Lorentz symmetry violation (LSV) presents a formulation where the effect of violation is described as a local interaction [R. Shaniv, et al, Phys. Rev. Lett. 120, 103202 (2018)]. An entangled ion pair in a decoherence free subspace (DFS) is shown to double the signal to noise ratio (SNR) of one ion, while (even)-N/2 such DFS pairs in a collective entangled state improve SNR by N times, provided the state parity or the even/odd numbers of ions can be measured. It remains to find out, however, how such fiducial entangled states can be prepared at nonexponentially small success rates. This work suggests two types of many particle entangled states for testing LSV: the maximally entangled NOON state, which can achieve Heisenberg limited precision; and the balanced spin-1 Dicke state, which is readily available in deterministic fashion. We show that the latter also lives in a DFS and is immune to stray magnetic fields. It can achieve classical precision limit or the standard quantum limit (SQL) based on collective population measurement without individual atom resolution. Given the high interests in LSV and in entanglement assisted quantum metrology, our observation offers additional incentives for pursuing practical applications of many atom entangled states.

quant-ph↗

arXiv2019

UniVSE: Robust Visual Semantic Embeddings via Structured Semantic Representations

We propose Unified Visual-Semantic Embeddings (UniVSE) for learning a joint space of visual and textual concepts. The space unifies the concepts at different levels, including objects, attributes, relations, and full scenes. A contrastive learning approach is proposed for the fine-grained alignment from only image-caption pairs. Moreover, we present an effective approach for enforcing the coverage of semantic components that appear in the sentence. We demonstrate the robustness of Unified VSE in defending text-domain adversarial attacks on cross-modal retrieval tasks. Such robustness also empowers the use of visual cues to resolve word dependencies in novel sentences.

cs.CV↗

arXiv2019

UAV Positioning and Power Control for Two-Way Wireless Relaying

This paper considers an unmanned-aerial-vehicle-enabled (UAV-enabled) wireless network where a relay UAV is used for two-way communications between a ground base station (BS) and a set of distant user equipment (UE). The UAV adopts the amplify-and-forward strategy for two-way relaying over orthogonal frequency bands. The UAV positioning and the transmission powers of all nodes are jointly designed to maximize the sum rate of both uplink and downlink subject to transmission power constraints and the signal-to-noise ratio constraint on the UAV control channel. The formulated joint positioning and power control (JPPC) problem has an intricate expression of the sum rate due to two-way transmissions and is difficult to solve in general. We propose a novel concave surrogate function for the sum rate and employ the successive convex approximation (SCA) technique for obtaining a high-quality approximate solution. We show that the proposed surrogate function has a small curvature and enables a fast convergence of SCA. Furthermore, we develop a computationally efficient JPPC algorithm by applying the FISTA-type accelerated gradient projection (AGP) algorithm to solve the SCA problem as well as one of the projection subproblem, resulting in a double-loop AGP method. Simulation results show that the proposed JPPC algorithms are not only computationally efficient but also greatly outperform the heuristic approaches.

cs.IT↗

arXiv2019

A stochastic version of Stein Variational Gradient Descent for efficient sampling

We propose in this work RBM-SVGD, a stochastic version of Stein Variational Gradient Descent (SVGD) method for efficiently sampling from a given probability measure and thus useful for Bayesian inference. The method is to apply the Random Batch Method (RBM) for interacting particle systems proposed by Jin et al to the interacting particle systems in SVGD. While keeping the behaviors of SVGD, it reduces the computational cost, especially when the interacting kernel has long range. Numerical examples verify the efficiency of this new version of SVGD.

stat.ML↗

arXiv2019

3D Face Mask Presentation Attack Detection Based on Intrinsic Image Analysis

Face presentation attacks have become a major threat to face recognition systems and many countermeasures have been proposed in the past decade. However, most of them are devoted to 2D face presentation attacks, rather than 3D face masks. Unlike the real face, the 3D face mask is usually made of resin materials and has a smooth surface, resulting in reflectance differences. So, we propose a novel detection method for 3D face mask presentation attack by modeling reflectance differences based on intrinsic image analysis. In the proposed method, the face image is first processed with intrinsic image decomposition to compute its reflectance image. Then, the intensity distribution histograms are extracted from three orthogonal planes to represent the intensity differences of reflectance images between the real face and 3D face mask. After that, the 1D convolutional network is further used to capture the information for describing different materials or surfaces react differently to changes in illumination. Extensive experiments on the 3DMAD database demonstrate the effectiveness of our proposed method in distinguishing a face mask from the real one and show that the detection performance outperforms other state-of-the-art methods.

cs.CV↗