arXiv subjects

Peng Jin

Publications and source records attributed to Peng Jin.

At least 91 records · Page 5Linked to original sources

arXiv2020

Distributional Discrepancy: A Metric for Unconditional Text Generation

The purpose of unconditional text generation is to train a model with real sentences, then generate novel sentences of the same quality and diversity as the training data. However, when different metrics are used for comparing the methods of unconditional text generation, contradictory conclusions are drawn. The difficulty is that both the diversity and quality of the sample should be considered simultaneously when the models are evaluated. To solve this problem, a novel metric of distributional discrepancy (DD) is designed to evaluate generators based on the discrepancy between the generated and real training sentences. However, it cannot compute the DD directly because the distribution of real sentences is unavailable. Thus, we propose a method for estimating the DD by training a neural-network-based text classifier. For comparison, three existing metrics, bi-lingual evaluation understudy (BLEU) versus self-BLEU, language model score versus reverse language model score, and Fréchet embedding distance, along with the proposed DD, are used to evaluate two popular generative models of long short-term memory and generative pretrained transformer 2 on both syntactic and real data. Experimental results show that DD is significantly better than the three existing metrics for ranking these generative models.

cs.CL↗

arXiv2020

Adding A Filter Based on The Discriminator to Improve Unconditional Text Generation

The autoregressive language model (ALM) trained with maximum likelihood estimation (MLE) is widely used in unconditional text generation. Due to exposure bias, the generated texts still suffer from low quality and diversity. This presents statistically as a discrepancy between the real text and generated text. Some research shows a discriminator can detect this discrepancy. Because the discriminator can encode more information than the generator, discriminator has the potentiality to improve generator. To alleviate the exposure bias, generative adversarial networks (GAN) use the discriminator to update the generator's parameters directly, but they fail by being evaluated precisely. A critical reason for the failure is the difference between the discriminator input and the ALM input. We propose a novel mechanism by adding a filter which has the same input as the discriminator. First, discriminator detects the discrepancy signals and passes to filter directly (or by learning). Then, we use the filter to reject some generated samples with a sampling-based method. Thus, the original generative distribution is revised to reduce the discrepancy. Two ALMs, RNN-based and Transformer-based, are experimented. Evaluated precisely by three metrics, our mechanism consistently outperforms the ALMs and all kinds of GANs across two benchmark data sets.

cs.CV↗

arXiv2020

Regularity of transition densities and ergodicity for affine jump-diffusion processes

In this paper we study the transition density and exponential ergodicity in total variation for an affine process on the canonical state space $\mathbb{R}_{\geq0}^{m}\times\mathbb{R}^{n}$. Under a Hörmander-type condition for diffusion components as well as a boundary non-attainment assumption, we derive the existence and regularity of the transition density for the affine process and then prove the strong Feller property. Moreover, we also show that under these and the additional subcritical conditions the corresponding affine process on the canonical state space is exponentially ergodic in the total variation distance. To prove existence and regularity of the transition density we derive some precise estimates for the real part of the characteristic function of the process. Our ergodicity result is a consequence of a suitable application of a Harris-type theorem based on a local Dobrushin condition combined with the regularity of the transition densities.

math.PR↗

arXiv2020

Pavement Image Datasets: A New Benchmark Dataset to Classify and Densify Pavement Distresses

Automated pavement distresses detection using road images remains a challenging topic in the computer vision research community. Recent developments in deep learning has led to considerable research activity directed towards improving the efficacy of automated pavement distress identification and rating. Deep learning models require a large ground truth data set, which is often not readily available in the case of pavements. In this study, a labeled dataset approach is introduced as a first step towards a more robust, easy-to-deploy pavement condition assessment system. The technique is termed herein as the Pavement Image Dataset (PID) method. The dataset consists of images captured from two camera views of an identical pavement segment, i.e., a wide-view and a top-down view. The wide-view images were used to classify the distresses and to train the deep learning frameworks, while the top-down view images allowed calculation of distress density, which will be used in future studies aimed at automated pavement rating. For the wide view group dataset, 7,237 images were manually annotated and distresses classified into nine categories. Images were extracted using the Google Application Programming Interface (API), selecting street-view images using a python-based code developed for this project. The new dataset was evaluated using two mainstream deep learning frameworks: You Only Look Once (YOLO v2) and Faster Region Convolution Neural Network (Faster R-CNN). Accuracy scores using the F1 index were found to be 0.84 for YOLOv2 and 0.65 for the Faster R-CNN model runs; both quite acceptable considering the convenience of utilizing Google maps images.

cs.CV↗

arXiv2020

Uniqueness in law for stable-like processes of variable order

Let $d\ge1$. Consider a stable-like operator of variable order \begin{align*} \mathcal{A}f(x) & =\int_{\mathbb{R}^{d} \backslash\{0\}} \left[f(x+h) -f(x) -\mathbf{1}_{\{|h|\le1\}}h \cdot\nabla f(x)\right]\frac{n(x,h)}{|h|^{d+α(x)}} \mathrm{d}h, \end{align*} where $0<\inf_{x}α(x) \le \sup_{x}α(x)<2$ and $n(x,h)$ satisfies \[ n(x,h)=n(x,-h),\quad0<κ_{1}\le n(x,h)\leκ_{2},\quad\forall x,h\in \mathbb{R}^{d}, \] with $κ_{1}$ and $κ_{2}$ being some positive constants. Under some further mild conditions on the functions $n(x,h)$ and $α(x)$, we show the uniqueness of solutions to the martingale problem for $\mathcal{A}$.

math.PR↗

arXiv2019

The Detection of Distributional Discrepancy for Text Generation

The text generated by neural language models is not as good as the real text. This means that their distributions are different. Generative Adversarial Nets (GAN) are used to alleviate it. However, some researchers argue that GAN variants do not work at all. When both sample quality (such as Bleu) and sample diversity (such as self-Bleu) are taken into account, the GAN variants even are worse than a well-adjusted language model. But, Bleu and self-Bleu can not precisely measure this distributional discrepancy. In fact, how to measure the distributional discrepancy between real text and generated text is still an open problem. In this paper, we theoretically propose two metric functions to measure the distributional difference between real text and generated text. Besides that, a method is put forward to estimate them. First, we evaluate language model with these two functions and find the difference is huge. Then, we try several methods to use the detected discrepancy signal to improve the generator. However the difference becomes even bigger than before. Experimenting on two existing language GANs, the distributional discrepancy between real text and generated text increases with more adversarial learning rounds. It demonstrates both of these language GANs fail.

cs.CV↗

arXiv2019

Two-dimensional modeling of the self-limiting oxidation in silicon and tungsten nanowires

Self-limiting oxidation of nanowires has been previously described as a reaction- or diffusion-controlled process. In this letter, the concept of finite reactive region is introduced into a diffusion-controlled model, based upon which a two-dimensional cylindrical kinetics model is developed for the oxidation of silicon nanowires and is extended for tungsten. In the model, diffusivity is affected by the expansive oxidation reaction induced stress. The dependency of the oxidation upon curvature and temperature is modeled. Good agreement between the model predictions and available experimental data is obtained. The developed model serves to quantify the oxidation in two-dimensional nanostructures and is expected to facilitate their fabrication via thermal oxidation techniques. https://doi.org/10.1016/j.taml.2016.08.002

cond-mat.mtrl-sci↗

arXiv2019

On the anisotropic stable JCIR process

We investigate the anisotropic stable JCIR process which is a multi-dimensional extension of the stable JCIR process but also a multi-dimensional analogue of the classical JCIR process. We prove that the heat kernel of the anisotropic stable JCIR process exists and it satisfies an a-priori bound in a weighted anisotropic Besov norm. Based on this regularity result we deduce the strong Feller property and prove, for the subcritical case, exponential ergodicity in total variation. Also, we show that in the one-dimensional case the corresponding heat kernel is smooth.

math.PR↗

arXiv2019

Adversarial Sub-sequence for Text Generation

Generative adversarial nets (GAN) has been successfully introduced for generating text to alleviate the exposure bias. However, discriminators in these models only evaluate the entire sequence, which causes feedback sparsity and mode collapse. To tackle these problems, we propose a novel mechanism. It first segments the entire sequence into several sub-sequences. Then these sub-sequences, together with the entire sequence, are evaluated individually by the discriminator. At last these feedback signals are all used to guide the learning of GAN. This mechanism learns the generation of both the entire sequence and the sub-sequences simultaneously. Learning to generate sub-sequences is easy and is helpful in generating an entire sequence. It is easy to improve the existing GAN-based models with this mechanism. We rebuild three previous well-designed models with our mechanism, and the experimental results on benchmark data show these models are improved significantly, the best one outperforms the state-of-the-art model.\footnote[1]{All code and data are available at https://github.com/liyzcj/seggan.git

cs.LG↗

arXiv2019

Ergodicity of affine processes on the cone of symmetric positive semidefinite matrices

This article investigates the long-time behavior of conservative affine processes on the cone of symmetric positive semidefinite $d\times d$-matrices. In particular, for conservative and subcritical affine processes on this cone we show that a finite $\log$-moment of the state-independent jump measure is sufficient for the existence of a unique limit distribution. Moreover, we study the convergence rate of the underlying transition kernel to the limit distribution: firstly, in a specific metric induced by the Laplace transform and secondly, in the Wasserstein distance under a first moment assumption imposed on the state-independent jump measure and an additional condition on the diffusion parameter.

math.PR↗

arXiv2019

Stochastic equation and exponential ergodicity in Wasserstein distances for affine processes

This work is devoted to the study of conservative affine processes on the canonical state space $D = $R_+^m \times \R^n$, where $m + n > 0$. We show that each affine process can be obtained as the pathwise unique strong solution to a stochastic equation driven by Brownian motions and Poisson random measures. Then we study the long-time behavior of affine processes, i.e., we show that under first moment condition on the state-dependent and log-moment conditions on the state-independent jump measures, respectively, each subcritical affine process is exponentially ergodic in a suitably chosen Wasserstein distance. Moments of affine processes are studied as well.

math.PR↗

arXiv2019

Note on the diversity of intersecting families

Let $n=2k+1$ be odd with $k\geq3$. In this note, we give two intersecting families with diversity larger than $\sum_{i=k+1}^{2k} \binom {2k}{i}$, which disprove a conjecture of Huang.

math.CO↗

arXiv2019

Exponential ergodicity for stochastic equations of nonnegative processes with jumps

In this work, we study ergodicity of continuous time Markov processes on state space $\mathbb{R}_{\geq 0} := [0,\infty)$ obtained as unique strong solutions to stochastic equations with jumps. Our first main result establishes exponential ergodicity in the Wasserstein distance, provided the stochastic equation satisfies a comparison principle and the drift is dissipative. In particular, it is applicable to continuous-state branching processes with immigration (shorted as CBI processes), possibly with nonlinear branching mechanisms or in Lévy random environments. Our second main result establishes exponential ergodicity in total variation distance for subcritical CBI processes under a first moment condition on the jump measure for branching and a $\log$-moment condition on the jump measure for immigration.

math.PR↗

arXiv2019

Boundary behavior of multi-type continuous-state branching processes with immigration

In this article we provide a sufficient condition for a continuous-state branching process with immigration (CBI process) to not hit its boundary, i.e. for non-extinction. Our result applies to arbitrary dimension $d \geq 1$ and is formulated in terms of an integrability condition for its immigration and branching mechanisms $F$ and $R$. The proof is based on a suitable comparison with one-dimensional CBI processes and an existing result for one-dimensional CBI processes. The same technique is also used to provide a sufficient condition for transience of multi-type CBI processes.

math.PR↗

arXiv2018

Existence of limiting distribution for affine processes

In this paper, sufficient conditions are given for the existence of limiting distribution of a conservative affine process on the canonical state space $\mathbb{R}_{\geqslant0}^{m}\times\mathbb{R}^{n}$, where $m,\thinspace n\in\mathbb{Z}_{\geqslant0}$ with $m+n>0$. Our main theorem extends and unifies some known results for OU-type processes on $\mathbb{R}^{n}$ and one-dimensional CBI processes (with state space $\mathbb{R}_{\geqslant0}$). To prove our result, we combine analytical and probabilistic techniques; in particular, the stability theory for ODEs plays an important role.

math.PR↗

arXiv2018

Restricted Boltzmann Machines with Gaussian Visible Units Guided by Pairwise Constraints

Restricted Boltzmann machines (RBMs) and their variants are usually trained by contrastive divergence (CD) learning, but the training procedure is an unsupervised learning approach, without any guidances of the background knowledge. To enhance the expression ability of traditional RBMs, in this paper, we propose pairwise constraints restricted Boltzmann machine with Gaussian visible units (pcGRBM) model, in which the learning procedure is guided by pairwise constraints and the process of encoding is conducted under these guidances. The pairwise constraints are encoded in hidden layer features of pcGRBM. Then, some pairwise hidden features of pcGRBM flock together and another part of them are separated by the guidances. In order to deal with real-valued data, the binary visible units are replaced by linear units with Gausian noise in the pcGRBM model. In the learning process of pcGRBM, the pairwise constraints are iterated transitions between visible and hidden units during CD learning procedure. Then, the proposed model is inferred by approximative gradient descent method and the corresponding learning algorithm is designed in this paper. In order to compare the availability of pcGRBM and traditional RBMs with Gaussian visible units, the features of the pcGRBM and RBMs hidden layer are used as input 'data' for K-means, spectral clustering (SP) and affinity propagation (AP) algorithms, respectively. A thorough experimental evaluation is performed with sixteen image datasets of Microsoft Research Asia Multimedia (MSRA-MM). The experimental results show that the clustering performance of K-means, SP and AP algorithms based on pcGRBM model are significantly better than traditional RBMs. In addition, the pcGRBM model for clustering task shows better performance than some semi-supervised clustering algorithms.

cs.LG↗

arXiv2018

Existence of densities for stochastic differential equations driven by Lévy processes with anisotropic jumps

We study existence of densities for solutions to stochastic differential equations with Hölder continuous coefficients and driven by a $d$-dimensional Lévy process $Z=(Z_{t})_{t\geq 0}$, where, for $t>0$, the density function $f_{t}$ of $Z_{t}$ exists and satisfies, for some $(α_{i})_{i=1,\dots,d}\subset(0,2)$ and $C>0$, \begin{align*} \limsup\limits _{t \to 0}t^{1/α_{i}}\int\limits _{\mathbb{R}^{d}}|f_{t}(z+e_{i}h)-f_{t}(z)|dz\leq C|h|,\ \ h\in \mathbb{R},\ \ i=1,\dots,d. \end{align*} Here $e_{1},\dots,e_{d}$ denote the canonical basis vectors in $\mathbb{R}^{d}$. The latter condition covers anisotropic $(α_{1},\dots,α_{d})$-stable laws but also particular cases of subordinate Brownian motion. To prove our result we use some ideas taken from \citep{DF13}.

math.PR↗

arXiv2018

Existence of densities for multi-type CBI processes

Let X be a multi-type continuous-state branching process with immigration (CBI process) on state space $\mathbb{R}^d$. Denote by $g_t$, $t \geq 0$, the law of $X_{t}$. We provide sufficient conditions under which $g_t$ has, for each $t > 0$, a density with respect to the Lebesgue measure. Such density has, by construction, some anisotropic Besov regularity. Our approach neither relies on the use of Malliavin calculus nor on the study of corresponding Laplace transform.

math.PR↗