arXiv subjects

Yao Li

Publications and source records attributed to Yao Li.

At least 199 records · Page 11Linked to original sources

arXiv2019

Entropy, mutual information, and systematic measures of structured spiking neural networks

The aim of this paper is to investigate various information-theoretic measures, including entropy, mutual information, and some systematic measures that based on mutual information, for a class of structured spiking neuronal network. In order to analyze and compute these information-theoretic measures for large networks, we coarse-grained the data by ignoring the order of spikes that fall into the same small time bin. The resultant coarse-grained entropy mainly capture the information contained in the rhythm produced by a local population of the network. We first proved that these information theoretical measures are well-defined and computable by proving the stochastic stability and the law of large numbers. Then we use three neuronal network examples, from simple to complex, to investigate these information-theoretic measures. Several analytical and computational results about properties of these information-theoretic measures are given.

q-bio.NC↗

arXiv2019

Uncertainty Quantification for High Dimensional Sparse Nonparametric Additive Models

Statistical inference in high dimensional settings has recently attracted enormous attention within the literature. However, most published work focuses on the parametric linear regression problem. This paper considers an important extension of this problem: statistical inference for high dimensional sparse nonparametric additive models. To be more precise, this paper develops a methodology for constructing a probability density function on the set of all candidate models. This methodology can also be applied to construct confidence intervals for various quantities of interest (such as noise variance) and confidence bands for the additive functions. This methodology is derived using a generalized fiducial inference framework. It is shown that results produced by the proposed methodology enjoy correct asymptotic frequentist properties. Empirical results obtained from numerical experimentation verify this theoretical claim. Lastly, the methodology is applied to a gene expression data set and discovers new findings for which most existing methods based on parametric linear modeling failed to observe.

stat.ME↗

arXiv2019

Improve variational autoEncoder with auxiliary softmax multiclassifier

As a general-purpose generative model architecture, VAE has been widely used in the field of image and natural language processing. VAE maps high dimensional sample data into continuous latent variables with unsupervised learning. Sampling in the latent variable space of the feature, VAE can construct new image or text data. As a general-purpose generation model, the vanilla VAE can not fit well with various data sets and neural networks with different structures. Because of the need to balance the accuracy of reconstruction and the convenience of latent variable sampling in the training process, VAE often has problems known as "posterior collapse". images reconstructed by VAE are also often blurred. In this paper, we analyze the main cause of these problem, which is the lack of mutual information between the sample variable and the latent feature variable during the training process. To maintain mutual information in model training, we propose to use the auxiliary softmax multi-classification network structure to improve the training effect of VAE, named VAE-AS. We use MNIST and Omniglot data sets to test the VAE-AS model. Based on the test results, It can be show that VAE-AS has obvious effects on the mutual information adjusting and solving the posterior collapse problem.

stat.ML↗

arXiv2019

Embracing a mechanized formalization gap

If a code base is so big and complicated that complete mechanical verification is intractable, can we still apply and benefit from verification methods? We show that by allowing a deliberate mechanized formalization gap we can shrink and simplify the model until it is manageable, while still retaining a meaningful, declaratively documented connection to the original, unmodified source code. Concretely, we translate core parts of the Haskell compiler GHC into Coq, using hs-to-coq, and verify invariants related to the use of term variables.

cs.PL↗

arXiv2019

A Double Residual Compression Algorithm for Efficient Distributed Learning

Large-scale machine learning models are often trained by parallel stochastic gradient descent algorithms. However, the communication cost of gradient aggregation and model synchronization between the master and worker nodes becomes the major obstacle for efficient learning as the number of workers and the dimension of the model increase. In this paper, we propose DORE, a DOuble REsidual compression stochastic gradient descent algorithm, to reduce over $95\%$ of the overall communication such that the obstacle can be immensely mitigated. Our theoretical analyses demonstrate that the proposed strategy has superior convergence properties for both strongly convex and nonconvex objective functions. The experimental results validate that DORE achieves the best communication efficiency while maintaining similar model accuracy and convergence speed in comparison with start-of-the-art baselines.

cs.LG↗

arXiv2019

Shape Transformations of Vesicles induced by Swim Pressure

While the behavior of vesicles in thermodynamic equilibrium has been studied extensively, how active forces control vesicle shape transformations is not understood. Here, we combine theory and simulations to study the shape behavior of vesicles containing active Brownian particles. We show that the combination of active forces, dimensionality and membrane bending free energy creates a plethora of novel phase transitions. At low swim pressure, the vesicle exhibits a discontinuous transition from a spherical to a prolate shape, which has no counterpart in two dimensions. At high swim pressure it exhibits stochastic spatio-temporal oscillations. Our work helps to understand and control the shape dynamics of membranes in active-matter systems.

cond-mat.soft↗

arXiv2019

An efficient data-driven solver for Fokker-Planck equations: algorithm and analysis

Computing the invariant probability measure of a randomly perturbed dynamical system usually means solving the stationary Fokker-Planck equation. This paper studies several key properties of a novel data-driven solver for low-dimensional Fokker-Planck equations proposed in [15]. Based on these results, we propose a new `block solver' for the stationary Fokker-Planck equation, which significantly improves the performance of the original algorithm. Some possible ways of reducing numerical artifacts caused by the block solver are discussed and tested with examples.

math.NA↗

arXiv2019

Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network

We present a new algorithm to train a robust neural network against adversarial attacks. Our algorithm is motivated by the following two ideas. First, although recent work has demonstrated that fusing randomness can improve the robustness of neural networks (Liu 2017), we noticed that adding noise blindly to all the layers is not the optimal way to incorporate randomness. Instead, we model randomness under the framework of Bayesian Neural Network (BNN) to formally learn the posterior distribution of models in a scalable way. Second, we formulate the mini-max problem in BNN to learn the best model distribution under adversarial attacks, leading to an adversarial-trained Bayesian neural net. Experiment results demonstrate that the proposed algorithm achieves state-of-the-art performance under strong attacks. On CIFAR-10 with VGG network, our model leads to 14\% accuracy improvement compared with adversarial training (Madry 2017) and random self-ensemble (Liu 2017) under PGD attack with $0.035$ distortion, and the gap becomes even larger on a subset of ImageNet.

cs.LG↗

arXiv2019

A data-driven method for the steady state of randomly perturbed dynamics

We demonstrate a data-driven method to solve for the invariant probability density function of a randomly perturbed dynamical system. The key idea is to replace the boundary condition of numerical schemes by a least squares problem corresponding to a reference solution, which is generated by Monte Carlo simulation. With this method we can solve for the invariant probability density function in any local area with high accuracy, regardless of whether the attractor is covered by the numerical domain.

math.NA↗

arXiv2019

Thermal conductivity and local thermodynamic equilibrium of stochastic energy exchange models

In this paper we study macroscopic thermodynamic properties of a stochastic microscopic heat conduction model that is reduced from deterministic problems. Our goal is to numerically check how the `low energy site effect' inherited from the deterministic model would affect the macroscopic thermodynamic properties such as the thermal conductivity and the local thermodynamic equilibrium. After a series of numerical computations, our conclusion is that neither the thermal conductivity nor the existence of local thermodynamic equilibrium is qualitatively changed by this effect.

math-ph↗

arXiv2018

Optimal Transport Classifier: Defending Against Adversarial Attacks by Regularized Deep Embedding

Recent studies have demonstrated the vulnerability of deep convolutional neural networks against adversarial examples. Inspired by the observation that the intrinsic dimension of image data is much smaller than its pixel space dimension and the vulnerability of neural networks grows with the input dimension, we propose to embed high-dimensional input images into a low-dimensional space to perform classification. However, arbitrarily projecting the input images to a low-dimensional space without regularization will not improve the robustness of deep neural networks. Leveraging optimal transport theory, we propose a new framework, Optimal Transport Classifier (OT-Classifier), and derive an objective that minimizes the discrepancy between the distribution of the true label and the distribution of the OT-Classifier output. Experimental results on several benchmark datasets show that, our proposed framework achieves state-of-the-art performance against strong adversarial attack methods.

cs.CV↗

arXiv2018

From C to Interaction Trees: Specifying, Verifying, and Testing a Networked Server

We present the first formal verification of a networked server implemented in C. Interaction trees, a general structure for representing reactive computations, are used to tie together disparate verification and testing tools (Coq, VST, and QuickChick) and to axiomatize the behavior of the operating system on which the server runs (CertiKOS). The main theorem connects a specification of acceptable server behaviors, written in a straightforward "one client at a time" style, with the CompCert semantics of the C program. The variability introduced by low-level buffering of messages and interleaving of multiple TCP connections is captured using network refinement, a variant of observational refinement.

cs.PL↗

arXiv2018

Calibration of the DAMPE Plastic Scintillator Detector and its on-orbit performance

DArk Matter Particle Explorer (DAMPE) is a space-borne apparatus for detecting the high-energy cosmic-rays like electrons, $γ$-rays, protons and heavy-ions. Plastic Scintillator Detector (PSD) is the top-most sub-detector of the DAMPE. The PSD is designed to measure the charge of incident high-energy particles and it also serves as a veto detector for discriminating $γ$-rays from charged particles. In this paper, PSD on-orbit calibration procedure is described, which includes five steps of pedestal, dynode correlation, response to minimum-ionizing particles (MIPs), light attenuation function and energy reconstruction. A method for reconstructing the charge of incident high energy cosmic-ray particles is introduced. The detection efficiency of each PSD strip is verified to be above 99.5%, the total efficiency of the PSD for charged particles is above 99.99%.

astro-ph.IM↗

arXiv2018

Image Super-Resolution Using VDSR-ResNeXt and SRCGAN

Over the past decade, many Super Resolution techniques have been developed using deep learning. Among those, generative adversarial networks (GAN) and very deep convolutional networks (VDSR) have shown promising results in terms of HR image quality and computational speed. In this paper, we propose two approaches based on these two algorithms: VDSR-ResNeXt, which is a deep multi-branch convolutional network inspired by VDSR and ResNeXt; and SRCGAN, which is a conditional GAN that explicitly passes class labels as input to the GAN. The two methods were implemented on common SR benchmark datasets for both quantitative and qualitative assessment.

cs.CV↗

arXiv2018

Adversarial 3D Human Pose Estimation via Multimodal Depth Supervision

In this paper, a novel deep-learning based framework is proposed to infer 3D human poses from a single image. Specifically, a two-phase approach is developed. We firstly utilize a generator with two branches for the extraction of explicit and implicit depth information respectively. During the training process, an adversarial scheme is also employed to further improve the performance. The implicit and explicit depth information with the estimated 2D joints generated by a widely used estimator, in the second step, are together fed into a deep 3D pose regressor for the final pose generation. Our method achieves MPJPE of 58.68mm on the ECCV2018 3D Human Pose Estimation Challenge.

cs.CV↗

arXiv2018

From billiards to thermodynamic laws: I. Stochastic energy exchange model

This paper studies a billiards-like microscopic heat conduction model, which describes the dynamics of gas molecules in a long tube with thermalized boundary. We numerically investigate the law of energy exchange between adjacent cells. A stochastic energy exchange model that preserves these properties is then derived. We further numerically justified that the stochastic energy exchange model preserves the ergodicity and the thermal conductivity of the original billiard model.

math.DS↗

arXiv2018

A Method of Alignment of the Plastic Scintillator Detector of DAMPE

The Plastic Scintillator Detector (PSD) of the DArk Matter Particle Explorer (DAMPE) is designed to measure cosmic ray charge (Z) and to act as a veto detector for gamma-ray identification. In order to fully exploit the charge identification potential of the PSD and to enhance its capability to identify the gamma ray events, we develop a PSD detector alignment method. The path length of a given track in the volume of a PSD bar is derived taking into account the shift and rotation alignment corrections. By examining energy spectra of corner-passing events and fully contained events, position shifts and rotations of all PSD bars are obtained, and found to be on average about 1mm and 0.0015 radian respectively. To validate the alignment method, we introduce the artificial shifts and rotations of PSD bars in the detector simulation. These shift and rotation parameters can be recovered successfully by the alignment procedure. As a result of the PSD alignment procedure, the charge resolution of the PSD is improved from $4\%$ to $8\%$ depending on the nuclei.

astro-ph.IM↗

arXiv2018

Firing rate and spatial correlation in a stochastic neural field model

This paper studies a stochastic neural field model that is extended from our previous paper [14]. The neural field model consists of many heterogeneous local populations of neurons. Rigorous results on the stochastic stability are proved, which further imply the well-definedness of quantities including mean firing rate and spike count correlation. Then we devote to address two main topics: the comparison with mean-field approximations and the spatial correlation of spike count. We showed that partial synchronization of spiking activities is a main cause for discrepancies of mean-field approximations. Furthermore, the spike count correlation between local populations are studied. We find that the spike count correlation decays quickly with the distance between corresponding local populations. Some mathematical justifications of the mechanism of this phenomenon is also provided.

math.PR↗