arXiv subjects

Feng Liang

Publications and source records attributed to Feng Liang.

83 records · Page 5Linked to original sources

arXiv2017

Heat engine in the three-dimensional spacetime

We define a kind of heat engine via three-dimensional charged BTZ black holes. This case is quite subtle and needs to be more careful. The heat flow along the isochores does not equal to zero since the specific heat $C_V\neq0$ and this point completely differs from the cases discussed before whose isochores and adiabats are identical. So one cannot simply apply the paradigm in the former literatures. However, if one introduces a new thermodynamic parameter associated with the renormalization length scale, the above problem can be solved. We obtain the analytical efficiency expression of the three-dimensional charged BTZ black hole heat engine for two different schemes. Moreover, we double check with the exact formula. Our result presents the first specific example for the sound correctness of the exact efficiency formula. We argue that the three-dimensional charged BTZ black hole can be viewed as a toy model for further investigation of holographic heat engine. Furthermore, we compare our result with that of the Carnot cycle and extend the former result to three-dimensional spacetime. In this sense, the result in this paper would be complementary to those obtained in four-dimensional spacetime or ever higher. Last but not the least, the heat engine efficiency discussed in this paper may serve as a criterion to discriminate the two thermodynamic approaches introduced in Ref.[29] and our result seems to support the approach which introduces a new thermodynamic parameter $R=r_0$.

gr-qc↗

arXiv2017

An Empirical Bayes Approach for High Dimensional Classification

We propose an empirical Bayes estimator based on Dirichlet process mixture model for estimating the sparse normalized mean difference, which could be directly applied to the high dimensional linear classification. In theory, we build a bridge to connect the estimation error of the mean difference and the misclassification error, also provide sufficient conditions of sub-optimal classifiers and optimal classifiers. In implementation, a variational Bayes algorithm is developed to compute the posterior efficiently and could be parallelized to deal with the ultra-high dimensional case.

stat.ML↗

arXiv2016

An Ensemble EM Algorithm for Bayesian Variable Selection

We study the Bayesian approach to variable selection in the context of linear regression. Motivated by a recent work by Rockova and George (2014), we propose an EM algorithm that returns the MAP estimate of the set of relevant variables. Due to its particular updating scheme, our algorithm can be implemented efficiently without inverting a large matrix in each iteration and therefore can scale up with big data. We also show that the MAP estimate returned by our EM algorithm achieves variable selection consistency even when $p$ diverges with $n$. In practice, our algorithm could get stuck with local modes, a common problem with EM algorithms. To address this issue, we propose an ensemble EM algorithm, in which we repeatedly apply the EM algorithm on a subset of the samples with a subset of the covariates, and then aggregate the variable selection results across those bootstrap replicates. Empirical studies have demonstrated the superior performance of the ensemble EM algorithm.

stat.CO↗

arXiv2016

A Variational Algorithm for Bayesian Variable Selection

There has been an intense development on the estimation of a sparse regression coefficient vector in statistics, machine learning and related fields. In this paper, we focus on the Bayesian approach to this problem, where sparsity is incorporated by the so-called spike-and-slab prior on the coefficients. Instead of replying on MCMC for posterior inference, we propose a fast and scalable algorithm based on variational approximation to the posterior distribution. The updating scheme employed by our algorithm is different from the one proposed by Carbonetto and Stephens (2012). Those changes seem crucial for us to show that our algorithm can achieve asymptotic consistency even when the feature dimension diverges exponentially fast with the sample size. Empirical results have demonstrated the effectiveness and efficiency of the proposed algorithm.

stat.CO↗

arXiv2015

Clustering With Side Information: From a Probabilistic Model to a Deterministic Algorithm

In this paper, we propose a model-based clustering method (TVClust) that robustly incorporates noisy side information as soft-constraints and aims to seek a consensus between side information and the observed data. Our method is based on a nonparametric Bayesian hierarchical model that combines the probabilistic model for the data instance and the one for the side-information. An efficient Gibbs sampling algorithm is proposed for posterior inference. Using the small-variance asymptotics of our probabilistic model, we then derive a new deterministic clustering algorithm (RDP-means). It can be viewed as an extension of K-means that allows for the inclusion of side information and has the additional property that the number of clusters does not need to be specified a priori. Empirical studies have been carried out to compare our work with many constrained clustering algorithms from the literature on both a variety of data sets and under a variety of conditions such as using noisy side information and erroneous k values. The results of our experiments show strong results for our probabilistic and deterministic approaches under these conditions when compared to other algorithms in the literature.

stat.ML↗

arXiv2014

PAC-Bayesian AUC classification and scoring

We develop a scoring and classification procedure based on the PAC-Bayesian approach and the AUC (Area Under Curve) criterion. We focus initially on the class of linear score functions. We derive PAC-Bayesian non-asymptotic bounds for two types of prior for the score parameters: a Gaussian prior, and a spike-and-slab prior; the latter makes it possible to perform feature selection. One important advantage of our approach is that it is amenable to powerful Bayesian computational tools. We derive in particular a Sequential Monte Carlo algorithm, as an efficient method which may be used as a gold standard, and an Expectation-Propagation algorithm, as a much faster but approximate method. We also extend our method to a class of non-linear score functions, essentially leading to a nonparametric procedure, by considering a Gaussian process prior.

stat.ML↗

arXiv2013

Heteroscedastic Relevance Vector Machine

In this work we propose a heteroscedastic generalization to RVM, a fast Bayesian framework for regression, based on some recent similar works. We use variational approximation and expectation propagation to tackle the problem. The work is still under progress and we are examining the results and comparing with the previous works.

stat.ML↗

arXiv2012

From Minimax Shrinkage Estimation to Minimax Shrinkage Prediction

In a remarkable series of papers beginning in 1956, Charles Stein set the stage for the future development of minimax shrinkage estimators of a multivariate normal mean under quadratic loss. More recently, parallel developments have seen the emergence of minimax shrinkage estimators of multivariate normal predictive densities under Kullback--Leibler risk. We here describe these parallels emphasizing the focus on Bayes procedures and the derivation of the superharmonic conditions for minimaxity as well as further developments of new minimax shrinkage predictive density estimators including multiple shrinkage estimators, empirical Bayes estimators, normal linear model regression estimators and nonparametric regression estimators.

stat.ME↗

arXiv2010

Asymptotic minimax risk of predictive density estimation for non-parametric regression

We consider the problem of estimating the predictive density of future observations from a non-parametric regression model. The density estimators are evaluated under Kullback--Leibler divergence and our focus is on establishing the exact asymptotics of minimax risk in the case of Gaussian errors. We derive the convergence rate and constant for minimax risk among Bayesian predictive densities under Gaussian priors and we show that this minimax risk is asymptotically equivalent to that among all density estimators.

math.ST↗

arXiv2007

The Use of Unlabeled Data in Predictive Modeling

The incorporation of unlabeled data in regression and classification analysis is an increasing focus of the applied statistics and machine learning literatures, with a number of recent examples demonstrating the potential for unlabeled data to contribute to improved predictive accuracy. The statistical basis for this semisupervised analysis does not appear to have been well delineated; as a result, the underlying theory and rationale may be underappreciated, especially by nonstatisticians. There is also room for statisticians to become more fully engaged in the vigorous research in this important area of intersection of the statistical and computer sciences. Much of the theoretical work in the literature has focused, for example, on geometric and structural properties of the unlabeled data in the context of particular algorithms, rather than probabilistic and statistical questions. This paper overviews the fundamental statistical foundations for predictive modeling and the general questions associated with unlabeled data, highlighting the relevance of venerable concepts of sampling design and prior specification. This theory, illustrated with a series of central illustrative examples and two substantial real data analyses, shows precisely when, why and how unlabeled data matter.

stat.ME↗

arXiv2006

Improved minimax predictive densities under Kullback--Leibler loss

Let $X| μ\sim N_p(μ,v_xI)$ and $Y| μ\sim N_p(μ,v_yI)$ be independent p-dimensional multivariate normal vectors with common unknown mean $μ$. Based on only observing $X=x$, we consider the problem of obtaining a predictive density $\hat{p}(y| x)$ for $Y$ that is close to $p(y| μ)$ as measured by expected Kullback--Leibler loss. A natural procedure for this problem is the (formal) Bayes predictive density $\hat{p}_{\mathrm{U}}(y| x)$ under the uniform prior $π_{\mathrm{U}}(μ)\equiv 1$, which is best invariant and minimax. We show that any Bayes predictive density will be minimax if it is obtained by a prior yielding a marginal that is superharmonic or whose square root is superharmonic. This yields wide classes of minimax procedures that dominate $\hat{p}_{\mathrm{U}}(y| x)$, including Bayes predictive densities under superharmonic priors. Fundamental similarities and differences with the parallel theory of estimating a multivariate normal mean under quadratic loss are described.

math.ST↗