arXiv subjects

Heng Lian

Publications and source records attributed to Heng Lian.

At least 55 records · Page 3Linked to original sources

arXiv2010

Gaussian Process Models for Nonparametric Functional Regression with Functional Responses

Recently nonparametric functional model with functional responses has been proposed within the functional reproducing kernel Hilbert spaces (fRKHS) framework. Motivated by its superior performance and also its limitations, we propose a Gaussian process model whose posterior mode coincide with the fRKHS estimator. The Bayesian approach has several advantages compared to its predecessor. Firstly, the multiple unknown parameters can be inferred together with the regression function in a unified framework. Secondly, as a Bayesian method, the statistical inferences are straightforward through the posterior distributions. We also use the predictive process models adapted from the spatial statistics literature to overcome the computational limitations, thus extending the applicability of this popular technique to a new problem. Modifications of predictive process models are nevertheless critical in our context to obtain valid inferences. The numerical results presented demonstrate the effectiveness of the modifications.

stat.ME↗

arXiv2010

A simple and efficient algorithm for fused lasso signal approximator with convex loss function

We consider the augmented Lagrangian method (ALM) as a solver for the fused lasso signal approximator (FLSA) problem. The ALM is a dual method in which squares of the constraint functions are added as penalties to the Lagrangian. In order to apply this method to FLSA, two types of auxiliary variables are introduced to transform the original unconstrained minimization problem into a linearly constrained minimization problem. Each updating in this iterative algorithm consists of just a simple one-dimensional convex programming problem, with closed form solution in many cases. While the existing literature mostly focused on the quadratic loss function, our algorithm can be easily implemented for general convex loss. The most attractive feature of this algorithm is its simplicity in implementation compared to other existing fast solvers. We also provide some convergence analysis of the algorithm. Finally, the method is illustrated with some simulation datasets.

stat.CO↗

arXiv2009

Time-varying Coefficients Estimation in Differential Equation Models with Noisy Time-varying Covariates

We study the problem of estimating time-varying coefficients in ordinary differential equations. Current theory only applies to the case when the associated state variables are observed without measurement errors as presented in \cite{chenwu08b,chenwu08}. The difficulty arises from the quadratic functional of observations that one needs to deal with instead of the linear functional that appears when state variables contain no measurement errors. We derive the asymptotic bias and variance for the previously proposed two-step estimators using quadratic regression functional theory.

math.ST↗

arXiv2009

Shrinkage Tuning Parameter Selection in Precision Matrices Estimation

Recent literature provides many computational and modeling approaches for covariance matrices estimation in a penalized Gaussian graphical models but relatively little study has been carried out on the choice of the tuning parameter. This paper tries to fill this gap by focusing on the problem of shrinkage parameter selection when estimating sparse precision matrices using the penalized likelihood approach. Previous approaches typically used K-fold cross-validation in this regard. In this paper, we first derived the generalized approximate cross-validation for tuning parameter selection which is not only a more computationally efficient alternative, but also achieves smaller error rate for model fitting compared to leave-one-out cross-validation. For consistency in the selection of nonzero entries in the precision matrix, we employ a Bayesian information criterion which provably can identify the nonzero conditional correlations in the Gaussian model. Our simulations demonstrate the general superiority of the two proposed selectors in comparison with leave-one-out cross-validation, ten-fold cross-validation and Akaike information criterion.

stat.ME↗

arXiv2009

Total Variation, Adaptive Total Variation and Nonconvex Smoothly Clipped Absolute Deviation Penalty for Denoising Blocky Images

The total variation-based image denoising model has been generalized and extended in numerous ways, improving its performance in different contexts. We propose a new penalty function motivated by the recent progress in the statistical literature on high-dimensional variable selection. Using a particular instantiation of the majorization-minimization algorithm, the optimization problem can be efficiently solved and the computational procedure realized is similar to the spatially adaptive total variation model. Our two-pixel image model shows theoretically that the new penalty function solves the bias problem inherent in the total variation model. The superior performance of the new penalty is demonstrated through several experiments. Our investigation is limited to "blocky" images which have small total variation.

cs.CV↗

arXiv2009

Sparse Bayesian Hierarchical Modeling of High-dimensional Clustering Problems

Clustering is one of the most widely used procedures in the analysis of microarray data, for example with the goal of discovering cancer subtypes based on observed heterogeneity of genetic marks between different tissues. It is well-known that in such high-dimensional settings, the existence of many noise variables can overwhelm the few signals embedded in the high-dimensional space. We propose a novel Bayesian approach based on Dirichlet process with a sparsity prior that simultaneous performs variable selection and clustering, and also discover variables that only distinguish a subset of the cluster components. Unlike previous Bayesian formulations, we use Dirichlet process (DP) for both clustering of samples as well as for regularizing the high-dimensional mean/variance structure. To solve the computational challenge brought by this double usage of DP, we propose to make use of a sequential sampling scheme embedded within Markov chain Monte Carlo (MCMC) updates to improve the naive implementation of existing algorithms for DP mixture models. Our method is demonstrated on a simulation study and illustrated with the leukemia gene expression dataset.

stat.ME↗

arXiv2009

Empirical Likelihood Confidence Intervals for Nonparametric Functional Data Analysis

We consider the problem of constructing confidence intervals for nonparametric functional data analysis using empirical likelihood. In this doubly infinite-dimensional context, we demonstrate the Wilks's phenomenon and propose a bias-corrected construction that requires neither undersmoothing nor direct bias estimation. We also extend our results to partially linear regression involving functional data. Our numerical results demonstrated the improved performance of empirical likelihood over approximation based on asymptotic normality.

stat.ME↗

arXiv2008

Nonparametric Estimation of Variance Function for Functional Data

This article investigates nonparametric estimation of variance functions for functional data when the mean function is unknown. We obtain asymptotic results for the kernel estimator based on squared residuals. Similar to the finite dimensional case, our asymptotic result shows the smoothness of the unknown mean function has an effect on the rate of convergence. Our simulaton studies demonstrate that estimator based on residuals performs much better than that based on conditional second moment of the responses.

stat.ME↗

arXiv2008

A note on conditional Akaike information for Poisson regression with random effects

A popular model selection approach for generalized linear mixed-effects models is the Akaike information criterion, or AIC. Among others, \cite{vaida05} pointed out the distinction between the marginal and conditional inference depending on the focus of research. The conditional AIC was derived for the linear mixed-effects model which was later generalized by \cite{liang08}. We show that the similar strategy extends to Poisson regression with random effects, where condition AIC can be obtained based on our observations. Simulation studies demonstrate the usage of the criterion.

stat.ME↗

arXiv2008

Posterior Convergence and Model Estimation in Bayesian Change-point Problems

We study the posterior distribution of the Bayesian multiple change-point regression problem when the number and the locations of the change-points are unknown. While it is relatively easy to apply the general theory to obtain the $O(1/\sqrt{n})$ rate up to some logarithmic factor, showing the exact parametric rate of convergence of the posterior distribution requires additional work and assumptions. Additionally, we demonstrate the asymptotic normality of the segment levels under these assumptions. For inferences on the number of change-points, we show that the Bayesian approach can produce a consistent posterior estimate. Finally, we argue that the point-wise posterior convergence property as demonstrated might have bad finite sample performance in that consistent posterior for model selection necessarily implies the maximal squared risk will be asymptotically larger than the optimal $O(1/\sqrt{n})$ rate. This is the Bayesian version of the same phenomenon that has been noted and studied by other authors.

math.ST↗

arXiv2008

Parameter estimation of high-dimensional linear differential equations

We study the problem of estimating the coefficients in linear ordinary differential equations (ODE's) with a diverging number of variables when the solutions are observed with noise. The solution trajectories are first smoothed with local polynomial regression and the coefficients are estimated with nonconcave penalty proposed by \cite{fan01}. Under some regularity and sparsity conditions, we show the procedure can correctly identifies nonzero coefficients with probability converging to one and the estimators for nonzero coefficients have the same asymptotic normal distribution as they would have when the zero coefficients are known and the same two-step procedure is used. Our asymptotic results are valid under the misspecified case where linear ODE's are only used as an approximation to nonlinear ODE's, and the estimates will converge to the coefficients of the best approximating linear system. From our results, when the solution trajectories of the ODE's are sufficiently smooth, the parametric $\sqrt{n}$ rate is achieved even though nonparametric regression estimator is used in the first step of the procedure. The performance of the two-step procedure is illustrated by a simulation study as well as an application to yeast cell-cycle data.

math.ST↗

arXiv2008

Inference of genetic networks from time course expression data using functional regression with lasso penalty

Statistical inference of genetic regulatory networks is essential for understanding temporal interactions of regulatory elements inside the cells. For inferences of large networks, identification of network structure is typical achieved under the assumption of sparsity of the networks. When the number of time points in the expression experiment is not too small, we propose to infer the parameters in the ordinary differential equations using the techniques from functional data analysis (FDA) by regarding the observed time course expression data as continuous-time curves. For networks with a large number of genes, we take advantage of the sparsity of the networks by penalizing the linear coefficients with a L_1 norm. The ability of the algorithm to infer network structure is demonstrated using the cell-cycle time course data for Saccharomyces cerevisiae.

q-bio.QM↗

arXiv2008

Bayesian Nonlinear Principal Component Analysis Using Random Fields

We propose a novel model for nonlinear dimension reduction motivated by the probabilistic formulation of principal component analysis. Nonlinearity is achieved by specifying different transformation matrices at different locations of the latent space and smoothing the transformation using a Markov random field type prior. The computation is made feasible by the recent advances in sampling from von Mises-Fisher distributions.

cs.CV↗

arXiv2007

Stochastic adaptation of importance sampler

Improving efficiency of importance sampler is at the center of research in Monte Carlo methods. While adaptive approach is usually difficult within the Markov Chain Monte Carlo framework, the counterpart in importance sampling can be justified and validated easily. We propose an iterative adaptation method for learning the proposal distribution of an importance sampler based on stochastic approximation. The stochastic approximation method can recruit general iterative optimization techniques like the minorization-maximization algorithm. The effectiveness of the approach in optimizing the Kullback divergence between the proposal distribution and the target is demonstrated using several simple examples.

stat.ME↗

arXiv2007

On posterior distribution of Bayesian wavelet thresholding

We investigate the posterior rate of convergence for wavelet shrinkage using a Bayesian approach in general Besov spaces. Instead of studying the Bayesian estimator related to a particular loss function, we focus on the posterior distribution itself from a nonparametric Bayesian asymptotics point of view and study its rate of convergence. We obtain the same rate as in \citet{abramovich04} where the authors studied the convergence of several Bayesian estimators.

math.ST↗

arXiv2007

Variational local structure estimation for image super-resolution

Super-resolution is an important but difficult problem in image/video processing. If a video sequence or some training set other than the given low-resolution image is available, this kind of extra information can greatly aid in the reconstruction of the high-resolution image. The problem is substantially more difficult with only a single low-resolution image on hand. The image reconstruction methods designed primarily for denoising is insufficient for super-resolution problem in the sense that it tends to oversmooth images with essentially no noise. We propose a new adaptive linear interpolation method based on variational method and inspired by local linear embedding (LLE). The experimental result shows that our method avoids the problem of oversmoothing and preserves image structures well.

cs.CV↗

arXiv2007

MOST: detecting cancer differential gene expression

We propose a new statistics for the detection of differentially expressed genes, when the genes are activated only in a subset of the samples. Statistics designed for this unconventional circumstance has proved to be valuable for most cancer studies, where oncogenes are activated for a small number of disease samples. Previous efforts made in this direction include COPA, OS and ORT. We propose a new statistics called maximum ordered subset t-statistics (MOST) which seems to be natural when the number of activated samples is unknown. We compare MOST to other statistics and find the proposed method often has more power then its competitors.

stat.AP↗

arXiv2007

Bayes and empirical Bayes changepoint problems

We generalize the approach of Liu and Lawrence (1999) for multiple changepoint problems where the number of changepoints is unknown. The approach is based on dynamic programming recursion for efficient calculation of the marginal probability of the data with the hidden parameters integrated out. For the estimation of the hyperparameters, we propose to use Monte Carlo EM when training data are available. We argue that there is some advantages of using samples from the posterior which takes into account the uncertainty of the changepoints, compared to the traditional MAP estimator, which is also more expensive to compute in this context. The samples from the posterior obtained by our algorithm are independent, getting rid of the convergence issue associated with the MCMC approach. We illustrate our approach on limited simulations and some real data set.

stat.CO↗