Searcharxiv⌕ Search

arXiv subjects

Jaeyong Lee

Publications and source records attributed to Jaeyong Lee.

At least 37 records · Page 2Linked to original sources
arXiv2022

Asymptotic Properties for Bayesian Neural Network in Besov Space

Neural networks have shown great predictive power when dealing with various unstructured data such as images and natural languages. The Bayesian neural network captures the uncertainty of prediction by putting a prior distribution for the parameter of the model and computing the posterior distribution. In this paper, we show that the Bayesian neural network using spike-and-slab prior has consistency with nearly minimax convergence rate when the true regression function is in the Besov space. Even when the smoothness of the regression function is unknown the same posterior convergence rate holds and thus the spike-and-slab prior is adaptive to the smoothness of the regression function. We also consider the shrinkage prior, which is more feasible than other priors, and show that it has the same convergence rate. In other words, we propose a practical Bayesian neural network with guaranteed asymptotic properties.

stat.ML↗
arXiv2022

Estimation of World Seroprevalence of SARS-CoV-2 antibodies

In this paper, we estimate the seroprevalence against COVID-19 by country and derive the seroprevalence over the world. To estimate seroprevalence, we use serological surveys (also called the serosurveys) conducted within each country. When the serosurveys are incorporated to estimate world seroprevalence, there are two issues. First, there are countries in which a serological survey has not been conducted. Second, the sample collection dates differ from country to country. We attempt to tackle these problems using the vaccination data, confirmed cases data, and national statistics. We construct Bayesian models to estimate the numbers of people who have antibodies produced by infection or vaccination separately. For the number of people with antibodies due to infection, we develop a hierarchical model for combining the information included in both confirmed cases data and national statistics. At the same time, we propose regression models to estimate missing values in the vaccination data. As of 31st of July 2021, using the proposed methods, we obtain the 95% credible interval of the world seroprevalence as [38.6%, 59.2%].

stat.AP↗
arXiv2021

Covariance Structure Estimation with Laplace Approximation

Gaussian covariance graph model is a popular model in revealing underlying dependency structures among random variables. A Bayesian approach to the estimation of covariance structures uses priors that force zeros on some off-diagonal entries of covariance matrices and put a positive definite constraint on matrices. In this paper, we consider a spike and slab prior on off-diagonal entries, which uses a mixture of point-mass and normal distribution. The point-mass naturally introduces sparsity to covariance structures so that the resulting posterior from this prior renders covariance structure learning. Under this prior, we calculate posterior model probabilities of covariance structures using Laplace approximation. We show that the error due to Laplace approximation becomes asymptotically marginal at some rate depending on the posterior convergence rate of covariance matrix under the Frobenius norm. With the approximated posterior model probabilities, we propose a new framework for estimating a covariance structure. Since the Laplace approximation is done around the mode of conditional posterior of covariance matrix, which cannot be obtained in the closed form, we propose a block coordinate descent algorithm to find the mode and show that the covariance matrix can be estimated using this algorithm once the structure is chosen. Through a simulation study based on five numerical models, we show that the proposed method outperforms graphical lasso and sample covariance matrix in terms of root mean squared error, max norm, spectral norm, specificity, and sensitivity. Also, the advantage of the proposed method is demonstrated in terms of accuracy compared to our competitors when it is applied to linear discriminant analysis (LDA) classification to breast cancer diagnostic dataset.

stat.ME↗
arXiv2021

Identifiability of Covariance Kernels in the Gaussian Process Regression Model

Gaussian process regression (GPR) model is a popular nonparametric regression model. In GPR, features of the regression function such as varying degrees of smoothness and periodicities are modeled through combining various covarinace kernels, which are supposed to model certain effects. The covariance kernels have unknown parameters which are estimated by the EM-algorithm or Markov Chain Monte Carlo. The estimated parameters are keys to the inference of the features of the regression functions, but identifiability of these parameters has not been investigated. In this paper, we prove identifiability of covariance kernel parameters in two radial basis mixed kernel GPR and radial basis and periodic mixed kernel GPR. We also provide some examples about non-identifiable cases in such mixed kernel GPRs.

math.ST↗
arXiv2021

Post-Processed Posteriors for Sparse Covariances and Its Application to Global Minimum Variance Portfolio

We consider Bayesian inference of sparse covariance matrices and propose a post-processed posterior. This method consists of two steps. In the first step, posterior samples are obtained from the conjugate inverse-Wishart posterior without considering the sparse structural assumption. The posterior samples are transformed in the second step to satisfy the sparse structural assumption through the hard-thresholding function. This non-traditional Bayesian procedure is justified by showing that the post-processed posterior attains the optimal minimax rates. We also investigate the application of the post-processed posterior to the estimation of the global minimum variance portfolio. We show that the post-processed posterior for the global minimum variance portfolio also attains the optimal minimax rate under the sparse covariance assumption. The advantages of the post-processed posterior for the global minimum variance portfolio are demonstrated by a simulation study and a real data analysis with S&P 400 data.

math.ST↗
arXiv2021

Laplace-aided variational inference for differential equation models

An ordinary differential equation (ODE) model, whose regression curves are a set of solution curves for some ODEs, poses a challenge in parameter estimation. The challenge, due to the frequent absence of analytic solutions and the complicated likelihood surface, tends to be more severe especially for larger models with many parameters and variables. Yang and Lee (2021) proposed a state-space model with variational Bayes (SSVB) for ODE, capable of fast and stable estimation in somewhat large ODE models. The method has shown excellent performance in parameter estimation but has a weakness of underestimation of the posterior covariance, which originates from the mean-field variational method. This paper proposes a way to overcome the weakness by using the Laplace approximation. In numerical experiments, the covariance modified by the Laplace approximation showed a high degree of improvement when checked against the covariances obtained by a standard Markov chain Monte Carlo method. With the improved covariance estimation, the SSVB renders fairly accurate posterior approximations.

stat.CO↗
arXiv2021

Variational Bayes method for ordinary differential equation models

Ordinary differential equations (ODEs) are a mathematical model used in many application areas such as climatology, bioinformatics, and chemical engineering with its intuitive appeal to modeling. Despite ODE's wide usage in modeling, the frequent absence of their analytic solutions makes it challenging to estimate ODE parameters from the data, especially when the model has lots of variables and parameters. This paper proposes a Bayesian ODE parameter estimating algorithm which is fast and accurate even for models with many parameters. The proposed method approximates an ODE model with a state-space model based on equations of a numeric solver. It allows fast estimation by avoiding computations of a complete numerical solution in the likelihood. The posterior is obtained by a variational Bayes method, more specifically, the approximate Riemannian conjugate gradient method (Honkela et al. 2010), which avoids samplings based on Markov chain Monte Carlo (MCMC). In simulation studies, we compared the speed and performance of the proposed method with existing methods. The proposed method showed the best performance in the reproduction of the true ODE curve with strong stability as well as the fastest computation, especially in a large model with more than 30 parameters. As a real-world data application, a SIR model with time-varying parameters was fitted to the COVID-19 data. Taking advantage of the proposed algorithm, more than 50 parameters were adequately estimated for each country.

stat.AP↗
arXiv2021

Bayesian Nonparametric Classification for Incomplete Data With a High Missing Rate: an Application to Semiconductor Manufacturing Data

During the semiconductor manufacturing process, predicting the yield of the semiconductor is an important problem. Early detection of defective product production in the manufacturing process can save huge production cost. The data generated from the semiconductor manufacturing process have characteristics of highly non-normal distributions, complicated missing patterns and high missing rate, which complicate the prediction of the yield. We propose Dirichlet process - naive Bayes model (DPNB), a classification method based on the mixtures of Dirichlet process and naive Bayes model. Since the DPNB is based on the mixtures of Dirichlet process and learns the joint distribution of all variables involved, it can handle highly non-normal data and can make predictions for the test dataset with any missing patterns. The DPNB also performs well for high missing rates since it uses all information of observed components. Experiments on various real datasets including semiconductor manufacturing data show that the DPNB has better performance than MICE and MissForest in terms of predicting missing values as percentage of missing values increases.

stat.AP↗
arXiv2021

Estimation of Conditional Mean Operator under the Bandable Covariance Structure

We consider high-dimensional multivariate linear regression models, where the joint distribution of covariates and response variables is a multivariate normal distribution with a bandable covariance matrix. The main goal of this paper is to estimate the regression coefficient matrix, which is a function of the bandable covariance matrix. Although the tapering estimator of covariance has the minimax optimal convergence rate for the class of bandable covariances, we show that it has a sub-optimal convergence rate for the regression coefficient; that is, a minimax estimator for the class of bandable covariances may not be a minimax estimator for its functionals. We propose the blockwise tapering estimator of the regression coefficient, which has the minimax optimal convergence rate for the regression coefficient under the bandable covariance assumption. We also propose a Bayesian procedure called the blockwise tapering post-processed posterior of the regression coefficient and show that the proposed Bayesian procedure has the minimax optimal convergence rate for the regression coefficient under the bandable covariance assumption. We show that the proposed methods outperform the existing methods via numerical studies.

math.ST↗
arXiv2021

Lévy Adaptive B-spline Regression via Overcomplete Systems

The estimation of functions with varying degrees of smoothness is a challenging problem in the nonparametric function estimation. In this paper, we propose the LABS (Lévy Adaptive B-Spline regression) model, an extension of the LARK models, for the estimation of functions with varying degrees of smoothness. LABS model is a LARK with B-spline bases as generating kernels. The B-spline basis consists of piecewise k degree polynomials with k-1 continuous derivatives and can express systematically functions with varying degrees of smoothness. By changing the orders of the B-spline basis, LABS can systematically adapt the smoothness of functions, i.e., jump discontinuities, sharp peaks, etc. Results of simulation studies and real data examples support that this model catches not only smooth areas but also jumps and sharp peaks of functions. The proposed model also has the best performance in almost all examples. Finally, we provide theoretical results that the mean function for the LABS model belongs to the certain Besov spaces based on the orders of the B-spline basis and that the prior of the model has the full support on the Besov spaces.

stat.ME↗
arXiv2021

Seroprevalence of SARS-CoV-2 antibodies in South Korea

In $2020$, Korea Disease Control and Prevention Agency reported three rounds of surveys on seroprevalence of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) antibodies in South Korea. We analyze the seroprevalence surveys using a Bayesian method with an informative prior distribution on the seroprevalence parameter, and the sensitivity and specificity of the diagnostic test. We construct the informative prior using the posterior distribution obtained from the clinical evaluation data based on the plaque reduction neutralization test. The constraint of the seroprevalence parameter induced from the known confirmed coronavirus 2019 cases can be imposed naturally in the proposed Bayesian model. We also prove that the confidence interval of the seroprevalence parameter based on the Rao's test can be the empty set, while the Bayesian method renders a reasonable interval estimator. As of the $30$th of October $2020$, the $95\%$ credible interval of the estimated SARS-CoV-2 positive population does not exceed $307,448$, approximately $0.6\%$ of the Korean population.

stat.AP↗
arXiv2021

The Beta-Mixture Shrinkage Prior for Sparse Covariances with Posterior Minimax Rates

Statistical inference for sparse covariance matrices is crucial to reveal dependence structure of large multivariate data sets, but lacks scalable and theoretically supported Bayesian methods. In this paper, we propose beta-mixture shrinkage prior, computationally more efficient than the spike and slab prior, for sparse covariance matrices and establish its minimax optimality in high-dimensional settings. The proposed prior consists of beta-mixture shrinkage and gamma priors for off-diagonal and diagonal entries, respectively. To ensure positive definiteness of the resulting covariance matrix, we further restrict the support of the prior to a subspace of positive definite matrices. We obtain the posterior convergence rate of the induced posterior under the Frobenius norm and establish a minimax lower bound for sparse covariance matrices. The class of sparse covariance matrices for the minimax lower bound considered in this paper is controlled by the number of nonzero off-diagonal elements and has more intuitive appeal than those appeared in the literature. The obtained posterior convergence rate coincides with the minimax lower bound unless the true covariance matrix is extremely sparse. In the simulation study, we show that the proposed method is computationally more efficient than competitors, while achieving comparable performance. Advantages of the shrinkage prior are demonstrated based on two real data sets.

math.ST↗
arXiv2020

Robust Sparse Bayesian Infinite Factor Models

Most of previous works and applications of Bayesian factor model have assumed the normal likelihood regardless of its validity. We propose a Bayesian factor model for heavy-tailed high-dimensional data based on multivariate Student-$t$ likelihood to obtain better covariance estimation. We use multiplicative gamma process shrinkage prior and factor number adaptation scheme proposed in Bhattacharya & Dunson [Biometrika (2011) 291-306]. Since a naive Gibbs sampler for the proposed model suffers from slow mixing, we propose a Markov Chain Monte Carlo algorithm where fast mixing of Hamiltonian Monte Carlo is exploited for some parameters in proposed model. Simulation results illustrate the gain in performance of covariance estimation for heavy-tailed high-dimensional data. We also provide a theoretical result that the posterior of the proposed model is weakly consistent under reasonable conditions. We conclude the paper with the application of proposed factor model on breast cancer metastasis prediction given DNA signature data of cancer cell.

stat.ME↗
arXiv2020

Post-Processed Posteriors for Banded Covariances

We consider Bayesian inference of banded covariance matrices and propose a post-processed posterior. The post-processing of the posterior consists of two steps. In the first step, posterior samples are obtained from the conjugate inverse-Wishart posterior which does not satisfy any structural restrictions. In the second step, the posterior samples are transformed to satisfy the structural restriction through a post-processing function. The conceptually straightforward procedure of the post-processed posterior makes its computation efficient and can render interval estimators of functionals of covariance matrices. We show that it has nearly optimal minimax rates for banded covariances among all possible pairs of priors and post-processing functions. Furthermore, we prove that the expected coverage probability of the $(1-α)100\%$ highest posterior density region of the post-processed posterior is asymptotically $1-α$ with respect to a conventional posterior distribution. It implies that the highest posterior density region of the post-processed posterior is, on average, a credible set of a conventional posterior. The advantages of the post-processed posterior are demonstrated by a simulation study and a real data analysis.

math.ST↗
arXiv2018

Minimax Posterior Convergence Rates and Model Selection Consistency in High-dimensional DAG Models based on Sparse Cholesky Factors

In this paper, we study the high-dimensional sparse directed acyclic graph (DAG) models under the empirical sparse Cholesky prior. Among our results, strong model selection consistency or graph selection consistency is obtained under more general conditions than those in the existing literature. Compared to Cao, Khare and Ghosh (2017), the required conditions are weakened in terms of the dimensionality, sparsity and lower bound of the nonzero elements in the Cholesky factor. Furthermore, our result does not require the irrepresentable condition, which is necessary for Lasso type methods. We also derive the posterior convergence rates for precision matrices and Cholesky factors with respect to various matrix norms. The obtained posterior convergence rates are the fastest among those of the existing Bayesian approaches. In particular, we prove that our posterior convergence rates for Cholesky factors are the minimax or at least nearly minimax depending on the relative size of true sparseness for the entire dimension. The simulation study confirms that the proposed method outperforms the competing methods.

stat.ME↗
arXiv2017

Optimal Bayesian Minimax Rates for Unconstrained Large Covariance Matrices

We obtain the optimal Bayesian minimax rate for the unconstrained large covariance matrix of multivariate normal sample with mean zero, when both the sample size, n, and the dimension, p, of the covariance matrix tend to infinity. Traditionally the posterior convergence rate is used to compare the frequentist asymptotic performance of priors, but defining the optimality with it is elusive. We propose a new decision theoretic framework for prior selection and define Bayesian minimax rate. Under the proposed framework, we obtain the optimal Bayesian minimax rate for the spectral norm for all rates of p. We also considered Frobenius norm, Bregman divergence and squared log-determinant loss and obtain the optimal Bayesian minimax rate under certain rate conditions on p. A simulation study is conducted to support the theoretical results.

math.ST↗
arXiv2017

Estimating Large Precision Matrices via Modified Cholesky Decomposition

We introduce the $k$-banded Cholesky prior for estimating a high-dimensional bandable precision matrix via the modified Cholesky decomposition. The bandable assumption is imposed on the Cholesky factor of the decomposition. We obtained the P-loss convergence rate under the spectral norm and the matrix $\ell_{\infty}$ norm and the minimax lower bounds. Since the P-loss convergence rate (Lee and Lee (2017)) is stronger than the posterior convergence rate, the rates obtained are also posterior convergence rates. Furthermore, when the true precision matrix is a $k_0$-banded matrix with some finite $k_0$, the obtained P-loss convergence rates coincide with the minimax rates. The established convergence rates are slightly slower than the minimax lower bounds, but these are the fastest rates for bandable precision matrices among the existing Bayesian approaches. A simulation study is conducted to compare the performance to the other competitive estimators in various scenarios.

math.ST↗