arXiv subjects

Jun Zhu

Publications and source records attributed to Jun Zhu.

At least 541 records · Page 30Linked to original sources

arXiv2015

Fast Scatter Artifacts Correction for Cone-Beam CT without System Modification and Repeat Scan

We provide a fast and accurate scatter artifacts correction algorithm for cone beam CT (CBCT) imaging. The method starts with an estimation of coarse scatter profile for a set of CBCT images. A total-variation denoising algorithm designed specifically for Poisson signal is then applied to derive the final scatter distribution. Qualitatively and quantitatively evaluations using Monte Carlo (MC) simulations, experimental CBCT phantom data, and \emph{in vivo} human data acquired for a clinical image guided radiation therapy were performed. Results show that the proposed algorithm can significantly reduce scatter artifacts and recover the correct HU within either projection domain or image domain. Further test shows the method is robust with respect to segmentation procedure.

physics.med-ph↗

arXiv2014

Local Adaptive Grouped Regularization and its Oracle Properties for Varying Coefficient Regression

Varying coefficient regression is a flexible technique for modeling data where the coefficients are functions of some effect-modifying parameter, often time or location in a certain domain. While there are a number of methods for variable selection in a varying coefficient regression model, the existing methods are mostly for global selection, which includes or excludes each covariate over the entire domain. Presented here is a new local adaptive grouped regularization (LAGR) method for local variable selection in spatially varying coefficient linear and generalized linear regression. LAGR selects the covariates that are associated with the response at any point in space, and simultaneously estimates the coefficients of those covariates by tailoring the adaptive group Lasso toward a local regression model with locally linear coefficient estimates. Oracle properties of the proposed method are established under local linear regression and local generalized linear regression. The finite sample properties of LAGR are assessed in a simulation study and for illustration, the Boston housing price data set is analyzed.

stat.ME↗

arXiv2014

Contrastive Feature Induction for Efficient Structure Learning of Conditional Random Fields

Structure learning of Conditional Random Fields (CRFs) can be cast into an L1-regularized optimization problem. To avoid optimizing over a fully linked model, gain-based or gradient-based feature selection methods start from an empty model and incrementally add top ranked features to it. However, for high-dimensional problems like statistical relational learning, training time of these incremental methods can be dominated by the cost of evaluating the gain or gradient of a large collection of candidate features. In this study we propose a fast feature evaluation algorithm called Contrastive Feature Induction (CFI), which only evaluates a subset of features that involve both variables with high signals (deviation from mean) and variables with high errors (residue). We prove that the gradient of candidate features can be represented solely as a function of signals and errors, and that CFI is an efficient approximation of gradient-based evaluation methods. Experiments on synthetic and real data sets show competitive learning speed and accuracy of CFI on pairwise CRFs, compared to state-of-the-art structure learning methods such as full optimization over all features, and Grafting.

cs.LG↗

arXiv2014

Secure Transmission in Multi-Cell Massive MIMO Systems

In this paper, we consider physical layer security provisioning in multi-cell massive multiple-input multiple-output (MIMO) systems. Specifically, we consider secure downlink transmission in a multi-cell massive MIMO system with matched-filter precoding and artificial noise (AN) generation at the base station (BS) in the presence of a passive multi-antenna eavesdropper. We investigate the resulting achievable ergodic secrecy rate and the secrecy outage probability for the cases of perfect training and pilot contamination. Thereby, we consider two different AN shaping matrices, namely, the conventional AN shaping matrix, where the AN is transmitted in the null space of the matrix formed by all user channels, and a random AN shaping matrix, which avoids the complexity associated with finding the null space of a large matrix. Our analytical and numerical results reveal that in multi-cell massive MIMO systems employing matched-filter precoding (1) AN generation is required to achieve a positive ergodic secrecy rate if the user and the eavesdropper experience the same path-loss, (2) even with AN generation secure transmission may not be possible if the number of eavesdropper antennas is too large and not enough power is allocated to channel estimation, (3) for a given fraction of power allocated to AN and a given number of users, in case of pilot contamination, the ergodic secrecy rate is not a monotonically increasing function of the number of BS antennas, and (4) random AN shaping matrices provide a favourable performance/complexity tradeoff and are an attractive alternative to conventional AN shaping matrices.

cs.IT↗

arXiv2014

Characterizations of all-derivable points in $B(H)$

Let ${\mathcal{K}}$ and ${\mathcal{H}}$ be two Hilbert space, and let $B({\mathcal{K}},{\mathcal{H}})$ be the algebra of all bounded linear operators from ${\mathcal{K}}$ into ${\mathcal{H}}$. We say that an element $G\in B({\mathcal{H}},{\mathcal{H}})$ is an all-derivable point in $B({\mathcal{H}},{\mathcal{H}})$ if every derivable linear mapping $φ$ at $G$ (i.e. $φ(ST)=φ(S)T+Sφ(T)$ for any $S,T\in B(H)$ with $ST=G$) is a derivation. Let both $φ: B({\mathcal{H}},{\mathcal{K}})\rightarrow B({\mathcal{H}},{\mathcal{K}})$ and $ψ: B({\mathcal{K}},{\mathcal{H}})\rightarrow B({\mathcal{K}},{\mathcal{H}})$ be two linear mappings. In this paper, the following results will be proved : if $Yφ(W)=ψ(Y)W$ for any $Y\in B({\mathcal{K}},{\mathcal{H}})$ and $W\in B({\mathcal{H}},{\mathcal{K}})$, then $φ(W)=DW$ and $ψ(Y)=YD$ for some $D\in B({\mathcal{K}})$. As an important application, we will show that an operator $G$ is an all-derivable point in $B({\mathcal{H}},{\mathcal{H}})$ if and only if $G\neq 0$.

math.OA↗

arXiv2014

Dropout Training for Support Vector Machines

Dropout and other feature noising schemes have shown promising results in controlling over-fitting by artificially corrupting the training data. Though extensive theoretical and empirical studies have been performed for generalized linear models, little work has been done for support vector machines (SVMs), one of the most successful approaches for supervised learning. This paper presents dropout training for linear SVMs. To deal with the intractable expectation of the non-smooth hinge loss under corrupting distributions, we develop an iteratively re-weighted least square (IRLS) algorithm by exploring data augmentation techniques. Our algorithm iteratively minimizes the expectation of a re-weighted least square problem, where the re-weights have closed-form solutions. The similar ideas are applied to develop a new IRLS algorithm for the expected logistic loss under corrupting distributions. Our algorithms offer insights on the connection and difference between the hinge loss and logistic loss in dropout training. Empirical results on several real datasets demonstrate the effectiveness of dropout training on significantly boosting the classification accuracy of linear SVMs.

cs.LG↗

arXiv2014

Bayesian Inference with Posterior Regularization and applications to Infinite Latent SVMs

Existing Bayesian models, especially nonparametric Bayesian methods, rely on specially conceived priors to incorporate domain knowledge for discovering improved latent representations. While priors can affect posterior distributions through Bayes' rule, imposing posterior regularization is arguably more direct and in some cases more natural and general. In this paper, we present regularized Bayesian inference (RegBayes), a novel computational framework that performs posterior inference with a regularization term on the desired post-data posterior distribution under an information theoretical formulation. RegBayes is more flexible than the procedure that elicits expert knowledge via priors, and it covers both directed Bayesian networks and undirected Markov networks whose Bayesian formulation results in hybrid chain graph models. When the regularization is induced from a linear operator on the posterior distributions, such as the expectation operator, we present a general convex-analysis theorem to characterize the solution of RegBayes. Furthermore, we present two concrete examples of RegBayes, infinite latent support vector machines (iLSVM) and multi-task infinite latent support vector machines (MT-iLSVM), which explore the large-margin idea in combination with a nonparametric Bayesian model for discovering predictive latent features for classification and multi-task learning, respectively. We present efficient inference methods and report empirical studies on several benchmark datasets, which appear to demonstrate the merits inherited from both large-margin learning and Bayesian nonparametrics. Such results were not available until now, and contribute to push forward the interface between these two important subfields, which have been largely treated as isolated in the community.

cs.LG↗

arXiv2013

Online Bayesian Passive-Aggressive Learning

Online Passive-Aggressive (PA) learning is an effective framework for performing max-margin online learning. But the deterministic formulation and estimated single large-margin model could limit its capability in discovering descriptive structures underlying complex data. This pa- per presents online Bayesian Passive-Aggressive (BayesPA) learning, which subsumes the online PA and extends naturally to incorporate latent variables and perform nonparametric Bayesian inference, thus providing great flexibility for explorative analysis. We apply BayesPA to topic modeling and derive efficient online learning algorithms for max-margin topic models. We further develop nonparametric methods to resolve the number of topics. Experimental results on real datasets show that our approaches significantly improve time efficiency while maintaining comparable results with the batch counterparts.

cs.LG↗

arXiv2013

Gibbs Max-margin Topic Models with Data Augmentation

Max-margin learning is a powerful approach to building classifiers and structured output predictors. Recent work on max-margin supervised topic models has successfully integrated it with Bayesian topic models to discover discriminative latent semantic structures and make accurate predictions for unseen testing data. However, the resulting learning problems are usually hard to solve because of the non-smoothness of the margin loss. Existing approaches to building max-margin supervised topic models rely on an iterative procedure to solve multiple latent SVM subproblems with additional mean-field assumptions on the desired posterior distributions. This paper presents an alternative approach by defining a new max-margin loss. Namely, we present Gibbs max-margin supervised topic models, a latent variable Gibbs classifier to discover hidden topic representations for various tasks, including classification, regression and multi-task learning. Gibbs max-margin supervised topic models minimize an expected margin loss, which is an upper bound of the existing margin loss derived from an expected prediction rule. By introducing augmented variables and integrating out the Dirichlet variables analytically by conjugacy, we develop simple Gibbs sampling algorithms with no restricting assumptions and no need to solve SVM subproblems. Furthermore, each step of the "augment-and-collapse" Gibbs sampling algorithms has an analytical conditional distribution, from which samples can be easily drawn. Experimental results demonstrate significant improvements on time efficiency. The classification performance is also significantly improved over competitors on binary, multi-class and multi-label classification tasks.

stat.ML↗

arXiv2013

Improved Bayesian Logistic Supervised Topic Models with Data Augmentation

Supervised topic models with a logistic likelihood have two issues that potentially limit their practical use: 1) response variables are usually over-weighted by document word counts; and 2) existing variational inference methods make strict mean-field assumptions. We address these issues by: 1) introducing a regularization constant to better balance the two parts based on an optimization formulation of Bayesian inference; and 2) developing a simple Gibbs sampling algorithm by introducing auxiliary Polya-Gamma variables and collapsing out Dirichlet variables. Our augment-and-collapse sampling algorithm has analytical forms of each conditional distribution without making any restricting assumptions and can be easily parallelized. Empirical results demonstrate significant improvements on prediction performance and time efficiency.

cs.LG↗

arXiv2013

Discriminative Relational Topic Models

Many scientific and engineering fields involve analyzing network data. For document networks, relational topic models (RTMs) provide a probabilistic generative process to describe both the link structure and document contents, and they have shown promise on predicting network structures and discovering latent topic representations. However, existing RTMs have limitations in both the restricted model expressiveness and incapability of dealing with imbalanced network data. To expand the scope and improve the inference accuracy of RTMs, this paper presents three extensions: 1) unlike the common link likelihood with a diagonal weight matrix that allows the-same-topic interactions only, we generalize it to use a full weight matrix that captures all pairwise topic interactions and is applicable to asymmetric networks; 2) instead of doing standard Bayesian inference, we perform regularized Bayesian inference (RegBayes) with a regularization parameter to deal with the imbalanced link structure issue in common real networks and improve the discriminative ability of learned latent representations; and 3) instead of doing variational approximation with strict mean-field assumptions, we present collapsed Gibbs sampling algorithms for the generalized relational topic models by exploring data augmentation without making restricting assumptions. Under the generic RegBayes framework, we carefully investigate two popular discriminative loss functions, namely, the logistic log-loss and the max-margin hinge loss. Experimental results on several real network datasets demonstrate the significance of these extensions on improving the prediction performance, and the time efficiency can be dramatically improved with a simple fast approximation method.

cs.LG↗

arXiv2013

An optimal problem for relative entropy

Relative entropy is an essential tool in quantum information theory. There are so many problems which are related to relative entropy. In this article, the optimal values which are defined by $\displaystyle\max_{U\in{U(\cX_{d})}} S(Uρ{U^{\ast}}\parallelσ)$ and $\displaystyle\min_{U\in{U(\cX_{d})}} S(Uρ{U^{\ast}}\parallelσ)$ for two positive definite operators $ρ,σ\in{\textmd{Pd}(\cX)}$ are obtained. And the set of $S(Uρ{U^{\ast}}\parallelσ)$ for every unitary operator $U$ is full of the interval $[\displaystyle\min_{U\in{U(\cX_{d})}} S(Uρ{U^{\ast}}\parallelσ),\displaystyle\max_{U\in{U(\cX_{d})}} S(Uρ{U^{\ast}}\parallelσ)]$

cs.IT↗

arXiv2013

Linear mappings of local preserving-majorization on matrix algebras

Let $\M_{n\times n}$ be the algebra of all $n\times n$ matrices. For $x,y\in {R}^{n}$ it is said that $x$ is majorized by $y$ if there is a double stochastic matrix $A\in {M}_{n\times n}$ such that $x=Ay$ (denoted by $x\prec y$). Suppose that $Φ$ is a linear mapping from ${R}^{n}$ into ${R}^{n}$, which is said to be strictly isotone if $Φ(x)\prec Φ(y)$ whenever $x\prec y$. We say that an element $α\in {R}^{n}$ is a strictly all-isotone point if every strictly isotone $φ$ at $α$ (i.e. $Φ(α)\precΦ(y)$ whenever $x\in {R}^{n}$ with $α\prec x$, and $Φ(x)\precΦ(α)$ whenever $x\in {R}^{n}$ with $x\prec α$) is a strictly isotone. In this paper we show that every $α=(α_{1},α_{2},...,α_{n})\in {R}^{n}$ with $α_{1}>α_{2}>...>α_{n}$ is a strictly all-isotone point.

math.QA↗

arXiv2012

Unifying Treatment of Discord via Relative Entropy

A new form of zero-discord state via Petz's monotonicity condition on relative entropy with equality has been derived systematically. A generalization of symmetric zero-discord states is presented and the related physical implications are discussed.

quant-ph↗

arXiv2012

Max-Margin Nonparametric Latent Feature Models for Link Prediction

We present a max-margin nonparametric latent feature model, which unites the ideas of max-margin learning and Bayesian nonparametrics to discover discriminative latent features for link prediction and automatically infer the unknown latent social dimension. By minimizing a hinge-loss using the linear expectation operator, we can perform posterior inference efficiently without dealing with a highly nonlinear link likelihood function; by using a fully-Bayesian formulation, we can avoid tuning regularization constants. Experimental results on real datasets appear to demonstrate the benefits inherited from max-margin learning and fully-Bayesian nonparametric inference.

cs.LG↗

arXiv2012

Penalized maximum likelihood estimation and variable selection in geostatistics

We consider the problem of selecting covariates in spatial linear models with Gaussian process errors. Penalized maximum likelihood estimation (PMLE) that enables simultaneous variable selection and parameter estimation is developed and, for ease of computation, PMLE is approximated by one-step sparse estimation (OSE). To further improve computational efficiency, particularly with large sample sizes, we propose penalized maximum covariance-tapered likelihood estimation (PMLE$_{\mathrm{T}}$) and its one-step sparse estimation (OSE$_{\mathrm{T}}$). General forms of penalty functions with an emphasis on smoothly clipped absolute deviation are used for penalized maximum likelihood. Theoretical properties of PMLE and OSE, as well as their approximations PMLE$_{\mathrm{T}}$ and OSE$_{\mathrm{T}}$ using covariance tapering, are derived, including consistency, sparsity, asymptotic normality and the oracle properties. For covariance tapering, a by-product of our theoretical results is consistency and asymptotic normality of maximum covariance-tapered likelihood estimates. Finite-sample properties of the proposed methods are demonstrated in a simulation study and, for illustration, the methods are applied to analyze two real data sets.

stat.ME↗

arXiv2012

Sparse Topical Coding

We present sparse topical coding (STC), a non-probabilistic formulation of topic models for discovering latent representations of large collections of data. Unlike probabilistic topic models, STC relaxes the normalization constraint of admixture proportions and the constraint of defining a normalized likelihood function. Such relaxations make STC amenable to: 1) directly control the sparsity of inferred representations by using sparsity-inducing regularizers; 2) be seamlessly integrated with a convex error function (e.g., SVM hinge loss) for supervised learning; and 3) be efficiently learned with a simply structured coordinate descent algorithm. Our results demonstrate the advantages of STC and supervised MedSTC on identifying topical meanings of words and improving classification accuracy and time efficiency.

cs.LG↗

arXiv2011

Jordan Higher All-Derivable Points in Nest Algebras

Let $\mathcal{N}$ be a non-trivial and complete nest on a Hilbert space $H$. Suppose $d=\{d_n: n\in N\}$ is a group of linear mappings from Alg$\mathcal{N}$ into itself. We say that $d=\{d_n: n\in N\}$ is a Jordan higher derivable mapping at a given point $G$ if $d_{n}(ST+ST)=\sum\limits_{i+j=n}\{d_{i}(S)d_{j}(T)+d_{j}(T)d_{i}(S)\}$ for any $S,T\in Alg \mathcal{N}$ with $ST=G$. An element $G\in Alg \mathcal{N}$ is called a Jordan higher all-derivable point if every Jordan higher derivable mapping at $G$ is a higher derivation. In this paper, we mainly prove that any given point $G$ of Alg$\mathcal{N}$ is a Jordan higher all-derivable point. This extends some results in \cite{Chen11} to the case of higher derivations.

math.OA↗