arXiv subjects

Jun Zhu

Publications and source records attributed to Jun Zhu.

At least 505 records · Page 28Linked to original sources

arXiv2016

Low-Complexity QoS-Aware Coordinated Scheduling for Heterogenous Networks

In this paper, we consider a heterogenous network (HetNet), where low-power indoor femtocells are deployed in the coverage area of the existing macro base station (MBS). This paper proposes a novel coordinated random beamforming and user scheduling strategy to improve the throughput of users served by the femtocell access point (FAP) while satisfying the quality-of-service (QoS) requirements of users served by both MBS and FAP. The strategy, termed as QoS-Aware Coodinated Scheduling (QACS), requires limited coordination between the MBS and FAP, i.e., only the indexes of the qualified beams are shared. Exact statistical analysis for the ergodic achievable rate of both FAP and MBS with the proposed strategy are presented. Scheduling fairness is also addressed for the proposed QACS.

cs.IT↗

arXiv2016

Kernel Bayesian Inference with Posterior Regularization

We propose a vector-valued regression problem whose solution is equivalent to the reproducing kernel Hilbert space (RKHS) embedding of the Bayesian posterior distribution. This equivalence provides a new understanding of kernel Bayesian inference. Moreover, the optimization problem induces a new regularization for the posterior embedding estimator, which is faster and has comparable performance to the squared regularization in kernel Bayes' rule. This regularization coincides with a former thresholding approach used in kernel POMDPs whose consistency remains to be established. Our theoretical work solves this open problem and provides consistency analysis in regression settings. Based on our optimizational formulation, we propose a flexible Bayesian posterior regularization framework which for the first time enables us to put regularization at the distribution level. We apply this method to nonparametric state-space filtering tasks with extremely nonlinear dynamics and show performance gains over all other baselines.

stat.ML↗

arXiv2016

Fast Sampling for Bayesian Max-Margin Models

Bayesian max-margin models have shown superiority in various practical applications, such as text categorization, collaborative prediction, social network link prediction and crowdsourcing, and they conjoin the flexibility of Bayesian modeling and predictive strengths of max-margin learning. However, Monte Carlo sampling for these models still remains challenging, especially for applications that involve large-scale datasets. In this paper, we present the stochastic subgradient Hamiltonian Monte Carlo (HMC) methods, which are easy to implement and computationally efficient. We show the approximate detailed balance property of subgradient HMC which reveals a natural and validated generalization of the ordinary HMC. Furthermore, we investigate the variants that use stochastic subsampling and thermostats for better scalability and mixing. Using stochastic subgradient Markov Chain Monte Carlo (MCMC), we efficiently solve the posterior inference task of various Bayesian max-margin models and extensive experimental results demonstrate the effectiveness of our approach.

stat.ML↗

arXiv2016

SaberLDA: Sparsity-Aware Learning of Topic Models on GPUs

Latent Dirichlet Allocation (LDA) is a popular tool for analyzing discrete count data such as text and images. Applications require LDA to handle both large datasets and a large number of topics. Though distributed CPU systems have been used, GPU-based systems have emerged as a promising alternative because of the high computational power and memory bandwidth of GPUs. However, existing GPU-based LDA systems cannot support a large number of topics because they use algorithms on dense data structures whose time and space complexity is linear to the number of topics. In this paper, we propose SaberLDA, a GPU-based LDA system that implements a sparsity-aware algorithm to achieve sublinear time complexity and scales well to learn a large number of topics. To address the challenges introduced by sparsity, we propose a novel data layout, a new warp-based sampling kernel, and an efficient sparse count matrix updating algorithm that improves locality, makes efficient utilization of GPU warps, and reduces memory consumption. Experiments show that SaberLDA can learn from billions-token-scale data with up to 10,000 topics, which is almost two orders of magnitude larger than that of the previous GPU-based systems. With a single GPU card, SaberLDA is able to learn 10,000 topics from a dataset of billions of tokens in a few hours, which is only achievable with clusters with tens of machines before.

cs.DC↗

arXiv2016

Nematic Quantum Critical Fluctuations in BaFe$_{2-x}$Ni$_x$As$_2$

We have systematically studied the nematic fluctuations in the electron-doped iron-based superconductor BaFe$_{2-x}$Ni$_x$As$_2$ by measuring the in-plane resistance change under uniaxial pressure. While the nematic quantum critical point can be identified through the measurements along the (110) direction as studied previously, quantum and thermal critical fluctuations cannot be distinguished due to similar Curie-Weiss-like behaviors. Here we find that a sizable pressure-dependent resistivity along the (100) direction is present in all doping levels, which is against the simple picture of an Ising-type nematic model. The signal along the (100) direction becomes maximum at optimal doping, suggesting that it is associated with nematic quantum critical fluctuations. Our results indicate that thermal fluctuations from striped antiferromagnetic order dominate the underdoped regime along the (110) direction. We argue that either there is a strong coupling between the quantum critical fluctuations and the fermions, or more exotically, a higher symmetry may be present around optimal doping.

cond-mat.supr-con↗

arXiv2016

Secure Massive MIMO Systems with Limited RF Chains

In future practical deployments of massive multi-input multi-output (MIMO) systems, the number of radio frequency (RF) chains at the base stations (BSs) may be much smaller than the number of BS antennas to reduce the overall expenditure. In this paper, we propose a novel design framework for joint data and artificial noise (AN) precoding in a multiuser massive MIMO system with limited number of RF chains, which improves the wireless security performance. With imperfect channel state information (CSI), we analytically derive an achievable lower bound on the ergodic secrecy rate of any mobile terminal (MT), for both analog and hybrid precoding schemes. The closed-form lower bound is used to determine optimal power splitting between data and AN that maximizes the secrecy rate through simple one-dimensional search. Analytical and numerical results together reveal that the proposed hybrid precoder, although suffers from reduced secrecy rate compared with theoretical full-dimensional precoder, is free of the high computational complexity of large-scale matrix inversion and null-space calculations, and largely reduces the hardware cost.

cs.IT↗

arXiv2016

Analysis and Design of Secure Massive MIMO Systems in the Presence of Hardware Impairments

To keep the hardware costs of future communications systems manageable, the use of low-cost hardware components is desirable. This is particularly true for the emerging massive multiple-input multiple-output (MIMO) systems which equip base stations (BSs) with a large number of antenna elements. However, low-cost transceiver designs will further accentuate the hardware impairments which are present in any practical communication system. In this paper, we investigate the impact of hardware impairments on the secrecy performance of downlink massive MIMO systems in the presence of a passive multiple-antenna eavesdropper. Thereby, for the BS and the legitimate users, the joint effects of multiplicative phase noise, additive distortion noise, and amplified receiver noise are taken into account, whereas the eavesdropper is assumed to employ ideal hardware. We derive a lower bound for the ergodic secrecy rate of a given user when matched filter (MF) data precoding and artificial noise (AN) transmission are employed at the BS. Based on the derived analytical expression, we investigate the impact of the various system parameters on the secrecy rate and optimize both the pilot sets used for uplink training and the AN precoding. Our analytical and simulation results reveal that 1) the additive distortion noise at the BS may be beneficial for the secrecy performance, especially if the power assigned for AN emission is not sufficient; 2) all other hardware impairments have a negative impact on the secrecy performance; 3) {\textcolor{blue}{despite their susceptibility to pilot interference in the presence of phase noise}}, so-called spatially orthogonal pilot sequences are preferable unless the phase noise is very strong; 4) the proposed generalized null-space (NS) AN precoding method can efficiently mitigate the negative effects of phase noise.

cs.IT↗

arXiv2016

Generative Topic Embedding: a Continuous Representation of Documents (Extended Version with Proofs)

Word embedding maps words into a low-dimensional continuous embedding space by exploiting the local word collocation patterns in a small context window. On the other hand, topic modeling maps documents onto a low-dimensional topic space, by utilizing the global word collocation patterns in the same document. These two types of patterns are complementary. In this paper, we propose a generative topic embedding model to combine the two types of patterns. In our model, topics are represented by embedding vectors, and are shared across documents. The probability of each word is influenced by both its local context and its topic. A variational inference method yields the topic embeddings as well as the topic mixing proportions for each document. Jointly they represent the document in a low-dimensional continuous space. In two document classification tasks, our method performs better than eight existing methods, with fewer features. In addition, we illustrate with an example that our method can generate coherent topics even based on only one document.

cs.CL↗

arXiv2016

Conditional Generative Moment-Matching Networks

Maximum mean discrepancy (MMD) has been successfully applied to learn deep generative models for characterizing a joint distribution of variables via kernel mean embedding. In this paper, we present conditional generative moment- matching networks (CGMMN), which learn a conditional distribution given some input variables based on a conditional maximum mean discrepancy (CMMD) criterion. The learning is performed by stochastic gradient descent with the gradient calculated by back-propagation. We evaluate CGMMN on a wide range of tasks, including predictive modeling, contextual generation, and Bayesian dark knowledge, which distills knowledge from a Bayesian model by learning a relatively small CGMMN student network. Our results demonstrate competitive performance in all the tasks.

cs.LG↗

arXiv2016

PSDVec: a Toolbox for Incremental and Scalable Word Embedding

PSDVec is a Python/Perl toolbox that learns word embeddings, i.e. the mapping of words in a natural language to continuous vectors which encode the semantic/syntactic regularities between the words. PSDVec implements a word embedding learning method based on a weighted low-rank positive semidefinite approximation. To scale up the learning process, we implement a blockwise online learning algorithm to learn the embeddings incrementally. This strategy greatly reduces the learning time of word embeddings on a large vocabulary, and can learn the embeddings of new words without re-learning the whole vocabulary. On 9 word similarity/analogy benchmark sets and 2 Natural Language Processing (NLP) tasks, PSDVec produces embeddings that has the best average performance among popular word embedding tools. PSDVec provides a new option for NLP practitioners.

cs.CL↗

arXiv2016

Learning to Generate with Memory

Memory units have been widely used to enrich the capabilities of deep networks on capturing long-term dependencies in reasoning and prediction tasks, but little investigation exists on deep generative models (DGMs) which are good at inferring high-level invariant representations from unlabeled data. This paper presents a deep generative model with a possibly large external memory and an attention mechanism to capture the local detail information that is often lost in the bottom-up abstraction process in representation learning. By adopting a smooth attention model, the whole network is trained end-to-end by optimizing a variational bound of data likelihood via auto-encoding variational Bayesian methods, where an asymmetric recognition network is learnt jointly to infer high-level invariant representations. The asymmetric architecture can reduce the competition between bottom-up invariant feature extraction and top-down generation of instance details. Our experiments on several datasets demonstrate that memory can significantly boost the performance of DGMs and even achieve state-of-the-art results on various tasks, including density estimation, image generation, and missing value imputation.

cs.LG↗

arXiv2016

Towards Better Analysis of Deep Convolutional Neural Networks

Deep convolutional neural networks (CNNs) have achieved breakthrough performance in many pattern recognition tasks such as image classification. However, the development of high-quality deep models typically relies on a substantial amount of trial-and-error, as there is still no clear understanding of when and why a deep model works. In this paper, we present a visual analytics approach for better understanding, diagnosing, and refining deep CNNs. We formulate a deep CNN as a directed acyclic graph. Based on this formulation, a hybrid visualization is developed to disclose the multiple facets of each neuron and the interactions between them. In particular, we introduce a hierarchical rectangle packing algorithm and a matrix reordering algorithm to show the derived features of a neuron cluster. We also propose a biclustering-based edge bundling method to reduce visual clutter caused by a large number of connections between neurons. We evaluated our method on a set of CNNs and the results are generally favorable.

cs.CV↗

arXiv2016

Statistically-estimated tree composition for the northeastern United States at the time of Euro-American settlement

We present a gridded 8 km-resolution data product of the estimated composition of tree taxa at the time of Euro-American settlement of the northeastern United States and the statistical methodology used to produce the product from trees recorded by land surveyors. Composition is defined as the proportion of stems larger than approximately 20 cm diameter at breast height for 22 tree taxa, generally at the genus level. The data come from settlement-era public survey records that are transcribed and then aggregated spatially, giving count data. The domain is divided into two regions, eastern (Maine to Ohio) and midwestern (Indiana to Minnesota). Public Land Survey point data in the midwestern region (ca. 0.8-km resolution) are aggregated to a regular 8 km grid, while data in the eastern region, from Town Proprietor Surveys, are aggregated at the township level in irregularly-shaped local administrative units. The product is based on a Bayesian statistical model fit to the count data that estimates composition on a regular 8 km grid across the entire domain. The statistical model is designed to handle data from both the regular grid and the irregularly-shaped townships and allows us to estimate composition at locations with no data and to smooth over noise caused by limited counts in locations with data. The model also allows us to quantify uncertainty in our composition estimates, making the product suitable for applications employing data assimilation. We expect this data product to be useful for understanding the state of vegetation in the northeastern United States prior to large-scale Euro-American settlement. In addition to specific regional questions, the data product can also serve as a baseline against which to investigate how forests and ecosystems change after intensive settlement. The data product is available at the NIS data portal as version 1.0.

stat.AP↗

arXiv2016

Learning Deep Generative Models with Doubly Stochastic MCMC

We present doubly stochastic gradient MCMC, a simple and generic method for (approximate) Bayesian inference of deep generative models (DGMs) in a collapsed continuous parameter space. At each MCMC sampling step, the algorithm randomly draws a mini-batch of data samples to estimate the gradient of log-posterior and further estimates the intractable expectation over hidden variables via a neural adaptive importance sampler, where the proposal distribution is parameterized by a deep neural network and learnt jointly. We demonstrate the effectiveness on learning various DGMs in a wide range of tasks, including density estimation, data generation and missing data imputation. Our method outperforms many state-of-the-art competitors.

cs.LG↗

arXiv2016

Physical Layer Security for Massive MIMO Systems Impaired by Phase Noise

In this paper, we investigate the impact of phase noise on the secrecy performance of downlink massive MIMO systems in the presence of a passive multiple-antenna eavesdropper. Thereby, for the base station (BS) and the legitimate users, the effect of multiplicative phase noise is taken into account, whereas the eavesdropper is assumed to employ ideal hardware. We derive a lower bound for the ergodic secrecy rate of a given user when matched filter data precoding and artificial noise transmission are employed at the BS. Based on the derived analytical expression, we investigate the impact of the various system parameters on the secrecy rate. Our analytical and simulation results reveal that distributively deployed local oscillators (LOs) can achieve a better performance than one common LO for all BS antennas as long as a sufficient amount of power is assigned for data transmission.

cs.IT↗

arXiv2016

WarpLDA: a Cache Efficient O(1) Algorithm for Latent Dirichlet Allocation

Developing efficient and scalable algorithms for Latent Dirichlet Allocation (LDA) is of wide interest for many applications. Previous work has developed an O(1) Metropolis-Hastings sampling method for each token. However, the performance is far from being optimal due to random accesses to the parameter matrices and frequent cache misses. In this paper, we first carefully analyze the memory access efficiency of existing algorithms for LDA by the scope of random access, which is the size of the memory region in which random accesses fall, within a short period of time. We then develop WarpLDA, an LDA sampler which achieves both the best O(1) time complexity per token and the best O(K) scope of random access. Our empirical results in a wide range of testing conditions demonstrate that WarpLDA is consistently 5-15x faster than the state-of-the-art Metropolis-Hastings based LightLDA, and is comparable or faster than the sparsity aware F+LDA. With WarpLDA, users can learn up to one million topics from hundreds of millions of documents in a few hours, at an unprecedentedly throughput of 11G tokens per second.

stat.ML↗

arXiv2016

A Model-Based Scatter Artifacts Correction for Cone Beam CT

The purpose of this work is to provide a fast and accurate scatter artifacts correction algorithm for cone beam CT (CBCT) imaging. The method starts with an estimation of coarse scatter profiles for a set of CBCT data in either image domain or projection domain. A denoising algorithm designed specifically for Poisson signals is then applied to derive the final scatter distribution. Qualitative and quantitative evaluations using thorax and abdomen phantoms with Monte Carlo (MC) simulations, experimental Catphan phantom data, and in vivo human data acquired for a clinical image guided radiation therapy were performed. Results show that the proposed algorithm can significantly reduce scatter artifacts and recover the correct HU in either projection domain or image domain. For the MC thorax phantom study, four components segmentation yield the best results, while the results of three components segmentation are still acceptable. For the Catphan phantom data, the mean value over all pixels in the residual image is reduced from -21.8 HU to -0.2 HU and 0.7 HU for projection domain and image domain, respectively. The contrast of the in vivo human images are greatly improved after correction. The software-based technique has a number of advantages, such as high computational efficiency and accuracy, and the capability of performing scatter correction without modifying the clinical workflow or modifying the imaging hardware. When implemented practically, this should improve the accuracy of CBCT image quantitation and significantly impact CBCT-based interventional procedures and adaptive radiation therapy.

physics.med-ph↗

arXiv2016

Max-Margin Nonparametric Latent Feature Models for Link Prediction

Link prediction is a fundamental task in statistical network analysis. Recent advances have been made on learning flexible nonparametric Bayesian latent feature models for link prediction. In this paper, we present a max-margin learning method for such nonparametric latent feature relational models. Our approach attempts to unite the ideas of max-margin learning and Bayesian nonparametrics to discover discriminative latent features for link prediction. It inherits the advances of nonparametric Bayesian methods to infer the unknown latent social dimension, while for discriminative link prediction, it adopts the max-margin learning principle by minimizing a hinge-loss using the linear expectation operator, without dealing with a highly nonlinear link likelihood function. For posterior inference, we develop an efficient stochastic variational inference algorithm under a truncated mean-field assumption. Our methods can scale up to large-scale real networks with millions of entities and tens of millions of positive links. We also provide a full Bayesian formulation, which can avoid tuning regularization hyper-parameters. Experimental results on a diverse range of real datasets demonstrate the benefits inherited from max-margin learning and Bayesian nonparametric inference.

cs.LG↗