arXiv subjects

Jun Zhou

Publications and source records attributed to Jun Zhou.

At least 505 records · Page 28Linked to original sources

arXiv2018

Multi-scale Convolution Aggregation and Stochastic Feature Reuse for DenseNets

Recently, Convolution Neural Networks (CNNs) obtained huge success in numerous vision tasks. In particular, DenseNets have demonstrated that feature reuse via dense skip connections can effectively alleviate the difficulty of training very deep networks and that reusing features generated by the initial layers in all subsequent layers has strong impact on performance. To feed even richer information into the network, a novel adaptive Multi-scale Convolution Aggregation module is presented in this paper. Composed of layers for multi-scale convolutions, trainable cross-scale aggregation, maxout, and concatenation, this module is highly non-linear and can boost the accuracy of DenseNet while using much fewer parameters. In addition, due to high model complexity, the network with extremely dense feature reuse is prone to overfitting. To address this problem, a regularization method named Stochastic Feature Reuse is also presented. Through randomly dropping a set of feature maps to be reused for each mini-batch during the training phase, this regularization method reduces training costs and prevents co-adaptation. Experimental results on CIFAR-10, CIFAR-100 and SVHN benchmarks demonstrated the effectiveness of the proposed methods.

cs.LG↗

arXiv2018

Review Helpfulness Prediction with Embedding-Gated CNN

Product reviews, in the form of texts dominantly, significantly help consumers finalize their purchasing decisions. Thus, it is important for e-commerce companies to predict review helpfulness to present and recommend reviews in a more informative manner. In this work, we introduce a convolutional neural network model that is able to extract abstract features from multi-granularity representations. Inspired by the fact that different words contribute to the meaning of a sentence differently, we consider to learn word-level embedding-gates for all the representations. Furthermore, as it is common that some product domains/categories have rich user reviews, other domains not. To help domains with less sufficient data, we integrate our model into a cross-domain relationship learning framework for effectively transferring knowledge from other domains. Extensive experiments show that our model yields better performance than the existing methods.

cs.CL↗

arXiv2018

Dropout with Tabu Strategy for Regularizing Deep Neural Networks

Dropout has proven to be an effective technique for regularization and preventing the co-adaptation of neurons in deep neural networks (DNN). It randomly drops units with a probability $p$ during the training stage of DNN. Dropout also provides a way of approximately combining exponentially many different neural network architectures efficiently. In this work, we add a diversification strategy into dropout, which aims at generating more different neural network architectures in a proper times of iterations. The dropped units in last forward propagation will be marked. Then the selected units for dropping in the current FP will be kept if they have been marked in the last forward propagation. We only mark the units from the last forward propagation. We call this new technique Tabu Dropout. Tabu Dropout has no extra parameters compared with the standard Dropout and also it is computationally cheap. The experiments conducted on MNIST, Fashion-MNIST datasets show that Tabu Dropout improves the performance of the standard dropout.

cs.LG↗

arXiv2018

Off-center Rattling Triggers High-Temperature Thermal Transport in Thermoelectric lathrates: Non-Perturbative Approach

Lattice thermal conductivities $κ_{\rm L}$ of type-I clathrates encapsulating "off-center" guest atoms behave in a way that is sharply different from those of conventional crystals such as on-center type-I clathrates. The latter decreases with increasing temperature according to 1/$T$ above a few tens\,K. While, $κ_{\rm L}$ of off-center clathrates increase T-linearly above a few tens K. Further increasing temperature above $T\gtrsim$100 K, $κ_{\rm L}$ saturates without exhibiting appreciable T-dependence. In this temperature regime, the standard formulation of $κ_{\rm L}$ based on the perturbation theory is unfeasible. We herein present a theory of $κ_{\rm L}$ based on "non-perturbative" approach by taking into account the interaction between non-vibrational off-center rattling states of guest atoms and cage shells. It is remarkable that our theory shows excellent agreement with observed $κ_{\rm L}(T)$ for off-center clathrates in magnitudes, the cage volume $Ω$-dependence and T-dependence.

cond-mat.mtrl-sci↗

arXiv2018

Distributed Collaborative Hashing and Its Applications in Ant Financial

Collaborative filtering, especially latent factor model, has been popularly used in personalized recommendation. Latent factor model aims to learn user and item latent factors from user-item historic behaviors. To apply it into real big data scenarios, efficiency becomes the first concern, including offline model training efficiency and online recommendation efficiency. In this paper, we propose a Distributed Collaborative Hashing (DCH) model which can significantly improve both efficiencies. Specifically, we first propose a distributed learning framework, following the state-of-the-art parameter server paradigm, to learn the offline collaborative model. Our model can be learnt efficiently by distributedly computing subgradients in minibatches on workers and updating model parameters on servers asynchronously. We then adopt hashing technique to speedup the online recommendation procedure. Recommendation can be quickly made through exploiting lookup hash tables. We conduct thorough experiments on two real large-scale datasets. The experimental results demonstrate that, comparing with the classic and state-of-the-art (distributed) latent factor models, DCH has comparable performance in terms of recommendation accuracy but has both fast convergence speed in offline model training procedure and realtime efficiency in online recommendation procedure. Furthermore, the encouraging performance of DCH is also shown for several real-world applications in Ant Financial.

cs.LG↗

arXiv2018

Design Identification of Curve Patterns on Cultural Heritage Objects: Combining Template Matching and CNN-based Re-Ranking

The surfaces of many cultural heritage objects were embellished with various patterns, especially curve patterns. In practice, most of the unearthed cultural heritage objects are highly fragmented, e.g., sherds of potteries or vessels, and each of them only shows a very small portion of the underlying full design, with noise and deformations. The goal of this paper is to address the challenging problem of automatically identifying the underlying full design of curve patterns from such a sherd. Specifically, we formulate this problem as template matching: curve structure segmented from the sherd is matched to each location with each possible orientation of each known full design. In this paper, we propose a new two-stage matching algorithm, with a different matching cost in each stage. In Stage 1, we use a traditional template matching, which is highly computationally efficient, over the whole search space and identify a small set of candidate matchings. In Stage 2, we derive a new matching cost by training a dual-source Convolutional Neural Network (CNN) and apply it to re-rank the candidate matchings identified in Stage 1. We collect 600 pottery sherds with 98 full designs from the Woodland Period in Southeastern North America for experiments and the performance of the proposed algorithm is very competitive.

cs.LG↗

arXiv2018

Thermal Conductivity of Polymers and Their Nanocomposites

Polymers are usually considered as thermal insulators and their applications are limited by their low thermal conductivity. However, recent studies showed that certain polymers have surprisingly high thermal conductivity, some of which are comparable to that in poor metals or even silicon. In this review, we outline the experimental achievements and theoretical progress of thermal transport in polymers and their nanocomposites. The open questions and challenges of existing theories are discussed. Special attention is given to the mechanism of thermal transport, the enhancement of thermal conductivity in polymer nanocomposites/fibers, and their potential application as thermal interface materials.

cond-mat.mes-hall↗

arXiv2018

A Boosting Framework of Factorization Machine

Recently, Factorization Machines (FM) has become more and more popular for recommendation systems, due to its effectiveness in finding informative interactions between features. Usually, the weights for the interactions is learnt as a low rank weight matrix, which is formulated as an inner product of two low rank matrices. This low rank can help improve the generalization ability of Factorization Machines. However, to choose the rank properly, it usually needs to run the algorithm for many times using different ranks, which clearly is inefficient for some large-scale datasets. To alleviate this issue, we propose an Adaptive Boosting framework of Factorization Machines (AdaFM), which can adaptively search for proper ranks for different datasets without re-training. Instead of using a fixed rank for FM, the proposed algorithm will adaptively gradually increases its rank according to its performance until the performance does not grow, using boosting strategy. To verify the performance of our proposed framework, we conduct an extensive set of experiments on many real-world datasets. Encouraging empirical results shows that the proposed algorithms are generally more effective than state-of-the-art other Factorization Machines.

cs.LG↗

arXiv2018

Feature Propagation on Graph: A New Perspective to Graph Representation Learning

We study feature propagation on graph, an inference process involved in graph representation learning tasks. It's to spread the features over the whole graph to the $t$-th orders, thus to expand the end's features. The process has been successfully adopted in graph embedding or graph neural networks, however few works studied the convergence of feature propagation. Without convergence guarantees, it may lead to unexpected numerical overflows and task failures. In this paper, we first define the concept of feature propagation on graph formally, and then study its convergence conditions to equilibrium states. We further link feature propagation to several established approaches such as node2vec and structure2vec. In the end of this paper, we extend existing approaches from represent nodes to edges (edge2vec) and demonstrate its applications on fraud transaction detection in real world scenario. Experiments show that it is quite competitive.

cs.SI↗

arXiv2018

Fast Subspace Clustering Based on the Kronecker Product

Subspace clustering is a useful technique for many computer vision applications in which the intrinsic dimension of high-dimensional data is often smaller than the ambient dimension. Spectral clustering, as one of the main approaches to subspace clustering, often takes on a sparse representation or a low-rank representation to learn a block diagonal self-representation matrix for subspace generation. However, existing methods require solving a large scale convex optimization problem with a large set of data, with computational complexity reaches O(N^3) for N data points. Therefore, the efficiency and scalability of traditional spectral clustering methods can not be guaranteed for large scale datasets. In this paper, we propose a subspace clustering model based on the Kronecker product. Due to the property that the Kronecker product of a block diagonal matrix with any other matrix is still a block diagonal matrix, we can efficiently learn the representation matrix which is formed by the Kronecker product of k smaller matrices. By doing so, our model significantly reduces the computational complexity to O(kN^{3/k}). Furthermore, our model is general in nature, and can be adapted to different regularization based subspace clustering methods. Experimental results on two public datasets show that our model significantly improves the efficiency compared with several state-of-the-art methods. Moreover, we have conducted experiments on synthetic data to verify the scalability of our model for large scale datasets.

cs.LG↗

arXiv2018

Exploring Linear Relationship in Feature Map Subspace for ConvNets Compression

While the research on convolutional neural networks (CNNs) is progressing quickly, the real-world deployment of these models is often limited by computing resources and memory constraints. In this paper, we address this issue by proposing a novel filter pruning method to compress and accelerate CNNs. Our work is based on the linear relationship identified in different feature map subspaces via visualization of feature maps. Such linear relationship implies that the information in CNNs is redundant. Our method eliminates the redundancy in convolutional filters by applying subspace clustering to feature maps. In this way, most of the representative information in the network can be retained in each cluster. Therefore, our method provides an effective solution to filter pruning for which most existing methods directly remove filters based on simple heuristics. The proposed method is independent of the network structure, thus it can be adopted by any off-the-shelf deep learning libraries. Experiments on different networks and tasks show that our method outperforms existing techniques before fine-tuning, and achieves the state-of-the-art results after fine-tuning.

cs.CV↗

arXiv2018

Time-sensitive Customer Churn Prediction based on PU Learning

With the fast development of Internet companies throughout the world, customer churn has become a serious concern. To better help the companies retain their customers, it is important to build a customer churn prediction model to identify the customers who are most likely to churn ahead of time. In this paper, we propose a Time-sensitive Customer Churn Prediction (TCCP) framework based on Positive and Unlabeled (PU) learning technique. Specifically, we obtain the recent data by shortening the observation period, and start to train model as long as enough positive samples are collected, ignoring the absence of the negative examples. We conduct thoroughly experiments on real industry data from Alipay.com. The experimental results demonstrate that TCCP outperforms the rule-based models and the traditional supervised learning models.

cs.LG↗

arXiv2018

Thermal Conductivity of Suspended Few-Layer MoS2

Modifying phonon thermal conductivity in nanomaterials is important not only for fundamental research but also for practical applications. However, the experiments on tailoring the thermal conductivity in nanoscale, especially in two-dimensional materials, are rare due to technical challenges. In this work, we demonstrate in-situ thermal conduction measurement of MoS2 and find that its thermal conductivity can be continuously tuned to a required value from crystalline to amorphous limits. The reduction of thermal conductivity is understood from phonon-defects scatterings that decrease the phonon transmission coefficient. Beyond a threshold, a sharp drop in thermal conductivity is observed, which is believed to be a crystalline-amorphous transition. Our method and results provide guidance for potential applications in thermoelectrics, photoelectronics, and energy harvesting where thermal management is critical with further integration and miniaturization.

cond-mat.mes-hall↗

arXiv2018

Dimensional crossover of heat conduction in amorphous Polyimide nanofibers

The mechanism of thermal conductivity in amorphous polymers, especially polymer fibers, is unclear in comparison with that in inorganic materials. Here, we report the observation of across over of heat conduction behavior from three dimensions (3D) to quasi-one dimension (1D) in Polyimide(PI) nanofibers at a given temperature. A theoretical model based on the random walk theory has been proposed to quantitatively describe the interplay between the inter-chain hopping and the intra-chain hopping in nanofibers. This model explains well the diameter dependence of thermal conductivity and also speculates the upper limit of thermal conductivity of amorphous polymers in the quasi-1D limit.

physics.app-ph↗

arXiv2018

Optimal Energy Management Strategies in Wireless Data and Energy Cooperative Communications

This paper presents a new cooperative wireless communication network strategy that incorporates energy cooperation and data cooperation. The model establishment, design goal formulations, and algorithms for throughput maximization of the proposed protocol are presented and illustrated using a three-node network with two energy harvesting (EH) user nodes and a destination node. Transmission models are established from the performance analysis for a total of four scenarios. Based on the models, we seek to find optimal energy management strategies by jointly optimizing time allocation for each user, power allocations over these time intervals, and data throughputs at user nodes so as to maximize the sum-throughput or, alternatively, the minimum throughput of the two users in all scenarios. An accelerated Newton barrier algorithm and an alternative algorithm based on local quadratic approximation of the transmission models are developed to solve the aforementioned optimization problems. Numerical experiments under practical settings provide supportive observations to our performance analysis.

eess.SP↗

arXiv2017

Conditional Random Field and Deep Feature Learning for Hyperspectral Image Segmentation

Image segmentation is considered to be one of the critical tasks in hyperspectral remote sensing image processing. Recently, convolutional neural network (CNN) has established itself as a powerful model in segmentation and classification by demonstrating excellent performances. The use of a graphical model such as a conditional random field (CRF) contributes further in capturing contextual information and thus improving the segmentation performance. In this paper, we propose a method to segment hyperspectral images by considering both spectral and spatial information via a combined framework consisting of CNN and CRF. We use multiple spectral cubes to learn deep features using CNN, and then formulate deep CRF with CNN-based unary and pairwise potential functions to effectively extract the semantic correlations between patches consisting of three-dimensional data cubes. Effective piecewise training is applied in order to avoid the computationally expensive iterative CRF inference. Furthermore, we introduce a deep deconvolution network that improves the segmentation masks. We also introduce a new dataset and experimented our proposed method on it along with several widely adopted benchmark datasets to evaluate the effectiveness of our method. By comparing our results with those from several state-of-the-art models, we show the promising potential of our method.

cs.CV↗

arXiv2017

Large valley splitting in monolayer WS$_2$ by proximity coupling to an insulating antiferromagnetic substrate

Lifting the valley degeneracy is an efficient way to achieve valley polarization for further valleytronics operations. In this work, we demonstrate that a large valley splitting can be obtained in monolayer transition metal dichalcogenides by magnetic proximity coupling to an insulating antiferromagnetic substrate. As an example, we perform first-principles calculations to investigate the electronic structures of monolayer WS$_2$ on the MnO(111) surface. Our calculation results suggest that a large valley splitting of 214 meV, which corresponds to a Zeeman magnetic field of 1516 T, is induced in the valence band of monolayer WS$_2$. The magnitude of valley splitting relies on the strength of interfacial orbital hybridization, and can be continually tuned by applying an external out-of-plane pressure and in-plane strain. More interestingly, we find that both spin and valley index will flip when the magnetic ordering of MnO is reversed. Besides, owing to the sizeable Berry curvature and time-reversal symmetry breaking in the WS$_2$/MnO heterostructure, a spin and valley polarized anomalous Hall current can be generated in the presence of an in-plane electric field, which allow one to detect valleys by the electrical approach. Our results shed light on the realization of valleytronic devices using the antiferromagnetic insulator as the substrate.

cond-mat.mtrl-sci↗

arXiv2017

Curve-Structure Segmentation from Depth Maps: A CNN-based Approach and Its Application to Exploring Cultural Heritage Objects

Motivated by the important archaeological application of exploring cultural heritage objects, in this paper we study the challenging problem of automatically segmenting curve structures that are very weakly stamped or carved on an object surface in the form of a highly noisy depth map. Different from most classical low-level image segmentation methods that are known to be very sensitive to the noise and occlusions, we propose a new supervised learning algorithm based on Convolutional Neural Network (CNN) to implicitly learn and utilize more curve geometry and pattern information for addressing this challenging problem. More specifically, we first propose a Fully Convolutional Network (FCN) to estimate the skeleton of curve structures and at each skeleton pixel, a scale value is estimated to reflect the local curve width. Then we propose a dense prediction network to refine the estimated curve skeletons. Based on the estimated scale values, we finally develop an adaptive thresholding algorithm to achieve the final segmentation of curve structures. In the experiment, we validate the performance of the proposed method on a dataset of depth images scanned from unearthed pottery sherds dating to the Woodland period of Southeastern North America.

cs.CV↗