arXiv subjects

Lijun Zhang

Publications and source records attributed to Lijun Zhang.

At least 235 records · Page 13Linked to original sources

arXiv2018

A Wasserstein GAN model with the total variational regularization

It is well known that the generative adversarial nets (GANs) are remarkably difficult to train. The recently proposed Wasserstein GAN (WGAN) creates principled research directions towards addressing these issues. But we found in practice that gradient penalty WGANs (GP-WGANs) still suffer from training instability. In this paper, we combine a Total Variational (TV) regularizing term into the WGAN formulation instead of weight clipping or gradient penalty, which implies that the Lipschitz constraint is enforced on the critic network. Our proposed method is more stable at training than GP-WGANs and works well across varied GAN architectures. We also present a method to control the trade-off between image diversity and visual quality. It does not bring any computation burden.

cs.CV↗

arXiv2018

Experimental identification of critical condition for drastically enhancing thermoelectric power factor of two-dimensional layered materials

Nano-structuring is an extremely promising path to high performance thermoelectrics. Favorable improvements in thermal conductivity are attainable in many material systems, and theoretical work points to large improvements in electronic properties. However, realization of the electronic benefits in practical materials has been elusive experimentally. A key challenge is that experimental identification of the quantum confinement length, below which the thermoelectric power factor is significantly enhanced, remains elusive due to lack of simultaneous control of size and carrier density. Here we investigate gate tunable and temperature-dependent thermoelectric transport in $γ$ phase indium selenide ($γ$ InSe, n type semiconductor) samples with thickness varying from 7 to 29 nm. This allows us to properly map out dimension and doping space. Combining theoretical and experimental studies, we reveal that the sharper pre-edge of the conduction-band density of states arising from quantum confinement gives rise to an enhancement of the Seebeck coefficient and the power factor in the thinner InSe samples. Most importantly, we experimentally identify the role of the competition between quantum confinement length and thermal de Broglie wavelength in the enhancement of power factor. Our results provide an important and general experimental guideline for optimizing the power factor and improving the thermoelectric performance of two-dimensional layered semiconductors.

cond-mat.mes-hall↗

arXiv2018

VR-SGD: A Simple Stochastic Variance Reduction Method for Machine Learning

In this paper, we propose a simple variant of the original SVRG, called variance reduced stochastic gradient descent (VR-SGD). Unlike the choices of snapshot and starting points in SVRG and its proximal variant, Prox-SVRG, the two vectors of VR-SGD are set to the average and last iterate of the previous epoch, respectively. The settings allow us to use much larger learning rates, and also make our convergence analysis more challenging. We also design two different update rules for smooth and non-smooth objective functions, respectively, which means that VR-SGD can tackle non-smooth and/or non-strongly convex problems directly without any reduction techniques. Moreover, we analyze the convergence properties of VR-SGD for strongly convex problems, which show that VR-SGD attains linear convergence. Different from its counterparts that have no convergence guarantees for non-strongly convex problems, we also provide the convergence guarantees of VR-SGD for this case, and empirically verify that VR-SGD with varying learning rates achieves similar performance to its momentum accelerated variant that has the optimal convergence rate $\mathcal{O}(1/T^2)$. Finally, we apply VR-SGD to solve various machine learning problems, such as convex and non-convex empirical risk minimization, and leading eigenvalue computation. Experimental results show that VR-SGD converges significantly faster than SVRG and Prox-SVRG, and usually outperforms state-of-the-art accelerated methods, e.g., Katyusha.

cs.LG↗

arXiv2018

$\ell_1$-regression with Heavy-tailed Distributions

In this paper, we consider the problem of linear regression with heavy-tailed distributions. Different from previous studies that use the squared loss to measure the performance, we choose the absolute loss, which is capable of estimating the conditional median. To address the challenge that both the input and output could be heavy-tailed, we propose a truncated minimization problem, and demonstrate that it enjoys an $\widetilde{O}(\sqrt{d/n})$ excess risk, where $d$ is the dimensionality and $n$ is the number of samples. Compared with traditional work on $\ell_1$-regression, the main advantage of our result is that we achieve a high-probability risk bound without exponential moment conditions on the input and output. Furthermore, if the input is bounded, we show that the classical empirical risk minimization is competent for $\ell_1$-regression even when the output is heavy-tailed.

cs.LG↗

arXiv2018

Adaptive Online Learning in Dynamic Environments

In this paper, we study online convex optimization in dynamic environments, and aim to bound the dynamic regret with respect to any sequence of comparators. Existing work have shown that online gradient descent enjoys an $O(\sqrt{T}(1+P_T))$ dynamic regret, where $T$ is the number of iterations and $P_T$ is the path-length of the comparator sequence. However, this result is unsatisfactory, as there exists a large gap from the $Ω(\sqrt{T(1+P_T)})$ lower bound established in our paper. To address this limitation, we develop a novel online method, namely adaptive learning for dynamic environment (Ader), which achieves an optimal $O(\sqrt{T(1+P_T)})$ dynamic regret. The basic idea is to maintain a set of experts, each attaining an optimal dynamic regret for a specific path-length, and combines them with an expert-tracking algorithm. Furthermore, we propose an improved Ader based on the surrogate loss, and in this way the number of gradient evaluations per round is reduced from $O(\log T)$ to $1$. Finally, we extend Ader to the setting that a sequence of dynamical models is available to characterize the comparators.

cs.LG↗

arXiv2018

Query-Efficient Black-Box Attack by Active Learning

Deep neural network (DNN) as a popular machine learning model is found to be vulnerable to adversarial attack. This attack constructs adversarial examples by adding small perturbations to the raw input, while appearing unmodified to human eyes but will be misclassified by a well-trained classifier. In this paper, we focus on the black-box attack setting where attackers have almost no access to the underlying models. To conduct black-box attack, a popular approach aims to train a substitute model based on the information queried from the target DNN. The substitute model can then be attacked using existing white-box attack approaches, and the generated adversarial examples will be used to attack the target DNN. Despite its encouraging results, this approach suffers from poor query efficiency, i.e., attackers usually needs to query a huge amount of times to collect enough information for training an accurate substitute model. To this end, we first utilize state-of-the-art white-box attack methods to generate samples for querying, and then introduce an active learning strategy to significantly reduce the number of queries needed. Besides, we also propose a diversity criterion to avoid the sampling bias. Our extensive experimental results on MNIST and CIFAR-10 show that the proposed method can reduce more than $90\%$ of queries while preserve attacking success rates and obtain an accurate substitute model which is more than $85\%$ similar with the target oracle.

cs.LG↗

arXiv2018

Bismuth and Antimony Based Oxyhalides and Chalcohalides as Potential Optoelectronic Materials

In the last decade the ns$^2$ cations (e.g., Pb$^{2+}$ and Sn$^{2+}$) based halides have emerged as one of the most exciting new classes of optoelectronic materials, as exemplified by for instance hybrid perovskite solar absorbers. These materials not only exhibit unprecedented performance in some cases, but they also appear to break new ground with their unexpected properties, such as extreme tolerance to defects. However, because of the relatively recent emergence of this class of materials, there remain many yet to be fully explored compounds. Here we assess a series of bismuth/antimony oxyhalides and chalcohalides using consistent first principles methods to ascertain their properties and obtain trends. Based on these calculations, we identify a subset consisting of three types of compounds that may be promising as solar absorbers, transparent conductors, and radiation detectors. Their electronic structure, connection to the crystal geometry, and impact on band-edge dispersion and carrier effective mass are discussed.

cond-mat.mtrl-sci↗

arXiv2018

Interlayer Coupling in Two-Dimensional Semiconductor Materials

Two-dimensional (2D) graphene-like layered semiconductors provide a new platform for materials research because of their unique mechanical, electronic and optical attributes. Their in-plane covalent bonding and dangling-bond-free surface allow them to assemble various van der Waals heterostructures (vdWHSs) with sharply atomic interfaces that are not limited by lattice matching and material compatibility. Interlayer coupling, as a ubiquitous phenomenon residing among 2D materials (2DMs) systems, controls a thin layer exfoliation process and the assembly of vdWHSs and behaves with a unique degree of freedom for engineering the properties of 2DMs. Interlayer coupling provides an opportunity to observe new physics and provides a novel strategy to modulate the electronic and optoelectronic properties of materials for practical device applications. We herein review recent progress in the exploration of interlayer coupling in 2D semiconducting vdWHSs for potential applications in electronics and optoelectronics.

cond-mat.mtrl-sci↗

arXiv2018

Intrinsic defect properties in halide double perovskites for optoelectronic applications

Lead-free halide double perovskites with the formula of quaternary A$_2^+$B'B'$^{3+}$X$_6^-$ have recently attracted intense interest as alternatives to lead-halide-perovskite-based optoelectronic materials for their non-toxicity and enhanced chemical and thermodynamic stability. However, the understanding of intrinsic defect properties and their effects on carrier transport and Fermi level tuning is still limited. In this paper, we show that, by exploring the phase diagram of a halide double perovskite, one can control the effects of intrinsic defects on carrier trapping and Fermi level pinning. We reveal the ideal growth conditions to grow p-type Cs$_2$AgInCl$_6$ and Cs$_2$AgBiCl$_6$ as well as semi-insulating Cs$_2$AgBiBr$_6$ with low trap density for targeted photovoltaic or visible-light/radiation detection application.

cond-mat.mtrl-sci↗

arXiv2018

Matrix Completion from Non-Uniformly Sampled Entries

In this paper, we consider matrix completion from non-uniformly sampled entries including fully observed and partially observed columns. Specifically, we assume that a small number of columns are randomly selected and fully observed, and each remaining column is partially observed with uniform sampling. To recover the unknown matrix, we first recover its column space from the fully observed columns. Then, for each partially observed column, we recover it by finding a vector which lies in the recovered column space and consists of the observed entries. When the unknown $m\times n$ matrix is low-rank, we show that our algorithm can exactly recover it from merely $Ω(rn\ln n)$ entries, where $r$ is the rank of the matrix. Furthermore, for a noisy low-rank matrix, our algorithm computes a low-rank approximation of the unknown matrix and enjoys an additive error bound measured by Frobenius norm. Experimental results on synthetic datasets verify our theoretical claims and demonstrate the effectiveness of our proposed algorithm.

cs.LG↗

arXiv2018

Verifying Probabilistic Timed Automata Against Omega-Regular Dense-Time Properties

Probabilistic timed automata (PTAs) are timed automata (TAs) extended with discrete probability distributions.They serve as a mathematical model for a wide range of applications that involve both stochastic and timed behaviours. In this work, we consider the problem of model-checking linear \emph{dense-time} properties over {PTAs}. In particular, we study linear dense-time properties that can be encoded by TAs with infinite acceptance criterion.First, we show that the problem of model-checking PTAs against deterministic-TA specifications can be solved through a product construction. Based on the product construction, we prove that the computational complexity of the problem with deterministic-TA specifications is EXPTIME-complete. Then we show that when relaxed to general (nondeterministic) TAs, the model-checking problem becomes undecidable.Our results substantially extend state of the art with both the dense-time feature and the nondeterminism in TAs.

cs.FL↗

arXiv2018

Dynamic Regret of Strongly Adaptive Methods

To cope with changing environments, recent developments in online learning have introduced the concepts of adaptive regret and dynamic regret independently. In this paper, we illustrate an intrinsic connection between these two concepts by showing that the dynamic regret can be expressed in terms of the adaptive regret and the functional variation. This observation implies that strongly adaptive algorithms can be directly leveraged to minimize the dynamic regret. As a result, we present a series of strongly adaptive algorithms that have small dynamic regrets for convex functions, exponentially concave functions, and strongly convex functions, respectively. To the best of our knowledge, this is the first time that exponential concavity is utilized to upper bound the dynamic regret. Moreover, all of those adaptive algorithms do not need any prior knowledge of the functional variation, which is a significant advantage over previous specialized methods for minimizing dynamic regret.

cs.LG↗

arXiv2018

A thermodynamically consistent approach to describe the effect of thermal vacancy on abnormal thermodynamic behaviors of pure metals: application to body centered cubic W

In this paper, we developed a thermodynamically consistent approach to account for the Gibbs energy of pure metallic element with thermal vacancy over wide temperature range. Taking body centered cubic (bcc) W for a demonstration, the strong nonlinear increase for temperature dependence of heat capacities at high temperatures and a nonlinear Arrhenius plots of vacancy concentration in bcc W can be nicely reproduced by the obtained Gibbs energy. The successful description of thermal vacancy on abnormal thermodynamic behaviors in bcc W indicates that the presently proposed thermodynamically consistent approach is a universal one, and applicable to the other metals.

cond-mat.mtrl-sci↗

arXiv2018

Fast Rates of ERM and Stochastic Approximation: Adaptive to Error Bound Conditions

Error bound conditions (EBC) are properties that characterize the growth of an objective function when a point is moved away from the optimal set. They have recently received increasing attention in the field of optimization for developing optimization algorithms with fast convergence. However, the studies of EBC in statistical learning are hitherto still limited. The main contributions of this paper are two-fold. First, we develop fast and intermediate rates of empirical risk minimization (ERM) under EBC for risk minimization with Lipschitz continuous, and smooth convex random functions. Second, we establish fast and intermediate rates of an efficient stochastic approximation (SA) algorithm for risk minimization with Lipschitz continuous random functions, which requires only one pass of $n$ samples and adapts to EBC. For both approaches, the convergence rates span a full spectrum between $\widetilde O(1/\sqrt{n})$ and $\widetilde O(1/n)$ depending on the power constant in EBC, and could be even faster than $O(1/n)$ in special cases for ERM. Moreover, these convergence rates are automatically adaptive without using any knowledge of EBC. Overall, this work not only strengthens the understanding of ERM for statistical learning but also brings new fast stochastic algorithms for solving a broad range of statistical learning problems.

stat.ML↗

arXiv2018

An Image dehazing approach based on the airlight field estimation

This paper proposes a scheme for single image haze removal based on the airlight field (ALF) estimation. Conventional image dehazing methods which are based on a physical model generally take the global atmospheric light as a constant. However, the constant-airlight assumption may be unsuitable for images with large sky regions, which causes unacceptable brightness imbalance and color distortion in recovery images. This paper models the atmospheric light as a field function, and presents a maximum a-priori (MAP) method for jointly estimating the airlight field, the transmission rate and the haze free image. We also introduce a valid haze-level prior for effective estimate of transmission. Evaluation on real world images shows that the proposed approach outperforms existing methods in single image dehazing, especially when the large sky region is included.

cs.CV↗

arXiv2018

InSe: a two-dimensional material with strong interlayer coupling

Atomically thin, two-dimensional (2D) indium selenide (InSe) has attracted considerable attention due to large tunability in the band gap (from 1.4 to 2.6 eV) and high carrier mobility. The intriguingly high dependence of band gap on layer thickness may lead to novel device applications, although its origin remains poorly understood, and generally attributed to quantum confinement effect. In this work, we demonstrate via first-principles calculations that strong interlayer coupling may be mainly responsible for this phenomenon, especially in the fewer-layer region, and it could also be an essential factor influencing other material properties of β-InSe and γ-InSe. Existence of strong interlayer coupling manifests itself in three aspects: (i) indirect-to-direct band gap transitions with increasing layer thickness; (ii) fan-like frequency diagrams of the shear and breathing modes of few-layer flakes; (iii) strong layer-dependent carrier mobilities. Our results indicate that multiple-layer InSe may be deserving of attention from FET-based technologies and also an ideal system to study interlayer coupling, possibly inherent in other 2D materials.

cond-mat.mtrl-sci↗

arXiv2018

Learning with Feature Evolvable Streams

Learning with streaming data has attracted much attention during the past few years. Though most studies consider data stream with fixed features, in real practice the features may be evolvable. For example, features of data gathered by limited-lifespan sensors will change when these sensors are substituted by new ones. In this paper, we propose a novel learning paradigm: \emph{Feature Evolvable Streaming Learning} where old features would vanish and new features would occur. Rather than relying on only the current features, we attempt to recover the vanished features and exploit it to improve performance. Specifically, we learn two models from the recovered features and the current features, respectively. To benefit from the recovered features, we develop two ensemble methods. In the first method, we combine the predictions from two models and theoretically show that with the assistance of old features, the performance on new features can be improved. In the second approach, we dynamically select the best single prediction and establish a better performance guarantee when the best model switches. Experiments on both synthetic and real data validate the effectiveness of our proposal.

cs.LG↗

arXiv2017

Scalable Demand-Aware Recommendation

Recommendation for e-commerce with a mix of durable and nondurable goods has characteristics that distinguish it from the well-studied media recommendation problem. The demand for items is a combined effect of form utility and time utility, i.e., a product must both be intrinsically appealing to a consumer and the time must be right for purchase. In particular for durable goods, time utility is a function of inter-purchase duration within product category because consumers are unlikely to purchase two items in the same category in close temporal succession. Moreover, purchase data, in contrast to ratings data, is implicit with non-purchases not necessarily indicating dislike. Together, these issues give rise to the positive-unlabeled demand-aware recommendation problem that we pose via joint low-rank tensor completion and product category inter-purchase duration vector estimation. We further relax this problem and propose a highly scalable alternating minimization approach with which we can solve problems with millions of users and millions of items in a single thread. We also show superior prediction accuracies on multiple real-world data sets.

cs.LG↗