arXiv subjects

Jingzhao Zhang

Publications and source records attributed to Jingzhao Zhang.

At least 73 records · Page 4Linked to original sources

arXiv2020

Complexity of Finding Stationary Points of Nonsmooth Nonconvex Functions

We provide the first non-asymptotic analysis for finding stationary points of nonsmooth, nonconvex functions. In particular, we study the class of Hadamard semi-differentiable functions, perhaps the largest class of nonsmooth functions for which the chain rule of calculus holds. This class contains examples such as ReLU neural networks and others with non-differentiable activation functions. We first show that finding an $ε$-stationary point with first-order methods is impossible in finite time. We then introduce the notion of $(δ, ε)$-stationarity, which allows for an $ε$-approximate gradient to be the convex combination of generalized gradients evaluated at points within distance $δ$ to the solution. We propose a series of randomized first-order methods and analyze their complexity of finding a $(δ, ε)$-stationary point. Furthermore, we provide a lower bound and show that our stochastic algorithm has min-max optimal dependence on $δ$. Empirically, our methods perform well for training ReLU neural networks.

math.OC↗

arXiv2020

Why gradient clipping accelerates training: A theoretical justification for adaptivity

We provide a theoretical explanation for the effectiveness of gradient clipping in training deep neural networks. The key ingredient is a new smoothness condition derived from practical neural network training examples. We observe that gradient smoothness, a concept central to the analysis of first-order optimization algorithms that is often assumed to be a constant, demonstrates significant variability along the training trajectory of deep neural networks. Further, this smoothness positively correlates with the gradient norm, and contrary to standard assumptions in the literature, it can grow with the norm of the gradient. These empirical observations limit the applicability of existing theoretical analyses of algorithms that rely on a fixed bound on smoothness. These observations motivate us to introduce a novel relaxation of gradient smoothness that is weaker than the commonly used Lipschitz smoothness assumption. Under the new condition, we prove that two popular methods, namely, \emph{gradient clipping} and \emph{normalized gradient}, converge arbitrarily faster than gradient descent with fixed stepsize. We further explain why such adaptively scaled gradient methods can accelerate empirical convergence and verify our results empirically in popular neural network training settings.

math.OC↗

arXiv2019

Acceleration in First Order Quasi-strongly Convex Optimization by ODE Discretization

We study gradient-based optimization methods obtained by direct Runge-Kutta discretization of the ordinary differential equation (ODE) describing the movement of a heavy-ball under constant friction coefficient. When the function is high order smooth and strongly convex, we show that directly simulating the ODE with known numerical integrators achieve acceleration in a nontrivial neighborhood of the optimal solution. In particular, the neighborhood can grow larger as the condition number of the function increases. Furthermore, our results also hold for nonconvex but quasi-strongly convex objectives. We provide numerical experiments that verify the theoretical rates predicted by our results.

math.OC↗

arXiv2019

Intrinsic long range antiferromagnetic coupling in dilutely V doped CuInTe$_2$

Despite the various magnetic orders mediated by superexchange mechanism, the existence of a long range antiferromagnetic (AFM) coupling is unknown. Based on DFT calculations, we discovered an intrinsic long range AFM coupling in V doped CuInTe$_2$. The AFM coupling is mainly due to the $p-d$ coupling and electron redistribution along the interacting chains. The relatively small energy difference between $p$ and $d$ orbitals and the small energy difference between d orbitals of the dopants and that of stepping stone sites can enhance the stability of this AFM configuration. A multi-bands Hubbard model was proposed to provide fundamental understanding to the long range AFM coupling in chalcopyrite diluted magnetic semiconductors(DMS).

cond-mat.mtrl-sci↗

arXiv2019

Towards understanding the special stability of ${\text{SrCo}\text{O}_{2.5}}$ and ${\text{HSrCo}\text{O}_{2.5}}$

Reversible hydrogen incorporation was recently attested [N. Lu, $\textit{et al.}$, Nature $\textbf{546}$, 124 (2017)] in ${\text{SrCo}\text{O}_{2.5}}$, the brownmillerite phase (BM) of strontium cobalt oxide (SCO), opening new avenues in catalysis and energy applications. However, existing theoretical studies of BM-SCO are insufficient, and that of ${\text{HSrCo}\text{O}_{2.5}}$, the newly-reported hydrogenated SCO (H-SCO), is especially scarce. In this work, we demonstrate how the electron-counting model (ECM) can be used in understanding the phases, particularly in explaining the stability of the oxygen-vacancy channels (OVCs), and in examining the Co valance problem. Using density-functional theoretical (DFT) methods, we analyze the crystalline, electronic, and magnetic structures of BM- and H-SCO. Based on our structure search, we discovered stable phases with large bandgaps (> 1 eV) for both BM-SCO and H-SCO, agreeing better with experiments on the electronic structures. Our calculations also indicate limited charge transfer from H to O that may explain the special stability of the H-SCO phase and the reversibility of H incorporation observed in experiments. In contrary to the initial study, our calculation also suggests intrinsic antiferromagnetism (AFM) of H-SCO, showing how the measured ferromagnetism (FM) has possible roots in hole doping.

cond-mat.mtrl-sci↗

arXiv2018

A Probe Towards Understanding GAN and VAE Models

This project report compares some known GAN and VAE models proposed prior to 2017. There has been significant progress after we finished this report. We upload this report as an introduction to generative models and provide some personal interpretations supported by empirical evidence. Both generative adversarial network models and variational autoencoders have been widely used to approximate probability distributions of data sets. Although they both use parametrized distributions to approximate the underlying data distribution, whose exact inference is intractable, their behaviors are very different. We summarize our experiment results that compare these two categories of models in terms of fidelity and mode collapse. We provide a hypothesis to explain their different behaviors and propose a new model based on this hypothesis. We further tested our proposed model on MNIST dataset and CelebA dataset.

cs.LG↗

arXiv2018

R-SPIDER: A Fast Riemannian Stochastic Optimization Algorithm with Curvature Independent Rate

We study smooth stochastic optimization problems on Riemannian manifolds. Via adapting the recently proposed SPIDER algorithm \citep{fang2018spider} (a variance reduced stochastic method) to Riemannian manifold, we can achieve faster rate than known algorithms in both the finite sum and stochastic settings. Unlike previous works, by \emph{not} resorting to bounding iterate distances, our analysis yields curvature independent convergence rates for both the nonconvex and strongly convex cases.

math.OC↗

arXiv2018

Direct Runge-Kutta Discretization Achieves Acceleration

We study gradient-based optimization methods obtained by directly discretizing a second-order ordinary differential equation (ODE) related to the continuous limit of Nesterov's accelerated gradient method. When the function is smooth enough, we show that acceleration can be achieved by a stable discretization of this ODE using standard Runge-Kutta integrators. Specifically, we prove that under Lipschitz-gradient, convexity and order-$(s+2)$ differentiability assumptions, the sequence of iterates generated by discretizing the proposed second-order ODE converges to the optimal solution at a rate of $\mathcal{O}({N^{-2\frac{s}{s+1}}})$, where $s$ is the order of the Runge-Kutta numerical integrator. Furthermore, we introduce a new local flatness condition on the objective, under which rates even faster than $\mathcal{O}(N^{-2})$ can be achieved with low-order integrators and only gradient information. Notably, this flatness condition is satisfied by several standard loss functions used in machine learning. We provide numerical experiments that verify the theoretical rates predicted by our results.

math.OC↗

arXiv2018

A protonated brownmillerite electrolyte for superior low-temperature proton conductivity

Design novel solid oxide electrolyte with enhanced ionic conductivity forms one of the Holy Grails in the field of materials science due to its great potential for wide range of energy applications. Conventional solid oxide electrolyte typically requires elevated temperature to activate the ionic transportation, while it has been increasing research interests to reduce the operating temperature due to the associated scientific and technological importance. Here, we report a conceptually new solid oxide electrolyte, HSrCoO2.5, which shows an exceptional enhanced proton conductivity at low temperature region (from room temperature to 140 oC). Combining both the experimental results and corresponding first-principles calculations, we attribute these intriguing properties to the extremely-high proton concentration as well as the well-ordered oxygen vacancy channels inherited from the novel crystalline structure of HSrCoO2.5. This result provides a new strategy to design novel solid oxide electrolyte with excellent proton conductivity for wide ranges of energy-related applications.

cond-mat.mtrl-sci↗

arXiv2018

Achieving Acceleration in Distributed Optimization via Direct Discretization of the Heavy-Ball ODE

We develop a distributed algorithm for convex Empirical Risk Minimization, the problem of minimizing large but finite sum of convex functions over networks. The proposed algorithm is derived from directly discretizing the second-order heavy-ball differential equation and results in an accelerated convergence rate, i.e, faster than distributed gradient descent-based methods for strongly convex objectives that may not be smooth. Notably, we achieve acceleration without resorting to the well-known Nesterov's momentum approach. We provide numerical experiments and contrast the proposed method with recently proposed optimal distributed optimization algorithms.

math.OC↗

arXiv2018

Missing links towards understanding equilibrium shapes of hexagonal boron nitride: algorithm, hydrogen passivation, and temperature effects

There is a large discrepancy between the experimental observations and the theoretical predictions in the morphology of hexagonal boron nitride (h-BN) nanosheets. Theoretically-predicted hexagons terminated by armchair edges are not observed in experiments; and experimentally-observed triangles terminated by zigzag edges are found theoretically unstable. There are two key issues in theoretical investigations, namely, an efficient and accurate algorithm of absolute formation energy of h-BN edges, and a good understanding of the role of hydrogen passivation during h-BN growth. Here, we first proposed an efficient algorithm to calculate asymmetric edges with a self-consistent accuracy of about 0.0014 eV/Å. This method can also potentially serve as a standard approach for other two-dimensional (2D) compound materials. Then, by using this method, we discovered that only when edges are passivated by hydrogen atoms and temperature effects are taken into account can experimental morphology be explained. We further employed Wulff construction to obtain the equilibrium shapes of H-passivated h-BN nanosheets under its typical growth conditions at T = 1300 K and p = 1 bar, and found out that the equilibrium shapes are sensitive to hydrogen passivation and the growth conditions. Our results resolved long-standing discrepancies between experimental observations and theoretical analysis, explaining the thermodynamic driving force of the triangular, truncated triangular, and hexagonal shapes, and revealing the key role of hydrogen in h-BN growth. These discoveries and the advancement in algorithm may open the gateway towards the realization of 2D electronic and spintronic devices based on h-BN.

cond-mat.mtrl-sci↗

arXiv2018

Stability of wurtzite semi-polar surfaces: algorithms and practices

A complete knowledge of absolute surface energies with any arbitrary crystal orientation is important for the improvements of semiconductor devices because it determines the equilibrium and nonequilibrium crystal shapes of thin films and nanostructures. It is also crucial in the control of thin film crystal growth and surface effect studies in broad research fields. However, obtaining accurate absolute formation energies is still a huge challenge for the semi-polar surfaces of compound semiconductors. It mainly results from the asymmetry nature of crystal structures and the complicated step morphologies and related reconstructions of these surface configurations. Here we propose a general approach to calculate the absolute formation energies of wurtzite semi-polar surfaces by first-principles calculations, taking GaN as an example. We mainly focused on two commonly seen sets of semi-polar surfaces: a-family (11-2X) and m-family (10-1X). For all the semi-polar surfaces that we have calculated in this paper, the self-consistent accuracy is within 1.5 meV/Å^2. Our work fills the last technical gap to fully investigate and understand the shape and morphology of compound semiconductors.

cond-mat.mtrl-sci↗

arXiv2018

Gate-tunable Room-temperature Ferromagnetism in Two-dimensional Fe$_3$GeTe$_2$

Material research has been a major driving force in the development of modern nano-electronic devices. In particular, research in magnetic thin films has revolutionized the development of spintronic devices; identifying new magnetic materials is key to better device performance and new device paradigm. The advent of two-dimensional van der Waals crystals creates new possibilities. This family of materials retain their chemical stability and structural integrity down to monolayers and, being atomically thin, are readily tuned by various kinds of gate modulation. Recent experiments have demonstrated that it is possible to obtain two-dimensional ferromagnetic order in insulating Cr$_2$Ge$_2$Te$_6$ and CrI$_3$ at low temperatures. Here, we developed a new device fabrication technique, and successfully isolated monolayers from layered metallic magnet Fe$_3$GeTe$_2$ for magnetotransport study. We found that the itinerant ferromagnetism persists in Fe$_3$GeTe$_2$ down to monolayer with an out-of-plane magnetocrystalline anisotropy. The ferromagnetic transition temperature, $T_c$, is suppressed in pristine Fe$_3$GeTe$_2$ thin flakes. An ionic gate, however, dramatically raises the $T_c$ up to room temperature, significantly higher than the bulk $T_c$ of 205 Kelvin. The gate-tunable room-temperature ferromagnetism in two-dimensional Fe$_3$GeTe$_2$ opens up opportunities for potential voltage-controlled magnetoelectronics based on atomically thin van der Waals crystals.

cond-mat.mtrl-sci↗

arXiv2018

Long Range Magnetic order stabilized by acceptors

Tuning magnetic order in magnetic semiconductors is a long sought goal. A proper concentration of acceptors can dramatically suppress local magnetic order in favor of the long one. Using Mn and an acceptor codoped LiZnAs as an example, we demonstrate, by first-principles calculation, the emergence of a long-range magnetic order. This intriguing phenomenon can be understood from an interplay between an acceptor-free magnetism and a band coupling magnetism. Our observation thus lays the ground for a precise control of the magnetic order in future spintronic devices.

cond-mat.mtrl-sci↗

arXiv2017

Hydrogen Surfactant Assisted Coherent Growth of GaN on ZnO Substrate

Heterostructures of wurtzite based devices have attracted great research interests since the tremendous success of GaN in light emitting diodes (LED) industry. Among the possible heterostructure material candidates, high quality GaN thin films on inexpensive and lattice matched ZnO substrate are both commercially and technologically desirable. However, the energy of ZnO polar surfaces is much lower than that of GaN polar surfaces. Therefore, the intrinsic wetting condition forbids such heterostructures. As a result, poor crystal quality and 3D growth mode were obtained. To dramatically change the growth mode of the heterostructure, we propose to use hydrogen as a surfactant, confirmed by our first principles calculations. Stable H involved surface configurations and interfaces are investigated, with the help of newly developed algorithms. By applying the experimental Gibbs free energy of H$_2$, we also predict the temperature and chemical potential of H, which is critical in experimental realizations of our strategy. This novel approach will for the first time make the growth of high quality GaN thin films on ZnO substrates possible. We believe that our new strategy may reduce the manufactory cost and improve the crystal quality and the efficiency of GaN based devices.

cond-mat.mtrl-sci↗

arXiv2017

Defect Properties of Na and K in Cu2ZnSnS4 from Hybrid Functional Calculation

In-growth or post-deposition treatment of $Cu_{2}ZnSnS_{4}$ (CZTS) absorber layer had led to improved photovoltaic efficiency, however, the underlying physical mechanism of such improvements are less studied. In this study, the thermodynamics of Na and K related defects in CZTS are investigated from first principle approach using hybrid functional, with chemical potential of Na and K established from various phases of their polysulphides. Both Na and K predominantly substitute on Cu sites similar to their behavior in $Cu(In,Ga)Se_{2}$, in contrast to previous results using the generalized gradient approximation (GGA). All substitutional and interstitial defects are shown to be either shallow levels or highly energetically unfavorable. Defect complexing between Na and abundant intrinsic defects did not show possibility of significant incorporation enhancement or introducing deep n-type levels. The possible benefit of Na incorporation on enhancing photovoltaic efficiency is discussed. The negligible defect solubility of K in CZTS also suggests possible surfactant candidate.

cond-mat.mtrl-sci↗

arXiv2017

Step Stone Effect: A sp anti-bonding Mediated Long-Range Ferromagnetism in Cr-doped Carrier-Free Bi2Te3

Despite the recent success in the realization of the quantum anomalous Hall effect, the underlying physical mechanism of the long range Ferromagnetism is still unclear. Based on our density functional theory calculations, we discovered an intriguing long-range ferromagnetic order in Cr-doped, carrier-free Bi2Te3, with the separation between dopants more than 8 Å. We found that this magnetic coupling is facilitated by an anti-bonding state originated from the lone pair of a Te 5p state and a Bi 6s state, despite this anti-bonding state lies below the valence band maximum. Such a state serves as a hidden step stone merged in the electron sea and mediates the long-range magnetic order. An effective electron hopping model is proposed to explain this mechanism. This novel mechanism sheds light on the understanding of long-range ferromagnetism in insulators and may lead to the realization of the long-range magnetic order in dilute magnetic semiconductors.

cond-mat.mtrl-sci↗

arXiv2015

Surface energy calculations from Zinc blende (111)/(-1-1-1) to Wurtzite (0001)/(000-1):a study of ZnO and GaN

The accurate absolute surface energies of (0001)/(000-1) surfaces of wurtzite structures are crucial in determining the thin film growth mode of important energy materials. However, the surface energies still remain to be solved due to the intrinsic difficulty of calculating dangling bond energy of asymmetrically bonded surface atoms. In this study, we used a pseudo-hydrogen passivation method to estimate the dangling bond energy and calculate the polar surfaces of ZnO and GaN. The calculations were based on the pseudo chemical potentials obtained from a set of tetrahedral clusters or simple pseudo-molecules, using density functional theory approaches. And the surface energies of (0001)/(000-1) surfaces of wurtzite ZnO and GaN we obtained showed relatively high self-consistencies. A wedge structure calculation with a new bottom surface passivation scheme of group I and group VII elements was also proposed and performed to show converged absolute surface energy of wurtzite ZnO polar surfaces, and the result were also compared with the above method. These calculations and comparisons may provide important insights to crystal growths of the above materials, thereby leading to significant performance enhancements of semiconductor devices.

cond-mat.mtrl-sci↗