arXiv subjects

Wei Hu

Publications and source records attributed to Wei Hu.

At least 289 records · Page 16Linked to original sources

arXiv2019

Influence of point defects on the electronic and topological properties of monolayer WTe$_2$

In some topological insulators, such as graphene and WTe$_2$, band inversion originates from chemical bonding and space group symmetry, in contrast to materials such as Bi$_2$Se$_3$, where the band inversion derives from relativistic effects in the atoms. In the former, band inversion is susceptible to changes of the chemical environment, e.g. by defects, while the latter are less affected by defects due to the larger energy scale associated with atomic relativistic effects. Motivated by recent experiments, we study the effect of Te-vacancies and Te-adatoms on the electronic properties of WTe$_2$. We find that the Te-vacancies have a formation energy of $2.21$ eV, while the formation energy of the Te-adatoms is much lower with $0.72$ eV. The vacancies strongly influence the band structure and we present evidence that band inversion is already reversed at the nominal composition of WTe$_{1.97}$. In contrast, we show that the adatoms do not change the electronic structure in the vicinity of the Fermi level and thus the topological properties. Our findings indicate that Te-adatoms should be present in thin films that are grown in a Te-rich environment, and we suggest that they have been observed in scanning tunneling microscopy experiments.

cond-mat.mtrl-sci↗

arXiv2019

On Exact Computation with an Infinitely Wide Neural Net

How well does a classic deep net architecture like AlexNet or VGG19 classify on a standard dataset such as CIFAR-10 when its width --- namely, number of channels in convolutional layers, and number of nodes in fully-connected internal layers --- is allowed to increase to infinity? Such questions have come to the forefront in the quest to theoretically understand deep learning and its mysteries about optimization and generalization. They also connect deep learning to notions such as Gaussian processes and kernels. A recent paper [Jacot et al., 2018] introduced the Neural Tangent Kernel (NTK) which captures the behavior of fully-connected deep nets in the infinite width limit trained by gradient descent; this object was implicit in some other recent papers. An attraction of such ideas is that a pure kernel-based method is used to capture the power of a fully-trained deep net of infinite width. The current paper gives the first efficient exact algorithm for computing the extension of NTK to convolutional neural nets, which we call Convolutional NTK (CNTK), as well as an efficient GPU implementation of this algorithm. This results in a significant new benchmark for the performance of a pure kernel-based method on CIFAR-10, being $10\%$ higher than the methods reported in [Novak et al., 2019], and only $6\%$ lower than the performance of the corresponding finite deep net architecture (once batch normalization, etc. are turned off). Theoretically, we also give the first non-asymptotic proof showing that a fully-trained sufficiently wide net is indeed equivalent to the kernel regression predictor using NTK.

cs.LG↗

arXiv2019

Enhanced Convolutional Neural Tangent Kernels

Recent research shows that for training with $\ell_2$ loss, convolutional neural networks (CNNs) whose width (number of channels in convolutional layers) goes to infinity correspond to regression with respect to the CNN Gaussian Process kernel (CNN-GP) if only the last layer is trained, and correspond to regression with respect to the Convolutional Neural Tangent Kernel (CNTK) if all layers are trained. An exact algorithm to compute CNTK (Arora et al., 2019) yielded the finding that classification accuracy of CNTK on CIFAR-10 is within 6-7% of that of that of the corresponding CNN architecture (best figure being around 78%) which is interesting performance for a fixed kernel. Here we show how to significantly enhance the performance of these kernels using two ideas. (1) Modifying the kernel using a new operation called Local Average Pooling (LAP) which preserves efficient computability of the kernel and inherits the spirit of standard data augmentation using pixel shifts. Earlier papers were unable to incorporate naive data augmentation because of the quadratic training cost of kernel regression. This idea is inspired by Global Average Pooling (GAP), which we show for CNN-GP and CNTK is equivalent to full translation data augmentation. (2) Representing the input image using a pre-processing technique proposed by Coates et al. (2011), which uses a single convolutional layer composed of random image patches. On CIFAR-10, the resulting kernel, CNN-GP with LAP and horizontal flip data augmentation, achieves 89% accuracy, matching the performance of AlexNet (Krizhevsky et al., 2012). Note that this is the best such result we know of for a classifier that is not a trained neural network. Similar improvements are obtained for Fashion-MNIST.

cs.LG↗

arXiv2019

A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks

We analyze speed of convergence to global optimum for gradient descent training a deep linear neural network (parameterized as $x \mapsto W_N W_{N-1} \cdots W_1 x$) by minimizing the $\ell_2$ loss over whitened data. Convergence at a linear rate is guaranteed when the following hold: (i) dimensions of hidden layers are at least the minimum of the input and output dimensions; (ii) weight matrices at initialization are approximately balanced; and (iii) the initial loss is smaller than the loss of any rank-deficient solution. The assumptions on initialization (conditions (ii) and (iii)) are necessary, in the sense that violating any one of them may lead to convergence failure. Moreover, in the important case of output dimension 1, i.e. scalar regression, they are met, and thus convergence to global optimum holds, with constant probability under a random initialization scheme. Our results significantly extend previous analyses, e.g., of deep linear residual networks (Bartlett et al., 2018).

cs.LG↗

arXiv2019

Implicit Regularization in Deep Matrix Factorization

Efforts to understand the generalization mystery in deep learning have led to the belief that gradient-based optimization induces a form of implicit regularization, a bias towards models of low "complexity." We study the implicit regularization of gradient descent over deep linear neural networks for matrix completion and sensing, a model referred to as deep matrix factorization. Our first finding, supported by theory and experiments, is that adding depth to a matrix factorization enhances an implicit tendency towards low-rank solutions, oftentimes leading to more accurate recovery. Secondly, we present theoretical and empirical arguments questioning a nascent view by which implicit regularization in matrix factorization can be captured using simple mathematical norms. Our results point to the possibility that the language of standard regularizers may not be rich enough to fully encompass the implicit regularization brought forth by gradient-based optimization.

cs.LG↗

arXiv2019

Overview to the Hard X-ray Modulation Telescope (Insight-HXMT) Satellite

As China's first X-ray astronomical satellite, the Hard X-ray Modulation Telescope (HXMT), which was dubbed as Insight-HXMT after the launch on June 15, 2017, is a wide-band (1-250 keV) slat-collimator-based X-ray astronomy satellite with the capability of all-sky monitoring in 0.2-3 MeV. It was designed to perform pointing, scanning and gamma-ray burst (GRB) observations and, based on the Direct Demodulation Method (DDM), the image of the scanned sky region can be reconstructed. Here we give an overview of the mission and its progresses, including payload, core sciences, ground calibration/facility, ground segment, data archive, software, in-orbit performance, calibration, background model, observations and some preliminary results.

astro-ph.IM↗

arXiv2019

The Low Energy X-ray telescope (LE) onboard the Insight-HXMT astronomy satellite

The low energy (LE) X-ray telescope is one of the three main instruments of the Insight-Hard X-ray Modulation Telescope (Insight-HXMT). It is equipped with Swept Charge Device (SCD) sensor arrays with a total geometrical area of 384 cm2 and an energy band from 0.7 keV to 13 keV. In order to evaluate the particle induced X-ray background and the cosmic X-ray background simultaneously, LE adopts collimators to define four types of Field Of Views (FOVs). LE is constituted of three detector boxes (LEDs) and an electric control box (LEB) and achieves a good energy resolution of 140 eV at 5.9 keV, an excellent time resolution of 0.98 ms, as well as an extremely low pileup (<1% at 18000 cts/s). Detailed performance tests and calibration on the ground have been performed, including energy-channel relation, energy response, detection efficiency and time response.

astro-ph.IM↗

arXiv2019

Exploring Structure-Adaptive Graph Learning for Robust Semi-Supervised Classification

Graph Convolutional Neural Networks (GCNNs) are generalizations of CNNs to graph-structured data, in which convolution is guided by the graph topology. In many cases where graphs are unavailable, existing methods manually construct graphs or learn task-driven adaptive graphs. In this paper, we propose Graph Learning Neural Networks (GLNNs), which exploit the optimization of graphs (the adjacency matrix in particular) from both data and tasks. Leveraging on spectral graph theory, we propose the objective of graph learning from a sparsity constraint, properties of a valid adjacency matrix as well as a graph Laplacian regularizer via maximum a posteriori estimation. The optimization objective is then integrated into the loss function of the GCNN, which adapts the graph topology to not only labels of a specific task but also the input data. Experimental results show that our proposed GLNN outperforms state-of-the-art approaches over widely adopted social network datasets and citation network datasets for semi-supervised classification.

cs.LG↗

arXiv2019

Experimental observation of anisotropic diffraction induced by orbital angular momentum

We reveal for the first time a direct relationship between the diffraction of optical beams and their carrying orbital angular momentum (OAM). We experimentally demonstrate a novel phenomenon that the anisotropic diffraction can be induced by the OAM, predicted by us [Opt. Express, \textbf{26}, 8084 (2018)], via the propagations of the elliptic beams with the OAM in linearly and both-linearly-and-nonlinearly isotropic media, respectively. In the former case, when its carrying OAM equals the so-called critical OAM, the spiraling elliptic Gaussian beam (fundamental eigenmode) is observed in the free space, where only the eigenmode with cylindrical-symmetry is supposed to exist for the beam without the OAM. In the latter case, the spiraling elliptic soliton, predicted by Desyatnikov et al. [Phys. Rev. Lett, \textbf{104}, 053902 (2010)], is observed to stably propagate in a cylindrical lead glass. The power-controllable rotation of such an elliptic beam is also experimentally demonstrated.

physics.optics↗

arXiv2019

Joint Learning of Graph Representation and Node Features in Graph Convolutional Neural Networks

Graph Convolutional Neural Networks (GCNNs) extend classical CNNs to graph data domain, such as brain networks, social networks and 3D point clouds. It is critical to identify an appropriate graph for the subsequent graph convolution. Existing methods manually construct or learn one fixed graph for all the layers of a GCNN. In order to adapt to the underlying structure of node features in different layers, we propose dynamic learning of graphs and node features jointly in GCNNs. In particular, we cast the graph optimization problem as distance metric learning to capture pairwise similarities of features in each layer. We deploy the Mahalanobis distance metric and further decompose the metric matrix into a low-dimensional matrix, which converts graph learning to the optimization of a low-dimensional matrix for efficient implementation. Extensive experiments on point clouds and citation network datasets demonstrate the superiority of the proposed method in terms of both accuracies and robustness.

cs.LG↗

arXiv2019

Leveraging Frequent Query Substructures to Generate Formal Queries for Complex Question Answering

Formal query generation aims to generate correct executable queries for question answering over knowledge bases (KBs), given entity and relation linking results. Current approaches build universal paraphrasing or ranking models for the whole questions, which are likely to fail in generating queries for complex, long-tail questions. In this paper, we propose SubQG, a new query generation approach based on frequent query substructures, which helps rank the existing (but nonsignificant) query structures or build new query structures. Our experiments on two benchmark datasets show that our approach significantly outperforms the existing ones, especially for complex questions. Also, it achieves promising performance with limited training data and noisy entity/relation linking results.

cs.CL↗

arXiv2019

Optically active beams: non-reciprocal optical activity in free space induced by spin-orbital interaction of light

Optical activity, the power of a medium to rotate the polarization of a light beam, has contributed significantly to molecular structure assessments in stereochemistry, biomolecular science and crystallography. Thus far, it is commonly believed that optical activity is manifested only in the chiral media which can give rise to circular birefringence of light. Here, we experimentally demonstrate that free space can also support the implementation of non-reciprocal optical activity with Bessel beams by spin-orbital interaction. Specifically, non-diffractive optically active beams are realized, with their optical rotatory power readily controlled by simple optical elements. We show that such free-space optical activity can be exploited to form non-reciprocal optical components such as polarization rotators, isolators, and circulators. Our results may bring about new possibilities of media-independent optical activity to other transverse waves ranging from radio to optical frequencies.

physics.optics↗

arXiv2019

Insight-HXMT observations of Swift J0243.6+6124 during its 2017-2018 outburst

The recently discovered neutron star transient Swift J0243.6+6124 has been monitored by {\it the Hard X-ray Modulation Telescope} ({\it Insight-\rm HXMT). Based on the obtained data, we investigate the broadband spectrum of the source throughout the outburst. We estimate the broadband flux of the source and search for possible cyclotron line in the broadband spectrum. No evidence of line-like features is, however, found up to $\rm 150~keV$. In the absence of any cyclotron line in its energy spectrum, we estimate the magnetic field of the source based on the observed spin evolution of the neutron star by applying two accretion torque models. In both cases, we get consistent results with $B\rm \sim 10^{13}~G$, $D\rm \sim 6~kpc$ and peak luminosity of $\rm >10^{39}~erg~s^{-1}$ which makes the source the first Galactic ultraluminous X-ray source hosting a neutron star.

astro-ph.HE↗

arXiv2019

Non-Volatile Superconductivity in an Insulating Copper Oxide Induced via Ionic Liquid Gating

Manipulating the superconducting states of high-T_c cuprate superconductors in an efficient and reliable way is of great importance for their applications in next-generation electronics. Traditional methods are mostly based on a trial-and-error method that is difficult to implement and time consuming. Here, employing ionic liquid gating, a selective control of volatile and non-volatile superconductivity is achieved in pristine insulating Pr_2CuO_{4\pmδ} film, based on two distinct mechanisms: 1) with positive electric fields, the film can be reversibly switched between non-superconducting and superconducting states, attributed to the carrier doping effect. 2) The film becomes more resistive by applying negative bias voltage up to -4 V, but strikingly, a non-volatile superconductivity is achieved once the gate voltage is removed. Such a persistent superconducting state represents a novel phenomenon in copper oxides, resulting from the doping healing of oxygen vacancies in copper-oxygen planes as unraveled by high-resolution scanning transmission electron microscope and in-situ x-ray diffraction experiments. The effective manipulation and mastering of volatile/non-volatile superconductivity in the same parent cuprate opens the door to more functionalities for superconducting electronics, as well as supplies flexible samples for investigating the nature of quantum phase transitions in high-T_c superconductors.

cond-mat.supr-con↗

arXiv2019

Multi-view Knowledge Graph Embedding for Entity Alignment

We study the problem of embedding-based entity alignment between knowledge graphs (KGs). Previous works mainly focus on the relational structure of entities. Some further incorporate another type of features, such as attributes, for refinement. However, a vast of entity features are still unexplored or not equally treated together, which impairs the accuracy and robustness of embedding-based entity alignment. In this paper, we propose a novel framework that unifies multiple views of entities to learn embeddings for entity alignment. Specifically, we embed entities based on the views of entity names, relations and attributes, with several combination strategies. Furthermore, we design some cross-KG inference methods to enhance the alignment between two KGs. Our experiments on real-world datasets show that the proposed framework significantly outperforms the state-of-the-art embedding-based entity alignment methods. The selected views, cross-KG inference and combination strategies all contribute to the performance improvement.

cs.AI↗

arXiv2019

Emergent superconductivity in single crystalline $\mathrm{MgTi}_2\mathrm{O}_4$ films via structural engineering

Spinel compounds have demonstrated rich functionalities but rarely shown superconductivity. Here, we report the emergence of superconductivity in the spinel $\mathrm{MgTi}_2\mathrm{O}_4$, known to be an insulator with a complicated order. The superconducting transition is achieved by engineering a superlattice of $\mathrm{MgTi}_2\mathrm{O}_4$ and $\mathrm{SrTiO}_3$. The onset transition temperature in the $\mathrm{MgTi}_2\mathrm{O}_4$ layer can be tuned from 0 to 5 K in such geometry, concurrently with a stretched $c$-axis (from 8.51 to 8.53 Å) compared to the bulk material. Such a positive correlation without saturation suggests ample room for the further enhancement. Intriguingly, the superlattice exhibits isotropic upper critical field $H_{\mathrm{c}2}$ that breaks the Pauli limit, distinct from the highly anisotropic feature of interface superconductivity. The origin of superconductivity in the $\mathrm{MgTi}_2\mathrm{O}_4$ layer is understood in combination with the electron energy loss spectra and the first-principles electronic structure calculations, which point to the birth of superconductivity in the $\mathrm{MgTi}_2\mathrm{O}_4$ layer by preventing the Ti-Ti dimerization. Our discovery not only provides a platform to explore the interplay between the superconductivity and other exotic states, but also opens a new window to realize superconductivity in the spinel compounds as well as other titanium oxides.

cond-mat.supr-con↗

arXiv2019

Width Provably Matters in Optimization for Deep Linear Neural Networks

We prove that for an $L$-layer fully-connected linear neural network, if the width of every hidden layer is $\tildeΩ(L \cdot r \cdot d_{\mathrm{out}} \cdot κ^3 )$, where $r$ and $κ$ are the rank and the condition number of the input data, and $d_{\mathrm{out}}$ is the output dimension, then gradient descent with Gaussian random initialization converges to a global minimum at a linear rate. The number of iterations to find an $ε$-suboptimal solution is $O(κ\log(\frac{1}ε))$. Our polynomial upper bound on the total running time for wide deep linear networks and the $\exp\left(Ω\left(L\right)\right)$ lower bound for narrow deep linear neural networks [Shamir, 2018] together demonstrate that wide layers are necessary for optimizing deep models.

cs.LG↗

arXiv2019

Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks

Recent works have cast some light on the mystery of why deep nets fit any data and generalize despite being very overparametrized. This paper analyzes training and generalization for a simple 2-layer ReLU net with random initialization, and provides the following improvements over recent works: (i) Using a tighter characterization of training speed than recent papers, an explanation for why training a neural net with random labels leads to slower training, as originally observed in [Zhang et al. ICLR'17]. (ii) Generalization bound independent of network size, using a data-dependent complexity measure. Our measure distinguishes clearly between random labels and true labels on MNIST and CIFAR, as shown by experiments. Moreover, recent papers require sample complexity to increase (slowly) with the size, while our sample complexity is completely independent of the network size. (iii) Learnability of a broad class of smooth functions by 2-layer ReLU nets trained via gradient descent. The key idea is to track dynamics of training and generalization via properties of a related kernel.

cs.LG↗