arXiv subjects

Lingxiao Zhao

Publications and source records attributed to Lingxiao Zhao.

At least 55 records · Page 3Linked to original sources

arXiv2023

DSV: An Alignment Validation Loss for Self-supervised Outlier Model Selection

Self-supervised learning (SSL) has proven effective in solving various problems by generating internal supervisory signals. Unsupervised anomaly detection, which faces the high cost of obtaining true labels, is an area that can greatly benefit from SSL. However, recent literature suggests that tuning the hyperparameters (HP) of data augmentation functions is crucial to the success of SSL-based anomaly detection (SSAD), yet a systematic method for doing so remains unknown. In this work, we propose DSV (Discordance and Separability Validation), an unsupervised validation loss to select high-performing detection models with effective augmentation HPs. DSV captures the alignment between an augmentation function and the anomaly-generating mechanism with surrogate losses, which approximate the discordance and separability of test data, respectively. As a result, the evaluation via DSV leads to selecting an effective SSAD model exhibiting better alignment, which results in high detection accuracy. We theoretically derive the degree of approximation conducted by the surrogate losses and empirically show that DSV outperforms a wide range of baselines on 21 real-world tasks.

cs.LG↗

arXiv2023

Signature of weakly coupled $f$ electrons and conduction electrons in magnetic Weyl semimetal candidates PrAlSi and SmAlSi

Magnetic topological materials are a class of compounds with the underlying interplay of nontrivial band topology and magnetic spin configuration. Extensive interests have been aroused due to their application potential involved with an array of exotic quantum states. With angle-resolved photoemission spectroscopy and first-principles calculations, here we study the electronic properties of two magnetic Weyl semimetal candidates PrAlSi and SmAlSi. Though the two compounds harbor distinct magnetic ground states (ferromagnetic and antiferromagnetic for PrAlSi and SmAlSi, respectively) and 4$f$ shell fillings, we find that they share quite analogous low-energy band structure. By the measurements across the magnetic transitions, we further reveal that there is no evident evolution of the band structure in both compounds and the experimental spectra can be well reproduced by the nonmagnetic calculations, together suggesting a negligible effect of the magnetism on their electronic structures and a possibly weak coupling between the localized 4$f$ electrons and the itinerant conduction electrons. Our results offer essential insights into the interactions between magnetism, electron correlations, and topological orders in the $R$Al$X$ ($R$ = light rare earth and $X$ = Si or Ge) family.

cond-mat.mtrl-sci↗

arXiv2023

Field-Induced Lifshitz Transition in the Magnetic Weyl Semimetal Candidate PrAlSi

Lifshitz transition (LT) refers to an abrupt change in the electronic structure and Fermi surface, and is associated to a variety of emergent quantum phenomena. Amongst the LTs observed in known materials, the field-induced LT has been rare and its origin remains elusive. To understand the origin of field-induced LT, it is important to extend the material basis beyond the usual setting of heavy fermion metals. Here, we report on a field-induced LT in PrAlSi, a magnetic Weyl semimetal candidate with localized 4$f$ electrons, through a study of magnetotransport up to 55 T. The quantum oscillation analysis reveals that across a threshold field $B^*\approx$14.5 T the oscillation frequency ($F_1$ = 43 T) is replaced by two new frequencies ($F_2$ = 62 T and $F_3$ = 103 T). Strikingly, the LT occurs well below quantum limit, with obvious temperature-dependent oscillation frequency and field-dependent cyclotron mass. Our work not only enriches the rare examples of field-induced LTs, but also paves the way for further investigation on the interplay among topology, magnetism and electronic correlation.

cond-mat.str-el↗

arXiv2022

A Practical, Progressively-Expressive GNN

Message passing neural networks (MPNNs) have become a dominant flavor of graph neural networks (GNNs) in recent years. Yet, MPNNs come with notable limitations; namely, they are at most as powerful as the 1-dimensional Weisfeiler-Leman (1-WL) test in distinguishing graphs in a graph isomorphism testing frame-work. To this end, researchers have drawn inspiration from the k-WL hierarchy to develop more expressive GNNs. However, current k-WL-equivalent GNNs are not practical for even small values of k, as k-WL becomes combinatorially more complex as k grows. At the same time, several works have found great empirical success in graph learning tasks without highly expressive models, implying that chasing expressiveness with a coarse-grained ruler of expressivity like k-WL is often unneeded in practical tasks. To truly understand the expressiveness-complexity tradeoff, one desires a more fine-grained ruler, which can more gradually increase expressiveness. Our work puts forth such a proposal: Namely, we first propose the (k, c)(<=)-SETWL hierarchy with greatly reduced complexity from k-WL, achieved by moving from k-tuples of nodes to sets with <=k nodes defined over <=c connected components in the induced original graph. We show favorable theoretical results for this model in relation to k-WL, and concretize it via (k, c)(<=)-SETGNN, which is as expressive as (k, c)(<=)-SETWL. Our model is practical and progressively-expressive, increasing in power with k and c. We demonstrate effectiveness on several benchmark datasets, achieving several state-of-the-art results with runtime and memory usage applicable to practical graphs. We open source our implementation at https://github.com/LingxiaoShawn/KCSetGNN.

cs.LG↗

arXiv2022

Graph Anomaly Detection with Unsupervised GNNs

Graph-based anomaly detection finds numerous applications in the real-world. Thus, there exists extensive literature on the topic that has recently shifted toward deep detection models due to advances in deep learning and graph neural networks (GNNs). A vast majority of prior work focuses on detecting node/edge/subgraph anomalies within a single graph, with much less work on graph-level anomaly detection in a graph database. This work aims to fill two gaps in the literature: We (1) design GLAM, an end-to-end graph-level anomaly detection model based on GNNs, and (2) focus on unsupervised model selection, which is notoriously hard due to lack of any labels, yet especially critical for deep NN based models with a long list of hyper-parameters. Further, we propose a new pooling strategy for graph-level embedding, called MMD-pooling, that is geared toward detecting distribution anomalies which has not been considered before. Through extensive experiments on 15 real-world datasets, we show that (i) GLAM outperforms node-level and two-stage (i.e. not end-to-end) baselines, and (ii) model selection picks a significantly more effective model than expectation (i.e. average) -- without using any labels -- among candidates with otherwise large variation in performance.

cs.LG↗

arXiv2022

Hyperparameter Sensitivity in Deep Outlier Detection: Analysis and a Scalable Hyper-Ensemble Solution

Outlier detection (OD) literature exhibits numerous algorithms as it applies to diverse domains. However, given a new detection task, it is unclear how to choose an algorithm to use, nor how to set its hyperparameter(s) (HPs) in unsupervised settings. HP tuning is an ever-growing problem with the arrival of many new detectors based on deep learning, which usually come with a long list of HPs. Surprisingly, the issue of model selection in the outlier mining literature has been "the elephant in the room"; a significant factor in unlocking the utmost potential of deep methods, yet little said or done to systematically tackle the issue. In the first part of this paper, we conduct the first large-scale analysis on the HP sensitivity of deep OD methods, and through more than 35,000 trained models, quantitatively demonstrate that model selection is inevitable. Next, we design a HP-robust and scalable deep hyper-ensemble model called ROBOD that assembles models with varying HP configurations, bypassing the choice paralysis. Importantly, we introduce novel strategies to speed up ensemble training, such as parameter sharing, batch/simultaneous training, and data subsampling, that allow us to train fewer models with fewer parameters. Extensive experiments on both image and tabular datasets show that ROBOD achieves and retains robust, state-of-the-art detection performance as compared to its modern counterparts, while taking only $2$-$10$\% of the time by the naive hyper-ensemble with independent training.

cs.LG↗

arXiv2022

Sign and Basis Invariant Networks for Spectral Graph Representation Learning

We introduce SignNet and BasisNet -- new neural architectures that are invariant to two key symmetries displayed by eigenvectors: (i) sign flips, since if $v$ is an eigenvector then so is $-v$; and (ii) more general basis symmetries, which occur in higher dimensional eigenspaces with infinitely many choices of basis eigenvectors. We prove that under certain conditions our networks are universal, i.e., they can approximate any continuous function of eigenvectors with the desired invariances. When used with Laplacian eigenvectors, our networks are provably more expressive than existing spectral methods on graphs; for instance, they subsume all spectral graph convolutions, certain spectral graph invariants, and previously proposed graph positional encodings as special cases. Experiments show that our networks significantly outperform existing baselines on molecular graph regression, learning expressive graph representations, and learning neural fields on triangle meshes. Our code is available at https://github.com/cptq/SignNet-BasisNet .

cs.LG↗

arXiv2022

Graph Condensation for Graph Neural Networks

Given the prevalence of large-scale graphs in real-world applications, the storage and time for training neural models have raised increasing concerns. To alleviate the concerns, we propose and study the problem of graph condensation for graph neural networks (GNNs). Specifically, we aim to condense the large, original graph into a small, synthetic and highly-informative graph, such that GNNs trained on the small graph and large graph have comparable performance. We approach the condensation problem by imitating the GNN training trajectory on the original graph through the optimization of a gradient matching loss and design a strategy to condense node futures and structural information simultaneously. Extensive experiments have demonstrated the effectiveness of the proposed framework in condensing different graph datasets into informative smaller graphs. In particular, we are able to approximate the original test accuracy by 95.3% on Reddit, 99.8% on Flickr and 99.0% on Citeseer, while reducing their graph size by more than 99.9%, and the condensed graphs can be used to train various GNN architectures.Code is released at https://github.com/ChandlerBang/GCond.

cs.LG↗

arXiv2022

From Stars to Subgraphs: Uplifting Any GNN with Local Structure Awareness

Message Passing Neural Networks (MPNNs) are a common type of Graph Neural Network (GNN), in which each node's representation is computed recursively by aggregating representations (messages) from its immediate neighbors akin to a star-shaped pattern. MPNNs are appealing for being efficient and scalable, how-ever their expressiveness is upper-bounded by the 1st-order Weisfeiler-Lehman isomorphism test (1-WL). In response, prior works propose highly expressive models at the cost of scalability and sometimes generalization performance. Our work stands between these two regimes: we introduce a general framework to uplift any MPNN to be more expressive, with limited scalability overhead and greatly improved practical performance. We achieve this by extending local aggregation in MPNNs from star patterns to general subgraph patterns (e.g.,k-egonets):in our framework, each node representation is computed as the encoding of a surrounding induced subgraph rather than encoding of immediate neighbors only (i.e. a star). We choose the subgraph encoder to be a GNN (mainly MPNNs, considering scalability) to design a general framework that serves as a wrapper to up-lift any GNN. We call our proposed method GNN-AK(GNN As Kernel), as the framework resembles a convolutional neural network by replacing the kernel with GNNs. Theoretically, we show that our framework is strictly more powerful than 1&2-WL, and is not less powerful than 3-WL. We also design subgraph sampling strategies which greatly reduce memory footprint and improve speed while maintaining performance. Our method sets new state-of-the-art performance by large margins for several well-known graph ML tasks; specifically, 0.08 MAE on ZINC,74.79% and 86.887% accuracy on CIFAR10 and PATTERN respectively.

cs.LG↗

arXiv2022

Electronic structure and open-orbit Fermi surface topology in isostructural semimetals NbAs$_2$ and W$_2$As$_3$ with extremely large magnetoresistance

In transition-metal dipnictides $TmPn_2$ ($Tm$ = Ta, Nb; $Pn$ = P, As, Sb), the origin of extremely large magnetoresistance (XMR) is yet to be studied by the direct visualization of the experimental band structures. Here, using angle-resolved photoemission spectroscopy, we map out the three-dimensional electronic structure of NbAs$_2$. The open-orbit topology contributes to a non-negligible part of the Fermi surfaces (FSs), like that of the isostructural compound MoAs$_2$, where the open FS is proposed to likely explain the origin of XMR. We further demonstrate the observation of open characters in the overall FSs of W$_2$As$_3$, which is also a XMR semimetal with the same space group of $C$12/$m$1 as $TmPn_2$ family and MoAs$_2$. Our results suggest that the open-orbit FS topology may be a shared feature between XMR materials with the space group of $C$12/$m$1, and thus could possibly play a role in determining the corresponding XMR effect together with the electron-hole compensation.

cond-mat.mtrl-sci↗

arXiv2021

Fast Attributed Graph Embedding via Density of States

Given a node-attributed graph, how can we efficiently represent it with few numerical features that expressively reflect its topology and attribute information? We propose A-DOGE, for Attributed DOS-based Graph Embedding, based on density of states (DOS, a.k.a. spectral density) to tackle this problem. A-DOGE is designed to fulfill a long desiderata of desirable characteristics. Most notably, it capitalizes on efficient approximation algorithms for DOS, that we extend to blend in node labels and attributes for the first time, making it fast and scalable for large attributed graphs and graph databases. Being based on the entire eigenspectrum of a graph, A-DOGE can capture structural and attribute properties at multiple ("glocal") scales. Moreover, it is unsupervised (i.e. agnostic to any specific objective) and lends itself to various interpretations, which makes it is suitable for exploratory graph mining tasks. Finally, it processes each graph independent of others, making it amenable for streaming settings as well as parallelization. Through extensive experiments, we show the efficacy and efficiency of A-DOGE on exploratory graph analysis and graph classification tasks, where it significantly outperforms unsupervised baselines and achieves competitive performance with modern supervised GNNs, while achieving the best trade-off between accuracy and runtime.

cs.LG↗

arXiv2021

Shubnikov-de Haas Oscillations and Nontrivial Topological State in a New Weyl Semimetal Candidate SmAlSi

We perform the quantum magnetotransport measurements and first-principles calculations on high quality single crystals of SmAlSi, a new topological Weyl semimetal candidate. At low temperatures, SmAlSi exhibits large non-saturated magnetoresistance (MR)~5200% (at 2 K, 48 T) and prominent Shubnikov-de Haas (SdH) oscillations, where MRs follow the power-law field dependence with exponent 1.52 at low fields (μ0H < 15 T) and linear behavior 1 under high fields (μ0H > 18 T). The analysis of angle dependent SdH oscillations reveal two fundamental frequencies originated from the Fermi surface (FS) pockets with non-trivial π Berry phases, small cyclotron mass and electron-hole compensation with high mobility at 2 K. In combination with the calculated nontrivial electronic band structure, SmAlSi is proposed to be a paradigm for understanding the Weyl fermions in the topological materials.

cond-mat.mtrl-sci↗

arXiv2021

On Using Classification Datasets to Evaluate Graph-Level Outlier Detection: Peculiar Observations and New Insights

It is common practice of the outlier mining community to repurpose classification datasets toward evaluating various detection models. To that end, often a binary classification dataset is used, where samples from one of the classes is designated as the inlier samples, and the other class is substantially down-sampled to create the ground-truth outlier samples. Graph-level outlier detection (GLOD) is rarely studied but has many potentially influential real-world applications. In this study, we identify an intriguing issue with repurposing graph classification datasets for GLOD. We find that ROC-AUC performance of the models changes significantly (flips from high to very low, even worse than random) depending on which class is down-sampled. Interestingly, ROC-AUCs on these two variants approximately sum to 1 and their performance gap is amplified with increasing propagations for a certain family of propagation based outlier detection models. We carefully study the graph embedding space produced by propagation based models and find two driving factors: (1) disparity between within-class densities which is amplified by propagation, and (2)overlapping support (mixing of embeddings) across classes. We also study other graph embedding methods and downstream outlier detectors, and find that the intriguing performance flip issue still widely exists but which version of the downsample achieves higher performance may vary. Thoughtful analysis over comprehensive results further deeper our understanding of the established issue.

cs.LG↗

arXiv2021

Connecting Graph Convolutional Networks and Graph-Regularized PCA

Graph convolution operator of the GCN model is originally motivated from a localized first-order approximation of spectral graph convolutions. This work stands on a different view; establishing a \textit{mathematical connection between graph convolution and graph-regularized PCA} (GPCA). Based on this connection, GCN architecture, shaped by stacking graph convolution layers, shares a close relationship with stacking GPCA. We empirically demonstrate that the \textit{unsupervised} embeddings by GPCA paired with a 1- or 2-layer MLP achieves similar or even better performance than GCN on semi-supervised node classification tasks across five datasets including Open Graph Benchmark \footnote{\url{https://ogb.stanford.edu/}}. This suggests that the prowess of GCN is driven by graph based regularization. In addition, we extend GPCA to the (semi-)supervised setting and show that it is equivalent to GPCA on a graph extended with "ghost" edges between nodes of the same label. Finally, we capitalize on the discovered relationship to design an effective initialization strategy based on stacking GPCA, enabling GCN to converge faster and achieve robust performance at large number of layers. Notably, the proposed initialization is general-purpose and applies to other GNNs.

cs.LG↗

arXiv2020

Beyond Homophily in Graph Neural Networks: Current Limitations and Effective Designs

We investigate the representation power of graph neural networks in the semi-supervised node classification task under heterophily or low homophily, i.e., in networks where connected nodes may have different class labels and dissimilar features. Many popular GNNs fail to generalize to this setting, and are even outperformed by models that ignore the graph structure (e.g., multilayer perceptrons). Motivated by this limitation, we identify a set of key designs -- ego- and neighbor-embedding separation, higher-order neighborhoods, and combination of intermediate representations -- that boost learning from the graph structure under heterophily. We combine them into a graph neural network, H2GCN, which we use as the base method to empirically evaluate the effectiveness of the identified designs. Going beyond the traditional benchmarks with strong homophily, our empirical analysis shows that the identified designs increase the accuracy of GNNs by up to 40% and 27% over models without them on synthetic and real networks with heterophily, respectively, and yield competitive performance under homophily.

cs.LG↗

arXiv2020

Graph Unrolling Networks: Interpretable Neural Networks for Graph Signal Denoising

We propose an interpretable graph neural network framework to denoise single or multiple noisy graph signals. The proposed graph unrolling networks expand algorithm unrolling to the graph domain and provide an interpretation of the architecture design from a signal processing perspective. We unroll an iterative denoising algorithm by mapping each iteration into a single network layer where the feed-forward process is equivalent to iteratively denoising graph signals. We train the graph unrolling networks through unsupervised learning, where the input noisy graph signals are used to supervise the networks. By leveraging the learning ability of neural networks, we adaptively capture appropriate priors from input noisy graph signals, instead of manually choosing signal priors. A core component of graph unrolling networks is the edge-weight-sharing graph convolution operation, which parameterizes each edge weight by a trainable kernel function where the trainable parameters are shared by all the edges. The proposed convolution is permutation-equivariant and can flexibly adjust the edge weights to various graph signals. We then consider two special cases of this class of networks, graph unrolling sparse coding (GUSC) and graph unrolling trend filtering (GUTF), by unrolling sparse coding and trend filtering, respectively. To validate the proposed methods, we conduct extensive experiments on both real-world datasets and simulated datasets, and demonstrate that our methods have smaller denoising errors than conventional denoising algorithms and state-of-the-art graph neural networks. For denoising a single smooth graph signal, the normalized mean square error of the proposed networks is around 40% and 60% lower than that of graph Laplacian denoising and graph wavelets, respectively.

eess.SP↗

arXiv2020

PairNorm: Tackling Oversmoothing in GNNs

The performance of graph neural nets (GNNs) is known to gradually decrease with increasing number of layers. This decay is partly attributed to oversmoothing, where repeated graph convolutions eventually make node embeddings indistinguishable. We take a closer look at two different interpretations, aiming to quantify oversmoothing. Our main contribution is PairNorm, a novel normalization layer that is based on a careful analysis of the graph convolution operator, which prevents all node embeddings from becoming too similar. What is more, PairNorm is fast, easy to implement without any change to network architecture nor any additional parameters, and is broadly applicable to any GNN. Experiments on real-world graphs demonstrate that PairNorm makes deeper GCN, GAT, and SGC models more robust against oversmoothing, and significantly boosts performance for a new problem setting that benefits from deeper GNNs. Code is available at https://github.com/LingxiaoShawn/PairNorm.

cs.LG↗

arXiv2020

PrBi: Topology meets quadrupolar degrees of freedom

Novel materials incorporating electronic degrees of freedom other than charge, including spin, orbital or valley \textit{et al} have manifested themselves to be of the great interests and applicable potentials. Recently, the multipolar degrees of freedom have attracted remarkable attention in the electronic correlated effects. In this work, we systematically studied the transport, magnetic and thermodynamic properties of the topological semimetal candidate PrBi in the framework of crystalline electric field theory. Our results demonstrate the $Γ_3$ non-Kramers doublet as the ground state of Pr$^{3+}$ (4$f^2$) ions. This ground state is nonmagnetic but carries a non-zero quadrupolar moment $\langle\hat{O}_2^0\rangle$. A quadrupolar phase transition is inferred below 0.08 K. No obvious quadrupolar Kondo effect can be identified. Ultrahigh-field quantum oscillation measurements confirm PrBi as a semimetal with non-trivial Berry phase and low total carrier density 0.06 /f.u. We discuss the interplay between low carrier density and $4f^2$ quadrupolar moment, and ascribe the weak quadrupolar ordering and Kondo effect to consequences of the low carrier density. PrBi, thus, opens a new window to the physics of topology and strongly correlated effect with quadrupolar degrees of freedom in the low-carrier-density limit, evoking the need for a reexamination of the Nozières exhaustion problem in the context of multi-channel Kondo effect.

cond-mat.str-el↗