Searcharxiv⌕ Search

arXiv subjects

Chun Chen

Publications and source records attributed to Chun Chen.

At least 73 records · Page 4Linked to original sources
arXiv2024

Motif-driven Subgraph Structure Learning for Graph Classification

To mitigate the suboptimal nature of graph structure, Graph Structure Learning (GSL) has emerged as a promising approach to improve graph structure and boost performance in downstream tasks. Despite the proposal of numerous GSL methods, the progresses in this field mostly concentrated on node-level tasks, while graph-level tasks (e.g., graph classification) remain largely unexplored. Notably, applying node-level GSL to graph classification is non-trivial due to the lack of find-grained guidance for intricate structure learning. Inspired by the vital role of subgraph in graph classification, in this paper we explore the potential of subgraph structure learning for graph classification by tackling the challenges of key subgraph selection and structure optimization. We propose a novel Motif-driven Subgraph Structure Learning method for Graph Classification (MOSGSL). Specifically, MOSGSL incorporates a subgraph structure learning module which can adaptively select important subgraphs. A motif-driven structure guidance module is further introduced to capture key subgraph-level structural patterns (motifs) and facilitate personalized structure learning. Extensive experiments demonstrate a significant and consistent improvement over baselines, as well as its flexibility and generalizability for various backbones and learning procedures.

cs.LG↗
arXiv2024

Distributionally Robust Graph-based Recommendation System

With the capacity to capture high-order collaborative signals, Graph Neural Networks (GNNs) have emerged as powerful methods in Recommender Systems (RS). However, their efficacy often hinges on the assumption that training and testing data share the same distribution (a.k.a. IID assumption), and exhibits significant declines under distribution shifts. Distribution shifts commonly arises in RS, often attributed to the dynamic nature of user preferences or ubiquitous biases during data collection in RS. Despite its significance, researches on GNN-based recommendation against distribution shift are still sparse. To bridge this gap, we propose Distributionally Robust GNN (DR-GNN) that incorporates Distributional Robust Optimization (DRO) into the GNN-based recommendation. DR-GNN addresses two core challenges: 1) To enable DRO to cater to graph data intertwined with GNN, we reinterpret GNN as a graph smoothing regularizer, thereby facilitating the nuanced application of DRO; 2) Given the typically sparse nature of recommendation data, which might impede robust optimization, we introduce slight perturbations in the training distribution to expand its support. Notably, while DR-GNN involves complex optimization, it can be implemented easily and efficiently. Our extensive experiments validate the effectiveness of DR-GNN against three typical distribution shifts. The code is available at https://github.com/WANGBohaO-jpg/DR-GNN.

cs.IR↗
arXiv2024

Eigenstate properties of the disordered Bose-Hubbard chain

Many-body localization (MBL) of a disordered interacting boson system in one dimension is studied numerically at the filling faction one-half. The von Neumann entanglement entropy SvN is commonly used to detect the MBL phase transition but remains challenging to be directly measured. Based on the U(1) symmetry from the particle number conservation, SvN can be decomposed into the particle number entropy SN and the configuration entropy SC. In light of the tendency that the eigenstate's SC nears zero in the localized phase, we introduce a quantity describing the deviation of SN from the ideal thermalization distribution; finite-size scaling analysis illustrates that it shares the same phase transition point with SvN but displays the better critical exponents. This observation hints that the phase transition to MBL might largely be determined by SN and its fluctuations. Notably, the recent experiments [A. Lukin et al., Science 364, 256 (2019); J. Leonard et al., Nat. Phys. 19, 481 (2023)] demonstrated that this deviation can potentially be measured through the SN measurement. Furthermore, our investigations reveal that the thermalized states primarily occupy the low-energy section of the spectrum, as indicated by measures of localization length, gap ratio, and energy density distribution. This low-energy spectrum of the Bose model closely resembles the entire spectrum of the Fermi (or spin XXZ) model, accommodating a transition from the thermalized to the localized states. While, owing to the bosonic statistics, the high-energy spectrum of the model allows the formation of distinct clusters of bosons in the random potential background. We analyze the resulting eigenstate properties and briefly summarize the associated dynamics. To distinguish between the phase regions at the low and high energies, a probing quantity based on the structure of SvN is also devised.

cond-mat.dis-nn↗
arXiv2024

Knowledge Translation: A New Pathway for Model Compression

Deep learning has witnessed significant advancements in recent years at the cost of increasing training, inference, and model storage overhead. While existing model compression methods strive to reduce the number of model parameters while maintaining high accuracy, they inevitably necessitate the re-training of the compressed model or impose architectural constraints. To overcome these limitations, this paper presents a novel framework, termed \textbf{K}nowledge \textbf{T}ranslation (KT), wherein a ``translation'' model is trained to receive the parameters of a larger model and generate compressed parameters. The concept of KT draws inspiration from language translation, which effectively employs neural networks to convert different languages, maintaining identical meaning. Accordingly, we explore the potential of neural networks to convert models of disparate sizes, while preserving their functionality. We propose a comprehensive framework for KT, introduce data augmentation strategies to enhance model performance despite restricted training data, and successfully demonstrate the feasibility of KT on the MNIST dataset. Code is available at \url{https://github.com/zju-SWJ/KT}.

cs.LG↗
arXiv2023

OpenGSL: A Comprehensive Benchmark for Graph Structure Learning

Graph Neural Networks (GNNs) have emerged as the de facto standard for representation learning on graphs, owing to their ability to effectively integrate graph topology and node attributes. However, the inherent suboptimal nature of node connections, resulting from the complex and contingent formation process of graphs, presents significant challenges in modeling them effectively. To tackle this issue, Graph Structure Learning (GSL), a family of data-centric learning approaches, has garnered substantial attention in recent years. The core concept behind GSL is to jointly optimize the graph structure and the corresponding GNN models. Despite the proposal of numerous GSL methods, the progress in this field remains unclear due to inconsistent experimental protocols, including variations in datasets, data processing techniques, and splitting strategies. In this paper, we introduce OpenGSL, the first comprehensive benchmark for GSL, aimed at addressing this gap. OpenGSL enables a fair comparison among state-of-the-art GSL methods by evaluating them across various popular datasets using uniform data processing and splitting strategies. Through extensive experiments, we observe that existing GSL methods do not consistently outperform vanilla GNN counterparts. We also find that there is no significant correlation between the homophily of the learned structure and task performance, challenging the common belief. Moreover, we observe that the learned graph structure demonstrates a strong generalization ability across different GNN models, despite the high computational and space consumption. We hope that our open-sourced library will facilitate rapid and equitable evaluation and inspire further innovative research in this field. The code of the benchmark can be found in https://github.com/OpenGSL/OpenGSL.

cs.LG↗
arXiv2023

CDR: Conservative Doubly Robust Learning for Debiased Recommendation

In recommendation systems (RS), user behavior data is observational rather than experimental, resulting in widespread bias in the data. Consequently, tackling bias has emerged as a major challenge in the field of recommendation systems. Recently, Doubly Robust Learning (DR) has gained significant attention due to its remarkable performance and robust properties. However, our experimental findings indicate that existing DR methods are severely impacted by the presence of so-called Poisonous Imputation, where the imputation significantly deviates from the truth and becomes counterproductive. To address this issue, this work proposes Conservative Doubly Robust strategy (CDR) which filters imputations by scrutinizing their mean and variance. Theoretical analyses show that CDR offers reduced variance and improved tail bounds.In addition, our experimental investigations illustrate that CDR significantly enhances performance and can indeed reduce the frequency of poisonous imputation.

cs.IR↗
arXiv2023

Graph Neural Networks-based Hybrid Framework For Predicting Particle Crushing Strength

Graph Neural Networks have emerged as an effective machine learning tool for multi-disciplinary tasks such as pharmaceutical molecule classification and chemical reaction prediction, because they can model non-euclidean relationships between different entities. Particle crushing, as a significant field of civil engineering, describes the breakage of granular materials caused by the breakage of particle fragment bonds under the modeling of numerical simulations, which motivates us to characterize the mechanical behaviors of particle crushing through the connectivity of particle fragments with Graph Neural Networks (GNNs). However, there lacks an open-source large-scale particle crushing dataset for research due to the expensive costs of laboratory tests or numerical simulations. Therefore, we firstly generate a dataset with 45,000 numerical simulations and 900 particle types to facilitate the research progress of machine learning for particle crushing. Secondly, we devise a hybrid framework based on GNNs to predict particle crushing strength in a particle fragment view with the advances of state of the art GNNs. Finally, we compare our hybrid framework against traditional machine learning methods and the plain MLP to verify its effectiveness. The usefulness of different features is further discussed through the gradient attribution explanation w.r.t the predictions. Our data and code are released at https://github.com/doujiang-zheng/GNN-For-Particle-Crushing.

cs.LG↗
arXiv2023

Theoretical foundations of studying criticality in the brain

Criticality is hypothesized as a physical mechanism underlying efficient transitions between cortical states and remarkable information processing capacities in the brain. While considerable evidence generally supports this hypothesis, non-negligible controversies persist regarding the ubiquity of criticality in neural dynamics and its role in information processing. Validity issues frequently arise during identifying potential brain criticality from empirical data. Moreover, the functional benefits implied by brain criticality are frequently misconceived or unduly generalized. These problems stem from the non-triviality and immaturity of the physical theories that analytically derive brain criticality and the statistic techniques that estimate brain criticality from empirical data. To help solve these problems, we present a systematic review and reformulate the foundations of studying brain criticality, i.e., ordinary criticality (OC), quasi-criticality (qC), self-organized criticality (SOC), and self-organized quasi-criticality (SOqC), using the terminology of neuroscience. We offer accessible explanations of the physical theories and statistic techniques of brain criticality, providing step-by-step derivations to characterize neural dynamics as a physical system with avalanches. We summarize error-prone details and existing limitations in brain criticality analysis and suggest possible solutions. Moreover, we present a forward-looking perspective on how optimizing the foundations of studying brain criticality can deepen our understanding of various neuroscience questions.

q-bio.NC↗
arXiv2023

Temporal Aggregation and Propagation Graph Neural Networks for Dynamic Representation

Temporal graphs exhibit dynamic interactions between nodes over continuous time, whose topologies evolve with time elapsing. The whole temporal neighborhood of nodes reveals the varying preferences of nodes. However, previous works usually generate dynamic representation with limited neighbors for simplicity, which results in both inferior performance and high latency of online inference. Therefore, in this paper, we propose a novel method of temporal graph convolution with the whole neighborhood, namely Temporal Aggregation and Propagation Graph Neural Networks (TAP-GNN). Specifically, we firstly analyze the computational complexity of the dynamic representation problem by unfolding the temporal graph in a message-passing paradigm. The expensive complexity motivates us to design the AP (aggregation and propagation) block, which significantly reduces the repeated computation of historical neighbors. The final TAP-GNN supports online inference in the graph stream scenario, which incorporates the temporal information into node embeddings with a temporal activation function and a projection layer besides several AP blocks. Experimental results on various real-life temporal networks show that our proposed TAP-GNN outperforms existing temporal graph methods by a large margin in terms of both predictive performance and online inference latency. Our code is available at \url{https://github.com/doujiang-zheng/TAP-GNN}.

cs.LG↗
arXiv2023

Transition Propagation Graph Neural Networks for Temporal Networks

Researchers of temporal networks (e.g., social networks and transaction networks) have been interested in mining dynamic patterns of nodes from their diverse interactions. Inspired by recently powerful graph mining methods like skip-gram models and Graph Neural Networks (GNNs), existing approaches focus on generating temporal node embeddings sequentially with nodes' sequential interactions. However, the sequential modeling of previous approaches cannot handle the transition structure between nodes' neighbors with limited memorization capacity. Detailedly, an effective method for the transition structures is required to both model nodes' personalized patterns adaptively and capture node dynamics accordingly. In this paper, we propose a method, namely Transition Propagation Graph Neural Networks (TIP-GNN), to tackle the challenges of encoding nodes' transition structures. The proposed TIP-GNN focuses on the bilevel graph structure in temporal networks: besides the explicit interaction graph, a node's sequential interactions can also be constructed as a transition graph. Based on the bilevel graph, TIP-GNN further encodes transition structures by multi-step transition propagation and distills information from neighborhoods by a bilevel graph convolution. Experimental results over various temporal networks reveal the efficiency of our TIP-GNN, with at most 7.2\% improvements of accuracy on temporal link prediction. Extensive ablation studies further verify the effectiveness and limitations of the transition propagation module. Our code is available at \url{https://github.com/doujiang-zheng/TIP-GNN}.

cs.IR↗
arXiv2023

Superdiffusive to Ballistic Transports in Nonintegrable Rydberg Chains

A common wisdom posits that transports of conserved quantities across clean nonintegrable quantum systems at high temperatures are diffusive when probed from the emergent hydrodynamic regime. We show that this empirical paradigm may alter if the strong interaction limit is taken. Using Krylov-typicality and purification matrix-product-state methods, we establish the following observations for the strongly interacting version of the mixed-field Ising chain, a nonintegrable lattice model imitating the experimental Rydberg blockade array. Given the strict projection owing to the infinite density-density repulsion $V$, the chain's energy transport in the presence of a transverse field $g$ is superdiffusive at infinite temperature featured by an anomalous scaling exponent $\frac{3}{4}$, indicating the existence of a novel dynamical universality class. Imposing, in addition, a growing longitudinal field $h$ causes a drastic factorization of the whole Hilbert space into smaller subsectors, evidenced by the spectral parsing of the eigenstate entanglement. Being a consequence of this approximate symmetry, a superdiffusion-to-ballistic transport transition arises at $h\approx g$. Interestingly, all the above results persist for large but finite interactions and temperatures, provided that the strongly interacting condition $g,h\ll k_\textrm{B}T\ll V$ is fulfilled. Our predictions are verifiable by current experimental facilities.

cond-mat.str-el↗
arXiv2023

Accelerating Diffusion Sampling with Classifier-based Feature Distillation

Although diffusion model has shown great potential for generating higher quality images than GANs, slow sampling speed hinders its wide application in practice. Progressive distillation is thus proposed for fast sampling by progressively aligning output images of $N$-step teacher sampler with $N/2$-step student sampler. In this paper, we argue that this distillation-based accelerating method can be further improved, especially for few-step samplers, with our proposed \textbf{C}lassifier-based \textbf{F}eature \textbf{D}istillation (CFD). Instead of aligning output images, we distill teacher's sharpened feature distribution into the student with a dataset-independent classifier, making the student focus on those important features to improve performance. We also introduce a dataset-oriented loss to further optimize the model. Experiments on CIFAR-10 show the superiority of our method in achieving high quality and fast sampling. Code is provided at \url{https://github.com/zju-SWJ/RCFD}.

cs.CV↗
arXiv2023

Robust Sequence Networked Submodular Maximization

In this paper, we study the \underline{R}obust \underline{o}ptimization for \underline{se}quence \underline{Net}worked \underline{s}ubmodular maximization (RoseNets) problem. We interweave the robust optimization with the sequence networked submodular maximization. The elements are connected by a directed acyclic graph and the objective function is not submodular on the elements but on the edges in the graph. Under such networked submodular scenario, the impact of removing an element from a sequence depends both on its position in the sequence and in the network. This makes the existing robust algorithms inapplicable. In this paper, we take the first step to study the RoseNets problem. We design a robust greedy algorithm, which is robust against the removal of an arbitrary subset of the selected elements. The approximation ratio of the algorithm depends both on the number of the removed elements and the network topology. We further conduct experiments on real applications of recommendation and link prediction. The experimental results demonstrate the effectiveness of the proposed algorithm.

cs.AI↗
arXiv2022

A possible 250-second X-ray quasi-periodicity in the fast blue optical transient AT2018cow

The fast blue optical transients (FBOTs) are a new population of extragalactic transients of unclear physical origin. A variety of mechanisms have been proposed including failed supernova explosion, shock interaction with a dense medium, young magnetar, accretion onto a compact object, and stellar tidal disruption event, but none is conclusive. Here we report the discovery of a possible X-ray quasi-periodicity signal with a period of $\sim$250 second (at a significance level of 99.76%) in the brightest FBOT AT2018cow through the analysis of XMM-Newton/PN data. The signal is independently detected at the same frequency in the average power density spectrum from data taken from the Swift telescope, with observations covering from 6 to 37 days after the optical discovery, though the significance level is lower (94.26%). This suggests that the QPO frequency may be stable over at least 1.1$\times$ 10$^{4}$ cycles. Assuming the $\sim$250 second QPO to be a scaled-down analogue of that typically seen in stellar mass black holes, a black hole mass of $\sim10^{3}-10^{5}$ solar masses could be inferred. The overall X-ray luminosity evolution could be modeled with the stellar tidal disruption by a black hole of $\sim10^4$ solar masses, providing a viable mechanism to produce AT2018cow. Our findings suggest that other bright FBOTs may also harbor intermediate-mass black holes.

astro-ph.HE↗
arXiv2022

Many-body localization in the infinite-interaction limit and the discontinuous eigenstate phase transition

Can localization persist when interaction grows infinitely stronger than randomness? If so, is it many-body Anderson localization? How about the associated localization transition in the infinite-interaction limit? To tackle these questions, we study many-body localization (MBL) in a spin-chain model mimicking the Rydberg-blockade quantum simulator with both infinite-strength projection and moderate quasiperiodic modulation. Employing exact diagonalization, Krylov-typicality technique, and time-evolving block decimation, we identify evidence for a constrained MBL phase stabilized by a pure quasirandom transverse field. Remarkably, the constrained MBL transition may embody a discontinuous eigenstate phase transition, whose discontinuity nature significantly suppresses the finite-size drifts that plague most numerical studies of conventional MBL transition. Through quantum dynamics, we find that rotating the modulated field from parallel toward perpendicular to the projection axis induces an eigenstate transition between the diagonal and constrained MBL phases. Intriguingly, the entanglement-entropy growth in constrained MBL follows a double-log form, whereas it changes to a power law in approaching the diagonal limit. By unveiling the significance of confined nonlocal effects in integrals of motion of constrained MBL, we show that this newfound insulating state is not a many-body Anderson insulator. Our predictions can be tested in Rydberg experiments.

cond-mat.dis-nn↗
arXiv2022

Lieb-Robinson bound for constrained many-body localization

Study how quantum information propagates through spacetime manifold provides a means of identifying, distinguishing, and classifying novel phases of matter fertilized by many-body effects in strongly interacting systems in and out of equilibrium. Via a fuller characterization of key aspects regarding dynamic behaviors of information, we perform such an analysis on constrained many-body localization -- a newly proposed fully localized state under infinite-interaction limit -- in quasirandom Rydberg blockade spin chain models using thermal out-of-time-order commutators (OTOCs). The OTOC light cones predict a hitherto unknown Lieb-Robinson bound for constrained many-body localization, which is qualitatively different from that of unconstrained many-body Anderson insulators stabilized at weak-interaction limit. Our corroborated numeric and analytic study suggests that constrained many-body localization is a distinct dynamical eigenstate phase whose nonergodicity is beyond local-integral-of-motion phenomenology. Together, these findings consolidate the hierarchy of unconventional quantum dynamics encompassing constrained, unconstrained, and diagonal many-body-localized regimes.

cond-mat.dis-nn↗
arXiv2022

Knowledge Distillation with the Reused Teacher Classifier

Knowledge distillation aims to compress a powerful yet cumbersome teacher model into a lightweight student model without much sacrifice of performance. For this purpose, various approaches have been proposed over the past few years, generally with elaborately designed knowledge representations, which in turn increase the difficulty of model development and interpretation. In contrast, we empirically show that a simple knowledge distillation technique is enough to significantly narrow down the teacher-student performance gap. We directly reuse the discriminative classifier from the pre-trained teacher model for student inference and train a student encoder through feature alignment with a single $\ell_2$ loss. In this way, the student model is able to achieve exactly the same performance as the teacher model provided that their extracted features are perfectly aligned. An additional projector is developed to help the student encoder match with the teacher classifier, which renders our technique applicable to various teacher and student architectures. Extensive experiments demonstrate that our technique achieves state-of-the-art results at the modest cost of compression ratio due to the added projector.

cs.CV↗
arXiv2022

Radiative diffusion in a time-dependent outflow: a model for fast blue optical transients

Fast Blue Optical Transients (FBOTs) are luminous transients with fast evolving (typically $t_{\rm rise}<12\ \rm days$) light curve and blue color (usually $\rm {-0.2\ >\ g-r\ >\ -0.3}$) that cannot be explained by a supernova-like explosion. We propose a radiative diffusion in a time-dependent outflow model to interpret such special transients. In this model, we assume a central engine ejects continuous outflow during a few days. We consider the ejection of the outflow to be time-dependent. The outflow is optically thick initially and photons are frozen in it. As the outflow expands over time, photons gradually escape, and our work is to model such an evolution. Numerical and analytical calculations are considered separately, and the results are consistent. We apply the model to three typical FBOTs: PS1-10bjp, ZTF18abukavn, and ATLAS19dqr. The modeling finds the total mass of the outflow ($\sim 1-5 {M_{\odot}}$), and the total time of the ejection ($\sim$ a few days) for them, leading us to speculate that they may be the result of the collapse of massive stars.

astro-ph.HE↗