arXiv subjects

Jun Zhou

Publications and source records attributed to Jun Zhou.

At least 397 records · Page 22Linked to original sources

arXiv2022

Transfer Attacks Revisited: A Large-Scale Empirical Study in Real Computer Vision Settings

One intriguing property of adversarial attacks is their "transferability" -- an adversarial example crafted with respect to one deep neural network (DNN) model is often found effective against other DNNs as well. Intensive research has been conducted on this phenomenon under simplistic controlled conditions. Yet, thus far, there is still a lack of comprehensive understanding about transferability-based attacks ("transfer attacks") in real-world environments. To bridge this critical gap, we conduct the first large-scale systematic empirical study of transfer attacks against major cloud-based MLaaS platforms, taking the components of a real transfer attack into account. The study leads to a number of interesting findings which are inconsistent to the existing ones, including: (1) Simple surrogates do not necessarily improve real transfer attacks. (2) No dominant surrogate architecture is found in real transfer attacks. (3) It is the gap between posterior (output of the softmax layer) rather than the gap between logit (so-called $κ$ value) that increases transferability. Moreover, by comparing with prior works, we demonstrate that transfer attacks possess many previously unknown properties in real-world environments, such as (1) Model similarity is not a well-defined concept. (2) $L_2$ norm of perturbation can generate high transferability without usage of gradient and is a more powerful source than $L_\infty$ norm. We believe this work sheds light on the vulnerabilities of popular MLaaS platforms and points to a few promising research directions.

cs.CV↗

arXiv2022

Revisiting Domain Generalized Stereo Matching Networks from a Feature Consistency Perspective

Despite recent stereo matching networks achieving impressive performance given sufficient training data, they suffer from domain shifts and generalize poorly to unseen domains. We argue that maintaining feature consistency between matching pixels is a vital factor for promoting the generalization capability of stereo matching networks, which has not been adequately considered. Here we address this issue by proposing a simple pixel-wise contrastive learning across the viewpoints. The stereo contrastive feature loss function explicitly constrains the consistency between learned features of matching pixel pairs which are observations of the same 3D points. A stereo selective whitening loss is further introduced to better preserve the stereo feature consistency across domains, which decorrelates stereo features from stereo viewpoint-specific style information. Counter-intuitively, the generalization of feature consistency between two viewpoints in the same scene translates to the generalization of stereo matching performance to unseen domains. Our method is generic in nature as it can be easily embedded into existing stereo networks and does not require access to the samples in the target domain. When trained on synthetic data and generalized to four real-world testing sets, our method achieves superior performance over several state-of-the-art networks.

cs.CV↗

arXiv2022

Benchmarking emergency department triage prediction models with machine learning and large public electronic health records

The demand for emergency department (ED) services is increasing across the globe, particularly during the current COVID-19 pandemic. Clinical triage and risk assessment have become increasingly challenging due to the shortage of medical resources and the strain on hospital infrastructure caused by the pandemic. As a result of the widespread use of electronic health records (EHRs), we now have access to a vast amount of clinical data, which allows us to develop predictive models and decision support systems to address these challenges. To date, however, there are no widely accepted benchmark ED triage prediction models based on large-scale public EHR data. An open-source benchmarking platform would streamline research workflows by eliminating cumbersome data preprocessing, and facilitate comparisons among different studies and methodologies. In this paper, based on the Medical Information Mart for Intensive Care IV Emergency Department (MIMIC-IV-ED) database, we developed a publicly available benchmark suite for ED triage predictive models and created a benchmark dataset that contains over 400,000 ED visits from 2011 to 2019. We introduced three ED-based outcomes (hospitalization, critical outcomes, and 72-hour ED reattendance) and implemented a variety of popular methodologies, ranging from machine learning methods to clinical scoring systems. We evaluated and compared the performance of these methods against benchmark tasks. Our codes are open-source, allowing anyone with MIMIC-IV-ED data access to perform the same steps in data processing, benchmark model building, and experiments. This study provides future researchers with insights, suggestions, and protocols for managing raw data and developing risk triaging tools for emergency care.

cs.LG↗

arXiv2022

Chiral Phonon Activated Spin Seebeck Effect

Efficient generation of spin polarization is the central focus of spintronics. In magnetic materials, spin currents can arise from heat currents by the conventional spin Seebeck effect. Recently, chiral phonons with definite handedness and angular momenta have also produced profound impacts on multiple research fields. In this paper, starting with nonequilibrium distribution of chiral phonons under temperature gradient, we find a new spin selectivity effect - chiral phonon activated spin Seebeck (CPASS) effect, in chiral materials without magnetic order nor spin-orbit coupling. With both phonon-drag and band transport contributions, the CPASS coefficients are computed based on the Boltzmann transport theory. The spin accumulations by the CPASS effect quadratically increase with temperature gradient, and vary with the chemical potential modulation, thus enabling highly efficient and tunable spin generation. The CPASS effect provides a promising explanation on the chiral-induced spin selectivity effect and opportunities for designing advanced spintronic devices based on nonmagnetic chiral materials.

cond-mat.mes-hall↗

arXiv2022

CrowdMLP: Weakly-Supervised Crowd Counting via Multi-Granularity MLP

Existing state-of-the-art crowd counting algorithms rely excessively on location-level annotations, which are burdensome to acquire. When only count-level (weak) supervisory signals are available, it is arduous and error-prone to regress total counts due to the lack of explicit spatial constraints. To address this issue, a novel and efficient counter (referred to as CrowdMLP) is presented, which probes into modelling global dependencies of embeddings and regressing total counts by devising a multi-granularity MLP regressor. In specific, a locally-focused pre-trained frontend is cascaded to extract crude feature maps with intrinsic spatial cues, which prevent the model from collapsing into trivial outcomes. The crude embeddings, along with raw crowd scenes, are tokenized at different granularity levels. The multi-granularity MLP then proceeds to mix tokens at the dimensions of cardinality, channel, and spatial for mining global information. An effective proxy task, namely Split-Counting, is also proposed to evade the barrier of limited samples and the shortage of spatial hints in a self-supervised manner. Extensive experiments demonstrate that CrowdMLP significantly outperforms existing weakly-supervised counting algorithms and performs on par with state-of-the-art location-level supervised approaches.

cs.CV↗

arXiv2022

Neural Graph Matching for Pre-training Graph Neural Networks

Recently, graph neural networks (GNNs) have been shown powerful capacity at modeling structural data. However, when adapted to downstream tasks, it usually requires abundant task-specific labeled data, which can be extremely scarce in practice. A promising solution to data scarcity is to pre-train a transferable and expressive GNN model on large amounts of unlabeled graphs or coarse-grained labeled graphs. Then the pre-trained GNN is fine-tuned on downstream datasets with task-specific fine-grained labels. In this paper, we present a novel Graph Matching based GNN Pre-Training framework, called GMPT. Focusing on a pair of graphs, we propose to learn structural correspondences between them via neural graph matching, consisting of both intra-graph message passing and inter-graph message passing. In this way, we can learn adaptive representations for a given graph when paired with different graphs, and both node- and graph-level characteristics are naturally considered in a single pre-training task. The proposed method can be applied to fully self-supervised pre-training and coarse-grained supervised pre-training. We further propose an approximate contrastive training strategy to significantly reduce time/memory consumption. Extensive experiments on multi-domain, out-of-distribution benchmarks have demonstrated the effectiveness of our approach. The code is available at: https://github.com/RUCAIBox/GMPT.

cs.LG↗

arXiv2022

An Effective Graph Learning based Approach for Temporal Link Prediction: The First Place of WSDM Cup 2022

Temporal link prediction, as one of the most crucial work in temporal graphs, has attracted lots of attention from the research area. The WSDM Cup 2022 seeks for solutions that predict the existence probabilities of edges within time spans over temporal graph. This paper introduces the solution of AntGraph, which wins the 1st place in the competition. We first analysis the theoretical upper-bound of the performance by removing temporal information, which implies that only structure and attribute information on the graph could achieve great performance. Based on this hypothesis, then we introduce several well-designed features. Finally, experiments conducted on the competition datasets show the superiority of our proposal, which achieved AUC score of 0.666 on dataset A and 0.902 on dataset B, the ablation studies also prove the efficiency of each feature. Code is publicly available at https://github.com/im0qianqian/WSDM2022TGP-AntGraph.

cs.SI↗

arXiv2022

Towards Scalable and Privacy-Preserving Deep Neural Network via Algorithmic-Cryptographic Co-design

Deep Neural Networks (DNNs) have achieved remarkable progress in various real-world applications, especially when abundant training data are provided. However, data isolation has become a serious problem currently. Existing works build privacy preserving DNN models from either algorithmic perspective or cryptographic perspective. The former mainly splits the DNN computation graph between data holders or between data holders and server, which demonstrates good scalability but suffers from accuracy loss and potential privacy risks. In contrast, the latter leverages time-consuming cryptographic techniques, which has strong privacy guarantee but poor scalability. In this paper, we propose SPNN - a Scalable and Privacy-preserving deep Neural Network learning framework, from algorithmic-cryptographic co-perspective. From algorithmic perspective, we split the computation graph of DNN models into two parts, i.e., the private data related computations that are performed by data holders and the rest heavy computations that are delegated to a server with high computation ability. From cryptographic perspective, we propose using two types of cryptographic techniques, i.e., secret sharing and homomorphic encryption, for the isolated data holders to conduct private data related computations privately and cooperatively. Furthermore, we implement SPNN in a decentralized setting and introduce user-friendly APIs. Experimental results conducted on real-world datasets demonstrate the superiority of SPNN.

cs.LG↗

arXiv2022

Confidence May Cheat: Self-Training on Graph Neural Networks under Distribution Shift

Graph Convolutional Networks (GCNs) have recently attracted vast interest and achieved state-of-the-art performance on graphs, but its success could typically hinge on careful training with amounts of expensive and time-consuming labeled data. To alleviate labeled data scarcity, self-training methods have been widely adopted on graphs by labeling high-confidence unlabeled nodes and then adding them to the training step. In this line, we empirically make a thorough study for current self-training methods on graphs. Surprisingly, we find that high-confidence unlabeled nodes are not always useful, and even introduce the distribution shift issue between the original labeled dataset and the augmented dataset by self-training, severely hindering the capability of self-training on graphs. To this end, in this paper, we propose a novel Distribution Recovered Graph Self-Training framework (DR-GST), which could recover the distribution of the original labeled dataset. Specifically, we first prove the equality of loss function in self-training framework under the distribution shift case and the population distribution if each pseudo-labeled node is weighted by a proper coefficient. Considering the intractability of the coefficient, we then propose to replace the coefficient with the information gain after observing the same changing trend between them, where information gain is respectively estimated via both dropout variational inference and dropedge variational inference in DR-GST. However, such a weighted loss function will enlarge the impact of incorrect pseudo labels. As a result, we apply the loss correction method to improve the quality of pseudo labels. Both our theoretical analysis and extensive experiments on five benchmark datasets demonstrate the effectiveness of the proposed DR-GST, as well as each well-designed component in DR-GST.

cs.LG↗

arXiv2022

Unravelling Distance-Dependent Inter-Site Interactions and Magnetic Transition Effects of Heteronuclear Single Atom Catalysts on Electrochemical Oxygen Reduction

Inter-site interactions between single atom catalysts (SACs) in the high loading regime are critical to tuning the catalytic performance. However, the understanding on such interactions and their distance dependent effects remains elusive, especially for the heteronuclear SACs. In this study, we reveal the effects of the distance-dependent inter-site interaction on the catalytic performance of SACs. Using the density functional theory calculations, we systematically investigate the heteronuclear iron and cobalt single atoms co-supported on the nitrogen-doped graphene (FeN4-C and CoN4-C) for oxygen reduction reaction (ORR). We find that as the distance between Fe and Co SACs decreases, FeN4-C exhibits a reduced catalytic activity, which can be mitigated by the presence of an axial hydroxyl ligand, whereas the activity of CoN4-C shows a volcano-like evolution with the optimum reached at the intermediate distance. We further unravel that the transition towards the high-spin state upon adsorption of ORR intermediate adsorbates is responsible for the decreased activity of both FeN4-C and CoN4-C at short inter-site distance. Such high-spin state transition is also found to significantly shift the linear relation between hydroxyl (*OH) and hydroperoxyl (*OOH) adsorbates. These findings not only shed light on the SAC-specific effect of the distance-dependent inter-site interaction between heteronuclear SACs, but also pave a way towards shifting the long-standing linear relations observed in multiple-electron chemical reactions.

physics.chem-ph↗

arXiv2022

Interpretable learning of voltage for electrode design of multivalent metal-ion batteries

Deep learning (DL) has indeed emerged as a powerful tool for rapidly and accurately predicting materials properties from big data, such as the design of current commercial Li-ion batteries. However, its practical utility for multivalent metal-ion batteries (MIBs), the most promising future solution of large-scale energy storage, is limited due to the scarce MIB data availability and poor DL model interpretability. Here, we develop an interpretable DL model as an effective and accurate method for learning electrode voltages of multivalent MIBs (divalent magnesium, calcium, zinc, and trivalent aluminum) at small dataset limits (150~500). Using the experimental results as validation, our model is much more accurate than machine-learning models which usually are better than DL in the small dataset regime. Besides the high accuracy, our feature-engineering-free DL model is explainable, which automatically extracts the atom covalent radius as the most important feature for the voltage learning by visualizing vectors from the layers of the neural network. The presented model potentially accelerates the design and optimization of multivalent MIB materials with fewer data and less domain-knowledge restriction, and is implemented into a publicly available online tool kit in http://batteries.2dmatpedia.org/ for the battery community.

cond-mat.mtrl-sci↗

arXiv2021

High-throughput Discovery and Intelligent Design of 2D Functional Materials for Various Applications

Novel technologies and new materials are in high demand for future energy-efficient electronic devices to overcome the fundamental limitations of miniaturization of current silicon-based devices. Two-dimensional (2D) materials show promising applications in the next generation devices because they can be tailored on the specific property that a technology is based on, and be compatible with other technologies, such as the silicon-based (opto)electronics. Although the number of experimentally discovered 2D materials is growing, the speed is very slow and only a few dozen 2D materials have been synthesized or exfoliated since the discovery of graphene. Recently, a novel computational technique, dubbed "high-throughput computational materials design", becomes a burgeoning area of materials science, which is the combination of the quantum-mechanical theory, materials genome, and database construction with intelligent data mining. This new and powerful tool can greatly accelerate the discovery, design and application of 2D materials by creating database containing a large amount of 2D materials with calculated fundamental properties, and then intelligently mining (via high-throughput automation or machine learning) the database in the search of 2D materials with the desired properties for particular applications, such as energy conversion, electronics, spintronics, and optoelectronics.

physics.comp-ph↗

arXiv2021

SAIL: Self-Augmented Graph Contrastive Learning

This paper studies learning node representations with graph neural networks (GNNs) for unsupervised scenario. Specifically, we derive a theoretical analysis and provide an empirical demonstration about the non-steady performance of GNNs over different graph datasets, when the supervision signals are not appropriately defined. The performance of GNNs depends on both the node feature smoothness and the locality of graph structure. To smooth the discrepancy of node proximity measured by graph topology and node feature, we proposed SAIL - a novel \underline{S}elf-\underline{A}ugmented graph contrast\underline{i}ve \underline{L}earning framework, with two complementary self-distilling regularization modules, \emph{i.e.}, intra- and inter-graph knowledge distillation. We demonstrate the competitive performance of SAIL on a variety of graph applications. Even with a single GNN layer, SAIL has consistently competitive or even better performance on various benchmark datasets, comparing with state-of-the-art baselines.

cs.LG↗

arXiv2021

Role of Magnon-Magnon Scattering in Magnon Polaron Spin Seebeck Effect

The spin Seebeck effect (SSE) signal of magnon polarons in bulk-Y3Fe5O12 (YIG)/Pt heterostructures is found to drastically change as a function of temperature. It appears as a dip in the total SSE signal at low temperatures, but as the temperature increases, the dip gradually decreases before turning to a peak. We attribute the observed dip-to-peak transition to the rapid rise of the four-magnon scattering rate. Our analysis provides important insights into the microscopic origin of the hybridized excitations and the overall temperature dependence of the SSE anomalies.

cond-mat.mtrl-sci↗

arXiv2021

Sliding Ferroelectric Tunnel Junctions

Very recently, ferroelectric polarization in staggered bilayer hexagonal boron nitride (BBN) and its novel sliding inversion mechanism was reported experimentally (Science 2021, 372, 1458; 2021, 372, 1462), which paves a new way to realize van der Waals (vdW) ferroelectric devices with new functionalities. Here, we develop vdW sliding ferroelectric tunnel junctions (FTJs) using the sliding ferroelectric BBN unit as ultrathin barriers and explore their transport properties with different ferroelectric states and metal contacts via the first principles. It is found that the electrode/BBN contact electric field quenches the ferroelectricity in the staggered BBN, resulting a very small tunnelling electroresistance (TER). Inserting high-mobility 2D materials between Au and BN can restore the BBN ferroelectricity, reaching a giant TER of ~10,000% in sliding FTJs. We finally investigate the metal-contact and thickness effect on the tunnelling property of sliding FTJs. The giant TER and multiple non-volatile resistance states in vdW sliding FTJs show the promising applications in voltage-controlled nano-memories with ultrahigh storage density.

cond-mat.mtrl-sci↗

arXiv2021

Dimension reduction induced anisotropic magnetic thermal conductivity in hematite nanowire

The thermophysical properties near the magnetic phase transition point is of great importance in the study of critical phenomenon. Low-dimensional materials are suggested to hold different thermophysical properties comparing to their bulk counterpart due to the dimension induced quantum confinement and anisotropy. In this work, we measured the thermal conductivity of $α$-Fe$_2$O$_3$ nanowires along [110] direction (growing direction) with temperature from 100K to 150K and found a dip of thermal conductivity near the Morin temperature. We found the thermal conductivity near Morin temperature varies with the angle between magnetic field and [110] direction of nanowire. More specifically, an angular-dependent thermal conductivity is observed, due to the magnetic field induced movement of magnetic domain wall. The angle corresponding to the maximum of thermal conductivity varies near the Morin transition temperature, due to the different magnetic easy axis as suggested by our calculation based on magnetic anisotropy energy. This angular dependence of thermal conductivity indicates that the easy axis of $α$-Fe$_2$O$_3$ nanowires is different from bulk $α$-Fe$_2$O$_3$ due to the geometric anisotropy.

cond-mat.mes-hall↗

arXiv2021

SMDS-Net: Model Guided Spectral-Spatial Network for Hyperspectral Image Denoising

Deep learning (DL) based hyperspectral images (HSIs) denoising approaches directly learn the nonlinear mapping between observed noisy images and underlying clean images. They normally do not consider the physical characteristics of HSIs, therefore making them lack of interpretability that is key to understand their denoising mechanism.. In order to tackle this problem, we introduce a novel model guided interpretable network for HSI denoising. Specifically, fully considering the spatial redundancy, spectral low-rankness and spectral-spatial properties of HSIs, we first establish a subspace based multi-dimensional sparse model. This model first projects the observed HSIs into a low-dimensional orthogonal subspace, and then represents the projected image with a multidimensional dictionary. After that, the model is unfolded into an end-to-end network named SMDS-Net whose fundamental modules are seamlessly connected with the denoising procedure and optimization of the model. This makes SMDS-Net convey clear physical meanings, i.e., learning the low-rankness and sparsity of HSIs. Finally, all key variables including dictionaries and thresholding parameters are obtained by the end-to-end training. Extensive experiments and comprehensive analysis confirm the denoising ability and interpretability of our method against the state-of-the-art HSI denoising methods.

eess.IV↗

arXiv2021

Multi-Interactive Attention Network for Fine-grained Feature Learning in CTR Prediction

In the Click-Through Rate (CTR) prediction scenario, user's sequential behaviors are well utilized to capture the user interest in the recent literature. However, despite being extensively studied, these sequential methods still suffer from three limitations. First, existing methods mostly utilize attention on the behavior of users, which is not always suitable for CTR prediction, because users often click on new products that are irrelevant to any historical behaviors. Second, in the real scenario, there exist numerous users that have operations a long time ago, but turn relatively inactive in recent times. Thus, it is hard to precisely capture user's current preferences through early behaviors. Third, multiple representations of user's historical behaviors in different feature subspaces are largely ignored. To remedy these issues, we propose a Multi-Interactive Attention Network (MIAN) to comprehensively extract the latent relationship among all kinds of fine-grained features (e.g., gender, age and occupation in user-profile). Specifically, MIAN contains a Multi-Interactive Layer (MIL) that integrates three local interaction modules to capture multiple representations of user preference through sequential behaviors and simultaneously utilize the fine-grained user-specific as well as context information. In addition, we design a Global Interaction Module (GIM) to learn the high-order interactions and balance the different impacts of multiple features. Finally, Offline experiment results from three datasets, together with an Online A/B test in a large-scale recommendation system, demonstrate the effectiveness of our proposed approach.

cs.IR↗