arXiv subjects

Yue Zhao

Publications and source records attributed to Yue Zhao.

At least 343 records · Page 19Linked to original sources

arXiv2022

Federated Learning with Non-IID Data

Federated learning enables resource-constrained edge compute devices, such as mobile phones and IoT devices, to learn a shared model for prediction, while keeping the training data local. This decentralized approach to train models provides privacy, security, regulatory and economic benefits. In this work, we focus on the statistical challenge of federated learning when local data is non-IID. We first show that the accuracy of federated learning reduces significantly, by up to 55% for neural networks trained for highly skewed non-IID data, where each client device trains only on a single class of data. We further show that this accuracy reduction can be explained by the weight divergence, which can be quantified by the earth mover's distance (EMD) between the distribution over classes on each device and the population distribution. As a solution, we propose a strategy to improve training on non-IID data by creating a small subset of data which is globally shared between all the edge devices. Experiments show that accuracy can be increased by 30% for the CIFAR-10 dataset with only 5% globally shared data.

cs.LG↗

arXiv2022

BA-Net: Bridge Attention for Deep Convolutional Neural Networks

In recent years, channel attention mechanism has been widely investigated due to its great potential in improving the performance of deep convolutional neural networks (CNNs) in many vision tasks. However, in most of the existing methods, only the output of the adjacent convolution layer is fed into the attention layer for calculating the channel weights. Information from other convolution layers has been ignored. With these observations, a simple strategy, named Bridge Attention Net (BA-Net), is proposed in this paper for better performance with channel attention mechanisms. The core idea of this design is to bridge the outputs of the previous convolution layers through skip connections for channel weights generation. Based on our experiment and theory analysis, we find that features from previous layers also contribute to the weights significantly. The Comprehensive evaluation demonstrates that the proposed approach achieves state-of-the-art(SOTA) performance compared with the existing methods in accuracy and speed. which shows that Bridge Attention provides a new perspective on the design of neural network architectures with great potential in improving performance. The code is available at https://github.com/zhaoy376/Bridge-Attention.

cs.CV↗

arXiv2022

Improved random batch Ewald method in molecular dynamics simulations

The random batch Ewald (RBE) is an efficient and accurate method for molecular dynamics (MD) simulations of physical systems at the nano-/micro- scale. The method shows great potential to solve the computational bottleneck of long-range interactions, motivating a necessity to accelerating short-range components of the non-bonded interactions for a further speedup of MD simulations. In this work, we present an improved RBE method for the non-bonding interactions by introducing the random batch idea to constructing neighbor lists for the treatment of both the short-range part of the Ewald splitting and the Lennard-Jones potential. The efficiency of the novel neighbor list algorithm owes to the stochastic minibatch strategy which can significantly reduce the total number of neighbors. We obtan the error estimate and convergence by theoretical analysis and implement the improved RBE method in the LAMMPS package. Benchmark simulations are performed to demonstrate the accuracy and stability of the algorithm. Numerical tests on computer performance by conducting large-scaled MD simulations for systems including up to 0.1 billion water molecules, run on massive cluster with up to 50 thousand CPU cores, demonstrating the attractive features such as the high parallel scalability and memory-saving of the method in comparison to the existing methods.

physics.comp-ph↗

arXiv2022

Intrinsically Motivated Self-supervised Learning in Reinforcement Learning

In vision-based reinforcement learning (RL) tasks, it is prevalent to assign auxiliary tasks with a surrogate self-supervised loss so as to obtain more semantic representations and improve sample efficiency. However, abundant information in self-supervised auxiliary tasks has been disregarded, since the representation learning part and the decision-making part are separated. To sufficiently utilize information in auxiliary tasks, we present a simple yet effective idea to employ self-supervised loss as an intrinsic reward, called Intrinsically Motivated Self-Supervised learning in Reinforcement learning (IM-SSR). We formally show that the self-supervised loss can be decomposed as exploration for novel states and robustness improvement from nuisance elimination. IM-SSR can be effortlessly plugged into any reinforcement learning with self-supervised auxiliary objectives with nearly no additional cost. Combined with IM-SSR, the previous underlying algorithms achieve salient improvements on both sample efficiency and generalization in various vision-based robotics tasks from the DeepMind Control Suite, especially when the reward signal is sparse.

cs.LG↗

arXiv2022

Deep Supervised Information Bottleneck Hashing for Cross-modal Retrieval based Computer-aided Diagnosis

Mapping X-ray images, radiology reports, and other medical data as binary codes in the common space, which can assist clinicians to retrieve pathology-related data from heterogeneous modalities (i.e., hashing-based cross-modal medical data retrieval), provides a new view to promot computeraided diagnosis. Nevertheless, there remains a barrier to boost medical retrieval accuracy: how to reveal the ambiguous semantics of medical data without the distraction of superfluous information. To circumvent this drawback, we propose Deep Supervised Information Bottleneck Hashing (DSIBH), which effectively strengthens the discriminability of hash codes. Specifically, the Deep Deterministic Information Bottleneck (Yu, Yu, and Principe 2021) for single modality is extended to the cross-modal scenario. Benefiting from this, the superfluous information is reduced, which facilitates the discriminability of hash codes. Experimental results demonstrate the superior accuracy of the proposed DSIBH compared with state-of-the-arts in cross-modal medical data retrieval tasks.

cs.LG↗

arXiv2022

A Deep Reinforcement Learning Framework for Rapid Diagnosis of Whole Slide Pathological Images

The deep neural network is a research hotspot for histopathological image analysis, which can improve the efficiency and accuracy of diagnosis for pathologists or be used for disease screening. The whole slide pathological image can reach one gigapixel and contains abundant tissue feature information, which needs to be divided into a lot of patches in the training and inference stages. This will lead to a long convergence time and large memory consumption. Furthermore, well-annotated data sets are also in short supply in the field of digital pathology. Inspired by the pathologist's clinical diagnosis process, we propose a weakly supervised deep reinforcement learning framework, which can greatly reduce the time required for network inference. We use neural network to construct the search model and decision model of reinforcement learning agent respectively. The search model predicts the next action through the image features of different magnifications in the current field of view, and the decision model is used to return the predicted probability of the current field of view image. In addition, an expert-guided model is constructed by multi-instance learning, which not only provides rewards for search model, but also guides decision model learning by the knowledge distillation method. Experimental results show that our proposed method can achieve fast inference and accurate prediction of whole slide images without any pixel-level annotations.

eess.IV↗

arXiv2022

Clues Before Answers: Generation-Enhanced Multiple-Choice QA

A trending paradigm for multiple-choice question answering (MCQA) is using a text-to-text framework. By unifying data in different tasks into a single text-to-text format, it trains a generative encoder-decoder model which is both powerful and universal. However, a side effect of twisting a generation target to fit the classification nature of MCQA is the under-utilization of the decoder and the knowledge that can be decoded. To exploit the generation capability and underlying knowledge of a pre-trained encoder-decoder model, in this paper, we propose a generation-enhanced MCQA model named GenMC. It generates a clue from the question and then leverages the clue to enhance a reader for MCQA. It outperforms text-to-text models on multiple MCQA datasets.

cs.CL↗

arXiv2022

Symmetry breaking and anomalous conductivity in a double moiré superlattice

A double moiré superlattice can be realized by stacking three layers of atomically thin two-dimensional materials with designer interlayer twisting or lattice mismatches. In this novel structure, atomic reconstruction of constituent layers could introduce significant modifications to the lattice symmetry and electronic structure at small twist angles. Here, we employ conductive atomic force microscopy (cAFM) to investigate symmetry breaking and local electrical properties in twisted trilayer graphene. We observe clear double moiré superlattices with two distinct moire periods all over the sample. At neighboring domains of the large moiré, the current exhibit either two- or six-fold rotational symmetry, indicating delicate symmetry breaking beyond the rigid model. Moreover, an anomalous current appears at the 'A-A' stacking site of the larger moiré, contradictory to previous observations on twisted bilayer graphene. Both behaviors can be understood by atomic reconstruction, and we also show that the cAFM signal of twisted graphene samples is dominated by the tip-graphene contact resistance that maps the local work function of twisted graphene and the metallic tip qualitatively. Our results unveil cAFM is an effective probe for visualizing atomic reconstruction and symmetry breaking in novel moiré superlattices, which could provide new insights for exploring and manipulating more exotic quantum states based on twisted van der Waals heterostructures.

cond-mat.mes-hall↗

arXiv2022

Gaussian Kernel Variance For an Adaptive Learning Method on Signals Over Graphs

This paper discusses a special kind of a simple yet possibly powerful algorithm, called single-kernel Gradraker (SKG), which is an adaptive learning method predicting unknown nodal values in a network using known nodal values and the network structure. We aim to find out how to configure the special kind of the model in applying the algorithm. To be more specific, we focus on SKG with a Gaussian kernel and specify how to find a suitable variance for the kernel. To do so, we introduce two variables with which we are able to set up requirements on the variance of the Gaussian kernel to achieve (near-) optimal performance and can better understand how SKG works. Our contribution is that we introduce two variables as analysis tools, illustrate how predictions will be affected under different Gaussian kernels, and provide an algorithm finding a suitable Gaussian kernel for SKG with knowledge about the training network. Simulation results on real datasets are provided.

eess.SP↗

arXiv2022

Two-Stream Graph Convolutional Network for Intra-oral Scanner Image Segmentation

Precise segmentation of teeth from intra-oral scanner images is an essential task in computer-aided orthodontic surgical planning. The state-of-the-art deep learning-based methods often simply concatenate the raw geometric attributes (i.e., coordinates and normal vectors) of mesh cells to train a single-stream network for automatic intra-oral scanner image segmentation. However, since different raw attributes reveal completely different geometric information, the naive concatenation of different raw attributes at the (low-level) input stage may bring unnecessary confusion in describing and differentiating between mesh cells, thus hampering the learning of high-level geometric representations for the segmentation task. To address this issue, we design a two-stream graph convolutional network (i.e., TSGCN), which can effectively handle inter-view confusion between different raw attributes to more effectively fuse their complementary information and learn discriminative multi-view geometric representations. Specifically, our TSGCN adopts two input-specific graph-learning streams to extract complementary high-level geometric representations from coordinates and normal vectors, respectively. Then, these single-view representations are further fused by a self-attention module to adaptively balance the contributions of different views in learning more discriminative multi-view representations for accurate and fully automatic tooth segmentation. We have evaluated our TSGCN on a real-patient dataset of dental (mesh) models acquired by 3D intraoral scanners. Experimental results show that our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation. Github: https://github.com/ZhangLingMing1/TSGCNet.

eess.IV↗

arXiv2022

GCN-MIF: Graph Convolutional Network with Multi-Information Fusion for Low-dose CT Denoising

Being low-level radiation exposure and less harmful to health, low-dose computed tomography (LDCT) has been widely adopted in the early screening of lung cancer and COVID-19. LDCT images inevitably suffer from the degradation problem caused by complex noises. It was reported that deep learning (DL)-based LDCT denoising methods using convolutional neural network (CNN) achieved impressive denoising performance. Although most existing DL-based methods (e.g., encoder-decoder framework) can implicitly utilize non-local and contextual information via downsampling operator and 3D CNN, the explicit multi-information (i.e., local, non-local, and contextual) integration may not be explored enough. To address this issue, we propose a novel graph convolutional network-based LDCT denoising model, namely GCN-MIF, to explicitly perform multi-information fusion for denoising purpose. Concretely, by constructing intra- and inter-slice graph, the graph convolutional network is introduced to leverage the non-local and contextual relationships among pixels. The traditional CNN is adopted for the extraction of local information. Finally, the proposed GCN-MIF model fuses all the extracted local, non-local, and contextual information. Extensive experiments show the effectiveness of our proposed GCN-MIF model by quantitative and visualized results. Furthermore, a double-blind reader study on a public clinical dataset is also performed to validate the usability of denoising results in terms of the structural fidelity, the noise suppression, and the overall score. Models and code are available at https://github.com/tonyckc/GCN-MIF_demo.

eess.IV↗

arXiv2022

Producing and detecting long-lived particles at different experiments at the LHC

We propose a new strategy to look for long-lived particles (LLP) at the LHC. The LLPs are produced at one experiment, but its decay products are detected by a detector at another experiment. We use a confining Hidden Valley scenario as a benchmark. Through showering and hadronization, the multiplicity of hidden mesons can be large, and their decay products, dimuon as chosen in this study, are typically too soft to pass triggers in traditional LHC searches. We find the best acceptance is achieved if we produce LLPs at collision points at the LHCb and ALICE experiments, and use the muon chamber of ATLAS for detection. This new search is cost-efficient since it does not require a new detector to be built. Meanwhile, it can provide coverage of interesting parameter space, which is complementary to other proposed LLP searches.

hep-ph↗

arXiv2022

Revisiting Skeleton-based Action Recognition

Human skeleton, as a compact representation of human action, has received increasing attention in recent years. Many skeleton-based action recognition methods adopt graph convolutional networks (GCN) to extract features on top of human skeletons. Despite the positive results shown in previous works, GCN-based methods are subject to limitations in robustness, interoperability, and scalability. In this work, we propose PoseC3D, a new approach to skeleton-based action recognition, which relies on a 3D heatmap stack instead of a graph sequence as the base representation of human skeletons. Compared to GCN-based methods, PoseC3D is more effective in learning spatiotemporal features, more robust against pose estimation noises, and generalizes better in cross-dataset settings. Also, PoseC3D can handle multiple-person scenarios without additional computation cost, and its features can be easily integrated with other modalities at early fusion stages, which provides a great design space to further boost the performance. On four challenging datasets, PoseC3D consistently obtains superior performance, when used alone on skeletons and in combination with the RGB modality.

cs.CV↗

arXiv2022

Combining Individual and Joint Networking Behavior for Intelligent IoT Analytics

The IoT vision of a trillion connected devices over the next decade requires reliable end-to-end connectivity and automated device management platforms. While we have seen successful efforts for maintaining small IoT testbeds, there are multiple challenges for the efficient management of large-scale device deployments. With Industrial IoT, incorporating millions of devices, traditional management methods do not scale well. In this work, we address these challenges by designing a set of novel machine learning techniques, which form a foundation of a new tool, it IoTelligent, for IoT device management, using traffic characteristics obtained at the network level. The design of our tool is driven by the analysis of 1-year long networking data, collected from 350 companies with IoT deployments. The exploratory analysis of this data reveals that IoT environments follow the famous Pareto principle, such as: (i) 10% of the companies in the dataset contribute to 90% of the entire traffic; (ii) 7% of all the companies in the set own 90% of all the devices. We designed and evaluated CNN, LSTM, and Convolutional LSTM models for demand forecasting, with a conclusion of the Convolutional LSTM model being the best. However, maintaining and updating individual company models is expensive. In this work, we design a novel, scalable approach, where a general demand forecasting model is built using the combined data of all the companies with a normalization factor. Moreover, we introduce a novel technique for device management, based on autoencoders. They automatically extract relevant device features to identify device groups with similar behavior to flag anomalous devices.

cs.LG↗

arXiv2022

Stringent axion constraints with Event Horizon Telescope polarimetric measurements of M87$^\star$

The hitherto unprecedented angular resolution of the Event Horizon Telescope (EHT) has created exciting opportunities in the search for new physics. Recently, the linear polarization of radiation emitted near the supermassive black hole M87$^\star$ was measured on four separate days, precisely enabling tests of the existence of a dense axion cloud produced by a spinning black hole. The presence of an axion cloud leads to a frequency-independent oscillation in the electric vector position angle (EVPA) of this linear polarization. For a nearly face-on M87$^\star$, this oscillation in the EVPA appears as a propagating wave along the photon ring. In this paper, we leverage the azimuthal distribution of EVPA measured by the EHT to study the axion-photon coupling. We propose a novel differential analysis procedure to reduce the astrophysical background, and derive stringent constraints on the existence of axions in the previously unexplored mass window $\sim (10^{-21}-10^{-20})$~eV.

hep-ph↗

arXiv2022

Stochastic Gravitational Wave Background from PBH-ABH Mergers

The measurement of gravitational waves produced by binary black-hole mergers at the Advanced LIGO has encouraged extensive studies on the stochastic gravitational wave background. Recent studies have focused on gravitational wave sources made of the same species, such as mergers from binary primordial black holes or those from binary astrophysical black holes. In this paper, we study a new possibility -- the stochastic gravitational wave background produced by mergers of one primordial black hole and one astrophysical black hole. Such systems are necessarily present if primordial black holes exist. We study the isotropic gravitational wave background produced through the history of the Universe. We find it is very challenging to detect such a signal. We also demonstrate that it is improper to treat the gravitational waves produced by such binaries in the Milky Way as a directional stochastic background, due to a very low binary formation rate.

astro-ph.CO↗

arXiv2022

Learning Robust Representation through Graph Adversarial Contrastive Learning

Existing studies show that node representations generated by graph neural networks (GNNs) are vulnerable to adversarial attacks, such as unnoticeable perturbations of adjacent matrix and node features. Thus, it is requisite to learn robust representations in graph neural networks. To improve the robustness of graph representation learning, we propose a novel Graph Adversarial Contrastive Learning framework (GraphACL) by introducing adversarial augmentations into graph self-supervised learning. In this framework, we maximize the mutual information between local and global representations of a perturbed graph and its adversarial augmentations, where the adversarial graphs can be generated in either supervised or unsupervised approaches. Based on the Information Bottleneck Principle, we theoretically prove that our method could obtain a much tighter bound, thus improving the robustness of graph representation learning. Empirically, we evaluate several methods on a range of node classification benchmarks and the results demonstrate GraphACL could achieve comparable accuracy over previous supervised methods.

cs.LG↗

arXiv2022

Stability for the multi-dimensional Borg--Levinson theorem of the biharmonic operator

In this paper, we prove a conditional Hölder stability estimate for the inverse spectral problem of the biharmonic operator. The proof employs the resolvent estimate and a Weyl-type law for the biharmonic operator which were obtained by the authors in \cite{LYZ}. This work extends nontrivially the result in \cite{stefanov} from the second order Schrödinger operator to the fourth order biharmonic operator.

math.AP↗