arXiv subjects

Liang Chen

Publications and source records attributed to Liang Chen.

At least 343 records · Page 19Linked to original sources

arXiv2022

Spiking Graph Convolutional Networks

Graph Convolutional Networks (GCNs) achieve an impressive performance due to the remarkable representation ability in learning the graph information. However, GCNs, when implemented on a deep network, require expensive computation power, making them difficult to be deployed on battery-powered devices. In contrast, Spiking Neural Networks (SNNs), which perform a bio-fidelity inference process, offer an energy-efficient neural architecture. In this work, we propose SpikingGCN, an end-to-end framework that aims to integrate the embedding of GCNs with the biofidelity characteristics of SNNs. The original graph data are encoded into spike trains based on the incorporation of graph convolution. We further model biological information processing by utilizing a fully connected layer combined with neuron nodes. In a wide range of scenarios (e.g. citation networks, image graph classification, and recommender systems), our experimental results show that the proposed method could gain competitive performance against state-of-the-art approaches. Furthermore, we show that SpikingGCN on a neuromorphic chip can bring a clear advantage of energy efficiency into graph data analysis, which demonstrates its great potential to construct environment-friendly machine learning models.

cs.LG↗

arXiv2022

Re-thinking and Re-labeling LIDC-IDRI for Robust Pulmonary Cancer Prediction

The LIDC-IDRI database is the most popular benchmark for lung cancer prediction. However, with subjective assessment from radiologists, nodules in LIDC may have entirely different malignancy annotations from the pathological ground truth, introducing label assignment errors and subsequent supervision bias during training. The LIDC database thus requires more objective labels for learning-based cancer prediction. Based on an extra small dataset containing 180 nodules diagnosed by pathological examination, we propose to re-label LIDC data to mitigate the effect of original annotation bias verified on this robust benchmark. We demonstrate in this paper that providing new labels by similar nodule retrieval based on metric learning would be an effective re-labeling strategy. Training on these re-labeled LIDC nodules leads to improved model performance, which is enhanced when new labels of uncertain nodules are added. We further infer that re-labeling LIDC is current an expedient way for robust lung cancer prediction while building a large pathological-proven nodule database provides the long-term solution.

eess.IV↗

arXiv2022

Hunting for neutrino emission from multi-frequency variable sources

Pinpointing the neutrino sources is crucial to unveil the mystery of high-energy cosmic rays. The search for neutrino-source candidates from coincident neutrino-photon signatures and electromagnetic objects with peculiar flaring behaviors have the potential to increase our chances of finding neutrino emitters. In this paper, we first study the temporal correlations of astrophysical flares with neutrinos, considering a few hundreds of multi-frequency sources from ALMA, WISE, Swift, and Fermi in the containment regions of IceCube high-energy alerts. Furthermore, the spatial correlations between blazars and neutrinos are investigated using the subset of 10-year IceCube track-like neutrinos with around 250 thousand events. In the second test, we account for 2700 blazars with different types of flaring phases in addition to sole position. No significant neutrino emissions were found from our analyses. Our results indicate an interesting trend showing the infrared flaring stages of WISE blazars might be correlated with arrival times of the neutrino alerts. Possible overflow of neutrinos associated with two of our blazar sub-samples are also illustrated. One is characterized by a significant flaring lag in infrared with respect to gamma-rays, like seen for TXS0506+056, and the other is characterized by highly simultaneous infrared and gamma-ray flares. These phenomena suggest the need to improve current multi-frequency light-curve catalogs to pair with the advent of more sensitive neutrino observatories.

astro-ph.HE↗

arXiv2022

Detection of DC electric forces with zeptonewton sensitivity by single-ion phonon laser

Detecting extremely small forces helps exploring new physics quantitatively. Here we demonstrate that the phonon laser made of a single trapped $^{40}$Ca$^{+}$ ion behaves as an exquisite sensor for small force measurement. We report our successful detection of small electric forces regarding the DC trapping potential with sensitivity of 2.41$\pm$0.49 zN/$\sqrt{\rm Hz}$, with the ion only under Doppler cooling, based on the injection-locking of the oscillation phase of the phonon laser in addition to the classical squeezing applied to suppress the measurement uncertainty. We anticipate that such a single-ion sensor would reach a much better force detection sensitivity in the future once the trapping system is further improved and the fluorescence collection efficiency is further enhanced.

physics.atom-ph↗

arXiv2022

CAR: Class-aware Regularizations for Semantic Segmentation

Recent segmentation methods, such as OCR and CPNet, utilizing "class level" information in addition to pixel features, have achieved notable success for boosting the accuracy of existing network modules. However, the extracted class-level information was simply concatenated to pixel features, without explicitly being exploited for better pixel representation learning. Moreover, these approaches learn soft class centers based on coarse mask prediction, which is prone to error accumulation. In this paper, aiming to use class level information more effectively, we propose a universal Class-Aware Regularization (CAR) approach to optimize the intra-class variance and inter-class distance during feature learning, motivated by the fact that humans can recognize an object by itself no matter which other objects it appears with. Three novel loss functions are proposed. The first loss function encourages more compact class representations within each class, the second directly maximizes the distance between different class centers, and the third further pushes the distance between inter-class centers and pixels. Furthermore, the class center in our approach is directly generated from ground truth instead of from the error-prone coarse prediction. Our method can be easily applied to most existing segmentation models during training, including OCR and CPNet, and can largely improve their accuracy at no additional inference overhead. Extensive experiments and ablation studies conducted on multiple benchmark datasets demonstrate that the proposed CAR can boost the accuracy of all baseline models by up to 2.23% mIOU with superior generalization ability. The complete code is available at https://github.com/edwardyehuang/CAR.

cs.CV↗

arXiv2022

Enhancing MR Image Segmentation with Realistic Adversarial Data Augmentation

The success of neural networks on medical image segmentation tasks typically relies on large labeled datasets for model training. However, acquiring and manually labeling a large medical image set is resource-intensive, expensive, and sometimes impractical due to data sharing and privacy issues. To address this challenge, we propose AdvChain, a generic adversarial data augmentation framework, aiming at improving both the diversity and effectiveness of training data for medical image segmentation tasks. AdvChain augments data with dynamic data augmentation, generating randomly chained photo-metric and geometric transformations to resemble realistic yet challenging imaging variations to expand training data. By jointly optimizing the data augmentation model and a segmentation network during training, challenging examples are generated to enhance network generalizability for the downstream task. The proposed adversarial data augmentation does not rely on generative networks and can be used as a plug-in module in general segmentation networks. It is computationally efficient and applicable for both low-shot supervised and semi-supervised learning. We analyze and evaluate the method on two MR image segmentation tasks: cardiac segmentation and prostate segmentation with limited labeled data. Results show that the proposed approach can alleviate the need for labeled data while improving model generalization ability, indicating its practical value in medical imaging applications.

eess.IV↗

arXiv2022

A Survey of Trustworthy Graph Learning: Reliability, Explainability, and Privacy Protection

Deep graph learning has achieved remarkable progresses in both business and scientific areas ranging from finance and e-commerce, to drug and advanced material discovery. Despite these progresses, how to ensure various deep graph learning algorithms behave in a socially responsible manner and meet regulatory compliance requirements becomes an emerging problem, especially in risk-sensitive domains. Trustworthy graph learning (TwGL) aims to solve the above problems from a technical viewpoint. In contrast to conventional graph learning research which mainly cares about model performance, TwGL considers various reliability and safety aspects of the graph learning framework including but not limited to robustness, explainability, and privacy. In this survey, we provide a comprehensive review of recent leading approaches in the TwGL field from three dimensions, namely, reliability, explainability, and privacy protection. We give a general categorization for existing work and review typical work for each category. To give further insights for TwGL research, we provide a unified view to inspect previous works and build the connection between them. We also point out some important open problems remaining to be solved in the future developments of TwGL.

cs.LG↗

arXiv2022

The Curvature of Spectral Energy Distribution and $γ$-ray Dominance of Fermi BL Lac Objects

The extragalactic $γ$-ray sky is dominated by blazars and their study plays an important role in understanding jet physics, cosmic evolution history and origin of ultra high energy cosmic rays. In this work, we study a large sample of BL Lac objects to investigate why some sources are detected in $γ$-rays, while others not. We selected 170 BL Lac objects, with measured synchrotron spectral curvature and Doppler factor, and divided them into Fermi-LAT detected (FBLs) and non-detected (NFBLs) sources. We show that FBLs have smaller curvature than NFBLs, even after getting rid of Doppler beaming effect. The BL Lac objects PKS 0048-09 and S5 0716+714 have similar synchrotron peak frequency and luminosity but different $γ$-ray dominance and their quasi-simultaneous broadband spectral energy distributions (SEDs) can be well fitted by a log-parabolic synchrotron self-Compton (SSC) model with same jet parameters except for the curvature and source size, assuming curvature being proportional to the size of emission region. Our results imply that for a given synchrotron luminosity, the different SED curvature and Compton dominance may account for the discrepancy between FBLs and NFBLs. We discuss these results in context of stochastic particle acceleration and radiation mechanisms.

astro-ph.HE↗

arXiv2022

Remarkably strong magnetic response in molecules with polar groups

For more than a century, electricity and magnetism have been believed to always exhibit inextricable link due to the symmetry in electromagnetism. At the interface, polar groups that have polar charges, are indispensable to be considered, which interact directly with other polar charges/external charges/external electric fields. However, there is no report on the corresponding magnetic properties on these polar groups. Clearly, such asymmetry, that is, only the interaction between the polar groups and charges, is out of bounds. Here we show that those molecules with considerable polar groups, such as cellulose acetate (CA) and other cellulose derivatives with different polar groups, can have strong magnetic response, indicating that they are strongly paramagnetic. Density functional theory (DFT) calculation shows that the polarity greatly reduces the excitation energy from the state without net spin (singlet) to the state with net spin (triplet), making the considerable existence of magnetic moments on the polar groups. We note that the hydrophobic groups in these molecules have no magnetic moments, however, they make the molecules aggregate to amply the magnetic effect of the magnetic moments in the polar groups, so that these magnetic moments can induce the strong paramagnetism. Our observations suggest a recovery of the symmetry with inextricable link between the electricity and magnetism at the interface. The findings leave many imaginations of the role of the magnetic interaction in biological systems as well as other magnetic applications considering that many of those polar materials are biological materials, pharmaceutical materials, chemical raw materials, and even an essential hormone in agricultural production.

cond-mat.soft↗

arXiv2022

A Survey of Deep Learning Models for Structural Code Understanding

In recent years, the rise of deep learning and automation requirements in the software industry has elevated Intelligent Software Engineering to new heights. The number of approaches and applications in code understanding is growing, with deep learning techniques being used in many of them to better capture the information in code data. In this survey, we present a comprehensive overview of the structures formed from code data. We categorize the models for understanding code in recent years into two groups: sequence-based and graph-based models, further make a summary and comparison of them. We also introduce metrics, datasets and the downstream tasks. Finally, we make some suggestions for future research in structural code understanding field.

cs.SE↗

arXiv2022

FastGCL: Fast Self-Supervised Learning on Graphs via Contrastive Neighborhood Aggregation

Graph contrastive learning (GCL), as a popular approach to graph self-supervised learning, has recently achieved a non-negligible effect. To achieve superior performance, the majority of existing GCL methods elaborate on graph data augmentation to construct appropriate contrastive pairs. However, existing methods place more emphasis on the complex graph data augmentation which requires extra time overhead, and pay less attention to developing contrastive schemes specific to encoder characteristics. We argue that a better contrastive scheme should be tailored to the characteristics of graph neural networks (e.g., neighborhood aggregation) and propose a simple yet effective method named FastGCL. Specifically, by constructing weighted-aggregated and non-aggregated neighborhood information as positive and negative samples respectively, FastGCL identifies the potential semantic information of data without disturbing the graph topology and node attributes, resulting in faster training and convergence speeds. Extensive experiments have been conducted on node classification and graph classification tasks, showing that FastGCL has competitive classification performance and significant training speedup compared to existing state-of-the-art methods.

cs.LG↗

arXiv2022

Hierarchical Curriculum Learning for AMR Parsing

Abstract Meaning Representation (AMR) parsing aims to translate sentences to semantic representation with a hierarchical structure, and is recently empowered by pretrained sequence-to-sequence models. However, there exists a gap between their flat training objective (i.e., equally treats all output tokens) and the hierarchical AMR structure, which limits the model generalization. To bridge this gap, we propose a Hierarchical Curriculum Learning (HCL) framework with Structure-level (SC) and Instance-level Curricula (IC). SC switches progressively from core to detail AMR semantic elements while IC transits from structure-simple to -complex AMR instances during training. Through these two warming-up processes, HCL reduces the difficulty of learning complex structures, thus the flat model can better adapt to the AMR hierarchy. Extensive experiments on AMR2.0, AMR3.0, structure-complex and out-of-distribution situations verify the effectiveness of HCL.

cs.CL↗

arXiv2022

ATP: AMRize Then Parse! Enhancing AMR Parsing with PseudoAMRs

As Abstract Meaning Representation (AMR) implicitly involves compound semantic annotations, we hypothesize auxiliary tasks which are semantically or formally related can better enhance AMR parsing. We find that 1) Semantic role labeling (SRL) and dependency parsing (DP), would bring more performance gain than other tasks e.g. MT and summarization in the text-to-AMR transition even with much less data. 2) To make a better fit for AMR, data from auxiliary tasks should be properly "AMRized" to PseudoAMR before training. Knowledge from shallow level parsing tasks can be better transferred to AMR Parsing with structure transform. 3) Intermediate-task learning is a better paradigm to introduce auxiliary tasks to AMR parsing, compared to multitask learning. From an empirical perspective, we propose a principled method to involve auxiliary tasks to boost AMR parsing. Extensive experiments show that our method achieves new state-of-the-art performance on different benchmarks especially in topology-related scores.

cs.CL↗

arXiv2022

Self-Calibrated Efficient Transformer for Lightweight Super-Resolution

Recently, deep learning has been successfully applied to the single-image super-resolution (SISR) with remarkable performance. However, most existing methods focus on building a more complex network with a large number of layers, which can entail heavy computational costs and memory storage. To address this problem, we present a lightweight Self-Calibrated Efficient Transformer (SCET) network to solve this problem. The architecture of SCET mainly consists of the self-calibrated module and efficient transformer block, where the self-calibrated module adopts the pixel attention mechanism to extract image features effectively. To further exploit the contextual information from features, we employ an efficient transformer to help the network obtain similar features over long distances and thus recover sufficient texture details. We provide comprehensive results on different settings of the overall network. Our proposed method achieves more remarkable performance than baseline methods. The source code and pre-trained models are available at https://github.com/AlexZou14/SCET.

cs.CV↗

arXiv2022

A Survey of Adversarial Learning on Graphs

Deep learning models on graphs have achieved remarkable performance in various graph analysis tasks, e.g., node classification, link prediction, and graph clustering. However, they expose uncertainty and unreliability against the well-designed inputs, i.e., adversarial examples. Accordingly, a line of studies has emerged for both attack and defense addressed in different graph analysis tasks, leading to the arms race in graph adversarial learning. Despite the booming works, there still lacks a unified problem definition and a comprehensive review. To bridge this gap, we investigate and summarize the existing works on graph adversarial learning tasks systemically. Specifically, we survey and unify the existing works w.r.t. attack and defense in graph analysis tasks, and give appropriate definitions and taxonomies at the same time. Besides, we emphasize the importance of related evaluation metrics, investigate and summarize them comprehensively. Hopefully, our works can provide a comprehensive overview and offer insights for the relevant researchers. Latest advances in graph adversarial learning are summarized in our GitHub repository https://github.com/EdisonLeeeee/Graph-Adversarial-Learning.

cs.LG↗

arXiv2022

Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detection

Recent studies in deepfake detection have yielded promising results when the training and testing face forgeries are from the same dataset. However, the problem remains challenging when one tries to generalize the detector to forgeries created by unseen methods in the training dataset. This work addresses the generalizable deepfake detection from a simple principle: a generalizable representation should be sensitive to diverse types of forgeries. Following this principle, we propose to enrich the "diversity" of forgeries by synthesizing augmented forgeries with a pool of forgery configurations and strengthen the "sensitivity" to the forgeries by enforcing the model to predict the forgery configurations. To effectively explore the large forgery augmentation space, we further propose to use the adversarial training strategy to dynamically synthesize the most challenging forgeries to the current model. Through extensive experiments, we show that the proposed strategies are surprisingly effective (see Figure 1), and they could achieve superior performance than the current state-of-the-art methods. Code is available at \url{https://github.com/liangchen527/SLADD}.

cs.CV↗

arXiv2022

Exact mean-field models for spiking neural networks with adaptation

Networks of spiking neurons with adaption have been shown to be able to reproduce a wide range of neural activities, including the emergent population bursting and spike synchrony that underpin brain disorders and normal function. Exact mean-field models derived from spiking neural networks are extremely valuable, as such models can be used to determine how individual neuron and network parameters interact to produce macroscopic network behaviour. In the paper, we derive and analyze a set of exact mean-field equations for the neural network with spike frequency adaptation. Specifically, our model is a network of Izhikevich neurons, where each neuron is modeled by a two dimensional system consisting of a quadratic integrate and fire equation plus an equation which implements spike frequency adaptation. Previous work deriving a mean-field model for this type of network, relied on the assumption of sufficiently slow dynamics of the adaptation variable. However, this approximation did not succeeded in establishing an exact correspondence between the macroscopic description and the realistic neural network, especially when the adaptation time constant was not large. The challenge lies in how to achieve a closed set of mean-field equations with the inclusion of the mean-field expression of the adaptation variable. We address this challenge by using a Lorentzian ansatz combined with the moment closure approach to arrive at the mean-field system in the thermodynamic limit. The resulting macroscopic description is capable of qualitatively and quantitatively describing the collective dynamics of the neural network, including transition between tonic firing and bursting.

q-bio.NC↗

arXiv2022

Focus on the Target's Vocabulary: Masked Label Smoothing for Machine Translation

Label smoothing and vocabulary sharing are two widely used techniques in neural machine translation models. However, we argue that simply applying both techniques can be conflicting and even leads to sub-optimal performance. When allocating smoothed probability, original label smoothing treats the source-side words that would never appear in the target language equally to the real target-side words, which could bias the translation model. To address this issue, we propose Masked Label Smoothing (MLS), a new mechanism that masks the soft label probability of source-side words to zero. Simple yet effective, MLS manages to better integrate label smoothing with vocabulary sharing. Our extensive experiments show that MLS consistently yields improvement over original label smoothing on different datasets, including bilingual and multilingual translation from both translation quality and model's calibration. Our code is released at https://github.com/PKUnlp-icler/MLS

cs.CL↗