arXiv subjects

Chunhui Zhang

Publications and source records attributed to Chunhui Zhang.

62 records · Page 4Linked to original sources

arXiv2022

Contrastive Graph Few-Shot Learning

Prevailing deep graph learning models often suffer from label sparsity issue. Although many graph few-shot learning (GFL) methods have been developed to avoid performance degradation in face of limited annotated data, they excessively rely on labeled data, where the distribution shift in the test phase might result in impaired generalization ability. Additionally, they lack a general purpose as their designs are coupled with task or data-specific characteristics. To this end, we propose a general and effective Contrastive Graph Few-shot Learning framework (CGFL). CGFL leverages a self-distilled contrastive learning procedure to boost GFL. Specifically, our model firstly pre-trains a graph encoder with contrastive learning using unlabeled data. Later, the trained encoder is frozen as a teacher model to distill a student model with a contrastive loss. The distilled model is finally fed to GFL. CGFL learns data representation in a self-supervised manner, thus mitigating the distribution shift impact for better generalization and making model task and data-independent for a general graph mining purpose. Furthermore, we introduce an information-based method to quantitatively measure the capability of CGFL. Comprehensive experiments demonstrate that CGFL outperforms state-of-the-art baselines on several graph mining tasks in the few-shot scenario. We also provide quantitative measurement of CGFL's success.

cs.LG↗

arXiv2022

Label-invariant Augmentation for Semi-Supervised Graph Classification

Recently, contrastiveness-based augmentation surges a new climax in the computer vision domain, where some operations, including rotation, crop, and flip, combined with dedicated algorithms, dramatically increase the model generalization and robustness. Following this trend, some pioneering attempts employ the similar idea to graph data. Nevertheless, unlike images, it is much more difficult to design reasonable augmentations without changing the nature of graphs. Although exciting, the current graph contrastive learning does not achieve as promising performance as visual contrastive learning. We conjecture the current performance of graph contrastive learning might be limited by the violation of the label-invariant augmentation assumption. In light of this, we propose a label-invariant augmentation for graph-structured data to address this challenge. Different from the node/edge modification and subgraph extraction, we conduct the augmentation in the representation space and generate the augmented samples in the most difficult direction while keeping the label of augmented data the same as the original samples. In the semi-supervised scenario, we demonstrate our proposed method outperforms the classical graph neural network based methods and recent graph contrastive learning on eight benchmark graph-structured data, followed by several in-depth experiments to further explore the label-invariant augmentation in several aspects.

cs.CV↗

arXiv2022

Towards Tailored Models on Private AIoT Devices: Federated Direct Neural Architecture Search

Neural networks often encounter various stringent resource constraints while deploying on edge devices. To tackle these problems with less human efforts, automated machine learning becomes popular in finding various neural architectures that fit diverse Artificial Intelligence of Things (AIoT) scenarios. Recently, to prevent the leakage of private information while enable automated machine intelligence, there is an emerging trend to integrate federated learning and neural architecture search (NAS). Although promising as it may seem, the coupling of difficulties from both tenets makes the algorithm development quite challenging. In particular, how to efficiently search the optimal neural architecture directly from massive non-independent and identically distributed (non-IID) data among AIoT devices in a federated manner is a hard nut to crack. In this paper, to tackle this challenge, by leveraging the advances in ProxylessNAS, we propose a Federated Direct Neural Architecture Search (FDNAS) framework that allows for hardware-friendly NAS from non- IID data across devices. To further adapt to both various data distributions and different types of devices with heterogeneous embedded hardware platforms, inspired by meta-learning, a Cluster Federated Direct Neural Architecture Search (CFDNAS) framework is proposed to achieve device-aware NAS, in the sense that each device can learn a tailored deep learning model for its particular data distribution and hardware constraint. Extensive experiments on non-IID datasets have shown the state-of-the-art accuracy-efficiency trade-offs achieved by the proposed solution in the presence of both data and device heterogeneity.

cs.LG↗

arXiv2021

USCL: Pretraining Deep Ultrasound Image Diagnosis Model through Video Contrastive Representation Learning

Most deep neural networks (DNNs) based ultrasound (US) medical image analysis models use pretrained backbones (e.g., ImageNet) for better model generalization. However, the domain gap between natural and medical images causes an inevitable performance bottleneck. To alleviate this problem, an US dataset named US-4 is constructed for direct pretraining on the same domain. It contains over 23,000 images from four US video sub-datasets. To learn robust features from US-4, we propose an US semi-supervised contrastive learning method, named USCL, for pretraining. In order to avoid high similarities between negative pairs as well as mine abundant visual features from limited US videos, USCL adopts a sample pair generation method to enrich the feature involved in a single step of contrastive optimization. Extensive experiments on several downstream tasks show the superiority of USCL pretraining against ImageNet pretraining and other state-of-the-art (SOTA) pretraining approaches. In particular, USCL pretrained backbone achieves fine-tuning accuracy of over 94% on POCUS dataset, which is 10% higher than 84% of the ImageNet pretrained model. The source codes of this work are available at https://github.com/983632847/USCL.

cs.CV↗

arXiv2021

Student Network Learning via Evolutionary Knowledge Distillation

Knowledge distillation provides an effective way to transfer knowledge via teacher-student learning, where most existing distillation approaches apply a fixed pre-trained model as teacher to supervise the learning of student network. This manner usually brings in a big capability gap between teacher and student networks during learning. Recent researches have observed that a small teacher-student capability gap can facilitate knowledge transfer. Inspired by that, we propose an evolutionary knowledge distillation approach to improve the transfer effectiveness of teacher knowledge. Instead of a fixed pre-trained teacher, an evolutionary teacher is learned online and consistently transfers intermediate knowledge to supervise student network learning on-the-fly. To enhance intermediate knowledge representation and mimicking, several simple guided modules are introduced between corresponding teacher-student blocks. In this way, the student can simultaneously obtain rich internal knowledge and capture its growth process, leading to effective student network learning. Extensive experiments clearly demonstrate the effectiveness of our approach as well as good adaptability in the low-resolution and few-sample visual recognition scenarios.

cs.LG↗

arXiv2020

FDNAS: Improving Data Privacy and Model Diversity in AutoML

To prevent the leakage of private information while enabling automated machine intelligence, there is an emerging trend to integrate federated learning and Neural Architecture Search (NAS). Although promising as it may seem, the coupling of difficulties from both two tenets makes the algorithm development quite challenging. In particular, how to efficiently search the optimal neural architecture directly from massive non-iid data of clients in a federated manner remains to be a hard nut to crack. To tackle this challenge, in this paper, by leveraging the advances in proxy-less NAS, we propose a Federated Direct Neural Architecture Search (FDNAS) framework that allows hardware-aware NAS from decentralized non-iid data of clients. To further adapt for various data distributions of clients, inspired by meta-learning, a cluster Federated Direct Neural Architecture Search (CFDNAS) framework is proposed to achieve client-aware NAS, in the sense that each client can learn a tailored deep learning model for its particular data distribution. Extensive experiments on real-world non-iid datasets show state-of-the-art accuracy-efficiency trade-offs for various hardware and data distributions of clients. Our codes will be released publicly upon paper acceptance.

cs.LG↗

arXiv2019

Asymmetric sending or not-sending twin-field quantum key distribution in practice

Quantum key distribution (QKD) offers a secret way to share keys between legitimate users which is guaranteed by the law of quantum mechanics. Most recently, the limitation of transmission distance without quantum repeaters was broken through by twin-field QKD [Nature (London) 557, 400 (2018)]. Based on its main idea, sending or not-sending (SNS) QKD protocol was proposed [Phys. Rev. A 98, 062323 (2018)], which filled the remaining security loopholes and can tolerate large misalignment errors. In this paper, we give a more general model for SNS QKD, where the two legitimate users, Alice and Bob, can possess asymmetric quantum channels. By applying the method present in the work, the legitimate users can achieve dramatically increased key generation rate and transmission distance compared with utilizing the original symmetric protocol. Therefore, our present work represents a further step along the progress of practical QKD.

quant-ph↗

arXiv2017

Proton tracking in a high-granularity Digital Tracking Calorimeter for proton CT purposes

Radiation therapy with protons as of today utilizes information from x-ray CT in order to estimate the proton stopping power of the traversed tissue in a patient. The conversion from x-ray attenuation to proton stopping power in tissue introduces range uncertainties of the order of 2-3% of the range, uncertainties that are contributing to an increase of the necessary planning margins added to the target volume in a patient. Imaging methods and modalities, such as Dual Energy CT and proton CT, have come into consideration in the pursuit of obtaining an as good as possible estimate of the proton stopping power. In this study, a Digital Tracking Calorimeter is benchmarked for proof-of-concept for proton CT purposes. The Digital Tracking Calorimeteris applied for reconstruction of the tracks and energies of individual high energy protons. The presented prototype forms the basis for a proton CT system using a single technology for tracking and calorimetry. This advantage simplifies the setup and reduces the cost of a proton CT system assembly, and it is a unique feature of the Digital Tracking Calorimeter. Data from the AGORFIRM beamline at KVI-CART in Groningen in the Netherlands and Monte Carlo simulation results are used to in order to develop a tracking algorithm for the estimation of the residual ranges of a high number of concurrent proton tracks. The range of the individual protons can at present be estimated with a resolution of 4%. The readout system for this prototype is able to handle an effective proton frequency of 1 MHz by using 500 concurrent proton tracks in each readout frame, which is at the high end range of present similar prototypes. A future further optimized prototype will enable a high-speed and more accurate determination of the ranges of individual protons in a therapeutic beam.

physics.med-ph↗