arXiv subjects

Peng Gao

Publications and source records attributed to Peng Gao.

At least 289 records · Page 16Linked to original sources

arXiv2022

Electric-field control of the nucleation and motion of isolated three-fold polar vertices

Recently various topological polar structures have been discovered in oxide thin films. Despite the increasing evidence of their switchability under electrical and/or mechanical fields, the dynamic property of isolated ones, which is usually required for applications such as data storage, is still absent. Here, we show the controlled nucleation and motion of isolated three-fold vertices under an applied electric field. At the PbTiO3/SrRuO3 interface, a two-unit-cell thick SrTiO3 layer provides electrical boundary conditions for the formation of three-fold vertices. Utilizing the SrTiO3 layer and in situ electrical testing system, we find that isolated three-fold vertices can move in a controllable and reversible manner with a velocity up to ~629 nm s-1. Microstructural evolution of the nucleation and propagation of isolated three-fold vertices is further revealed by phase-field simulations. This work demonstrates the ability to electrically manipulate isolated three-fold vertices, shedding light on the dynamic property of isolated topological polar structures.

cond-mat.mtrl-sci↗

arXiv2022

First moment of central values of quadratic Hecke $L$-functions in the Gaussian field

We evaluate the smoothed first moment of central values of a family of qudratic Hecke $L$-functions in the Gaussian field using the method of double Dirichlet series. The asymptotic formula we obtain has an error term of size $O(X^{1/4+\varepsilon})$ under the generalized Riemann hypothesis. The same approach also allows us to obtain asymptotic formulas for all $X$, $Y$ for a smoothed double character sum involving $\sum_{N(m) \leq X, N(n)\leq Y}\left( \frac{m}{n} \right)$, where $\left( \frac{\cdot}{n} \right)$ denotes the quadratic symbol in the Gaussian field.

math.NT↗

arXiv2022

Atomically localized ingredient-dependent interface phonon in heterogeneous solids

Phonons are the primary heat carriers in non-metallic solids. In compositionally heterogeneous materials, the thermal properties are believed to be mainly governed by the disrupted phonon transport due to mass disorder and strain fluctuations, while the effects of compositional fluctuation induced local phonon states are usually ignored. Here, by scanning transmission electron microscopy electron energy loss spectroscopy and sophisticated calculations, we identify the vibrational properties of ingredient-dependent interface phonon modes in AlxGa1-xN and quantify their various contributions to the local interface thermal conductance. We demonstrate that atomic-scale compositional fluctuation has significant influence on the vibrational thermodynamic properties, highly affecting the mode ratio and vibrational amplitude of interface phonon modes and subsequently redistributing their modal contribution to the ITC. Our work provides fundamental insights into understanding of local phonon-boundary interactions in nanoscale inhomogeneities, which reveal new opportunities for optimization of thermal properties via engineering ingredient distribution.

cond-mat.mtrl-sci↗

arXiv2022

HCL: Improving Graph Representation with Hierarchical Contrastive Learning

Contrastive learning has emerged as a powerful tool for graph representation learning. However, most contrastive learning methods learn features of graphs with fixed coarse-grained scale, which might underestimate either local or global information. To capture more hierarchical and richer representation, we propose a novel Hierarchical Contrastive Learning (HCL) framework that explicitly learns graph representation in a hierarchical manner. Specifically, HCL includes two key components: a novel adaptive Learning to Pool (L2Pool) method to construct more reasonable multi-scale graph topology for more comprehensive contrastive objective, a novel multi-channel pseudo-siamese network to further enable more expressive learning of mutual information within each scale. Comprehensive experimental results show HCL achieves competitive performance on 12 datasets involving node classification, node clustering and graph classification. In addition, the visualization of learned representation reveals that HCL successfully captures meaningful characteristics of graphs.

cs.LG↗

arXiv2022

Bounds for moments of symmetric square $L$-functions

We study the $2k$-th moment at the central point of the family of symmetric square $L$-functions attached to holomorphic Hecke cusp forms of level one, weight $κ$. We establish sharp lower bounds for all real $k \geq 1/2$ unconditionally. Assuming the truth of the generalized Riemann hypothesis, we also obtain sharp lower bounds for all real $0 \leq k < 1/2$ and sharp upper bounds for all real $k \geq 0$.

math.NT↗

arXiv2022

Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-training

Masked Autoencoders (MAE) have shown great potentials in self-supervised pre-training for language and 2D image transformers. However, it still remains an open question on how to exploit masked autoencoding for learning 3D representations of irregular point clouds. In this paper, we propose Point-M2AE, a strong Multi-scale MAE pre-training framework for hierarchical self-supervised learning of 3D point clouds. Unlike the standard transformer in MAE, we modify the encoder and decoder into pyramid architectures to progressively model spatial geometries and capture both fine-grained and high-level semantics of 3D shapes. For the encoder that downsamples point tokens by stages, we design a multi-scale masking strategy to generate consistent visible regions across scales, and adopt a local spatial self-attention mechanism during fine-tuning to focus on neighboring patterns. By multi-scale token propagation, the lightweight decoder gradually upsamples point tokens with complementary skip connections from the encoder, which further promotes the reconstruction from a global-to-local perspective. Extensive experiments demonstrate the state-of-the-art performance of Point-M2AE for 3D representation learning. With a frozen encoder after pre-training, Point-M2AE achieves 92.9% accuracy for linear SVM on ModelNet40, even surpassing some fully trained methods. By fine-tuning on downstream tasks, Point-M2AE achieves 86.43% accuracy on ScanObjectNN, +3.36% to the second-best, and largely benefits the few-shot classification, part segmentation and 3D object detection with the hierarchical pre-training scheme. Code is available at https://github.com/ZrrSkywalker/Point-M2AE.

cs.CV↗

arXiv2022

Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer

The large pre-trained vision transformers (ViTs) have demonstrated remarkable performance on various visual tasks, but suffer from expensive computational and memory cost problems when deployed on resource-constrained devices. Among the powerful compression approaches, quantization extremely reduces the computation and memory consumption by low-bit parameters and bit-wise operations. However, low-bit ViTs remain largely unexplored and usually suffer from a significant performance drop compared with the real-valued counterparts. In this work, through extensive empirical analysis, we first identify the bottleneck for severe performance drop comes from the information distortion of the low-bit quantized self-attention map. We then develop an information rectification module (IRM) and a distribution guided distillation (DGD) scheme for fully quantized vision transformers (Q-ViT) to effectively eliminate such distortion, leading to a fully quantized ViTs. We evaluate our methods on popular DeiT and Swin backbones. Extensive experimental results show that our method achieves a much better performance than the prior arts. For example, our Q-ViT can theoretically accelerates the ViT-S by 6.14x and achieves about 80.9% Top-1 accuracy, even surpassing the full-precision counterpart by 1.0% on ImageNet dataset. Our codes and models are attached on https://github.com/YanjingLi0202/Q-ViT

cs.CV↗

arXiv2022

You Only Need 90K Parameters to Adapt Light: A Light Weight Transformer for Image Enhancement and Exposure Correction

Challenging illumination conditions (low-light, under-exposure and over-exposure) in the real world not only cast an unpleasant visual appearance but also taint the computer vision tasks. After camera captures the raw-RGB data, it renders standard sRGB images with image signal processor (ISP). By decomposing ISP pipeline into local and global image components, we propose a lightweight fast Illumination Adaptive Transformer (IAT) to restore the normal lit sRGB image from either low-light or under/over-exposure conditions. Specifically, IAT uses attention queries to represent and adjust the ISP-related parameters such as colour correction, gamma correction. With only ~90k parameters and ~0.004s processing speed, our IAT consistently achieves superior performance over SOTA on the current benchmark low-light enhancement and exposure correction datasets. Competitive experimental performance also demonstrates that our IAT significantly enhances object detection and semantic segmentation tasks under various light conditions. Training code and pretrained model is available at https://github.com/cuiziteng/Illumination-Adaptive-Transformer.

cs.CV↗

arXiv2022

IDa-Det: An Information Discrepancy-aware Distillation for 1-bit Detectors

Knowledge distillation (KD) has been proven to be useful for training compact object detection models. However, we observe that KD is often effective when the teacher model and student counterpart share similar proposal information. This explains why existing KD methods are less effective for 1-bit detectors, caused by a significant information discrepancy between the real-valued teacher and the 1-bit student. This paper presents an Information Discrepancy-aware strategy (IDa-Det) to distill 1-bit detectors that can effectively eliminate information discrepancies and significantly reduce the performance gap between a 1-bit detector and its real-valued counterpart. We formulate the distillation process as a bi-level optimization formulation. At the inner level, we select the representative proposals with maximum information discrepancy. We then introduce a novel entropy distillation loss to reduce the disparity based on the selected proposals. Extensive experiments demonstrate IDa-Det's superiority over state-of-the-art 1-bit detectors and KD methods on both PASCAL VOC and COCO datasets. IDa-Det achieves a 76.9% mAP for a 1-bit Faster-RCNN with ResNet-18 backbone. Our code is open-sourced on https://github.com/SteveTsui/IDa-Det.

cs.CV↗

arXiv2022

Phonon transition across an isotopic interface

Natural materials usually consist of isotopic mixtures, for which different isotopic ratios can lead to distinct material properties such as thermal conductivity and nucleation process. However, the knowledge of isotopic interface remains largely unexplored mainly due to the challenges in isotopic identification and property measurement at an atomic scale. Here, by using monochromated electron energy-loss spectroscopy in a scanning transmission electron microscope, we reveal momentum-transfer-dependent lattice vibration behavior at an artificial h-10BN/h-11BN heterostructure with sub-unit-cell resolution. We find the vibrational energy changes across the isotopic interface gradually, featuring a wide transition regime, which suggests strong delocalization of the out-of-plane optical phonons at the interface. In addition, we identify phonons near the Brillouin zone center have a transition regime ~3.34 nm (10 atomic layers), whereas phonons at the Brillouin zone boundary transition in ~1.66 nm (5 atomic layers). We propose that the isotope-induced charge effect at the interface accounts for the distinct delocalization behavior. Moreover, intra-atomic layer variation of vibration energy is also sensitive to the momentum transfer, thus at the interface it depends on both of momentum transfer and mass change. The revealed lattice vibration behavior at an isotopic interface provides new insights to understand the isotopic effects on properties in natural materials.

cond-mat.mtrl-sci↗

arXiv2022

Consecutive Pretraining: A Knowledge Transfer Learning Strategy with Relevant Unlabeled Data for Remote Sensing Domain

Currently, under supervised learning, a model pretrained by a large-scale nature scene dataset and then fine-tuned on a few specific task labeling data is the paradigm that has dominated the knowledge transfer learning. It has reached the status of consensus solution for task-aware model training in remote sensing domain (RSD). Unfortunately, due to different categories of imaging data and stiff challenges of data annotation, there is not a large enough and uniform remote sensing dataset to support large-scale pretraining in RSD. Moreover, pretraining models on large-scale nature scene datasets by supervised learning and then directly fine-tuning on diverse downstream tasks seems to be a crude method, which is easily affected by inevitable labeling noise, severe domain gaps and task-aware discrepancies. Thus, in this paper, considering the self-supervised pretraining and powerful vision transformer (ViT) architecture, a concise and effective knowledge transfer learning strategy called ConSecutive PreTraining (CSPT) is proposed based on the idea of not stopping pretraining in natural language processing (NLP), which can gradually bridge the domain gap and transfer knowledge from the nature scene domain to the RSD. The proposed CSPT also can release the huge potential of unlabeled data for task-aware model training. Finally, extensive experiments are carried out on twelve datasets in RSD involving three types of downstream tasks (e.g., scene classification, object detection and land cover classification) and two types of imaging data (e.g., optical and SAR). The results show that by utilizing the proposed CSPT for task-aware model training, almost all downstream tasks in RSD can outperform the previous method of supervised pretraining-then-fine-tuning and even surpass the state-of-the-art (SOTA) performance without any expensive labeling consumption and careful model design.

cs.CV↗

arXiv2022

Recurrent Bilinear Optimization for Binary Neural Networks

Binary Neural Networks (BNNs) show great promise for real-world embedded devices. As one of the critical steps to achieve a powerful BNN, the scale factor calculation plays an essential role in reducing the performance gap to their real-valued counterparts. However, existing BNNs neglect the intrinsic bilinear relationship of real-valued weights and scale factors, resulting in a sub-optimal model caused by an insufficient training process. To address this issue, Recurrent Bilinear Optimization is proposed to improve the learning process of BNNs (RBONNs) by associating the intrinsic bilinear variables in the back propagation process. Our work is the first attempt to optimize BNNs from the bilinear perspective. Specifically, we employ a recurrent optimization and Density-ReLU to sequentially backtrack the sparse real-valued weight filters, which will be sufficiently trained and reach their performance limits based on a controllable learning process. We obtain robust RBONNs, which show impressive performance over state-of-the-art BNNs on various models and datasets. Particularly, on the task of object detection, RBONNs have great generalization performance. Our code is open-sourced on https://github.com/SteveTsui/RBONN .

cs.CV↗

arXiv2022

Moments of quadratic Dirichlet $L$-functions over Function Fields

In this paper, we establish the expected order of magnitude of the $k$th-moment of quadratic Dirichlet $L$-functions associated to hyperelliptic curves of genus $g$ as well as of prime moduli over a fixed finite field $\mathbb{F}_{q}$ for all real $k \geq 0$.

math.NT↗

arXiv2022

Tunable Interband Transitions in Twisted h-BN/Graphene Heterostructures

In twisted h-BN/graphene heterostructures, the complex electronic properties of the fast-traveling electron gas in graphene are usually considered to be fully revealed. However, the randomly twisted heterostructures may also have unexpected transition behaviors, which may influence the device performance. Here, we study the twist angle-dependent coupling effects of h-BN/graphene heterostructures using monochromatic electron energy loss spectroscopy. We find that the moiré potentials alter the band structure of graphene, resulting in a redshift of the intralayer transition at the M-point, which becomes more pronounced up to 0.25 eV with increasing twist angle. Furthermore, the twisting of the Brillouin zone of h-BN relative to the graphene M-point leads to tunable vertical transition energies in the range of 5.1-5.6 eV. Our findings indicate that twist-coupling effects of van der Waals heterostructures should be carefully considered in device fabrications, and the continuously tunable interband transitions through the twist angle can serve as a new degree of freedom to design optoelectrical devices.

cond-mat.mes-hall↗

arXiv2022

CandidateDrug4Cancer: An Open Molecular Graph Learning Benchmark on Drug Discovery for Cancer

Anti-cancer drug discoveries have been serendipitous, we sought to present the Open Molecular Graph Learning Benchmark, named CandidateDrug4Cancer, a challenging and realistic benchmark dataset to facilitate scalable, robust, and reproducible graph machine learning research for anti-cancer drug discovery. CandidateDrug4Cancer dataset encompasses multiple most-mentioned 29 targets for cancer, covering 54869 cancer-related drug molecules which are ranged from pre-clinical, clinical and FDA-approved. Besides building the datasets, we also perform benchmark experiments with effective Drug Target Interaction (DTI) prediction baselines using descriptors and expressive graph neural networks. Experimental results suggest that CandidateDrug4Cancer presents significant challenges for learning molecular graphs and targets in practical application, indicating opportunities for future researches on developing candidate drugs for treating cancers.

cs.LG↗

arXiv2022

Lower bounds for negative moments of $ζ'(ρ)$

We establish lower bounds for the discrete $2k$-th moment of the derivative of the Riemann zeta function at nontrivial zeros for all $k<0$ under the Riemann hypothesis (RH) and the assumption that all zeros of $ζ(s)$ are simple.

math.NT↗

arXiv2022

Frozen CLIP Models are Efficient Video Learners

Video recognition has been dominated by the end-to-end learning paradigm -- first initializing a video recognition model with weights of a pretrained image model and then conducting end-to-end training on videos. This enables the video network to benefit from the pretrained image model. However, this requires substantial computation and memory resources for finetuning on videos and the alternative of directly using pretrained image features without finetuning the image backbone leads to subpar results. Fortunately, recent advances in Contrastive Vision-Language Pre-training (CLIP) pave the way for a new route for visual recognition tasks. Pretrained on large open-vocabulary image-text pair data, these models learn powerful visual representations with rich semantics. In this paper, we present Efficient Video Learning (EVL) -- an efficient framework for directly training high-quality video recognition models with frozen CLIP features. Specifically, we employ a lightweight Transformer decoder and learn a query token to dynamically collect frame-level spatial features from the CLIP image encoder. Furthermore, we adopt a local temporal module in each decoder layer to discover temporal clues from adjacent frames and their attention maps. We show that despite being efficient to train with a frozen backbone, our models learn high quality video representations on a variety of video recognition datasets. Code is available at https://github.com/OpenGVLab/efficient-video-recognition.

cs.CV↗

arXiv2022

Moments and One level density of sextic Hecke $L$-functions of $\mathbb{Q}(ω)$

In this paper, we study moments of central values of sextic Hecke $L$-functions of $\mathbb{Q}(ω)$ and one level density result for the low-lying zeros of sextic Hecke $L$-functions of $\mathbb{Q}(ω)$. As a corollary, we deduce that, assuming GRH, at least $2/45$ of the members of the sextic family do not vanish at $s=1/2$.

math.NT↗