arXiv subjects

Zhenyu Li

Publications and source records attributed to Zhenyu Li.

At least 127 records · Page 7Linked to original sources

arXiv2022

AutoAlignV2: Deformable Feature Aggregation for Dynamic Multi-Modal 3D Object Detection

Point clouds and RGB images are two general perceptional sources in autonomous driving. The former can provide accurate localization of objects, and the latter is denser and richer in semantic information. Recently, AutoAlign presents a learnable paradigm in combining these two modalities for 3D object detection. However, it suffers from high computational cost introduced by the global-wise attention. To solve the problem, we propose Cross-Domain DeformCAFA module in this work. It attends to sparse learnable sampling points for cross-modal relational modeling, which enhances the tolerance to calibration error and greatly speeds up the feature aggregation across different modalities. To overcome the complex GT-AUG under multi-modal settings, we design a simple yet effective cross-modal augmentation strategy on convex combination of image patches given their depth information. Moreover, by carrying out a novel image-level dropout training scheme, our model is able to infer in a dynamic manner. To this end, we propose AutoAlignV2, a faster and stronger multi-modal 3D detection framework, built on top of AutoAlign. Extensive experiments on nuScenes benchmark demonstrate the effectiveness and efficiency of AutoAlignV2. Notably, our best model reaches 72.4 NDS on nuScenes test leaderboard, achieving new state-of-the-art results among all published multi-modal 3D object detectors. Code will be available at https://github.com/zehuichen123/AutoAlignV2.

cs.CV↗

arXiv2022

Effective Few-Shot Named Entity Linking by Meta-Learning

Entity linking aims to link ambiguous mentions to their corresponding entities in a knowledge base, which is significant and fundamental for various downstream applications, e.g., knowledge base completion, question answering, and information extraction. While great efforts have been devoted to this task, most of these studies follow the assumption that large-scale labeled data is available. However, when the labeled data is insufficient for specific domains due to labor-intensive annotation work, the performance of existing algorithms will suffer an intolerable decline. In this paper, we endeavor to solve the problem of few-shot entity linking, which only requires a minimal amount of in-domain labeled data and is more practical in real situations. Specifically, we firstly propose a novel weak supervision strategy to generate non-trivial synthetic entity-mention pairs based on mention rewriting. Since the quality of the synthetic data has a critical impact on effective model training, we further design a meta-learning mechanism to assign different weights to each synthetic entity-mention pair automatically. Through this way, we can profoundly exploit rich and precious semantic information to derive a well-trained entity linking model under the few-shot setting. The experiments on real-world datasets show that the proposed method can extensively improve the state-of-the-art few-shot entity linking model and achieve impressive performance when only a small amount of labeled data is available. Moreover, we also demonstrate the outstanding ability of the model's transferability.

cs.CL↗

arXiv2022

Large-Scale Simulation of Quantum Computational Chemistry on a New Sunway Supercomputer

Quantum computational chemistry (QCC) is the use of quantum computers to solve problems in computational quantum chemistry. We develop a high performance variational quantum eigensolver (VQE) simulator for simulating quantum computational chemistry problems on a new Sunway supercomputer. The major innovations include: (1) a Matrix Product State (MPS) based VQE simulator to reduce the amount of memory needed and increase the simulation efficiency; (2) a combination of the Density Matrix Embedding Theory with the MPS-based VQE simulator to further extend the simulation range; (3) A three-level parallelization scheme to scale up to 20 million cores; (4) Usage of the Julia script language as the main programming language, which both makes the programming easier and enables cutting edge performance as native C or Fortran; (5) Study of real chemistry systems based on the VQE simulator, achieving nearly linearly strong and weak scaling. Our simulation demonstrates the power of VQE for large quantum chemistry systems, thus paves the way for large-scale VQE experiments on near-term quantum computers.

quant-ph↗

arXiv2022

Graph-DETR3D: Rethinking Overlapping Regions for Multi-View 3D Object Detection

3D object detection from multiple image views is a fundamental and challenging task for visual scene understanding. Due to its low cost and high efficiency, multi-view 3D object detection has demonstrated promising application prospects. However, accurately detecting objects through perspective views in the 3D space is extremely difficult due to the lack of depth information. Recently, DETR3D introduces a novel 3D-2D query paradigm in aggregating multi-view images for 3D object detection and achieves state-of-the-art performance. In this paper, with intensive pilot experiments, we quantify the objects located at different regions and find that the "truncated instances" (i.e., at the border regions of each image) are the main bottleneck hindering the performance of DETR3D. Although it merges multiple features from two adjacent views in the overlapping regions, DETR3D still suffers from insufficient feature aggregation, thus missing the chance to fully boost the detection performance. In an effort to tackle the problem, we propose Graph-DETR3D to automatically aggregate multi-view imagery information through graph structure learning (GSL). It constructs a dynamic 3D graph between each object query and 2D feature maps to enhance the object representations, especially at the border regions. Besides, Graph-DETR3D benefits from a novel depth-invariant multi-scale training strategy, which maintains the visual depth consistency by simultaneously scaling the image size and the object depth. Extensive experiments on the nuScenes dataset demonstrate the effectiveness and efficiency of our Graph-DETR3D. Notably, our best model achieves 49.5 NDS on the nuScenes test leaderboard, achieving new state-of-the-art in comparison with various published image-view 3D object detectors.

cs.CV↗

arXiv2022

Towards Model Generalization for Monocular 3D Object Detection

Monocular 3D object detection (Mono3D) has achieved tremendous improvements with emerging large-scale autonomous driving datasets and the rapid development of deep learning techniques. However, caused by severe domain gaps (e.g., the field of view (FOV), pixel size, and object size among datasets), Mono3D detectors have difficulty in generalization, leading to drastic performance degradation on unseen domains. To solve these issues, we combine the position-invariant transform and multi-scale training with the pixel-size depth strategy to construct an effective unified camera-generalized paradigm (CGP). It fully considers discrepancies in the FOV and pixel size of images captured by different cameras. Moreover, we further investigate the obstacle in quantitative metrics when cross-dataset inference through an exhaustive systematic study. We discern that the size bias of prediction leads to a colossal failure. Hence, we propose the 2D-3D geometry-consistent object scaling strategy (GCOS) to bridge the gap via an instance-level augment. Our method called DGMono3D achieves remarkable performance on all evaluated datasets and surpasses the SoTA unsupervised domain adaptation scheme even without utilizing data on the target domain.

cs.CV↗

arXiv2022

Exploring accurate potential energy surfaces via integrating variational quantum eigensovler with machine learning

The potential energy surface (PES) is crucial for interpreting a variety of chemical reaction processes. However, predicting accurate PESs with high-level electronic structure methods is a challenging task due to the high computational cost. As an appealing application of quantum computing, we show in this work that variational quantum algorithms can be integrated with machine learning (ML) techniques as a promising scheme for exploring accurate PESs. Different from using a ML model to represent the potential energy, we encode the molecular geometry information into a deep neural network (DNN) for representing parameters of the variational quantum eigensolver (VQE), leaving the PES to the wave function ansatz. Once the DNN model is trained, the variational optimization procedure that hinders the application of the VQE to complex systems is avoided and thus the evaluation of PESs is significantly accelerated. Numerical results demonstrate that a simple DNN model is able to reproduce accurate PESs for small molecules.

physics.chem-ph↗

arXiv2022

Unsupervised Domain Adaptation for Monocular 3D Object Detection via Self-Training

Monocular 3D object detection (Mono3D) has achieved unprecedented success with the advent of deep learning techniques and emerging large-scale autonomous driving datasets. However, drastic performance degradation remains an unwell-studied challenge for practical cross-domain deployment as the lack of labels on the target domain. In this paper, we first comprehensively investigate the significant underlying factor of the domain gap in Mono3D, where the critical observation is a depth-shift issue caused by the geometric misalignment of domains. Then, we propose STMono3D, a new self-teaching framework for unsupervised domain adaptation on Mono3D. To mitigate the depth-shift, we introduce the geometry-aligned multi-scale training strategy to disentangle the camera parameters and guarantee the geometry consistency of domains. Based on this, we develop a teacher-student paradigm to generate adaptive pseudo labels on the target domain. Benefiting from the end-to-end framework that provides richer information of the pseudo labels, we propose the quality-aware supervision strategy to take instance-level pseudo confidences into account and improve the effectiveness of the target-domain training process. Moreover, the positive focusing training strategy and dynamic threshold are proposed to handle tremendous FN and FP pseudo samples. STMono3D achieves remarkable performance on all evaluated datasets and even surpasses fully supervised results on the KITTI 3D object detection dataset. To the best of our knowledge, this is the first study to explore effective UDA methods for Mono3D.

cs.CV↗

arXiv2022

AutoAlign: Pixel-Instance Feature Aggregation for Multi-Modal 3D Object Detection

Object detection through either RGB images or the LiDAR point clouds has been extensively explored in autonomous driving. However, it remains challenging to make these two data sources complementary and beneficial to each other. In this paper, we propose \textit{AutoAlign}, an automatic feature fusion strategy for 3D object detection. Instead of establishing deterministic correspondence with camera projection matrix, we model the mapping relationship between the image and point clouds with a learnable alignment map. This map enables our model to automate the alignment of non-homogenous features in a dynamic and data-driven manner. Specifically, a cross-attention feature alignment module is devised to adaptively aggregate \textit{pixel-level} image features for each voxel. To enhance the semantic consistency during feature alignment, we also design a self-supervised cross-modal feature interaction module, through which the model can learn feature aggregation with \textit{instance-level} feature guidance. Extensive experimental results show that our approach can lead to 2.3 mAP and 7.0 mAP improvements on the KITTI and nuScenes datasets, respectively. Notably, our best model reaches 70.9 NDS on the nuScenes testing leaderboard, achieving competitive performance among various state-of-the-arts.

cs.CV↗

arXiv2022

BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation

Monocular depth estimation is a fundamental task in computer vision and has drawn increasing attention. Recently, some methods reformulate it as a classification-regression task to boost the model performance, where continuous depth is estimated via a linear combination of predicted probability distributions and discrete bins. In this paper, we present a novel framework called BinsFormer, tailored for the classification-regression-based depth estimation. It mainly focuses on two crucial components in the specific task: 1) proper generation of adaptive bins and 2) sufficient interaction between probability distribution and bins predictions. To specify, we employ the Transformer decoder to generate bins, novelly viewing it as a direct set-to-set prediction problem. We further integrate a multi-scale decoder structure to achieve a comprehensive understanding of spatial geometry information and estimate depth maps in a coarse-to-fine manner. Moreover, an extra scene understanding query is proposed to improve the estimation accuracy, which turns out that models can implicitly learn useful information from an auxiliary environment classification task. Extensive experiments on the KITTI, NYU, and SUN RGB-D datasets demonstrate that BinsFormer surpasses state-of-the-art monocular depth estimation methods with prominent margins. Code and pretrained models will be made publicly available at \url{https://github.com/zhyever/Monocular-Depth-Estimation-Toolbox}.

cs.CV↗

arXiv2022

DepthFormer: Exploiting Long-Range Correlation and Local Information for Accurate Monocular Depth Estimation

This paper aims to address the problem of supervised monocular depth estimation. We start with a meticulous pilot study to demonstrate that the long-range correlation is essential for accurate depth estimation. Therefore, we propose to leverage the Transformer to model this global context with an effective attention mechanism. We also adopt an additional convolution branch to preserve the local information as the Transformer lacks the spatial inductive bias in modeling such contents. However, independent branches lead to a shortage of connections between features. To bridge this gap, we design a hierarchical aggregation and heterogeneous interaction module to enhance the Transformer features via element-wise interaction and model the affinity between the Transformer and the CNN features in a set-to-set translation manner. Due to the unbearable memory cost caused by global attention on high-resolution feature maps, we introduce the deformable scheme to reduce the complexity. Extensive experiments on the KITTI, NYU, and SUN RGB-D datasets demonstrate that our proposed model, termed DepthFormer, surpasses state-of-the-art monocular depth estimation methods with prominent margins. Notably, it achieves the most competitive result on the highly competitive KITTI depth estimation benchmark. Our codes and models are available at https://github.com/zhyever/Monocular-Depth-Estimation-Toolbox.

cs.CV↗

arXiv2022

Image Fragile Watermarking Algorithm Based on Deneighborhood Mapping

To address the security risk caused by fixed offset mapping and the limited recoverability of random mapping used in image watermarking, we propose an image self-embedding fragile watermarking algorithm based on deneighborhood mapping. First, the image is divided into several 2*2 blocks, and authentication watermark and recovery watermark are generated based on the average value of the image blocks. Then, the denighborhood mapping is implemented as, for each image block, its mapping block is randomly selected outside it's neighborhood whose size is specified by a parameter. Finally, the authentication watermark and the recovery watermark are embedded in the image block itself and its corresponding mapping block. Theoretical analysis indicates that in the case of continuous region tampering, the proposed watermarking method can achieve better the recovery rate of the tampered image block than the method based on the random mapping. The experimental results verify the rationality and effectiveness of the theoretical analysis. Moreover, compared with the existing embedding algorithms based on random mapping, chaos mapping and Arnold mapping, in the case of continuous region tampering, the average recovery rate of the tampered region achieved by the proposed algorithm is higher.

cs.CR↗

arXiv2022

Reducing circuit depth in adaptive variational quantum algorithms via effective Hamiltonian theories

Electronic structure simulation is an anticipated application for quantum computers. Due to high-dimensional quantum entanglement in strongly correlated systems, the quantum resources required to perform such simulations are far beyond the capacity of current quantum devices. To reduce the quantum circuit complexity, it has been suggested to incorporate a part of the electronic correlation into an effective Hamiltonian, which is often obtained from a similarity transformation of the electronic Hamiltonian. In this work, we introduce a new transformation in the form of a product of a linear combination of excitation operators to construct the effective Hamiltonian with finite terms. To demonstrate its accuracy, we also consider an equivalent adaptive variational algorithm with this transformation and show that it can obtain an accurate ground state wave function. The effective Hamiltonian defined with this new transformation is incorporated into the adaptive variational quantum algorithms to maintain constant-size quantum circuits. The new computational scheme is assessed by performing numerical simulations for small molecules. Chemical accuracy is achieved with a much shallower circuit depth.

quant-ph↗

arXiv2022

SimIPU: Simple 2D Image and 3D Point Cloud Unsupervised Pre-Training for Spatial-Aware Visual Representations

Pre-training has become a standard paradigm in many computer vision tasks. However, most of the methods are generally designed on the RGB image domain. Due to the discrepancy between the two-dimensional image plane and the three-dimensional space, such pre-trained models fail to perceive spatial information and serve as sub-optimal solutions for 3D-related tasks. To bridge this gap, we aim to learn a spatial-aware visual representation that can describe the three-dimensional space and is more suitable and effective for these tasks. To leverage point clouds, which are much more superior in providing spatial information compared to images, we propose a simple yet effective 2D Image and 3D Point cloud Unsupervised pre-training strategy, called SimIPU. Specifically, we develop a multi-modal contrastive learning framework that consists of an intra-modal spatial perception module to learn a spatial-aware representation from point clouds and an inter-modal feature interaction module to transfer the capability of perceiving spatial information from the point cloud encoder to the image encoder, respectively. Positive pairs for contrastive losses are established by the matching algorithm and the projection matrix. The whole framework is trained in an unsupervised end-to-end fashion. To the best of our knowledge, this is the first study to explore contrastive learning pre-training strategies for outdoor multi-modal datasets, containing paired camera images and LIDAR point clouds. Codes and models are available at https://github.com/zhyever/SimIPU.

cs.CV↗

arXiv2022

Understanding High-Temperature Chemical Reactions on Metal Surfaces

Chemical reactions on metal surfaces are important in various processes such as heterogeneous catalysis and nanostructure growth. At moderate or lower temperatures, these reactions generally follow the minimum energy path and temperature effects can be reasonably described by a harmonic oscillator model. At a high temperature approaching the melting point of the substrate, general behaviors of surface reactions remain elusive. In this study, by taking hydrocarbon species adsorbed on Cu(111) as a model system and performing extensive molecular dynamics simulations powered by machine learning potentials, we identify several important high-temperature effects, including local chemical environment, surface atom mobility, and substrate thermal expansion. They affect different aspects of a high-temperature surface reaction in different ways. These results deepen our understanding of high-temperature reactions.

physics.chem-ph↗

arXiv2021

ForgeryNet -- Face Forgery Analysis Challenge 2021: Methods and Results

The rapid progress of photorealistic synthesis techniques has reached a critical point where the boundary between real and manipulated images starts to blur. Recently, a mega-scale deep face forgery dataset, ForgeryNet which comprised of 2.9 million images and 221,247 videos has been released. It is by far the largest publicly available in terms of data-scale, manipulations (7 image-level approaches, 8 video-level approaches), perturbations (36 independent and more mixed perturbations), and annotations (6.3 million classification labels, 2.9 million manipulated area annotations, and 221,247 temporal forgery segment labels). This paper reports methods and results in the ForgeryNet - Face Forgery Analysis Challenge 2021, which employs the ForgeryNet benchmark. The model evaluation is conducted offline on the private test set. A total of 186 participants registered for the competition, and 11 teams made valid submissions. We will analyze the top-ranked solutions and present some discussion on future work directions.

cs.CV↗

arXiv2021

A Quantum Algorithm to Calculate Band Structure at the EOM Level of Theory

Band structure is a cornerstone to understand electronic properties of materials. Accurate band structure calculations using a high-level quantum-chemistry theory can be computationally very expensive. It is promising to speed up such calculations with a quantum computer. In this study, we present a quantum algorithm for band structure calculation based on the equation-of-motion (EOM) theory. First, we introduce a new variational quantum eigensolver algorithm named ADAPT-C for ground-state quantum simulation of solids, where the wave function is built adaptively from a complete set of anti-Hermitian operators. Then, on top of the ADAPT-C ground state, quasiparticle energies and the band structure can be calculated using the EOM theory in a quantum-subspace-expansion (QSE) style, where the projected excitation operators guarantee that the killer condition is satisfied. As a proof of principle, such an EOM-ADAPT-C protocol is used to calculate the band structures of silicon and diamond using a quantum computer simulator.

quant-ph↗

arXiv2021

Motion-Aware Robotic 3D Ultrasound

Robotic three-dimensional (3D) ultrasound (US) imaging has been employed to overcome the drawbacks of traditional US examinations, such as high inter-operator variability and lack of repeatability. However, object movement remains a challenge as unexpected motion decreases the quality of the 3D compounding. Furthermore, attempted adjustment of objects, e.g., adjusting limbs to display the entire limb artery tree, is not allowed for conventional robotic US systems. To address this challenge, we propose a vision-based robotic US system that can monitor the object's motion and automatically update the sweep trajectory to provide 3D compounded images of the target anatomy seamlessly. To achieve these functions, a depth camera is employed to extract the manually planned sweep trajectory after which the normal direction of the object is estimated using the extracted 3D trajectory. Subsequently, to monitor the movement and further compensate for this motion to accurately follow the trajectory, the position of firmly attached passive markers is tracked in real-time. Finally, a step-wise compounding was performed. The experiments on a gel phantom demonstrate that the system can resume a sweep when the object is not stationary during scanning.

cs.RO↗

arXiv2021

BBRv2+:Towards Balancing Aggressiveness and Fairness with Delay-based Bandwidth Probing

BBRv2, proposed by Google, aims at addressing BBR's shortcomings of unfairness against loss-based congestion control algorithms (CCAs) and excessive retransmissions in shallow-buffered networks. In this paper, we first comprehensively study BBRv2's performance under various network conditions and show that BBRv2 mitigates the shortcomings of BBR. Nevertheless, BBRv2's benefits come with several costs, including the slow responsiveness to bandwidth dynamics as well as the low resilience to random losses. We then propose BBRv2+ to address BBRv2's performance issues without sacrificing its advantages over BBR. To this end, BBRv2+ incorporates delay information into its path model, which cautiously guides the aggressiveness of its bandwidth probing to not reduce its fairness against loss-based CCAs. BBRv2+ also integrates mechanisms for improved resilience to random losses as well as network jitters. Extensive experiments demonstrate the effectiveness of BBRv2+. Especially, it achieves 25% higher throughput and comparable queuing delay in comparison with BBRv2 in high-mobility network scenarios.

cs.NI↗