arXiv subjects

Zheng Ge

Publications and source records attributed to Zheng Ge.

At least 73 records · Page 4Linked to original sources

arXiv2023

Mid-Infrared Upconversion Imaging Under Different Illumination Conditions

Converting the medium infrared field to the visible band is an effective image detection method. We propose a comprehensive theory of image up-conversion under continuous optical pumping, and discuss the relationship between the experimental parameters and imaging field of view, resolution, quantum efficiency, and conversion bandwidth. Theoretical predictions of upconversion imaging results are given based on numerical simulations, which show good agreement with experimental results. In particular, coherent and incoherent light illumination are studied separately and the advantages and disadvantages of their imaging performance are compared and analysed. This work provides a study of the upconversion image detection performance of the system, which is of great value in guiding the design of the detection system and bringing it to practical applications.

physics.optics↗

arXiv2023

BEVStereo++: Accurate Depth Estimation in Multi-view 3D Object Detection via Dynamic Temporal Stereo

Bounded by the inherent ambiguity of depth perception, contemporary multi-view 3D object detection methods fall into the performance bottleneck. Intuitively, leveraging temporal multi-view stereo (MVS) technology is the natural knowledge for tackling this ambiguity. However, traditional attempts of MVS has two limitations when applying to 3D object detection scenes: 1) The affinity measurement among all views suffers expensive computational cost; 2) It is difficult to deal with outdoor scenarios where objects are often mobile. To this end, we propose BEVStereo++: by introducing a dynamic temporal stereo strategy, BEVStereo++ is able to cut down the harm that is brought by introducing temporal stereo when dealing with those two scenarios. Going one step further, we apply Motion Compensation Module and long sequence Frame Fusion to BEVStereo++, which shows further performance boosting and error reduction. Without bells and whistles, BEVStereo++ achieves state-of-the-art(SOTA) on both Waymo and nuScenes dataset.

cs.CV↗

arXiv2023

Thermal camera based on frequency upconversion and its noise-equivalent temperature difference characterization

We present a scheme for estimating the noise-equivalent temperature difference (NETD) of frequency upconversion detectors (UCDs) that detect mid-infrared (MIR) light. In particular, this letter investigates the frequency upconversion of a periodically polarized crystal based on lithium niobate, where a mid-infrared conversion bandwidth of 220 nm can be achieved in a single poled period by a special design. Experimentally for a temperature target with a central wavelength of 7.89 μm in mid-infrared radiation, we estimated the NETD of the device to be 56 mK. Meanwhile, a direct measurement of the NETD was performed utilizing conventional methods, which resulted in 48 mK. We also compared the NETD of our UCD with commercially available direct mid-infrared detectors. Here, we showed that the limiting factor for further NETD reduction of our device is not primarily from the upconversion process and camera noise, but from the limitations of the heat source performance. Our detectors have good temperature measurement performance and can be used for a variety of applications involving temperature object identification and material structure detection.

physics.optics↗

arXiv2023

Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalization

In this paper, we analyse the generalization ability of binary classifiers for the task of deepfake detection. We find that the stumbling block to their generalization is caused by the unexpected learned identity representation on images. Termed as the Implicit Identity Leakage, this phenomenon has been qualitatively and quantitatively verified among various DNNs. Furthermore, based on such understanding, we propose a simple yet effective method named the ID-unaware Deepfake Detection Model to reduce the influence of this phenomenon. Extensive experimental results demonstrate that our method outperforms the state-of-the-art in both in-dataset and cross-dataset evaluation. The code is available at https://github.com/megvii-research/CADDM.

cs.CV↗

arXiv2023

Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?

The success of deep learning heavily relies on large-scale data with comprehensive labels, which is more expensive and time-consuming to fetch in 3D compared to 2D images or natural languages. This promotes the potential of utilizing models pretrained with data more than 3D as teachers for cross-modal knowledge transferring. In this paper, we revisit masked modeling in a unified fashion of knowledge distillation, and we show that foundational Transformers pretrained with 2D images or natural languages can help self-supervised 3D representation learning through training Autoencoders as Cross-Modal Teachers (ACT). The pretrained Transformers are transferred as cross-modal 3D teachers using discrete variational autoencoding self-supervision, during which the Transformers are frozen with prompt tuning for better knowledge inheritance. The latent features encoded by the 3D teachers are used as the target of masked point modeling, wherein the dark knowledge is distilled to the 3D Transformer students as foundational geometry understanding. Our ACT pretrained 3D learner achieves state-of-the-art generalization capacity across various downstream benchmarks, e.g., 88.21% overall accuracy on ScanObjectNN. Codes have been released at https://github.com/RunpeiDong/ACT.

cs.CV↗

arXiv2022

BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection

In this research, we propose a new 3D object detector with a trustworthy depth estimation, dubbed BEVDepth, for camera-based Bird's-Eye-View (BEV) 3D object detection. Our work is based on a key observation -- depth estimation in recent approaches is surprisingly inadequate given the fact that depth is essential to camera 3D detection. Our BEVDepth resolves this by leveraging explicit depth supervision. A camera-awareness depth estimation module is also introduced to facilitate the depth predicting capability. Besides, we design a novel Depth Refinement Module to counter the side effects carried by imprecise feature unprojection. Aided by customized Efficient Voxel Pooling and multi-frame mechanism, BEVDepth achieves the new state-of-the-art 60.9% NDS on the challenging nuScenes test set while maintaining high efficiency. For the first time, the NDS score of a camera model reaches 60%.

cs.CV↗

arXiv2022

MatrixVT: Efficient Multi-Camera to BEV Transformation for 3D Perception

This paper proposes an efficient multi-camera to Bird's-Eye-View (BEV) view transformation method for 3D perception, dubbed MatrixVT. Existing view transformers either suffer from poor transformation efficiency or rely on device-specific operators, hindering the broad application of BEV models. In contrast, our method generates BEV features efficiently with only convolutions and matrix multiplications (MatMul). Specifically, we propose describing the BEV feature as the MatMul of image feature and a sparse Feature Transporting Matrix (FTM). A Prime Extraction module is then introduced to compress the dimension of image features and reduce FTM's sparsity. Moreover, we propose the Ring \& Ray Decomposition to replace the FTM with two matrices and reformulate our pipeline to reduce calculation further. Compared to existing methods, MatrixVT enjoys a faster speed and less memory footprint while remaining deploy-friendly. Extensive experiments on the nuScenes benchmark demonstrate that our method is highly efficient but obtains results on par with the SOTA method in object detection and map segmentation tasks

cs.CV↗

arXiv2022

Towards 3D Object Detection with 2D Supervision

The great progress of 3D object detectors relies on large-scale data and 3D annotations. The annotation cost for 3D bounding boxes is extremely expensive while the 2D ones are easier and cheaper to collect. In this paper, we introduce a hybrid training framework, enabling us to learn a visual 3D object detector with massive 2D (pseudo) labels, even without 3D annotations. To break through the information bottleneck of 2D clues, we explore a new perspective: Temporal 2D Supervision. We propose a temporal 2D transformation to bridge the 3D predictions with temporal 2D labels. Two steps, including homography wraping and 2D box deduction, are taken to transform the 3D predictions into 2D ones for supervision. Experiments conducted on the nuScenes dataset show strong results (nearly 90% of its fully-supervised performance) with only 25% 3D annotations. We hope our findings can provide new insights for using a large number of 2D annotations for 3D perception.

cs.CV↗

arXiv2022

Harmonics-assisted optical phase amplifier

The change in the relative phase between two light fields serves as a basic principle for the measurement of the physical quantity that guides this change. It would therefore be highly advantageous if the relative phase could be amplified to enhance the measurement resolution. One well-known method for phase amplification involves the use of the multi-photon number and path entangled state known as the NOON state; however, a high-number NOON state is very difficult to prepare and is highly sensitive to optical losses. Here we propose and experimentally demonstrate in principle a phase amplifier scheme with the assistance of a harmonic generation process. The relative phase difference between two polarization modes in a polarized interferometer is amplified coherently four times with cascaded second-harmonic generation processes. We demonstrate that these amplification processes can be recycled and therefore have the potential to realize much higher numbers of multiple amplification steps. The phase amplification method presented here shows considerable advantages over the method based on NOON states and will be highly promising for use in precision optical measurements.

physics.optics↗

arXiv2022

BEVStereo: Enhancing Depth Estimation in Multi-view 3D Object Detection with Dynamic Temporal Stereo

Bounded by the inherent ambiguity of depth perception, contemporary camera-based 3D object detection methods fall into the performance bottleneck. Intuitively, leveraging temporal multi-view stereo (MVS) technology is the natural knowledge for tackling this ambiguity. However, traditional attempts of MVS are flawed in two aspects when applying to 3D object detection scenes: 1) The affinity measurement among all views suffers expensive computation cost; 2) It is difficult to deal with outdoor scenarios where objects are often mobile. To this end, we introduce an effective temporal stereo method to dynamically select the scale of matching candidates, enable to significantly reduce computation overhead. Going one step further, we design an iterative algorithm to update more valuable candidates, making it adaptive to moving candidates. We instantiate our proposed method to multi-view 3D detector, namely BEVStereo. BEVStereo achieves the new state-of-the-art performance (i.e., 52.5% mAP and 61.0% NDS) on the camera-only track of nuScenes dataset. Meanwhile, extensive experiments reflect our method can deal with complex outdoor scenarios better than contemporary MVS approaches. Codes have been released at https://github.com/Megvii-BaseDetection/BEVStereo.

cs.CV↗

arXiv2022

STS: Surround-view Temporal Stereo for Multi-view 3D Detection

Learning accurate depth is essential to multi-view 3D object detection. Recent approaches mainly learn depth from monocular images, which confront inherent difficulties due to the ill-posed nature of monocular depth learning. Instead of using a sole monocular depth method, in this work, we propose a novel Surround-view Temporal Stereo (STS) technique that leverages the geometry correspondence between frames across time to facilitate accurate depth learning. Specifically, we regard the field of views from all cameras around the ego vehicle as a unified view, namely surroundview, and conduct temporal stereo matching on it. The resulting geometrical correspondence between different frames from STS is utilized and combined with the monocular depth to yield final depth prediction. Comprehensive experiments on nuScenes show that STS greatly boosts 3D detection ability, notably for medium and long distance objects. On BEVDepth with ResNet-50 backbone, STS improves mAP and NDS by 2.6% and 1.4%, respectively. Consistent improvements are observed when using a larger backbone and a larger image resolution, demonstrating its effectiveness

cs.CV↗

arXiv2022

PersDet: Monocular 3D Detection in Perspective Bird's-Eye-View

Currently, detecting 3D objects in Bird's-Eye-View (BEV) is superior to other 3D detectors for autonomous driving and robotics. However, transforming image features into BEV necessitates special operators to conduct feature sampling. These operators are not supported on many edge devices, bringing extra obstacles when deploying detectors. To address this problem, we revisit the generation of BEV representation and propose detecting objects in perspective BEV -- a new BEV representation that does not require feature sampling. We demonstrate that perspective BEV features can likewise enjoy the benefits of the BEV paradigm. Moreover, the perspective BEV improves detection performance by addressing issues caused by feature sampling. We propose PersDet for high-performance object detection in perspective BEV space based on this discovery. While implementing a simple and memory-efficient structure, PersDet outperforms existing state-of-the-art monocular methods on the nuScenes benchmark, reaching 34.6% mAP and 40.8% NDS when using ResNet-50 as the backbone.

cs.CV↗

arXiv2022

Dense Teacher: Dense Pseudo-Labels for Semi-supervised Object Detection

To date, the most powerful semi-supervised object detectors (SS-OD) are based on pseudo-boxes, which need a sequence of post-processing with fine-tuned hyper-parameters. In this work, we propose replacing the sparse pseudo-boxes with the dense prediction as a united and straightforward form of pseudo-label. Compared to the pseudo-boxes, our Dense Pseudo-Label (DPL) does not involve any post-processing method, thus retaining richer information. We also introduce a region selection technique to highlight the key information while suppressing the noise carried by dense labels. We name our proposed SS-OD algorithm that leverages the DPL as Dense Teacher. On COCO and VOC, Dense Teacher shows superior performance under various settings compared with the pseudo-box-based methods.

cs.CV↗

arXiv2022

Up-conversion detection of mid-infrared light carrying orbital angular momentum

Frequency up-conversion is an effective method of mid-infrared (MIR) detection by converting the long-wavelength photons to the visible domain, where efficient detectors are readily available. Here, we generate the MIR light carrying orbital angular momentum (OAM) from a difference frequency generation process and perform the up-conversion of it via sum frequency conversion in a bulk quasi-phase-matching crystal. The maximum quantum conversion efficiencies from MIR to visible are 34.0%, 10.4%, and 3.5% for light with topological charges of 0, 1, and 2, respectively, which is achieved by utilizing an optimized strong pump light. We also verify the OAM conservation with a specially designed interferometer, and the results agree well with the numerical simulations. Our study opens up the possibilities for generating, manipulating, and detecting MIR light that carries OAM, and will have great potential for optical communications and remote sensing in the MIR regime.

physics.optics↗

arXiv2022

Measuring the tuning curve of spontaneous parameter down-conversion using a comet-tail-like pattern

The comet-tail-like interference patterns are observed using photons from the spontaneous parametric down-conversion (SPDC) process. The patterns are caused by the angular-spectrum-dependent interference and the diffraction of a blazed grating. We present the theoretical explanation and simulation results for these patterns, which are in good agreement with the experimental results. The most significant feature of the patterns is the bright parabolic contour profile, from which, one can deduce the parameter of the parabolic tuning curve of the SPDC process. This method could be helpful in designing experiments based on SPDC.

quant-ph↗

arXiv2021

Angular-spectrum-dependent interference

Optical interference is not only a fundamental phenomenon that has enabled new theories of light to be derived but it has also been used in interferometry for the measurement of small displacements, refractive index changes and surface irregularities. In a two-beam interferometer, variations in the interference fringes are used as a diagnostic for anything that causes the optical path difference (OPD) to change; therefore, for a specified OPD, greater variation in the fringes indicates better measurement sensitivity. Here, we introduce and experimentally validate an interesting optical interference phenomenon that uses photons with a structured frequency-angular spectrum, which are generated from a spontaneous parametric down-conversion process in a nonlinear crystal. This interference phenomenon is manifested as interference fringes that vary much more rapidly with increasing OPD than the corresponding fringes for equal-inclination interference; the phenomenon is parameterised using an equivalent wavelength, which under our experimental conditions is 29.38 nm or about 1/27 of the real wavelength. This phenomenon not only enriches the knowledge with regard to optical interference but also offers promise for applications in interferometry.

physics.optics↗

arXiv2021

Interference fringes in a nonlinear Michelson interferometer based on spontaneous parametric down-conversion

Quantum nonlinear interferometers (QNIs) can measure the infrared physical quantities of a sample by detecting visible photons. A QNI with Michelson geometry based on the spontaneous parametric down-conversion in a second-order nonlinear crystal is studied systematically. A simplified theoretical model of the QNI is presented. The interference visibility, coherence length, equal-inclination interference, and equal-thickness interference for the QNI are demonstrated theoretically and experimentally. As an application example of the QNI, the refractive index and the angle between two surfaces of a BBO crystal are measured using equal-inclination interference and equal-thickness interference.

quant-ph↗

arXiv2021

YOLOX: Exceeding YOLO Series in 2021

In this report, we present some experienced improvements to YOLO series, forming a new high-performance detector -- YOLOX. We switch the YOLO detector to an anchor-free manner and conduct other advanced detection techniques, i.e., a decoupled head and the leading label assignment strategy SimOTA to achieve state-of-the-art results across a large scale range of models: For YOLO-Nano with only 0.91M parameters and 1.08G FLOPs, we get 25.3% AP on COCO, surpassing NanoDet by 1.8% AP; for YOLOv3, one of the most widely used detectors in industry, we boost it to 47.3% AP on COCO, outperforming the current best practice by 3.0% AP; for YOLOX-L with roughly the same amount of parameters as YOLOv4-CSP, YOLOv5-L, we achieve 50.0% AP on COCO at a speed of 68.9 FPS on Tesla V100, exceeding YOLOv5-L by 1.8% AP. Further, we won the 1st Place on Streaming Perception Challenge (Workshop on Autonomous Driving at CVPR 2021) using a single YOLOX-L model. We hope this report can provide useful experience for developers and researchers in practical scenes, and we also provide deploy versions with ONNX, TensorRT, NCNN, and Openvino supported. Source code is at https://github.com/Megvii-BaseDetection/YOLOX.

cs.CV↗