arXiv subjects

Qing Zhu

Publications and source records attributed to Qing Zhu.

49 records · Page 3Linked to original sources

arXiv2021

Depth-Enhanced Feature Pyramid Network for Occlusion-Aware Verification of Buildings from Oblique Images

Detecting the changes of buildings in urban environments is essential. Existing methods that use only nadir images suffer from severe problems of ambiguous features and occlusions between buildings and other regions. Furthermore, buildings in urban environments vary significantly in scale, which leads to performance issues when using single-scale features. To solve these issues, this paper proposes a fused feature pyramid network, which utilizes both color and depth data for the 3D verification of existing buildings 2D footprints from oblique images. First, the color data of oblique images are enriched with the depth information rendered from 3D mesh models. Second, multiscale features are fused in the feature pyramid network to convolve both the color and depth data. Finally, multi-view information from both the nadir and oblique images is used in a robust voting procedure to label changes in existing buildings. Experimental evaluations using both the ISPRS benchmark datasets and Shenzhen datasets reveal that the proposed method outperforms the ResNet and EfficientNet networks by 5\% and 2\%, respectively, in terms of recall rate and precision. We demonstrate that the proposed method can successfully detect all changed buildings; therefore, only those marked as changed need to be manually checked during the pipeline updating procedure; this significantly reduces the manual quality control requirements. Moreover, ablation studies indicate that using depth data, feature pyramid modules, and multi-view voting strategies can lead to clear and progressive improvements.

cs.CV↗

arXiv2020

MAP-Net: Multi Attending Path Neural Network for Building Footprint Extraction from Remote Sensed Imagery

Accurately and efficiently extracting building footprints from a wide range of remote sensed imagery remains a challenge due to their complex structure, variety of scales and diverse appearances. Existing convolutional neural network (CNN)-based building extraction methods are complained that they cannot detect the tiny buildings because the spatial information of CNN feature maps are lost during repeated pooling operations of the CNN, and the large buildings still have inaccurate segmentation edges. Moreover, features extracted by a CNN are always partial which restricted by the size of the respective field, and large-scale buildings with low texture are always discontinuous and holey when extracted. This paper proposes a novel multi attending path neural network (MAP-Net) for accurately extracting multiscale building footprints and precise boundaries. MAP-Net learns spatial localization-preserved multiscale features through a multi-parallel path in which each stage is gradually generated to extract high-level semantic features with fixed resolution. Then, an attention module adaptively squeezes channel-wise features from each path for optimization, and a pyramid spatial pooling module captures global dependency for refining discontinuous building footprints. Experimental results show that MAP-Net outperforms state-of-the-art (SOTA) algorithms in boundary localization accuracy as well as continuity of large buildings. Specifically, our method achieved 0.68\%, 1.74\%, 1.46\% precision, and 1.50\%, 1.53\%, 0.82\% IoU score improvement without increasing computational complexity compared with the latest HRNetv2 on the Urban 3D, Deep Globe and WHU datasets, respectively. The TensorFlow implementation is available at https://github.com/lehaifeng/MAPNet.

cs.CV↗

arXiv2020

Minimum Potential Energy of Point Cloud for Robust Global Registration

In this paper, we propose a novel minimum gravitational potential energy (MPE)-based algorithm for global point set registration. The feature descriptors extraction algorithms have emerged as the standard approach to align point sets in the past few decades. However, the alignment can be challenging to take effect when the point set suffers from raw point data problems such as noises (Gaussian and Uniformly). Different from the most existing point set registration methods which usually extract the descriptors to find correspondences between point sets, our proposed MPE alignment method is able to handle large scale raw data offset without depending on traditional descriptors extraction, whether for the local or global registration methods. We decompose the solution into a global optimal convex approximation and the fast descent process to a local minimum. For the approximation step, the proposed minimum potential energy (MPE) approach consists of two main steps. Firstly, according to the construction of the force traction operator, we could simply compute the position of the potential energy minimum; Secondly, with respect to the finding of the MPE point, we propose a new theory that employs the two flags to observe the status of the registration procedure. The method of fast descent process to the minimum that we employed is the iterative closest point algorithm; it can achieve the global minimum. We demonstrate the performance of the proposed algorithm on synthetic data as well as on real data. The proposed method outperforms the other global methods in terms of both efficiency, accuracy and noise resistance.

cs.CV↗

arXiv2020

Leveraging Photogrammetric Mesh Models for Aerial-Ground Feature Point Matching Toward Integrated 3D Reconstruction

Integration of aerial and ground images has been proved as an efficient approach to enhance the surface reconstruction in urban environments. However, as the first step, the feature point matching between aerial and ground images is remarkably difficult, due to the large differences in viewpoint and illumination conditions. Previous studies based on geometry-aware image rectification have alleviated this problem, but the performance and convenience of this strategy is limited by several flaws, e.g. quadratic image pairs, segregated extraction of descriptors and occlusions. To address these problems, we propose a novel approach: leveraging photogrammetric mesh models for aerial-ground image matching. The methods of this proposed approach have linear time complexity with regard to the number of images, can explicitly handle low overlap using multi-view images and can be directly injected into off-the-shelf structure-from-motion (SfM) and multi-view stereo (MVS) solutions. First, aerial and ground images are reconstructed separately and initially co-registered through weak georeferencing data. Second, aerial models are rendered to the initial ground views, in which the color, depth and normal images are obtained. Then, the synthesized color images and the corresponding ground images are matched by comparing the descriptors, filtered by local geometrical information, and then propagated to the aerial views using depth images and patch-based matching. Experimental evaluations using various datasets confirm the superior performance of the proposed methods in aerial-ground image matching. In addition, incorporation of the existing SfM and MVS solutions into these methods enables more complete and accurate models to be directly obtained.

cs.CV↗

arXiv2020

Fast and Regularized Reconstruction of Building Façades from Street-View Images using Binary Integer Programming

Regularized arrangement of primitives on building façades to aligned locations and consistent sizes is important towards structured reconstruction of urban environment. Mixed integer linear programing was used to solve the problem, however, it is extreamly time consuming even for state-of-the-art commercial solvers. Aiming to alleviate this issue, we cast the problem into binary integer programming, which omits the requirements for real value parameters and is more efficient to be solved . Firstly, the bounding boxes of the primitives are detected using the YOLOv3 architecture in real-time. Secondly, the coordinates of the upper left corners and the sizes of the bounding boxes are automatically clustered in a binary integer programming optimization, which jointly considers the geometric fitness, regularity and additional constraints; this step does not require \emph{a priori} knowledge, such as the number of clusters or pre-defined grammars. Finally, the regularized bounding boxes can be directly used to guide the façade reconstruction in an interactive envinronment. Experimental evaluations have revealed that the accuracies for the extraction of primitives are above 0.82, which is sufficient for the following 3D reconstruction. The proposed approach only takes about $ 10\% $ to $ 20\% $ of the runtime than previous approach and reduces the diversity of the bounding boxes to about $20\%$ to $50\%$.

cs.CV↗

arXiv2020

Deep Fusion of Local and Non-Local Features for Precision Landslide Recognition

Precision mapping of landslide inventory is crucial for hazard mitigation. Most landslides generally co-exist with other confusing geological features, and the presence of such areas can only be inferred unambiguously at a large scale. In addition, local information is also important for the preservation of object boundaries. Aiming to solve this problem, this paper proposes an effective approach to fuse both local and non-local features to surmount the contextual problem. Built upon the U-Net architecture that is widely adopted in the remote sensing community, we utilize two additional modules. The first one uses dilated convolution and the corresponding atrous spatial pyramid pooling, which enlarged the receptive field without sacrificing spatial resolution or increasing memory usage. The second uses a scale attention mechanism to guide the up-sampling of features from the coarse level by a learned weight map. In implementation, the computational overhead against the original U-Net was only a few convolutional layers. Experimental evaluations revealed that the proposed method outperformed state-of-the-art general-purpose semantic segmentation approaches. Furthermore, ablation studies have shown that the two models afforded extensive enhancements in landslide-recognition performance.

eess.IV↗

arXiv2019

PointNLM: Point Nonlocal-Means for vegetation segmentation based on middle echo point clouds

Middle-echo, which covers one or a few corresponding points, is a specific type of 3D point cloud acquired by a multi-echo laser scanner. In this paper, we propose a novel approach for automatic segmentation of trees that leverages middle-echo information from LiDAR point clouds. First, using a convolution classification method, the proposed type of point clouds reflected by the middle echoes are identified from all point clouds. The middle-echo point clouds are distinguished from the first and last echoes. Hence, the crown positions of the trees are quickly detected from the huge number of point clouds. Second, to accurately extract trees from all point clouds, we propose a 3D deep learning network, PointNLM, to semantically segment tree crowns. PointNLM captures the long-range relationship between the point clouds via a non-local branch and extracts high-level features via max-pooling applied to unordered points. The whole framework is evaluated using the Semantic 3D reduced-test set. The IoU of tree point cloud segmentation reached 0.864. In addition, the semantic segmentation network was tested using the Paris-Lille-3D dataset. The average IoU outperformed several other popular methods. The experimental results indicate that the proposed algorithm provides an excellent solution for vegetation segmentation from LiDAR point clouds.

cs.CV↗

arXiv2019

Measuring Road Network Topology Vulnerability by Ricci Curvature

Describing the basic properties of road network systems, such as their robustness, vulnerability, and reliability, has been a very important research topic in the field of urban transportation. Current research mainly uses several statistical indicators of complex networks to analyze the road network systems. However, these methods are essentially node-based. These node-based methods are more concerned with the number of connections between nodes, and lack of consideration for interactions. So, this leads to the well-known node paradox problem, and their ability of characterizing the local and intrinsic properties of a network is weak. From the perspective of network intrinsic geometry, this paper proposes a method for measuring road network vulnerability using a discrete Ricci curvature, which can identify the key sections of a road network and indicate its fragile elements. The results show that our method performs better than complex network statistics on measuring the vulnerability of a road network. Additionally, it can characterize the evolution of the road network vulnerability among different periods of time in the same city through our method. Finally, we compare our method with the previous method of centrality and show the different between them. This article provides a new perspective on a geometry to analyze the vulnerability of a road network and describes the inherent nature of the vulnerability of a road system from a new perspective. It also contributes to enriching the analytical methods of complex road networks.

cs.SI↗

arXiv2018

Dynamic interfacial polaron enhanced superconductivity of FeSe/SrTiO3

The observation of substantially enhanced superconductivity of single-layer FeSe films on SrTiO3 has stimulated intensive research interest. At present, conclusive experimental data on the corresponding electron-boson interaction is still missing. Here we use inelastic electron scattering spectroscopy and angle resolved photoemission spectroscopy to show that the electrons in these systems are dressed by the strongly polarized lattice distortions of the SrTiO3, and the indispensable non-adiabatic nature of such a coupling leads to the formation of dynamic interfacial polarons. Furthermore, the collective motion of the polarons results in a polaronic plasmon mode, which is unambiguously correlated with the surface phonons of SrTiO3 in the presence of the FeSe films. A microscopic model is developed showing that the interfacial polaron-polaron interaction leads to the superconductivity enhancement.

cond-mat.supr-con↗

arXiv2018

Overcoming Catastrophic Forgetting by Soft Parameter Pruning

Catastrophic forgetting is a challenge issue in continual learning when a deep neural network forgets the knowledge acquired from the former task after learning on subsequent tasks. However, existing methods try to find the joint distribution of parameters shared with all tasks. This idea can be questionable because this joint distribution may not present when the number of tasks increase. On the other hand, It also leads to "long-term" memory issue when the network capacity is limited since adding tasks will "eat" the network capacity. In this paper, we proposed a Soft Parameters Pruning (SPP) strategy to reach the trade-off between short-term and long-term profit of a learning model by freeing those parameters less contributing to remember former task domain knowledge to learn future tasks, and preserving memories about previous tasks via those parameters effectively encoding knowledge about tasks at the same time. The SPP also measures the importance of parameters by information entropy in a label free manner. The experiments on several tasks shows SPP model achieved the best performance compared with others state-of-the-art methods. Experiment results also indicate that our method is less sensitive to hyper-parameter and better generalization. Our research suggests that a softer strategy, i.e. approximate optimize or sub-optimal solution, will benefit alleviating the dilemma of memory. The source codes are available at https://github.com/lehaifeng/Learning_by_memory.

cs.LG↗

arXiv2018

Learning to Measure Change: Fully Convolutional Siamese Metric Networks for Scene Change Detection

A critical challenge problem of scene change detection is that noisy changes generated by varying illumination, shadows and camera viewpoint make variances of a scene difficult to define and measure since the noisy changes and semantic ones are entangled. Following the intuitive idea of detecting changes by directly comparing dissimilarities between a pair of features, we propose a novel fully Convolutional siamese metric Network(CosimNet) to measure changes by customizing implicit metrics. To learn more discriminative metrics, we utilize contrastive loss to reduce the distance between the unchanged feature pairs and to enlarge the distance between the changed feature pairs. Specifically, to address the issue of large viewpoint differences, we propose Thresholded Contrastive Loss (TCL) with a more tolerant strategy to punish noisy changes. We demonstrate the effectiveness of the proposed approach with experiments on three challenging datasets: CDnet, PCD2015, and VL-CMU-CD. Our approach is robust to lots of challenging conditions, such as illumination changes, large viewpoint difference caused by camera motion and zooming. In addition, we incorporate the distance metric into the segmentation framework and validate the effectiveness through visualization of change maps and feature distribution. The source code is available at https://github.com/gmayday1997/ChangeDet.

cs.CV↗

arXiv2018

Lattice Dynamics of Ultrathin FeSe Films on SrTiO3

Charge transfer and electron-phonon coupling (EPC) are proposed to be two important constituents associated with enhanced superconductivity in the single unit cell FeSe films on oxide surfaces. Using high-resolution electron energy loss spectroscopy combined with first-principles calculations, we have explored the lattice dynamics of ultrathin FeSe films grown on SrTiO3. We show that, despite the significant effect from the substrate on the electronic structure and superconductivity of the system, the FeSe phonons in the films are unaffected. The energy dispersion and linewidth associated with the Fe- and Se-derived vibrational modes are thickness- and temperature-independent. Theoretical calculations indicate the crucial role of antiferromagnetic correlation in FeSe to reproduce the experimental phonon dispersion. Importantly, the only detectable change due to the growth of FeSe films is the broadening of the Fuchs-Kliewer (F-K) phonons associated with the lattice vibrations of SrTiO$_3$(001) substrate. If EPC plays any role in the enhancement of film superconductivity, it must be the interfacial coupling between the electrons in FeSe film and the F-K phonons from substrate rather than the phonons of FeSe.

cond-mat.supr-con↗

arXiv2017

Superconducting transition of FeSe/SrTiO3 induced by adsorption of semiconducting organic molecules

We prepared superconducting and non-superconducting FeSe films on SrTiO3(001) substrates (FeSe/STO) and investigated the superconducting transition induced by charge transfer between organic molecules and FeSe layers by low temperature scanning tunneling microscopy and spectroscopy. At low coverage, donor- and acceptor-type molecules adsorbed preferentially on the non-superconducting and superconducting FeSe layers, respectively. Superconductivity was induced by donor molecules on non-superconducting FeSe layer, while the superconductivity was suppressed near acceptor molecules. The corresponding evolutions of electronic states and work function were also resolved by scanning tunneling microscopy. These results illustrate the important role played by local electron concentration in the superconducting transition of FeSe/STO.

cond-mat.supr-con↗