arXiv subjects

Fan Xue

Publications and source records attributed to Fan Xue.

6 recordsLinked to original sources

arXiv2026

Image-Guided Pavement Defect Recognition in GPR Data with novel 3D Deep Learning Architecture

Ground Penetrating Radar (GPR) is a widely adopted non-destructive sensing technology for subsurface inspection in civil and transportation engineering. Despite its potential for pavement condition assessment, the large-scale application of GPR in automated inspection has two key challenges: the scarcity of annotated real-world datasets and the lack of deep learning models designed for the unique characteristics of 3-Dimensional (3D) GPR data. This study addresses these limitations by firstly introducing a cost-effective data preparation pipeline that integrates orthomosaic Red Green Blue (RGB) imagery with 3D GPR scans to generate annotated 3D GPR datasets. The proposed method uses the aligned segments of RGB and GPR data, using pavement surface images as a reference to transfer labels of surface-visible defects to corresponding GPR segments, enabling efficient large-scale annotation in a real-world dataset collected on a highway section under operation. In addition to the dataset contribution, we propose a specialised 3D Convolutional Neural Network (CNN) architecture incorporating residual connections, mixed convolutional kernel sizes, and both depthwise and channelwise attention mechanisms to enhance feature representation and defect classification. The model is evaluated on binary classification tasks for detecting patch and crack defects in pavement structures. Experimental results demonstrate that the proposed network outperforms baseline architectures across multiple evaluation metrics. Ablation studies further confirm the effectiveness of the designed architectural components. This work contributes a scalable and practical method for real-world dataset generation, along with a novel deep learning framework.

cs.CV↗

arXiv2024

Stop Reasoning! When Multimodal LLM with Chain-of-Thought Reasoning Meets Adversarial Image

Multimodal LLMs (MLLMs) with a great ability of text and image understanding have received great attention. To achieve better reasoning with MLLMs, Chain-of-Thought (CoT) reasoning has been widely explored, which further promotes MLLMs' explainability by giving intermediate reasoning steps. Despite the strong power demonstrated by MLLMs in multimodal reasoning, recent studies show that MLLMs still suffer from adversarial images. This raises the following open questions: Does CoT also enhance the adversarial robustness of MLLMs? What do the intermediate reasoning steps of CoT entail under adversarial attacks? To answer these questions, we first generalize existing attacks to CoT-based inferences by attacking the two main components, i.e., rationale and answer. We find that CoT indeed improves MLLMs' adversarial robustness against the existing attack methods by leveraging the multi-step reasoning process, but not substantially. Based on our findings, we further propose a novel attack method, termed as stop-reasoning attack, that attacks the model while bypassing the CoT reasoning process. Experiments on three MLLMs and two visual reasoning datasets verify the effectiveness of our proposed method. We show that stop-reasoning attack can result in misled predictions and outperform baseline attacks by a significant margin.

cs.CV↗

arXiv2023

Efficient assessment of window views in high-rise, high-density urban areas using 3D color City Information Models

Urban-scale quantification of window views can inform housing selection and valuation, landscape management, and urban planning. However, window views are numerous in high-rise, high-density urban areas and current automatic assessments of window views are inaccurate and time-consuming. Thus, both accurate and efficient assessment of window views is significant in improving the automation for urban-scale window view applications. The paper presents an automatic, accurate, and efficient assessment of window view indices (WVIs) of greenery, sky, waterbody, and construction using 3D color City Information Models (CIMs). The workflow includes: i) 3D semantic segmentation of photorealistic CIM and Digital Surface Model (DSM), and ii) batch computation of WVIs. Experimental results showed the estimated WVIs were more accurate (RMSE < 0.01), and the proposed method was more efficient (3.68 times faster) than Li et al.'s (2022) 2D semantic segmentation. Thus, the proposed method can facilitate large-scale WVI assessment and update in healthy high-rise, high-density urban development.

cs.CE↗

arXiv2023

Automated Machine Learning in the smart construction era:Significance and accessibility for industrial classification and regression tasks

This paper explores the application of automated machine learning (AutoML) techniques to the construction industry, a sector vital to the global economy. Traditional ML model construction methods were complex, time-consuming, reliant on data science expertise, and expensive. AutoML shows the potential to automate many tasks in ML construction and to create outperformed ML models. This paper aims to verify the feasibility of applying AutoML to industrial datasets for the smart construction domain, with a specific case study demonstrating its effectiveness. Two data challenges that were unique to industrial construction datasets are focused on, in addition to the normal steps of dataset preparation, model training, and evaluation. A real-world application case of construction project type prediction is provided to illustrate the accessibility of AutoML. By leveraging AutoML, construction professionals without data science expertise can now utilize software to process industrial data into ML models that assist in project management. The findings in this paper may bridge the gap between data-intensive smart construction practices and the emerging field of AutoML, encouraging its adoption for improved decision-making, project outcomes, and efficiency

cs.SE↗

arXiv2023

HRHD-HK: A benchmark dataset of high-rise and high-density urban scenes for 3D semantic segmentation of photogrammetric point clouds

Many existing 3D semantic segmentation methods, deep learning in computer vision notably, claimed to achieve desired results on urban point clouds. Thus, it is significant to assess these methods quantitatively in diversified real-world urban scenes, encompassing high-rise, low-rise, high-density, and low-density urban areas. However, existing public benchmark datasets primarily represent low-rise scenes from European cities and cannot assess the methods comprehensively. This paper presents a benchmark dataset of high-rise urban point clouds, namely High-Rise, High-Density urban scenes of Hong Kong (HRHD-HK). HRHD-HK arranged in 150 tiles contains 273 million colorful photogrammetric 3D points from diverse urban settings. The semantic labels of HRHD-HK include building, vegetation, road, waterbody, facility, terrain, and vehicle. To our best knowledge, HRHD-HK is the first photogrammetric dataset that focuses on HRHD urban areas. This paper also comprehensively evaluates eight popular semantic segmentation methods on the HRHD-HK dataset. Experimental results confirmed plenty of room for enhancing the current 3D semantic segmentation of point clouds, especially for city objects with small volumes. Our dataset is publicly available at https://doi.org/10.25442/hku.23701866.v2.

cs.CV↗

arXiv2021

FloorPP-Net: Reconstructing Floor Plans using Point Pillars for Scan-to-BIM

This paper presents a deep learning-based point cloud processing method named FloorPP-Net for the task of Scan-to-BIM (building information model). FloorPP-Net first converts the input point cloud of a building story into point pillars (PP), then predicts the corners and edges to output the floor plan. Altogether, FloorPP-Net establishes an end-to-end supervised learning framework for the Scan-to-Floor-Plan (Scan2FP) task. In the 1st International Scan-to-BIM Challenge held in conjunction with CVPR 2021, FloorPP-Net was ranked the second runner-up in the floor plan reconstruction track. Future work includes general edge proposals, 2D plan regularization, and 3D BIM reconstruction.

cs.CV↗