arXiv subjects

Di Feng

Publications and source records attributed to Di Feng.

At least 37 records · Page 2Linked to original sources

arXiv2023

Revisiting Multi-modal 3D Semantic Segmentation in Real-world Autonomous Driving

LiDAR and camera are two critical sensors for multi-modal 3D semantic segmentation and are supposed to be fused efficiently and robustly to promise safety in various real-world scenarios. However, existing multi-modal methods face two key challenges: 1) difficulty with efficient deployment and real-time execution; and 2) drastic performance degradation under weak calibration between LiDAR and cameras. To address these challenges, we propose CPGNet-LCF, a new multi-modal fusion framework extending the LiDAR-only CPGNet. CPGNet-LCF solves the first challenge by inheriting the easy deployment and real-time capabilities of CPGNet. For the second challenge, we introduce a novel weak calibration knowledge distillation strategy during training to improve the robustness against the weak calibration. CPGNet-LCF achieves state-of-the-art performance on the nuScenes and SemanticKITTI benchmarks. Remarkably, it can be easily deployed to run in 20ms per frame on a single Tesla V100 GPU using TensorRT TF16 mode. Furthermore, we benchmark performance over four weak calibration levels, demonstrating the robustness of our proposed approach.

cs.CV↗

arXiv2023

Priors are Powerful: Improving a Transformer for Multi-camera 3D Detection with 2D Priors

Transfomer-based approaches advance the recent development of multi-camera 3D detection both in academia and industry. In a vanilla transformer architecture, queries are randomly initialised and optimised for the whole dataset, without considering the differences among input frames. In this work, we propose to leverage the predictions from an image backbone, which is often highly optimised for 2D tasks, as priors to the transformer part of a 3D detection network. The method works by (1). augmenting image feature maps with 2D priors, (2). sampling query locations via ray-casting along 2D box centroids, as well as (3). initialising query features with object-level image features. Experimental results shows that 2D priors not only help the model converge faster, but also largely improve the baseline approach by up to 12% in terms of average precision.

cs.CV↗

arXiv2022

On the Asymptotic Performance of Affirmative Actions in School Choice

This paper analyzes the asymptotic performance of two popular affirmative action policies, majority quota and minority reserve, under the immediate acceptance mechanism (IAM) and the top trading cycles mechanism (TTCM) in the contest of school choice. The matching outcomes of these two affirmative actions are asymptotically equivalent under the IAM when all students are sincere. Given the possible preference manipulations under the IAM, we characterize the asymptotically equivalent sets of Nash equilibrium outcomes of the IAM with these two affirmative actions. However, these two affirmative actions induce different matching outcomes under the TTCM with non-negligible probability even in large markets.

econ.TH↗

arXiv2022

DeepFusion: A Robust and Modular 3D Object Detector for Lidars, Cameras and Radars

We propose DeepFusion, a modular multi-modal architecture to fuse lidars, cameras and radars in different combinations for 3D object detection. Specialized feature extractors take advantage of each modality and can be exchanged easily, making the approach simple and flexible. Extracted features are transformed into bird's-eye-view as a common representation for fusion. Spatial and semantic alignment is performed prior to fusing modalities in the feature space. Finally, a detection head exploits rich multi-modal features for improved 3D detection performance. Experimental results for lidar-camera, lidar-camera-radar and camera-radar fusion show the flexibility and effectiveness of our fusion approach. In the process, we study the largely unexplored task of faraway car detection up to 225 meters, showing the benefits of our lidar-camera fusion. Furthermore, we investigate the required density of lidar points for 3D object detection and illustrate implications at the example of robustness against adverse weather conditions. Moreover, ablation studies on our camera-radar fusion highlight the importance of accurate depth estimation.

cs.CV↗

arXiv2022

Impatient Queuing for Intelligent Task Offloading in Multi-Access Edge Computing

Multi-access edge computing (MEC) emerges as an essential part of the upcoming Fifth Generation (5G) and future beyond-5G mobile communication systems. It adds computational power towards the edge of cellular networks, much closer to energy-constrained user devices, and therewith allows the users to offload tasks to the edge computing nodes for low-latency applications with very-limited battery consumption. However, due to the high dynamics of user demand and server load, task congestion may occur at the edge nodes resulting in long queuing delay. Such delays can significantly degrade the quality of experience (QoE) of some latency-sensitive applications, raise the risk of service outage, and cannot be efficiently resolved by conventional queue management solutions. In this article, we study a latency-outage critical scenario, where users intend to limit the risk of latency outage. We propose an impatience-based queuing strategy for such users to intelligently choose between MEC offloading and local computation, allowing them to rationally renege from the task queue. The proposed approach is demonstrated by numerical simulations to be efficient for generic service model, when a perfect queue status information is available. For the practical case where the users obtain only imperfect queue status information, we design an optimal online learning strategy to enable its application in Poisson service scenarios.

cs.NI↗

arXiv2022

Understanding the Domain Gap in LiDAR Object Detection Networks

In order to make autonomous driving a reality, artificial neural networks have to work reliably in the open-world. However, the open-world is vast and continuously changing, so it is not technically feasible to collect and annotate training datasets which accurately represent this domain. Therefore, there are always domain gaps between training datasets and the open-world which must be understood. In this work, we investigate the domain gaps between high-resolution and low-resolution LiDAR sensors in object detection networks. Using a unique dataset, which enables us to study sensor resolution domain gaps independent of other effects, we show two distinct domain gaps - an inference domain gap and a training domain gap. The inference domain gap is characterised by a strong dependence on the number of LiDAR points per object, while the training gap shows no such dependence. These fndings show that different approaches are required to close these inference and training domain gaps.

cs.CV↗

arXiv2021

A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving

Capturing uncertainty in object detection is indispensable for safe autonomous driving. In recent years, deep learning has become the de-facto approach for object detection, and many probabilistic object detectors have been proposed. However, there is no summary on uncertainty estimation in deep object detection, and existing methods are not only built with different network architectures and uncertainty estimation methods, but also evaluated on different datasets with a wide range of evaluation metrics. As a result, a comparison among methods remains challenging, as does the selection of a model that best suits a particular application. This paper aims to alleviate this problem by providing a review and comparative study on existing probabilistic object detection methods for autonomous driving applications. First, we provide an overview of generic uncertainty estimation in deep learning, and then systematically survey existing methods and evaluation metrics for probabilistic object detection. Next, we present a strict comparative study for probabilistic object detection based on an image detector and three public autonomous driving datasets. Finally, we present a discussion of the remaining challenges and future works. Code has been made available at https://github.com/asharakeh/pod_compare.git

cs.CV↗

arXiv2021

Where can I drive? A System Approach: Deep Ego Corridor Estimation for Robust Automated Driving

Lane detection is an essential part of the perception sub-architecture of any automated driving (AD) or advanced driver assistance system (ADAS). When focusing on low-cost, large scale products for automated driving, model-driven approaches for the detection of lane markings have proven good performance. More recently, data-driven approaches have been proposed that target the drivable area / freespace mainly in inner-city applications. Focus of these approaches is less on lane-based driving due to the fact that the lane concept does not fully apply in unstructured, residential inner-city environments. So-far the concept of drivable area is seldom used for highway and inter-urban applications due to the specific requirements of these scenarios that require clear lane associations of all traffic participants. We believe that lane-based, mapless driving in inter-urban and highway scenarios is still not fully handled with sufficient robustness and availability. Especially for challenging weather situations such as heavy rain, fog, low-standing sun, darkness or reflections in puddles, the mapless detection of lane markings decreases significantly or completely fails. We see potential in applying specifically designed data-driven freespace approaches in more lane-based driving applications for highways and inter-urban use. Therefore, we propose to classify specifically a drivable corridor of the ego lane on pixel level with a deep learning approach. Our approach is kept computationally efficient with only 0.66 million parameters allowing its application in large scale products. Thus, we were able to easily integrate into an online AD system of a test vehicle. We demonstrate the performance of our approach under challenging conditions qualitatively and quantitatively in comparison to a state-of-the-art model-driven approach.

cs.CV↗

arXiv2021

A Simple and Efficient Multi-task Network for 3D Object Detection and Road Understanding

Detecting dynamic objects and predicting static road information such as drivable areas and ground heights are crucial for safe autonomous driving. Previous works studied each perception task separately, and lacked a collective quantitative analysis. In this work, we show that it is possible to perform all perception tasks via a simple and efficient multi-task network. Our proposed network, LidarMTL, takes raw LiDAR point cloud as inputs, and predicts six perception outputs for 3D object detection and road understanding. The network is based on an encoder-decoder architecture with 3D sparse convolution and deconvolution operations. Extensive experiments verify the proposed method with competitive accuracies compared to state-of-the-art object detectors and other task-specific networks. LidarMTL is also leveraged for online localization. Code and pre-trained model have been made available at https://github.com/frankfengdi/LidarMTL.

cs.CV↗

arXiv2020

Labels Are Not Perfect: Inferring Spatial Uncertainty in Object Detection

The availability of many real-world driving datasets is a key reason behind the recent progress of object detection algorithms in autonomous driving. However, there exist ambiguity or even failures in object labels due to error-prone annotation process or sensor observation noise. Current public object detection datasets only provide deterministic object labels without considering their inherent uncertainty, as does the common training process or evaluation metrics for object detectors. As a result, an in-depth evaluation among different object detection methods remains challenging, and the training process of object detectors is sub-optimal, especially in probabilistic object detection. In this work, we infer the uncertainty in bounding box labels from LiDAR point clouds based on a generative model, and define a new representation of the probabilistic bounding box through a spatial uncertainty distribution. Comprehensive experiments show that the proposed model reflects complex environmental noises in LiDAR perception and the label quality. Furthermore, we propose Jaccard IoU (JIoU) as a new evaluation metric that extends IoU by incorporating label uncertainty. We conduct an in-depth comparison among several LiDAR-based object detectors using the JIoU metric. Finally, we incorporate the proposed label uncertainty in a loss function to train a probabilistic object detector and to improve its detection accuracy. We verify our proposed methods on two public datasets (KITTI, Waymo), as well as on simulation data. Code is released at https://bit.ly/2W534yo.

cs.CV↗

arXiv2020

Labels Are Not Perfect: Improving Probabilistic Object Detection via Label Uncertainty

Reliable uncertainty estimation is crucial for robust object detection in autonomous driving. However, previous works on probabilistic object detection either learn predictive probability for bounding box regression in an un-supervised manner, or use simple heuristics to do uncertainty regularization. This leads to unstable training or suboptimal detection performance. In this work, we leverage our previously proposed method for estimating uncertainty inherent in ground truth bounding box parameters (which we call label uncertainty) to improve the detection accuracy of a probabilistic LiDAR-based object detector. Experimental results on the KITTI dataset show that our method surpasses both the baseline model and the models based on simple heuristics by up to 3.6% in terms of Average Precision.

cs.CV↗

arXiv2020

Inferring Spatial Uncertainty in Object Detection

The availability of real-world datasets is the prerequisite for developing object detection methods for autonomous driving. While ambiguity exists in object labels due to error-prone annotation process or sensor observation noises, current object detection datasets only provide deterministic annotations without considering their uncertainty. This precludes an in-depth evaluation among different object detection methods, especially for those that explicitly model predictive probability. In this work, we propose a generative model to estimate bounding box label uncertainties from LiDAR point clouds, and define a new representation of the probabilistic bounding box through spatial distribution. Comprehensive experiments show that the proposed model represents uncertainties commonly seen in driving scenarios. Based on the spatial distribution, we further propose an extension of IoU, called the Jaccard IoU (JIoU), as a new evaluation metric that incorporates label uncertainty. Experiments on the KITTI and the Waymo Open Datasets show that JIoU is superior to IoU when evaluating probabilistic object detectors.

cs.CV↗

arXiv2020

Multiservice-based Network Slicing Orchestration with Impatient Tenants

The combination of recent emerging technologies such as network function virtualization (NFV) and network programmability (SDN) gave birth to the novel Network Slicing paradigm. 5G networks consist of multi-tenant infrastructures capable of offering leased network "slices" to new customers (e.g., vertical industries) enabling a new telecom business model: Slice-as-a-Service (SlaaS). However, as the service demand gets increasingly dense, slice requests congestion may occur leading to undesired waiting periods. This may turn into impatient tenant behaviors that increase potential loss of the business attractiveness to customers. In this paper, we aim to i study the slicing admission control problem by means of a multi-queuing system for heterogeneous tenant requests, ii) derive its statistical behavior model, iii) find out the rational strategy of impatient tenants waiting in queue-based slice admission control systems, iv) prove mathematically and empirically the benefits of allowing infrastructure providers to share its information with the upcoming tenants, and v) provide a utility model for network slices admission optimization. Our results analyze the capability of the proposed SlaaS system to be approximately Markovian and evaluate its performance as compared to a baseline solution.

cs.NI↗

arXiv2020

Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges

Recent advancements in perception for autonomous driving are driven by deep learning. In order to achieve robust and accurate scene understanding, autonomous vehicles are usually equipped with different sensors (e.g. cameras, LiDARs, Radars), and multiple sensing modalities can be fused to exploit their complementary properties. In this context, many methods have been proposed for deep multi-modal perception problems. However, there is no general guideline for network architecture design, and questions of "what to fuse", "when to fuse", and "how to fuse" remain open. This review paper attempts to systematically summarize methodologies and discuss challenges for deep multi-modal object detection and semantic segmentation in autonomous driving. To this end, we first provide an overview of on-board sensors on test vehicles, open datasets, and background information for object detection and semantic segmentation in autonomous driving research. We then summarize the fusion methodologies and discuss challenges and open questions. In the appendix, we provide tables that summarize topics and methods. We also provide an interactive online platform to navigate each reference: https://boschresearch.github.io/multimodalperception/.

cs.RO↗

arXiv2020

Leveraging Uncertainties for Deep Multi-modal Object Detection in Autonomous Driving

This work presents a probabilistic deep neural network that combines LiDAR point clouds and RGB camera images for robust, accurate 3D object detection. We explicitly model uncertainties in the classification and regression tasks, and leverage uncertainties to train the fusion network via a sampling mechanism. We validate our method on three datasets with challenging real-world driving scenarios. Experimental results show that the predicted uncertainties reflect complex environmental uncertainty like difficulties of a human expert to label objects. The results also show that our method consistently improves the Average Precision by up to 7% compared to the baseline method. When sensors are temporally misaligned, the sampling method improves the Average Precision by up to 20%, showing its high robustness against noisy sensor inputs.

cs.RO↗

arXiv2019

Rational Impatience Admission Control in 5G-sliced Networks: Shall I Bide my Slice Opportunity?

Recently, the specifications of the fifth generation (5G) of mobile networks have been released witnessing the industrial interest around the novel network slicing concept. This context is further enriched by the evolution of an emerging type of public cloud environment: the Slice-as-a-Service (SlaaS). Infrastructure providers or network operators deploy different admission strategy while processing network slice requests issued by infrastructure tenants based on service level agreements (SLAs) and current (and predicted) resource availability. However, when the service demand will reasonably get dense, congestions of slice requests may occur leading to long waiting periods. This may turn into impatient tenant behaviors that mitigate potential loss. This paper studies the rational strategies of impatient tenants waiting in queue-based slice admission control system, proving mathematically and empirically the benefits of allowing infrastructure provider to share its information with the upcoming tenants.

cs.NI↗

arXiv2019

Can We Trust You? On Calibration of a Probabilistic Object Detector for Autonomous Driving

Reliable uncertainty estimation is crucial for perception systems in safe autonomous driving. Recently, many methods have been proposed to model uncertainties in deep learning based object detectors. However, the estimated probabilities are often uncalibrated, which may lead to severe problems in safety critical scenarios. In this work, we identify such uncertainty miscalibration problems in a probabilistic LiDAR 3D object detection network, and propose three practical methods to significantly reduce errors in uncertainty calibration. Extensive experiments on several datasets show that our methods produce well-calibrated uncertainties, and generalize well between different datasets.

cs.RO↗

arXiv2019

Leveraging Heteroscedastic Aleatoric Uncertainties for Robust Real-Time LiDAR 3D Object Detection

We present a robust real-time LiDAR 3D object detector that leverages heteroscedastic aleatoric uncertainties to significantly improve its detection performance. A multi-loss function is designed to incorporate uncertainty estimations predicted by auxiliary output layers. Using our proposed method, the network ignores to train from noisy samples, and focuses more on informative ones. We validate our method on the KITTI object detection benchmark. Our method surpasses the baseline method which does not explicitly estimate uncertainties by up to nearly 9% in terms of Average Precision (AP). It also produces state-of-the-art results compared to other methods while running with an inference time of only 72 ms. In addition, we conduct extensive experiments to understand how aleatoric uncertainties behave. Extracting aleatoric uncertainties brings almost no additional computation cost during the deployment, making our method highly desirable for autonomous driving applications.

cs.RO↗