arXiv subjects

Jilin Mei

Publications and source records attributed to Jilin Mei.

27 records · Page 2Linked to original sources

arXiv2025

Proto-OOD: Enhancing OOD Object Detection with Prototype Feature Similarity

Neural networks that are trained on limited category samples often mispredict out-of-distribution (OOD) objects. We observe that features of the same category are more tightly clustered in feature space, while those of different categories are more dispersed. Based on this, we propose using prototype similarity for OOD detection. Drawing on widely used prototype features in few-shot learning, we introduce a novel OOD detection network structure (Proto-OOD). Proto-OOD enhances the representativeness of category prototypes using contrastive loss and detects OOD data by evaluating the similarity between input features and category prototypes. During training, Proto-OOD generates OOD samples for training the similarity module with a negative embedding generator. When Pascal VOC are used as the in-distribution dataset and MS-COCO as the OOD dataset, Proto-OOD significantly reduces the FPR (false positive rate). Moreover, considering the limitations of existing evaluation metrics, we propose a more reasonable evaluation protocol. The code will be released.

cs.CV↗

arXiv2024

WildOcc: A Benchmark for Off-Road 3D Semantic Occupancy Prediction

3D semantic occupancy prediction is an essential part of autonomous driving, focusing on capturing the geometric details of scenes. Off-road environments are rich in geometric information, therefore it is suitable for 3D semantic occupancy prediction tasks to reconstruct such scenes. However, most of researches concentrate on on-road environments, and few methods are designed for off-road 3D semantic occupancy prediction due to the lack of relevant datasets and benchmarks. In response to this gap, we introduce WildOcc, to our knowledge, the first benchmark to provide dense occupancy annotations for off-road 3D semantic occupancy prediction tasks. A ground truth generation pipeline is proposed in this paper, which employs a coarse-to-fine reconstruction to achieve a more realistic result. Moreover, we introduce a multi-modal 3D semantic occupancy prediction framework, which fuses spatio-temporal information from multi-frame images and point clouds at voxel level. In addition, a cross-modality distillation function is introduced, which transfers geometric knowledge from point clouds to image features.

cs.CV↗

arXiv2024

TeFF: Tracking-enhanced Forgetting-free Few-shot 3D LiDAR Semantic Segmentation

In autonomous driving, 3D LiDAR plays a crucial role in understanding the vehicle's surroundings. However, the newly emerged, unannotated objects presents few-shot learning problem for semantic segmentation. This paper addresses the limitations of current few-shot semantic segmentation by exploiting the temporal continuity of LiDAR data. Employing a tracking model to generate pseudo-ground-truths from a sequence of LiDAR frames, our method significantly augments the dataset, enhancing the model's ability to learn on novel classes. However, this approach introduces a data imbalance biased to novel data that presents a new challenge of catastrophic forgetting. To mitigate this, we incorporate LoRA, a technique that reduces the number of trainable parameters, thereby preserving the model's performance on base classes while improving its adaptability to novel classes. This work represents a significant step forward in few-shot 3D LiDAR semantic segmentation for autonomous driving. Our code is available at https://github.com/junbao-zhou/Track-no-forgetting.

cs.CV↗

arXiv2023

Few-shot 3D LiDAR Semantic Segmentation for Autonomous Driving

In autonomous driving, the novel objects and lack of annotations challenge the traditional 3D LiDAR semantic segmentation based on deep learning. Few-shot learning is a feasible way to solve these issues. However, currently few-shot semantic segmentation methods focus on camera data, and most of them only predict the novel classes without considering the base classes. This setting cannot be directly applied to autonomous driving due to safety concerns. Thus, we propose a few-shot 3D LiDAR semantic segmentation method that predicts both novel classes and base classes simultaneously. Our method tries to solve the background ambiguity problem in generalized few-shot semantic segmentation. We first review the original cross-entropy and knowledge distillation losses, then propose a new loss function that incorporates the background information to achieve 3D LiDAR few-shot semantic segmentation. Extensive experiments on SemanticKITTI demonstrate the effectiveness of our method.

cs.RO↗

arXiv2023

PA&DA: Jointly Sampling PAth and DAta for Consistent NAS

Based on the weight-sharing mechanism, one-shot NAS methods train a supernet and then inherit the pre-trained weights to evaluate sub-models, largely reducing the search cost. However, several works have pointed out that the shared weights suffer from different gradient descent directions during training. And we further find that large gradient variance occurs during supernet training, which degrades the supernet ranking consistency. To mitigate this issue, we propose to explicitly minimize the gradient variance of the supernet training by jointly optimizing the sampling distributions of PAth and DAta (PA&DA). We theoretically derive the relationship between the gradient variance and the sampling distributions, and reveal that the optimal sampling probability is proportional to the normalized gradient norm of path and training data. Hence, we use the normalized gradient norm as the importance indicator for path and training data, and adopt an importance sampling strategy for the supernet training. Our method only requires negligible computation cost for optimizing the sampling distributions of path and data, but achieves lower gradient variance during supernet training and better generalization performance for the supernet, resulting in a more consistent NAS. We conduct comprehensive comparisons with other improved approaches in various search spaces. Results show that our method surpasses others with more reliable ranking performance and higher accuracy of searched architectures, showing the effectiveness of our method. Code is available at https://github.com/ShunLu91/PA-DA.

cs.CV↗

arXiv2020

Scene Context Based Semantic Segmentation for 3D LiDAR Data in Dynamic Scene

We propose a graph neural network(GNN) based method to incorporate scene context for the semantic segmentation of 3D LiDAR data. The problem is defined as building a graph to represent the topology of a center segment with its neighborhoods, then inferring the segment label. The node of graph is generated from the segment on range image, which is suitable for both sparse and dense point cloud. Edge weights that evaluate the correlations of center node and its neighborhoods are automatically encoded by a neural network, therefore the number of neighbor nodes is no longer a sensitive parameter. A system consists of segment generation, graph building, edge weight estimation, node updating, and node prediction is designed. Quantitative evaluation on a dataset of dynamic scene shows that our method has better performance than unary CNN with 8% improvement, as well as normal GNN with 17% improvement.

cs.RO↗

arXiv2020

SemanticPOSS: A Point Cloud Dataset with Large Quantity of Dynamic Instances

3D semantic segmentation is one of the key tasks for autonomous driving system. Recently, deep learning models for 3D semantic segmentation task have been widely researched, but they usually require large amounts of training data. However, the present datasets for 3D semantic segmentation are lack of point-wise annotation, diversiform scenes and dynamic objects. In this paper, we propose the SemanticPOSS dataset, which contains 2988 various and complicated LiDAR scans with large quantity of dynamic instances. The data is collected in Peking University and uses the same data format as SemanticKITTI. In addition, we evaluate several typical 3D semantic segmentation models on our SemanticPOSS dataset. Experimental results show that SemanticPOSS can help to improve the prediction accuracy of dynamic objects as people, car in some degree. SemanticPOSS will be published at \url{www.poss.pku.edu.cn}.

cs.RO↗

arXiv2019

Incorporating Human Domain Knowledge in 3D LiDAR-based Semantic Segmentation

This work studies semantic segmentation using 3D LiDAR data. Popular deep learning methods applied for this task require a large number of manual annotations to train the parameters. We propose a new method that makes full use of the advantages of traditional methods and deep learning methods via incorporating human domain knowledge into the neural network model to reduce the demand for large numbers of manual annotations and improve the training efficiency. We first pretrain a model with autogenerated samples from a rule-based classifier so that human knowledge can be propagated into the network. Based on the pretrained model, only a small set of annotations is required for further fine-tuning. Quantitative experiments show that the pretrained model achieves better performance than random initialization in almost all cases; furthermore, our method can achieve similar performance with fewer manual annotations.

cs.RO↗

arXiv2018

Semantic Segmentation of 3D LiDAR Data in Dynamic Scene Using Semi-supervised Learning

This work studies the semantic segmentation of 3D LiDAR data in dynamic scenes for autonomous driving applications. A system of semantic segmentation using 3D LiDAR data, including range image segmentation, sample generation, inter-frame data association, track-level annotation and semi-supervised learning, is developed. To reduce the considerable requirement of fine annotations, a CNN-based classifier is trained by considering both supervised samples with manually labeled object classes and pairwise constraints, where a data sample is composed of a segment as the foreground and neighborhood points as the background. A special loss function is designed to account for both annotations and constraints, where the constraint data are encouraged to be assigned to the same semantic class. A dataset containing 1838 frames of LiDAR data, 39934 pairwise constraints and 57927 human annotations is developed. The performance of the method is examined extensively. Qualitative and quantitative experiments show that the combination of a few annotations and large amount of constraint data significantly enhances the effectiveness and scene adaptability, resulting in greater than 10% improvement

cs.RO↗