arXiv subjects

Ming Liu

Publications and source records attributed to Ming Liu.

At least 343 records · Page 19Linked to original sources

arXiv2021

On Bundle Adjustment for Multiview PointCloud Registration

Multiview registration is used to estimate Rigid Body Transformations (RBTs) from multiple frames and reconstruct a scene with corresponding scans. Despite the success of pairwise registration and pose synchronization, the concept of Bundle Adjustment (BA) has been proven to better maintain global consistency. So in this work, we make the multiview point-cloud registration more tractable from a different perspective in resolving range-based BA. Based on this analysis, we propose an objective function that takes both measurement noises and computational cost into account. For the feature parameter update, instead of calculating the global distribution parameters from the raw measurements, we aggregate the local distributions upon the pose update at each iteration. The computational cost of feature update is then only dependent on the number of scans. Finally, we develop a multiview registration system using voxel-based quantization that can be applied in real-world scenarios. The experimental results demonstrate our superiority over the baselines in terms of both accuracy and speed. Moreover, the results also show that our average positioning errors achieve the centimeter level.

cs.RO↗

arXiv2021

Incorporating Learnt Local and Global Embeddings into Monocular Visual SLAM

Traditional approaches for Visual Simultaneous Localization and Mapping (VSLAM) rely on low-level vision information for state estimation, such as handcrafted local features or the image gradient. While significant progress has been made through this track, under more challenging configuration for monocular VSLAM, e.g., varying illumination, the performance of state-of-the-art systems generally degrades. As a consequence, robustness and accuracy for monocular VSLAM are still widely concerned. This paper presents a monocular VSLAM system that fully exploits learnt features for better state estimation. The proposed system leverages both learnt local features and global embeddings at different modules of the system: direct camera pose estimation, inter-frame feature association, and loop closure detection. With a probabilistic explanation of keypoint prediction, we formulate the camera pose tracking in a direct manner and parameterize local features with uncertainty taken into account. To alleviate the quantization effect, we adapt the mapping module to generate 3D landmarks better to guarantee the system's robustness. Detecting temporal loop closure via deep global embeddings further improves the robustness and accuracy of the proposed system. The proposed system is extensively evaluated on public datasets (Tsukuba, EuRoC, and KITTI), and compared against the state-of-the-art methods. The competitive performance of camera pose estimation confirms the effectiveness of our method.

cs.RO↗

arXiv2021

DiGNet: Learning Scalable Self-Driving Policies for Generic Traffic Scenarios with Graph Neural Networks

Traditional decision and planning frameworks for self-driving vehicles (SDVs) scale poorly in new scenarios, thus they require tedious hand-tuning of rules and parameters to maintain acceptable performance in all foreseeable cases. Recently, self-driving methods based on deep learning have shown promising results with better generalization capability but less hand engineering effort. However, most of the previous learning-based methods are trained and evaluated in limited driving scenarios with scattered tasks, such as lane-following, autonomous braking, and conditional driving. In this paper, we propose a graph-based deep network to achieve scalable self-driving that can handle massive traffic scenarios. Specifically, more than 7,000 km of evaluation is conducted in a high-fidelity driving simulator, in which our method can obey the traffic rules and safely navigate the vehicle in a large variety of urban, rural, and highway environments, including unprotected left turns, narrow roads, roundabouts, and pedestrian-rich intersections. Demonstration videos are available at https://caipeide.github.io/dignet/.

cs.RO↗

arXiv2021

One-dimensional van der Waals Heterostructures: Growth Mechanism and Handedness Correlation Revealed by Non-destructive TEM

In this work, we perform a follow-up and comprehensive study on the structural details and formation mechanism of chemical vapor deposition (CVD) synthesized one-dimensional (1D) van der Waals heterostructures. Edge structures, nucleation sites, crystal epitaxial relationships are clearly revealed using transmission electron microscopy (TEM). This is achieved by the direct synthesis of heteronanotubes on a CVD-compatible Si/SiO2 TEM grid, which enabled a transfer-free and non-destructive access to many intrinsic structural details. In particular, we have distinguished different shaped boron nitride nanotube (BNNT) edges, which are confirmed, by electron diffraction at the same location, to be strictly associated with its own chiral angle and polarity. We also demonstrate the importance of surface cleanness and isolation for the formation of perfect 1D heterostructures. Furthermore, we elucidate the handedness correlation between SWCNT template and BNNT crystals. This work not only provides an in-depth understanding of this new 1D heterostructure material group, but also, in a more general perspective, serves as an interesting investigation on crystal growth on highly curved (radius of a couple of nm) atomic substrates.

cond-mat.mtrl-sci↗

arXiv2021

Federated Learning Meets Natural Language Processing: A Survey

Federated Learning aims to learn machine learning models from multiple decentralized edge devices (e.g. mobiles) or servers without sacrificing local data privacy. Recent Natural Language Processing techniques rely on deep learning and large pre-trained language models. However, both big deep neural and language models are trained with huge amounts of data which often lies on the server side. Since text data is widely originated from end users, in this work, we look into recent NLP models and techniques which use federated learning as the learning framework. Our survey discusses major challenges in federated natural language processing, including the algorithm challenges, system challenges as well as the privacy issues. We also provide a critical review of the existing Federated NLP evaluation methods and tools. Finally, we highlight the current research gaps and future directions.

cs.CL↗

arXiv2021

CP-loss: Connectivity-preserving Loss for Road Curb Detection in Autonomous Driving with Aerial Images

Road curb detection is important for autonomous driving. It can be used to determine road boundaries to constrain vehicles on roads, so that potential accidents could be avoided. Most of the current methods detect road curbs online using vehicle-mounted sensors, such as cameras or 3-D Lidars. However, these methods usually suffer from severe occlusion issues. Especially in highly-dynamic traffic environments, most of the field of view is occupied by dynamic objects. To alleviate this issue, we detect road curbs offline using high-resolution aerial images in this paper. Moreover, the detected road curbs can be used to create high-definition (HD) maps for autonomous vehicles. Specifically, we first predict the pixel-wise segmentation map of road curbs, and then conduct a series of post-processing steps to extract the graph structure of road curbs. To tackle the disconnectivity issue in the segmentation maps, we propose an innovative connectivity-preserving loss (CP-loss) to improve the segmentation performance. The experimental results on a public dataset demonstrate the effectiveness of our proposed loss function. This paper is accompanied with a demonstration video and a supplementary document, which are available at \texttt{\url{https://sites.google.com/view/cp-loss}}.

cs.CV↗

arXiv2021

Nanotube-based one-dimensional heterostructures coupled by van der Waals forces

One-dimensional (1D) van der Waals heterostructures based on carbon nanotube templates are raising a lot of excitement due to the possibility of creating new optical and electronic properties, by either confining molecules inside their hollow core or by adding layers on the outside of the nanotube. In contrast to their 2D analogues, where the number of layers, atomic type and relative orientation of the constituting layers are the main parameters defining physical properties, 1D heterostructures provide an additional degree of freedom, i.e. their specific diameter and chiral structure, for engineering their characteristics. This review discusses the current state-of-the-art in synthesizing 1D heterostructures, in particular focusing on their resulting optical properties, and details the vast parameter space that can be used to design heterostructures with custom-built properties that can be integrated into a large variety of applications. The review starts from describing the effects of van der Waals coupling on the properties of the simplest and best-studied 1D heterostructure, namely a double-wall carbon nanotube, then considers heterostructures built from the inside and the outside, which all use a nanotube as a template, and, finally, provides an outlook for the future of this research field.

cond-mat.mtrl-sci↗

arXiv2021

Vision-Based Autonomous Car Racing Using Deep Imitative Reinforcement Learning

Autonomous car racing is a challenging task in the robotic control area. Traditional modular methods require accurate mapping, localization and planning, which makes them computationally inefficient and sensitive to environmental changes. Recently, deep-learning-based end-to-end systems have shown promising results for autonomous driving/racing. However, they are commonly implemented by supervised imitation learning (IL), which suffers from the distribution mismatch problem, or by reinforcement learning (RL), which requires a huge amount of risky interaction data. In this work, we present a general deep imitative reinforcement learning approach (DIRL), which successfully achieves agile autonomous racing using visual inputs. The driving knowledge is acquired from both IL and model-based RL, where the agent can learn from human teachers as well as perform self-improvement by safely interacting with an offline world model. We validate our algorithm both in a high-fidelity driving simulation and on a real-world 1/20-scale RC-car with limited onboard computation. The evaluation results demonstrate that our method outperforms previous IL and RL methods in terms of sample efficiency and task performance. Demonstration videos are available at https://caipeide.github.io/autorace-dirl/

cs.RO↗

arXiv2021

Feature Mining: A Novel Training Strategy for Convolutional Neural Network

In this paper, we propose a novel training strategy for convolutional neural network(CNN) named Feature Mining, that aims to strengthen the network's learning of the local feature. Through experiments, we find that semantic contained in different parts of the feature is different, while the network will inevitably lose the local information during feedforward propagation. In order to enhance the learning of local feature, Feature Mining divides the complete feature into two complementary parts and reuse these divided feature to make the network learn more local information, we call the two steps as feature segmentation and feature reusing. Feature Mining is a parameter-free method and has plug-and-play nature, and can be applied to any CNN models. Extensive experiments demonstrate the wide applicability, versatility, and compatibility of our method.

cs.CV↗

arXiv2021

Co-Teaching: An Ark to Unsupervised Stereo Matching

Stereo matching is a key component of autonomous driving perception. Recent unsupervised stereo matching approaches have received adequate attention due to their advantage of not requiring disparity ground truth. These approaches, however, perform poorly near occlusions. To overcome this drawback, in this paper, we propose CoT-Stereo, a novel unsupervised stereo matching approach. Specifically, we adopt a co-teaching framework where two networks interactively teach each other about the occlusions in an unsupervised fashion, which greatly improves the robustness of unsupervised stereo matching. Extensive experiments on the KITTI Stereo benchmarks demonstrate the superior performance of CoT-Stereo over all other state-of-the-art unsupervised stereo matching approaches in terms of both accuracy and speed. Our project webpage is https://sites.google.com/view/cot-stereo.

cs.CV↗

arXiv2021

SCV-Stereo: Learning Stereo Matching from a Sparse Cost Volume

Convolutional neural network (CNN)-based stereo matching approaches generally require a dense cost volume (DCV) for disparity estimation. However, generating such cost volumes is computationally-intensive and memory-consuming, hindering CNN training and inference efficiency. To address this problem, we propose SCV-Stereo, a novel CNN architecture, capable of learning dense stereo matching from sparse cost volume (SCV) representations. Our inspiration is derived from the fact that DCV representations are somewhat redundant and can be replaced with SCV representations. Benefiting from these SCV representations, our SCV-Stereo can update disparity estimations in an iterative fashion for accurate and efficient stereo matching. Extensive experiments carried out on the KITTI Stereo benchmarks demonstrate that our SCV-Stereo can significantly minimize the trade-off between accuracy and efficiency for stereo matching. Our project page is https://sites.google.com/view/scv-stereo.

cs.CV↗

arXiv2021

Topo-boundary: A Benchmark Dataset on Topological Road-boundary Detection Using Aerial Images for Autonomous Driving

Road-boundary detection is important for autonomous driving. It can be used to constrain autonomous vehicles running on road areas to ensure driving safety. Compared with online road-boundary detection using on-vehicle cameras/Lidars, offline detection using aerial images could alleviate the severe occlusion issue. Moreover, the offline detection results can be directly employed to annotate high-definition (HD) maps. In recent years, deep-learning technologies have been used in offline detection. But there still lacks a publicly available dataset for this task, which hinders the research progress in this area. So in this paper, we propose a new benchmark dataset, named \textit{Topo-boundary}, for offline topological road-boundary detection. The dataset contains 25,295 $1000\times1000$-sized 4-channel aerial images. Each image is provided with 8 training labels for different sub-tasks. We also design a new entropy-based metric for connectivity evaluation, which could better handle noises or outliers. We implement and evaluate 3 segmentation-based baselines and 5 graph-based baselines using the dataset. We also propose a new imitation-learning-based baseline which is enhanced from our previous work. The superiority of our enhancement is demonstrated from the comparison. The dataset and our-implemented code for the baselines are available at \texttt{\url{https://tonyxuqaq.github.io/Topo-boundary/}}.

cs.CV↗

arXiv2021

A Fuzzy Post-project Evaluation Approach for Security Video Surveillance System

Video surveillance is an essential component of the public security system. The security video surveillance system is a powerful means to prevent violence and crimes, and it is closely coupled with the construction of smart cities. A post-project evaluation is an evaluation of a project's actions and outcomes after its completion. Post-project evaluation can scientifically and objectively evaluate the construction effectiveness of video surveillance system at a certain stage. Utilizing post-project evaluation can find out the causes of success or failure to make recommendations for the construction of a security video surveillance system in the next stage. Therefore, we propose a fuzzy post-project evaluation approach for the security video surveillance system in a real-world community. The fuzzy theory and fuzzy multi-level evaluation method are applied. The evaluation result demonstrates that the proposed approach is practically applicable to real-world security video surveillance systems.

cs.OH↗

arXiv2021

Go Small and Similar: A Simple Output Decay Brings Better Performance

Regularization and data augmentation methods have been widely used and become increasingly indispensable in deep learning training. Researchers who devote themselves to this have considered various possibilities. But so far, there has been little discussion about regularizing outputs of the model. This paper begins with empirical observations that better performances are significantly associated with output distributions, that have smaller average values and variances. By audaciously assuming there is causality involved, we propose a novel regularization term, called Output Decay, that enforces the model to assign smaller and similar output values on each class. Though being counter-intuitive, such a small modification result in a remarkable improvement on performance. Extensive experiments demonstrate the wide applicability, versatility, and compatibility of Output Decay.

cs.CV↗

arXiv2021

Self-supervised Feature Enhancement: Applying Internal Pretext Task to Supervised Learning

Traditional self-supervised learning requires CNNs using external pretext tasks (i.e., image- or video-based tasks) to encode high-level semantic visual representations. In this paper, we show that feature transformations within CNNs can also be regarded as supervisory signals to construct the self-supervised task, called \emph{internal pretext task}. And such a task can be applied for the enhancement of supervised learning. Specifically, we first transform the internal feature maps by discarding different channels, and then define an additional internal pretext task to identify the discarded channels. CNNs are trained to predict the joint labels generated by the combination of self-supervised labels and original labels. By doing so, we let CNNs know which channels are missing while classifying in the hope to mine richer feature information. Extensive experiments show that our approach is effective on various models and datasets. And it's worth noting that we only incur negligible computational overhead. Furthermore, our approach can also be compatible with other methods to get better results.

cs.CV↗

arXiv2021

Self-supervision of Feature Transformation for Further Improving Supervised Learning

Self-supervised learning, which benefits from automatically constructing labels through pre-designed pretext task, has recently been applied for strengthen supervised learning. Since previous self-supervised pretext tasks are based on input, they may incur huge additional training overhead. In this paper we find that features in CNNs can be also used for self-supervision. Thus we creatively design the \emph{feature-based pretext task} which requires only a small amount of additional training overhead. In our task we discard different particular regions of features, and then train the model to distinguish these different features. In order to fully apply our feature-based pretext task in supervised learning, we also propose a novel learning framework containing multi-classifiers for further improvement. Original labels will be expanded to joint labels via self-supervision of feature transformations. With more semantic information provided by our self-supervised tasks, this approach can train CNNs more effectively. Extensive experiments on various supervised learning tasks demonstrate the accuracy improvement and wide applicability of our method.

cs.CV↗

arXiv2021

A universal deep learning strategy for designing high-quality-factor photonic resonances

Resonance is instrumental in modern optics and photonics for novel phenomena such as cavity quantum electrodynamics and electric-field-induced transparency. While one can use numerical simulations to sweep geometric and material parameters of optical structures, these simulations usually require considerably long calculation time (spanning from several hours to several weeks) and substantial computational resources. Such requirements significantly limit their applicability in understanding and inverse designing structures with desired resonance performances. Recently, the introduction of artificial intelligence allows for faster predictions of resonance with less demanding computational requirements. However, current end-to-end deep learning approaches generally fail to predict resonances with high quality-factors (Q-factor). Here, we introduce a universal deep learning strategy that can predict ultra-high Q-factor resonances by decomposing spectra with an adaptive data acquisition (ADA) method while incorporating resonance information. We exploit bound states in the continuum (BICs) with an infinite Q-factor to testify this resonance-informed deep learning (RIDL) strategy. The trained RIDL strategy achieves high-accuracy prediction of reflection spectra and photonic band structures while using a considerably small training dataset. We further develop an inverse design algorithm based on the RIDL strategy for a symmetry-protected BIC on a suspended silicon nitride photonic crystal (PhC) slab. The predicted and measured angle-resolved band structures show minimum differences. We expect the RIDL strategy to apply to many other physical phenomena which exhibit Gaussian, Lorentzian, and Fano resonances.

physics.optics↗

arXiv2021

Robust Odometry and Mapping for Multi-LiDAR Systems with Online Extrinsic Calibration

Combining multiple LiDARs enables a robot to maximize its perceptual awareness of environments and obtain sufficient measurements, which is promising for simultaneous localization and mapping (SLAM). This paper proposes a system to achieve robust and simultaneous extrinsic calibration, odometry, and mapping for multiple LiDARs. Our approach starts with measurement preprocessing to extract edge and planar features from raw measurements. After a motion and extrinsic initialization procedure, a sliding window-based multi-LiDAR odometry runs onboard to estimate poses with online calibration refinement and convergence identification. We further develop a mapping algorithm to construct a global map and optimize poses with sufficient features together with a method to model and reduce data uncertainty. We validate our approach's performance with extensive experiments on ten sequences (4.60km total length) for the calibration and SLAM and compare them against the state-of-the-art. We demonstrate that the proposed work is a complete, robust, and extensible system for various multi-LiDAR setups. The source code, datasets, and demonstrations are available at https://ram-lab.com/file/site/m-loam.

cs.RO↗