arXiv subjects

Danping Zou

Publications and source records attributed to Danping Zou.

46 records · Page 3Linked to original sources

arXiv2021

M2DGR: A Multi-sensor and Multi-scenario SLAM Dataset for Ground Robots

We introduce M2DGR: a novel large-scale dataset collected by a ground robot with a full sensor-suite including six fish-eye and one sky-pointing RGB cameras, an infrared camera, an event camera, a Visual-Inertial Sensor (VI-sensor), an inertial measurement unit (IMU), a LiDAR, a consumer-grade Global Navigation Satellite System (GNSS) receiver and a GNSS-IMU navigation system with real-time kinematic (RTK) signals. All those sensors were well-calibrated and synchronized, and their data were recorded simultaneously. The ground truth trajectories were obtained by the motion capture device, a laser 3D tracker, and an RTK receiver. The dataset comprises 36 sequences (about 1TB) captured in diverse scenarios including both indoor and outdoor environments. We evaluate state-of-the-art SLAM algorithms on M2DGR. Results show that existing solutions perform poorly in some scenarios. For the benefit of the research community, we make the dataset and tools public. The webpage of our project is https://github.com/SJTU-ViSYS/M2DGR.

cs.RO↗

arXiv2021

StructDepth: Leveraging the structural regularities for self-supervised indoor depth estimation

Self-supervised monocular depth estimation has achieved impressive performance on outdoor datasets. Its performance however degrades notably in indoor environments because of the lack of textures. Without rich textures, the photometric consistency is too weak to train a good depth network. Inspired by the early works on indoor modeling, we leverage the structural regularities exhibited in indoor scenes, to train a better depth network. Specifically, we adopt two extra supervisory signals for self-supervised training: 1) the Manhattan normal constraint and 2) the co-planar constraint. The Manhattan normal constraint enforces the major surfaces (the floor, ceiling, and walls) to be aligned with dominant directions. The co-planar constraint states that the 3D points be well fitted by a plane if they are located within the same planar region. To generate the supervisory signals, we adopt two components to classify the major surface normal into dominant directions and detect the planar regions on the fly during training. As the predicted depth becomes more accurate after more training epochs, the supervisory signals also improve and in turn feedback to obtain a better depth model. Through extensive experiments on indoor benchmark datasets, the results show that our network outperforms the state-of-the-art methods. The source code is available at https://github.com/SJTU-ViSYS/StructDepth .

cs.CV↗

arXiv2020

Attention-SLAM: A Visual Monocular SLAM Learning from Human Gaze

This paper proposes a novel simultaneous localization and mapping (SLAM) approach, namely Attention-SLAM, which simulates human navigation mode by combining a visual saliency model (SalNavNet) with traditional monocular visual SLAM. Most SLAM methods treat all the features extracted from the images as equal importance during the optimization process. However, the salient feature points in scenes have more significant influence during the human navigation process. Therefore, we first propose a visual saliency model called SalVavNet in which we introduce a correlation module and propose an adaptive Exponential Moving Average (EMA) module. These modules mitigate the center bias to enable the saliency maps generated by SalNavNet to pay more attention to the same salient object. Moreover, the saliency maps simulate the human behavior for the refinement of SLAM results. The feature points extracted from the salient regions have greater importance in optimization process. We add semantic saliency information to the Euroc dataset to generate an open-source saliency SLAM dataset. Comprehensive test results prove that Attention-SLAM outperforms benchmarks such as Direct Sparse Odometry (DSO), ORB-SLAM, and Salient DSO in terms of efficiency, accuracy, and robustness in most test cases.

cs.CV↗

arXiv2020

TextSLAM: Visual SLAM with Planar Text Features

We propose to integrate text objects in man-made scenes tightly into the visual SLAM pipeline. The key idea of our novel text-based visual SLAM is to treat each detected text as a planar feature which is rich of textures and semantic meanings. The text feature is compactly represented by three parameters and integrated into visual SLAM by adopting the illumination-invariant photometric error. We also describe important details involved in implementing a full pipeline of text-based visual SLAM. To our best knowledge, this is the first visual SLAM method tightly coupled with the text features. We tested our method in both indoor and outdoor environments. The results show that with text features, the visual SLAM system becomes more robust and produces much more accurate 3D text maps that could be useful for navigation and scene understanding in robotic or augmented reality applications.

cs.CV↗

arXiv2020

A Deep Learning Method for Complex Human Activity Recognition Using Virtual Wearable Sensors

Sensor-based human activity recognition (HAR) is now a research hotspot in multiple application areas. With the rise of smart wearable devices equipped with inertial measurement units (IMUs), researchers begin to utilize IMU data for HAR. By employing machine learning algorithms, early IMU-based research for HAR can achieve accurate classification results on traditional classical HAR datasets, containing only simple and repetitive daily activities. However, these datasets rarely display a rich diversity of information in real-scene. In this paper, we propose a novel method based on deep learning for complex HAR in the real-scene. Specially, in the off-line training stage, the AMASS dataset, containing abundant human poses and virtual IMU data, is innovatively adopted for enhancing the variety and diversity. Moreover, a deep convolutional neural network with an unsupervised penalty is proposed to automatically extract the features of AMASS and improve the robustness. In the on-line testing stage, by leveraging advantages of the transfer learning, we obtain the final result by fine-tuning the partial neural network (optimizing the parameters in the fully-connected layers) using the real IMU data. The experimental results show that the proposed method can surprisingly converge in a few iterations and achieve an accuracy of 91.15% on a real IMU dataset, demonstrating the efficiency and effectiveness of the proposed method.

cs.CV↗

arXiv2019

StructVIO : Visual-inertial Odometry with Structural Regularity of Man-made Environments

We propose a novel visual-inertial odometry approach that adopts structural regularity in man-made environments. Instead of using Manhattan world assumption, we use Atlanta world model to describe such regularity. An Atlanta world is a world that contains multiple local Manhattan worlds with different heading directions. Each local Manhattan world is detected on-the-fly, and their headings are gradually refined by the state estimator when new observations are coming. With fully exploration of structural lines that aligned with each local Manhattan worlds, our visual-inertial odometry method become more accurate and robust, as well as much more flexible to different kinds of complex man-made environments. Through extensive benchmark tests and real-world tests, the results show that the proposed approach outperforms existing visual-inertial systems in large-scale man-made environments

cs.RO↗

arXiv2019

Efficient Outlier Removal in Large Scale Global Structure-from-Motion

This work addresses the outlier removal problem in large-scale global structure-from-motion. In such applications, global outlier removal is very useful to mitigate the deterioration caused by mismatches in the feature point matching step. Unlike existing outlier removal methods, we exploit the structure in multiview geometry problems to propose a dimension reduced formulation, based on which two methods have been developed. The first method considers a convex relaxed $\ell_1$ minimization and is solved by a single linear programming (LP), whilst the second one approximately solves the ideal $\ell_0$ minimization by an iteratively reweighted method. The dimension reduction results in a significant speedup of the new algorithms. Further, the iteratively reweighted method can significantly reduce the possibility of removing true inliers. Realistic multiview reconstruction experiments demonstrated that, compared with state-of-the-art algorithms, the new algorithms are much more efficient and meanwhile can give improved solution. Matlab code for reproducing the results is available at \textit{https://github.com/FWen/OUTLR.git}.

cs.CV↗

arXiv2018

Reconstructing 3D Motion Trajectory of Large Swarm of Flying Objects

This paper addresses the problem of reconstructing the motion trajectories of the individuals in a large collection of flying objects using two temporally synchronized and geometrically calibrated cameras. The 3D trajectory reconstruction problem involves two challenging tasks - stereo matching and temporal tracking. Existing methods separate the two and process them one at a time sequentially, and suffer from frequent irresolvable ambiguities in stereo matching and in tracking. We unify the two tasks, and propose an optimization approach to solving stereo matching and temporal tracking simultaneously. It treats 3D trajectory acquisition problem as selecting appropriate stereo correspondence out of all possible ones for each object via minimizing a cost function. Experiment results show that the proposed method offers significant performance advantage over existing approaches. The proposed method has successfully been applied to reconstruct 3D motion trajectories of hundreds of simultaneously flying fruit flies (Drosophila Melanogaster), which could facilitate the study the insect's collective behavior.

q-bio.QM↗

arXiv2018

Active Image-based Modeling with a Toy Drone

Image-based modeling techniques can now generate photo-realistic 3D models from images. But it is up to users to provide high quality images with good coverage and view overlap, which makes the data capturing process tedious and time consuming. We seek to automate data capturing for image-based modeling. The core of our system is an iterative linear method to solve the multi-view stereo (MVS) problem quickly and plan the Next-Best-View (NBV) effectively. Our fast MVS algorithm enables online model reconstruction and quality assessment to determine the NBVs on the fly. We test our system with a toy unmanned aerial vehicle (UAV) in simulated, indoor and outdoor experiments. Results show that our system improves the efficiency of data acquisition and ensures the completeness of the final model.

cs.CV↗

arXiv2016

Dynamic Magnetometer Calibration and Alignment to Inertial Sensors by Kalman Filtering

Magnetometer and inertial sensors are widely used for orientation estimation. Magnetometer usage is often troublesome, as it is prone to be interfered by onboard or ambient magnetic disturbance. The onboard soft-iron material distorts not only the magnetic field, but the magnetometer sensor frame coordinate and the cross-sensor misalignment relative to inertial sensors. It is desirable to conveniently put magnetic and inertial sensors information in a common frame. Existing methods either split the problem into successive intrinsic and cross-sensor calibrations, or rely on stationary accelerometer measurements which is infeasible in dynamic conditions. This paper formulates the magnetometer calibration and alignment to inertial sensors as a state estimation problem, and collectively solves the magnetometer intrinsic and cross-sensor calibrations, as well as the gyroscope bias estimation. Sufficient conditions are derived for the problem to be globally observable, even when no accelerometer information is used at all. An extended Kalman filter is designed to implement the state estimation and comprehensive test data results show the superior performance of the proposed approach. It is immune to acceleration disturbance and applicable potentially in any dynamic conditions.

cs.RO↗