Searcharxiv⌕ Search

arXiv subjects

Fu Zhang

Publications and source records attributed to Fu Zhang.

At least 91 records · Page 5Linked to original sources

Kalman Filters on Differentiable Manifolds

Kalman filter is presumably one of the most important and extensively used filtering techniques in modern control systems. Yet, nearly all current variants of Kalman filters are formulated in the Euclidean space $\mathbb{R}^n$, while many real-world systems (e.g., robotic systems) are really evolving on manifolds. In this paper, we propose a method to develop Kalman filters for such on-manifold systems. Utilizing $\boxplus$, $\boxminus$ operations and further defining an oplus operation on the respective manifold, we propose a canonical representation of the on-manifold system. Such a canonical form enables us to separate the manifold constraints from the system behaviors in each step of the Kalman filter, ultimately leading to a generic and symbolic Kalman filter framework that are naturally evolving on the manifold. Furthermore, the on-manifold Kalman filter is implemented as a toolkit in $C$++ packages which enables users to implement an on-manifold Kalman filter just like the normal one in $\mathbb{R}^n$: the user needs only to provide the system-specific descriptions, and then call the respective filter steps (e.g., predict, update) without dealing with any of the manifold constraints. The existing implementation supports full iterated Kalman filtering for systems on any manifold composed of $\mathbb{R}^n$, $SO(3)$ and $\mathbb{S}^2$, and is extendable to other types of manifold when necessary. The proposed symbolic Kalman filter and the developed toolkit are verified by implementing a tightly-coupled lidar-inertial navigation system. Results show that the developed toolkit leads to superior filtering performances and computation efficiency comparable to hand-engineered counterparts. Finally, the toolkit is opened sourced at https://github.com/hku-mars/IKFoM to assist practitioners to quickly deploy an on-manifold Kalman filter.

cs.RO↗

Pixel-level Extrinsic Self Calibration of High Resolution LiDAR and Camera in Targetless Environments

In this letter, we present a novel method for automatic extrinsic calibration of high-resolution LiDARs and RGB cameras in targetless environments. Our approach does not require checkerboards but can achieve pixel-level accuracy by aligning natural edge features in the two sensors. On the theory level, we analyze the constraints imposed by edge features and the sensitivity of calibration accuracy with respect to edge distribution in the scene. On the implementation level, we carefully investigate the physical measuring principles of LiDARs and propose an efficient and accurate LiDAR edge extraction method based on point cloud voxel cutting and plane fitting. Due to the edges' richness in natural scenes, we have carried out experiments in many indoor and outdoor scenes. The results show that this method has high robustness, accuracy, and consistency. It can promote the research and application of the fusion between LiDAR and camera. We have open-sourced our code on GitHub to benefit the community.

cs.RO↗

FAST-LIO: A Fast, Robust LiDAR-inertial Odometry Package by Tightly-Coupled Iterated Kalman Filter

This paper presents a computationally efficient and robust LiDAR-inertial odometry framework. We fuse LiDAR feature points with IMU data using a tightly-coupled iterated extended Kalman filter to allow robust navigation in fast-motion, noisy or cluttered environments where degeneration occurs. To lower the computation load in the presence of large number of measurements, we present a new formula to compute the Kalman gain. The new formula has computation load depending on the state dimension instead of the measurement dimension. The proposed method and its implementation are tested in various indoor and outdoor environments. In all tests, our method produces reliable navigation results in real-time: running on a quadrotor onboard computer, it fuses more than 1,200 effective feature points in a scan and completes all iterations of an iEKF step within 25 ms. Our codes are open-sourced on Github.

cs.RO↗

Control of a Tail-Sitter VTOL UAV Based on Recurrent Neural Networks

Tail-sitter vertical takeoff and landing (VTOL) unmanned aerial vehicles (UAVs) have the capability of hovering and performing efficient level flight with compact mechanical structures. We present a unified controller design for such UAVs, based on recurrent neural networks. An advantage of this design method is that the various flight modes (i.e., hovering, transition and level flight) of a VTOL UAV are controlled in a unified manner, as opposed to treating them separately and in the runtime switching one from another. The proposed controller consists of an outer-loop position controller and an inner-loop attitude controller. The inner-loop controller is composed of a proportional attitude controller and a loop-shaping linear angular rate controller. For the outer-loop controller, we propose a nonlinear solver to compute the desired attitude and thrust, based on the UAV dynamics and an aerodynamic model, in addition to a cascaded PID controller for the position and velocity tracking. We employ a recurrent neural network (RNN) to approximate the behavior of the nonlinear solver, which suffers from high computational complexity. The proposed RNN has negligible approximation errors, and can be implemented in real-time (e.g., 50 Hz). Moreover, the RNN generates much smoother outputs than the nonlinear solver. We provide an analysis of the stability and robustness of the overall closed-loop system. Simulation and experiments are also presented to demonstrate the effectiveness of the proposed method.

cs.RO↗

Avoiding dynamic small obstacles with onboard sensing and computating on aerial robots

In practical applications, autonomous quadrotors are still facing significant challenges, such as the detection and avoidance of very small and even dynamic obstacles (e.g., tree branches, power lines). In this paper, we propose a compact, integrated, and fully autonomous quadrotor system, which can fly safely in cluttered environments while avoiding dynamic small obstacles. Our quadrotor platform is equipped with a forward-looking three-dimensional (3D) light detection and ranging (lidar) sensor to perceive the environment and an onboard embedded computer to perform all the estimation, mapping, and planning tasks. Specifically, the computer estimates the current pose of the UAV, maintains a local map (time-accumulated point clouds KD-Trees), and computes a safe trajectory using kinodynamic A* search to the goal point. The whole perception and planning system can run onboard at 50Hz with careful optimization. Various indoor and outdoor experiments show that the system can avoid dynamic small obstacles (down to 20mm diameter bar) while flying at 2m/s in cluttered environments. Our codes and hardware design are open-sourced on Github.

cs.RO↗

R2LIVE: A Robust, Real-time, LiDAR-Inertial-Visual tightly-coupled state Estimator and mapping

In this letter, we propose a robust, real-time tightly-coupled multi-sensor fusion framework, which fuses measurement from LiDAR, inertial sensor, and visual camera to achieve robust and accurate state estimation. Our proposed framework is composed of two parts: the filter-based odometry and factor graph optimization. To guarantee real-time performance, we estimate the state within the framework of error-state iterated Kalman-filter, and further improve the overall precision with our factor graph optimization. Taking advantage of measurement from all individual sensors, our algorithm is robust enough to various visual failure, LiDAR-degenerated scenarios, and is able to run in real-time on an on-board computation platform, as shown by extensive experiments conducted in indoor, outdoor, and mixed environment of different scale. Moreover, the results show that our proposed framework can improve the accuracy of state-of-the-art LiDAR-inertial or visual-inertial odometry. To share our findings and to make contributions to the community, we open source our codes on our Github.

cs.RO↗

ikd-Tree: An Incremental K-D Tree for Robotic Applications

This paper proposes an efficient data structure, ikd-Tree, for dynamic space partition. The ikd-Tree incrementally updates a k-d tree with new coming points only, leading to much lower computation time than existing static k-d trees. Besides point-wise operations, the ikd-Tree supports several features such as box-wise operations and down-sampling that are practically useful in robotic applications. In parallel to the incremental operations (i.e., insert, re-insert, and delete), ikd-Tree actively monitors the tree structure and partially re-balances the tree, which enables efficient nearest point search in later stages. The ikd-Tree is carefully engineered and supports multi-thread parallel computing to maximize the overall efficiency. We validate the ikd-Tree in both theory and practical experiments. On theory level, a complete time complexity analysis is presented to prove the high efficiency. On experiment level, the ikd-Tree is tested on both randomized datasets and real-world LiDAR point data in LiDAR-inertial odometry and mapping application. In all tests, ikd-Tree consumes only 4% of the running time in a static k-d tree.

cs.RO↗

BALM: Bundle Adjustment for Lidar Mapping

A local Bundle Adjustment (BA) on a sliding window of keyframes has been widely used in visual SLAM and proved to be very effective in lowering the drift. But in lidar SLAM, BA method is hardly used because the sparse feature points (e.g., edge and plane) make the exact point matching impossible. In this paper, we formulate the lidar BA as minimizing the distance from a feature point to its matched edge or plane. Unlike the visual SLAM (and prior plane adjustment method in lidar SLAM) where the feature has to be co-determined along with the pose, we show that the feature can be analytically solved and removed from the BA, the resultant BA is only dependent on the scan poses. This greatly reduces the optimization scale and allows large-scale dense plane and edge features to be used. To speedup the optimization, we derive the analytical derivatives of the cost function, up to second order, in closed form. Moreover, we propose a novel adaptive voxelization method to search feature correspondence efficiently. The proposed formulations are incorporated into a LOAM back-end for map refinement. Results show that, although as a back-end, the local BA can be solved very efficiently, even in real-time at 10Hz when optimizing 20 scans of point-cloud. The local BA also considerably lowers the LOAM drift. Our implementation of the BA optimization and LOAM are open-sourced to benefit the community.

cs.RO↗

A Q-learning algorithm for discrete-time linear-quadratic control with random parameters of unknown distribution: convergence and stabilization

This paper studies an infinite horizon optimal control problem for discrete-time linear systems and quadratic criteria, both with random parameters which are independent and identically distributed with respect to time. A classical approach is to solve an algebraic Riccati equation that involves mathematical expectations and requires certain statistical information of the parameters. In this paper, we propose an online iterative algorithm in the spirit of Q-learning for the situation where only one random sample of parameters emerges at each time step. The first theorem proves the equivalence of three properties: the convergence of the learning sequence, the well-posedness of the control problem, and the solvability of the algebraic Riccati equation. The second theorem shows that the adaptive feedback control in terms of the learning sequence stabilizes the system as long as the control problem is well-posed. Numerical examples are presented to illustrate our results.

math.OC↗

Robots State Estimation and Observability Analysis Based on Statistical Motion Models

This paper presents a generic motion model to capture mobile robots' dynamic behaviors (translation and rotation). The model is based on statistical models driven by white random processes and is formulated into a full state estimation algorithm based on the error-state extended Kalman filtering framework (ESEKF). Major benefits of this method are its versatility, being applicable to different robotic systems without accurately modeling the robots' specific dynamics, and ability to estimate the robot's (angular) acceleration, jerk, or higher-order dynamic states with low delay. Mathematical analysis with numerical simulations are presented to show the properties of the statistical model-based estimation framework and to reveal its connection to existing low-pass filters. Furthermore, a new paradigm is developed for robots observability analysis by developing Lie derivatives and associated partial differentiation directly on manifolds. It is shown that this new paradigm is much simpler and more natural than existing methods based on quaternion parameterizations. It is also scalable to high dimensional systems. A novel \textbf{\textit{thin}} set concept is introduced to characterize the unobservable subset of the system states, providing the theoretical foundation to observability analysis of robotic systems operating on manifolds and in high dimension. Finally, extensive experiments including full state estimation and extrinsic calibration (both POS-IMU and IMU-IMU) on a quadrotor UAV, a handheld platform and a ground vehicle are conducted. Comparisons with existing methods show that the proposed method can effectively estimate all extrinsic parameters, the robot's translation/angular acceleration and other state variables (e.g., position, velocity, attitude) of high accuracy and low delay.

cs.RO↗

Learning Pugachev's Cobra Maneuver for Tail-sitter UAVs Using Acceleration Model

The Pugachev's cobra maneuver is a dramatic and demanding maneuver requiring the aircraft to fly at extremely high Angle of Attacks (AOA) where stalling occurs. This paper considers this maneuver on tail-sitter UAVs. We present a simple yet very effective feedback-iterative learning position control structure to regulate the altitude error and lateral displacement during the maneuver. Both the feedback controller and the iterative learning controller are based on the aircraft acceleration model, which is directly measurable by the onboard accelerometer. Moreover, the acceleration model leads to an extremely simple dynamic model that does not require any model identification in designing the position controller, greatly simplifying the implementation of the iterative learning control. Real-world outdoor flight experiments on the "Hong Hu" UAV, an aerobatic yet efficient quadrotor tail-sitter UAV of small-size, are provided to show the effectiveness of the proposed controller.

eess.SY↗

A decentralized framework for simultaneous calibration, localization and mapping with multiple LiDARs

LiDAR is playing a more and more essential role in autonomous driving vehicles for objection detection, self localization and mapping. A single LiDAR frequently suffers from hardware failure (e.g., temporary loss of connection) due to the harsh vehicle environment (e.g., temperature, vibration, etc.), or performance degradation due to the lack of sufficient geometry features, especially for solid-state LiDARs with small field of view (FoV). To improve the system robustness and performance in self-localization and mapping, we develop a decentralized framework for simultaneous calibration, localization and mapping with multiple LiDARs. Our proposed framework is based on an extended Kalman filter (EKF), but is specially formulated for decentralized implementation. Such an implementation could potentially distribute the intensive computation among smaller computing devices or resources dedicated for each LiDAR and remove the single point of failure problem. Then this decentralized formulation is implemented on an unmanned ground vehicle (UGV) carrying 5 low-cost LiDARs and moving at $1.3m/s$ in urban environments. Experiment results show that the proposed method can successfully and simultaneously estimate the vehicle state (i.e., pose and velocity) and all LiDAR extrinsic parameters. The localization accuracy is up to 0.2% on the two datasets we collected. To share our findings and to make contributions to the community, meanwhile enable the readers to verify our work, we will release all our source codes and hardware design blueprint on our Github.

cs.RO↗

Light-controlled room temperature ferromagnetism in vanadium-doped tungsten diselenide semiconducting monolayers

Atomically thin transition metal dichalcogenide (TMD) semiconductors hold enormous potential for modern optoelectronic devices and quantum computing applications. By inducing long-range ferromagnetism (FM) in these semiconductors through the introduction of small amounts of a magnetic dopant, it is possible to extend their potential in emerging spintronic applications. Here, we demonstrate light-mediated, room temperature (RT) FM, in V-doped WS2 (V-WS2) monolayers. We probe this effect using the principle of magnetic LC resonance, which employs a soft ferromagnetic Co-based microwire coil driven near its resonance in the radio frequency (RF) regime. The combination of LC resonance with an extraordinary giant magneto-impedance effect, renders the coil highly sensitive to changes in the magnetic flux through its core. We then place the V-WS2 monolayer at the core of the coil where it is excited with a laser while its change in magnetic permeability is measured. Notably, the magnetic permeability of the monolayer is found to depend on the laser intensity, thus confirming light control of RT magnetism in this two-dimensional (2D) material. Guided by density functional calculations, we attribute this phenomenon to the presence of excess holes in the conduction and valence bands, as well as carriers trapped in the magnetic doping states, which in turn mediates the magnetization of the V-WS2 monolayer. These findings provide a unique route to exploit light-controlled ferromagnetism in low powered 2D spintronic devices capable of operating at RT.

cond-mat.mtrl-sci↗

Consolidating Commonsense Knowledge

Commonsense reasoning is an important aspect of building robust AI systems and is receiving significant attention in the natural language understanding, computer vision, and knowledge graphs communities. At present, a number of valuable commonsense knowledge sources exist, with different foci, strengths, and weaknesses. In this paper, we list representative sources and their properties. Based on this survey, we propose principles and a representation model in order to consolidate them into a Common Sense Knowledge Graph (CSKG). We apply this approach to consolidate seven separate sources into a first integrated CSKG. We present statistics of CSKG, present initial investigations of its utility on four QA datasets, and list learned lessons.

cs.AI↗

Low-cost Retina-like Robotic Lidars Based on Incommensurable Scanning

High performance lidars are essential in autonomous robots such as self-driving cars, automated ground vehicles and intelligent machines. Traditional mechanical scanning lidars offer superior performance in autonomous vehicles, but the potential mass application is limited by the inherent manufacturing difficulty. We propose a robotic lidar sensor based on incommensurable scanning that allows straightforward mass production and adoption in autonomous robots. Some unique features are additionally permitted by this incommensurable scanning. Similar to the fovea in human retina, this lidar features a peaked central angular density, enabling in applications that prefers eye-like attention. The incommensurable scanning method of this lidar could also provide a much higher resolution than conventional lidars which is beneficial in robotic applications such as sensor calibration. Examples making use of these advantageous features are demonstrated.

cs.RO↗

Monolayer Vanadium-doped Tungsten Disulfide: A Room-Temperature Dilute Magnetic Semiconductor

Dilute magnetic semiconductors, achieved through substitutional doping of spin-polarized transition metals into semiconducting systems, enable experimental modulation of spin dynamics in ways that hold great promise for novel magneto-electric or magneto-optical devices, especially for two-dimensional systems such as transition metal dichalcogenides that accentuate interactions and activate valley degrees of freedom. Practical applications of 2D magnetism will likely require room-temperature operation, air stability, and (for magnetic semiconductors) the ability to achieve optimal doping levels without dopant aggregation. Here we describe room-temperature ferromagnetic order obtained in semiconducting vanadium-doped tungsten disulfide monolayers produced by a reliable single-step film sulfidation method across an exceptionally wide range of vanadium concentrations, up to 12 at% with minimal dopant aggregation. These monolayers develop p-type transport as a function of vanadium incorporation and rapidly reach ambipolarity. Ferromagnetism peaks at an intermediate vanadium concentration of a few atomic percent and decreases for higher concentrations, which is consistent with quenching due to orbital hybridization at closer vanadium-vanadium spacings, as supported by transmission electron microscopy, magnetometry and first-principles calculations. Room-temperature two-dimensional dilute magnetic semiconductors provide a new component to expand the functional scope of van der Waals heterostructures and bring semiconducting magnetic 2D heterostructures them into the realm of practical application.

cond-mat.mtrl-sci↗

Tunable Ferromagnetism and Thermally Induced Spin Flip in Vanadium-doped Tungsten Diselenide Monolayers at Room Temperature

The outstanding optoelectronic and valleytronic properties of transition metal dichalcogenides (TMDs) have triggered intense research efforts by the scientific community. An alternative to induce long-range ferromagnetism (FM) in TMDs is by introducing magnetic dopants to form a dilute magnetic semiconductor. Enhancing ferromagnetism in these semiconductors not only represents a key step towards modern TMD-based spintronics, but also enables exploration of new and exciting dimensionality-driven magnetic phenomena. To this end, we show tunable ferromagnetism at room temperature and a thermally induced spin flip (TISF) in monolayers of V-doped WSe2. As vanadium concentrations increase within the WSe2 monolayers the saturation magnetization increases, and it is optimal at ~4at.% vanadium; the highest doping/alloying level ever achieved for V-doped WSe2 monolayers. The TISF occurs at ~175 K and becomes more pronounced upon increasing the temperature towards room temperature. We demonstrate that TISF can be manipulated by changing the vanadium concentration within the WSe2 monolayers. We attribute TISF to the magnetic field and temperature dependent flipping of the nearest W-site magnetic moments that are antiferromagnetically coupled to the V magnetic moments in the ground state. This is fully supported by a recent spin-polarized density functional theory calculation. Our findings pave the way for the development of novel spintronic and valleytronic nanodevices based on atomically thin magnetic semiconductors and stimulate further studies in this rapidly expanding research field of 2D magnetism.

cond-mat.mtrl-sci↗

Hybrid aerial ground locomotion with a single passive wheel

Exploiting contacts with environment structures provides extra force support to a UAV, often reducing the power consumption and hence extending the mission time. This paper investigates one such way to exploit flat surfaces in the environment by a novel aerial-ground hybrid locomotion. Our design is a single passive wheel integrated at the UAV bottom, serving a minimal design to date. We present the principle and implementation of such a simple design as well as its control. Flight experiments are conducted to verify the feasibility and the power saving caused by the ground locomotion. Results show that our minimal design allows successful aerial-ground hybrid locomotion even with a less-controllable bi-copter UAV. The ground locomotion saves up to 77% battery without much tuning effort.

cs.RO↗