arXiv subjects

Fumin Zhang

Publications and source records attributed to Fumin Zhang.

At least 37 records · Page 2Linked to original sources

arXiv2025

Design of a Formation Control System to Assist Human Operators in Flying a Swarm of Robotic Blimps

Formation control is essential for swarm robotics, enabling coordinated behavior in complex environments. In this paper, we introduce a novel formation control system for an indoor blimp swarm using a specialized leader-follower approach enhanced with a dynamic leader-switching mechanism. This strategy allows any blimp to take on the leader role, distributing maneuvering demands across the swarm and enhancing overall formation stability. Only the leader blimp is manually controlled by a human operator, while follower blimps use onboard monocular cameras and a laser altimeter for relative position and altitude estimation. A leader-switching scheme is proposed to assist the human operator to maintain stability of the swarm, especially when a sharp turn is performed. Experimental results confirm that the leader-switching mechanism effectively maintains stable formations and adapts to dynamic indoor environments while assisting human operator.

cs.RO↗

arXiv2025

NuRF: Nudging the Particle Filter in Radiance Fields for Robot Visual Localization

Can we localize a robot on a map only using monocular vision? This study presents NuRF, an adaptive and nudged particle filter framework in radiance fields for 6-DoF robot visual localization. NuRF leverages recent advancements in radiance fields and visual place recognition. Conventional visual place recognition meets the challenges of data sparsity and artifact-induced inaccuracies. By utilizing radiance field-generated novel views, NuRF enhances visual localization performance and combines coarse global localization with the fine-grained pose tracking of a particle filter, ensuring continuous and precise localization. Experimentally, our method converges 7 times faster than existing Monte Carlo-based methods and achieves localization accuracy within 1 meter, offering an efficient and resilient solution for indoor visual localization.

cs.RO↗

arXiv2025

Speak the Same Language: Global LiDAR Registration on BIM Using Pose Hough Transform

Light detection and ranging (LiDAR) point clouds and building information modeling (BIM) represent two distinct data modalities in the fields of robot perception and construction. These modalities originate from different sources and are associated with unique reference frames. The primary goal of this study is to align these modalities within a shared reference frame using a global registration approach, effectively enabling them to ``speak the same language''. To achieve this, we propose a cross-modality registration method, spanning from the front end to the back end. At the front end, we extract triangle descriptors by identifying walls and intersected corners, enabling the matching of corner triplets with a complexity independent of the BIM's size. For the back-end transformation estimation, we utilize the Hough transform to map the matched triplets to the transformation space and introduce a hierarchical voting mechanism to hypothesize multiple pose candidates. The final transformation is then verified using our designed occupancy-aware scoring method. To assess the effectiveness of our approach, we conducted real-world multi-session experiments in a large-scale university building, employing two different types of LiDAR sensors. We make the collected datasets and codes publicly available to benefit the community.

cs.RO↗

arXiv2025

SLABIM: A SLAM-BIM Coupled Dataset in HKUST Main Building

Existing indoor SLAM datasets primarily focus on robot sensing, often lacking building architectures. To address this gap, we design and construct the first dataset to couple the SLAM and BIM, named SLABIM. This dataset provides BIM and SLAM-oriented sensor data, both modeling a university building at HKUST. The as-designed BIM is decomposed and converted for ease of use. We employ a multi-sensor suite for multi-session data collection and mapping to obtain the as-built model. All the related data are timestamped and organized, enabling users to deploy and test effectively. Furthermore, we deploy advanced methods and report the experimental results on three tasks: registration, localization and semantic mapping, demonstrating the effectiveness and practicality of SLABIM. We make our dataset open-source at https://github.com/HKUST-Aerial-Robotics/SLABIM.

cs.RO↗

arXiv2024

BEINGS: Bayesian Embodied Image-goal Navigation with Gaussian Splatting

Image-goal navigation enables a robot to reach the location where a target image was captured, using visual cues for guidance. However, current methods either rely heavily on data and computationally expensive learning-based approaches or lack efficiency in complex environments due to insufficient exploration strategies. To address these limitations, we propose Bayesian Embodied Image-goal Navigation Using Gaussian Splatting, a novel method that formulates ImageNav as an optimal control problem within a model predictive control framework. BEINGS leverages 3D Gaussian Splatting as a scene prior to predict future observations, enabling efficient, real-time navigation decisions grounded in the robot's sensory experiences. By integrating Bayesian updates, our method dynamically refines the robot's strategy without requiring extensive prior experience or data. Our algorithm is validated through extensive simulations and physical experiments, showcasing its potential for embodied robot systems in visually complex scenarios.

cs.RO↗

arXiv2024

A Hybrid Controller Design for Human-Assistive Piloting of an Underactuated Blimp

This paper introduces a novel solution to the manual control challenge for indoor blimps. The problem's complexity arises from the conflicting demands of executing human commands while maintaining stability through automatic control for underactuated robots. To tackle this challenge, we introduced an assisted piloting hybrid controller with a preemptive mechanism, that seamlessly switches between executing human commands and activating automatic stabilization control. Our algorithm ensures that the automatic stabilization controller operates within the time delay between human observation and perception, providing assistance to the driver in a way that remains imperceptible.

cs.RO↗

arXiv2024

OceanPlan: Hierarchical Planning and Replanning for Natural Language AUV Piloting in Large-scale Unexplored Ocean Environments

We develop a hierarchical LLM-task-motion planning and replanning framework to efficiently ground an abstracted human command into tangible Autonomous Underwater Vehicle (AUV) control through enhanced representations of the world. We also incorporate a holistic replanner to provide real-world feedback with all planners for robust AUV operation. While there has been extensive research in bridging the gap between LLMs and robotic missions, they are unable to guarantee success of AUV applications in the vast and unknown ocean environment. To tackle specific challenges in marine robotics, we design a hierarchical planner to compose executable motion plans, which achieves planning efficiency and solution quality by decomposing long-horizon missions into sub-tasks. At the same time, real-time data stream is obtained by a replanner to address environmental uncertainties during plan execution. Experiments validate that our proposed framework delivers successful AUV performance of long-duration missions through natural language piloting.

cs.RO↗

arXiv2023

Real-time Autonomous Glider Navigation Software

Underwater gliders are widely utilized for ocean sampling, surveillance, and other various oceanic applications. In the context of complex ocean environments, gliders may yield poor navigation performance due to strong ocean currents, thus requiring substantial human effort during the manual piloting process. To enhance navigation accuracy, we developed a real-time autonomous glider navigation software, named GENIoS Python, which generates waypoints based on flow predictions to assist human piloting. The software is designed to closely check glider status, provide customizable experiment settings, utilize lightweight computing resources, offer stably communicate with dockservers, robustly run for extended operation time, and quantitatively compare flow estimates, which add to its value as an autonomous tool for underwater glider navigation.

cs.RO↗

arXiv2023

General Anomaly Detection of Underwater Gliders Validated by Large-scale Deployment Datasets

Underwater gliders have been widely used in oceanography for a range of applications. However, unpredictable events like shark strikes or remora attachments can lead to abnormal glider behavior or even loss of the instrument. This paper employs an anomaly detection algorithm to assess operational conditions of underwater gliders in the real-world ocean environment. Prompt alerts are provided to glider pilots upon detecting any anomaly, so that they can take control of the glider to prevent further harm. The detection algorithm is applied to multiple datasets collected in real glider deployments led by the University of Georgia's Skidaway Institute of Oceanography (SkIO) and the University of South Florida (USF). In order to demonstrate the algorithm generality, the experimental evaluation is applied to four glider deployment datasets, each highlighting various anomalies happening in different scenes. Specifically, we utilize high resolution datasets only available post-recovery to perform detailed analysis of the anomaly and compare it with pilot logs. Additionally, we simulate the online detection based on the real-time subsets of data transmitted from the glider at the surfacing events. While the real-time data may not contain as much rich information as the post-recovery one, the online detection is of great importance as it allows glider pilots to monitor potential abnormal conditions in real time.

cs.RO↗

arXiv2023

OceanChat: Piloting Autonomous Underwater Vehicles in Natural Language

In the trending research of fusing Large Language Models (LLMs) and robotics, we aim to pave the way for innovative development of AI systems that can enable Autonomous Underwater Vehicles (AUVs) to seamlessly interact with humans in an intuitive manner. We propose OceanChat, a system that leverages a closed-loop LLM-guided task and motion planning framework to tackle AUV missions in the wild. LLMs translate an abstract human command into a high-level goal, while a task planner further grounds the goal into a task sequence with logical constraints. To assist the AUV with understanding the task sequence, we utilize a motion planner to incorporate real-time Lagrangian data streams received by the AUV, thus mapping the task sequence into an executable motion plan. Considering the highly dynamic and partially known nature of the underwater environment, an event-triggered replanning scheme is developed to enhance the system's robustness towards uncertainty. We also build a simulation platform HoloEco that generates photo-realistic simulation for a wide range of AUV applications. Experimental evaluation verifies that the proposed system can achieve improved performance in terms of both success rate and computation time. Project website: \url{https://sites.google.com/view/oceanchat}

cs.RO↗

arXiv2022

Anomaly Detection of Underwater Gliders Verified by Deployment Data

This paper utilizes an anomaly detection algorithm to check if underwater gliders are operating normally in the unknown ocean environment. Glider pilots can be warned of the detected glider anomaly in real time, thus taking over the glider appropriately and avoiding further damage to the glider. The adopted algorithm is validated by two valuable sets of data in real glider deployments, the University of South Florida (USF) glider Stella and the Skidaway Institute of Oceanography (SkIO) glider Angus.

cs.RO↗

arXiv2022

Lyapunov-Net: A Deep Neural Network Architecture for Lyapunov Function Approximation

We develop a versatile deep neural network architecture, called Lyapunov-Net, to approximate Lyapunov functions of dynamical systems in high dimensions. Lyapunov-Net guarantees positive definiteness, and thus it can be easily trained to satisfy the negative orbital derivative condition, which only renders a single term in the empirical risk function in practice. This significantly reduces the number of hyper-parameters compared to existing methods. We also provide theoretical justifications on the approximation power of Lyapunov-Net and its complexity bounds. We demonstrate the efficiency of the proposed method on nonlinear dynamical systems involving up to 30-dimensional state spaces, and show that the proposed approach significantly outperforms the state-of-the-art methods.

cs.LG↗

arXiv2022

The Rational Selection of Goal Operations and the Integration ofSearch Strategies with Goal-Driven Autonomy

Intelligent physical systems as embodied cognitive systems must perform high-level reasoning while concurrently managing an underlying control architecture. The link between cognition and control must manage the problem of converting continuous values from the real world to symbolic representations (and back). To generate effective behaviors, reasoning must include a capacity to replan, acquire and update new information, detect and respond to anomalies, and perform various operations on system goals. But, these processes are not independent and need further exploration. This paper examines an agent's choices when multiple goal operations co-occur and interact, and it establishes a method of choosing between them. We demonstrate the benefits and discuss the trade offs involved with this and show positive results in a dynamic marine search task.

cs.AI↗

arXiv2021

Method of evolving junction on optimal path planning in flow fields

We propose an algorithm using method of evolving junctions to solve the optimal path planning problems with piece-wise constant flow fields. In such flow fields with a convex Lagrangian in the objective function, we can prove that the optimal trajectories must be formed by constant velocity motions. Taking advantage of this property, we transform the infinite dimensional optimal control problem into a finite dimensional optimization and use intermittent diffusion to solve the problems. The algorithm is proven to be complete. At last, we demonstrate the performance of the algorithm with various simulation examples.

math.OC↗

arXiv2021

From Instantaneous Schedulability to Worst Case Schedulability: A Significant Moment Approach

The method of significant moment analysis has been employed to derive instantaneous schedulability tests for real-time systems. However, the instantaneous schedulability can only be checked within a finite time window. On the other hand, worst-case schedulability guarantees schedulability of systems for infinite time. This paper derives the classical worst-case schedulability conditions for preemptive periodic systems starting from instantaneous schedulability, hence unifying the two notions of schedulability. The results provide a rigorous justification on the critical time instants being the worst case for scheduling of preemptive periodic systems. The paper also show that the critical time instant is not the only worst case moments.

eess.SY↗

arXiv2021

Dissensus Algorithms for Opinion Dynamics on the Sphere

In this paper, novel dissensus algorithms based on the Oja principal component analysis (PCA) flow are proposed to model opinion dynamics on the unit sphere. The information of the covariance formed by the opinion state of each agent is used to achieve a dissensus equilibrium on unsigned graphs. This differs from most of the existing work where antagonistic interactions represented by negative weights in signed graphs are used to achieve a dissensus equilibrium. The nonlinear algorithm is analyzed under both constant covariance and time-varying covariance leading to different behaviors. Stability analysis for the unstable consensus and stable dissensus equilibria is provided under various conditions. The performance of the algorithm is illustrated through a simulation experiment of a multi-agent system.

eess.SY↗

arXiv2021

A Distributed Active Perception Strategy for Source Seeking and Level Curve Tracking

Algorithms for multi-agent systems to locate a source or to follow a desired level curve of spatially distributed scalar fields generally require sharing field measurements among the agents for gradient estimation. Yet, in this paper, we propose a distributed active perception strategy that enables swarms of various sizes and graph structures to perform source seeking and level curve tracking without the need to explicitly estimate the field gradient or explicitly share measurements. The proposed method utilizes a consensus-like Principal Component Analysis perception algorithm that does not require explicit communication in order to compute a local body frame. This body frame is used to design a distributed control law where each agent modulates its motion based only on its instantaneous field measurement. Several stability results are obtained within a singular perturbation framework which justifies the convergence and robustness of the strategy. Additionally, efficiency is validated through various computer simulations and robots implementation in $2$-D scalar fields. The active perception strategy leverages the available local information and has the potential to be used in various applications such as modeling information propagation in biological and robotic swarms.

eess.SY↗

arXiv2020

Long distance measurement using single soliton microcomb

Dispersive interferometry (DPI) takes a major interest in optical frequency comb (OFC) based long distance laser-based light detection and ranging (LIDAR) for the merits of strong anti-interference ability and long coherent length. However, the mismatch between the repetition rate of OFC and the resolution of optical spectrum acquisition system induces a large dead-zone which is a major obstacle for practical applications. Here, a new DPI LIDAR on the strength of high-repetition-rate soliton microcomb is demonstrated, which reaches a minimum Allan deviation of 27 nm for an outdoor 1179 m ranging experiment. The proposed scheme approaches a compact, high-accuracy, and none-dead-zone long distance ranging system, opening up new opportunities for emerging applications of frontier scientific researches and advanced manufacturing.

physics.ins-det↗