arXiv subjects

Yanbing Mao

Publications and source records attributed to Yanbing Mao.

At least 19 recordsLinked to original sources

arXiv2026

Cognitively-Grounded On-Device Runtime Learning for Ground Robots in Unknown Physical Environments

This paper presents \ul{CogRun}, a framework that enables safety-critical ground robots to perform cognitively-grounded runtime learning entirely on edge-AI devices in unknown physical environments, without prior maps or perceptual knowledge. CogRun consists of three components: a Learning-Agent, a Rational-Agent, and a Coordinator. The Learning-Agent is novel in cognitive-neural learning architecture, which featurs dedicated replay buffers, cognition-driven experience sampling, and a safety-aware action blending of actor-critic reinforcement learning (RL) with instance-based learning (IBL). The Rational-Agent is a non-learning module that complements the Learning-Agent by exclusively handling safety-critical functions, while the Coordinator manages interactions between the two agents to promote safe and efficient runtime learning. CogRun's full autonomy stack (i.e., perception, learning, and control) on edge-AI devices eliminates dependence on wireless communications, enabling broader applications in challenging environments with limited or no connectivity. Experiments on a quadruped robot in real-world wild forests and on an off-road autonomous vehicle in a simulated wild forest demonstrate that CogRun enables safe and efficient runtime learning, allowing robots to safely and continuously interact with the physical world for enhancing task performance in complex, unknown environments.

cs.RO↗

arXiv2025

Real-DRL: Teach and Learn in Reality

This paper introduces the Real-DRL framework for safety-critical autonomous systems, enabling runtime learning of a deep reinforcement learning (DRL) agent to develop safe and high-performance action policies in real plants (i.e., real physical systems to be controlled), while prioritizing safety! The Real-DRL consists of three interactive components: a DRL-Student, a PHY-Teacher, and a Trigger. The DRL-Student is a DRL agent that innovates in the dual self-learning and teaching-to-learn paradigm and the real-time safety-informed batch sampling. On the other hand, PHY-Teacher is a physics-model-based design of action policies that focuses solely on safety-critical functions. PHY-Teacher is novel in its real-time patch for two key missions: i) fostering the teaching-to-learn paradigm for DRL-Student and ii) backing up the safety of real plants. The Trigger manages the interaction between the DRL-Student and the PHY-Teacher. Powered by the three interactive components, the Real-DRL can effectively address safety challenges that arise from the unknown unknowns and the Sim2Real gap. Additionally, Real-DRL notably features i) assured safety, ii) automatic hierarchy learning (i.e., safety-first learning and then high-performance learning), and iii) safety-informed batch sampling to address the learning experience imbalance caused by corner cases. Experiments with a real quadruped robot, a quadruped robot in NVIDIA Isaac Gym, and a cart-pole system, along with comparisons and ablation studies, demonstrate the Real-DRL's effectiveness and unique features.

cs.RO↗

arXiv2025

Runtime Learning of Quadruped Robots in Wild Environments

This paper presents a runtime learning framework for quadruped robots, enabling them to learn and adapt safely in dynamic wild environments. The framework integrates sensing, navigation, and control, forming a closed-loop system for the robot. The core novelty of this framework lies in two interactive and complementary components within the control module: the high-performance (HP)-Student and the high-assurance (HA)-Teacher. HP-Student is a deep reinforcement learning (DRL) agent that engages in self-learning and teaching-to-learn to develop a safe and high-performance action policy. HA-Teacher is a simplified yet verifiable physics-model-based controller, with the role of teaching HP-Student about safety while providing a backup for the robot's safe locomotion. HA-Teacher is innovative due to its real-time physics model, real-time action policy, and real-time control goals, all tailored to respond effectively to real-time wild environments, ensuring safety. The framework also includes a coordinator who effectively manages the interaction between HP-Student and HA-Teacher. Experiments involving a Unitree Go2 robot in Nvidia Isaac Gym and comparisons with state-of-the-art safe DRLs demonstrate the effectiveness of the proposed runtime learning framework.

cs.RO↗

arXiv2024

Physics-model-guided Worst-case Sampling for Safe Reinforcement Learning

Real-world accidents in learning-enabled CPS frequently occur in challenging corner cases. During the training of deep reinforcement learning (DRL) policy, the standard setup for training conditions is either fixed at a single initial condition or uniformly sampled from the admissible state space. This setup often overlooks the challenging but safety-critical corner cases. To bridge this gap, this paper proposes a physics-model-guided worst-case sampling strategy for training safe policies that can handle safety-critical cases toward guaranteed safety. Furthermore, we integrate the proposed worst-case sampling strategy into the physics-regulated deep reinforcement learning (Phy-DRL) framework to build a more data-efficient and safe learning algorithm for safety-critical CPS. We validate the proposed training strategy with Phy-DRL through extensive experiments on a simulated cart-pole system, a 2D quadrotor, a simulated and a real quadruped robot, showing remarkably improved sampling efficiency to learn more robust safe policies.

cs.RO↗

arXiv2024

Simplex-enabled Safe Continual Learning Machine

This paper proposes the SeC-Learning Machine: Simplex-enabled safe continual learning for safety-critical autonomous systems. The SeC-learning machine is built on Simplex logic (that is, ``using simplicity to control complexity'') and physics-regulated deep reinforcement learning (Phy-DRL). The SeC-learning machine thus constitutes HP (high performance)-Student, HA (high assurance)-Teacher, and Coordinator. Specifically, the HP-Student is a pre-trained high-performance but not fully verified Phy-DRL, continuing to learn in a real plant to tune the action policy to be safe. In contrast, the HA-Teacher is a mission-reduced, physics-model-based, and verified design. As a complementary, HA-Teacher has two missions: backing up safety and correcting unsafe learning. The Coordinator triggers the interaction and the switch between HP-Student and HA-Teacher. Powered by the three interactive components, the SeC-learning machine can i) assure lifetime safety (i.e., safety guarantee in any continual-learning stage, regardless of HP-Student's success or convergence), ii) address the Sim2Real gap, and iii) learn to tolerate unknown unknowns in real plants. The experiments on a cart-pole system and a real quadruped robot demonstrate the distinguished features of the SeC-learning machine, compared with continual learning built on state-of-the-art safe DRL frameworks with approaches to addressing the Sim2Real gap.

cs.LG↗

arXiv2024

Physics-Regulated Deep Reinforcement Learning: Invariant Embeddings

This paper proposes the Phy-DRL: a physics-regulated deep reinforcement learning (DRL) framework for safety-critical autonomous systems. The Phy-DRL has three distinguished invariant-embedding designs: i) residual action policy (i.e., integrating data-driven-DRL action policy and physics-model-based action policy), ii) automatically constructed safety-embedded reward, and iii) physics-model-guided neural network (NN) editing, including link editing and activation editing. Theoretically, the Phy-DRL exhibits 1) a mathematically provable safety guarantee and 2) strict compliance of critic and actor networks with physics knowledge about the action-value function and action policy. Finally, we evaluate the Phy-DRL on a cart-pole system and a quadruped robot. The experiments validate our theoretical results and demonstrate that Phy-DRL features guaranteed safety compared to purely data-driven DRL and solely model-based design while offering remarkably fewer learning parameters and fast training towards safety guarantee.

cs.AI↗

arXiv2023

Physical Deep Reinforcement Learning Towards Safety Guarantee

Deep reinforcement learning (DRL) has achieved tremendous success in many complex decision-making tasks of autonomous systems with high-dimensional state and/or action spaces. However, the safety and stability still remain major concerns that hinder the applications of DRL to safety-critical autonomous systems. To address the concerns, we proposed the Phy-DRL: a physical deep reinforcement learning framework. The Phy-DRL is novel in two architectural designs: i) Lyapunov-like reward, and ii) residual control (i.e., integration of physics-model-based control and data-driven control). The concurrent physical reward and residual control empower the Phy-DRL the (mathematically) provable safety and stability guarantees. Through experiments on the inverted pendulum, we show that the Phy-DRL features guaranteed safety and stability and enhanced robustness, while offering remarkably accelerated training and enlarged reward.

cs.LG↗

arXiv2022

Phy-Taylor: Physics-Model-Based Deep Neural Networks

Purely data-driven deep neural networks (DNNs) applied to physical engineering systems can infer relations that violate physics laws, thus leading to unexpected consequences. To address this challenge, we propose a physics-model-based DNN framework, called Phy-Taylor, that accelerates learning compliant representations with physical knowledge. The Phy-Taylor framework makes two key contributions; it introduces a new architectural Physics-compatible neural network (PhN), and features a novel compliance mechanism, we call {\em Physics-guided Neural Network Editing\}. The PhN aims to directly capture nonlinearities inspired by physical quantities, such as kinetic energy, potential energy, electrical power, and aerodynamic drag force. To do so, the PhN augments neural network layers with two key components: (i) monomials of Taylor series expansion of nonlinear functions capturing physical knowledge, and (ii) a suppressor for mitigating the influence of noise. The neural-network editing mechanism further modifies network links and activation functions consistently with physical knowledge. As an extension, we also propose a self-correcting Phy-Taylor framework that introduces two additional capabilities: (i) physics-model-based safety relationship learning, and (ii) automatic output correction when violations of safety occur. Through experiments, we show that (by expressing hard-to-learn nonlinearities directly and by constraining dependencies) Phy-Taylor features considerably fewer parameters, and a remarkably accelerated training process, while offering enhanced model robustness and accuracy.

cs.LG↗

arXiv2022

Cost Function Learning in Memorized Social Networks with Cognitive Behavioral Asymmetry

This paper investigates the cost function learning in social information networks, wherein the influence of humans' memory on information consumption is explicitly taken into account. We first propose a model for social information-diffusion dynamics with a focus on systematic modeling of asymmetric cognitive bias, represented by confirmation bias and novelty bias. Building on the proposed social model, we then propose the M$^{3}$IRL: a model and maximum-entropy based inverse reinforcement learning framework for learning the cost functions of target individuals in the memorized social networks. Compared with the existing Bayesian IRL, maximum entropy IRL, relative entropy IRL and maximum causal entropy IRL, the characteristics of M$^{3}$IRL are significantly different here: no dependency on the Markov Decision Process principle, the need of only a single finite-time trajectory sample, and bounded decision variables. Finally, the effectiveness of the proposed social information-diffusion model and the M$^{3}$IRL algorithm are validated by the online social media data.

cs.SI↗

arXiv2022

SL1-Simplex: Safe Velocity Regulation of Self-Driving Vehicles in Dynamic and Unforeseen Environments

This paper proposes a novel extension of the Simplex architecture with model switching and model learning to achieve safe velocity regulation of self-driving vehicles in dynamic and unforeseen environments. To guarantee the reliability of autonomous vehicles, an $\mathcal{L}_{1}$ adaptive controller that compensates for uncertainties and disturbances is employed by the Simplex architecture as a verified safe controller to tolerate concurrent software and physical failures. Meanwhile, safe switching controller is incorporated into the Simplex for safe velocity regulation through the integration of the traction control system and anti-lock braking system. Specifically, the vehicle's angular and longitudinal velocities asymptotically track the provided references that vary with driving environments, while the wheel slips are restricted to safety envelopes to prevent slipping and sliding. Due to the high dependence of vehicle dynamics on the driving environments, the proposed Simplex leverages the finite-time model learning to timely learn and update the vehicle model for $\mathcal{L}_{1}$ adaptive controller, when any deviation from the safety envelope or the uncertainty measurement threshold occurs in the unforeseen driving environments. Finally, the effectiveness of the proposed Simplex architecture for safe velocity regulation is validated by the AutoRally platform.

eess.SY↗

arXiv2021

Social System Inference from Noisy Observations

This paper studies social system inference from a single trajectory of public evolving opinions, wherein observation noise leads to the statistical dependence of samples on time and coordinates. We first propose a cyber-social system that comprises individuals in a social network and a set of information sources in a cyber layer, whose opinion dynamics explicitly takes confirmation bias, novelty bias and process noise into account. Based on the proposed social model, we then study the sample complexity of least-square auto-regressive model estimation, which governs the number of observations that are sufficient for the identified model to achieve the prescribed levels of accuracy and confidence. Building on the identified social model, we then investigate social inference, with particular focus on the weighted network topology, the subconscious bias and the model parameters of confirmation bias and novelty bias. Finally, the theoretical results and the effectiveness of the proposed social model and inference algorithm are validated by the US Senate Member Ideology data.

cs.SI↗

arXiv2021

Dynamics of Public Opinion Evolution with Asymmetric Cognitive Bias

In this paper, we propose a pubic opinion model with incorporation of asymmetric cognitive bias: confirmation bias and negativity bias. We then investigate the generic modeling guidance of capturing asymmetric confirmation bias and negativity bias. A numerical examples is provided to demonstrate the correctness of asymmetric cognitive bias model.

cs.SI↗

arXiv2021

Finite-Time Model Inference From A Single Noisy Trajectory

This paper proposes a novel model inference procedure to identify system matrix from a single noisy trajectory over a finite-time interval. The proposed inference procedure comprises an observation data processor, a redundant data processor and an ordinary least-square estimator, wherein the data processors mitigate the influence of observation noise on inference error. We first systematically investigate the comparisons with naive least-square-regression based model inference and uncover that 1) the same observation data has identical influence on the feasibility of the proposed and the naive model inferences, 2) the naive model inference uses all of the redundant data, while the proposed model inference optimally uses the basis and the redundant data. We then study the sample complexity of the proposed model inference in the presence of observation noise, which leads to the dependence of the processed bias in the observed system trajectory on time and coordinates. Particularly, we derive the sample-complexity upper bound (on the number of observations sufficient to infer a model with prescribed levels of accuracy and confidence) and the sample-complexity lower bound (high-probability lower bound on model error). Finally, the proposed model inference is numerically validated and analyzed.

eess.SY↗

arXiv2020

Adaptive Robust Quadratic Programs using Control Lyapunov and Barrier Functions

This paper presents adaptive robust quadratic program (QP) based control using control Lyapunov and barrier functions for nonlinear systems subject to time-varying and state-dependent uncertainties. An adaptive estimation law is proposed to estimate the pointwise value of the uncertainties with pre-computable estimation error bounds. The estimated uncertainty and the error bounds are then used to formulate a robust QP, which ensures that the actual uncertain system will not violate the safety constraints defined by the control barrier function. Additionally, the accuracy of the uncertainty estimation can be systematically improved by reducing the estimation sampling time, leading subsequently to reduced conservatism of the formulated robust QP. The proposed approach is validated in simulations on an adaptive cruise control problem and through comparisons with existing approaches.

math.OC↗

arXiv2020

Impact of Confirmation Bias on Competitive Information Spread in Social Networks

This paper investigates the impact of confirmation bias on competitive information spread in the cyber-social network that comprises individuals in a social network and competitive information sources in cyber layer. We formulate the problem as a zero-sum game, which admits a unique Nash equilibrium in pure strategies. We characterize the dependence of pure Nash equilibrium on the public's innate opinions, the social network topology, as well as the parameters of confirmation bias. We uncover that confirmation bias moves the equilibrium towards the center only when the innate opinions are not neutral, and this move does not occur for the competitive information sources simultaneously. Numerical examples in the context of well-known Krackhardt's advice network are provided to demonstrate the correctness of theoretical results.

cs.SI↗

arXiv2020

Novel Stealthy Attack and Defense Strategies for Networked Control Systems

This paper studies novel attack and defense strategies, based on a class of stealthy attacks, namely the zero-dynamics attack (ZDA), for multi-agent control systems. ZDA poses a formidable security challenge since its attack signal is hidden in the null-space of the state-space representation of the control system and hence it can evade conventional detection methods. An intuitive defense strategy builds on changing the aforementioned representation via switching through a set of carefully crafted topologies. In this paper, we propose realistic ZDA variations where the attacker is aware of this topology-switching strategy, and hence employs the following policies to avoid detection: (i) pause, update and resume ZDA according to the knowledge of switching topologies; (ii) cooperate with a concurrent stealthy topology attack that alters network topology at switching times, such that the original ZDA is feasible under the corrupted topology. We first systematically study the proposed ZDA variations, and then develop defense strategies against them under the realistic assumption that the defender has no knowledge of attack starting, pausing, and resuming times and the number of misbehaving agents. Particularly, we characterize conditions for detectability of the proposed ZDA variations, in terms of the network topologies to be maintained, the set of agents to be monitored, and the measurements of the monitored agents that should be extracted, while simultaneously preserving the privacy of the states of the non-monitored agents. We then propose an attack detection algorithm based on the Luenberger observer, using the characterized detectability conditions. We provide numerical simulation results to demonstrate our theoretical findings.

eess.SY↗

arXiv2020

On Inference of Network Topology and Confirmation Bias in Cyber-Social Networks

This paper studies topology inference, from agent states, of a directed cyber-social network with opinion spreading dynamics model that explicitly takes confirmation bias into account. The cyber-social network comprises a set of partially connected directed network of agents at the social level, and a set of information sources at the cyber layer. The necessary and sufficient conditions for the existence of exact inference solution are characterized. A method for exact inference, when it is possible, of entire network topology as well as confirmation bias model parameters is proposed for the case where the bias mentioned earlier follows a piece-wise linear model. The particular case of no confirmation bias is analyzed in detail. For the setting where the model of confirmation bias is unknown, an algorithm that approximates the network topology, building on the exact inference method, is presented. This algorithm can exactly infer the weighted communication from the neighbors to the non-followers of information sources. Numerical simulations demonstrate the effectiveness of the proposed methods for different scenarios.

cs.SI↗

arXiv2019

Novel Defense Strategy Against Zero-Dynamics Attack in Multi-Agent Systems

This paper considers the defense strategy of strategic topology switching for the second-order multi-agent system under zero-dynamics attack (ZDA) whose attack-starting time is allowed to be not the initial time. We first study the detectability of ZDA, which is a sufficient and necessary condition of switching topologies in detecting the attack. Based on the detectability, a Luenberger observer under switching topology is proposed to detect the stealthy attack. The primary advantages of the attack-detection algorithm are twofold: (i) in detecting ZDA, the algorithm allows the defender (system operator) to have no knowledge of the attack-starting time and the misbehaving agents (i.e., agents under attack); (ii) in tracking system in the absence of attacks, Luenberger observer has no constraint on the magnitudes of observer gains and the number of observed outputs, i.e., only one agent's observed output is sufficient. Simulations are provided to verify the effectiveness of the strategic topology-switching algorithm.

eess.SY↗