arXiv subjects

He Yin

Publications and source records attributed to He Yin.

At least 19 recordsLinked to original sources

arXiv2026

VOFA: Visual Object Goal Pushing with Force-Adaptive Control for Humanoids

The ability to push large objects in a goal-directed manner using onboard egocentric perception is an essential skill for humanoid robots to perform complex tasks such as material handling in warehouses. To robustly manipulate heavy objects to arbitrary goal configurations, the robot must cope with unknown object mass and ground friction, noisy onboard perception, and actuation errors; all in a real-time feedback loop. Existing solutions either rely on privileged object-state information without onboard perception or lack robustness to variations in goal configurations and object physical properties. In this work, we present VOFA, a visual goal-conditioned humanoid loco-manipulation system capable of pushing objects with unknown physical properties to arbitrary goal positions. VOFA consists of a two-level hierarchical architecture with a high-level visuomotor policy and a low-level force-adaptive whole-body controller. The high-level policy processes noisy onboard observations and generates goal-conditioned commands to operate in closed loop across diverse object-goal configurations, while the low-level whole-body controller provides robustness to variations in object physical properties. VOFA is extensively evaluated in both simulation and real-world experiments on the Booster T1 humanoid robot. Our results demonstrate strong performance, achieving over 90% success in simulation and over 80% success in real-world trials. Moreover, VOFA successfully pushes objects weighing up to 17kg, exceeding half of the Booster T1's body weight.

cs.RO↗

arXiv2026

GaussianSSC: Triplane-Guided Directional Gaussian Fields for 3D Semantic Completion

We present \emph{GaussianSSC}, a two-stage, grid-native and triplane-guided approach to semantic scene completion (SSC) that injects the benefits of Gaussians without replacing the voxel grid or maintaining a separate Gaussian set. We introduce \emph{Gaussian Anchoring}, a sub-pixel, Gaussian-weighted image aggregation over fused FPN features that tightens voxel--image alignment and improves monocular occupancy estimation. We further convert point-like voxel features into a learned per-voxel Gaussian field and refine triplane features via a triplane-aligned \emph{Gaussian--Triplane Refinement} module that combines \emph{local gathering} (target-centric) and \emph{global aggregation} (source-centric). This directional, anisotropic support captures surface tangency, scale, and occlusion-aware asymmetry while preserving the efficiency of triplane representations. On SemanticKITTI~\cite{behley2019semantickitti}, GaussianSSC improves Stage~1 occupancy by +1.0\% Recall, +2.0\% Precision, and +1.8\% IoU over state-of-the-art baselines, and improves Stage~2 semantic prediction by +1.8\% IoU and +0.8\% mIoU.

cs.RO↗

arXiv2025

Region Based SLAM-Aware Exploration: Efficient and Robust Autonomous Mapping Strategy That Can Scale

Autonomous exploration for mapping unknown large scale environments is a fundamental challenge in robotics, with efficiency in time, stability against map corruption and computational resources being crucial. This paper presents a novel approach to indoor exploration that addresses these key issues in existing methods. We introduce a Simultaneous Localization and Mapping (SLAM)-aware region-based exploration strategy that partitions the environment into discrete regions, allowing the robot to incrementally explore and stabilize each region before moving to the next one. This approach significantly reduces redundant exploration and improves overall efficiency. As the device finishes exploring a region and stabilizes it, we also perform SLAM keyframe marginalization, a technique which reduces problem complexity by eliminating variables, while preserving their essential information. To improves robustness and further enhance efficiency, we develop a checkpoint system that enables the robot to resume exploration from the last stable region in case of failures, eliminating the need for complete re-exploration. Our method, tested in real homes, office and simulations, outperforms state-of-the-art approaches. The improvements demonstrate substantial enhancements in various real world environments, with significant reductions in keyframe usage (85%), submap usage (50% office, 32% home), pose graph optimization time (78-80%), and exploration duration (10-15%). This region-based strategy with keyframe marginalization offers an efficient solution for autonomous robotic mapping.

cs.RO↗

arXiv2024

ET-Former: Efficient Triplane Deformable Attention for 3D Semantic Scene Completion From Monocular Camera

We introduce ET-Former, a novel end-to-end algorithm for semantic scene completion using a single monocular camera. Our approach generates a semantic occupancy map from single RGB observation while simultaneously providing uncertainty estimates for semantic predictions. By designing a triplane-based deformable attention mechanism, our approach improves geometric understanding of the scene than other SOTA approaches and reduces noise in semantic predictions. Additionally, through the use of a Conditional Variational AutoEncoder (CVAE), we estimate the uncertainties of these predictions. The generated semantic and uncertainty maps will help formulate navigation strategies that facilitate safe and permissible decision making in the future. Evaluated on the Semantic-KITTI dataset, ET-Former achieves the highest Intersection over Union (IoU) and mean IoU (mIoU) scores while maintaining the lowest GPU memory usage, surpassing state-of-the-art (SOTA) methods. It improves the SOTA scores of IoU from 44.71 to 51.49 and mIoU from 15.04 to 16.30 on SeamnticKITTI test, with a notably low training memory consumption of 10.9 GB. Project page: https://github.com/jingGM/ET-Former.git.

cs.CV↗

arXiv2022

Synthesis of Stabilizing Recurrent Equilibrium Network Controllers

We propose a parameterization of a nonlinear dynamic controller based on the recurrent equilibrium network, a generalization of the recurrent neural network. We derive constraints on the parameterization under which the controller guarantees exponential stability of a partially observed dynamical system with sector bounded nonlinearities. Finally, we present a method to synthesize this controller using projected policy gradient methods to maximize a reward function with arbitrary structure. The projection step involves the solution of convex optimization problems. We demonstrate the proposed method with simulated examples of controlling nonlinear plants, including plants modeled with neural networks.

eess.SY↗

arXiv2022

Learning Neural Networks under Input-Output Specifications

In this paper, we examine an important problem of learning neural networks that certifiably meet certain specifications on input-output behaviors. Our strategy is to find an inner approximation of the set of admissible policy parameters, which is convex in a transformed space. To this end, we address the key technical challenge of convexifying the verification condition for neural networks, which is derived by abstracting the nonlinear specifications and activation functions with quadratic constraints. In particular, we propose a reparametrization scheme of the original neural network based on loop transformation, which leads to a convex condition that can be enforced during learning. This theoretical construction is validated in an experiment that specifies reachable sets for different regions of inputs.

eess.SY↗

arXiv2022

Safe-by-Design Planner-Tracker Synthesis

We present a safe-by-design trajectory planning and tracking framework for nonlinear dynamical systems using a hierarchy of system models. The planning layer uses a low-fidelity model to plan a feasible trajectory satisfying the planning constraints, and the tracking layer utilizes the high-fidelity model to design a controller that restricts the error states between the low- and high-fidelity models to a bounded set. The low-fidelity model enables the planning to be performed online (e.g. using Model Predictive Control) and the tracking controller and error bound are derived offline (e.g. using sum-of-squares programming). To provide freedom in the choice of the low-fidelity model, we allow the tracking error to depend on both the states and inputs of the planner. The goal of this article is to provide a tutorial review of this hierarchical framework and to illustrate it with examples, including a design for vehicle obstacle avoidance.

eess.SY↗

arXiv2021

Recurrent Neural Network Controllers Synthesis with Stability Guarantees for Partially Observed Systems

Neural network controllers have become popular in control tasks thanks to their flexibility and expressivity. Stability is a crucial property for safety-critical dynamical systems, while stabilization of partially observed systems, in many cases, requires controllers to retain and process long-term memories of the past. We consider the important class of recurrent neural networks (RNN) as dynamic controllers for nonlinear uncertain partially-observed systems, and derive convex stability conditions based on integral quadratic constraints, S-lemma and sequential convexification. To ensure stability during the learning and control process, we propose a projected policy gradient method that iteratively enforces the stability conditions in the reparametrized space taking advantage of mild additional information on system dynamics. Numerical experiments show that our method learns stabilizing controllers while using fewer samples and achieving higher final performance compared with policy gradient.

eess.SY↗

arXiv2020

Imitation Learning with Stability and Safety Guarantees

A method is presented to learn neural network (NN) controllers with stability and safety guarantees through imitation learning (IL). Convex stability and safety conditions are derived for linear time-invariant plant dynamics with NN controllers by merging Lyapunov theory with local quadratic constraints to bound the nonlinear activation functions in the NN. These conditions are incorporated in the IL process, which minimizes the IL loss, and maximizes the volume of the region of attraction associated with the NN controller simultaneously. An alternating direction method of multipliers based algorithm is proposed to solve the IL problem. The method is illustrated on an inverted pendulum system, aircraft longitudinal dynamics, and vehicle lateral dynamics.

eess.SY↗

arXiv2020

Co-design of Control and Planning for Multi-rotor UAVs with Signal Temporal Logic Specifications

Urban Air Mobility (UAM), or the scenario where multiple manned and Unmanned Aerial Vehicles (UAVs) carry out various tasks over urban airspaces, is a transportation concept of the future that is gaining prominence. UAM missions with complex spatial, temporal and reactive requirements can be succinctly represented using Signal Temporal Logic (STL), a behavioral specification language. However, planning and control of systems with STL specifications is computationally intensive, usually resulting in planning approaches that do not guarantee dynamical feasibility, or control approaches that cannot handle complex STL specifications. Here, we present an approach to co-design the planner and control such that a given STL specification (possibly over multiple UAVs) is satisfied with trajectories that are dynamically feasible and our controller can track them with a bounded tracking-error that the planner accounts for. The tracking controller is formulated for the non-linear dynamics of the individual UAVs, and the tracking error bound is computed for this controller when the trajectories satisfy some kinematic constraints. We also augment an existing multi-UAV STL-based trajectory generator in order to generate trajectories that satisfy such constraints. We show that this co-design allows for trajectories that satisfy a given STL specification, and are also dynamically feasible in the sense that they can be tracked with bounded error. The applicability of this approach is demonstrated through simulations of multi-UAV missions.

eess.SY↗

arXiv2020

Stability Analysis using Quadratic Constraints for Systems with Neural Network Controllers

A method is presented to analyze the stability of feedback systems with neural network controllers. Two stability theorems are given to prove asymptotic stability and to compute an ellipsoidal inner-approximation to the region of attraction (ROA). The first theorem addresses linear time-invariant systems, and merges Lyapunov theory with local (sector) quadratic constraints to bound the nonlinear activation functions in the neural network. The second theorem allows the system to include perturbations such as unmodeled dynamics, slope-restricted nonlinearities, and time delay, using integral quadratic constraint (IQCs) to capture their input/output behavior. This in turn allows for off-by-one IQCs to refine the description of activation functions by capturing their slope restrictions. Both results rely on semidefinite programming to approximate the ROA. The method is illustrated on systems with neural networks trained to stabilize a nonlinear inverted pendulum as well as vehicle lateral dynamics with actuator uncertainty.

eess.SY↗

arXiv2020

Passivity-based distributed acquisition and station-keeping control of a satellite constellation in areostationary orbit

We present a distributed control law to assemble a cluster of satellites into an equally-spaced, planar constellation in a desired circular orbit about a planet. We assume each satellite only uses local information, transmitted through communication links with neighboring satellites. The same control law is used to maintain relative angular positions in the presence of disturbance forces. The stability of the constellation in the desired orbit is proved using a compositional approach. We first show the existence and uniqueness of an equilibrium of the interconnected system. We then certify each satellite and communication link is equilibrium-independent passive with respective storage functions. By leveraging the skew symmetric coupling structure of the constellation and the equilibrium-independent passivity property of each subsystem, we show that the equilibrium of the interconnected system is stable with a Lyapunov function composed of the individual subsystem storage functions. We further prove that the angular velocity of each satellite converges to the desired value necessary to maintain circular, areostationary orbit. Finally, we present simulation results to demonstrate the efficacy of the proposed control law in acquisition and station-keeping of an equally-spaced satellite constellation in areostationary orbit despite the presence of unmodeled disturbance forces.

eess.SY↗

arXiv2020

Bayesian Safe Learning and Control with Sum-of-Squares Analysis and Polynomial Kernels

We propose an iterative method to safely learn the unmodeled dynamics of a nonlinear system using Bayesian Gaussian process (GP) models with polynomial kernel functions. The method maintains safety by ensuring that the system state stays within the region of attraction (ROA) of a stabilizing control policy while collecting data. A quadratic programming based exploration control policy is computed to keep the exploration trajectory inside an inner-approximation of the ROA and to maximize the information gained from the trajectory. A prior GP model, which incorporates prior information about the unknown dynamics, is used to construct an initial stabilizing policy. As the GP model is updated with data, it is used to synthesize a new policy and a larger ROA, which increases the range of safe exploration. The use of polynomial kernels allows us to compute ROA inner-approximations and stabilizing control laws for the model using sum-of-squares programming. We also provide a probabilistic guarantee of safety which ensures that the policy computed using the learned model stabilizes the true dynamics with high confidence.

eess.SY↗

arXiv2020

Backward Reachability using Integral Quadratic Constraints for Uncertain Nonlinear Systems

A method is proposed to compute robust inner-approximations to the backward reachable set for uncertain nonlinear systems. It also produces a robust control law that drives trajectories starting in these sets to the target set. The method merges dissipation inequalities and integral quadratic constraints (IQCs) with both hard and soft IQC factorizations. Computational algorithms are presented using the generalized S-procedure and sum-of-squares techniques. The use of IQCs in backward reachability analysis allows for a variety of perturbations including parametric uncertainty, unmodeled dynamics, nonlinearities, and uncertain time delays. The method is demonstrated on two examples, including a 6-state quadrotor with actuator uncertainties.

eess.SY↗

arXiv2019

Continuous and discrete abstractions for planning, applied to ship docking

We propose a hierarchical control framework for the synthesis of correct-by-construction controllers for nonlinear control-affine systems with respect to reach-avoid-stay specifications. We first create a low-dimensional continuous abstraction of the system and use Sum-of-Squares (SOS) programming to obtain a low-level controller ensuring a bounded error between the two models. We then create a discrete abstraction of the continuous abstraction and use formal methods to synthesize a controller satisfying the specifications shrunk by the obtained error bound. Combining both controllers finally solves the main control problem on the initial system. This two-step framework allows the discrete abstraction methods to deal with higher-dimensional systems which may be computationally expensive without the prior continuous abstraction. The main novelty of the proposed SOS continuous abstraction is that it allows the error between abstract and concrete models to explicitly depend on the control input of the abstract model, which offers more freedom in the choice of the continuous abstraction model and provides lower error bounds than when only the states of both models are considered. This approach is illustrated on the docking problem of a marine vessel.

eess.SY↗

arXiv2019

Optimization Based Planner Tracker Design for Safety Guarantees

We present a safe-by-design approach to path planning and control for nonlinear systems. The planner uses a low fidelity model of the plant to compute reference trajectories by solving an MPC problem, while the plant being controlled utilizes a feedback control law that tracks those trajectories with an upper-bound on the tracking error. Our main goal is to allow for maximum permissiveness (that is, room for constraint feasibility) of the planner, while maintaining safety after accounting for the tracking error bound. We achieve this by parametrizing the state and input constraints imposed on the planner and deriving corresponding parametrized tracking control laws and tracking error bounds, which are computed offline through Sum-of-Squares programming. The parameters are then optimally chosen to maximize planner permissiveness, while guaranteeing safety.

eess.SY↗

arXiv2019

Continuous Abstraction of Nonlinear Systems using Sum-of-Squares Programming

We present a control design procedure for nonlinear control systems in which we represent a potentially high dimensional system with a low dimensional continuous-state abstraction. The abstraction generates a reference which the original system follows with a low level controller. We propose sum-of-squares programming as a tool to design this controller and to provide an upper bound on the relative error between the system and its abstraction. We compute the low level controller simultaneously with a simulation function that gives the boundedness guarantee for the relative error.

math.OC↗

arXiv2019

Backward Reachability for Polynomial Systems on A Finite Horizon

A method is presented to obtain an inner-approximation of the backward reachable set (BRS) of a given target tube, along with an admissible controller that maintains trajectories inside this tube. The proposed optimization algorithms are formulated as nonlinear optimization problems, which are decoupled into tractable subproblems and solved by an iterative algorithm using the polynomial S-procedure and sum-of-squares techniques. This framework is also extended to uncertain nonlinear systems with L_2 disturbances and L_{\infty} parametric uncertainties. The effectiveness of the method is demonstrated on several nonlinear robotics and aircraft systems with control saturation.

eess.SY↗