arXiv subjects

Martin Buss

Publications and source records attributed to Martin Buss.

17 recordsLinked to original sources

arXiv2026

Branch-Stochastic Model Predictive Control for Motion Planning under Multi-Modal Uncertainty with Scenario Clustering

Motion planning for autonomous driving must account for multi-modal uncertainty in both the intentions and trajectories of surrounding vehicles. Handling uncertainty in a worst-case manner guarantees robustness but often leads to excessive conservatism. Stochastic Model Predictive Control (SMPC) reduces trajectory-level conservatism through chance constraints, yet remains conservative with respect to intention uncertainty since constraints must hold across all intentions. We present a novel combination of SMPC and the branching structure, enabling the planner to generate distinct trajectories for different possible intentions while maintaining safety under trajectory uncertainty. A novel scenario clustering is proposed to merge prediction scenarios based on high-level decision similarity, thereby ensuring real-time tractability. Furthermore, an adaptive branching-time computation postpones commitment to separate plans until intention uncertainty is sufficiently reduced. Simulation studies in challenging highway scenarios demonstrate that the proposed method improves safety, reduces conservatism, and achieves real-time computational performance.

cs.RO↗

arXiv2024

Coevolution of Opinion Dynamics and Recommendation System: Modeling, Analysis and Reinforcement Learning Based Manipulation

In this work, we develop an analytical framework that integrates opinion dynamics with a recommendation system. By incorporating elements such as collaborative filtering, we provide a precise characterization of how recommendation systems shape interpersonal interactions and influence opinion formation. Moreover, the property of the coevolution of both opinion dynamics and recommendation systems is also shown. Specifically, the convergence of this coevolutionary system is theoretically proved, and the mechanisms behind filter bubble formation are elucidated. Our analysis of the maximum number of opinion clusters shows how recommendation system parameters affect opinion grouping and polarization. Additionally, we incorporate the influence of propagators into our model and propose a reinforcement learning-based solution. The analysis and the propagation solution are demonstrated in simulations using the Yelp data set.

eess.SY↗

arXiv2023

Incorporating Target Vehicle Trajectories Predicted by Deep Learning Into Model Predictive Controlled Vehicles

Model Predictive Control (MPC) has been widely applied to the motion planning of autonomous vehicles. An MPC-controlled vehicle is required to predict its own trajectories in a finite prediction horizon according to its model. Beyond this, the vehicle should also incorporate the prediction of the trajectory of its nearby vehicles, or target vehicles (TVs) into its decision-making. The conventional trajectory prediction methods, such as the constant-speed-based ones, are too trivial to accurately capture the potential collision risks. In this report, we propose a novel MPC-based motion planning method for an autonomous vehicle with a set of risk-aware constraints. These constraints incorporate the predicted trajectory of a TV learned using a deep-learning-based method. A recurrent neural network (RNN) is used to predict the TV's future trajectory based on its historical data. Then, the predicted TV trajectory is incorporated into the optimization of the MPC of the ego vehicle to generate collision-free motion. Simulation studies are conducted to showcase the prediction accuracy of the RNN model and the collision-free trajectories generated by the MPC.

cs.RO↗

arXiv2023

Identifying Reaction-Aware Driving Styles of Stochastic Model Predictive Controlled Vehicles by Inverse Reinforcement Learning

The driving style of an Autonomous Vehicle (AV) refers to how it behaves and interacts with other AVs. In a multi-vehicle autonomous driving system, an AV capable of identifying the driving styles of its nearby AVs can reliably evaluate the risk of collisions and make more reasonable driving decisions. However, there has not been a consistent definition of driving styles for an AV in the literature, although it is considered that the driving style is encoded in the AV's trajectories and can be identified using Maximum Entropy Inverse Reinforcement Learning (ME-IRL) methods as a cost function. Nevertheless, an important indicator of the driving style, i.e., how an AV reacts to its nearby AVs, is not fully incorporated in the feature design of previous ME-IRL methods. In this paper, we describe the driving style as a cost function of a series of weighted features. We design additional novel features to capture the AV's reaction-aware characteristics. Then, we identify the driving styles from the demonstration trajectories generated by the Stochastic Model Predictive Control (SMPC) using a modified ME-IRL method with our newly proposed features. The proposed method is validated using MATLAB simulation and an off-the-shelf experiment.

cs.RO↗

arXiv2023

A Persistent-Excitation-Free Method for System Disturbance Estimation Using Concurrent Learning

Observer-based methods are widely used to estimate the disturbances of different dynamic systems. However, a drawback of the conventional disturbance observers is that they all assume persistent excitation (PE) of the systems. As a result, they may lead to poor estimation precision when PE is not ensured, for instance, when the disturbance gain of the system is close to the singularity. In this paper, we propose a novel disturbance observer based on concurrent learning (CL) with time-variant history stacks, which ensures high estimation precision even in PE-free cases. The disturbance observer is designed in both continuous and discrete time. The estimation errors of the proposed method are proved to converge to a bounded set using the Lyapunov method. A history-sample-selection procedure is proposed to reduce the estimation error caused by the accumulation of old history samples. A simulation study on epidemic control shows that the proposed method produces higher estimation precision than the conventional disturbance observer when PE is not satisfied. This justifies the correctness of the proposed CL-based disturbance observer and verifies its applicability to solving practical problems.

eess.SY↗

arXiv2023

Simultaneous Recursive Identification of Parameters and Switching Manifolds Identification of Discrete-Time Switched Linear Systems

A novel procedure for the online identification of a class of discrete-time switched linear systems, which simultaneously estimates the parameters and switching manifolds of the systems, is proposed in this paper. Firstly, to estimate the parameters of the subsystems, a discrete-time concurrent learning-based recursive parameter estimator is designed to guarantee the exponential convergence of the estimation errors to zero. Secondly, as an assistant procedure of the identification framework, an online switching detection method is proposed by making use of the history stacks produced by the concurrent learning estimators. Thirdly, techniques of incremental support vector machine are applied to develop the recursive algorithm to estimate the system switching manifolds, with its stability proven by a Lynapunov-based method. At the end of the paper, the stability and precision of the proposed identification methods are confirmed by the numerical simulation of a 2-order switched linear system. Compared to the traditional offline identification methods, the proposed online identification framework possesses superior efficiency with respect to large amounts of data, while the limitations and outlook of this framework are also discussed within the conclusion.

eess.SY↗

arXiv2022

Adaptive Observer for a Class of Systems with Switched Unknown Parameters Using DREM

In this note, we develop an adaptive observer for a class of nonlinear systems with switched unknown parameters to estimate the states and parameters simultaneously. The main challenge lies in how to eliminate the disturbance effect of zero-input responses caused by the switching on the parameter estimation. These responses depend on the unknown states at switching instants (SASI) and constitute an additive disturbance to the parameter estimation, which obstructs parameter convergence to zero. Our solution is to treat the zero-input responses as excitations instead of disturbances. This is realized by first augmenting the system parameter with the SASI and then developing an estimator for the augmented parameter using the \textit{dynamic regression extension and mixing} (DREM) technique. Thanks to its property of element-wise parameter adaptation, the system parameter estimation is decoupled from the SASI. As a result, the estimation errors of system states and parameters converge to zero asymptotically. Furthermore, the robustness of the proposed adaptive observer is guaranteed in the presence of disturbances and noise. A numerical example validates the effectiveness of the proposed approach.

eess.SY↗

arXiv2021

Data Informed Residual Reinforcement Learning for High-Dimensional Robotic Tracking Control

The learning inefficiency of reinforcement learning (RL) from scratch hinders its practical application towards continuous robotic tracking control, especially for high-dimensional robots. This work proposes a data-informed residual reinforcement learning (DR-RL) based robotic tracking control scheme applicable to robots with high dimensionality. The proposed DR-RL methodology outperforms common RL methods regarding sample efficiency and scalability. Specifically, we first decouple the original robot into low-dimensional robotic subsystems; and further utilize one-step backward (OSBK) data to construct incremental subsystems that are equivalent model-free representations of the above decoupled robotic subsystems. The formulated incremental subsystems allow for parallel learning to relieve computation load and offer us mathematical descriptions of robotic movements for conducting theoretical analysis. Then, we apply DR-RL to learn the tracking control policy, a combination of incremental base policy and incremental residual policy, under a parallel learning architecture. The incremental residual policy uses the guidance from the incremental base policy as the learning initialization and further learns from interactions with environments to endow the tracking control policy with adaptability towards dynamically changing environments. Our proposed DR-RL based tracking control scheme is developed with rigorous theoretical analysis of system stability and weight convergence. The effectiveness of our proposed method is validated numerically on a 7-DoF KUKA iiwa robot manipulator and experimentally on a 3-DoF robot manipulator that would fail for other counterpart RL methods.

eess.SY↗

arXiv2021

Data Generation Method for Learning a Low-dimensional Safe Region in Safe Reinforcement Learning

Safe reinforcement learning aims to learn a control policy while ensuring that neither the system nor the environment gets damaged during the learning process. For implementing safe reinforcement learning on highly nonlinear and high-dimensional dynamical systems, one possible approach is to find a low-dimensional safe region via data-driven feature extraction methods, which provides safety estimates to the learning algorithm. As the reliability of the learned safety estimates is data-dependent, we investigate in this work how different training data will affect the safe reinforcement learning approach. By balancing between the learning performance and the risk of being unsafe, a data generation method that combines two sampling methods is proposed to generate representative training data. The performance of the method is demonstrated with a three-link inverted pendulum example.

eess.SY↗

arXiv2021

Safe Feedback Motion Planning in Unknown Environments: An Instantaneous Local Control Barrier Function Approach

Mobile robots are desired with resilience to safely interact with prior-unknown environments and finally accomplish given tasks. This paper utilizes instantaneous local sensory data to stimulate the safe feedback motion planning (SFMP) strategy with adaptability to diverse prior-unknown environments without building a global map. This is achieved by the numerical optimization with the constraints, referred to as instantaneous local control barrier functions (IL-CBFs) and goal-driven control Lyapunov functions (GD-CLFs), learned from perceptional signals. In particular, the IL-CBFs reflecting potential collisions and GD-CLFs encoding incrementally discovered subgoals are first online learned from local perceptual data. Then, the learned IL-CBFs are united with GD-CLFs in the context of quadratic programming (QP) to generate the safe feedback motion planning strategy. Rather importantly, an optimization over the admissible control space of IL-CBFs is conducted to enhance the solution feasibility of QP. The SFMP strategy is developed with theoretically guaranteed collision avoidance and convergence to destinations. Numerical simulations are conducted to reveal the effectiveness of the proposed SFMP strategy that drives mobile robots to safely reach the destination incrementally in diverse prior-unknown environments.

eess.SY↗

arXiv2021

Model-Free Incremental Adaptive Dynamic Programming Based Approximate Robust Optimal Regulation

This paper presents a new formulation for model-free robust optimal regulation of continuous-time nonlinear systems. The proposed reinforcement learning based approach, referred to as incremental adaptive dynamic programming (IADP), exploits measured data to allow the design of the approximate optimal incremental control strategy, which stabilizes the controlled system incrementally under model uncertainties, environmental disturbances, and input saturation. By leveraging the time delay estimation (TDE) technique, we first exploit sensory data to reduce the requirement of a complete dynamics, where measured data are adopted to construct an incremental dynamics that reflects the system evolution in an incremental form. Then, the resulting incremental dynamics serves to design the approximate optimal incremental control strategy based on adaptive dynamic programming, which is implemented as a simplified single critic structure to get the approximate solution to the value function of the Hamilton-Jacobi-Bellman equation. Furthermore, for the critic artificial neural network, experience data are used to design an off-policy weight update law with guaranteed weight convergence. Rather importantly, to address the unintentionally introduced TDE error, we incorporate a TDE error bound related term into the cost function, whereby the TDE error is attenuated during the optimization process. The system stability proof and the weight convergence proof are provided. Numerical simulations are conducted to validate the effectiveness and superiority of our proposed IADP, especially regarding the reduced control energy expenditure and the enhanced robustness.

eess.SY↗

arXiv2021

Model Reference Adaptive Control of Piecewise Affine Systems with State Tracking Performance Guarantees

In this paper, we investigate the model reference adaptive control approach for uncertain piecewise affine systems with performance guarantees. The proposed approach ensures the error metric, defined as the weighted Euclidean norm of the state tracking error, to be confined within a user-defined time-varying performance bound. We introduce an auxiliary performance function to construct a barrier Lyapunov function. This auxiliary performance signal is reset at each switching instant, which prevents the transgression of the barriers caused by the jumps of the error metric at switching instants. The dwell time constraints are derived based on the parameters of the user-defined performance bound and the auxiliary performance function. We also prove that the Lyapunov function is non-increasing even at the switching instants and thus does not impose extra dwell time constraints. Furthermore, we propose the robust modification of the adaptive controller for the uncertain piecewise affine systems subject to unmatched disturbances. A Numerical example validates the correctness of the proposed approach.

eess.SY↗

arXiv2020

Learning a Low-dimensional Representation of a Safe Region for Safe Reinforcement Learning on Dynamical Systems

For safely applying reinforcement learning algorithms on high-dimensional nonlinear dynamical systems, a simplified system model is used to formulate a safe reinforcement learning framework. Based on the simplified system model, a low-dimensional representation of the safe region is identified and is used to provide safety estimates for learning algorithms. However, finding a satisfying simplified system model for complex dynamical systems usually requires a considerable amount of effort. To overcome this limitation, we propose in this work a general data-driven approach that is able to efficiently learn a low-dimensional representation of the safe region. Through an online adaptation method, the low-dimensional representation is updated by using the feedback data such that more accurate safety estimates are obtained. The performance of the proposed approach for identifying the low-dimensional representation of the safe region is demonstrated with a quadcopter example. The results show that, compared to previous work, a more reliable and representative low-dimensional representation of the safe region is derived, which then extends the applicability of the safe reinforcement learning framework.

cs.RO↗

arXiv2020

Distributed Link Removal Strategy for Networked Meta-Population Epidemics and its Application to the Control of the COVID-19 Pandemic

In this paper, we investigate the distributed link removal strategy for networked meta-population epidemics. In particular, a deterministic networked susceptible-infected-recovered (SIR) model is considered to describe the epidemic evolving process. In order to curb the spread of epidemics, we present the spectrum-based optimization problem involving the Perron-Frobenius eigenvalue of the matrix constructed by the network topology and transition rates. A modified distributed link removal strategy is developed such that it can be applied to the SIR model with heterogeneous transition rates on weighted digraphs. The proposed approach is implemented to control the COVID-19 pandemic by using the reported infected and recovered data in each state of Germany. The numerical experiment shows that the infected percentage can be significantly reduced by using the distributed link removal strategy.

physics.soc-ph↗

arXiv2020

Off-Policy Risk-Sensitive Reinforcement Learning Based Constrained Robust Optimal Control

This paper proposes an off-policy risk-sensitive reinforcement learning based control framework for stabilization of a continuous-time nonlinear system that subjects to additive disturbances, input saturation, and state constraints. By introducing pseudo controls and risk-sensitive input and state penalty terms, the constrained robust stabilization problem of the original system is converted into an equivalent optimal control problem of an auxiliary system. Then, aiming at the transformed optimal control problem, we adopt adaptive dynamic programming (ADP) implemented as a single critic structure to get the approximate solution to the value function of the Hamilton-Jacobi-Bellman (HJB) equation, which results in the approximate optimal control policy that is able to satisfy both input and state constraints under disturbances. By replaying experience data to the off-policy weight update law of the critic artificial neural network, the weight convergence is guaranteed. Moreover, to get experience data to achieve a sufficient excitation required for the weight convergence, online and offline algorithms are developed to serve as principled ways to record informative experience data. The equivalence proof demonstrates that the optimal control strategy of the auxiliary system robustly stabilizes the original system without violating input and state constraints. The proofs of system stability and weight convergence are provided. Simulation results reveal the validity of the proposed control framework.

eess.SY↗

arXiv2020

Interplay Between Homophily-Based Appraisal Dynamics and Influence-Based Opinion Dynamics: Modeling and Analysis

In social systems, the evolution of interpersonal appraisals and individual opinions are not independent processes but intertwine with each other. Despite extensive studies on both opinion dynamics and appraisal dynamics separately, no previous work has ever combined these two processes together. In this paper, we propose a novel and intuitive model on the interplay between homophily-based appraisal dynamics and influence-based opinion dynamics. We assume that individuals' opinions are updated via the influence network constructed from their interpersonal appraisals, which are in turn updated based on the individual opinions via the homophily mechanism. By theoretical analysis, we characterize the set of equilibria and some transient behavior of our model. Moreover, we establish the equivalence among the convergence of the appraisal network to social balance, the modulus consensus of individual opinions, and the non-vanishing appraisals. Monte Carlo validations further show that the non-vanishing appraisals condition holds for generic initial conditions. Compared with previous works that explain the emergence of social balance via person-to-person homophily mechanism, our model provides an alternative explanation in terms of the person-to-entity homophily mechanism. In addition, our model also describes how individuals' opinions on multiple irrelevant issues become correlated and converge to modulus consensus over time-varying influence networks.

eess.SY↗

arXiv2020

On the Stability of the Endemic Equilibrium of A Discrete-Time Networked Epidemic Model

Networked epidemic models have been widely adopted to describe propagation phenomena. The endemic equilibrium of these models is of great significance in the field of viral marketing, innovation dissemination, and information diffusion. However, its stability conditions have not been fully explored. In this paper we study the stability of the endemic equilibrium of a networked Susceptible-Infected-Susceptible (SIS) epidemic model with heterogeneous transition rates in a discrete-time manner. We show that the endemic equilibrium, if it exists, is asymptotically stable for any nontrivial initial condition. Under mild assumptions on initial conditions, we further prove that during the spreading process there exists no overshoot with respect to the endemic equilibrium. Finally, we conduct numerical experiments on real-world networks to demonstrate our results.

eess.SY↗