arXiv subjects

Steven W. Su

Publications and source records attributed to Steven W. Su.

14 recordsLinked to original sources

arXiv2026

Dual-Agent Multiple-Model Reinforcement Learning for Event-Triggered Human-Robot Co-Adaptation in Decoupled Task Spaces

This paper presents a shared-control rehabilitation policy for a custom 6-degree-of-freedom (6-DoF) upper-limb robot that decomposes complex reaching tasks into decoupled spatial axes. The patient governs the primary reaching direction using binary commands, while the robot autonomously manages orthogonal corrective motions. Because traditional fixed-frequency control often induces trajectory oscillations due to variable inverse-kinematics execution times, an event-driven progression strategy is proposed. This architecture triggers subsequent control actions only when the end-effector enters an admission sphere centred on the immediate target waypoint, and was validated in a semi-virtual setup linking a physical pressure sensor to a MuJoCo simulation. To optimise human--robot co-adaptation safely and efficiently, this study introduces Dual Agent Multiple Model Reinforcement Learning (DAMMRL). This framework discretises decision characteristics: the human agent selects the admission sphere radius to reflect their inherent speed--accuracy trade-off, while the robot agent dynamically adjusts its 3D Cartesian step magnitudes to complement the user's cognitive state. Trained in simulation and deployed across mixed environments, this event-triggered DAMMRL approach effectively suppresses waypoint chatter, balances spatial precision with temporal efficiency, and significantly improves success rates in object acquisition tasks.

cs.RO↗

arXiv2026

Feasible Pairings for Decentralized Integral Controllability of Non-Square Systems

This paper investigates the determination of feasible input-output pairings for the decentralized integral controllability of non-square systems. The relevance of this problem extends beyond traditional industrial processes into modern AI research, particularly Multi-Agent Reinforcement Learning (MARL), where environments frequently act as strongly non-square mappings that evaluate high-dimensional joint action spaces via comparatively low-dimensional global rewards. To address the stability of these complex distributed architectures, we extend the concept of D-stability to non-square matrices, providing a crucial mathematical foundation. We formally define D-stability for non-square matrices as a direct generalization of the square case. By introducing the concept of ``Squared Matrices'', which are derived from specific column selections of the non-square formulation and directly correspond to candidate control pairings, we establish a fundamental link between the stability of these square sub-components and the original non-square system. Ultimately, we propose sufficient conditions under which the individual Volterra-Lyapunov stability of these squared components guarantees the extended D-stability of the non-square matrix, thereby providing a rigorous method to identify feasible pairings that ensure robust decentralized control across both classical and data-driven applications.

math.OC↗

arXiv2026

D-Optimality-Guided Reinforcement Learning for Efficient Open-Loop Calibration of a 3-DOF Ankle Rehabilitation Robot

Accurate alignment of multi-degree-of-freedom rehabilitation robots is essential for safe and effective patient training. This paper proposes a two-stage calibration framework for a self-designed three-degree-of-freedom (3-DOF) ankle rehabilitation robot. First, a Kronecker-product-based open-loop calibration method is developed to cast the input-output alignment into a linear parameter identification problem, which in turn defines the associated experimental design objective through the resulting information matrix. Building on this formulation, calibration posture selection is posed as a combinatorial design-of-experiments problem guided by a D-optimality criterion, i.e., selecting a small subset of postures that maximises the determinant of the information matrix. To enable practical selection under constraints, a Proximal Policy Optimization (PPO) agent is trained in simulation to choose 4 informative postures from a candidate set of 50. Across simulation and real-robot evaluations, the learned policy consistently yields substantially more informative posture combinations than random selection: the mean determinant of the information matrix achieved by PPO is reported to be more than two orders of magnitude higher with reduced variance. In addition, real-world results indicate that a parameter vector identified from only four D-optimality-guided postures provides stronger cross-episode prediction consistency than estimates obtained from a larger but unstructured set of 50 postures. The proposed framework therefore improves calibration efficiency while maintaining robust parameter estimation, offering practical guidance for high-precision alignment of multi-DOF rehabilitation robots.

cs.RO↗

arXiv2025

Human Machine Co-Adaptation Model and Its Convergence Analysis

The key to robot-assisted rehabilitation lies in the design of the human-machine interface, which must accommodate the needs of both patients and machines. Current interface designs primarily focus on machine control algorithms, often requiring patients to spend considerable time adapting. In this paper, we introduce a novel approach based on the Cooperative Adaptive Markov Decision Process (CAMDPs) model to address the fundamental aspects of the interactive learning process, offering theoretical insights and practical guidance. We establish sufficient conditions for the convergence of CAMDPs and ensure the uniqueness of Nash equilibrium points. Leveraging these conditions, we guarantee the system's convergence to a unique Nash equilibrium point. Furthermore, we explore scenarios with multiple Nash equilibrium points, devising strategies to adjust both Value Evaluation and Policy Improvement algorithms to enhance the likelihood of converging to the global minimal Nash equilibrium point. Through numerical experiments, we illustrate the effectiveness of the proposed conditions and algorithms, demonstrating their applicability and robustness in practical settings. The proposed conditions for convergence and the identification of a unique optimal Nash equilibrium contribute to the development of more effective adaptive systems for human users in robot-assisted rehabilitation.

cs.AI↗

arXiv2024

A Field Calibration Approach for Triaxial MEMS Gyroscopes Based on Gravity and Rotation Consistency

This paper developed an efficient method for calibrating triaxial MEMS gyroscopes, which can be effectively utilized in the field environment. The core strategy is to utilize the criterion that the dot product of the measured gravity and the rotation speed in a fixed frame remains constant. To eliminate the impact of external acceleration, the calibration process involves separate procedures for measuring local gravity and rotation speed. Moreover, unlike existing approaches for auto calibration of triaxial sensors that often result in nonlinear optimization problems, the proposed method simplifies the estimation of the gyroscope scale factor by employing a linear least squares algorithm. Extensive numerical simulations have been conducted to analyze the proposed method's performance in calibrating the six-parameter triaxial gyroscope model, taking into consideration measurements corrupted by simulated noise. Experimental validation was also carried out using two commercially available MEMS inertial measurement units (LSM9DS1) and a servo motor. The experimental results effectively demonstrate the efficacy of the proposed calibration approach.

cs.RO↗

arXiv2024

Enhancing EEG Signal Generation through a Hybrid Approach Integrating Reinforcement Learning and Diffusion Models

The present study introduces an innovative approach to the synthesis of Electroencephalogram (EEG) signals by integrating diffusion models with reinforcement learning. This integration addresses key challenges associated with traditional EEG data acquisition, including participant burden, privacy concerns, and the financial costs of obtaining high-fidelity clinical data. Our methodology enhances the generation of EEG signals with detailed temporal and spectral features, enriching the authenticity and diversity of synthetic datasets. The uniqueness of our approach lies in its capacity to concurrently model time-domain characteristics, such as waveform morphology, and frequency-domain features, including rhythmic brainwave patterns, within a cohesive generative framework. This is executed through the reinforcement learning model's autonomous selection of parameter update strategies, which steers the diffusion process to accurately reflect the complex dynamics inherent in EEG signals. We validate the efficacy of our approach using both the BCI Competition IV 2a dataset and a proprietary dataset, each collected under stringent experimental conditions. Our results indicate that the method preserves participant privacy by generating synthetic data that lacks biometric identifiers and concurrently improves the efficiency of model training by minimizing reliance on large annotated datasets. This research offers dual contributions: firstly, it advances EEG research by providing a novel tool for data augmentation and the advancement of machine learning algorithms; secondly, it enhances brain-computer interface technologies by offering a robust solution for training models on diverse and representative EEG datasets. Collectively, this study establishes a foundation for future investigations in neurological care and the development of tailored treatment protocols in neurorehabilitation.

eess.SP↗

arXiv2024

In-Field Gyroscope Autocalibration with Iterative Attitude Estimation

This paper presents an efficient in-field calibration method tailored for low-cost triaxial MEMS gyroscopes often used in healthcare applications. Traditional calibration techniques are challenging to implement in clinical settings due to the unavailability of high-precision equipment. Unlike the auto-calibration approaches used for triaxial MEMS accelerometers, which rely on local gravity, gyroscopes lack a reliable reference since the Earth's self-rotation speed is insufficient for accurate calibration. To address this limitation, we propose a novel method that uses manual rotation of the MEMS gyroscope to a specific angle (360°) as the calibration reference. This approach iteratively estimates the sensor's attitude without requiring any external equipment. Numerical simulations and empirical tests validate that the calibration error is low and that parameter estimation is unbiased. The method can be implemented in real-time on a low-energy microcontroller and completed in under 30 seconds. Comparative results demonstrate that the proposed technique outperforms existing state-of-the-art methods, achieving scale factor and bias errors of less than $2.5\times10^{-2}$ for LSM9DS1 and less than $1\times10^{-2}$ for ICM20948.

eess.SY↗

arXiv2024

Human-Machine Co-Adaptation for Robot-Assisted Rehabilitation via Dual-Agent Multiple Model Reinforcement Learning (DAMMRL)

This study introduces a novel approach to robot-assisted ankle rehabilitation by proposing a Dual-Agent Multiple Model Reinforcement Learning (DAMMRL) framework, leveraging multiple model adaptive control (MMAC) and co-adaptive control strategies. In robot-assisted rehabilitation, one of the key challenges is modelling human behaviour due to the complexity of human cognition and physiological systems. Traditional single-model approaches often fail to capture the dynamics of human-machine interactions. Our research employs a multiple model strategy, using simple sub-models to approximate complex human responses during rehabilitation tasks, tailored to varying levels of patient incapacity. The proposed system's versatility is demonstrated in real experiments and simulated environments. Feasibility and potential were evaluated with 13 healthy young subjects, yielding promising results that affirm the anticipated benefits of the approach. This study not only introduces a new paradigm for robot-assisted ankle rehabilitation but also opens the way for future research in adaptive, patient-centred therapeutic interventions.

cs.RO↗

arXiv2024

Sufficient D-Stability Conditions for Non-Square Matrices

This note explores the extension of D-stability to non-square matrices, applicable to distributed/decentralized controllability analysis. We first present a definition of D-stability for non-square matrices, directly extending from square matrices. We propose sufficient conditions for specific configurations of non-square matrices. Finally, we consider the selection of configurations to ensure the D-stability of a given non-square system.

math.OC↗

arXiv2024

EEG_RL-Net: Enhancing EEG MI Classification through Reinforcement Learning-Optimised Graph Neural Networks

Brain-Computer Interfaces (BCIs) rely on accurately decoding electroencephalography (EEG) motor imagery (MI) signals for effective device control. Graph Neural Networks (GNNs) outperform Convolutional Neural Networks (CNNs) in this regard, by leveraging the spatial relationships between EEG electrodes through adjacency matrices. The EEG_GLT-Net framework, featuring the state-of-the-art EEG_GLT adjacency matrix method, has notably enhanced EEG MI signal classification, evidenced by an average accuracy of 83.95% across 20 subjects on the PhysioNet dataset. This significantly exceeds the 76.10% accuracy rate achieved using the Pearson Correlation Coefficient (PCC) method within the same framework. In this research, we advance the field by applying a Reinforcement Learning (RL) approach to the classification of EEG MI signals. Our innovative method empowers the RL agent, enabling not only the classification of EEG MI data points with higher accuracy, but effective identification of EEG MI data points that are less distinct. We present the EEG_RL-Net, an enhancement of the EEG_GLT-Net framework, which incorporates the trained EEG GCN Block from EEG_GLT-Net at an adjacency matrix density of 13.39% alongside the RL-centric Dueling Deep Q Network (Dueling DQN) block. The EEG_RL-Net model showcases exceptional classification performance, achieving an unprecedented average accuracy of 96.40% across 20 subjects within 25 milliseconds. This model illustrates the transformative effect of the RL in EEG MI time point classification.

eess.SP↗

arXiv2024

EEG_GLT-Net: Optimising EEG Graphs for Real-time Motor Imagery Signals Classification

Brain-Computer Interfaces connect the brain to external control devices, necessitating the accurate translation of brain signals such as from electroencephalography (EEG) into executable commands. Graph Neural Networks (GCN) have been increasingly applied for classifying EEG Motor Imagery signals, primarily because they incorporates the spatial relationships among EEG channels, resulting in improved accuracy over traditional convolutional methods. Recent advances by GCNs-Net in real-time EEG MI signal classification utilised Pearson Coefficient Correlation (PCC) for constructing adjacency matrices, yielding significant results on the PhysioNet dataset. Our paper introduces the EEG Graph Lottery Ticket (EEG_GLT) algorithm, an innovative technique for constructing adjacency matrices for EEG channels. It does not require pre-existing knowledge of inter-channel relationships, and it can be tailored to suit both individual subjects and GCN model architectures. Our findings demonstrated that the PCC method outperformed the Geodesic approach by 9.65% in mean accuracy, while our EEG_GLT matrix consistently exceeded the performance of the PCC method by a mean accuracy of 13.39%. Also, we found that the construction of the adjacency matrix significantly influenced accuracy, to a greater extent than GCN model configurations. A basic GCN configuration utilising our EEG_GLT matrix exceeded the performance of even the most complex GCN setup with a PCC matrix in average accuracy. Our EEG_GLT method also reduced MACs by up to 97% compared to the PCC method, while maintaining or enhancing accuracy. In conclusion, the EEG_GLT algorithm marks a breakthrough in the development of optimal adjacency matrices, effectively boosting both computational accuracy and efficiency, making it well-suited for real-time classification of EEG MI signals that demand intensive computational resources.

cs.LG↗

arXiv2023

Special Stable Matrices and Their Non-square Counterpart

In this note, we discuss the extension of several important stable square matrices, e.g., D-stable matrices, diagonal dominance matrices, Volterra-Lyapunov stable matrices, to their corresponding non-square matrices. The extension is motivated by some distributed control-related problems, such as decentralized unconditional stability and decentralized integral controllability for non-square processes. We will provide the connections of conditions between these special square matrices and their associated non-square counterparts. Some conjectures for these special matrices are proposed for future research.

math.OC↗

arXiv2023

Human Machine Co-adaption Interface via Cooperation Markov Decision Process System

This paper aims to develop a new human-machine interface to improve rehabilitation performance from the perspective of both the user (patient) and the machine (robot) by introducing the co-adaption techniques via model-based reinforcement learning. Previous studies focus more on robot assistance, i.e., to improve the control strategy so as to fulfill the objective of Assist-As-Needed. In this study, we treat the full process of robot-assisted rehabilitation as a co-adaptive or mutual learning process and emphasize the adaptation of the user to the machine. To this end, we proposed a Co-adaptive MDPs (CaMDPs) model to quantify the learning rates based on cooperative multi-agent reinforcement learning (MARL) in the high abstraction layer of the systems. We proposed several approaches to cooperatively adjust the Policy Improvement among the two agents in the framework of Policy Iteration. Based on the proposed co-adaptive MDPs, the simulation study indicates the non-stationary problem can be mitigated using various proposed Policy Improvement approaches.

cs.AI↗

arXiv2018

Stability Analysis for Switched Systems with Sequence-based Average Dwell Time

This note investigates the stability of both linear and nonlinear switched systems with average dwell time. Two new analysis methods are proposed. Different from existing approaches, the proposed methods take into account the sequence in which the subsystems are switched. Depending on the predecessor or successor subsystems to be considered, sequence-based average preceding dwell time (SBAPDT) and sequence-based average subsequence dwell time (SBASDT) approaches are proposed and discussed for both continuous and discrete time systems. These proposed methods, when considering the switch sequence, have the potential to further reduce the conservativeness of the existing approaches. A comparative numerical example is also given to demonstrate the advantages of the proposed approaches.

eess.SY↗