arXiv subjects

Cheng Zhou

Publications and source records attributed to Cheng Zhou.

At least 19 recordsLinked to original sources

arXiv2026

Cooperative-Competitive Team Play of Real-World Craft Robots

Multi-agent deep Reinforcement Learning (RL) has made significant progress in developing intelligent game-playing agents in recent years. However, the efficient training of collective robots using multi-agent RL and the transfer of learned policies to real-world applications remain open research questions. In this work, we first develop a comprehensive robotic system, including simulation, distributed learning framework, and physical robot components. We then propose and evaluate reinforcement learning techniques designed for efficient training of cooperative and competitive policies on this platform. To address the challenges of multi-agent sim-to-real transfer, we introduce Out of Distribution State Initialization (OODSI) to mitigate the impact of the sim-to-real gap. In the experiments, OODSI improves the Sim2Real performance by 20%. We demonstrate the effectiveness of our approach through experiments with a multi-robot car competitive game and a cooperative task in real-world settings.

cs.RO↗

arXiv2025

Unified Hierarchical MPC in Task Executing for Modular Manipulators across Diverse Morphologies

This work proposes a unified Hierarchical Model Predictive Control (H-MPC) for modular manipulators across various morphologies, as the controller can adapt to different configurations to execute the given task without extensive parameter tuning in the controller. The H-MPC divides the control process into two levels: a high-level MPC and a low-level MPC. The high-level MPC predicts future states and provides trajectory information, while the low-level MPC refines control actions by updating the predictive model based on this high-level information. This hierarchical structure allows for the integration of kinematic constraints and ensures smooth joint-space trajectories, even near singular configurations. Moreover, the low-level MPC incorporates secondary linearization by leveraging predictive information from the high-level MPC, effectively capturing the second-order Taylor expansion information of the kinematic model while still maintaining a linearized model formulation. This approach not only preserves the simplicity of a linear control model but also enhances the accuracy of the kinematic representation, thereby improving overall control precision and reliability. To validate the effectiveness of the control policy, we conduct extensive evaluations across different manipulator morphologies and demonstrate the execution of pick-and-place tasks in real-world scenarios.

cs.RO↗

arXiv2024

Whole-Body Impedance Coordinative Control of Wheel-Legged Robot on Uncertain Terrain

This article propose a whole-body impedance coordinative control framework for a wheel-legged humanoid robot to achieve adaptability on complex terrains while maintaining robot upper body stability. The framework contains a bi-level control strategy. The outer level is a variable damping impedance controller, which optimizes the damping parameters to ensure the stability of the upper body while holding an object. The inner level employs Whole-Body Control (WBC) optimization that integrates real-time terrain estimation based on wheel-foot position and force data. It generates motor torques while accounting for dynamic constraints, joint limits,friction cones, real-time terrain updates, and a model-free friction compensation strategy. The proposed whole-body coordinative control method has been tested on a recently developed quadruped humanoid robot. The results demonstrate that the proposed algorithm effectively controls the robot, maintaining upper body stability to successfully complete a water-carrying task while adapting to varying terrains.

cs.RO↗

arXiv2024

A Deconfounding Framework for Human Behavior Prediction: Enhancing Robotic Systems in Dynamic Environments

Accurate prediction of human behavior is crucial for effective human-robot interaction (HRI) systems, especially in dynamic environments where real-time decisions are essential. This paper addresses the challenge of forecasting future human behavior using multivariate time series data from wearable sensors, which capture various aspects of human movement. The presence of hidden confounding factors in this data often leads to biased predictions, limiting the reliability of traditional models. To overcome this, we propose a robust predictive model that integrates deconfounding techniques with advanced time series prediction methods, enhancing the model's ability to isolate true causal relationships and improve prediction accuracy. Evaluation on real-world datasets demonstrates that our approach significantly outperforms traditional methods, providing a more reliable foundation for responsive and adaptive HRI systems.

cs.RO↗

arXiv2024

Uncovering the Secrets of Human-Like Movement: A Fresh Perspective on Motion Planning

This article explores human-like movement from a fresh perspective on motion planning. We analyze the coordinated and compliant movement mechanisms of the human body from the perspective of biomechanics. Based on these mechanisms, we propose an optimal control framework that integrates compliant control dynamics, optimizing robotic arm motion through a response time matrix. This matrix sets the timing parameters for joint movements, turning the system into a time-parameterized optimal control problem. The model focuses on the interaction between active and passive joints under external disturbances, improving adaptability and compliance. This method achieves optimal trajectory generation and balances precision and compliance. Experimental results on both a manipulator and a humanoid robot validate the approach.

cs.RO↗

arXiv2024

A Fairness-Oriented Control Framework for Safety-Critical Multi-Robot Systems: Alternative Authority Control

This paper proposes a fair control framework for multi-robot systems, which integrates the newly introduced Alternative Authority Control (AAC) and Flexible Control Barrier Function (F-CBF). Control authority refers to a single robot which can plan its trajectory while considering others as moving obstacles, meaning the other robots do not have authority to plan their own paths. The AAC method dynamically distributes the control authority, enabling fair and coordinated movement across the system. This approach significantly improves computational efficiency, scalability, and robustness in complex environments. The proposed F-CBF extends traditional CBFs by incorporating obstacle shape, velocity, and orientation. F-CBF enhances safety by accurate dynamic obstacle avoidance. The framework is validated through simulations in multi-robot scenarios, demonstrating its safety, robustness and computational efficiency.

cs.RO↗

arXiv2024

Combined optimization ghost imaging based on random speckle field

Ghost imaging is a non local imaging technology, which can obtain target information by measuring the second-order intensity correlation between the reference light field and the target detection light field. However, the current imaging environment requires a large number of measurement data, and the imaging results also have the problems of low image resolution and long reconstruction time. Therefore, using orthogonal methods such as QR decomposition, a variety of optimization methods for speckle patterns are designed combined with Kronecker product,which can help to shorten the imaging time, improve the imaging quality and image noise resistance.

physics.optics↗

arXiv2024

Learning Highly Dynamic Behaviors for Quadrupedal Robots

Learning highly dynamic behaviors for robots has been a longstanding challenge. Traditional approaches have demonstrated robust locomotion, but the exhibited behaviors lack diversity and agility. They employ approximate models, which lead to compromises in performance. Data-driven approaches have been shown to reproduce agile behaviors of animals, but typically have not been able to learn highly dynamic behaviors. In this paper, we propose a learning-based approach to enable robots to learn highly dynamic behaviors from animal motion data. The learned controller is deployed on a quadrupedal robot and the results show that the controller is able to reproduce highly dynamic behaviors including sprinting, jumping and sharp turning. Various behaviors can be activated through human interaction using a stick with markers attached to it. Based on the motion pattern of the stick, the robot exhibits walking, running, sitting and jumping, much like the way humans interact with a pet.

cs.RO↗

arXiv2023

Lifelike Agility and Play in Quadrupedal Robots using Reinforcement Learning and Generative Pre-trained Models

Knowledge from animals and humans inspires robotic innovations. Numerous efforts have been made to achieve agile locomotion in quadrupedal robots through classical controllers or reinforcement learning approaches. These methods usually rely on physical models or handcrafted rewards to accurately describe the specific system, rather than on a generalized understanding like animals do. Here we propose a hierarchical framework to construct primitive-, environmental- and strategic-level knowledge that are all pre-trainable, reusable and enrichable for legged robots. The primitive module summarizes knowledge from animal motion data, where, inspired by large pre-trained models in language and image understanding, we introduce deep generative models to produce motor control signals stimulating legged robots to act like real animals. Then, we shape various traversing capabilities at a higher level to align with the environment by reusing the primitive module. Finally, a strategic module is trained focusing on complex downstream tasks by reusing the knowledge from previous levels. We apply the trained hierarchical controllers to the MAX robot, a quadrupedal robot developed in-house, to mimic animals, traverse complex obstacles and play in a designed challenging multi-agent chase tag game, where lifelike agility and strategy emerge in the robots.

cs.RO↗

arXiv2023

Learning Terrain-Adaptive Locomotion with Agile Behaviors by Imitating Animals

In this paper, we present a general learning framework for controlling a quadruped robot that can mimic the behavior of real animals and traverse challenging terrains. Our method consists of two steps: an imitation learning step to learn from motions of real animals, and a terrain adaptation step to enable generalization to unseen terrains. We capture motions from a Labrador on various terrains to facilitate terrain adaptive locomotion. Our experiments demonstrate that our policy can traverse various terrains and produce a natural-looking behavior. We deployed our method on the real quadruped robot Max via zero-shot simulation-to-reality transfer, achieving a speed of 1.1 m/s on stairs climbing.

cs.RO↗

arXiv2022

Relative Policy-Transition Optimization for Fast Policy Transfer

We consider the problem of policy transfer between two Markov Decision Processes (MDPs). We introduce a lemma based on existing theoretical results in reinforcement learning to measure the relativity gap between two arbitrary MDPs, that is the difference between any two cumulative expected returns defined on different policies and environment dynamics. Based on this lemma, we propose two new algorithms referred to as Relative Policy Optimization (RPO) and Relative Transition Optimization (RTO), which offer fast policy transfer and dynamics modelling, respectively. RPO transfers the policy evaluated in one environment to maximize the return in another, while RTO updates the parameterized dynamics model to reduce the gap between the dynamics of the two environments. Integrating the two algorithms results in the complete Relative Policy-Transition Optimization (RPTO) algorithm, in which the policy interacts with the two environments simultaneously, such that data collections from two environments, policy and transition updates are completed in one closed loop to form a principled learning framework for policy transfer. We demonstrate the effectiveness of RPTO on a set of MuJoCo continuous control tasks by creating policy transfer problems via variant dynamics.

cs.LG↗

arXiv2022

Online Motion Style Transfer for Interactive Character Control

Motion style transfer is highly desired for motion generation systems for gaming. Compared to its offline counterpart, the research on online motion style transfer under interactive control is limited. In this work, we propose an end-to-end neural network that can generate motions with different styles and transfer motion styles in real-time under user control. Our approach eliminates the use of handcrafted phase features, and could be easily trained and directly deployed in game systems. In the experiment part, we evaluate our approach from three aspects that are essential for industrial game design: accuracy, flexibility, and variety, and our model performs a satisfying result.

cs.GR↗

arXiv2021

Predicting antimicrobial activity of conjugated oligoelectrolyte molecules via machine learning

New antibiotics are needed to battle growing antibiotic resistance, but the development process from hit, to lead, and ultimately to a useful drug, takes decades. Although progress in molecular property prediction using machine-learning methods has opened up new pathways for aiding the antibiotics development process, many existing solutions rely on large datasets and finding structural similarities to existing antibiotics. Challenges remain in modelling of unconventional antibiotics classes that are drawing increasing research attention. In response, we developed an antimicrobial activity prediction model for conjugated oligoelectrolyte molecules, a new class of antibiotics that lacks extensive prior structure-activity relationship studies. Our approach enables us to predict minimum inhibitory concentration for E. coli K12, with 21 molecular descriptors selected by recursive elimination from a set of 5,305 descriptors. This predictive model achieves an R2 of 0.65 with no prior knowledge of the underlying mechanism. We find the molecular representation optimum for the domain is the key to good predictions of antimicrobial activity. In the case of conjugated oligoelectrolytes, a representation reflecting the 3-dimensional shape of the molecules is most critical. Although it is demonstrated with a specific example of conjugated oligoelectrolytes, our proposed approach for creating the predictive model can be readily adapted to other novel antibiotic candidate domains.

physics.app-ph↗

arXiv2020

Quantum Annealing for Semi-Supervised Learning

Recent advances in quantum technology have led to the development and the manufacturing of programmable quantum annealers that promise to solve certain combinatorial optimization problems faster than their classical counterparts. Semi-supervised learning is a machine learning technique that makes use of both labeled and unlabeled data for training, which enables a good classifier with only a small amount of labeled data. In this paper, we propose and theoretically analyze a graph-based semi-supervised learning method with the aid of the quantum annealing technique, which efficiently utilize the quantum resources while maintaining a good accuracy. We illustrate two classification examples, suggesting the feasibility of this method even with a small portion (20%) of labeled data is involved.

quant-ph↗

arXiv2019

Gold Matrix Ghost Imaging

Light field modulation matrix is closely related to the quality of reconstructed image in ghost imaging. The orthogonality light field modulation matrix with better noise immunity and high quality reconstructed image is urgently needed in the practical application of ghost imaging. In this work, we propose a Gold matrix ghost imaging method with excellent imaging and anti-noise performance, which is proud of the pseudo-randomness and orthogonality of the deterministic Gold matrix. Numerical simulation and experimental results show that the Gold matrix has the characteristics of both random speckle and Hadamard matrix, and the reconstructed images are superior to the two modulation matrixes under the condition of noise, which shows better imaging robustness.

eess.IV↗

arXiv2019

Hadamard `Pipeline' Coding Computational Ghost Imaging

The Hadamard matrix with orthogonality is a more important modulation matrix for computational ghost imaging (CGI), especially its optimized Hadamard matrix. However, as far as we know, little mention has been paid to efficient and convenient Hadamard matrix generation for CGI. The existing methods are to reconstruct any row of Hadamard matrix into two-dimensional matrix and then optimize it. In this work, we propose a Hadamard `pipeline' coding computational ghost imaging approach, which can directly generate two-dimensional Hadamard derived pattern and Hadamard optimization sequence, whereby both the memory consumption and the complexity of coding implementation for CGI can be significantly reduced. The optimization method of commonly used hadamard optimization sequence implementation is also discussed. This method provides a new approach for Hadamard sequence optimization and ghost imaging applications.

eess.IV↗

arXiv2019

Edge detection based on joint iteration ghost imaging

Imaging and edge detection have been widely applied and played an important role in security checking and medical diagnosis. However, as we know, most edge detection based on ghost imaging system require a large measurement times and the target object image cannot be provided directly. In this work, a new edge detection based on joint iteration of projected Landweber iteration regularization and guided filter ghost imaging method have been proposed which can be improved the feature detection quality in ghost imaging. This method can also achieve high quality imaging. Simulation and experiment results show that the spatial information and edge information of target object are successfully recovered from the random speckle patterns without special coding under a low measurement times, and the edge image quality is improved remarkably. This approach improves the the applicability of ghost imaging, and can satisfy the practical application fields of imaging and edge detection at the same time.

eess.IV↗

arXiv2019

An Action Recognition network for specific target based on rMC and RPN

The traditional methods of action recognition are not specific for the operator, thus results are easy to be disturbed when other actions are operated in videos. The network based on mixed convolutional resnet and RPN is proposed in this paper. The rMC is tested in the data set of UCF-101 to compare with the method of R3D. The result shows that its correct rate reaches 71.07%. Meanwhile, the action recognition network is tested in our gesture and body posture data sets for specific target. The simulation achieves a good performance in which the running speed reaches 200 FPS. Finally, our model is improved by introducing the regression block and performs better, which shows the great potential of this model.

cs.CV↗