arXiv subjects

Shivesh Kumar

Publications and source records attributed to Shivesh Kumar.

29 records · Page 2Linked to original sources

arXiv2023

Deriving Rewards for Reinforcement Learning from Symbolic Behaviour Descriptions of Bipedal Walking

Generating physical movement behaviours from their symbolic description is a long-standing challenge in artificial intelligence (AI) and robotics, requiring insights into numerical optimization methods as well as into formalizations from symbolic AI and reasoning. In this paper, a novel approach to finding a reward function from a symbolic description is proposed. The intended system behaviour is modelled as a hybrid automaton, which reduces the system state space to allow more efficient reinforcement learning. The approach is applied to bipedal walking, by modelling the walking robot as a hybrid automaton over state space orthants, and used with the compass walker to derive a reward that incentivizes following the hybrid automaton cycle. As a result, training times of reinforcement learning controllers are reduced while final walking speed is increased. The approach can serve as a blueprint how to generate reward functions from symbolic AI and reasoning.

cs.RO↗

arXiv2023

Towards Continuous Time Finite Horizon LQR Control in SE(3)

The control of free-floating robots requires dealing with several challenges. The motion of such robots evolves on a continuous manifold described by the Special Euclidean Group of dimension 3, known as SE(3). Methods from finite horizon Linear Quadratic Regulators (LQR) control have gained recent traction in the robotics community. However, such approaches are inherently solving an unconstrained optimization problem and hence are unable to respect the manifold constraints imposed by the group structure of SE(3). This may lead to small errors, singularity problems and double cover issues depending on the choice of coordinates to model the floating base motion. In this paper, we propose the use of canonical exponential coordinates of SE(3) and the associated Exponential map along with its differentials to embed this structure in the theory of finite horizon LQR controllers.

cs.RO↗

arXiv2023

AcroMonk: A Minimalist Underactuated Brachiating Robot

Brachiation is a dynamic, coordinated swinging maneuver of body and arms used by monkeys and apes to move between branches. As a unique underactuated mode of locomotion, it is interesting to study from a robotics perspective since it can broaden the deployment scenarios for humanoids and animaloids. While several brachiating robots of varying complexity have been proposed in the past, this paper presents the simplest possible prototype of a brachiation robot, using only a single actuator and unactuated grippers. The novel passive gripper design allows it to snap on and release from monkey bars, while guaranteeing well defined start and end poses of the swing. The brachiation behavior is realized in three different ways, using trajectory optimization via direct collocation and stabilization by a model-based time-varying linear quadratic regulator (TVLQR) or model-free proportional derivative (PD) control, as well as by a reinforcement learning (RL) based control policy. The three control schemes are compared in terms of robustness to disturbances, mass uncertainty, and energy consumption. The system design and controllers have been open-sourced. Due to its minimal and open design, the system can serve as a canonical underactuated platform for education and research.

cs.RO↗

arXiv2022

Analytic Estimation of Region of Attraction of an LQR Controller for Torque Limited Simple Pendulum

Linear-quadratic regulators (LQR) are a well known and widely used tool in control theory for both linear and nonlinear dynamics. For nonlinear problems, an LQR-based controller is usually only locally viable, thus, raising the problem of estimating the region of attraction (ROA). The need for good ROA estimations becomes especially pressing for underactuated systems, as a failure of controls might lead to unsafe and unrecoverable system states. Known approaches based on optimization or sampling, while working well, might be too slow in time critical applications and are hard to verify formally. In this work, we propose a novel approach to estimate the ROA based on the analytic solutions to linear ODEs for the torque limited simple pendulum. In simulation and physical experiments, we compared our approach to a Lyapunov-sampling baseline approach and found that our approach was faster to compute, while yielding ROA estimations of similar phase space area.

cs.RO↗

arXiv2022

An efficient combined local and global search strategy for optimization of parallel kinematic mechanisms with joint limits and collision constraints

The optimization of parallel kinematic manipulators (PKM) involve several constraints that are difficult to formalize, thus making optimal synthesis problem highly challenging. The presence of passive joint limits as well as the singularities and self-collisions lead to a complicated relation between the input and output parameters. In this article, a novel optimization methodology is proposed by combining a local search, Nelder-Mead algorithm, with global search methodologies such as low discrepancy distribution for faster and more efficient exploration of the optimization space. The effect of the dimension of the optimization problem and the different constraints are discussed to highlight the complexities of closed-loop kinematic chain optimization. The work also presents the approaches used to consider constraints for passive joint boundaries as well as singularities to avoid internal collisions in such mechanisms. The proposed algorithm can also optimize the length of the prismatic actuators and the constraints can be added in modular fashion, allowing to understand the impact of given criteria on the final result. The application of the presented approach is used to optimize two PKMs of different degrees of freedom.

cs.RO↗

arXiv2021

A Development Cycle for Automated Self-Exploration of Robot Behaviors

In this paper we introduce Q-Rock, a development cycle for the automated self-exploration and qualification of robot behaviors. With Q-Rock, we suggest a novel, integrative approach to automate robot development processes. Q-Rock combines several machine learning and reasoning techniques to deal with the increasing complexity in the design of robotic systems. The Q-Rock development cycle consists of three complementary processes: (1) automated exploration of capabilities that a given robotic hardware provides, (2) classification and semantic annotation of these capabilities to generate more complex behaviors, and (3) mapping between application requirements and available behaviors. These processes are based on a graph-based representation of a robot's structure, including hardware and software components. A central, scalable knowledge base enables collaboration of robot designers including mechanical, electrical and systems engineers, software developers and machine learning experts. In this paper we formalize Q-Rock's integrative development cycle and highlight its benefits with a proof-of-concept implementation and a use case demonstration.

cs.RO↗

arXiv2021

Nth Order Analytical Time Derivatives of Inverse Dynamics in Recursive and Closed Forms

Derivatives of equations of motion describing the rigid body dynamics are becoming increasingly relevant for the robotics community and find many applications in design and control of robotic systems. Controlling robots, and multibody systems comprising elastic components in particular, not only requires smooth trajectories but also the time derivatives of the control forces/torques, hence of the equations of motion (EOM). This paper presents novel nth order time derivatives of the EOM in both closed and recursive forms. While the former provides a direct insight into the structure of these derivatives,the latter leads to their highly efficient implementation for large degree of freedom robotic system.

cs.RO↗

arXiv2021

Design, analysis and control of the series-parallel hybrid RH5 humanoid robot

Last decades of humanoid research has shown that humanoids developed for high dynamic performance require a stiff structure and optimal distribution of mass--inertial properties. Humanoid robots built with a purely tree type architecture tend to be bulky and usually suffer from velocity and force/torque limitations. This paper presents a novel series-parallel hybrid humanoid called RH5 which is 2 m tall and weighs only 62.5 kg capable of performing heavy-duty dynamic tasks with 5 kg payloads in each hand. The analysis and control of this humanoid is performed with whole-body trajectory optimization technique based on differential dynamic programming (DDP). Additionally, we present an improved contact stability soft-constrained DDP algorithm which is able to generate physically consistent walking trajectories for the humanoid that can be tracked via a simple PD position control in a physics simulator. Finally, we showcase preliminary experimental results on the RH5 humanoid robot.

cs.RO↗

arXiv2020

A Comparative Study on 2-DOF Variable Stiffness Mechanisms

Based on the idea of variable stiffness mechanisms, a variety of such mechanisms is shown in this work. Specifically, 2-DOF parallel kinematic machines equipped with redundant actuators and non-linear springs in the actuated joints are presented and a comparative overview is drawn. Accordingly, a general stiffness formulation in task space of all mechanisms is given. Under fixed geometric parameters, optimization of task space stiffness is carried out on the designs comprising all kinematic solutions. Finally, a stiffness metric is introduced that allows a quantitative comparison of the given mechanism designs. This gives rise to design guidelines for engineers but also shows an interesting outline for future applications of variable stiffness mechanisms.

cs.RO↗

arXiv2020

Combinatorics of a Discrete Trajectory Space for Robot Motion Planning

Motion planning is a difficult problem in robot control. The complexity of the problem is directly related to the dimension of the robot's configuration space. While in many theoretical calculations and practical applications the configuration space is modeled as a continuous space, we present a discrete robot model based on the fundamental hardware specifications of a robot. Using lattice path methods, we provide estimates for the complexity of motion planning by counting the number of possible trajectories in a discrete robot configuration space.

cs.RO↗

arXiv2020

Comparison of Distal Teacher Learning with Numerical and Analytical Methods to Solve Inverse Kinematics for Rigid-Body Mechanisms

Several publications are concerned with learning inverse kinematics, however, their evaluation is often limited and none of the proposed methods is of practical relevance for rigid-body kinematics with a known forward model. We argue that for rigid-body kinematics one of the first proposed machine learning (ML) solutions to inverse kinematics -- distal teaching (DT) -- is actually good enough when combined with differentiable programming libraries and we provide an extensive evaluation and comparison to analytical and numerical solutions. In particular, we analyze solve rate, accuracy, sample efficiency and scalability. Further, we study how DT handles joint limits, singularities, unreachable poses, trajectories and provide a comparison of execution times. The three approaches are evaluated on three different rigid body mechanisms with varying complexity. With enough training data and relaxed precision requirements, DT has a better solve rate and is faster than state-of-the-art numerical solvers for a 15-DoF mechanism. DT is not affected by singularities while numerical solutions are vulnerable to them. In all other cases numerical solutions are usually better. Analytical solutions outperform the other approaches by far if they are available.

cs.RO↗