arXiv subjects

Kevin Haninger

Publications and source records attributed to Kevin Haninger.

At least 19 recordsLinked to original sources

arXiv2026

Human Sensorimotor Model Adaptation to Account for Robot Accuracy and Speed Augmentation in Human-Robot Co-Manipulation

In human-robot co-manipulation, the robot's trajectory should accommodate the human's, where human sensorimotor behavior is affected by the task's spatial and temporal constraints, as well as personal factors. Several robot trajectory planning methods have been proposed to embed human sensorimotor models in the robot trajectory planning, but these methods do not consider temporal constraints that adapt to how the human behaves when interacting with the robot. To better align human and robot collaborative motion, this work adapts a stochastic optimal control problem with a final state constraint from the task constraints and a temporal constraint from Fitts' law. A recursive least squares method is used to estimate the Fitts' law parameters, adapting the robot's motion duration to that of the human, thus allowing the robot to adapt to how the human behaves when interacting with the robot. It is hypothesized that a speed increase can result from the robot augmenting the accuracy. The feasibility of the approach is validated on a positioning task where the robot controls degrees of freedom which have repeatable motion, and the human adapts motion in the complementary degrees of freedom. The approach is shown to be computationally suited to online use, it converges as the human and robot mutually adapt, and can increase the movement speed, although thorough human studies are needed in future work.

cs.RO↗

arXiv2026

Towards Neuro-Symbolic Procedural Reasoning for Long-Horizon Vision-Language-Action Manipulation

Vision-language-action (VLA) models can execute short manipulation skills, but remain brittle in long-horizon procedures requiring persistent task state, dependency-aware reasoning, conditional decisions, and reliable grounding. We investigate a neuro-symbolic framework that combines learned VLA control with explicit task graphs and multimodal procedural memory. Task graphs encode action dependencies, valid transitions, and branch conditions, while memory maintains the active step, completed actions, textual context, and task-relevant visual evidence. Together, these structures guide object selection, destination grounding, subgoal dispatch, and verification of expected state transitions. Human demonstrations provide additional spatial and temporal guidance through gaze or saliency cues. To isolate their effect on policy learning, our initial study bypasses cross-view gaze transfer and directly annotates pseudo-gaze in robot-view teleoperation videos. The resulting guidance is used during VLA fine-tuning and inference. We study two long-horizon manipulation domains, workspace clearing and surgical-instrument handling, which require ordered execution, visually grounded decisions, and conditional branching. We evaluate correct-object and destination selection, subtask completion, task progress, step-order consistency, complete-task success, and procedural or execution mistakes. This work positions structured symbolic reasoning and demonstration-derived visual guidance as complementary mechanisms for reliable long-horizon VLA manipulation.

cs.RO↗

arXiv2026

What Matters, When? Diagnosing and Improving Conditional Visual Grounding in Visuomotor Imitation Policies

Visuomotor imitation policies can achieve high performance under in-distribution visual conditions yet fail when visually similar objects or receptacles are introduced. We study this behavior as a problem of conditional visual grounding: the visual target required for successful control changes with the manipulation phase and, in more complex tasks, with the observed task state. Using Action Chunking with Transformers (ACT), we systematically introduce distractor objects and receptacles with controlled color and shape similarity and localize failures to picking and placement. We find that distractor sensitivity is specific to both the type of visual similarity and the manipulation stage. Guided by this diagnosis, we evaluate distractor augmentation, phase-dependent attention regularization, and appearance-based visual prompting as complementary interventions for improving target selection while preserving spatial information required for control. These interventions substantially improve robustness in simulation and on a physical UR3e. We further examine the same failure pattern in a pretrained vision-language-action policy on a state-conditioned instrument-handling task, where the observed state of a medical instrument determines the correct destination. Together, the results show that visual distractors can cause incorrect object or destination selection even when the underlying manipulation skill remains intact, and that explicitly improving target selection can substantially recover performance across distinct visuomotor policy-learning regimes.

cs.RO↗

arXiv2026

Anomaly detection for generic failure monitoring in robotic assembly, screwing and manipulation

Out-of-distribution states in robot manipulation often lead to unpredictable robot behavior or task failure, limiting success rates and increasing risk of damage. Anomaly detection (AD) can identify deviations from expected patterns in data, which can be used to trigger failsafe behaviors and recovery strategies. Prior work has applied data-driven AD on time series data for specific robotic tasks, however the transferability of an AD approach between different robot control strategies and task types has not been shown. Leveraging time series data, such as force/torque signals, allows to directly capture robot-environment interactions, crucial for manipulation and online failure detection. As robotic tasks can have widely signal characteristics and requirements, AD methods which can be applied in the same way to a wide range of tasks is needed, ideally with good data efficiency. We examine three industrial robotic tasks, robotic cabling, screwing, and sanding, each with multi-modal time series data and several anomalies. Several autoencoderbased methods are compared, and we evaluate the generalization across different robotic tasks and control methods (diffusion policy-, position-, and impedance-controlled). This allows us to validate the integration of AD in complex tasks involving tighter tolerances and variation from both the robot and its environment. Additionally, we evaluate data efficiency, detection latency, and task characteristics which support robust detection. The results indicate reliable detection with AUROC exceeding 0.96 in failures in the cabling and screwing task, such as incorrect or misaligned parts and obstructed targets. In the polishing task, only severe failures were reliably detected, while more subtle failure types remained undetected.

cs.RO↗

arXiv2025

Towards simulation-based optimization of compliant fingers for high-speed connector assembly

Mechanical compliance is a key design parameter for dynamic contact-rich manipulation, affecting task success and safety robustness over contact geometry variation. Design of soft robotic structures, such as compliant fingers, requires choosing design parameters which affect geometry and stiffness, and therefore manipulation performance and robustness. Today, these parameters are chosen through either hardware iteration, which takes significant development time, or simplified models (e.g. planar), which can't address complex manipulation task objectives. Improvements in dynamic simulation, especially with contact and friction modeling, present a potential design tool for mechanical compliance. We propose a simulation-based design tool for compliant mechanisms which allows design with respect to task-level objectives, such as success rate. This is applied to optimize design parameters of a structured compliant finger to reduce failure cases inside a tolerance window in insertion tasks. The improvement in robustness is then validated on a real robot using tasks from the benchmark NIST task board. The finger stiffness affects the tolerance window: optimized parameters can increase tolerable ranges by a factor of 2.29, with workpiece variation up to 8.6 mm being compensated. However, the trends remain task-specific. In some tasks, the highest stiffness yields the widest tolerable range, whereas in others the opposite is observed, motivating need for design tools which can consider application-specific geometry and dynamics.

cs.RO↗

arXiv2024

Planning Human-Robot Co-manipulation with Human Motor Control Objectives and Multi-component Reaching Strategies

For successful goal-directed human-robot interaction, the robot should adapt to the intentions and actions of the collaborating human. This can be supported by musculoskeletal or data-driven human models, where the former are limited to lower-level functioning such as ergonomics, and the latter have limited generalizability or data efficiency. What is missing, is the inclusion of human motor control models that can provide generalizable human behavior estimates and integrate into robot planning methods. We use well-studied models from human motor control based on the speed-accuracy and cost-benefit trade-offs to plan collaborative robot motions. In these models, the human trajectory minimizes an objective function, a formulation we adapt to numerical trajectory optimization. This can then be extended with constraints and new variables to realize collaborative motion planning and goal estimation. We deploy this model, as well as a multi-component movement strategy, in physical collaboration with uncertain goal-reaching and synchronized motion tasks, showing the ability of the approach to produce human-like trajectories over a range of conditions.

cs.RO↗

arXiv2024

Energy Consumption in Robotics: A Simplified Modeling Approach

The energy use of a robot is trajectory-dependent, and thus can be reduced by optimization of the trajectory. Current methods for robot trajectory optimization can reduce energy up to 15\% for fixed start and end points, however their use in industrial robot planning is still restricted due to model complexity and lack of integration with planning tools which address other concerns (e.g. collision avoidance). We propose an approach that uses differentiable inertial and kinematic models from standard open-source tools, integrating with standard ROS planning methods. An inverse dynamics-based energy model is optionally extended with a single-parameter electrical model, simplifying the model identification process. We compare the inertial and electrical models on a collaborative robot, showing that simplified models provide competitive accuracy and are easier to deploy in practice.

cs.RO↗

arXiv2024

Soft Finger Grasp Force and Contact State Estimation from Tactile Sensors

Soft robotic fingers can improve adaptability in grasping and manipulation, compensating for geometric variation in object or environmental contact, but today lack force capacity and fine dexterity. Integrated tactile sensors can provide grasp and task information which can improve dexterity,but should ideally not require object-specific training. The total force vector exerted by a finger provides general information to the internal grasp forces (e.g. for grasp stability) and, when summed over fingers, an estimate of the external force acting on the grasped object (e.g. for task-level control). In this study, we investigate the efficacy of estimating finger force from integrated soft sensors and use it to estimate contact states. We use a neural network for force regression, collecting labelled data with a force/torque sensor and a range of test objects. Subsequently, we apply this model in a plug-in task scenario and demonstrate its validity in estimating contact states.

cs.RO↗

arXiv2024

Industrial Cabling in Constrained Environments: a Practical Approach and Current Challenges

Cabling tasks (pulling, clipping, and plug insertion) are today mostly manual work, limiting the cost-effectiveness of electrification. Feasibility for the robotic grasping and insertion of plugs, as well as the manipulation of cables, have been shown in research settings. However, in many industrial tasks the complete process from picking, insertion, routing, and validation must be solved with one system. This often means the cable must be directly manipulated for routing, and the plug must be manipulated for insertion, often in cluttered environments with tight space constraints. Here we introduce an analysis of the complete industrial cabling tasks and demonstrate a solution from grasp, plug insertion, clipping, and final plug insertion. Industrial requirements are summarized, considering the space limitations, tolerances, and possible ways that the cabling process can be integrated into the production process. This paper proposes gripper designs and general robotic assembly methods for the widely used FASTON and a cubical industrial connector. The proposed methods cover the cable gripping, handling, routing, and inserting processes of the connector. Customized grippers are designed to ensure the reliable gripping of the plugs and the pulling and manipulation of the cable segments. A passive component to correct the cable orientation is proposed, allowing the robot to re-grip the plug before insertion. In general, the proposed method can perform cable assembly with mere position control, foregoing complex control approaches. This solution is demonstrated with an industrial product with realistic space requirements and tolerances, identifying difficult aspects of current cabling scenarios and potential to improve the automation-friendliness in the product design.

cs.RO↗

arXiv2024

Combining Sampling- and Gradient-based Planning for Contact-rich Manipulation

Planning over discontinuous dynamics is needed for robotics tasks like contact-rich manipulation, which presents challenges in the numerical stability and speed of planning methods when either neural network or analytical models are used. On the one hand, sampling-based planners require higher sample complexity in high-dimensional problems and cannot describe safety constraints such as force limits. On the other hand, gradient-based solvers can suffer from local optima and convergence issues when the Hessian is poorly conditioned. We propose a planning method with both sampling- and gradient-based elements, using the Cross-entropy Method to initialize a gradient-based solver, providing better search over local minima and the ability to handle explicit constraints. We show the approach allows smooth, stable contact-rich planning for an impedance-controlled robot making contact with a stiff environment, benchmarking against gradient-only MPC and CEM.

cs.RO↗

arXiv2024

Soft finger rotational stability for precision grasps

Soft robotic fingers can safely grasp fragile or variable form objects, but their force capacity is limited, especially with less contact area: precision grasps and when objects are smaller or not spherical. Current research is improving force capacity through mechanical design by increasing contact area or stiffness, typically without models which explain soft finger force limitations. To address this, this paper considers two types of soft grip failure, slip and dynamic rotational stability. For slip, the validity of a Coulomb model investigated, identifying the effect of contact area, pressure, and relative pose. For rotational stability, bulk linear stiffness of the fingers is used to develop conditions for dynamic stability and identify when rotation leads to slip. Together, these models suggest contact area improves force capacity by increasing transverse stiffness and normal force. The models are validated on pneumatic fingers, both custom PneuNets-based and commercially available. The models are used to find grip parameters which increase force capacity without failure.

cs.RO↗

arXiv2023

Differentiable Compliant Contact Primitives for Estimation and Model Predictive Control

Control techniques like MPC can realize contact-rich manipulation which exploits dynamic information, maintaining friction limits and safety constraints. However, contact geometry and dynamics are required to be known. This information is often extracted from CAD, limiting scalability and the ability to handle tasks with varying geometry. To reduce the need for a priori models, we propose a framework for estimating contact models online based on torque and position measurements. To do this, compliant contact models are used, connected in parallel to model multi-point contact and constraints such as a hinge. They are parameterized to be differentiable with respect to all of their parameters (rest position, stiffness, contact location), allowing the coupled robot/environment dynamics to be linearized or efficiently used in gradient-based optimization. These models are then applied for: offline gradient-based parameter fitting, online estimation via an extended Kalman filter, and online gradient-based MPC. The proposed approach is validated on two robots, showing the efficacy of sensorless contact estimation and the effects of online estimation on MPC performance.

cs.RO↗

arXiv2023

High-speed electrical connector assembly by structured compliance in a finray-effect gripper

Fine assembly tasks such as electrical connector insertion have tight tolerances and sensitive components, requiring compensation of alignment errors while applying sufficient force in the insertion direction, ideally at high speeds and while grasping a range of components. Vision, tactile, or force sensors can compensate alignment errors, but have limited bandwidth, limiting the safe assembly speed. Passive compliance such as silicone-based fingers can reduce collision forces and grasp a range of components, but often cannot provide the accuracy or assembly forces required. To support high-speed mechanical search and self-aligning insertion, this paper proposes monolithic additively manufactured fingers which realize a moderate, structured compliance directly proximal to the gripped object. The geometry of finray-effect fingers are adapted to add form-closure features and realize a directionally-dependent stiffness at the fingertip, with a high stiffness to apply insertion forces and lower transverse stiffness to support alignment. Design parameters and mechanical properties of the fingers are investigated with FEM and empirical studies, analyzing the stiffness, maximum load, and viscoelastic effects. The fingers realize a remote center of compliance, which is shown to depend on the rib angle, and a directional stiffness ratio of $14-36$. The fingers are applied to a plug insertion task, realizing a tolerance window of $7.5$ mm and approach speeds of $1.3$ m/s.

cs.RO↗

arXiv2023

Teaching contact-rich tasks from visual demonstrations by constraint extraction

Contact-rich manipulation involves kinematic constraints on the task motion, typically with discrete transitions between these constraints during the task. Allowing the robot to detect and reason about these contact constraints can support robust and dynamic manipulation, but how can these contact models be efficiently learned? Purely visual observations are an attractive data source, allowing passive task demonstrations with unmodified objects. Existing approaches for vision-only learning from demonstration are effective in pick-and-place applications and planar tasks. Nevertheless, accuracy/occlusions and unobserved task dynamics can limit their robustness in contact-rich manipulation. To use visual demonstrations for contact-rich robotic tasks, we consider the demonstration of pose trajectories with transitions between holonomic kinematic constraints, first clustering the trajectories into discrete contact modes, then fitting kinematic constraints per each mode. The fit constraints are then used to (i) detect contact online with force/torque measurements and (ii) plan the robot policy with respect to the active constraint. We demonstrate the approach with real experiments, on cabling and rake tasks, showing the approach gives robust manipulation through contact transitions.

cs.RO↗

arXiv2023

Compliant finray-effect gripper for high-speed robotic assembly of electrical components

Fine assembly tasks such as electrical connector insertion have tight tolerances and sensitive components, limiting the speed and robustness of robot assembly, even when using vision, tactile, or force sensors. Connector insertion is a common industrial task, requiring horizontal alignment errors to be compensated with minimal force, then sufficient force to be brought in the insertion direction. The ability to handle a variety of objects, achieve high-speeds, and handle a wide range in object position variation are also desired. Soft grippers can allow the gripping of parts with variation in surface geometry, but often focus on gripping alone and may not be able to bring the assembly forces required. To achieve high-speed connector insertion, this paper proposes monolithic fingers with structured compliance and form-closure features. A finray-effect gripper is adapted to realize structured (i.e. directional) stiffness that allows high-speed mechanical search, self-alignment in insertion, and sufficient assembly force. The design of the finray ribs and fingertips are investigated, with a final design allowing plug insertion with a tolerance window of up to 7.5 mm at high speed.

cs.RO↗

arXiv2022

Model Predictive Impedance Control with Gaussian Processes for Human and Environment Interaction

Robotic tasks which involve uncertainty--due to variation in goal, environment configuration, or confidence in task model--may require human input to instruct or adapt the robot. In tasks with physical contact, several existing methods for adapting robot trajectory or impedance according to individual uncertainties have been proposed, e.g., realizing intention detection or uncertainty-aware learning from demonstration. However, isolated methods cannot address the wide range of uncertainties jointly present in many tasks. To improve generality, this paper proposes a model predictive control (MPC) framework which plans both trajectory and impedance online, can consider discrete and continuous uncertainties, includes safety constraints, and can be efficiently applied to a new task. This framework can consider uncertainty from: contact constraint variation, uncertainty in human goals, or task disturbances. An uncertainty-aware task model is learned from a few ($\leq3$) demonstrations using Gaussian Processes. This task model is used in a nonlinear MPC problem to optimize robot trajectory and impedance according to belief in discrete human goals, human kinematics, safety constraints, contact stability, and frequency-domain disturbance rejection. This MPC formulation is introduced, analyzed with respect to convexity, and validated in co-manipulation with multiple goals, a collaborative polishing task, and a collaborative assembly task.

cs.RO↗

arXiv2022

Impedance Adaptation by Reinforcement Learning with Contact Dynamic Movement Primitives

Dynamic movement primitives (DMPs) allow complex position trajectories to be efficiently demonstrated to a robot. In contact-rich tasks, where position trajectories alone may not be safe or robust over variation in contact geometry, DMPs have been extended to include force trajectories. However, different task phases or degrees of freedom may require the tracking of either position or force -- e.g., once contact is made, it may be more important to track the force demonstration trajectory in the contact direction. The robot impedance balances between following a position or force reference trajectory, where a high stiffness tracks position and a low stiffness tracks force. This paper proposes using DMPs to learn position and force trajectories from demonstrations, then adapting the impedance parameters online with a higher-level control policy trained by reinforcement learning. This allows one-shot demonstration of the task with DMPs, and improved robustness and performance from the impedance adaptation. The approach is validated on peg-in-hole and adhesive strip application tasks.

cs.RO↗

arXiv2022

Contact Information Flow and Design of Compliance

Identifying changes in contact during contact-rich manipulation can detect task state or errors, enabling improved robustness and autonomy. The ability to detect contact is affected by the mechatronic design of the robot, especially its physical compliance. Established methods can design physical compliance for many aspects of contact performance (e.g. peak contact force, motion/force control bandwidth), but are based on time-invariant dynamic models. A change in contact mode is a discrete change in coupled robot-environment dynamics, not easily considered in existing design methods. Towards designing robots which can robustly detect changes in contact mode online, this paper investigates how mechatronic design can improve contact estimation, with a focus on the impact of the location and degree of compliance. A design metric of information gain is proposed which measures how much position/force measurements reduce uncertainty in the contact mode estimate. This information gain is developed for fully- and partially-observed systems, as partial observability can arise from joint flexibility in the robot or environmental inertia. Hardware experiments with various compliant setups validate that information gain predicts the speed and certainty with which contact is detected in (i) monitoring of contact-rich assembly and (ii) collision detection.

cs.RO↗