arXiv subjects

Emiland Garrabe

Publications and source records attributed to Emiland Garrabe.

4 recordsLinked to original sources

arXiv2024

Task-Aware Robotic Grasping by evaluating Quality Diversity Solutions through Foundation Models

Task-aware robotic grasping is a challenging problem that requires the integration of semantic understanding and geometric reasoning. This paper proposes a novel framework that leverages Large Language Models (LLMs) and Quality Diversity (QD) algorithms to enable zero-shot task-conditioned grasp synthesis. The framework segments objects into meaningful subparts and labels each subpart semantically, creating structured representations that can be used to prompt an LLM. By coupling semantic and geometric representations of an object's structure, the LLM's knowledge about tasks and which parts to grasp can be applied in the physical world. The QD-generated grasp archive provides a diverse set of grasps, allowing us to select the most suitable grasp based on the task. We evaluated the proposed method on a subset of the YCB dataset with a Franka Emika robot. A consolidated ground truth for task-specific grasp regions is established through a survey. Our work achieves a weighted intersection over union (IoU) of 73.6% in predicting task-conditioned grasp regions in 65 task-object combinations. An end-to-end validation study on a smaller subset further confirms the effectiveness of our approach, with 88% of responses favoring the task-aware grasp over the control group. A binomial test shows that participants significantly prefer the task-aware grasp.

cs.RO↗

arXiv2023

On Convex Data-Driven Inverse Optimal Control for Nonlinear, Non-stationary and Stochastic Systems

This paper is concerned with a finite-horizon inverse control problem, which has the goal of reconstructing, from observations, the possibly non-convex and non-stationary cost driving the actions of an agent. In this context, we present a result enabling cost reconstruction by solving an optimization problem that is convex even when the agent cost is not and when the underlying dynamics is nonlinear, non-stationary and stochastic. To obtain this result, we also study a finite-horizon forward control problem that has randomized policies as decision variables. We turn our findings into algorithmic procedures and show the effectiveness of our approach via in-silico and hardware validations. All experiments confirm the effectiveness of our approach.

math.OC↗

arXiv2023

Optimal Decision-Making for Autonomous Agents via Data Composition

We consider the problem of designing agents able to compute optimal decisions by composing data from multiple sources to tackle tasks involving: (i) tracking a desired behavior while minimizing an agent-specific cost; (ii) satisfying safety constraints. After formulating the control problem, we show that this is convex under a suitable assumption and find the optimal solution. The effectiveness of the results, which are turned in an algorithm, is illustrated on a connected cars application via in-silico and in-vivo experiments with real vehicles and drivers. All the experiments confirm our theoretical predictions and the deployment of the algorithm on a real vehicle shows its suitability for in-car operation.

math.OC↗

arXiv2022

Probabilistic design of optimal sequential decision-making algorithms in learning and control

This survey is focused on certain sequential decision-making problems that involve optimizing over probability functions. We discuss the relevance of these problems for learning and control. The survey is organized around a framework that combines a problem formulation and a set of resolution methods. The formulation consists of an infinite-dimensional optimization problem. The methods come from approaches to search optimal solutions in the space of probability functions. Through the lenses of this overarching framework we revisit popular learning and control algorithms, showing that these naturally arise from suitable variations on the formulation mixed with different resolution methods. A running example, for which we make the code available, complements the survey. Finally, a number of challenges arising from the survey are also outlined.

math.OC↗