Searcharxiv⌕ Search

arXiv subjects

Tom Lefebvre

Publications and source records attributed to Tom Lefebvre.

At least 19 recordsLinked to original sources

Costate-Conditioned Dynamic Programming for Parallel-in-Time Solution of Equality-Constrained Optimal Control Problems

Gradient based trajectory optimisation algorithms spend most of their per-iteration time on a structured Newton--KKT system. The system corresponds to an equality-constrained linear-quadratic optimal control problem and dedicated Riccati-type solvers exhibit linear complexity in the horizon length. Parallel-in-time algorithms exhibit the potential to reduce the complexity to logarithmic in the horizon by distributing several subtrajectories over multiple cores and then stitch together the results by means of an associative scan. Existing exact formulations condition each subtrajectory on its terminal state. Each subproblem then carries the reachability to that terminal state explicitly as constraint rows. The associated reachability decision however is ill-conditioned for weakly actuated systems. In this work we condition each subtrajectory on the terminal costate instead. The resulting subproblems can now be computed by a free-end Riccati recursion without reachability carrying rows. This approach effectively compensates for the ill-conditioning of weakly actuated systems but requires additional bookkeeping with Lagrangian multipliers when stitching together the various subproblems. Numerically the method reproduces a direct solution of the primal-dual Newton-KKT system to $10^{-12}$ in relative terms, on a weakly controllable chain of masses. The accurracy rivals that of a sequential recursion, whereas terminal-state conditioning loses up to six digits or fails. Operation counts on a problem with 64 subtrajectories and a horizon of 4096 stages gives a speed-up of 26 compared with a sequential solve.

math.OC↗

Aeroelastically Constrained Optimal Path Planning of a Fixed-Wing Airborne Wind Energy System

Multi-megawatt fixed-wing airborne wind energy systems (AWES) use an aircraft with large, slender wings that deform over the pumping cycle. To harvest maximum energy, the cycle is usually designed by means of dedicated reference trajectory optimization routines. It is well known that such routines rely heavily on the system model and that close alignment between the model and the physical system is required for the results to hold in practice. Nevertheless, the wing is usually treated as rigid and structural limits are respected indirectly through upper bounding of states and inputs. In this work, we directly enforce bounds on the elastic deflection and twist of the wing in the power-maximizing periodic optimal control problem. To that end, we model the wing by root-clamped modes, loaded by strip-theory aerodynamics and by the gravitational and inertial loads of the manoeuvring aircraft. To keep the problem tractable, the modal dynamics are replaced by their quasi-static equilibrium, which enters every discretization interval as algebraic constraints. For a multi-megawatt reference aircraft, the elastic twist near the wing tips reaches $1.03^\circ$ along the optimal rigid-wing trajectory. With a twist limit of $0.6^\circ$, the optimal trajectory delivers $31\,\%$ less mean power than the rigid-wing one, mainly through a lower tether tension during reel-out. Rigid-wing trajectory optimization can thus overestimate the power that the wing limits permit. Forward simulations that retain the modal dynamics demonstrate that the quasi-static deflection and twist deviate from the dynamic ones by less than $2\,\%$ of their limits. This work shows that elastic deformation can be accounted for in the trajectory optimization and that, depending on the wing limits, it can strongly reduce the achievable power.

eess.SY↗

KL-Regularized Policy Iteration for Adaptive LQR: Gain Updates and Probing from One Policy Improvement

Policy-gradient methods for adaptive linear quadratic control improve the feedback gain of an unknown plant online by gradient-type steps computed from closed-loop data. A recent framework of such methods, policy-gradient adaptive control (PGAC), takes one gradient-type step per sample, which its indirect form computes from an estimated model and its direct form directly from the data. Because data generated by a fixed linear feedback cannot identify the plant, PGAC adds a probing perturbation to the input and leaves its covariance as a design parameter. We study the adaptive LQR problem through a KL-regularized formulation, in which the probing is the randomness of a Gaussian policy kept close to a Gaussian reference policy, so that the cost of the probing is part of the objective. One policy improvement of this formulation gives a gain update together with a probing covariance. We show that, for suitable references centred at the current gain, this gain update is exactly PGAC's Gauss-Newton or natural-gradient update with any step size below a bound. We repeat this policy improvement, each time against a tempered copy of the current policy, and refer to this scheme as KL-regularized policy iteration (KLPI). On a known plant and with tempering, the cost of KLPI's gains converges linearly to the optimal cost, and the probing covariance keeps a floor proportional to the regularization weight. On an estimated plant, under conditions like PGAC's (small updates and sufficiently informative data), the closed loop stays stable, the cost of the gains converges linearly up to a bias, and the probing covariance again keeps this floor. In simulations at equal probing cost, the shape of KLPI's probing covariance, which PGAC can adopt, lowers the extra cost caused by suboptimal gains on nearly all tested plants, whereas KLPI's use of past iterations in its gain update changes this cost little.

math.OC↗

Learning On The Job: Zero-Shot Task Execution under Parametric Uncertainty via Trajectory-Parametrized Dual Control

Model-based control can achieve reliable task performance, but its effectiveness depends on the accuracy of the underlying model. Robots operating under model uncertainty must often adapt to previously unseen payloads, objects, and interaction dynamics to complete a task successfully. Conventional approaches typically rely on a dedicated task- and control-agnostic excitation phase for estimating physics parameters, delaying task execution and collecting potentially irrelevant data. In this paper, we instead consider zero-shot task execution under parametric uncertainty, where online model learning and control proceed concurrently during task execution. Our approach formulates reference generation within a dual control framework to produce task-relevant, informative trajectories that reduce parameter uncertainty in directions critical to task success. We predefine a feedback policy with an explicit parameter adaptation law and optimize the reference through the resulting adaptive closed-loop dynamics. We propose two formulations: 1) minimizing expected task cost under parameter uncertainty, and 2) minimizing optimality loss, which quantifies the degradation in task performance caused by planning with an incorrect parameter estimate. By actively optimizing for informative trajectories via a natural Fisher information measure, we tightly approach a lower bound on task-relevant parameter uncertainty while simultaneously achieving reliable task execution. Across diverse tasks, controllers, and adaptation laws, targeted exploration enables the identification required for successful task execution, allowing robots to learn relevant physical parameters while completing the task in a single attempt.

cs.RO↗

Distributionally Robust Control via Stein Variational Inference for Contact-Rich Manipulation

Reliable robotic manipulation requires control policies that can accurately represent and adapt to uncertainty arising from contact-rich interactions. Modern data-driven methods mitigate uncertainty through large-scale training and computation, and degrade significantly in performance with limited number of training samples. By contrast, classical model-based controllers are computationally efficient and reliable, but their limited ability to represent task-relevant uncertainty can hinder performance in contact-rich interactions. In this work, we propose to expand the capabilities of model-based manipulation control through more flexible uncertainty modeling that retains performance while exactly adapting to uncertainty. Our approach casts the manipulation problem as a distributionally robust control optimization and proposes a novel deterministic formulation based on Stein variational inference that preserves performance while explicitly modeling task-sensitive parameter uncertainty. As a result, the derived controllers are more aware of task sensitivities to uncertainty, yielding high reliability without compromising performance. Experimental results demonstrate up to 3$\times$ improved robustness across a range of contact-rich manipulation tasks under broad parametric uncertainty, outperforming existing model-based control methods.

cs.RO↗

Optimal Path Planning of Airborne Wind Energy Systems with a Flexible Tether

In this work, we establish an optimal control framework for airborne wind energy systems (AWESs) with flexible tethers. The AWES configuration, consisting of a six-degree-of-freedom aircraft, a flexible tether, and a winch, is formulated as an index-1 differential-algebraic system of equations (DAE). We achieve this by adopting a minimal coordinate representation that uses Euler angles to characterize the aircraft's attitude and employing a quasi-static approach for the tether. The presented method contrasts with other recent optimization studies that use an index-3 DAE approach. By doing so, our approach avoids related inconsistency condition problems. We use a homotopy strategy to solve the optimal control problem that ultimately generates optimal trajectories of the AWES with a flexible tether. We furthermore compare with a rigid tether model by investigating the resulting mechanical powers and tether forces. Simulation results demonstrate the efficacy of the presented methodology and the necessity to incorporate the flexibility of the tether when solving the optimal control problem.

math.OC↗

Unifying Entropy Regularization in Optimal Control: From and Back to Classical Objectives via Iterated Soft Policies and Path Integral Solutions

This paper develops a unified perspective on several optimal control formulations through the lens of Kullback-Leibler (KL) regularization. We propose a central problem that separates the KL penalties on policies and transitions with independent weights, thus generalizing the standard trajectory-level KL-regularization used in probabilistic optimal control. This umbrella formulation recovers various control problems: the classical Stochastic Optimal Control (SOC), Risk-Sensitive Stochastic Optimal Control (RSOC), and their policy-based KL-regularized counterparts, termed soft-policy SOC and RSOC, which yield tractable surrogates. Beyond being regularized variants, these soft-policy formulations majorize the original SOC and RSOC, thus, iterating their solutions recovers the original objectives. We further identify a synchronized case of soft-policy RSOC where the policy and transition KL weights coincide, yielding a linear Bellman operator, path-integral solution, and compositionality -- extending these computationally favourable properties to a broad class of control problems.

math.OC↗

Gaussian Process Dual MPC using Active Inference: An Autonomous Vehicle Usecase

Designing controllers under uncertainty requires balancing the need to explore system dynamics with the requirement to maintain reliable control performance. Dual control addresses this challenge by selecting actions that both regulate the system and actively gather informative data. This paper investigates the use of the Active Inference framework, grounded in the Free Energy Principle, for developing a dual model-predictive controller (MPC). To identify and quantify uncertainty, we introduce an online sparse semi-parametric Gaussian Process model that combines the flexibility of nonparametric with the efficiency of parametric learning for real-time updates. By applying the expected free energy functional to this adaptive probabilistic model, we derive an MPC objective that incorporates an information-theoretic term, which captures uncertainty arising from both the learned model and measurement noise. This formulation leads to a stochastic optimal control problem for dual controller design, which is solved using a novel dynamic-programming-based method. Simulation results on a vehicle use case demonstrate that the proposed algorithm enhances autonomous driving control performance across different settings and scenarios.

math.OC↗

How to Capture Human Preference: Commissioning of a Robotic Use-Case via Preferential Bayesian Optimisation

The popularity of Bayesian Optimization (BO) to automate or support the commissioning of engineering systems is rising. Conventional BO, however, relies on the availability of a scalar objective function. The latter is often difficult to define and rarely captures the nuanced judgement of expert operators in industrial settings. Preferential Bayesian Optimization (PBO) addresses this limitation by relying solely on pairwise preference feedback of a human expert, so-called duels. In this paper, we study PBO's capacity to commission a particular setup where a manipulator needs to push a block towards a target position. We benchmark state-of-the-art algorithms in both simulations and in the real world. Our results confirm that PBO can commission the set-up to the satisfaction of an expert operator whilst relying solely on binary preference feedback. To evaluate to what extend the same result can be achieved using conventional BO we investigate the experts decision consistency against an expert-designed cost function. Our study reveals that the experts fail to define a cost function that is in full agreement with their own decision process as witnessed in the PBO experiments. We then show that the auxiliary cost function that is constructed as a by-product of the PBO algorithms outperforms the expert-designed cost function in terms of decision consistency. Furthermore we demonstrate that this cost function can be used with conventional BO algorithms in an effort to reproduce the optimal design. This proofs the preference based cost function captures the experts' preferences perhaps more effectively than the experts could articulate preference themselves. In conclusion, we discuss downsides and propose directions for future research.

eess.SY↗

Differential Flatness of Quasi-Static Slider-Pusher Models with Applications in Control

This paper investigates the dynamic properties of planar slider-pusher systems as a motion primitive in manipulation tasks. To that end, we construct a differential kinematic model deriving from the limit surface approach under the quasi-static assumption and with negligible contact friction. The quasi-static model applies to generic slider shapes and circular pusher geometries, enabling a differential kinematic representation of the system. From this model, we analyze differential flatness - a property advantageous for control synthesis and planning - and find that slider-pusher systems with polygon sliders and circular pushers exhibit flatness with the centre of mass as a flat output. Leveraging this property, we propose two control strategies for trajectory tracking: a cascaded quasi-static feedback strategy and a dynamic feedback linearization approach. We validate these strategies through closed-loop simulations incorporating perturbed models and input noise, as well as experimental results using a physical setup with a finger-like pusher and vision-based state detection. The real-world experiments confirm the applicability of the simulation gains, highlighting the potential of the proposed methods for

eess.SY↗

Probabilistic Latent Variable Modeling for Dynamic Friction Identification and Estimation

Precise identification of dynamic models in robotics is essential to support control design, friction compensation, output torque estimation, etc. A longstanding challenge remains in the identification of friction models for robotic joints, given the numerous physical phenomena affecting the underlying friction dynamics which result into nonlinear characteristics and hysteresis behaviour in particular. These phenomena proof difficult to be modelled and captured accurately using physical analogies alone. This has motivated researchers to shift from physics-based to data-driven models. Currently, these methods are still limited in their ability to generalize effectively to typical industrial robot deployement, characterized by high- and low-velocity operations and frequent direction reversals. Empirical observations motivate the use of dynamic friction models but these remain particulary challenging to establish. To address the current limitations, we propose to account for unidentified dynamics in the robot joints using latent dynamic states. The friction model may then utilize both the dynamic robot state and additional information encoded in the latent state to evaluate the friction torque. We cast this stochastic and partially unsupervised identification problem as a standard probabilistic representation learning problem. In this work both the friction model and latent state dynamics are parametrized as neural networks and integrated in the conventional lumped parameter dynamic robot model. The complete dynamics model is directly learned from the noisy encoder measurements in the robot joints. We use the Expectation-Maximisation (EM) algorithm to find a Maximum Likelihood Estimate (MLE) of the model parameters. The effectiveness of the proposed method is validated in terms of open-loop prediction accuracy in comparison with baseline methods, using the Kuka KR6 R700 as a test platform.

cs.RO↗

Deterministic Trajectory Optimization through Probabilistic Optimal Control

In this article, we discuss two algorithms tailored to discrete-time deterministic finite-horizon nonlinear optimal control problems or so-called deterministic trajectory optimization problems. Both algorithms can be derived from an emerging theoretical paradigm that we refer to as probabilistic optimal control. The paradigm reformulates stochastic optimal control as an equivalent probabilistic inference problem and can be viewed as a generalisation of the former. The merit of this perspective is that it allows to address the problem using the Expectation-Maximization algorithm. It is shown that the application of this algorithm results in a fixed point iteration of probabilistic policies that converge to the deterministic optimal policy. Two strategies for policy evaluation are discussed, using state-of-the-art uncertainty quantification methods resulting into two distinct algorithms. The algorithms are structurally closest related to the differential dynamic programming algorithm and related methods that use sigma-point methods to avoid direct gradient evaluations. The main advantage of the algorithms is an improved balance between exploration and exploitation over the iterations, leading to improved numerical stability and accelerated convergence. These properties are demonstrated on different nonlinear systems.

math.OC↗

Introducing DAIMYO: a first-time-right dynamic design architecture and its application to tail-sitter UAS development

In recent years, there has been a notable evolution in various multidisciplinary design methodologies for dynamic systems. Among these approaches, a noteworthy concept is that of concurrent conceptual and control design or co-design. This approach involves the tuning of feedforward and/or feedback control strategies in conjunction with the conceptual design of the dynamic system. The primary aim is to discover integrated solutions that surpass those attainable through a disjointed or decoupled approach. This concurrent design paradigm exhibits particular promise in the context of hybrid unmanned aerial systems (UASs), such as tail-sitters, where the objectives of versatility (driven by control considerations) and efficiency (influenced by conceptual design) often present conflicting demands. Nevertheless, a persistent challenge lies in the potential disparity between the theoretical models that underpin the design process and the real-world operational environment, the so-called reality gap. Such disparities can lead to suboptimal performance when the designed system is deployed in reality. To address this issue, this paper introduces DAIMYO, a novel design architecture that incorporates a high-fidelity environment, which emulates real-world conditions, into the procedure in pursuit of a `first-time-right' design. The outcome of this innovative approach is a design procedure that yields versatile and efficient UAS designs capable of withstanding the challenges posed by the reality gap.

math.OC↗

Probabilistic Control and Majorization of Optimal Control

Probabilistic control design is founded on the principle that a rational agent attempts to match modelled with an arbitrary desired closed-loop system trajectory density. The framework was originally proposed as a tractable alternative to traditional optimal control design, parametrizing desired behaviour through fictitious transition and policy densities and using the information projection as a proximity measure. In this work we introduce an alternative parametrization of desired closed-loop behaviour and explore alternative proximity measures between densities. It is then illustrated how the associated probabilistic control problems solve into uncertain or probabilistic policies. Our main result is to show that the probabilistic control objectives majorize conventional, stochastic and risk sensitive, optimal control objectives. This observation allows us to identify two probabilistic fixed point iterations that converge to the deterministic optimal control policies establishing an explicit connection between either formulations. Further we demonstrate that the risk sensitive optimal control formulation is also technically equivalent to a Maximum Likelihood estimation problem on a probabilistic graph model where the notion of costs is directly encoded into the model. The associated treatment of the estimation problem is then shown to coincide with the moment projected probabilistic control formulation. That way optimal decision making can be reformulated as an iterative inference problem. Based on these insights we discuss directions for algorithmic development.

cs.LG↗

Flatness-based MPC using B-splines transcription with application to a Pusher-Slider System

This work discusses the use of model predictive control (MPC) for the manipulation of a pusher-slider system. In particular we leverage the differential flatness of the pusher-slider in combination with a B-splines transcription to address the computational demand that is typically associated to real-time implementation of an MPC controller. We demonstrate the flatness based B-spline MPC controller in simulation and compare it to a standard MPC implementation approach using direct multiple shooting. We evaluate the computational advantage of the flatness based MPC empirically and document computational acceleration up to 65%.

math.OC↗

Differential Flatness of Slider-Pusher Systems for Constrained Time Optimal Collision Free Path Planning

In this work we show that the differential kinematics of slider-pusher systems are differentially flat assuming quasi-static behaviour and frictionless contact. Second we demonstrate that the state trajectories are invariant to time-differential transformations of the path parametrizing coordinate. For one this property allows to impose arbitrary velocity profiles on the slider without impacting the geometry of the state trajectory. This property implies that certain path planning problems may be decomposed approximately into a strictly geometric path planning and an auxiliary throughput speed optimization problem. Building on these insights we elaborate a numerical approach tailored to constrained time optimal collision free path planning and apply it to the slider-pusher system.

math.DS↗

A Supervisory Learning Control Framework for Autonomous & Real-time Task Planning for an Underactuated Cooperative Robotic task

We introduce a framework for cooperative manipulation, applied on an underactuated manipulation problem. Two stationary robotic manipulators are required to cooperate in order to reposition an object within their shared work space. Control of multi-agent systems for manipulation tasks cannot rely on individual control strategies with little to no communication between the agents that serve the common objective through swarming. Instead a coordination strategy is required that queries subtasks to the individual agents. We formulate the problem in a Task And Motion Planning (TAMP) setting, while considering a decomposition strategy that allows us to treat the task and motion planning problems separately. We solve the supervisory planning problem offline using deep Reinforcement Learning techniques resulting into a supervisory policy capable of coordinating the two manipulators into a successful execution of the pick-and-place task. Additionally, a benefit of solving the task planning problem offline is the possibility of real-time (re)planning, demonstrating robustness in the event of subtask execution failure or on-the-fly task changes. The framework achieved zero-shot deployment on the real setup with a success rate that is higher than 90%.

cs.RO↗

Information-Theoretic Policy Learning from Partial Observations with Fully Informed Decision Makers

In this work we formulate and treat an extension of the Imitation from Observations problem. Imitation from Observations is a generalisation of the well-known Imitation Learning problem where state-only demonstrations are considered. In our treatment we extend the scope of Imitation from Observations to feature-only demonstrations which could arguably be described as partial observations. Therewith we mean that the full state of the decision makers is unknown and imitation must take place on the basis of a limited set of features. We set out for methods that extract an executable policy directly from those features which, in the literature, would be referred to as Behavioural Cloning methods. Our treatment combines elements from probability and information theory and draws connections with entropy regularized Markov Decision Processes.

eess.SY↗