arXiv subjects

Carsten Hartmann

Publications and source records attributed to Carsten Hartmann.

50 records · Page 3Linked to original sources

arXiv2018

Singularly perturbed forward-backward stochastic differential equations: application to the optimal control of bilinear systems

We study linear-quadratic stochastic optimal control problems with bilinear state dependence for which the underlying stochastic differential equation (SDE) consists of slow and fast degrees of freedom. We show that, in the same way in which the underlying dynamics can be well approximated by a reduced order effective dynamics in the time scale limit (using classical homogenziation results), the associated optimal expected cost converges in the time scale limit to an effective optimal cost. This entails that we can well approximate the stochastic optimal control for the whole system by the reduced order stochastic optimal control, which is clearly easier to solve because of lower dimensionality. The approach uses an equivalent formulation of the Hamilton-Jacobi-Bellman (HJB) equation, in terms of forward-backward SDEs (FBSDEs). We exploit the efficient solvability of FBSDEs via a least squares Monte Carlo algorithm and show its applicability by a suitable numerical example.

math.DS↗

arXiv2018

WavePacket: A Matlab package for numerical quantum dynamics. II: Open quantum systems, optimal control, and model reduction

WavePacket is an open-source program package for numeric simulations in quantum dynamics. It can solve time-independent or time-dependent linear Schrödinger and Liouville-von Neumann-equations in one or more dimensions. Also coupled equations can be treated, which allows, e.g., to simulate molecular quantum dynamics beyond the Born-Oppenheimer approximation. Optionally accounting for the interaction with external electric fields within the semi-classical dipole approximation, WavePacket can be used to simulate experiments involving tailored light pulses in photo-induced physics or chemistry. Being highly versatile and offering visualization of quantum dynamics 'on the fly', WavePacket is well suited for teaching or research projects in atomic, molecular and optical physics as well as in physical or theoretical chemistry. Building on the previous Part I which dealt with closed quantum systems and discrete variable representations, the present Part II focuses on the dynamics of open quantum systems, with Lindblad operators modeling dissipation and dephasing. This part also describes the WavePacket function for optimal control of quantum dynamics, building on rapid monotonically convergent iteration methods. Furthermore, two different approaches to dimension reduction implemented in WavePacket are documented here. In the first one, a balancing transformation based on the concepts of controllability and observability Gramians is used to identify states that are neither well controllable nor well observable. Those states are either truncated or averaged out. In the other approach, the H2-error for a given reduced dimensionality is minimized by H2 optimal model reduction techniques, utilizing a bilinear iterative rational Krylov algorithm.

quant-ph↗

arXiv2017

Learning Inverse Statics Models Efficiently

Online Goal Babbling and Direction Sampling are recently proposed methods for direct learning of inverse kinematics mappings from scratch even in high-dimensional sensorimotor spaces following the paradigm of "learning while behaving". To learn inverse statics mappings - primarily for gravity compensation - from scratch and without using any closed-loop controller, we modify and enhance the Online Goal Babbling and Direction Sampling schemes. Moreover, we exploit symmetries in the inverse statics mappings to drastically reduce the number of samples required for learning inverse statics models. Results for a 2R planar robot, a 3R simplified human arm, and a 4R humanoid robot arm clearly demonstrate that their inverse statics mappings can be learned successfully with our modified online Goal Babbling scheme. Furthermore, we show that the number of samples required for the 2R and 3R arms can be reduced by a factor of at least 8 and 16 resp. -depending on the number of discovered symmetries.

cs.LG↗

arXiv2017

Optimal control of Markov jump processes : Asymptotic analysis, algorithms and applications to the modelling of chemical reaction systems

Markov jump processes are widely used to model natural and engineered processes. In the context of biological or chemical applications one typically refers to the chemical master equation (CME), which models the evolution of the probability mass of any copy-number combination of the interacting particles. When many interacting particles ("species") are considered, the complexity of the CME quickly increases, making direct numerical simulations impossible. This is even more problematic when one aims at controlling the Markov jump processes defined by the CME. In this work, we study both open loop and feedback optimal control problems of the Markov jump processes in the case that the controls can only be switched at fixed control stages. Based on Kurtz's limit theorems, we prove the convergence of the respective control value functions of the underlying Markov decision problem as the copy numbers of the species go to infinity. In the case of the optimal control problem on a finite time-horizon, we propose a hybrid control policy algorithm to overcome the difficulties due to the curse of dimensionality when the copy number of the involved species is large. Two numerical examples demonstrate the suitability of both the analysis and the proposed algorithms.

math.OC↗

arXiv2017

Model reduction of controlled Fokker--Planck and Liouville-von Neumann equations

Model reduction methods for bilinear control systems are compared by means of practical examples of Liouville-von Neumann and Fokker--Planck type. Methods based on balancing generalized system Gramians and on minimizing an H2-type cost functional are considered. The focus is on the numerical implementation and a thorough comparison of the methods. Structure and stability preservation are investigated, and the competitiveness of the approaches is shown for practically relevant, large-scale examples.

math.NA↗

arXiv2017

Partitioning a macroscopic system into independent subsystems

We discuss the problem of partitioning a macroscopic system into a collection of independent subsystems. The partitioning of a system into replica-like subsystems is nowadays a subject of major interest in several field of theoretical and applied physics, and the thermodynamic approach currently favoured by practitioners is based on a phenomenological definition of an interface energy associated with the partition, due to a lack of easily computable expressions for a microscopic (i.e.~particle-based) interface energy. In this article, we outline a general approach to derive sharp and computable bounds for the interface free energy in terms of microscopic statistical quantities. We discuss potential applications in nanothermodynamics and outline possible future directions.

math-ph↗

arXiv2017

Girsanov reweighting for path ensembles and Markov state models

The sensitivity of molecular dynamics on changes in the potential energy function plays an important role in understanding the dynamics and function of complex molecules.We present a method to obtain path ensemble averages of a perturbed dynamics from a set of paths generated by a reference dynamics. It is based on the concept of path probability measure and the Girsanov theorem, a result from stochastic analysis to estimate a change of measure of a path ensemble. Since Markov state models (MSM) of the molecular dynamics can be formulated as a combined phase-space and path ensemble average, the method can be extended toreweight MSMs by combining it with a reweighting of the Boltzmann distribution. We demonstrate how to efficiently implement the Girsanov reweighting in a molecular dynamics simulation program by calculating parts of the reweighting factor "on the fly" during the simulation, and we benchmark the method on test systems ranging from a two-dimensional diffusion process to an artificial many-body system and alanine dipeptide and valine dipeptide in implicit and explicit water. The method can be used to study the sensitivity of molecular dynamics on external perturbations as well as to reweight trajectories generated by enhanced sampling schemes to the original dynamics.

cond-mat.stat-mech↗

arXiv2016

Importance sampling in path space for diffusion processes with slow-fast variables

Importance sampling is a widely used technique to reduce the variance of a Monte Carlo estimator by an appropriate change of measure. In this work, we study importance sam- pling in the framework of diffusion process and consider the change of measure which is realized by adding a control force to the original dynamics. For certain exponential type expectation, the corresponding control force of the optimal change of measure leads to a zero-variance estimator and is related to the solution of a Hamilton-Jacobi-Bellmann equation. We focus on certain diffu- sions with both slow and fast variables, and the main result is that we obtain an upper bound of the relative error for the importance sampling estimators with control obtained from the limiting dynamics. We demonstrate our approximation strategy with an illustrative numerical example.

math.PR↗

arXiv2015

Molecular Dynamics in a Grand Ensemble: Bergmann-Lebowitz model and Adaptive Resolution Simulation

This article deals with the molecular dynamics simulation of open systems that can exchange energy and matter with a reservoir; the physics of the reservoir and its interactions with the system are described by the model introduced by Bergmann and Lebowitz.Despite its conceptual appeal, the model did not gain popularity in the field of molecular simulation and, as a consequence, did not play a role in the development of open system molecular simulation techniques, even though it can provide the conceptual legitimation of simulation techniques that mimic open systems. We shall demonstrate that the model can serve as a tool to devise both numerical procedures and conceptual definitions of physical quantities that cannot be defined in a straightforward way by systems with a fixed number of molecules. In particular, we discuss the utility of the Bergmann-Lebowitz (BL) model for the calculation of equilibrium time correlation functions within the Grand Canonical Adaptive Resolution method (GC-AdResS) and report numerical results for the case of liquid water.

physics.comp-ph↗

arXiv2015

Pseudo generators for under-resolved molecular dynamics

Many features of a molecule which are of physical interest (e.g. molecular conformations, reaction rates) are described in terms of its dynamics in configuration space. This article deals with the projection of molecular dynamics in phase space onto configuration space. Specifically, we study the situation that the phase space dynamics is governed by a stochastic Langevin equation and study its relation with the configurational Smoluchowski equation in the three different scaling regimes: Firstly, the Smoluchowski equations in non-Cartesian geometries are derived from the overdamped limit of the Langevin equation. Secondly, transfer operator methods are used to describe the metastable behaviour of the system at hand, and an explicit small-time asymptotics is derived on which the Smoluchowski equation turns out to govern the dynamics of the position coordinate (without any assumptions on the damping). By using an adequate reduction technique, these considerations are then extended to one-dimensional reaction coordinates. Thirdly, we sketch three different approaches to approximate the metastable dynamics based on time-local information only.

math.DS↗

arXiv2014

Optimal control of multiscale systems using reduced-order models

We study optimal control of diffusions with slow and fast variables and address a question raised by practitioners: is it possible to first eliminate the fast variables before solving the optimal control problem and then use the optimal control computed from the reduced-order model to control the original, high-dimensional system? The strategy "first reduce, then optimize"--rather than "first optimize, then reduce"--is motivated by the fact that solving optimal control problems for high-dimensional multiscale systems is numerically challenging and often computationally prohibitive. We state sufficient and necessary conditions, under which the "first reduce, then control" strategy can be employed and discuss when it should be avoided. We further give numerical examples that illustrate the "first reduce, then optmize" approach and discuss possible pitfalls.

math.OC↗

arXiv2013

Meshless discretization of LQ-type stochastic control problems

We propose a novel Galerkin discretization scheme for stochastic optimal control problems on an indefinite time horizon. The control problems are linear-quadratic in the controls, but possibly nonlinear in the state variables, and the discretization is based on the fact that problems of this kind can be transformed into linear boundary value problems by a logarithmic transformation. We show that the discretized linear problem is dual to a Markov decision problem, the precise form of which depends on the chosen Galerkin basis. We prove a strong error bound in $L^{2}$ for the general scheme and discuss two special cases: a variant of the known Markov chain approximation obtained from a basis of characteristic functions of a box discretization, and a sparse approximation that uses the basis of committor functions of metastable sets of the dynamics; the latter is particularly suited for high-dimensional systems, e.g., control problems in molecular dynamics. We illustrate the method with several numerical examples, one being the optimal control of Alanine dipeptide to its helical conformation.

math.OC↗

arXiv2013

Grand-canonical-like molecular-dynamics simulations by using an adaptive-resolution technique

In this work, we provide a detailed theoretical analysis, supported by numerical tests, of the reliability of the adaptive resolution simulation (AdResS) technique in sampling the Grand Canonical ensemble. We demonstrate that the correct density and radial distribution functions in the hybrid region, where molecules change resolution, are two necessary conditions for considering the atomistic and coarse-grained regions in AdResS equivalent to subsystems of a full atomistic system with an accuracy up to the second order with respect to the probability distribution of the system. Moreover, we show that the work done by the thermostat and a thermodynamic force in the transition region is formally equivalent to balance the chemical potential difference between the different resolutions. From these results follows the main conclusion that the atomistic region exchanges molecules with the coarse-grained region in a Grand Canonical fashion with an accuracy up to (at least) second order. Numerical tests, for the relevant case of liquid water at ambient conditions, are carried out to strengthen the conclusions of the theoretical analysis. Finally, in order to show the computational convenience of AdResS as a Grand Canonical set up, we compare our method to the Insertion Particle Method (IMP) in its most efficient computational implementation. This fruitful combination of theoretical principles and numerical evidence candidates the adaptive resolution technique as a natural, general and efficient protocol for Grand Canonical Molecular Dynamics for the case of large systems.

cond-mat.stat-mech↗

arXiv2012

Efficient Rare Event Simulation by Optimal Nonequilibrium Forcing

Rare event simulation and estimation for systems in equilibrium are among the most challenging topics in molecular dynamics. As was shown by Jarzynski and others, nonequilibrium forcing can theoretically be used to obtain equilibrium rare event statistics. The advantage seems to be that the external force can speed up the sampling of the rare events by biasing the equilibrium distribution towards a distribution under which the rare events is no longer rare. Yet algorithmic methods based on Jarzynski's and related results often fail to be efficient because they are based on sampling in path space. We present a new method that replaces the path sampling problem by minimization of a cross-entropy-like functional which boils down to finding the optimal nonequilibrium forcing. We show how to solve the related optimization problem in an efficient way by using an iterative strategy based on milestoning.

math.OC↗