Searcharxiv⌕ Search

arXiv subjects

Behcet Acikmese

Publications and source records attributed to Behcet Acikmese.

46 records · Page 3Linked to original sources
arXiv2018

Optimal mass transport and kernel density estimation for state-dependent networked dynamic systems

State-dependent networked dynamical systems are ones where the interconnections between agents change as a function of the states of the agents. Such systems are highly nonlinear, and a cohesive strategy for their control is lacking in the literature. In this paper, we present two techniques pertaining to the density control of such systems. Agent states are initially distributed according to some density, and a feedback law is designed to move the agents to a target density profile. We use optimal mass transport to design a feedforward control law propelling the agents towards this target density. Kernel density estimation, with constraints imposed by the state-dependent dynamics, is then used to allow each agent to estimate the local density of the agents.

math.OC↗
arXiv2018

Successive Convexification for 6-DoF Mars Rocket Powered Landing with Free-Final-Time

In this paper, we employ successive convexification to solve the minimum-time 6-DoF rocket powered landing problem. The contribution of this paper is the development and demonstration of a free-final-time problem formulation that can be solved iteratively using a successive convexification framework. This paper is an extension of our previous work on the 3-DoF free-final-time and the 6-DoF fixed-final-time minimum-fuel problems. Herein, the vehicle is modeled as a 6-DoF rigid-body controlled by a single gimbaled rocket engine. The trajectory is subject to a variety of convex and non-convex state- and control-constraints, and aerodynamic effects are assumed negligible. The objective of the problem is to determine the optimal thrust commands that will minimize the time-of-flight while satisfying the aforementioned constraints. Solving this problem quickly and reliably is challenging because (a) it is nonlinear and non-convex, (b) the validity of the solution is heavily dependent on the accuracy of the discretization scheme, and (c) it can be difficult to select a suitable reference trajectory to initialize an iterative solution process. To deal with these issues, our algorithm (a) uses successive convexification to eliminate non-convexities, (b) computes the discrete linear-time-variant system matrices to ensure that the converged solution perfectly satisfies the original nonlinear dynamics, and (c) can be initialized with a simple, dynamically inconsistent reference trajectory. Using the proposed convex formulation and successive convexification framework, we are able to convert the original non-convex problem into a sequence of convex second-order cone programming (SOCP) sub-problems. Through the use of Interior Point Method (IPM) solvers, this sequence can be solved quickly and reliably, thus enabling higher fidelity real-time guidance for rocket powered landings on Mars.

math.OC↗
arXiv2017

Velocity Field Generation for Density Control of Swarms using Heat Equation and Smoothing Kernels

This paper presents a method to control the probability density distribution of a swarm of vehicles via velocity fields. The proposed approach synthesizes smooth velocity fields, which specify a desired velocity as a function of time and position in a decentralized manner i.e., each agent calculates the desired velocity locally by utilizing the number of agents within a prescribed communication distance. Swarm converges to the desired/commanded density distribution by following the velocity field. The local information consists, only, of agents' positions and it is utilized to estimate the density around each agent. Local density estimation is performed by using kernel density estimation. Then the local density estimates and the desired density are utilized to generate the velocity field, which propagates the swarm probability density distribution via the well-known heat equation. The key advantage of using smooth velocity fields to control swarm density with respect to our earlier Markov chain based probabilistic control methods is that the agents move more smoothly in an organized manner and their relative velocities go to zero as they get closer to each other, i.e., they facilitate conflict/collision avoidance. The desired density distribution, which is commanded to the local controllers, can be computed by using a Markov chain that propagates the desired density distribution within prescribed requirements. Along with convergence and stability analysis, the effectiveness of the approach is illustrated via numerical examples.

math.OC↗
arXiv2017

Successive Convexification of Non-Convex Optimal Control Problems and Its Convergence Properties

This paper presents an algorithm to solve non-convex optimal control problems, where non-convexity can arise from nonlinear dynamics, and non-convex state and control constraints. This paper assumes that the state and control constraints are already convex or convexified, the proposed algorithm convexifies the nonlinear dynamics, via a linearization, in a successive manner. Thus at each succession, a convex optimal control subproblem is solved. Since the dynamics are linearized and other constraints are convex, after a discretization, the subproblem can be expressed as a finite dimensional convex programming subproblem. Since convex optimization problems can be solved very efficiently, especially with custom solvers, this subproblem can be solved in time-critical applications, such as real-time path planning for autonomous vehicles. Several safe-guarding techniques are incorporated into the algorithm, namely virtual control and trust regions, which add another layer of algorithmic robustness. A convergence analysis is presented in continuous- time setting. By doing so, our convergence results will be independent from any numerical schemes used for discretization. Numerical simulations are performed for an illustrative trajectory optimization example.

math.OC↗
arXiv2016

Safe Markov Chains for Density Control of ON/OFF Agents with Observed Transitions

This paper presents a convex optimization approach to control the density distribution of autonomous mobile agents with two control modes: ON and OFF. The main new characteristic distinguishing this model from standard Markov decision models is the existence of the ON control mode and its observed actions. When an agent is in the ON mode, it can measure the instantaneous outcome of one of the actions corresponding to the ON mode and decides whether it should take this action or not based on this new observation. If it does not take this action, it switches to the OFF mode where it transitions to the next state based on a predetermined set of transitional probabilities, without making any additional observations. In this decision-making model, each agent acts autonomously according to an ON/OFF decision policy, and the discrete probability distribution for the agent's state evolves according to a discrete-time Markov chain that is a linear function of the stochastic environment (i.e., transition probabilities) and the ON/OFF decision policy. The relevant policy synthesis is formulated as a convex optimization problem where hard safety and convergence constraints are imposed on the resulting Markov matrix. We first consider the case where the ON mode has a single action and the OFF mode has deterministic transitions (rather than stochastic) to demonstrate the model and the ideas behind our approach, which is then generalized to the case where ON mode has multiple actions and OFF mode has stochastic transitions.

math.OC↗
arXiv2016

Mixed Strategy for Constrained Stochastic Optimal Control

Choosing control inputs randomly can result in a reduced expected cost in optimal control problems with stochastic constraints, such as stochastic model predictive control (SMPC). We consider a controller with initial randomization, meaning that the controller randomly chooses from K+1 control sequences at the beginning (called K-randimization).It is known that, for a finite-state, finite-action Markov Decision Process (MDP) with K constraints, K-randimization is sufficient to achieve the minimum cost. We found that the same result holds for stochastic optimal control problems with continuous state and action spaces.Furthermore, we show the randomization of control input can result in reduced cost when the optimization problem is nonconvex, and the cost reduction is equal to the duality gap. We then provide the necessary and sufficient conditions for the optimality of a randomized solution, and develop an efficient solution method based on dual optimization. Furthermore, in a special case with K=1 such as a joint chance-constrained problem, the dual optimization can be solved even more efficiently by root finding. Finally, we test the theories and demonstrate the solution method on multiple practical problems ranging from path planning to the planning of entry, descent, and landing (EDL) for future Mars missions.

cs.RO↗
arXiv2015

Finite-Horizon Markov Decision Processes with State Constraints

Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (minimize costs) in a given stochastic dynamical environment. In many practical scenarios (multi-agent systems, telecommunication, queuing, etc.), the decision-making problem can have state constraints that must be satisfied, which leads to Constrained MDP (CMDP) problems. In the presence of such state constraints, the optimal policies can be very hard to characterize. This paper introduces a new approach for finding non-stationary randomized policies for finite-horizon CMDPs. An efficient algorithm based on Linear Programming (LP) and duality theory is proposed, which gives the convex set of feasible policies and ensures that the expected total reward is above a computable lower-bound. The resulting decision policy is a randomized policy, which is the projection of the unconstrained deterministic MDP policy on this convex set. To the best of our knowledge, this is the first result in state constrained MDPs to give an efficient algorithm for generating finite horizon randomized policies for CMDP with optimality guarantees. A simulation example of a swarm of autonomous agents running MDPs is also presented to demonstrate the proposed CMDP solution algorithm.

math.OC↗
arXiv2015

Finite-Horizon Markov Decision Processes with Sequentially-Observed Transitions

Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize costs) in a given stochastic dynamical environment. In this paper, we extend this model by incorporating additional information that the transitions due to actions can be sequentially observed. The proposed model benefits from this information and produces policies with better performance than those of standard MDPs. The paper also presents an efficient offline linear programming based algorithm to synthesize optimal policies for the extended model.

math.OC↗
arXiv2015

Spectrum of Laplacians for Graphs with Self-Loops

This note introduces a result on the location of eigenvalues, i.e., the spectrum, of the Laplacian for a family of undirected graphs with self-loops. We extend on the known results for the spectrum of undirected graphs without self-loops or multiple edges. For this purpose, we introduce a new concept of pseudo-connected graphs and apply a lifting of the graph with self-loops to a graph without self-loops, which is then used to quantify the spectrum of the Laplacian for the graph with self-loops.

math.OC↗