arXiv subjects

Tomonori Sadamoto

Publications and source records attributed to Tomonori Sadamoto.

11 recordsLinked to original sources

arXiv2025

Policy Gradient Method for LQG Control via Input-Output-History Representation: Convergence to $O(ε)$-Stationary Points

We study the policy gradient method (PGM) for the linear quadratic Gaussian (LQG) dynamic output-feedback control problem using an input-output-history (IOH) representation of the closed-loop system. First, we show that any dynamic output-feedback controller is equivalent to a static partial-state feedback gain for a new system representation characterized by a finite-length IOH. Leveraging this equivalence, we reformulate the search for an optimal dynamic output feedback controller as an optimization problem over the corresponding partial-state feedback gain. Next, we introduce a relaxed version of the IOH-based LQG problem by incorporating a small process noise with covariance $εI$ into the new system to ensure coerciveness, a key condition for establishing gradient-based convergence guarantees. Consequently, we show that a vanilla PGM for the relaxed problem converges to an $\mathcal{O}(ε)$-stationary point, i.e., $\overline{K}$ satisfying $\|\nabla J(\overline{K})\|_F \leq \mathcal{O}(ε)$, where $J$ denotes the original LQG cost. Numerical experiments empirically indicate convergence to the vicinity of the globally optimal LQG controller.

math.OC↗

arXiv2025

Co-Optimization of EV Charging Control and Incentivization for Enhanced Power System Stability

We study how high charging rate demands from electric vehicles (EVs) in a power distribution grid may collectively cause poor dynamic performance, and propose a price incentivization strategy to steer customers to settle for lesser charging rate demands so that such performance degradation can be avoided. We pose the problem as a joint optimization and optimal control formulation. The optimization determines the optimal charging setpoints for EVs to minimize the $\mathcal{H}_2$-norm of the transfer function of the grid model, while the optimal control simultaneously develops a linear quadratic regulator (LQR) based state-feedback control signal for the battery currents of those EVs to jointly improve the small-signal dynamic performance of the system states. A subsequent algorithm is developed to determine how much customers may be willing to sacrifice their intended charging rate demands in return for financial incentives. Results are derived for both unidirectional and bidirectional charging, and validated using numerical simulations of multiple EV charging stations (EVCSs) in the IEEE 33-bus power distribution model.

math.OC↗

arXiv2024

Policy Gradient Methods for Designing Dynamic Output Feedback Controllers

This paper proposes model-based and model-free policy gradient methods (PGMs) for designing dynamic output feedback controllers for discrete-time partially observable systems. To fulfill this objective, we first show that any dynamic output feedback controller design is equivalent to a state-feedback controller design for a newly introduced system whose internal state is a finite-length input-output history (IOH). Next, based on this equivalency, we propose a model-based PGM and show its global linear convergence by proving that the Polyak-Lojasiewicz inequality holds for a reachability-based lossless projection of the IOH dynamics. Moreover, we propose two model-free implementations of the PGM: the multi- and single-episodic PGM. The former is a Monte Carlo approximation of the model-based PGM, whereas the latter is a simplified version of the former for ease of use in real systems. A sample complexity analysis of both methods is also presented. Finally, the effectiveness of the model-based/model-free PGMs is investigated through a numerical simulation.

eess.SY↗

arXiv2024

A Globally Convergent Policy Gradient Method for Linear Quadratic Gaussian (LQG) Control

We present a model-based globally convergent policy gradient method (PGM) for linear quadratic Gaussian (LQG) control. Firstly, we establish equivalence between optimizing dynamic output feedback controllers and designing a static feedback gain for a system represented by a finite-length input-output history (IOH). This IOH-based approach allows us to explore LQG controllers within a parameter space defined by IOH gains. Secondly, by considering a control law comprising the IOH gain and a sufficiently small random perturbation, we show that the cost function, evaluated through the control law over IOH gains, is gradient-dominant and locally smooth, ensuring the global linear convergence of the PGM. Numerical simulations show that the dynamic controller learned by the proposed PGM is almost same as the LQG optimal controller, indicating promising results even in a reduced-order controller design.

math.OC↗

arXiv2023

Input-Output History Feedback Controller for Encrypted Control with Leveled Fully Homomorphic Encryption

Protecting the parameters, states, and input/output signals of a dynamic controller is essential for securely outsourcing its computation to an untrusted third party. Although a fully homomorphic encryption scheme allows the evaluation of controller operations with encrypted data, an encrypted dynamic controller with the encryption scheme destabilizes a closed-loop system or degrades the control performance due to overflow. This paper presents a novel controller representation based on input-output history data to implement an encrypted dynamic controller that operates without destabilization and performance degradation. Implementation of this encrypted dynamic controller representation can be optimized via batching techniques to reduce the time and space complexities. Furthermore, this study analyzes the stability and performance degradation of a closed-loop system caused by the effects of controller encryption. A numerical simulation demonstrates the feasibility of the proposed encrypted control scheme, which inherits the control performance of the original controller at a sufficient level.

eess.SY↗

arXiv2022

Designing Optimal Key Lengths and Control Laws for Encrypted Control Systems based on Sample Identifying Complexity and Deciphering Time

In the state-of-the-art literature on cryptography and control theory, there has been no systematic methodology of constructing cyber-physical systems that can achieve desired control performance while being protected against eavesdropping attacks. In this paper, we tackle this challenging problem. We first propose two novel notions referred to as sample identifying complexity and sample deciphering time in an encrypted-control framework. The former explicitly captures the relation between the dynamical characteristics of control systems and the level of identifiability of the systems while the latter shows the relation between the computation time for the identification and the key length of a cryptosystem. Based on these two tractable new notions, we propose a systematic method for designing the both of an optimal key length to prevent system identification with a given precision within a given life span of systems, and of an optimal controller to maximize both of the control performance and the difficulty of the identification. The efficiency of the proposed method in terms of security level and realtime-ness is investigated through numerical simulations. To the best of our knowledge, this paper first connect the relationship between the security of cryptography and dynamical systems from a control-theoretic perspective.

eess.SY↗

arXiv2020

Fast Online Reinforcement Learning Control using State-Space Dimensionality Reduction

In this paper, we propose a fast reinforcement learning (RL) control algorithm that enables online control of large-scale networked dynamic systems. RL is an effective way of designing model-free linear quadratic regulator (LQR) controllers for linear time-invariant (LTI) networks with unknown state-space models. However, when the network size is large, conventional RL can result in unacceptably long learning times. The proposed approach is to construct a compressed state vector by projecting the measured state through a projective matrix. This matrix is constructed from online measurements of the states in a way that it captures the dominant controllable subspace of the open-loop network model. Next, a RL-controller is learned using the reduced-dimensional state instead of the original state such that the resultant cost is close to the optimal LQR cost. Numerical benefits as well as the cyber-physical implementation benefits of the approach are verified using illustrative examples including an example of wide-area control of the IEEE 68-bus benchmark power system.

eess.SY↗

arXiv2018

Dynamic Modeling, Stability, and Control of Power Systems with Distributed Energy Resources

This article presents a suite of new control designs for next-generation electric smart grids. The future grid will consist of thousands of non-conventional renewable generation sources such as wind, solar, and energy storage. These new components are collectively referred to as distributed energy resources (DER). The article presents a comprehensive list of dynamic models for DERs, and shows their coupling with the conventional generators and loads. It then presents several innovative control designs that can be used for facilitating large-scale DER integration. Ideas from decentralized retrofit control and distributed sparsity-promoting optimal control are used for developing these designs, followed by illustrations on an IEEE power system test model.

eess.SY↗

arXiv2018

Retrofit Control: Localization of Controller Design and Implementation

In this paper, we propose a retrofit control method for stable network systems. The proposed approach is a control method that, rather than an entire system model, requires a model of the subsystem of interest for controller design. To design the retrofit controller, we use a novel approach based on hierarchical state-space expansion that generates a higher-dimensional cascade realization of a given network system. The upstream dynamics of the cascade realization corresponds to an isolated model of the subsystem of interest, which is stabilized by a local controller. The downstream dynamics can be seen as a dynamical model representing the propagation of interference signals among subsystems, the stability of which is equivalent to that of the original system. This cascade structure enables a systematic analysis of both the stability and control performance of the resultant closed-loop system. The resultant retrofit controller is formed as a cascade interconnection of the local controller and an output rectifier that rectifies an output signal of the subsystem of interest so as to conform to an output signal of the isolated subsystem model while acquiring complementary signals neglected in the local controller design, such as interconnection signals from neighboring subsystems. Finally, the efficiency of the retrofit control method is demonstrated through numerical examples of power systems control and vehicle platoon control.

eess.SY↗

arXiv2017

A Retrofitting-based Supplementary Controller Design for Enhancing Damping Performance of Wind Power Systems

In this paper we address the growing concerns of wind power integration from the perspective of power system dynamics and stability. We propose a new retrofit control technique where an additional controller is designed at the doubly-fed induction generator site inside the wind power plant. This controller cancels the adverse impacts of the power flow from the wind side to the grid side on the dynamics of the overall power system. The main advantage of this controller is that it can be implemented by feeding back only the wind states and wind bus voltage without depending on any of the other synchronous machines in the rest of the system. Through simulations of a 4-machine Kundur power system model we show that the retrofit can efficiently enhance the damping performance of the system variable despite very high values of wind penetration.

eess.SY↗

arXiv2015

Weak Resilience of Networked Control Systems

In this paper, we propose a method to establish a networked control system that maintains its stability in the presence of certain undesirable incidents on local controllers. We call such networked control systems weakly resilient. We first derive a necessary and sufficient condition for the weak resilience of networked systems. Networked systems do not generally satisfy this condition. Therefore, we provide a method for designing a compensator which ensures the weak resilience of the compensated system. Finally, we illustrate the efficiency of the proposed method by a power system example based on the IEEE 14-bus test system.

eess.SY↗