arXiv subjects

Sheung Chi Phillip Yam

Publications and source records attributed to Sheung Chi Phillip Yam.

27 records · Page 2Linked to original sources

arXiv2023

Maximum Principle for Mean Field Type Control Problems with General Volatility Functions

In this paper, we study the maximum principle of mean field type control problems when the volatility function depends on the state and its measure and also the control, by using our recently developed method. Our method is to embed the mean field type control problem into a Hilbert space to bypass the evolution in the Wasserstein space. We here give a necessary condition and a sufficient condition for these control problems in Hilbert spaces, and we also derive a system of forward-backward stochastic differential equations.

math.OC↗

arXiv2023

Control on Hilbert Spaces and Application to Some Mean Field Type Control Problems

We propose a new approach to studying classical solutions of the Bellman equation and Master equation for mean field type control problems, using a novel form of the "lifting" idea introduced by P.-L. Lions. Rather than studying the usual system of Hamilton-Jacobi/Fokker-Planck PDEs using analytic techniques, we instead study a stochastic control problem on a specially constructed Hilbert space, which is reminiscent of a tangent space on the Wasserstein space in optimal transport. On this Hilbert space we can use classical control theory techniques, despite the fact that it is infinite dimensional. A consequence of our construction is that the mean field type control problem appears as a special case. Thus we preserve the advantages of the lifiting procedure, while removing some of the difficulties. Our approach extends previous work by two of the coauthors, which dealt with a deterministic control problem for which the Hilbert space could be generic.

math.PR↗

arXiv2023

Mean Field Type Control Problems, Some Hilbert-space-valued FBSDEs, and Related Equations

In this article, we provide an original systematic global-in-time analysis of mean field type control problems on $\mathbb{R}^n$ with generic cost functionals by the modified approach but not the same, firstly proposed in [7], as the ``lifting'' idea introduced by P. L. Lions. As an alternative to the recent popular analytical method by tackling the master equation, we resolve the control problem in a certain proper Hilbert subspace of the whole space of $L^2$ random variables, it can be regarded as tangent space attached at the initial probability measure. The present work also fills the gap of the global-in-time solvability and extends the previous works of [7,11] which only dealt with quadratic cost functionals in control; the problem is linked to the global solvability of the Hilbert-space-valued forward-backward stochastic differential equation (FBSDE), which is solved by variational techniques here. We also rely on the Jacobian flow of the solution to this FBSDE to establish the regularities of the value function, including its linearly functional differentiability, which leads to the classical well-posedness of the Bellman equation. Together with the linear functional derivatives and the gradient of the linear functional derivatives of the solution to the FBSDE, we also obtain the classical well-posedness of the master equation.

math.OC↗

arXiv2023

An Effective Multivariate Normality Test via Hessians of Empirical Cumulant Generating Functions

In this article, we propose a new class of consistent tests for $p$-variate normality. These tests are based on the characterization of the standard multivariate normal distribution, that the Hessian of the corresponding cumulant generating function is identical to the $p\times p$ identity matrix and the idea of decomposing the information from the joint distribution into the dependence copula and all marginal distributions. Under the null hypothesis of multivariate normality, our proposed test statistic is independent of the unknown mean vector and covariance matrix so that the distribution-free critical value of the test can be obtained by Monte Carlo simulation. We also derive the asymptotic null distribution of proposed test statistic and establish the consistency of the test against different fixed alternatives. Last but not least, a comprehensive and extensive Monte Carlo study also illustrates that our test is a superb yet computationally convenient competitor to many well-known existing test statistics.

stat.ME↗

arXiv2021

Value-Gradient based Formulation of Optimal Control Problem and Machine Learning Algorithm

Optimal control problem is typically solved by first finding the value function through Hamilton-Jacobi equation (HJE) and then taking the minimizer of the Hamiltonian to obtain the control. In this work, instead of focusing on the value function, we propose a new formulation for the gradient of the value function (value-gradient) as a decoupled system of partial differential equations in the context of continuous-time deterministic discounted optimal control problem. We develop an efficient iterative scheme for this system of equations in parallel by utilizing the properties that they share the same characteristic curves as the HJE for the value function. For the theoretical part, we prove that this iterative scheme converges linearly in $L_α^2$ sense for some suitable exponent $α$ in a weight function. For the numerical method, we combine characteristic line method with machine learning techniques. Specifically, we generate multiple characteristic curves at each policy iteration from an ensemble of initial states, and compute both the value function and its gradient simultaneously on each curve as the labelled data. Then supervised machine learning is applied to minimize the weighted squared loss for both the value function and its gradients. Experimental results demonstrate that this new method not only significantly increases the accuracy but also improves the efficiency and robustness of the numerical estimates, particularly with less amount of characteristics data or fewer training steps.

math.OC↗

arXiv2021

Control in Hilbert Space and First Order Mean Field Type Problem

We extend the work \cite{bensoussan2019control} by two of the coauthors, which dealt with a deterministic control problem for which the Hilbert space could be generic and investigated a novel form of the `lifting' technique proposed by P. L. Lions. In \cite{bensoussan2019control}, we only showed the local existence and uniqueness of solutions to the FBODEs in the Hilbert space which were associated to the control problems with drift function consisting of the control only. In this article, we establish the global existence and uniqueness of the solutions to the FBODEs in Hilbert space corresponding to control problems with separable drift function which is nonlinear in state and linear in control. We shall also prove the sufficiency of the Pontryagin Maximum Principle and derive the corresponding Bellman equation. Besides, we shall show an analogue in the stationary case. Finally, by using the `lifting' idea as in \cite{stochasticv2,stochasticv1}, we shall apply the result to solve the linear quadratic mean field type control problems, and to show the global existence of the corresponding Bellman equations.

math.OC↗

arXiv2020

Machine Learning and Control Theory

We survey in this article the connections between Machine Learning and Control Theory. Control Theory provide useful concepts and tools for Machine Learning. Conversely Machine Learning can be used to solve large control problems. In the first part of the paper, we develop the connections between reinforcement learning and Markov Decision Processes, which are discrete time control problems. In the second part, we review the concept of supervised learning and the relation with static optimization. Deep learning which extends supervised learning, can be viewed as a control problem. In the third part, we present the links between stochastic gradient descent and mean-field theory. Conversely, in the fourth and fifth parts, we review machine learning approaches to stochastic control problems, and focus on the deterministic case, to explain, more easily, the numerical algorithms.

cs.LG↗

arXiv2019

Mean Field approach to stochastic control with partial information

The classical stochastic control problem under partial information can be formulated as a control problem for Zakai equation, whose solution is the unnormalized conditional probability distribution of the state of the system. Zakai equation is a stochastic Fokker-Planck equation. Therefore, the problem to be solved is similar to that met in Mean Field Control theory. Since Mean Field Control theory is much posterior to the development of Stochastic Control with partial information, the tools, techniques, and concepts obtained in the last decade, for Mean Field Games and Mean field type Control theory, have not been used for the control of Zakai equation. Our objective is to connect the two theories. We get the power of new tools, and we get new insights for the problem of stochastic control with partial information. For mean field theory, we get new interesting applications, but also new problems. Indeed, Mean Field Control Theory leads to very complex equations, like the Master equation, which is a nonlinear infinite dimensional P.D.E., for which general theorems are hardly available, although active research in this direction is performed. Direct methods are useful to obtain regularity results. We will develop in detail the LQ regulator problem, but since we cannot just consider the Gaussian case, well-known results, such as the separation principle is not available. An important result is available in the literature, due to A. Makowsky. It describes the solution of Zakai equation for linear systems with general initial condition (non-gaussian). We show that the separation principle can be extended for quadratic pay-off functionals, but the Kalman filter is much more complex than in the gaussian case. Finally we compare our work to the work of E. Bandini et al. and we show that the example E. Bandini et al. provided does not cover ours. Our system remains nonlinear in their setting.

math.OC↗

arXiv2014

Oracle, Multiple Robust and Multipurpose Calibration in a Missing Response Problem

In the presence of a missing response, reweighting the complete case subsample by the inverse of nonmissing probability is both intuitive and easy to implement. When the population totals of some auxiliary variables are known and when the inclusion probabilities are known by design, survey statisticians have developed calibration methods for improving efficiencies of the inverse probability weighting estimators and the methods can be applied to missing data analysis. Model-based calibration has been proposed in the survey sampling literature, where multidimensional auxiliary variables are first summarized into a predictor function from a working regression model. Usually, one working model is being proposed for each parameter of interest and results in different sets of calibration weights for estimating different parameters. This paper considers calibration using multiple working regression models for estimating a single or multiple parameters. Contrary to a common belief that overfitting hurts efficiency, we present three rather unexpected results. First, when the missing probability is correctly specified and multiple working regression models for the conditional mean are posited, calibration enjoys an oracle property: the same semiparametric efficiency bound is attained as if the true outcome model is known in advance. Second, when the missing data mechanism is misspecified, calibration can still be a consistent estimator when any one of the outcome regression models is correctly specified. Third, a common set of calibration weights can be used to improve efficiency in estimating multiple parameters of interest and can simultaneously attain semiparametric efficiency bounds for all parameters of interest. We provide connections of a wide class of calibration estimators, constructed based on generalized empirical likelihood, to many existing estimators in biostatistics, econometrics and survey sampling and perform simulation studies to show that the finite sample properties of calibration estimators conform well with the theoretical results being studied.

stat.ME↗