arXiv subjects

Huyên Pham

Publications and source records attributed to Huyên Pham.

At least 55 records · Page 3Linked to original sources

arXiv2022

Differential learning methods for solving fully nonlinear PDEs

We propose machine learning methods for solving fully nonlinear partial differential equations (PDEs) with convex Hamiltonian. Our algorithms are conducted in two steps. First the PDE is rewritten in its dual stochastic control representation form, and the corresponding optimal feedback control is estimated using a neural network. Next, three different methods are presented to approximate the associated value function, i.e., the solution of the initial PDE, on the entire space-time domain of interest. The proposed deep learning algorithms rely on various loss functions obtained either from regression or pathwise versions of the martingale representation and its differential relation, and compute simultaneously the solution and its derivatives. Compared to existing methods, the addition of a differential loss function associated to the gradient, and augmented training sets with Malliavin derivatives of the forward process, yields a better estimation of the PDE's solution derivatives, in particular of the second derivative, which is usually difficult to approximate. Furthermore, we leverage our methods to design algorithms for solving families of PDEs when varying terminal condition (e.g. option payoff in the context of mathematical finance) by means of the class of DeepOnet neural networks aiming to approximate functional operators. Numerical tests illustrate the accuracy of our methods on the resolution of a fully nonlinear PDE associated to the pricing of options with linear market impact, and on the Merton portfolio selection problem.

q-fin.CP↗

arXiv2022

Master Bellman equation in the Wasserstein space: Uniqueness of viscosity solutions

We study the Bellman equation in the Wasserstein space arising in the study of mean field control problems, namely stochastic optimal control problems for McKean-Vlasov diffusion processes.Using the standard notion of viscosity solution à la Crandall-Lions extended to our Wasserstein setting,we prove a comparison result under general conditions on the drift and reward coefficients, whichcoupled with the dynamic programming principle, implies that the value function is the unique viscosity solution of the Master Bellman equation.This is the first uniqueness result in such a second-order context. The classical arguments used in the standard cases of equations in finite-dimensional spaces or in infinite-dimensional separable Hilbert spaces do not extend to the present framework, due to the awkward nature of the underlying Wasserstein space. The adopted strategy is based on finite-dimensional approximations of the value function obtained in terms of the related cooperative $n$-player game, and on the construction of a smooth gauge-type function, built starting from a regularization of a sharp estimate of the Wasserstein metric; such a gauge-type function is used to generate maxima/minima through a suitable extension of the Borwein-Preiss generalization of Ekeland's variational principle on the Wasserstein space.

math.AP↗

arXiv2022

DeepSets and their derivative networks for solving symmetric PDEs

Machine learning methods for solving nonlinear partial differential equations (PDEs) are hot topical issues, and different algorithms proposed in the literature show efficient numerical approximation in high dimension. In this paper, we introduce a class of PDEs that are invariant to permutations, and called symmetric PDEs. Such problems are widespread, ranging from cosmology to quantum mechanics, and option pricing/hedging in multi-asset market with exchangeable payoff. Our main application comes actually from the particles approximation of mean-field control problems. We design deep learning algorithms based on certain types of neural networks, named PointNet and DeepSet (and their associated derivative networks), for computing simultaneously an approximation of the solution and its gradient to symmetric PDEs. We illustrate the performance and accuracy of the PointNet/DeepSet networks compared to classical feedforward ones, and provide several numerical results of our algorithm for the examples of a mean-field systemic risk, mean-variance problem and a min/max linear quadratic McKean-Vlasov control problem.

math.OC↗

arXiv2021

Rate of convergence for particle approximation of PDEs in Wasserstein space

We prove a rate of convergence for the $N$-particle approximation of a second-order partial differential equation in the space of probability measures, like the Master equation or Bellman equation of mean-field control problem under common noise. The rate is of order $1/N$ for the pathwise error on the solution $v$ and of order $1/\sqrt{N}$ for the $L^2$-error on its $L$-derivative $\partial_μv$. The proof relies on backward stochastic differential equations techniques.

math.OC↗

arXiv2021

Optimal bidding strategies for digital advertising

With the emergence of new online channels and information technology, digital advertising tends to substitute more and more to traditional advertising by offering the opportunity to companies to target the consumers/users that are really interested by their products or services. We introduce a novel framework for the study of optimal bidding strategies associated to different types of advertising, namely, commercial advertising for triggering purchases or subscriptions, and social marketing for alerting population about unhealthy behaviours (anti-drug, vaccination, road-safety campaigns). Our continuoustime models are based on a common framework encoding users online behaviours via their web-browsing at random times, and the targeted advertising auction mechanism widely used on Internet, the objective being to efficiently diffuse advertising information by means of digital channels. Our main results are to provide semi-explicit formulas for the optimal value and bidding policy for each of these problems. We show some sensitivity properties of the solution with respect to model parameters, and analyse how the different sources of digital information accessible to users including the social interactions affect the optimal bid for advertising auctions. We also study how to efficiently combine targeted advertising and non-targeted advertising mechanisms. Finally, some classes of examples with fully explicit formulas are derived.

math.OC↗

arXiv2021

Mean-field Markov decision processes with common noise and open-loop controls

We develop an exhaustive study of Markov decision process (MDP) under mean field interaction both on states and actions in the presence of common noise, and when optimization is performed over open-loop controls on infinite horizon. Such model, called CMKV-MDP for conditional McKean-Vlasov MDP, arises and is obtained here rigorously with a rate of convergence as the asymptotic problem of N-cooperative agents controlled by a social planner/influencer that observes the environment noises but not necessarily the individual states of the agents. We highlight the crucial role of relaxed controls and randomization hypothesis for this class of models with respect to classical MDP theory. We prove the correspondence between CMKV-MDP and a general lifted MDP on the space of probability measures, and establish the dynamic programming Bellman fixed point equation satisfied by the value function, as well as the existence of-optimal randomized feedback controls. The arguments of proof involve an original measurable optimal coupling for the Wasserstein distance. This provides a procedure for learning strategies in a large population of interacting collaborative agents. MSC Classification: 90C40, 49L20.

math.OC↗

arXiv2021

Neural networks-based algorithms for stochastic control and PDEs in finance

This paper presents machine learning techniques and deep reinforcement learningbased algorithms for the efficient resolution of nonlinear partial differential equations and dynamic optimization problems arising in investment decisions and derivative pricing in financial engineering. We survey recent results in the literature, present new developments, notably in the fully nonlinear case, and compare the different schemes illustrated by numerical tests on various financial applications. We conclude by highlighting some future research directions.

math.OC↗

arXiv2021

Deep neural networks algorithms for stochastic control problems on finite horizon: convergence analysis

This paper develops algorithms for high-dimensional stochastic control problems based on deep learning and dynamic programming. Unlike classical approximate dynamic programming approaches, we first approximate the optimal policy by means of neural networks in the spirit of deep reinforcement learning, and then the value function by Monte Carlo regression. This is achieved in the dynamic programming recursion by performance or hybrid iteration, and regress now methods from numerical probabilities. We provide a theoretical justification of these algorithms. Consistency and rate of convergence for the control and value function estimates are analyzed and expressed in terms of the universal approximation error of the neural networks, and of the statistical error when estimating network function, leaving aside the optimization error. Numerical results on various applications are presented in a companion paper (arxiv.org/abs/1812.05916) and illustrate the performance of the proposed algorithms.

math.PR↗

arXiv2021

Markowitz portfolio selection for multivariate affine and quadratic Volterra models

This paper concerns portfolio selection with multiple assets under rough covariance matrix. We investigate the continuous-time Markowitz mean-variance problem for a multivariate class of affine and quadratic Volterra models. In this incomplete non-Markovian and non-semimartingale market framework with unbounded random coefficients, the optimal portfolio strategy is expressed by means of a Riccati backward stochastic differential equation (BSDE). In the case of affine Volterra models, we derive explicit solutions to this BSDE in terms of multi-dimensional Riccati-Volterra equations. This framework includes multivariate rough Heston models and extends the results of \cite{han2019mean}. In the quadratic case, we obtain new analytic formulae for the the Riccati BSDE and we establish their link with infinite dimensional Riccati equations. This covers rough Stein-Stein and Wishart type covariance models. Numerical results on a two dimensional rough Stein-Stein model illustrate the impact of rough volatilities and stochastic correlations on the optimal Markowitz strategy. In particular for positively correlated assets, we find that the optimal strategy in our model is a `buy rough sell smooth' one.

math.OC↗

arXiv2020

Linear--Quadratic control for a class of stochastic Volterra equations: solvability and approximation

We provide an exhaustive treatment of Linear-Quadratic control problems for a class of stochastic Volterra equations of convolution type, whose kernels are Laplace transforms of certain signed matrix measures which are not necessarily finite. These equations are in general neither Markovian nor semimartingales, and include the fractional Brownian motion with Hurst index smaller than $1/2$ as a special case. We establish the correspondence of the initial problem with a possibly infinite dimensional Markovian one in a Banach space, which allows us to identify the Markovian controlled state variables. Using a refined martingale verification argument combined with a squares completion technique, we prove that the value function is of linear quadratic form in these state variables with a linear optimal feedback control, depending on non-standard Banach space valued Riccati equations. Furthermore, we show that the value function of the stochastic Volterra optimization problem can be approximated by that of conventional finite dimensional Markovian Linear--Quadratic problems, which is of crucial importance for numerical implementation.

math.OC↗

arXiv2020

Discrete-time portfolio optimization under maximum drawdown constraint with partial information and deep learning resolution

We study a discrete-time portfolio selection problem with partial information and maxi\-mum drawdown constraint. Drift uncertainty in the multidimensional framework is modeled by a prior probability distribution. In this Bayesian framework, we derive the dynamic programming equation using an appropriate change of measure, and obtain semi-explicit results in the Gaussian case. The latter case, with a CRRA utility function is completely solved numerically using recent deep learning techniques for stochastic optimal control problems. We emphasize the informative value of the learning strategy versus the non-learning one by providing empirical performance and sensitivity analysis with respect to the uncertainty of the drift. Furthermore, we show numerical evidence of the close relationship between the non-learning strategy and a no short-sale constrained Merton problem, by illustrating the convergence of the former towards the latter as the maximum drawdown constraint vanishes.

q-fin.PM↗

arXiv2020

Equilibrium price in intraday electricity markets

We formulate an equilibrium model of intraday trading in electricity markets. Agents face balancing constraints between their customers consumption plus intraday sales and their production plus intraday purchases. They have continuously updated forecast of their customers consumption at maturity with decreasing volatility error. Forecasts are prone to idiosyncratic noise as well as common noise (weather). Agents production capacities are subject to independent random outages, which are each modelled by a Markov chain. The equilibrium price is defined as the price that minimises trading cost plus imbalance cost of each agent and satisfies the usual market clearing condition. Existence and uniqueness of the equilibrium are proved, and we show that the equilibrium price and the optimal trading strategies are martingales. The main economic insights are the following. (i) When there is no uncertainty on generation, it is shown that the market price is a convex combination of forecasted marginal cost of each agent, with deterministic weights. Furthermore, the equilibrium market price follows Almgren and Chriss's model and we identify the fundamental part as well as the permanent market impact. It turns out that heterogeneity across agents is a necessary condition for the Samuelson's effect to hold. (ii) When there is production uncertainty, the price volatility becomes stochastic but converges to the case without production uncertainty when the number of agents increases to infinity. Further, on a two-agent case, we show that the potential outages of a low marginal cost producer reduces her sales position.

q-fin.CP↗

arXiv2020

Mean-variance portfolio selection with tracking error penalization

This paper studies a variation of the continuous-time mean-variance portfolio selection where a tracking-error penalization is added to the mean-variance criterion. The tracking error term penalizes the distance between the allocation controls and a reference portfolio with same wealth and fixed weights. Such consideration is motivated as follows: (i) On the one hand, it is a way to robustify the mean-variance allocation in case of misspecified parameters, by "fitting" it to a reference portfolio that can be agnostic to market parameters; (ii) On the other hand, it is a procedure to track a benchmark and improve the Sharpe ratio of the resulting portfolio by considering a mean-variance criterion in the objective function. This problem is formulated as a McKean-Vlasov control problem. We provide explicit solutions for the optimal portfolio strategy and asymptotic expansions of the portfolio strategy and efficient frontier for small values of the tracking error parameter. Finally, we compare the Sharpe ratios obtained by the standard mean-variance allocation and the penalized one for four different reference portfolios: equal-weights, minimum-variance, equal risk contributions and shrinking portfolio. This comparison is done on a simulated misspecified model, and on a backtest performed with historical data. Our results show that in most cases, the penalized portfolio outperforms in terms of Sharpe ratio both the standard mean-variance and the reference portfolio.

q-fin.CP↗

arXiv2020

Deep backward schemes for high-dimensional nonlinear PDEs

We propose new machine learning schemes for solving high dimensional nonlinear partial differential equations (PDEs). Relying on the classical backward stochastic differential equation (BSDE) representation of PDEs, our algorithms estimate simultaneously the solution and its gradient by deep neural networks. These approximations are performed at each time step from the minimization of loss functions defined recursively by backward induction. The methodology is extended to variational inequalities arising in optimal stopping problems. We analyze the convergence of the deep learning schemes and provide error estimates in terms of the universal approximation of neural networks. Numerical results show that our algorithms give very good results till dimension 50 (and certainly above), for both PDEs and variational inequalities problems. For the PDEs resolution, our results are very similar to those obtained by the recent method in \cite{weinan2017deep} when the latter converges to the right solution or does not diverge. Numerical tests indicate that the proposed methods are not stuck in poor local minimaas it can be the case with the algorithm designed in \cite{weinan2017deep}, and no divergence is experienced. The only limitation seems to be due to the inability of the considered deep neural networks to represent a solution with a too complex structure in high dimension.

math.PR↗

arXiv2020

Algorithmic trading in a microstructural limit order book model

We propose a microstructural modeling framework for studying optimal market making policies in a FIFO (first in first out) limit order book (LOB). In this context, the limit orders, market orders, and cancel orders arrivals in the LOB are modeled as Cox point processes with intensities that only depend on the state of the LOB. These are high-dimensional models which are realistic from a micro-structure point of view and have been recently developed in the literature. In this context, we consider a market maker who stands ready to buy and sell stock on a regular and continuous basis at a publicly quoted price, and identifies the strategies that maximize her P\&L penalized by her inventory. We apply the theory of Markov Decision Processes and dynamic programming method to characterize analytically the solutions to our optimal market making problem. The second part of the paper deals with the numerical aspect of the high-dimensional trading problem. We use a control randomization method combined with quantization method to compute the optimal strategies. Several computational tests are performed on simulated data to illustrate the efficiency of the computed optimal strategy. In particular, we simulated an order book with constant/ symmet-ric/ asymmetrical/ state dependent intensities, and compared the computed optimal strategy with naive strategies. Some codes are available on https://github.com/comeh.

q-fin.TR↗

arXiv2019

A McKean-Vlasov approach to distributed electricity generation development

This paper analyses the interaction between centralised carbon emissive technologies and distributed intermittent non-emissive technologies. In our model, there is a representative consumer who can satisfy her electricity demand by investing in distributed generation (solar panels) and by buying power from a centralised firm at a price the firm sets. Distributed generation is intermittent and induces an externality cost to the consumer. The firm provides non-random electricity generation subject to a carbon tax and to transmission costs. The objective of the consumer is to satisfy her demand while mini\-mising investment costs, payments to the firm and intermittency costs. The objective of the firm is to satisfy the consumer's residual demand while minimising investment costs, demand deviation costs, and maximising the payments from the consumer. We formulate the investment decisions as McKean-Vlasov control problems with stochastic coefficients. We provide explicit, price model-free solutions to the optimal decision problems faced by each player, the solution of the Pareto optimum, and the Stackelberg equilibrium where the firm is the leader. We find that, from the social planner's point of view, the carbon tax or transmission costs are necessary to justify a positive share of distributed capacity in the long-term, whatever the respective investment costs of both technologies are. The Stackelberg equilibrium is far from the Pareto equilibrium and leads to an over-investment in distributed energy and to a much higher price for centralised energy.

math.OC↗

arXiv2018

Bayesian learning for the Markowitz portfolio selection problem

We study the Markowitz portfolio selection problem with unknown drift vector in the multidimensional framework. The prior belief on the uncertain expected rate of return is modeled by an arbitrary probability law, and a Bayesian approach from filtering theory is used to learn the posterior distribution about the drift given the observed market data of the assets. The Bayesian Markowitz problem is then embedded into an auxiliary standard control problem that we characterize by a dynamic programming method and prove the existence and uniqueness of a smooth solution to the related semi-linear partial differential equation (PDE). The optimal Markowitz portfolio strategy is explicitly computed in the case of a Gaussian prior distribution. Finally, we measure the quantitative impact of learning, updating the strategy from observed data, compared to non-learning, using a constant drift in an uncertain context, and analyze the sensitivity of the value of information w.r.t. various relevant parameters of our model.

q-fin.PM↗

arXiv2018

A Weak Martingale Approach to Linear-Quadratic McKean-Vlasov Stochastic Control Problems

We propose a simple and original approach for solving linear-quadratic mean-field stochastic control problems. We study both finite-horizon and infinite-horizon pro\-blems, and allow notably some coefficients to be stochastic. Extension to the common noise case is also addressed. Our method is based on a suitable version of the martingale formulation for verification theorems in control theory. The optimal control involves the solution to a system of Riccati ordinary differential equations and to a linear mean-field backward stochastic differential equation; existence and uniqueness conditions are provided for such a system. Finally, we illustrate our results through an application to the production of an exhaustible resource. MSC Classification: 49N10, 49L20, 93E20.

math.PR↗