arXiv subjects

George Yin

Publications and source records attributed to George Yin.

At least 37 records · Page 2Linked to original sources

arXiv2021

Multi-kernel Passive Stochastic Gradient Algorithms and Transfer Learning

This paper develops a novel passive stochastic gradient algorithm. In passive stochastic approximation, the stochastic gradient algorithm does not have control over the location where noisy gradients of the cost function are evaluated. Classical passive stochastic gradient algorithms use a kernel that approximates a Dirac delta to weigh the gradients based on how far they are evaluated from the desired point. In this paper we construct a multi-kernel passive stochastic gradient algorithm. The algorithm performs substantially better in high dimensional problems and incorporates variance reduction. We analyze the weak convergence of the multi-kernel algorithm and its rate of convergence. In numerical examples, we study the multi-kernel version of the passive least mean squares (LMS) algorithm for transfer learning to compare the performance with the classical passive version.

cs.LG↗

arXiv2021

Langevin Dynamics for Adaptive Inverse Reinforcement Learning of Stochastic Gradient Algorithms

Inverse reinforcement learning (IRL) aims to estimate the reward function of optimizing agents by observing their response (estimates or actions). This paper considers IRL when noisy estimates of the gradient of a reward function generated by multiple stochastic gradient agents are observed. We present a generalized Langevin dynamics algorithm to estimate the reward function $R(θ)$; specifically, the resulting Langevin algorithm asymptotically generates samples from the distribution proportional to $\exp(R(θ))$. The proposed IRL algorithms use kernel-based passive learning schemes. We also construct multi-kernel passive Langevin algorithms for IRL which are suitable for high dimensional data. The performance of the proposed IRL algorithms are illustrated on examples in adaptive Bayesian learning, logistic regression (high dimensional problem) and constrained Markov decision processes. We prove weak convergence of the proposed IRL algorithms using martingale averaging methods. We also analyze the tracking performance of the IRL algorithms in non-stationary environments where the utility function $R(θ)$ jump changes over time as a slow Markov chain.

cs.LG↗

arXiv2021

Large Deviations Principles for Langevin Equations in Random Environment and Applications

In contrast to the study of Langevin equations in a homogeneous environment in the literature, the study on Langevin equations in randomly-varying environments is relatively scarce. Almost all the existing works require random environments to have a specific formulation that is independent of the systems. This paper aims to consider large deviations principles (LDPs) of Langevin equations involving a random environment that is a process taking value in a measurable space and that is allowed to interact with the systems, without specified formulation on the random environment. Examples and applications to statistical physics are provided. Our formulation of the random environment presents the main challenges and requires new approaches. Our approach stems from the intuition of the Smoluchowski-Kramers approximation. The techniques developed in this paper focus on the relation between the solutions of the second-order equations and the associate first-order equations. We obtain the desired LDPs by showing a family of processes enjoy the exponential tightness and local LDPs with an appropriate rate function.

math.PR↗

arXiv2021

Optimal Control and Numerical Methods for Hybrid Stochastic SIS Models

This work focuses on optimal controls of a class of stochastic SIS epidemic models under regime switching. By assuming that a decision maker can either influence the infectivity period or isolate infected individuals, our aim is to minimize the expected discounted cost due to illness, medical treatment, and the adverse effect on the society. In addition, a model with the incorporation of vaccination is proposed. Numerical schemes are developed by approximating the continuous-time dynamics using Markov chain approximation methods. It is demonstrated that the approximation schemes converge to the optimal strategy as the mesh size goes to zero. Numerical examples are provided to illustrate our results.

math.OC↗

arXiv2020

Adaptive Non-reversible Stochastic Gradient Langevin Dynamics

It is well known that adding any skew symmetric matrix to the gradient of Langevin dynamics algorithm results in a non-reversible diffusion with improved convergence rate. This paper presents a gradient algorithm to adaptively optimize the choice of the skew symmetric matrix. The resulting algorithm involves a non-reversible diffusion algorithm cross coupled with a stochastic gradient algorithm that adapts the skew symmetric matrix. The algorithm uses the same data as the classical Langevin algorithm. A weak convergence proof is given for the optimality of the choice of the skew symmetric matrix. The improved convergence rate of the algorithm is illustrated numerically in Bayesian learning and tracking examples.

cs.LG↗

arXiv2020

Solving A Class of Mean-Field LQG Problems

In this work, we study a class of mean-field linear quadratic Gaussian (LQG) problems. Under suitable conditions, explicit solutions of the distribution-dependent optimal control problems are obtained. Riccati systems are derived by directly solving the associated master equations. Some extensions on controls with partial observations are also considered.

math.PR↗

arXiv2020

Deep Filtering

This paper develops a deep learning method for linear and nonlinear filtering. The idea is to start with a nominal dynamic model and generate Monte Carlo sample paths. Then these samples are used to train a deep neutral network. A least square error is used as a loss function for network training. Then the resulting weights are applied to Monte Carlo sampl\ es from an actual dynamic model. The deep filter obtained in such a way compares favorably to the traditional Kalman filter in linear cases and the extended Kalman filter in nonlinear cases. Moreover, a switching model with jumps is studied to show the adaptiveness and power of our deep filtering method. A main advantage of deep filtering is its robustness when the nominal model and actual model differ. Another advantage of deep filtering is that real data can be used directly to train the deep neutral network. Therefore, one does not need to calibrate the model.

math.OC↗

arXiv2020

General Nonlinear Stochastic Systems Motivated by Chemostat Models: Complete Characterization of Long-Time Behavior, Optimal Controls, and Applications to Wastewater Treatment

The paper considers a chemostat model describing an activated sludge process in wastewater treatment. The model is assumed to be subject to environment noise in terms of both white noise and color noise. The paper fully characterizes the asymptotic behavior of the model that is a hybrid switching diffusion. We show that the long-term properties of the system can be classified using a value $λ$. More precisely, if $λ\leq 0$, the bacteria in the sewage will die out, which means that the process does not operate. If $λ>0$, the system has an invariant probability measure to which the transition probability of the solution process converges exponentially fast. One of the distinctive contributions of this paper is that the critical case $λ=0$ is considered. Numerical examples are given to illustrate our results.

math.PR↗

arXiv2019

Analysis of A Spatially Inhomogeneous Stochastic Partial Differential Equation Epidemic Model

This work proposes and analyzes a family of spatially inhomogeneous epidemic models. This is our first effort to use stochastic partial differential equations (SPDEs) to model epidemic dynamics with spatial variations and environmental noise. After setting up the problem, existence and uniqueness of solutions of the underlying SPDEs are examined. Then definitions of permanence and extinction are given. Certain sufficient conditions are provided for the permanence and extinction. Our hope is that this paper will open up windows for investigation of epidemic models from a new angle.

math.DS↗

arXiv2019

Time-Inconsistent Problems for Controlled Markov Chains with Distribution-Dependent Costs: Equilibrium Solutions

This paper focuses on a class of continuous-time controlled Markov chains with time-inconsistent and distribution-dependent cost functional (in some appropriate sense). A new definition of time-inconsistent distribution-dependent equilibrium in closed-loop sense is given and its existence and uniqueness have been established. Because of the time-inconsistency, it is proved that the equilibrium is locally optimal in an appropriate sense. Moreover, it has been shown that our problem is essentially equivalent to an infinite-player mean-field game with time-inconsistent cost.

math.OC↗

arXiv2019

On Laws of Large Numbers for Systems with Mean-Field Interactions and Markovian Switching

Focusing on stochastic systems arising in mean-field models, the systems under consideration belong to the class of switching diffusions, in which continuous dynamics and discrete events coexist and interact. The discrete events are modeled by a continuous-time Markov chain. Different from the usual switching diffusions, the systems include mean-field interactions. Our effort is devoted to obtaining laws of large numbers for the underlying systems. One of the distinct features of the paper is the limit of the empirical measures is not deterministic but a random measure depending on the history of the Markovian switching process. A main difficulty is that the standard martingale approach cannot be used to characterize the limit because of the coupling due to the random switching process. In this paper, in contrast to the classical approach, the limit is characterized as the conditional distribution (given the history of the switching process) of the solution to a stochastic McKean-Vlasov differential equation with Markovian switching.

math.PR↗

arXiv2018

Regime-Switching Jump Diffusions with Non-Lipschitz Coefficients and Countably Many Switching States: Existence and Uniqueness, Feller, and Strong Feller Properties

This work focuses on a class of regime-switching jump diffusion processes, which is a two component Markov processes $(X(t),Λ(t))$, where $Λ(t)$ is a component representing discrete events taking values in a countably infinite set. Considering the corresponding stochastic differential equations, our main focus is on treating those with non-Lipschitz coefficients. We first show that there exists a unique strong solution to the corresponding stochastic differential equation. Then Feller and strong Feller properties are investigated.

math.PR↗

arXiv2018

Harvesting of interacting stochastic populations

We analyze the optimal harvesting problem for an ecosystem of species that experience environmental stochasticity. Our work generalizes the current literature significantly by taking into account non-linear interactions between species, state-dependent prices, and species injections. The key generalization is making it possible to not only harvest, but also `seed' individuals into the ecosystem. This is motivated by how fisheries and certain endangered species are controlled. The harvesting problem becomes finding the optimal harvesting-seeding strategy that maximizes the expected total income from the harvest minus the lost income from the species injections. Our analysis shows that new phenomena emerge due to the possibility of species injections. It is well-known that multidimensional harvesting problems are very hard to tackle. We are able to make progress, by characterizing the value function as a viscosity solution of the associated Hamilton-Jacobi-Bellman (HJB) equations. Moreover, we provide a verification theorem, which tells us that if a function has certain properties, then it will be the value function. This allows us to show heuristically, as was shown in Lungu and $Ø$ksendal (Bernoulli '01), that it is almost surely never optimal to harvest or seed from more than one population at a time. We approximate the continuous-time systems by Markov chains and show that the optimal harvesting-seeding strategies of the Markov chain approximations converge to the correct optimal harvesting strategy. This is used to provide numerical approximations to the optimal harvesting-seeding strategies and is a first step towards a full understanding of the intricacies of how one should harvest and seed interacting species. In particular, we look at three examples: one species modeled by a Verhulst-Pearl diffusion, two competing species and a two-species predator-prey system.

math.PR↗

arXiv2018

Properties of Switching Jump Diffusions: Maximum Principles and Harnack Inequalities

This work examines a class of switching jump diffusion processes. The main effort is devoted to proving the maximum principle and obtaining the Harnack inequalities. Compared with the diffusions and switching diffusions, the associated operators for switching jump diffusions are non-local, resulting in more difficulty in treating such systems. Our study is carried out by taking into consideration of the interplay of stochastic processes and the associated systems of integro-differential equations.

math.PR↗

arXiv2018

Recurrence and Ergodicity for A Class of Regime-Switching Jump Diffusions

This work develops asymptotic properties of a class of switching jump diffusion processes. The processes under consideration may be viewed as a number of jump diffusion processes modulated by a random switching mechanism. The underlying processes feature in the switching process depends on the jump diffusions. In this paper, conditions for recurrence and positive recurrence are derived. Ergodicity is examined in detail. Existence of invariant probability measures is proved.

math.PR↗

arXiv2018

Hybrid Competitive Lotka-Volterra Ecosystems: Countable Switching States and Two-time-scale Models

This work is concerned with competitive Lotka-Volterra model with Markov switching. A novelty of the contribution is that the Markov chain has a countable state space. Our main objective of the paper is to reduce the computational complexity by using the two-time-scale systems. Because existence and uniqueness as well as continuity of solutions for Lotka-Volterra ecosystems with Markovian switching in which the switching takes place in a countable set are not available, such properties are studied first. The two-time scale feature is highlighted by introducing a small parameter into the generator of the Markov chain. When the small parameter goes to 0, there is a limit system or reduced system. It is established in this paper that if the reduced system possesses certain properties such as permanence and extinction, etc., then the complex system also has the same properties when the parameter is sufficiently small. These results are obtained by using the perturbed Lyapunov function methods.

math.DS↗

arXiv2018

Approximation of A Class of Non-Zero-Sum Investment and Reinsurance Games for Regime-Switching Jump-Diffusion Models

This work develops an approximation procedure for a class of non-zero-sum stochastic differential investment and reinsurance games between two insurance companies. Both proportional reinsurance and excess-of loss reinsurance policies are considered. We develop numerical algorithms to obtain the Nash equilibrium by adopting the Markov chain approximation methodology and applying the dynamical programming principle for the nonlinear integro-differential Hamilton-Jacobi-Isaacs (HJI) equations. Furthermore, we establish the convergence of the approximation sequences and the approximation to the value functions. Numerical examples are presented to illustrate the applicability of the algorithms.

math.OC↗

arXiv2017

Stability of Regime-Switching Diffusion Systems with Discrete States Belonging to a Countable Set

This work focuses on stability of regime-switching diffusions consisting of continuous and discrete components, in which the discrete component switches in a countably infinite set and its switching rates at current time depend on the continuous component. In contrast to the existing approach, this work provides more practically viable approach with more feasible conditions for stability. A classical approach for asymptotic stabilityusing Lyapunov function techniques shows the Lyapunov function evaluated at the solution process goes to 0 as time $t\to \infty$. A distinctive feature of this paper is to obtain estimates of path-wise rates of convergence, which pinpoints how fast the aforementioned convergence to 0 taking place. Finally, some examples are given to illustrate our findings.

math.PR↗