arXiv subjects

Nan Jiang

Publications and source records attributed to Nan Jiang.

At least 271 records · Page 15Linked to original sources

arXiv2017

Markov Decision Processes with Continuous Side Information

We consider a reinforcement learning (RL) setting in which the agent interacts with a sequence of episodic MDPs. At the start of each episode the agent has access to some side-information or context that determines the dynamics of the MDP for that episode. Our setting is motivated by applications in healthcare where baseline measurements of a patient at the start of a treatment episode form the context that may provide information about how the patient might respond to treatment decisions. We propose algorithms for learning in such Contextual Markov Decision Processes (CMDPs) under an assumption that the unobserved MDP parameters vary smoothly with the observed context. We also give lower and upper PAC bounds under the smoothness assumption. Because our lower bound has an exponential dependence on the dimension, we consider a tractable linear setting where the context is used to create linear combinations of a finite set of MDPs. For the linear setting, we give a PAC learning algorithm based on KWIK learning techniques.

stat.ML↗

arXiv2017

Repeated Inverse Reinforcement Learning

We introduce a novel repeated Inverse Reinforcement Learning problem: the agent has to act on behalf of a human in a sequence of tasks and wishes to minimize the number of tasks that it surprises the human by acting suboptimally with respect to how the human would have acted. Each time the human is surprised, the agent is provided a demonstration of the desired behavior by the human. We formalize this problem, including how the sequence of tasks is chosen, in a few different ways and provide some foundational results.

cs.AI↗

arXiv2017

A higher-order ensemble/proper orthogonal decomposition method for the nonstationary Navier-Stokes equations

Partial differential equations (PDE) often involve parameters, such as viscosity or density. An analysis of the PDE may involve considering a large range of parameter values, as occurs in uncertainty quantification, control and optimization, inference, and several statistical techniques. The solution for even a single case may be quite expensive; whereas parallel computing may be applied, this reduces the total elapsed time but not the total computational effort. In the case of flows governed by the Navier-Stokes equations, a method has been devised for computing an ensemble of solutions. Recently, a reduced-order model derived from a proper orthogonal decomposition (POD) approach was incorporated into a first-order accurate in time version of the ensemble algorithm. In this work, we expand on that work by incorporating the POD reduced order model into a second-order accurate ensemble algorithm. Stability and convergence results for this method are updated to account for the POD/ROM approach. Numerical experiments illustrate the accuracy and efficiency of the new approach.

math.NA↗

arXiv2017

An efficient, partitioned ensemble algorithm for simulating ensembles of evolutionary MHD flows at low magnetic Reynolds number

Studying the propagation of uncertainties in a nonlinear dynamical system usually involves generating a set of samples in the stochastic parameter space and then repeated simulations with different sampled parameters. The main difficulty faced in the process is the excessive computational cost. In this paper, we present an efficient, partitioned ensemble algorithm to determine multiple realizations of a reduced Magnetohydrodynamics (MHD) system, which models MHD flows at low magnetic Reynolds number. The algorithm decouples the fully coupled problem into two smaller sub-physics problems, which reduces the size of the linear systems that to be solved and allows the use of optimized codes for each sub-physics problem. Moreover, the resulting coefficient matrices are the same for all realizations at each time step, which allows faster computation of all realizations and significant savings in computational cost. We prove this algorithm is first order accurate and long time stable under a time step condition. Numerical examples are provided to verify the theoretical results and demonstrate the efficiency of the algorithm.

math.NA↗

arXiv2017

A second-order time-stepping scheme for simulating ensembles of parameterized flow problems

We consider settings for which one needs to perform multiple flow simulations based on the Navier-Stokes equations, each having different values for the physical parameters and/or different initial condition data, boundary conditions data, and/or forcing functions. For such settings, we propose a second-order time accurate ensemble-based method that to simulate the whole set of solutions, requires, at each time step, the solution of only a single linear system with multiple right-hand-side vectors. Rigorous analyses are given proving the conditional stability and error estimates for the proposed algorithm. Numerical experiments are provided that illustrate the analyses.

math.NA↗

arXiv2017

An efficient algorithm for simulating ensembles of parameterized flow problems

Many applications of computational fluid dynamics require multiple simulations of a flow under different input conditions. In this paper, a numerical algorithm is developed to efficiently determine a set of such simulations in which the individually independent members of the set are subject to different viscosity coefficients, initial conditions, and/or body forces. The proposed scheme applied to the flow ensemble leads to need to solve a single linear system with multiple right-hand sides, and thus is computationally more efficient than solving for all the simulations separately. We show that the scheme is nonlinearly and long-term stable under certain conditions on the time-step size and a parameter deviation ratio. Rigorous numerical error estimate shows the scheme is of first-order accuracy in time and optimally accurate in space. Several numerical experiments are presented to illustrate the theoretical results.

math.NA↗

arXiv2017

Predicting Atomic Decay Rates Using an Informational-Entropic Approach

We show that a newly proposed Shannon-like entropic measure of shape complexity applicable to spatially-localized or periodic mathematical functions known as configurational entropy (CE) can be used as a predictor of spontaneous decay rates for one-electron atoms. The CE is constructed from the Fourier transform of the atomic probability density. For the hydrogen atom with degenerate states labeled with the principal quantum number n, we obtain a scaling law relating the n-averaged decay rates to the respective CE. The scaling law allows us to predict the n-averaged decay rate without relying on the traditional computation of dipole matrix elements. We tested the predictive power of our approach up to n=20, obtaining an accuracy better than 3.7% within our numerical precision, as compared to spontaneous decay tables listed in the literature.

physics.atom-ph↗

arXiv2017

A Novel Quantum Image Compression Method Based on JPEG

Quantum image processing has been a hot topic. The first step of it is to store an image into qubits, which is called quantum image preparation. Different quantum image representations may have different preparation methods. In this paper, we use GQIR (the generalized quantum image representation) to represent an image, and try to decrease the operations used in preparation, which is also known as quantum image compression. Our compression scheme is based on JPEG (named from its inventor: the Joint Photographic Experts Group) --- the most widely used method for still image compression in classical computers. We input the quantized JPEG coefficients into qubits and then convert them into pixel values. Theoretical analysis and experimental results show that the compression ratio of our scheme is obviously higher than that of the previous compression method.

quant-ph↗

arXiv2016

Contextual Decision Processes with Low Bellman Rank are PAC-Learnable

This paper studies systematic exploration for reinforcement learning with rich observations and function approximation. We introduce a new model called contextual decision processes, that unifies and generalizes most prior settings. Our first contribution is a complexity measure, the Bellman rank, that we show enables tractable learning of near-optimal behavior in these processes and is naturally small for many well-studied reinforcement learning settings. Our second contribution is a new reinforcement learning algorithm that engages in systematic exploration to learn contextual decision processes with low Bellman rank. Our algorithm provably learns near-optimal behavior with a number of samples that is polynomial in all relevant parameters but independent of the number of unique observations. The approach uses Bellman error minimization with optimistic exploration and provides new insights into efficient exploration for reinforcement learning with function approximation.

cs.LG↗

arXiv2016

Word Embedding based Correlation Model for Question/Answer Matching

With the development of community based question answering (Q&A) services, a large scale of Q&A archives have been accumulated and are an important information and knowledge resource on the web. Question and answer matching has been attached much importance to for its ability to reuse knowledge stored in these systems: it can be useful in enhancing user experience with recurrent questions. In this paper, we try to improve the matching accuracy by overcoming the lexical gap between question and answer pairs. A Word Embedding based Correlation (WEC) model is proposed by integrating advantages of both the translation model and word embedding, given a random pair of words, WEC can score their co-occurrence probability in Q&A pairs and it can also leverage the continuity and smoothness of continuous space word representation to deal with new pairs of words that are rare in the training parallel text. An experimental study on Yahoo! Answers dataset and Baidu Zhidao dataset shows this new method's promising potential.

cs.CL↗

arXiv2016

An Ensemble-Proper Orthogonal Decomposition Method for the Nonstationary Navier-Stokes Equations

The definition of partial differential equation (PDE) models usually involves a set of parameters whose values may vary over a wide range. The solution of even a single set of parameter values may be quite expensive. In many cases, e.g., optimization, control, uncertainty quantification, and other settings, solutions are needed for many sets of parameter values. We consider the case of the time-dependent Navier-Stokes equations for which a recently developed ensemble-based method allows for the efficient determination of the multiple solutions corresponding to many parameter sets. The method uses the average of the multiple solutions at any time step to define a linear set of equations that determines the solutions at the next time step. To significantly further reduce the costs of determining multiple solutions of the Navier-Stokes equations, we incorporate a proper orthogonal decomposition (POD) reduced-order model into the ensemble-based method. The stability and convergence results for the ensemble-based method are extended to the ensemble-POD approach. Numerical experiments are provided that illustrate the accuracy and efficiency of computations determined using the new approach.

math.NA↗

arXiv2016

Analysis and approximation of a fractional Laplacian-based closure model for turbulent flows and its connection to Richardson pair dispersion

We study a turbulence closure model in which the fractional Laplacian $(-Δ)^α$ of the velocity field represents the turbulence diffusivity. We investigate the energy spectrum of the model by applying Pao's energy transfer theory. For the case $α=1/3$, the corresponding power law of the energy spectrum in the inertial range has a correction exponent on the regular Kolmogorov -5/3 scaling exponent. For this case, this model represents Richardson's particle pair-distance superdiffusion of a fully developed homogeneous turbulent flow as well as Lévy jumps that lead to the superdiffusion. For other values of $α$, the power law of the energy spectrum is consistent with the regular Kolmogorov -5/3 scaling exponent. We also propose and study a modular time-stepping algorithm in semi-discretized form. The algorithm is minimally intrusive to a given legacy code for solving Navier-Stokes equations by decoupling the local part and nonlocal part of the equations for the unknowns. We prove the algorithm is unconditionally stable and unconditionally, first-order convergent. We also derive error estimates for full discretizations of the model which, in addition to the time stepping algorithm, involves a finite element spatial discretization and a domain truncation approximation to the range of the fractional Laplacian.

math.NA↗

arXiv2016

Quantum Image Matching

Quantum image processing (QIP) means the quantum based methods to speed up image processing algorithms. Many quantum image processing schemes claim that their efficiency are theoretically higher than their corresponding classical schemes. However, most of them do not consider the problem of measurement. As we all know, measurement will lead to collapse. That is to say, executing the algorithm once, users can only measure the final state one time. Therefore, if users want to regain the results (the processed images), they must execute the algorithms many times and then measure the final state many times to get all the pixels' values. If the measurement process is taken into account, whether or not the algorithms are really efficient needs to be reconsidered. In this paper, we try to solve the problem of measurement and give a quantum image matching algorithm. Unlike most of the QIP algorithms, our scheme interests only one pixel (the target pixel) instead of the whole image. It modifies the probability of pixels based on Grover's algorithm to make the target pixel to be measured with higher probability, and the measurement step is executed only once. An example is given to explain the algorithm more vividly. Complexity analysis indicates that the quantum scheme's complexity is $O(2^{n})$ in contradistinction to the classical scheme's complexity $O(2^{2n+2m})$, where $m$ and $n$ are integers related to the size of images.

quant-ph↗

arXiv2016

Doubly Robust Off-policy Value Evaluation for Reinforcement Learning

We study the problem of off-policy value evaluation in reinforcement learning (RL), where one aims to estimate the value of a new policy based on data collected by a different policy. This problem is often a critical step when applying RL in real-world problems. Despite its importance, existing general methods either have uncontrolled bias or suffer high variance. In this work, we extend the doubly robust estimator for bandits to sequential decision-making problems, which gets the best of both worlds: it is guaranteed to be unbiased and can have a much lower variance than the popular importance sampling estimators. We demonstrate the estimator's accuracy in several benchmark problems, and illustrate its use as a subroutine in safe policy improvement. We also provide theoretical results on the hardness of the problem, and show that our estimator can match the lower bound in certain scenarios.

cs.LG↗

arXiv2015

Stability Bounds on Compact Astrophysical Objects from Information-Entropic Measure

We obtain bounds on the stability of various self-gravitating astrophysical objects using a new measure of shape complexity known as configurational entropy. We apply the method to Newtonian polytropes, neutron stars with an Oppenheimer-Volkoff equation of state, and to self-gravitating configurations of complex scalar field (boson stars) with different self-couplings, showing that the critical stability region of these stellar configurations obtained from traditional perturbation methods correlates well with critical points of the configurational entropy with accuracy of a few percent or better.

gr-qc↗

arXiv2015

The electromagnetic decays of the charmed and bottom baryons in chiral perturbation theory

We have investigated the electromagnetic decays of the antitriplet and sextet charmed baryon systems with $J^P= \frac{1}{2}^+, \frac{3}{2}^+$ in the framework of the heavy baryon chiral perturbation theory. We first construct the chiral Lagrangians at $O(p^2)$ and $O(p^3)$. Then we calculate the electromagnetic (EM) decay amplitudes of the charmed baryon systems up to $O(p^3)$. With the help of the quark model, we estimate the low energy constants. The numerical results of the EM decay widths show good convergence of the chiral expansion. We notice that the two neutral EM decay processes $Ξ_c'^0\rightarrowγ+Ξ_c^0$ and ${Ξ_c^*}'^0\rightarrowγ+Ξ_c^0$ are strongly suppressed by the SU(3) U-spin flavor symmetry. With the same formalism, we also estimate the EM decay widths of the bottomed baryons. The EM decay widths of the heavy baryons may be measured at facilities such as LHCb and JPARC. The explicit chiral structures of the heavy baryon decay amplitudes derived in this work may be useful to the possible chiral extrapolations of the future lattice simulations of these EM decay amplitudes.

hep-ph↗

arXiv2015

Algorithms and Models for Turbulence Not at Statistical Equilibrium

Standard eddy viscosity models, while robust, cannot represent backscatter and have severe difficulties with complex turbulence not at statistical equilibrium. This report gives a new derivation of eddy viscosity models from an equation for the evolution of variance in a turbulent flow. The new derivation also shows how to correct eddy viscosity models. The report proves the corrected models preserve important features of the true Reynolds stresses. It gives algorithms for their discretization including a minimally invasive modular step to adapt an eddy viscosity code to the extended models. A numerical test is given with the usual and over diffusive Smagorinsky model. The correction (scaled by $10^{-8}$ ) does successfully exhibit intermittent backscatter.

math.NA↗

arXiv2014

Mass and axial charge of heavy baryons

We investigate the antitriplet and sextet heavy baryon systems with $J^P= \frac{1}{2}^+, \frac{3}{2}^+$ in the framework of the heavy baryon chiral perturbation theory. We first calculate the chiral corrections to the heavy baryon mass from the SU(3) flavor breaking effect up to $O(p^3)$. Then we extend the same formalism to calculate the chiral corrections to the axial charges of the heavy baryons in the isospin symmetry limit.

hep-ph↗