arXiv subjects

Zhiming Ma

Publications and source records attributed to Zhiming Ma.

56 records · Page 4Linked to original sources

arXiv2020

Constructing Basis Path Set by Eliminating Path Dependency

The way the basis path set works in neural network remains mysterious, and the generalization of newly appeared G-SGD algorithm to more practical network is hindered. The Basis Path Set Searching problem is formulated from the perspective of graph theory, to find the basis path set in a regular complicated neural network. Our paper aims to discover the underlying cause of path dependency between two independent substructures. Algorithm DEAH is designed to solve the Basis Path Set Searching problem by eliminating such path dependency. The path subdivision chain is proposed to effectively eliminate the path dependency inside the chain and between chains. The theoretical proofs and analysis of polynomial time complexity are presented. The paper therefore provides one methodology to find the basis path set in a more general neural network, which offers theoretical and algorithmic support for the application of G-SGD algorithm in more practical scenarios.

cs.DS↗

arXiv2020

Robust Reinforcement Learning with Wasserstein Constraint

Robust Reinforcement Learning aims to find the optimal policy with some extent of robustness to environmental dynamics. Existing learning algorithms usually enable the robustness through disturbing the current state or simulating environmental parameters in a heuristic way, which lack quantified robustness to the system dynamics (i.e. transition probability). To overcome this issue, we leverage Wasserstein distance to measure the disturbance to the reference transition kernel. With Wasserstein distance, we are able to connect transition kernel disturbance to the state disturbance, i.e. reduce an infinite-dimensional optimization problem to a finite-dimensional risk-aware problem. Through the derived risk-aware optimal Bellman equation, we show the existence of optimal robust policies, provide a sensitivity analysis for the perturbations, and then design a novel robust learning algorithm--Wasserstein Robust Advantage Actor-Critic algorithm (WRAAC). The effectiveness of the proposed algorithm is verified in the Cart-Pole environment.

cs.LG↗