Searcharxiv⌕ Search

arXiv subjects

Miao Liu

Publications and source records attributed to Miao Liu.

144 records · Page 8Linked to original sources
arXiv2018

Screening magnetic two-dimensional atomic crystals with nontrivial electronic topology

To date only a few two-dimensional (2D) magnetic crystals were experimentally confirmed, such as CrI3 and CrGeTe3, all with very low Curie temperatures (TC). High-throughput first-principles screening over a large set of materials yields 89 magnetic monolayers including 56 ferromagnetic (FM) and 33 antiferromagnetic compounds. Among them, 24 FM monolayers are promising candidates possessing TC higher than that of CrI3. High TC monolayers with fascinating electronic phases are identified: (i) quantum anomalous and valley Hall effects coexist in a single material RuCl3 or VCl3, leading to a valley-polarized quantum anomalous Hall state; (ii) TiBr3, Co2NiO6 and V2H3O5 are revealed to be half-metals. More importantly, a new type of fermion dubbed type-II Weyl ring is discovered in ScCl. Our work provides a database of 2D magnetic materials, which could guide experimental realization of high-temperature magnetic monolayers with exotic electronic states for future spintronics and quantum computing applications.

cond-mat.mtrl-sci↗
arXiv2018

Learning to Teach in Cooperative Multiagent Reinforcement Learning

Collective human knowledge has clearly benefited from the fact that innovations by individuals are taught to others through communication. Similar to human social groups, agents in distributed learning systems would likely benefit from communication to share knowledge and teach skills. The problem of teaching to improve agent learning has been investigated by prior works, but these approaches make assumptions that prevent application of teaching to general multiagent problems, or require domain expertise for problems they can apply to. This learning to teach problem has inherent complexities related to measuring long-term impacts of teaching that compound the standard multiagent coordination challenges. In contrast to existing works, this paper presents the first general framework and algorithm for intelligent agents to learn to teach in a multiagent environment. Our algorithm, Learning to Coordinate and Teach Reinforcement (LeCTR), addresses peer-to-peer teaching in cooperative multiagent reinforcement learning. Each agent in our approach learns both when and what to advise, then uses the received advice to improve local learning. Importantly, these roles are not fixed; these agents learn to assume the role of student and/or teacher at the appropriate moments, requesting and providing advice in order to improve teamwide performance and learning. Empirical comparisons against state-of-the-art teaching methods show that our teaching agents not only learn significantly faster, but also learn to coordinate in tasks where existing methods fail.

cs.MA↗
arXiv2018

Socially Aware Motion Planning with Deep Reinforcement Learning

For robotic vehicles to navigate safely and efficiently in pedestrian-rich environments, it is important to model subtle human behaviors and navigation rules (e.g., passing on the right). However, while instinctive to humans, socially compliant navigation is still difficult to quantify due to the stochasticity in people's behaviors. Existing works are mostly focused on using feature-matching techniques to describe and imitate human paths, but often do not generalize well since the feature values can vary from person to person, and even run to run. This work notes that while it is challenging to directly specify the details of what to do (precise mechanisms of human navigation), it is straightforward to specify what not to do (violations of social norms). Specifically, using deep reinforcement learning, this work develops a time-efficient navigation policy that respects common social norms. The proposed method is shown to enable fully autonomous navigation of a robotic vehicle moving at human walking speed in an environment with many pedestrians.

cs.RO↗
arXiv2018

Eigenoption Discovery through the Deep Successor Representation

Options in reinforcement learning allow agents to hierarchically decompose a task into subtasks, having the potential to speed up learning and planning. However, autonomously learning effective sets of options is still a major challenge in the field. In this paper we focus on the recently introduced idea of using representation learning methods to guide the option discovery process. Specifically, we look at eigenoptions, options obtained from representations that encode diffusive information flow in the environment. We extend the existing algorithms for eigenoption discovery to settings with stochastic transitions and in which handcrafted features are not available. We propose an algorithm that discovers eigenoptions while learning non-linear state representations from raw pixels. It exploits recent successes in the deep reinforcement learning literature and the equivalence between proto-value functions and the successor representation. We use traditional tabular domains to provide intuition about our approach and Atari 2600 games to demonstrate its potential.

cs.LG↗
arXiv2017

The Eigenoption-Critic Framework

Eigenoptions (EOs) have been recently introduced as a promising idea for generating a diverse set of options through the graph Laplacian, having been shown to allow efficient exploration. Despite its initial promising results, a couple of issues in current algorithms limit its application, namely: (1) EO methods require two separate steps (eigenoption discovery and reward maximization) to learn a control policy, which can incur a significant amount of storage and computation; (2) EOs are only defined for problems with discrete state-spaces and; (3) it is not easy to take the environment's reward function into consideration when discovering EOs. To addresses these issues, we introduce an algorithm termed eigenoption-critic (EOC) based on the Option-critic (OC) framework [Bacon17], a general hierarchical reinforcement learning (RL) algorithm that allows learning the intra-option policies simultaneously with the policy over options. We also propose a generalization of EOC to problems with continuous state-spaces through the Nyström approximation. EOC can also be seen as extending OC to nonstationary settings, where the discovered options are not tailored for a single task.

cs.AI↗
arXiv2017

Learning for Multi-robot Cooperation in Partially Observable Stochastic Environments with Macro-actions

This paper presents a data-driven approach for multi-robot coordination in partially-observable domains based on Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) and macro-actions (MAs). Dec-POMDPs provide a general framework for cooperative sequential decision making under uncertainty and MAs allow temporally extended and asynchronous action execution. To date, most methods assume the underlying Dec-POMDP model is known a priori or a full simulator is available during planning time. Previous methods which aim to address these issues suffer from local optimality and sensitivity to initial conditions. Additionally, few hardware demonstrations involving a large team of heterogeneous robots and with long planning horizons exist. This work addresses these gaps by proposing an iterative sampling based Expectation-Maximization algorithm (iSEM) to learn polices using only trajectory data containing observations, MAs, and rewards. Our experiments show the algorithm is able to achieve better solution quality than the state-of-the-art learning-based methods. We implement two variants of multi-robot Search and Rescue (SAR) domains (with and without obstacles) on hardware to demonstrate the learned policies can effectively control a team of distributed robots to cooperate in a partially observable stochastic environment.

cs.MA↗
arXiv2017

Semantic-level Decentralized Multi-Robot Decision-Making using Probabilistic Macro-Observations

Robust environment perception is essential for decision-making on robots operating in complex domains. Intelligent task execution requires principled treatment of uncertainty sources in a robot's observation model. This is important not only for low-level observations (e.g., accelerometer data), but also for high-level observations such as semantic object labels. This paper formalizes the concept of macro-observations in Decentralized Partially Observable Semi-Markov Decision Processes (Dec-POSMDPs), allowing scalable semantic-level multi-robot decision making. A hierarchical Bayesian approach is used to model noise statistics of low-level classifier outputs, while simultaneously allowing sharing of domain noise characteristics between classes. Classification accuracy of the proposed macro-observation scheme, called Hierarchical Bayesian Noise Inference (HBNI), is shown to exceed existing methods. The macro-observation scheme is then integrated into a Dec-POSMDP planner, with hardware experiments running onboard a team of dynamic quadrotors in a challenging domain where noise-agnostic filtering fails. To the best of our knowledge, this is the first demonstration of a real-time, convolutional neural net-based classification framework running fully onboard a team of quadrotors in a multi-robot decision-making domain.

cs.MA↗
arXiv2017

Scalable Accelerated Decentralized Multi-Robot Policy Search in Continuous Observation Spaces

This paper presents the first ever approach for solving \emph{continuous-observation} Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) and their semi-Markovian counterparts, Dec-POSMDPs. This contribution is especially important in robotics, where a vast number of sensors provide continuous observation data. A continuous-observation policy representation is introduced using Stochastic Kernel-based Finite State Automata (SK-FSAs). An SK-FSA search algorithm titled Entropy-based Policy Search using Continuous Kernel Observations (EPSCKO) is introduced and applied to the first ever continuous-observation Dec-POMDP/Dec-POSMDP domain, where it significantly outperforms state-of-the-art discrete approaches. This methodology is equally applicable to Dec-POMDPs and Dec-POSMDPs, though the empirical analysis presented focuses on Dec-POSMDPs due to their higher scalability. To improve convergence, an entropy injection policy search acceleration approach for both continuous and discrete observation cases is also developed and shown to improve convergence rates without degrading policy quality.

cs.MA↗
arXiv2017

Quickest Change Detection Approach to Optimal Control in Markov Decision Processes with Model Changes

Optimal control in non-stationary Markov decision processes (MDP) is a challenging problem. The aim in such a control problem is to maximize the long-term discounted reward when the transition dynamics or the reward function can change over time. When a prior knowledge of change statistics is available, the standard Bayesian approach to this problem is to reformulate it as a partially observable MDP (POMDP) and solve it using approximate POMDP solvers, which are typically computationally demanding. In this paper, the problem is analyzed through the viewpoint of quickest change detection (QCD), a set of tools for detecting a change in the distribution of a sequence of random variables. Current methods applying QCD to such problems only passively detect changes by following prescribed policies, without optimizing the choice of actions for long term performance. We demonstrate that ignoring the reward-detection trade-off can cause a significant loss in long term rewards, and propose a two threshold switching strategy to solve the issue. A non-Bayesian problem formulation is also proposed for scenarios where a Bayesian formulation cannot be defined. The performance of the proposed two threshold strategy is examined through numerical analysis on a non-stationary MDP task, and the strategy outperforms the state-of-the-art QCD methods in both Bayesian and non-Bayesian settings.

stat.AP↗
arXiv2016

Decentralized Non-communicating Multiagent Collision Avoidance with Deep Reinforcement Learning

Finding feasible, collision-free paths for multiagent systems can be challenging, particularly in non-communicating scenarios where each agent's intent (e.g. goal) is unobservable to the others. In particular, finding time efficient paths often requires anticipating interaction with neighboring agents, the process of which can be computationally prohibitive. This work presents a decentralized multiagent collision avoidance algorithm based on a novel application of deep reinforcement learning, which effectively offloads the online computation (for predicting interaction patterns) to an offline learning procedure. Specifically, the proposed approach develops a value network that encodes the estimated time to the goal given an agent's joint configuration (positions and velocities) with its neighbors. Use of the value network not only admits efficient (i.e., real-time implementable) queries for finding a collision-free velocity vector, but also considers the uncertainty in the other agents' motion. Simulation results show more than 26 percent improvement in paths quality (i.e., time to reach the goal) when compared with optimal reciprocal collision avoidance (ORCA), a state-of-the-art collision avoidance strategy.

cs.MA↗
arXiv2015

Stick-Breaking Policy Learning in Dec-POMDPs

Expectation maximization (EM) has recently been shown to be an efficient algorithm for learning finite-state controllers (FSCs) in large decentralized POMDPs (Dec-POMDPs). However, current methods use fixed-size FSCs and often converge to maxima that are far from optimal. This paper considers a variable-size FSC to represent the local policy of each agent. These variable-size FSCs are constructed using a stick-breaking prior, leading to a new framework called \emph{decentralized stick-breaking policy representation} (Dec-SBPR). This approach learns the controller parameters with a variational Bayesian algorithm without having to assume that the Dec-POMDP model is available. The performance of Dec-SBPR is demonstrated on several benchmark problems, showing that the algorithm scales to large problems while outperforming other state-of-the-art methods.

cs.AI↗
arXiv2013

Dynamic Clustering via Asymptotics of the Dependent Dirichlet Process Mixture

This paper presents a novel algorithm, based upon the dependent Dirichlet process mixture model (DDPMM), for clustering batch-sequential data containing an unknown number of evolving clusters. The algorithm is derived via a low-variance asymptotic analysis of the Gibbs sampling algorithm for the DDPMM, and provides a hard clustering with convergence guarantees similar to those of the k-means algorithm. Empirical results from a synthetic test with moving Gaussian clusters and a test with real ADS-B aircraft trajectory data demonstrate that the algorithm requires orders of magnitude less computational time than contemporary probabilistic and hard clustering algorithms, while providing higher accuracy on the examined datasets.

cs.LG↗
arXiv2012

Quantum Manifestation of Elastic Constants in Nanostructures

Generally, there are two distinct effects in modifying the properties of low-dimensional nanostructures: surface effect (SS) due to increased surface-volume ratio and quantum size effect (QSE) due to quantum confinement in reduced dimension. The SS has been widely shown to affect the elastic constants and mechanical properties of nanostructures. Here, using Pb nanofilm and graphene nanoribbon as model systems, we demonstrate the QSE on the elastic constants of nanostructures by first-principles calculations. We show that generally QSE is dominant in affecting the elastic constants of metallic nanostructures while SS is more pronounced in semiconductor and insulator nanostructures. Our findings have broad implications in quantum aspects of nanomechanics.

cond-mat.mtrl-sci↗
arXiv2012

Interplay between Quantum Size Effect and Strain Effect on Growth of Nanoscale Metal Thin Film

We develop a theoretical framework to investigate the interplay between quantum size effect (QSE) and strain effect on the stability of metal nanofilms. The QSE and strain effect are shown to be coupled through the concept of "quantum electronic stress. First-principles calculations reveal large quantum oscillations in the surface stress of metal nanofilms as a function of film thickness. This adds extrinsically additional strain-coupled quantum oscillations to surface energy of strained metal nanofilms. Our theory enables a quantitative estimation of the amount of strain in experimental samples, and suggests strain be an important factor contributing to the discrepancies between the existing theories and experiments.

cond-mat.mtrl-sci↗
arXiv2011

Quantum Stress: Density Functional Theory Formulation and Physical Manifestation

The concept of "quantum stress (QS)" is introduced and formulated within density functional theory (DFT), to elucidate extrinsic electronic effects on the stress state of solids and thin films in the absence of lattice strain. A formal expression of QS (σ^Q) is derived in relation to deformation potential of electronic states (Ξ) and variation of electron density (Δn), σ^Q = ΞΔn, as a quantum analog of classical Hook's law. Two distinct QS manifestations are demonstrated quantitatively by DFT calculations: (1) in the form of bulk stress induced by charge carriers; and (2) in the form of surface stress induced by quantum confinement. Implications of QS in some physical phenomena are discussed to underlie its importance.

cond-mat.other↗
arXiv2010

Quantum Manifestations of Graphene Edge Stress and Edge Instability: A First-Principles Study

We have performed first-principles calculations of graphene edge stresses, which display two interesting quantum manifestations absent from the classical interpretation: the armchair edge stress oscillates with a nanoribbon width, and the zigzag edge stress is noticeably reduced by spin polarization. Such quantum stress effects in turn manifest in mechanical edge twisting and warping instability, showing features not captured by empirical potentials or continuum theory. Edge adsorption of H and Stone-Wales reconstruction are shown to provide alternative mechanisms in relieving the edge compression and hence to stabilize the planar edge structure.

cond-mat.mes-hall↗
arXiv2008

Magnetic Graphene Nanohole Superlattices

We investigate the magnetic properties of nano-holes (NHs) patterned in graphene using first principles calculations. We show that superlattices consisting of a periodic array of NHs form a new family of 2D crystalline "bulk" magnets whose collective magnetic behavior is governed by inter-NH spin-spin interaction. They exhibit long-range magnetic order well above room temperature. Furthermore, magnetic semiconductors can be made by doping magnetic NHs into semiconducting NH superlattices. Our findings offer a new material system for fundamental studies of spin-spin interaction and magnetic ordering in low dimensions, and open up the exciting opportunities of making engineered magnetic materials for storage media and spintronics applications.

physics.comp-ph↗