arXiv subjects

Miao Liu

Publications and source records attributed to Miao Liu.

At least 109 records · Page 6Linked to original sources

arXiv2023

Learning in Factored Domains with Information-Constrained Visual Representations

Humans learn quickly even in tasks that contain complex visual information. This is due in part to the efficient formation of compressed representations of visual information, allowing for better generalization and robustness. However, compressed representations alone are insufficient for explaining the high speed of human learning. Reinforcement learning (RL) models that seek to replicate this impressive efficiency may do so through the use of factored representations of tasks. These informationally simplistic representations of tasks are similarly motivated as the use of compressed representations of visual information. Recent studies have connected biological visual perception to disentangled and compressed representations. This raises the question of how humans learn to efficiently represent visual information in a manner useful for learning tasks. In this paper we present a model of human factored representation learning based on an altered form of a $β$-Variational Auto-encoder used in a visual learning task. Modelling results demonstrate a trade-off in the informational complexity of model latent dimension spaces, between the speed of learning and the accuracy of reconstructions.

cs.AI↗

arXiv2023

Lu-H-N phase diagram from first-principles calculations

Using a comprehensive structure search and high-throughput first-principles calculations of 1483 compounds, this study presents the phase diagram of Lu-H-N. The formation energy landscape of Lu-H-N was derived and utilized to assess the thermodynamic stability of compounds. Results indicate that there are no stable Lu-H-N ternary structures in this system, but metastable ternary structures, such as Lu20H2N17 (C2/m), Lu2H2N (P3-m1), were observed with small Ehull (< 100 meV/atom). Moreover, applying hydrostatic pressure up to 10 GPa causes the energy convex hull of the Lu-H-N to shift its shape and stabilizes binary phases such as LuN9 and Lu10H21. Additionally, interstitial empty sites in LuH2 were noted, which may explain the formation of Lu10H21 and LuH3-xNy. To provide a basis for comparison, X-ray diffraction patterns and electronic structures of some compounds are also presented.

cond-mat.supr-con↗

arXiv2023

Predicting structure-dependent Hubbard U parameters for assessing hybrid functional-level exchange via machine learning

DFT+U is a widely used treatment in the density functional theory (DFT) to deal with correlated materials that contain open-shell elements, whereby the quantitative and sometimes even qualitative failures of local and semilocal approximations can be corrected without much computational overhead. However, finding appropriate U parameters for a given system is non-trivial and usually requires computationally intensive and cumbersome first-principles calculations. In this Letter, we address this issue by building a machine learning (ML) model to predict material-specific U parameters only from the structural information. An ML model is trained for the Mn-O chemical system by calibrating their DFT+U electronic structures with the hybrid functional results of more than Mn-O 3000 structures. The model allows us to determine a reliable U value (MAE=0.128 eV, R2=0.97) for any given structure at nearly no computational cost; yet the obtained U value is as good as that obtained from the conventional first-principles methods. Further analysis reveals that the U value is primarily determined by the local chemical structure, especially the bond lengths, and this property is well captured by the ML model developed in this work. This concept of the ML U model is universally applicable and can considerably ease the usage of the DFT+U method by providing structure-specific, readily accessible U values.

physics.comp-ph↗

arXiv2023

Joint Edge-Model Sparse Learning is Provably Efficient for Graph Neural Networks

Due to the significant computational challenge of training large-scale graph neural networks (GNNs), various sparse learning techniques have been exploited to reduce memory and storage costs. Examples include \textit{graph sparsification} that samples a subgraph to reduce the amount of data aggregation and \textit{model sparsification} that prunes the neural network to reduce the number of trainable weights. Despite the empirical successes in reducing the training cost while maintaining the test accuracy, the theoretical generalization analysis of sparse learning for GNNs remains elusive. To the best of our knowledge, this paper provides the first theoretical characterization of joint edge-model sparse learning from the perspective of sample complexity and convergence rate in achieving zero generalization error. It proves analytically that both sampling important nodes and pruning neurons with the lowest-magnitude can reduce the sample complexity and improve convergence without compromising the test accuracy. Although the analysis is centered on two-layer GNNs with structural constraints on data, the insights are applicable to more general setups and justified by both synthetic and practical citation datasets.

cs.LG↗

arXiv2022

Werewolf Among Us: A Multimodal Dataset for Modeling Persuasion Behaviors in Social Deduction Games

Persuasion modeling is a key building block for conversational agents. Existing works in this direction are limited to analyzing textual dialogue corpus. We argue that visual signals also play an important role in understanding human persuasive behaviors. In this paper, we introduce the first multimodal dataset for modeling persuasion behaviors. Our dataset includes 199 dialogue transcriptions and videos captured in a multi-player social deduction game setting, 26,647 utterance level annotations of persuasion strategy, and game level annotations of deduction game outcomes. We provide extensive experiments to show how dialogue context and visual signals benefit persuasion strategy prediction. We also explore the generalization ability of language models for persuasion modeling and the role of persuasion strategies in predicting social deduction game outcomes. Our dataset, code, and models can be found at https://persuasion-deductiongame.socialai-data.org.

cs.LG↗

arXiv2022

Local Explanations for Reinforcement Learning

Many works in explainable AI have focused on explaining black-box classification models. Explaining deep reinforcement learning (RL) policies in a manner that could be understood by domain users has received much less attention. In this paper, we propose a novel perspective to understanding RL policies based on identifying important states from automatically learned meta-states. The key conceptual difference between our approach and many previous ones is that we form meta-states based on locality governed by the expert policy dynamics rather than based on similarity of actions, and that we do not assume any particular knowledge of the underlying topology of the state space. Theoretically, we show that our algorithm to find meta-states converges and the objective that selects important states from each meta-state is submodular leading to efficient high quality greedy selection. Experiments on four domains (four rooms, door-key, minipacman, and pong) and a carefully conducted user study illustrate that our perspective leads to better understanding of the policy. We conjecture that this is a result of our meta-states being more intuitive in that the corresponding important states are strong indicators of tractable intermediate goals that are easier for humans to interpret and follow.

cs.LG↗

arXiv2022

Game-Theoretical Perspectives on Active Equilibria: A Preferred Solution Concept over Nash Equilibria

Multiagent learning settings are inherently more difficult than single-agent learning because each agent interacts with other simultaneously learning agents in a shared environment. An effective approach in multiagent reinforcement learning is to consider the learning process of agents and influence their future policies toward desirable behaviors from each agent's perspective. Importantly, if each agent maximizes its long-term rewards by accounting for the impact of its behavior on the set of convergence policies, the resulting multiagent system reaches an active equilibrium. While this new solution concept is general such that standard solution concepts, such as a Nash equilibrium, are special cases of active equilibria, it is unclear when an active equilibrium is a preferred equilibrium over other solution concepts. In this paper, we analyze active equilibria from a game-theoretic perspective by closely studying examples where Nash equilibria are known. By directly comparing active equilibria to Nash equilibria in these examples, we find that active equilibria find more effective solutions than Nash equilibria, concluding that an active equilibrium is the desired solution for multiagent learning settings.

cs.GT↗

arXiv2022

Engineered Kondo screening and nonzero Berry phase in SrTiO3/LaTiO3/SrTiO3 heterostructures

Controlling the interplay between localized spins and itinerant electrons at the oxide interfaces can lead to exotic magnetic states. Here we devise SrTiO3/LaTiO3/SrTiO3 heterostructures with varied thickness of the LaTiO3 layer (n monolayers) to investigate the magnetic interactions in the two-dimensional electron gas system. The heterostructures exhibit significant Kondo effect when the LaTiO3 layer is rather thin (n = 2, 10), manifesting the strong interaction between the itinerant electrons and the localized magnetic moments at the interfaces, while the Kondo effect is greatly inhibited when n = 20. Notably, distinct Shubnikov-de Haas oscillations are observed and a nonzero Berry phase of π is extracted when the LaTiO3 layer is rather thin (n = 2, 10), which is absent in the heterostructure with thicker LaTiO3 layer (n = 20). The observed phenomena are consistently interpreted as a result of sub-band splitting and symmetry breaking due to the interplay between the interfacial Rashba spin-orbit coupling and the magnetic orderings in the heterostructures. Our findings provide a route for exploring and manipulating nontrivial electronic band structures at complex oxide interfaces.

cond-mat.str-el↗

arXiv2022

Learning Multi-Objective Curricula for Robotic Policy Learning

Various automatic curriculum learning (ACL) methods have been proposed to improve the sample efficiency and final performance of deep reinforcement learning (DRL). They are designed to control how a DRL agent collects data, which is inspired by how humans gradually adapt their learning processes to their capabilities. For example, ACL can be used for subgoal generation, reward shaping, environment generation, or initial state generation. However, prior work only considers curriculum learning following one of the aforementioned predefined paradigms. It is unclear which of these paradigms are complementary, and how the combination of them can be learned from interactions with the environment. Therefore, in this paper, we propose a unified automatic curriculum learning framework to create multi-objective but coherent curricula that are generated by a set of parametric curriculum modules. Each curriculum module is instantiated as a neural network and is responsible for generating a particular curriculum. In order to coordinate those potentially conflicting modules in unified parameter space, we propose a multi-task hyper-net learning framework that uses a single hyper-net to parameterize all those curriculum modules. In addition to existing hand-designed curricula paradigms, we further design a flexible memory mechanism to learn an abstract curriculum, which may otherwise be difficult to design manually. We evaluate our method on a series of robotic manipulation tasks and demonstrate its superiority over other state-of-the-art ACL methods in terms of sample efficiency and final performance.

cs.LG↗

arXiv2022

Influencing Long-Term Behavior in Multiagent Reinforcement Learning

The main challenge of multiagent reinforcement learning is the difficulty of learning useful policies in the presence of other simultaneously learning agents whose changing behaviors jointly affect the environment's transition and reward dynamics. An effective approach that has recently emerged for addressing this non-stationarity is for each agent to anticipate the learning of other agents and influence the evolution of future policies towards desirable behavior for its own benefit. Unfortunately, previous approaches for achieving this suffer from myopic evaluation, considering only a finite number of policy updates. As such, these methods can only influence transient future policies rather than achieving the promise of scalable equilibrium selection approaches that influence the behavior at convergence. In this paper, we propose a principled framework for considering the limiting policies of other agents as time approaches infinity. Specifically, we develop a new optimization objective that maximizes each agent's average reward by directly accounting for the impact of its behavior on the limiting set of policies that other agents will converge to. Our paper characterizes desirable solution concepts within this problem setting and provides practical approaches for optimizing over possible outcomes. As a result of our farsighted objective, we demonstrate better long-term performance than state-of-the-art baselines across a suite of diverse multiagent benchmark domains.

cs.LG↗

arXiv2022

Hierarchical Reinforcement Learning with AI Planning Models

Two common approaches to sequential decision-making are AI planning (AIP) and reinforcement learning (RL). Each has strengths and weaknesses. AIP is interpretable, easy to integrate with symbolic knowledge, and often efficient, but requires an up-front logical domain specification and is sensitive to noise; RL only requires specification of rewards and is robust to noise but is sample inefficient and not easily supplied with external knowledge. We propose an integrative approach that combines high-level planning with RL, retaining interpretability, transfer, and efficiency, while allowing for robust learning of the lower-level plan actions. Our approach defines options in hierarchical reinforcement learning (HRL) from AIP operators by establishing a correspondence between the state transition model of AI planning problem and the abstract state transition system of a Markov Decision Process (MDP). Options are learned by adding intrinsic rewards to encourage consistency between the MDP and AIP transition models. We demonstrate the benefit of our integrated approach by comparing the performance of RL and HRL algorithms in both MiniGrid and N-rooms environments, showing the advantage of our method over the existing ones.

cs.AI↗

arXiv2022

Egocentric Activity Recognition and Localization on a 3D Map

Given a video captured from a first person perspective and the environment context of where the video is recorded, can we recognize what the person is doing and identify where the action occurs in the 3D space? We address this challenging problem of jointly recognizing and localizing actions of a mobile user on a known 3D map from egocentric videos. To this end, we propose a novel deep probabilistic model. Our model takes the inputs of a Hierarchical Volumetric Representation (HVR) of the 3D environment and an egocentric video, infers the 3D action location as a latent variable, and recognizes the action based on the video and contextual cues surrounding its potential locations. To evaluate our model, we conduct extensive experiments on the subset of Ego4D dataset, in which both human naturalistic actions and photo-realistic 3D environment reconstructions are captured. Our method demonstrates strong results on both action recognition and 3D action localization across seen and unseen environments. We believe our work points to an exciting research direction in the intersection of egocentric vision, and 3D scene understanding.

cs.CV↗

arXiv2022

Generative Adversarial Network for Future Hand Segmentation from Egocentric Video

We introduce the novel problem of anticipating a time series of future hand masks from egocentric video. A key challenge is to model the stochasticity of future head motions, which globally impact the head-worn camera video analysis. To this end, we propose a novel deep generative model -- EgoGAN, which uses a 3D Fully Convolutional Network to learn a spatio-temporal video representation for pixel-wise visual anticipation, generates future head motion using Generative Adversarial Network (GAN), and then predicts the future hand masks based on the video representation and the generated future head motion. We evaluate our method on both the EPIC-Kitchens and the EGTEA Gaze+ datasets. We conduct detailed ablation studies to validate the design choices of our approach. Furthermore, we compare our method with previous state-of-the-art methods on future image segmentation and show that our method can more accurately predict future hand masks.

cs.CV↗

arXiv2022

Context-Specific Representation Abstraction for Deep Option Learning

Hierarchical reinforcement learning has focused on discovering temporally extended actions, such as options, that can provide benefits in problems requiring extensive exploration. One promising approach that learns these options end-to-end is the option-critic (OC) framework. We examine and show in this paper that OC does not decompose a problem into simpler sub-problems, but instead increases the size of the search over policy space with each option considering the entire state space during learning. This issue can result in practical limitations of this method, including sample inefficient learning. To address this problem, we introduce Context-Specific Representation Abstraction for Deep Option Learning (CRADOL), a new framework that considers both temporal abstraction and context-specific representation abstraction to effectively reduce the size of the search over policy space. Specifically, our method learns a factored belief state representation that enables each option to learn a policy over only a subsection of the state space. We test our method against hierarchical, non-hierarchical, and modular recurrent neural network baselines, demonstrating significant sample efficiency improvements in challenging partially observable environments.

cs.LG↗

arXiv2022

Ego4D: Around the World in 3,000 Hours of Egocentric Video

We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite. It offers 3,670 hours of daily-life activity video spanning hundreds of scenarios (household, outdoor, workplace, leisure, etc.) captured by 931 unique camera wearers from 74 worldwide locations and 9 different countries. The approach to collection is designed to uphold rigorous privacy and ethics standards with consenting participants and robust de-identification procedures where relevant. Ego4D dramatically expands the volume of diverse egocentric video footage publicly available to the research community. Portions of the video are accompanied by audio, 3D meshes of the environment, eye gaze, stereo, and/or synchronized videos from multiple egocentric cameras at the same event. Furthermore, we present a host of new benchmark challenges centered around understanding the first-person visual experience in the past (querying an episodic memory), present (analyzing hand-object manipulation, audio-visual conversation, and social interactions), and future (forecasting activities). By publicly sharing this massive annotated dataset and benchmark suite, we aim to push the frontier of first-person perception. Project page: https://ego4d-data.org/

cs.CV↗

arXiv2022

Superconductive materials with MgB2-like structures from data-driven screening

Finding viable superconducting materials is of interest to the physics community as the superconductors are the playground to manifest many appealing quantum phenomena. This work exemplifies an end-to-end materials discovery towards novel MgB2-like superconductors, starting from the data-driven compound screening all the way to the experimental materialization. In addition to the known superconducting compounds, CaB2 (Tc = 9.4 ~ 28.6 K), SrGa2 (Tc = 0.1 ~ 2.4 K), BaGa2 (Tc = 0.3 ~ 3.3 K), BaAu2 (Tc = 0.0 ~ 0.5 K), and LaCu2 (Tc = 0.1 ~ 2.2 K) are newly discovered, out of ~182,000 starting structures, to be the most promising superconducting compounds that share similar atomic structures with MgB2. Moreover, BaGa2 is experimentally synthesized and measured to confirm that the compound is a BCS superconductor with Tc = 0.36 K, in good agreement with our theoretical predictions. This work provides a "once and for all" study for the MgB2-like superconductors and showcases that it is feasible to discover new materials via a data-driven approach.

cond-mat.supr-con↗

arXiv2022

Towards the Maximum Traffic Demand and Throughput Supported by Relay-Assisted mmWave Backhaul Networks

This paper investigates the throughput performance issue of the relay-assisted mmWave backhaul network. The maximum traffic demand of small-cell base stations (BSs) and the maximum throughput at the macro-cell BS have been found in a tree-style backhaul network through linear programming under different network settings, which concern both the number of radio chains available on BSs and the interference relationship between logical links in the backhaul network. A novel interference model for the relay-assisted mmWave backhaul network in the dense urban environment is proposed, which demonstrates the limited interference footprint of mmWave directional communications. Moreover, a scheduling algorithm is developed to find the optimal scheduling for tree-style mmWave backhaul networks. Extensive numerical analysis and simulations are conducted to show and validate the network throughput performance and the scheduling algorithm.

cs.NI↗

arXiv2022

An efficient distributed scheduling algorithm for relay-assisted mmWave backhaul networks

In this paper, a novel distributed scheduling algorithm is proposed, which aims to efficiently schedule both the uplink and downlink backhaul traffic in the relay-assisted mmWave backhaul network with a tree topology. The handshaking of control messages, calculation of local schedules, and the determination of final valid schedule are all discussed. Simulation results show that the performance of the distributed algorithm can reach very close to the maximum traffic demand of the backhaul network, and it can also adapt to the dynamic traffic with sharp traffic demand change of small-cell BSs quickly and accurately.

cs.NI↗