arXiv subjects

Dong Li

Publications and source records attributed to Dong Li.

At least 397 records · Page 22Linked to original sources

arXiv2022

A Survey on Interpretable Reinforcement Learning

Although deep reinforcement learning has become a promising machine learning approach for sequential decision-making problems, it is still not mature enough for high-stake domains such as autonomous driving or medical applications. In such contexts, a learned policy needs for instance to be interpretable, so that it can be inspected before any deployment (e.g., for safety and verifiability reasons). This survey provides an overview of various approaches to achieve higher interpretability in reinforcement learning (RL). To that aim, we distinguish interpretability (as a property of a model) and explainability (as a post-hoc operation, with the intervention of a proxy) and discuss them in the context of RL with an emphasis on the former notion. In particular, we argue that interpretable RL may embrace different facets: interpretable inputs, interpretable (transition/reward) models, and interpretable decision-making. Based on this scheme, we summarize and analyze recent work related to interpretable RL with an emphasis on papers published in the past 10 years. We also discuss briefly some related research areas and point to some potential promising research directions.

cs.LG↗

arXiv2022

Persistent fast kink magnetohydrodynamic waves detected in a quiescent prominence

Small-scale, cyclic, transverse motions of plasma threads are usually seen in solar prominences, which are often interpreted as magnetohydrodynamic (MHD) waves. Here, we observed small-scale decayless transverse oscillations in a quiescent prominence, and they appear to be omnipresent. The oscillatory periods of the emission intensity and a proxy for the line-of-sight Doppler shift are about half period of the displacement oscillations. This feature agrees well with the fast kink-mode waves in a flux tube. All the moving threads oscillate transversally spatially in phase and exhibit no significant damping throughout the visible segments, indicating that the fast kink MHD waves are persistently powered and ongoing dissipating energy is transferred to the ambient plasma in the quiet corona. However, our calculations suggest that the energy taken by the fast kink MHD waves alone can not support the coronal heating on the quiet Sun.

astro-ph.SR↗

arXiv2022

Representing Videos as Discriminative Sub-graphs for Action Recognition

Human actions are typically of combinatorial structures or patterns, i.e., subjects, objects, plus spatio-temporal interactions in between. Discovering such structures is therefore a rewarding way to reason about the dynamics of interactions and recognize the actions. In this paper, we introduce a new design of sub-graphs to represent and encode the discriminative patterns of each action in the videos. Specifically, we present MUlti-scale Sub-graph LEarning (MUSLE) framework that novelly builds space-time graphs and clusters the graphs into compact sub-graphs on each scale with respect to the number of nodes. Technically, MUSLE produces 3D bounding boxes, i.e., tubelets, in each video clip, as graph nodes and takes dense connectivity as graph edges between tubelets. For each action category, we execute online clustering to decompose the graph into sub-graphs on each scale through learning Gaussian Mixture Layer and select the discriminative sub-graphs as action prototypes for recognition. Extensive experiments are conducted on both Something-Something V1 & V2 and Kinetics-400 datasets, and superior results are reported when comparing to state-of-the-art methods. More remarkably, our MUSLE achieves to-date the best reported accuracy of 65.0% on Something-Something V2 validation set.

cs.CV↗

arXiv2021

Detections of Multi-Periodic Oscillations during a Circular Ribbon Flare

We present the analysis of three kinds of oscillating behavior using multi-wavelength observations of the 10 November 2013 (SOL2013-11-10T05:14) circular-ribbon flare. This event is a typical circular-ribbon flare with an outer spine structure and homologous jets. We found three kinds of oscillations (or perturbations): i) flux oscillation (or QPP) with a dominant period of about 20 seconds in X-ray, EUV, and microwave emissions, ii) periodic jets with an intermittent cadence of around 72 seconds, iii) an outer loop perturbing half a cycle with a duration of about 168 seconds. Similar to the periodic jets that could be produced by a nonthermal process, like repeated magnetic reconnection, the flare QPP detected in the thermal emissions could have the same origin as the oscillation seen in the nonthermal emissions. The outer-loop perturbation is possibly triggered by a blast wave driven by the circular-ribbon flare, or it might be modulated by the sausage wave or the slow magnetoacoustic wave. The results obtained provide data for further numerical studies on the physical origin of the flare oscillations.

astro-ph.SR↗

arXiv2021

Neuro-Symbolic Hierarchical Rule Induction

We propose an efficient interpretable neuro-symbolic model to solve Inductive Logic Programming (ILP) problems. In this model, which is built from a set of meta-rules organised in a hierarchical structure, first-order rules are invented by learning embeddings to match facts and body predicates of a meta-rule. To instantiate it, we specifically design an expressive set of generic meta-rules, and demonstrate they generate a consequent fragment of Horn clauses. During training, we inject a controlled \pw{Gumbel} noise to avoid local optima and employ interpretability-regularization term to further guide the convergence to interpretable rules. We empirically validate our model on various tasks (ILP, visual genome, reinforcement learning) against several state-of-the-art methods.

cs.LG↗

arXiv2021

Dispersionless orbital excitations in (Li,Fe)OHFeSe superconductors

The superconducting critical temperature $T_{\mathrm{c}}$ of intercalated iron-selenide superconductor (Li,Fe)OHFeSe (FeSe11111) can be increased to 42 K from 8 K of bulk FeSe. It shows remarkably similar electronic properties as the high-$T_{\mathrm{c}}$ monolayer FeSe and provides a bulk counterpart to investigate the origin of enhanced superconductivity. Unraveling the nature of excitations is crucial for understanding the pairing mechanism in high-$T_{\mathrm{c}}$ iron selenides. Here we use resonant inelastic x-ray scattering (RIXS) to investigate the excitations in FeSe11111. Our high-quality data exhibit several Raman-like excitations, which are dispersionless and isotropic in momentum transfer and robust against varying $T_{\mathrm{c}}$. Using atomic multiplet calculations, we assign the low-energy $\sim 0.3$ and 0.7 eV Raman peaks as local $e_g-e_g$ and $e_g-t_{2g}$ orbital excitations. The intensity of these two features decreases with increasing temperature, suggesting a primary contribution of the orbital fluctuations. Our results highlight the importance of orbital degree of freedom for high-$T_{\mathrm{c}}$ iron selenides.

cond-mat.supr-con↗

arXiv2021

What About Inputing Policy in Value Function: Policy Representation and Policy-extended Value Function Approximator

We study Policy-extended Value Function Approximator (PeVFA) in Reinforcement Learning (RL), which extends conventional value function approximator (VFA) to take as input not only the state (and action) but also an explicit policy representation. Such an extension enables PeVFA to preserve values of multiple policies at the same time and brings an appealing characteristic, i.e., \emph{value generalization among policies}. We formally analyze the value generalization under Generalized Policy Iteration (GPI). From theoretical and empirical lens, we show that generalized value estimates offered by PeVFA may have lower initial approximation error to true values of successive policies, which is expected to improve consecutive value approximation during GPI. Based on above clues, we introduce a new form of GPI with PeVFA which leverages the value generalization along policy improvement path. Moreover, we propose a representation learning framework for RL policy, providing several approaches to learn effective policy embeddings from policy network parameters or state-action pairs. In our experiments, we evaluate the efficacy of value generalization offered by PeVFA and policy representation learning in several OpenAI Gym continuous control tasks. For a representative instance of algorithm implementation, Proximal Policy Optimization (PPO) re-implemented under the paradigm of GPI with PeVFA achieves about 40\% performance improvement on its vanilla counterpart in most environments.

cs.LG↗

arXiv2021

The global landscape of phase retrieval I: perturbed amplitude models

A fundamental task in phase retrieval is to recover an unknown signal $\vx\in \Rn$ from a set of magnitude-only measurements $y_i=\abs{\nj{\va_i,\vx}}, \; i=1,\ldots,m$. In this paper, we propose two novel perturbed amplitude models (PAMs) which have non-convex and quadratic-type loss function. When the measurements $ \va_i \in \Rn$ are Gaussian random vectors and the number of measurements $m\ge Cn$, we rigorously prove that the PAMs admit no spurious local minimizers with high probability, i.e., the target solution $ \vx$ is the unique global minimizer (up to a global phase) and the loss function has a negative directional curvature around each saddle point. Thanks to the well-tamed benign geometric landscape, one can employ the vanilla gradient descent method to locate the global minimizer $\vx$ (up to a global phase) without spectral initialization. We carry out extensive numerical experiments to show that the gradient descent algorithm with random initialization outperforms state-of-the-art algorithms with spectral initialization in empirical success rate and convergence speed.

math.NA↗

arXiv2021

The global landscape of phase retrieval II: quotient intensity models

A fundamental problem in phase retrieval is to reconstruct an unknown signal from a set of magnitude-only measurements. In this work we introduce three novel quotient intensity-based models (QIMs) based a deep modification of the traditional intensity-based models. A remarkable feature of the new loss functions is that the corresponding geometric landscape is benign under the optimal sampling complexity. When the measurements $ a_i\in \Rn$ are Gaussian random vectors and the number of measurements $m\ge Cn$, the QIMs admit no spurious local minimizers with high probability, i.e., the target solution $ x$ is the unique global minimizer (up to a global phase) and the loss function has a negative directional curvature around each saddle point. Such benign geometric landscape allows the gradient descent methods to find the global solution $x$ (up to a global phase) without spectral initialization.

math.NA↗

arXiv2021

Simultaneous Observations of Chromospheric Evaporation and Condensation during a C-class Flare

We explored simultaneous observations of chromospheric evaporation and condensation during the impulsive phase of a C6.7 flare on 9 May 2019. The solar flare was simultaneously observed by multiple instruments, i.e., the New Vacuum Solar Telescope (NVST), the Interface Region Imaging Spectrograph, the Atmospheric Imaging Assembly (AIA), the Fermi, the Mingantu Spectral Radioheliograph, and the Nobeyama Radio Polarimeters. Using the single Gaussian fitting and the moment analysis technique, redshifted velocities at slow speeds of 15-19 km/s are found in the cool lines of C II and Si IV at one flare footpoint location. Red shifts are also seen in the H-alpha line-of-sight (LOS) velocity image measured by the NVST at double footpoints. Those red shifts with slow speeds can be regarded as the low-velocity downflows driven by the chromospheric condensation. Meanwhile, the converging motions from double footpoints to the loop top are found in the high-temperature EUV images, such as AIA 131 A, 94 A, and 335 A. Their apparent speeds are estimated to be roughly 126-210 km/s, which could be regarded as the high-velocity upflows caused by the chromospheric evaporation. The nonthermal energy flux is estimated to be about 5.7x10^10 erg/s/cm^2. The characteristic timescale is roughly equal to 1 minute. All these observational results suggest an explosive chromospheric evaporation during the flare impulsive phase. While a HXR/microwave pulse and a type III radio burst are found simultaneously, indicating that the explosive chromospheric evaporation is driven by the nonthermal electron.

astro-ph.SR↗

arXiv2021

A global quadratic speed-up for computing the principal eigenvalue of Perron-like operators

We consider a new algorithm in light of the min-max Collatz-Wielandt formalism to compute the principal eigenvalue and the eigenvector (eigen-function) for a class of positive Perron-Frobenius-like operators. Such operators are natural generalizations of the usual nonnegative primitive matrices. These have nontrivial applications in PDE problems such as computing the principal eigenvalue of Dirichlet Laplacian operators on general domains. We rigorously prove that for general initial data the corresponding numerical iterates converge globally to the unique principal eigenvalue with quadratic convergence. We show that the quadratic convergence is sharp with compatible upper and lower bounds. We demonstrate the effectiveness of the scheme via several illustrative numerical examples.

math.NA↗

arXiv2021

SEIHAI: A Sample-efficient Hierarchical AI for the MineRL Competition

The MineRL competition is designed for the development of reinforcement learning and imitation learning algorithms that can efficiently leverage human demonstrations to drastically reduce the number of environment interactions needed to solve the complex \emph{ObtainDiamond} task with sparse rewards. To address the challenge, in this paper, we present \textbf{SEIHAI}, a \textbf{S}ample-\textbf{e}ff\textbf{i}cient \textbf{H}ierarchical \textbf{AI}, that fully takes advantage of the human demonstrations and the task structure. Specifically, we split the task into several sequentially dependent subtasks, and train a suitable agent for each subtask using reinforcement learning and imitation learning. We further design a scheduler to select different agents for different subtasks automatically. SEIHAI takes the first place in the preliminary and final of the NeurIPS-2020 MineRL competition.

cs.LG↗

arXiv2021

Authentication of optical physical unclonable functions based on single-pixel detection

Physical unclonable function (PUF) has been proposed as a promising and trustworthy solution to a variety of cryptographic applications. Here we propose a non-imaging based authentication scheme for optical PUFs materialized by random scattering media, in which the characteristic fingerprints of optical PUFs are extracted from stochastical fluctuations of the scattered light intensity with respect to laser challenges which are detected by a single-pixel detector. The randomness, uniqueness, unpredictability, and robustness of the extracted fingerprints are validated to be qualified for real authentication applications. By increasing the key length and improving the signal to noise ratio, the false accept rate of a fake PUF can be dramatically lowered to the order of 10^-28. In comparison to the conventional laser-speckle-imaging based authentication with unique identity information obtained from textures of laser speckle patterns, this non-imaging scheme can be implemented at small speckle size bellowing the Nyquist--Shannon sampling criterion of the commonly used CCD or CMOS cameras, offering benefits in system miniaturization and immunity against reverse engineering attacks simultaneously.

cs.CR↗

arXiv2021

Why large time-stepping methods for the Cahn-Hilliard equation is stable

We consider the Cahn-Hilliard equation with standard double-well potential. We employ a prototypical class of first order in time semi-implicit methods with implicit treatment of the linear dissipation term and explicit extrapolation of the nonlinear term. When the dissipation coefficient is held small, a conventional wisdom is to add a judiciously chosen stabilization term in order to afford relatively large time stepping and speed up the simulation. In practical numerical implementations it has been long observed that the resulting system exhibits remarkable stability properties in the regime where the stabilization parameter is $\mathcal O(1)$, the dissipation coefficient is vanishingly small and the size of the time step is moderately large. In this work we develop a new stability theory to address this perplexing phenomenon.

math.NA↗

arXiv2021

Generalizable Cross-Graph Embedding for GNN-based Congestion Prediction

Presently with technology node scaling, an accurate prediction model at early design stages can significantly reduce the design cycle. Especially during logic synthesis, predicting cell congestion due to improper logic combination can reduce the burden of subsequent physical implementations. There have been attempts using Graph Neural Network (GNN) techniques to tackle congestion prediction during the logic synthesis stage. However, they require informative cell features to achieve reasonable performance since the core idea of GNNs is built on the message passing framework, which would be impractical at the early logic synthesis stage. To address this limitation, we propose a framework that can directly learn embeddings for the given netlist to enhance the quality of our node features. Popular random-walk based embedding methods such as Node2vec, LINE, and DeepWalk suffer from the issue of cross-graph alignment and poor generalization to unseen netlist graphs, yielding inferior performance and costing significant runtime. In our framework, we introduce a superior alternative to obtain node embeddings that can generalize across netlist graphs using matrix factorization methods. We propose an efficient mini-batch training method at the sub-graph level that can guarantee parallel training and satisfy the memory restriction for large-scale netlists. We present results utilizing open-source EDA tools such as DREAMPLACE and OPENROAD frameworks on a variety of openly available circuits. By combining the learned embedding on top of the netlist with the GNNs, our method improves prediction performance, generalizes to new circuit lines, and is efficient in training, potentially saving over $90 \%$ of runtime.

cs.LG↗

arXiv2021

An F-modulated stability framework for multistep methods

We introduce a new $\mathbf F$-modulated energy stability framework for general linear multistep methods. We showcase the theory for the two dimensional molecular beam epitaxy model with no slope selection which is a prototypical gradient flow with Lipschitz-bounded nonlinearity. We employ a class of representative BDF$k$, $2\le k \le 5$ discretization schemes with explicit $k^{\mathrm{th}}$-order extrapolation of the nonlinear term. We prove the uniform-in-time boundedness of high Sobolev norms of the numerical solution. The upper bound is unconditional, i.e. regardless of the size of the time step. We develop a new algebraic theory and calibrate nearly optimal and \emph{explicit} maximal time step constraints which guarantee monotonic $\mathbf F$-modulated energy dissipation.

math.NA↗

arXiv2021

Cross-Dataset Collaborative Learning for Semantic Segmentation in Autonomous Driving

Semantic segmentation is an important task for scene understanding in self-driving cars and robotics, which aims to assign dense labels for all pixels in the image. Existing work typically improves semantic segmentation performance by exploring different network architectures on a target dataset. Little attention has been paid to build a unified system by simultaneously learning from multiple datasets due to the inherent distribution shift across different datasets. In this paper, we propose a simple, flexible, and general method for semantic segmentation, termed Cross-Dataset Collaborative Learning (CDCL). Our goal is to train a unified model for improving the performance in each dataset by leveraging information from all the datasets. Specifically, we first introduce a family of Dataset-Aware Blocks (DAB) as the fundamental computing units of the network, which help capture homogeneous convolutional representations and heterogeneous statistics across different datasets. Second, we present a Dataset Alternation Training (DAT) mechanism to facilitate the collaborative optimization procedure. We conduct extensive evaluations on diverse semantic segmentation datasets for autonomous driving. Experiments demonstrate that our method consistently achieves notable improvements over prior single-dataset and cross-dataset training methods without introducing extra FLOPs. Particularly, with the same architecture of PSPNet (ResNet-18), our method outperforms the single-dataset baseline by 5.65\%, 6.57\%, and 5.79\% mIoU on the validation sets of Cityscapes, BDD100K, CamVid, respectively. We also apply CDCL for point cloud 3D semantic segmentation and achieve improved performance, which further validates the superiority and generality of our method. Code and models will be released.

cs.CV↗

arXiv2021

CMML: Contextual Modulation Meta Learning for Cold-Start Recommendation

Practical recommender systems experience a cold-start problem when observed user-item interactions in the history are insufficient. Meta learning, especially gradient based one, can be adopted to tackle this problem by learning initial parameters of the model and thus allowing fast adaptation to a specific task from limited data examples. Though with significant performance improvement, it commonly suffers from two critical issues: the non-compatibility with mainstream industrial deployment and the heavy computational burdens, both due to the inner-loop gradient operation. These two issues make them hard to be applied in practical recommender systems. To enjoy the benefits of meta learning framework and mitigate these problems, we propose a recommendation framework called Contextual Modulation Meta Learning (CMML). CMML is composed of fully feed-forward operations so it is computationally efficient and completely compatible with the mainstream industrial deployment. CMML consists of three components, including a context encoder that can generate context embedding to represent a specific task, a hybrid context generator that aggregates specific user-item features with task-level context, and a contextual modulation network, which can modulate the recommendation model to adapt effectively. We validate our approach on both scenario-specific and user-specific cold-start setting on various real-world datasets, showing CMML can achieve comparable or even better performance with gradient based methods yet with much higher computational efficiency and better interpretability.

cs.IR↗