arXiv subjects

Bin Liang

Publications and source records attributed to Bin Liang.

At least 145 records · Page 8Linked to original sources

arXiv2021

Probability Density Estimation Based Imitation Learning

Imitation Learning (IL) is an effective learning paradigm exploiting the interactions between agents and environments. It does not require explicit reward signals and instead tries to recover desired policies using expert demonstrations. In general, IL methods can be categorized into Behavioral Cloning (BC) and Inverse Reinforcement Learning (IRL). In this work, a novel reward function based on probability density estimation is proposed for IRL, which can significantly reduce the complexity of existing IRL methods. Furthermore, we prove that the theoretically optimal policy derived from our reward function is identical to the expert policy as long as it is deterministic. Consequently, an IRL problem can be gracefully transformed into a probability density estimation problem. Based on the proposed reward function, we present a "watch-try-learn" style framework named Probability Density Estimation based Imitation Learning (PDEIL), which can work in both discrete and continuous action spaces. Finally, comprehensive experiments in the Gym environment show that PDEIL is much more efficient than existing algorithms in recovering rewards close to the ground truth.

cs.LG↗

arXiv2021

The Partially Observable Asynchronous Multi-Agent Cooperation Challenge

Multi-agent reinforcement learning (MARL) has received increasing attention for its applications in various domains. Researchers have paid much attention on its partially observable and cooperative settings for meeting real-world requirements. For testing performance of different algorithms, standardized environments are designed such as the StarCraft Multi-Agent Challenge, which is one of the most successful MARL benchmarks. To our best knowledge, most of current environments are synchronous, where agents execute actions in the same pace. However, heterogeneous agents usually have their own action spaces and there is no guarantee for actions from different agents to have the same executed cycle, which leads to asynchronous multi-agent cooperation. Inspired from the Wargame, a confrontation game between two armies abstracted from real world environment, we propose the first Partially Observable Asynchronous multi-agent Cooperation challenge (POAC) for the MARL community. Specifically, POAC supports two teams of heterogeneous agents to fight with each other, where an agent selects actions based on its own observations and cooperates asynchronously with its allies. Moreover, POAC is a light weight, flexible and easy to use environment, which can be configured by users to meet different experimental requirements such as self-play model, human-AI model and so on. Along with our benchmark, we offer six game scenarios of varying difficulties with the built-in rule-based AI as opponents. Finally, since most MARL algorithms are designed for synchronous agents, we revise several representatives to meet the asynchronous setting, and the relatively poor experimental results validate the challenge of POAC. Source code is released in \url{http://turingai.ia.ac.cn/data\_center/show}.

cs.MA↗

arXiv2021

Experimental Realization of Weyl Exceptional Rings in a Synthetic Three-Dimensional Non-Hermitian Phononic Crystal

Weyl points (WPs) are isolated degeneracies carrying quantized topological charges, and are therefore robust against Hermitian perturbations. WPs are predicted to spread to the Weyl exceptional rings (WERs) in the presence of non-Hermiticity. Here, we use a one-dimensional (1D) Aubry-Andre-Harper (AAH) model to construct a Weyl semimetal in a 3D parameter space comprised of one reciprocal dimension and two synthetic dimensions. The inclusion of non-Hermiticity in the form of gain and loss produces a WER. The topology of the WER is characterized by both its topological charge and non-Hermitian winding numbers. The WER is experimentally observed in a 1D phononic crystal with the non-Hermiticity introduced by active acoustic components. In addition, Fermi arcs are observed to survive the presence of non-Hermitian effect. We envision our findings to pave the way for studying the high-dimensional non-Hermitian topological physics in acoustics.

cond-mat.mes-hall↗

arXiv2021

Offline Reinforcement Learning with Value-based Episodic Memory

Offline reinforcement learning (RL) shows promise of applying RL to real-world problems by effectively utilizing previously collected data. Most existing offline RL algorithms use regularization or constraints to suppress extrapolation error for actions outside the dataset. In this paper, we adopt a different framework, which learns the V-function instead of the Q-function to naturally keep the learning procedure within the support of an offline dataset. To enable effective generalization while maintaining proper conservatism in offline learning, we propose Expectile V-Learning (EVL), which smoothly interpolates between the optimal value learning and behavior cloning. Further, we introduce implicit planning along offline trajectories to enhance learned V-values and accelerate convergence. Together, we present a new offline method called Value-based Episodic Memory (VEM). We provide theoretical analysis for the convergence properties of our proposed VEM method, and empirical results in the D4RL benchmark show that our method achieves superior performance in most tasks, particularly in sparse-reward tasks.

cs.LG↗

arXiv2021

Multiple-Pilot Collaboration for Advanced Remote Intervention using Reinforcement Learning

The traditional master-slave teleoperation relies on human expertise without correction mechanisms, resulting in excessive physical and mental workloads. To address these issues, a co-pilot-in-the-loop control framework is investigated for cooperative teleoperation. A deep deterministic policy gradient(DDPG) based agent is realised to effectively restore the master operators' intents without prior knowledge on time delay. The proposed framework allows for introducing an operator (i.e., co-pilot) to generate commands at the slave side, whose weights are optimally assigned online through DDPG-based arbitration, thereby enhancing the command robustness in the case of possible human operational errors. With the help of interval type-2(IT2) Takagi-Sugeno (T-S) fuzzy identification, force feedback can be reconstructed at the master side without a sense of delay, thus ensuring the telepresence performance in the force-sensor-free scenarios. Two experimental applications validate the effectiveness of the proposed framework.

cs.RO↗

arXiv2021

Experimental Observation of Efficient Nonreciprocal Mode Transitions via Spatiotemporally-Modulated Acoustic Metamaterials

In lossless acoustic systems, mode transitions are always time-reversible, consistent with Lorentz reciprocity, giving rise to symmetric sound manipulation in space-time. To overcome this fundamental limitation and break space-time symmetry, nonreciprocal sound steering is realized by designing and experimentally implementing spatiotemporally-modulated acoustic metamaterials. Relying on no slow mechanical parts, unstable and noisy airflow or complicated piezoelectric array, our mechanism uses the coupling between an ultrathin membrane and external electromagnetic field to realize programmable, dynamic control of acoustic impedance in a motionless and noiseless manner. The fast and flexible impedance modulation at the deeply subwavelength scale enabled by our compact metamaterials provides an effective unidirectional momentum in space-time to realize irreversible transition in k-ω space between different diffraction modes. The nonreciprocal wave-steering functionality of the proposed metamaterial is elucidated by theoretically deriving the time-varying acoustic response and demonstrated both numerically and experimentally via two distinctive examples of unidirectional evanescent wave conversion and nonreciprocal blue-shift focusing. This work can be further extended into the paradigm of Bloch waves and impact other vibrant domains, such as non-Hermitian topological acoustics and parity-time-symmetric acoustics.

physics.app-ph↗

arXiv2021

Orientation to Pose: Continuum Robots Shape Sensing Based on Piecewise Polynomial Curvature Model

Continuum robots are typically slender and flexible with infinite freedoms in theory, which poses a challenge for their control and application. The shape sensing of continuum robots is vital to realise accuracy control. This letter proposed a novel general real-time shape sensing framework of continuum robots based on the piecewise polynomial curvature (PPC) kinematics model. We illustrate the coupling between orientation and position at any given location of the continuum robots. Further, the coupling relation could be bridged by the PPC kinematics. Therefore, we propose to estimate the shape of continuum robots through orientation estimation, using the off-the-shelf orientation sensors, e.g., IMUs, mounted on certain locations. The approach gives a valuable framework to the shape sensing of continuum robots, universality, accuracy and convenience. The accuracy of the general approach is verified in the experiments of multi-type physical prototypes.

cs.RO↗

arXiv2021

Revised $f_{\rm NL}$ parameter in Curvaton Scenario

We revise the Non-Gaussianity of canonical curvaton scenario with a generalized $δN$ formalism, in which it could handle the generic potentials. In various curvaton models, the energy density is dominant in different period including the secondary inflation of curvaton, matter domination and radiation domination. Our method could unify to deal with these periods since the non-linearity parameter $f_{\rm NL}$ associated with Non-Gaussianity is a function of equation of state $w$. We firstly investigate the most simple curvaton scenario, namely the chaotic curvaton with quadratic potential. Our study shows that most parameter space satisfies with observational constraints. And our formula will nicely recover the well-known value of $f_{\rm NL}$ in the absence of non-linear evolution. From the micro origin of curvaton, we also investigate the Pseudo-Nambu-Goldstone curvaton. Our result clearly indicates that the second short inflationary process for Pseudo-Nambu-Goldstone curvaton is ruled out in light of observations. Finally, our method sheds a new way for investigating the Non-Gaussianity of curvaton mechanism, espeically for exploring the Non-Gaussianity in MSSM curvaton model.

astro-ph.CO↗

arXiv2020

Approximate Piecewise Constant Curvature Equivalent Model and Their Application to Continuum Robot Configuration Estimation

The continuum robot has attracted more attention for its flexibility. Continuum robot kinematics models are the basis for further perception, planning, and control. The design and research of continuum robots are usually based on the assumption of piecewise constant curvature (PCC). However, due to the influence of friction, etc., the actual motion of the continuum robot is approximate piecewise constant curvature (APCC). To address this, we present a kinematic equivalent model for continuum robots, i.e. APCC 2L-5R. Using classical rigid linkages to replace the original model in kinematic, the APCC 2L-5R model effectively reduces complexity and improves numerical stability. Furthermore, based on the model, the configuration self-estimation of the continuum robot is realized by monocular cameras installed at the end of each approximate constant curvature segment. The potential of APCC 2L-5R in perception, planning, and control of continuum robots remains to be explored.

cs.RO↗

arXiv2020

Utilizing machine learning to prevent water main breaks by understanding pipeline failure drivers

Data61 and Western Water worked collaboratively to apply engineering expertise and Machine Learning tools to find a cost-effective solution to the pipe failure problem in the region west of Melbourne, where on average 400 water main failures occur per year. To achieve this objective, we constructed a detailed picture and understanding of the behaviour of the water pipe network by 1) discovering the underlying drivers of water main breaks, and 2) developing a Machine Learning system to assess and predict the failure likelihood of water main breaking using historical failure records, descriptors of pipes, and other environmental factors. The ensuing results open up an avenue for Western Water to identify the priority of pipe renewals

cs.CY↗

arXiv2019

Meta-neural-network for Realtime and Passive Deep-learning-based Object Recognition

Deep-learning recently show great success across disciplines yet conventionally require time-consuming computer processing or bulky-sized diffractive elements. Here we theoretically propose and experimentally demonstrate a purely-passive "meta-neural-network" with compactness and high-resolution for real-time recognizing complicated objects by analyzing acoustic scattering. We prove our meta-neural-network mimics standard neural network despite its small footprint, thanks to unique capability of its metamaterial unit cells, dubbed "meta-neurons", to produce deep-subwavelength-distribution of discrete phase shift as learnable parameters during training. The resulting device exhibits the "intelligence" to perform desired tasks with potential to address the current trade-off between reducing device's size, cost and energy consumption and increasing recognition speed and accuracy, showcased by an example of handwritten digit recognition. Our mechanism opens the route to new metamaterial-based deep-learning paradigms and enable conceptual devices such as smart transducers automatically analyzing signals, with far-reaching implications for acoustics, optics and related fields.

cs.NE↗

arXiv2019

Context-aware Embedding for Targeted Aspect-based Sentiment Analysis

Attention-based neural models were employed to detect the different aspects and sentiment polarities of the same target in targeted aspect-based sentiment analysis (TABSA). However, existing methods do not specifically pre-train reasonable embeddings for targets and aspects in TABSA. This may result in targets or aspects having the same vector representations in different contexts and losing the context-dependent information. To address this problem, we propose a novel method to refine the embeddings of targets and aspects. Such pivotal embedding refinement utilizes a sparse coefficient vector to adjust the embeddings of target and aspect from the context. Hence the embeddings of targets and aspects can be refined from the highly correlative words instead of using context-independent or randomly initialized vectors. Experiment results on two benchmark datasets show that our approach yields the state-of-the-art performance in TABSA task.

cs.CL↗

arXiv2019

Quasinormal Modes and Van der Waals like phase transition of charged AdS black holes in Lorentz symmetry breaking massive gravity

Using the quasinormal modes of a massless scalar perturbation, we investigate the small/large black hole phase transition in the Lorentz symmetry breaking massive gravity. We mainly focus on two issues: i) the sign change of slope of the quasinormal mode frequencies in the complex-$ω$ diagram; ii) the behaviors of the imaginary part of the quasinormal mode frequencies along the isobaric or isothermal processes. For the first issue, our result shows that, at low fixed temperature or pressure, the phase transition can be probed by the sign change of slope. While increasing the temperature or pressure to some certain values near the critical point, there will appear the deflection point, which indicates that such method may not be appropriate to test the phase transition. In particular, the behavior of the quasinormal mode frequencies for the small and large black holes tend to the same at the critical point. For the second issue, it is shown that the non-monotonic behavior is observed only when the small/large black hole phase transition occurs. Therefore, this property can provide us with an additional method to probe the phase transition through the quasinormal modes.

gr-qc↗

arXiv2019

Acoustic manipulation through zero-thickness perforated plane with strong-coupling effects

How to manipulate acoustic waves through thinner structures is always a challenging problem due to the linear proportional relationship between the structural thickness and the acoustic wavelength. Here, we show the possibility of breaking this relationship by the strong-coupling effects of the radiated waves on the zero-thickness two-dimensional perforated plane, rather than reducing the thickness of the three-dimensional structure by the resonance mechanism of the cavity structure. The strong-coupling effects can be achieved and regulated by the self and mutual radiation between acoustic waves from different holes in the zero-thickness plane. We experimentally demonstrate the effectiveness of our approach by implementing acoustic focusing and holography. Our work introduces a different perspective for manipulating acoustic waves and will enable the application of ultrathin acoustic devices.

physics.app-ph↗

arXiv2019

Weak cosmic censorship conjecture in Kerr black holes of modified gravity

By neglecting the effects of self-force and radiation, we investigate the possibility of destroying the Kerr-MOG black hole through the point particle absorption process. Using the instability of event horizon and equation of particle motion, we get the upper and lower energy bounds allowed for a matter particle to produce the naked singularity. We find that the energy gap always exists between the upper and lower energy bounds for both extremal and near-extremal black holes, which means some tailored particles can actually lead to the violation of the weak cosmic censorship conjecture. However, when considering the effect of the adiabatic process, the result shows that the Kerr-MOG black hole gets more stable instead of a naked singularity, and thus the weak cosmic censorship conjecture can be restored at some level.

gr-qc↗

arXiv2019

Detecting Adversarial Image Examples in Deep Networks with Adaptive Noise Reduction

Recently, many studies have demonstrated deep neural network (DNN) classifiers can be fooled by the adversarial example, which is crafted via introducing some perturbations into an original sample. Accordingly, some powerful defense techniques were proposed. However, existing defense techniques often require modifying the target model or depend on the prior knowledge of attacks. In this paper, we propose a straightforward method for detecting adversarial image examples, which can be directly deployed into unmodified off-the-shelf DNN models. We consider the perturbation to images as a kind of noise and introduce two classic image processing techniques, scalar quantization and smoothing spatial filter, to reduce its effect. The image entropy is employed as a metric to implement an adaptive noise reduction for different kinds of images. Consequently, the adversarial example can be effectively detected by comparing the classification results of a given sample and its denoised version, without referring to any prior knowledge of attacks. More than 20,000 adversarial examples against some state-of-the-art DNN models are used to evaluate the proposed method, which are crafted with different attack techniques. The experiments show that our detection method can achieve a high overall F1 score of 96.39% and certainly raises the bar for defense-aware attacks.

cs.CR↗

arXiv2019

Deep Text Classification Can be Fooled

In this paper, we present an effective method to craft text adversarial samples, revealing one important yet underestimated fact that DNN-based text classifiers are also prone to adversarial sample attack. Specifically, confronted with different adversarial scenarios, the text items that are important for classification are identified by computing the cost gradients of the input (white-box attack) or generating a series of occluded test samples (black-box attack). Based on these items, we design three perturbation strategies, namely insertion, modification, and removal, to generate adversarial samples. The experiment results show that the adversarial samples generated by our method can successfully fool both state-of-the-art character-level and word-level DNN-based text classifiers. The adversarial samples can be perturbed to any desirable classes without compromising their utilities. At the same time, the introduced perturbation is difficult to be perceived.

cs.CR↗

arXiv2018

Ultrathin acoustic parity-time symmetric metasurface cloak

Invisibility or unhearability cloaks have been made possible by using metamaterials making light or sound flow around obstacle without the trace of reflections or shadows. Metamaterials are known for being flexible building units that can mimic a host of unusual and extreme material responses, which are essential when engineering artificial material properties to realize a coordinate transforming cloak. Bending and stretching the coordinate grid in space requires stringent material parameters, therefore, small inaccuracies and inevitable material losses become sources for unwanted scattering that are decremental to the desired effect. These obstacles further limit the possibility to achieve a robust concealment of sizeable objects from either radar or sonar detection. By using a elaborate arrangement of gain and lossy acoustic media respecting parity-time symmetry, we built an one-way unhearability cloak capable to hide objects seven times larger than acoustic wavelength. Generally speaking, our approach has no limits in terms of working frequency, shape, or size, specifically though, we demonstrate how, in principle, an object of the size of a human can be hidden from audible sound.

physics.app-ph↗