Searcharxiv⌕ Search

arXiv subjects

Yuan Zhou

Publications and source records attributed to Yuan Zhou.

At least 253 records · Page 14Linked to original sources
arXiv2020

$\sqrt{n}$-Regret for Learning in Markov Decision Processes with Function Approximation and Low Bellman Rank

In this paper, we consider the problem of online learning of Markov decision processes (MDPs) with very large state spaces. Under the assumptions of realizable function approximation and low Bellman ranks, we develop an online learning algorithm that learns the optimal value function while at the same time achieving very low cumulative regret during the learning process. Our learning algorithm, Adaptive Value-function Elimination (AVE), is inspired by the policy elimination algorithm proposed in (Jiang et al., 2017), known as OLIVE. One of our key technical contributions in AVE is to formulate the elimination steps in OLIVE as contextual bandit problems. This technique enables us to apply the active elimination and expert weighting methods from (Dudik et al., 2011), instead of the random action exploration scheme used in the original OLIVE algorithm, for more efficient exploration and better control of the regret incurred in each policy elimination step. To the best of our knowledge, this is the first $\sqrt{n}$-regret result for reinforcement learning in stochastic MDPs with general value function approximation.

cs.LG↗
arXiv2020

Almost Optimal Model-Free Reinforcement Learning via Reference-Advantage Decomposition

We study the reinforcement learning problem in the setting of finite-horizon episodic Markov Decision Processes (MDPs) with $S$ states, $A$ actions, and episode length $H$. We propose a model-free algorithm UCB-Advantage and prove that it achieves $\tilde{O}(\sqrt{H^2SAT})$ regret where $T = KH$ and $K$ is the number of episodes to play. Our regret bound improves upon the results of [Jin et al., 2018] and matches the best known model-based algorithms as well as the information theoretic lower bound up to logarithmic factors. We also show that UCB-Advantage achieves low local switching cost and applies to concurrent reinforcement learning, improving upon the recent results of [Bai et al., 2019].

cs.LG↗
arXiv2020

Improving mobility of silicon metal-oxide-semiconductor devices for quantum dots by high vacuum activation annealing

To improve mobility of fabricated silicon metal-oxide-semiconductor (MOS) quantum devices, forming gas annealing is a common method used to mitigate the effects of disorder at the Si/SiO2 interface. However, the importance of activation annealing is usually ignored. Here, we show that a high vacuum environment for implantation activation is beneficial for improving mobility compared to nitrogen atmosphere. Low-temperature transport measurements of Hall bars show that peak mobility can be improved by a factor of two, reaching 1.5 m^2/(Vs) using high vacuum annealing during implantation activation. Moreover, the charge stability diagram of a single quantum dot is mapped, with no visible disturbance caused by disorder, suggesting possibility of fabricating high-quality quantum dots on commercial wafers. Our results may provide valuable insights into device optimization in silicon-based quantum computing.

cond-mat.mes-hall↗
arXiv2020

Multi-IF : An Approach to Anomaly Detection in Self-Driving Systems

Autonomous driving vehicles (ADVs) are implemented with rich software functions and equipped with many sensors, which in turn brings broad attack surface. Moreover, the execution environment of ADVs is often open and complex. Hence, ADVs are always at risk of safety and security threats. This paper proposes a fast method called Multi-IF, using multiple invocation features of system calls to detect anomalies in self-driving systems. Since self-driving functions take most of the computation resources and upgrade frequently, Multi-IF is designed to work under such resource constraints and support frequent updates. Given the collected sequences of system calls, the combination of different syntax patterns is used to analyze and construct feature vectors of those sequences. By taking the feature vectors as inputs, one-class support vector machine is adopted to determine whether the current sequence of system calls is abnormal, which is trained with the feature vectors from the normal sequences. The evaluations on both simulated and real data prove that the proposed method is effective in identifying the abnormal behavior after minutes of feature extraction and training. Further comparisons with the existing methods on the ADFA-LD data set also validate that the proposed approach achieves a higher accuracy with less time overhead.

cs.CR↗
arXiv2020

Dual-reference Age Synthesis

Age synthesis methods typically take a single image as input and use a specific number to control the age of the generated image. In this paper, we propose a novel framework taking two images as inputs, named dual-reference age synthesis (DRAS), which approaches the task differently; instead of using "hard" age information, i.e. a fixed number, our model determines the target age in a "soft" way, by employing a second reference image. Specifically, the proposed framework consists of an identity agent, an age agent and a generative adversarial network. It takes two images as input - an identity reference and an age reference - and outputs a new image that shares corresponding features with each. Experimental results on two benchmark datasets (UTKFace and CACD) demonstrate the appealing performance and flexibility of the proposed framework.

cs.CV↗
arXiv2020

Guardauto: A Decentralized Runtime Protection System for Autonomous Driving

Due to the broad attack surface and the lack of runtime protection, potential safety and security threats hinder the real-life adoption of autonomous vehicles. Although efforts have been made to mitigate some specific attacks, there are few works on the protection of the self-driving system. This paper presents a decentralized self-protection framework called Guardauto to protect the self-driving system against runtime threats. First, Guardauto proposes an isolation model to decouple the self-driving system and isolate its components with a set of partitions. Second, Guardauto provides self-protection mechanisms for each target component, which combines different methods to monitor the target execution and plan adaption actions accordingly. Third, Guardauto provides cooperation among local self-protection mechanisms to identify the root-cause component in the case of cascading failures affecting multiple components. A prototype has been implemented and evaluated on the open-source autonomous driving system Autoware. Results show that Guardauto could effectively mitigate runtime failures and attacks, and protect the control system with acceptable performance overhead.

cs.CR↗
arXiv2020

Stochastically Differentiable Probabilistic Programs

Probabilistic programs with mixed support (both continuous and discrete latent random variables) commonly appear in many probabilistic programming systems (PPSs). However, the existence of the discrete random variables prohibits many basic gradient-based inference engines, which makes the inference procedure on such models particularly challenging. Existing PPSs either require the user to manually marginalize out the discrete variables or to perform a composing inference by running inference separately on discrete and continuous variables. The former is infeasible in most cases whereas the latter has some fundamental shortcomings. We present a novel approach to run inference efficiently and robustly in such programs using stochastic gradient Markov Chain Monte Carlo family of algorithms. We compare our stochastic gradient-based inference algorithm against conventional baselines in several important cases of probabilistic programs with mixed support, and demonstrate that it outperforms existing composing inference baselines and works almost as well as inference in marginalized versions of the programs, but with less programming effort and at a lower computation cost.

cs.LG↗
arXiv2020

Soft-Root-Sign Activation Function

The choice of activation function in deep networks has a significant effect on the training dynamics and task performance. At present, the most effective and widely-used activation function is ReLU. However, because of the non-zero mean, negative missing and unbounded output, ReLU is at a potential disadvantage during optimization. To this end, we introduce a novel activation function to manage to overcome the above three challenges. The proposed nonlinearity, namely "Soft-Root-Sign" (SRS), is smooth, non-monotonic, and bounded. Notably, the bounded property of SRS distinguishes itself from most state-of-the-art activation functions. In contrast to ReLU, SRS can adaptively adjust the output by a pair of independent trainable parameters to capture negative information and provide zero-mean property, which leading not only to better generalization performance, but also to faster learning speed. It also avoids and rectifies the output distribution to be scattered in the non-negative real number space, making it more compatible with batch normalization (BN) and less sensitive to initialization. In experiments, we evaluated SRS on deep networks applied to a variety of tasks, including image classification, machine translation and generative modelling. Our SRS matches or exceeds models with ReLU and other state-of-the-art nonlinearities, showing that the proposed activation function is generalized and can achieve high performance across tasks. Ablation study further verified the compatibility with BN and self-adaptability for different initialization.

cs.CV↗
arXiv2020

Phononic waveguide assisted steady state entanglement of SiV centers

Multiparticle entanglement is of great significance for quantum metrology and quantum information processing. We here present an efficient scheme to generate stable multiparticle entanglement in a solid state setup, where an array of silicon-vacancy centers are embedded in a quasi-one-dimensional acoustic diamond waveguide. In this scheme, the continuum of phonon modes induces a controllable dissipative coupling among the SiV centers. We show that, by an appropriate choice of the distance between the SiV centers, the dipole-dipole interactions can be switched off due to destructive interferences, thus realizing a Dicke superradiance model. This gives rise to an entangled steady state of SiV centers with high fidelities. The protocol provides a feasible setup for the generation of multiparticle entanglement in a solid state system.

quant-ph↗
arXiv2020

Anypath Routing Protocol Design via Q-Learning for Underwater Sensor Networks

As a promising technology in the Internet of Underwater Things, underwater sensor networks have drawn a widespread attention from both academia and industry. However, designing a routing protocol for underwater sensor networks is a great challenge due to high energy consumption and large latency in the underwater environment. This paper proposes a Q-learning-based localization-free anypath routing (QLFR) protocol to prolong the lifetime as well as reduce the end-to-end delay for underwater sensor networks. Aiming at optimal routing policies, the Q-value is calculated by jointly considering the residual energy and depth information of sensor nodes throughout the routing process. More specifically, we define two reward functions (i.e., depth-related and energy-related rewards) for Q-learning with the objective of reducing latency and extending network lifetime. In addition, a new holding time mechanism for packet forwarding is designed according to the priority of forwarding candidate nodes. Furthermore, a mathematical analysis is presented to analyze the performance of the proposed routing protocol. Extensive simulation results demonstrate the superiority performance of the proposed routing protocol in terms of the end-to-end delay and the network lifetime.

cs.NI↗
arXiv2020

Domain Adaptive Adversarial Learning Based on Physics Model Feedback for Underwater Image Enhancement

Owing to refraction, absorption, and scattering of light by suspended particles in water, raw underwater images suffer from low contrast, blurred details, and color distortion. These characteristics can significantly interfere with the visibility of underwater images and the result of visual tasks, such as segmentation and tracking. To address this problem, we propose a new robust adversarial learning framework via physics model based feedback control and domain adaptation mechanism for enhancing underwater images to get realistic results. A new method for simulating underwater-like training dataset from RGB-D data by underwater image formation model is proposed. Upon the synthetic dataset, a novel enhancement framework, which introduces a domain adaptive mechanism as well as a physics model constraint feedback control, is trained to enhance the underwater scenes. Final enhanced results on synthetic and real underwater images demonstrate the superiority of the proposed method, which outperforms nondeep and deep learning methods in both qualitative and quantitative evaluations. Furthermore, we perform an ablation study to show the contributions of each component we proposed.

cs.CV↗
arXiv2020

Furnishing Your Room by What You See: An End-to-End Furniture Set Retrieval Framework with Rich Annotated Benchmark Dataset

Understanding interior scenes has attracted enormous interest in computer vision community. However, few works focus on the understanding of furniture within the scenes and a large-scale dataset is also lacked to advance the field. In this paper, we first fill the gap by presenting DeepFurniture, a richly annotated large indoor scene dataset, including 24k indoor images, 170k furniture instances and 20k unique furniture identities. On the dataset, we introduce a new benchmark, named furniture set retrieval. Given an indoor photo as input, the task requires to detect all the furniture instances and search a matched set of furniture identities. To address this challenging task, we propose a feature and context embedding based framework. It contains 3 major contributions: (1) An improved Mask-RCNN model with an additional mask-based classifier is introduced for better utilizing the mask information to relieve the occlusion problems in furniture detection context. (2) A multi-task style Siamese network is proposed to train the feature embedding model for retrieval, which is composed of a classification subnet supervised by self-clustered pseudo attributes and a verification subnet to estimate whether the input pair is matched. (3) In order to model the relationship of the furniture entities in an interior design, a context embedding model is employed to re-rank the retrieval results. Extensive experiments demonstrate the effectiveness of each module and the overall system.

cs.CV↗
arXiv2020

Equivariant Perturbation in Gomory and Johnson's Infinite Group Problem. VII. Inverse semigroup theory, closures, decomposition of perturbations

In this self-contained paper, we present a theory of the piecewise linear minimal valid functions for the 1-row Gomory-Johnson infinite group problem. The non-extreme minimal valid functions are those that admit effective perturbations. We give a precise description of the space of these perturbations as a direct sum of certain finite- and infinite-dimensional subspaces. The infinite-dimensional subspaces have partial symmetries; to describe them, we develop a theory of inverse semigroups of partial bijections, interacting with the functional equations satisfied by the perturbations. Our paper provides the foundation for grid-free algorithms for the Gomory-Johnson model, in particular for testing extremality of piecewise linear functions whose breakpoints are rational numbers with huge denominators.

math.OC↗
arXiv2019

Exploration via Hindsight Goal Generation

Goal-oriented reinforcement learning has recently been a practical framework for robotic manipulation tasks, in which an agent is required to reach a certain goal defined by a function on the state space. However, the sparsity of such reward definition makes traditional reinforcement learning algorithms very inefficient. Hindsight Experience Replay (HER), a recent advance, has greatly improved sample efficiency and practical applicability for such problems. It exploits previous replays by constructing imaginary goals in a simple heuristic way, acting like an implicit curriculum to alleviate the challenge of sparse reward signal. In this paper, we introduce Hindsight Goal Generation (HGG), a novel algorithmic framework that generates valuable hindsight goals which are easy for an agent to achieve in the short term and are also potential for guiding the agent to reach the actual goal in the long term. We have extensively evaluated our goal generation algorithm on a number of robotic manipulation tasks and demonstrated substantially improvement over the original HER in terms of sample efficiency.

cs.LG↗
arXiv2019

Exploiting Operation Importance for Differentiable Neural Architecture Search

Recently, differentiable neural architecture search methods significantly reduce the search cost by constructing a super network and relax the architecture representation by assigning architecture weights to the candidate operations. All the existing methods determine the importance of each operation directly by architecture weights. However, architecture weights cannot accurately reflect the importance of each operation; that is, the operation with the highest weight might not related to the best performance. To alleviate this deficiency, we propose a simple yet effective solution to neural architecture search, termed as exploiting operation importance for effective neural architecture search (EoiNAS), in which a new indicator is proposed to fully exploit the operation importance and guide the model search. Based on this new indicator, we propose a gradual operation pruning strategy to further improve the search efficiency and accuracy. Experimental results have demonstrated the effectiveness of the proposed method. Specifically, we achieve an error rate of 2.50\% on CIFAR-10, which significantly outperforms state-of-the-art methods. When transferred to ImageNet, it achieves the top-1 error of 25.6\%, comparable to the state-of-the-art performance under the mobile setting.

cs.CV↗
arXiv2019

Quasicrystalline Chern Insulators

Chern insulator or quantum anomalous Hall state is a topological state with integer Hall conductivity but in absence of Landau level. It had been well established on various two-dimensional lattices with periodic structure. Here, we report similar Chern insulators can also be realized on the quasicrystal with $5$-fold rotational symmetry. Providing the staggered flux through plaquettes, we propose two types of quasicrystalline Chern insulators. Their topological characterizations are well identified by the robustness of edge states, non-zero real-space Chern number, and quantized conductance. We further find the failure of integer conductivity but with quantized Chern number at some special energies. Our study therefore provide a new opportunity to searching topological materials in aperiodic system.

cond-mat.mes-hall↗
arXiv2019

Subcarrier Assignment Schemes Based on Q-Learning in Wideband Cognitive Radio Networks

Subcarrier assignment is of crucial importance in wideband cognitive radio (CR) networks. In order to tackle the challenge that the traditional optimization-based methods are inappropriate in the dynamic spectrum access environment, an independent Q-learning-based scheme is proposed for the case that the secondary users (SUs) cannot exchange information while a collaborative Q-learning-based scheme is proposed for the case that information can be exchange among SUs. Simulation results show that the performance achieved with the proposed collaborative Q-learning-based assignment is better than that obtained with the proposed independent Q-learning-based assignment at the cost of the computation cost.

eess.SP↗