arXiv subjects

Ziyu Wang

Publications and source records attributed to Ziyu Wang.

At least 181 records · Page 10Linked to original sources

arXiv2019

Towards a Multi-Chain Future of Proof-of-Space

Proof-of-Space provides an intriguing alternative for consensus protocol of permissionless blockchains due to its recyclable nature and the potential to support multiple chains simultaneously. However, a direct shared proof of the same storage, which was adopted in the existing multi-chain schemes based on Proof-of-Space, could give rise to newborn attack on new chain launching. To fix this gap, we propose an innovative framework of single-chain Proof-of-Space and further present a novel multi-chain scheme which can resist newborn attack effectively by elaborately combining shared proof and chain-specific proof of storage. Moreover, we analyze the security of the multi-chain scheme and prove that it is incentive-compatible. This means that participants in such multi-chain system can achieve their greatest utility with our proposed strategy of storage resource partition.

cs.CR↗

arXiv2019

Function Space Particle Optimization for Bayesian Neural Networks

While Bayesian neural networks (BNNs) have drawn increasing attention, their posterior inference remains challenging, due to the high-dimensional and over-parameterized nature. To address this issue, several highly flexible and scalable variational inference procedures based on the idea of particle optimization have been proposed. These methods directly optimize a set of particles to approximate the target posterior. However, their application to BNNs often yields sub-optimal performance, as such methods have a particular failure mode on over-parameterized models. In this paper, we propose to solve this issue by performing particle optimization directly in the space of regression functions. We demonstrate through extensive experiments that our method successfully overcomes this issue, and outperforms strong baselines in a variety of tasks including prediction, defense against adversarial examples, and reinforcement learning.

stat.ML↗

arXiv2018

A Framework for Automated Pop-song Melody Generation with Piano Accompaniment Arrangement

We contribute a pop-song automation framework for lead melody generation and accompaniment arrangement. The framework reflects the major procedures of human music composition, generating both lead melody and piano accompaniment by a unified strategy. Specifically, we take chord progression as an input and propose three models to generate a structured melody with piano accompaniment textures. First, the harmony alternation model transforms a raw input chord progression to an altered one to better fit the specified music style. Second, the melody generation model generates the lead melody and other voices (melody lines) of the accompaniment using seasonal ARMA (Autoregressive Moving Average) processes. Third, the melody integration model integrates melody lines (voices) together as the final piano accompaniment. We evaluate the proposed framework using subjective listening tests. Experimental results show that the generated melodies are rated significantly higher than the ones generated by bi-directional LSTM, and our accompaniment arrangement result is comparable with a state-of-the-art commercial software, Band in a Box.

cs.SD↗

arXiv2018

Bayesian Optimization in AlphaGo

During the development of AlphaGo, its many hyper-parameters were tuned with Bayesian optimization multiple times. This automatic tuning process resulted in substantial improvements in playing strength. For example, prior to the match with Lee Sedol, we tuned the latest AlphaGo agent and this improved its win-rate from 50% to 66.5% in self-play games. This tuned version was deployed in the final match. Of course, since we tuned AlphaGo many times during its development cycle, the compounded contribution was even higher than this percentage. It is our hope that this brief case study will be of interest to Go fans, and also provide Bayesian optimization practitioners with some insights and inspiration.

cs.LG↗

arXiv2018

Playing hard exploration games by watching YouTube

Deep reinforcement learning methods traditionally struggle with tasks where environment rewards are particularly sparse. One successful method of guiding exploration in these domains is to imitate trajectories provided by a human demonstrator. However, these demonstrations are typically collected under artificial conditions, i.e. with access to the agent's exact environment setup and the demonstrator's action and reward trajectories. Here we propose a two-stage method that overcomes these limitations by relying on noisy, unaligned footage without access to such data. First, we learn to map unaligned videos from multiple sources to a common representation using self-supervised objectives constructed over both time and modality (i.e. vision and sound). Second, we embed a single YouTube video in this representation to construct a reward function that encourages an agent to imitate human gameplay. This method of one-shot imitation allows our agent to convincingly exceed human-level performance on the infamously hard exploration games Montezuma's Revenge, Pitfall! and Private Eye for the first time, even if the agent is not presented with any environment rewards.

cs.LG↗

arXiv2018

One-Shot High-Fidelity Imitation: Training Large-Scale Deep Nets with RL

Humans are experts at high-fidelity imitation -- closely mimicking a demonstration, often in one attempt. Humans use this ability to quickly solve a task instance, and to bootstrap learning of new tasks. Achieving these abilities in autonomous agents is an open problem. In this paper, we introduce an off-policy RL algorithm (MetaMimic) to narrow this gap. MetaMimic can learn both (i) policies for high-fidelity one-shot imitation of diverse novel skills, and (ii) policies that enable the agent to solve tasks more efficiently than the demonstrators. MetaMimic relies on the principle of storing all experiences in a memory and replaying these to learn massive deep neural network policies by off-policy RL. This paper introduces, to the best of our knowledge, the largest existing neural networks for deep RL and shows that larger networks with normalization are needed to achieve one-shot high-fidelity imitation on a challenging manipulation task. The results also show that both types of policy can be learned from vision, in spite of the task rewards being sparse, and without access to demonstrator actions.

cs.LG↗

arXiv2018

Bright room temperature single photon source at telecom range in cubic silicon carbide

Single photon emitters (SPEs) play an important role in a number of quantum information tasks such as quantum key distributions. In these protocols, telecom wavelength photons are desired due to their low transmission loss in optical fibers. In this paper, we present a study of bright single-photon emitters in cubic silicon carbide (3C-SiC) emitting in the telecom range. We find that these emitters are photostable and bright at room temperature with a count rate of ~ MHz. Together with the fact that SiC is a growth and fabrication-friendly material, our result may pave the way for its future application in quantum communication technology applications.

physics.optics↗

arXiv2018

Reinforcement and Imitation Learning for Diverse Visuomotor Skills

We propose a model-free deep reinforcement learning method that leverages a small amount of demonstration data to assist a reinforcement learning agent. We apply this approach to robotic manipulation tasks and train end-to-end visuomotor policies that map directly from RGB camera inputs to joint velocities. We demonstrate that our approach can solve a wide variety of visuomotor tasks, for which engineering a scripted controller would be laborious. In experiments, our reinforcement and imitation agent achieves significantly better performances than agents trained with reinforcement learning or imitation learning alone. We also illustrate that these policies, trained with large visual and dynamics variations, can achieve preliminary successes in zero-shot sim2real transfer. A brief visual description of this work can be viewed in https://youtu.be/EDl8SQUNjj0

cs.RO↗

arXiv2018

High Thermoelectric Performance of Au@Sb2Te3 Heterostructure Derived from the Potential Barriers

The correlated couple of electrical and thermal property is the challenge to realize a substantial leap in thermoelectric materials.Synthesis of semiconductor and metal composites is a significant and versatile design strategy to optimize the thermoelectric performance driven by tailored interface between nanoinclusions and matrix.In this study, we present the simultaneous increase of electrical conductivity and Seebeck coefficient, and reduction of thermal conductivity in Sb2Te3-Au system.The enhanced electrical conductivity lies in the incorporated Au nanostructures contributing to injecting carriers to Sb2Te3 matrix.The appropriate barriers originated from the Au-Sb2Te3 interface, which filter low energy carriers, results in enhancement of Seebeck coefficient.The increased boundaries and nanodomains block the transport of phonons, subsequently reducing the thermal conductivity.As a consequence, combination of these effects promote double of ZT value in 1% Au@ Sb2Te3 composites with respect to the pristine Sb2Te3.

physics.app-ph↗

arXiv2017

Room-temperature solid state quantum emitters in the telecom range

On demand single photon emitters (SPEs) play a key role across a broad range of quantum technologies, including quantum computation, quantum simulation, quantum metrology and quantum communications. In quantum networks and quantum key distribution protocols, where photons are employed as flying qubits, telecom wavelength operation is preferred due to the reduced fibre loss. However, despite the tremendous efforts to develop various triggered SPE platforms, a robust source of triggered SPEs operating at room temperature and the telecom wavelength is still missing. Here we report a triggered, optically stable, room temperature solid state SPE operating at telecom wavelengths. The emitters exhibit high photon purity (~ 5% multiphoton events) and a record-high brightness of ~ 1.5 MHz. The emission is attributed to localized defects in a gallium nitride (GaN) crystal. The high performance SPEs embedded in a technologically mature semiconductor are promising for on-chip quantum simulators and practical quantum communication technologies.

quant-ph↗

arXiv2017

Robust Imitation of Diverse Behaviors

Deep generative models have recently shown great promise in imitation learning for motor control. Given enough data, even supervised approaches can do one-shot imitation learning; however, they are vulnerable to cascading failures when the agent trajectory diverges from the demonstrations. Compared to purely supervised methods, Generative Adversarial Imitation Learning (GAIL) can learn more robust controllers from fewer demonstrations, but is inherently mode-seeking and more difficult to train. In this paper, we show how to combine the favourable aspects of these two approaches. The base of our model is a new type of variational autoencoder on demonstration trajectories that learns semantic policy embeddings. We show that these embeddings can be learned on a 9 DoF Jaco robot arm in reaching tasks, and then smoothly interpolated with a resulting smooth interpolation of reaching behavior. Leveraging these policy representations, we develop a new version of GAIL that (1) is much more robust than the purely-supervised controller, especially with few demonstrations, and (2) avoids mode collapse, capturing many diverse behaviors when GAIL on its own does not. We demonstrate our approach on learning diverse gaits from demonstration on a 2D biped and a 62 DoF 3D humanoid in the MuJoCo physics environment.

cs.LG↗

arXiv2017

The Intentional Unintentional Agent: Learning to Solve Many Continuous Control Tasks Simultaneously

This paper introduces the Intentional Unintentional (IU) agent. This agent endows the deep deterministic policy gradients (DDPG) agent for continuous control with the ability to solve several tasks simultaneously. Learning to solve many tasks simultaneously has been a long-standing, core goal of artificial intelligence, inspired by infant development and motivated by the desire to build flexible robot manipulators capable of many diverse behaviours. We show that the IU agent not only learns to solve many tasks simultaneously but it also learns faster than agents that target a single task at-a-time. In some cases, where the single task DDPG method completely fails, the IU agent successfully solves the task. To demonstrate this, we build a playroom environment using the MuJoCo physics engine, and introduce a grounded formal language to automatically generate tasks.

cs.AI↗

arXiv2017

Sample Efficient Actor-Critic with Experience Replay

This paper presents an actor-critic deep reinforcement learning agent with experience replay that is stable, sample efficient, and performs remarkably well on challenging environments, including the discrete 57-game Atari domain and several continuous control problems. To achieve this, the paper introduces several innovations, including truncated importance sampling with bias correction, stochastic dueling network architectures, and a new trust region policy optimization method.

cs.LG↗

arXiv2017

Learning human behaviors from motion capture by adversarial imitation

Rapid progress in deep reinforcement learning has made it increasingly feasible to train controllers for high-dimensional humanoid bodies. However, methods that use pure reinforcement learning with simple reward functions tend to produce non-humanlike and overly stereotyped movement behaviors. In this work, we extend generative adversarial imitation learning to enable training of generic neural network policies to produce humanlike movement patterns from limited demonstrations consisting only of partially observed state features, without access to actions, even when the demonstrations come from a body with different and unknown physical parameters. We leverage this approach to build sub-skill policies from motion capture data and show that they can be reused to solve tasks when controlled by a higher level controller.

cs.RO↗

arXiv2017

Emergence of Locomotion Behaviours in Rich Environments

The reinforcement learning paradigm allows, in principle, for complex behaviours to be learned directly from simple reward signals. In practice, however, it is common to carefully hand-design the reward function to encourage a particular solution, or to derive it from demonstration data. In this paper explore how a rich environment can help to promote the learning of complex behavior. Specifically, we train agents in diverse environmental contexts, and find that this encourages the emergence of robust behaviours that perform well across a suite of tasks. We demonstrate this principle for locomotion -- behaviours that are known for their sensitivity to the choice of reward. We train several simulated bodies on a diverse set of challenging terrains and obstacles, using a simple reward function based on forward progress. Using a novel scalable variant of policy gradient reinforcement learning, our agents learn to run, jump, crouch and turn as required by the environment without explicit reward-based guidance. A visual depiction of highlights of the learned behavior can be viewed following https://youtu.be/hx_bgoTF7bs .

cs.AI↗

arXiv2017

Optical properties of Xe color centers in diamond

Optical properties of color centers in diamond have been the subject of intense research due to their promising applications in quantum photonics. In this work we study the optical properties of Xe related color centers implanted into nitrogen rich (type IIA) and an ultrapure, electronic grade diamond. The Xe defect has two zero phonon lines at ~ 794 and 811 nm, which can be effectively excited using both green and red excitation, however, its emission in the nitrogen rich diamond is brighter. Near resonant excitation is performed at cryogenic temperatures and luminescence is probed under strong magnetic field. Our results are important towards the understanding of the Xe related defect and other near infrared color centers in diamond.

cond-mat.other↗

arXiv2017

Parallel Multiscale Autoregressive Density Estimation

PixelCNN achieves state-of-the-art results in density estimation for natural images. Although training is fast, inference is costly, requiring one network evaluation per pixel; O(N) for N pixels. This can be sped up by caching activations, but still involves generating each pixel sequentially. In this work, we propose a parallelized PixelCNN that allows more efficient inference by modeling certain pixel groups as conditionally independent. Our new PixelCNN model achieves competitive density estimation and orders of magnitude speedup - O(log N) sampling instead of O(N) - enabling the practical generation of 512x512 images. We evaluate the model on class-conditional image generation, text-to-image synthesis, and action-conditional video generation, showing that our model achieves the best results among non-pixel-autoregressive density models that allow efficient sampling.

cs.CV↗

arXiv2017

Scalable fabrication of single silicon vacancy defect arrays in silicon carbide using focused ion beam

In this work, we present a method for targeted, maskless, and scalable fabrication of single silicon vacancy (VSi) defect arrays in silicon carbide (SiC) using focused ion beam. The resolution of implanted VSi defects is limited to a few tens of nanometers, defined by the diameter of the ion beam. Firstly, we studied the photoluminescence (PL) spectrum and optically detected magnetic resonance (ODMR) of the generated defect spin ensemble, confirming that the synthesized centers were in the desired defect state. Then we investigated the fluorescence properties of single VSi defects and our measurements indicate the presence of a photostable single photon source. Finally, we find that the Si++ ion to VSi defect conversion yield increases as the implanted dose decreases. The reliable production of VSi defects in silicon carbide could pave the way for its applications in quantum photonics and quantum information processing.

quant-ph↗