arXiv subjects

Hans J. Briegel

Publications and source records attributed to Hans J. Briegel.

At least 73 records · Page 4Linked to original sources

arXiv2016

Meta-learning within Projective Simulation

Learning models of artificial intelligence can nowadays perform very well on a large variety of tasks. However, in practice different task environments are best handled by different learning models, rather than a single, universal, approach. Most non-trivial models thus require the adjustment of several to many learning parameters, which is often done on a case-by-case basis by an external party. Meta-learning refers to the ability of an agent to autonomously and dynamically adjust its own learning parameters, or meta-parameters. In this work we show how projective simulation, a recently developed model of artificial intelligence, can naturally be extended to account for meta-learning in reinforcement learning settings. The projective simulation approach is based on a random walk process over a network of clips. The suggested meta-learning scheme builds upon the same design and employs clip networks to monitor the agent's performance and to adjust its meta-parameters "on the fly". We distinguish between "reflexive adaptation" and "adaptation through learning", and show the utility of both approaches. In addition, a trade-off between flexibility and learning-time is addressed. The extended model is examined on three different kinds of reinforcement learning tasks, in which the agent has different optimal values of the meta-parameters, and is shown to perform well, reaching near-optimal to optimal success rates in all of them, without ever needing to manually adjust any meta-parameter.

cs.AI↗

arXiv2016

Quantum machine learning with glow for episodic tasks and decision games

We consider a general class of models, where a reinforcement learning (RL) agent learns from cyclic interactions with an external environment via classical signals. Perceptual inputs are encoded as quantum states, which are subsequently transformed by a quantum channel representing the agent's memory, while the outcomes of measurements performed at the channel's output determine the agent's actions. The learning takes place via stepwise modifications of the channel properties. They are described by an update rule that is inspired by the projective simulation (PS) model and equipped with a glow mechanism that allows for a backpropagation of policy changes, analogous to the eligibility traces in RL and edge glow in PS. In this way, the model combines features of PS with the ability for generalization, offered by its physical embodiment as a quantum system. We apply the agent to various setups of an invasion game and a grid world, which serve as elementary model tasks allowing a direct comparison with a basic classical PS agent.

quant-ph↗

arXiv2015

Estimation of coherent error sources from stabilizer measurements

In the context of measurement-based quantum computation a way of maintaining the coherence of a graph state is to measure its stabilizer operators. Aside from performing quantum error correction, it is possible to exploit the information gained from these measurements to characterize and then counteract a coherent source of errors; that is, to determine all the parameters of an error channel that applies a fixed - but unknown - unitary operation to the physical qubits. Such a channel is generated, e.g., by local stray fields that act on the qubits. We study the case in which each qubit of a given graph state may see a different error channel and we focus on channels given by a rotation on the Bloch sphere around either the x, y or z axis, for which analytical results can be given in a compact form. The possibility of reconstructing the channels at all qubits depends non-trivially on the topology of the graph state. We prove via perturbation methods that the reconstruction process is robust and supplement the analytic results with numerical evidence.

quant-ph↗

arXiv2015

Coherent controlization using superconducting qubits

Coherent controlization, i.e., coherent conditioning of arbitrary single- or multi-qubit operations on the state of one or more control qubits, is an important ingredient for the flexible implementation of many algorithms in quantum computation. This is of particular significance when certain subroutines are changing over time or when they are frequently modified, such as in decision-making algorithms for learning agents. We propose a scheme to realize coherent controlization for any number of superconducting qubits coupled to a microwave resonator. For two and three qubits, we present an explicit construction that is of high relevance for quantum learning agents. We demonstrate the feasibility of our proposal, taking into account loss, dephasing, and the cavity self-Kerr effect.

quant-ph↗

arXiv2015

Framework for learning agents in quantum environments

In this paper we provide a broad framework for describing learning agents in general quantum environments. We analyze the types of classically specified environments which allow for quantum enhancements in learning, by contrasting environments to quantum oracles. We show that whether or not quantum improvements are at all possible depends on the internal structure of the quantum environment. If the environments are constructed and the internal structure is appropriately chosen, or if the agent has limited capacities to influence the internal states of the environment, we show that improvements in learning times are possible in a broad range of scenarios. Such scenarios we call luck-favoring settings. The case of constructed environments is particularly relevant for the class of model-based learning agents, where our results imply a near-generic improvement.

quant-ph↗

arXiv2015

Quantum mixing of Markov chains for special distributions

The preparation of the stationary distribution of irreducible, time-reversible Markov chains is a fundamental building block in many heuristic approaches to algorithmically hard problems. It has been conjectured that quantum analogs of classical mixing processes may offer a generic quadratic speed-up in realizing such stationary distributions. Such a speed-up would also imply a speed-up of a broad family of heuristic algorithms. However, a true quadratic speed up has thus far only been demonstrated for special classes of Markov chains. These results often presuppose a regular structure of the underlying graph of the Markov chain, and also a regularity in the transition probabilities. In this work, we demonstrate a true quadratic speed-up for a class of Markov chains where the restriction is only on the form of the stationary distribution, rather than directly on the Markov chain structure itself. In particular, we show efficient mixing can be achieved when it is beforehand known that the distribution is monotonically decreasing relative to a known order on the state space. Following this, we show that our approach extends to a wider class of distributions, where only a fraction of the shape of the distribution is known to be monotonic. Our approach is built on the Szegedy-type quantization of transition operators.

quant-ph↗

arXiv2015

Quantum-enhanced deliberation of learning agents using trapped ions

A scheme that successfully employs quantum mechanics in the design of autonomous learning agents has recently been reported in the context of the projective simulation (PS) model for artificial intelligence. In that approach, the key feature of a PS agent, a specific type of memory which is explored via random walks, was shown to be amenable to quantization. In particular, classical random walks were substituted by Szegedy-type quantum walks, allowing for a speed-up. In this work we propose how such classical and quantum agents can be implemented in systems of trapped ions. We employ a generic construction by which the classical agents are `upgraded' to their quantum counterparts by nested coherent controlization, and we outline how this construction can be realized in ion traps. Our results provide a flexible modular architecture for the design of PS agents. Furthermore, we present numerical simulations of simple PS agents which analyze the robustness of our proposal under certain noise models.

quant-ph↗

arXiv2014

Projective simulation for classical learning agents: a comprehensive investigation

We study the model of projective simulation (PS), a novel approach to artificial intelligence based on stochastic processing of episodic memory which was recently introduced [H.J. Briegel and G. De las Cuevas. Sci. Rep. 2, 400, (2012)]. Here we provide a detailed analysis of the model and examine its performance, including its achievable efficiency, its learning times and the way both properties scale with the problems' dimension. In addition, we situate the PS agent in different learning scenarios, and study its learning abilities. A variety of new scenarios are being considered, thereby demonstrating the model's flexibility. Furthermore, to put the PS scheme in context, we compare its performance with those of Q-learning and learning classifier systems, two popular models in the field of reinforcement learning. It is shown that PS is a competitive artificial intelligence model of unique properties and strengths.

nlin.AO↗

arXiv2014

Quantum speedup for active learning agents

Can quantum mechanics help us in building intelligent robots and agents? One of the defining characteristics of intelligent behavior is the capacity to learn from experience. However, a major bottleneck for agents to learn in any real-life situation is the size and complexity of the corresponding task environment. Owing to, e.g., a large space of possible strategies, learning is typically slow. Even for a moderate task environment, it may simply take too long to rationally respond to a given situation. If the environment is impatient, allowing only a certain time for a response, an agent may then be unable to cope with the situation and to learn at all. Here we show that quantum physics can help and provide a significant speed-up for active learning as a genuine problem of artificial intelligence. We introduce a large class of quantum learning agents for which we show a quadratic boost in their active learning efficiency over their classical analogues. This result will be particularly relevant for applications involving complex task environments.

quant-ph↗

arXiv2014

In-situ characterization of quantum devices with error correction

Syndrome measurements made in quantum error correction contain more information than is typically used. We show that the statistics of data from syndrome measurements can be used to do the following: (i) estimation of parameters of an error channel, including the ability correct away the invertible part of the error channel, once it is estimated; (ii) hypothesis testing (or model selection) to distinguish error channels, e.g., to determine if the errors are correlated. The unifying theme is to make use of all of the information in the statistics of the data collected from syndrome measurements using machine learning and control algorithms.

quant-ph↗

arXiv2014

Projective simulation applied to the grid-world and the mountain-car problem

We study the model of projective simulation (PS) which is a novel approach to artificial intelligence (AI). Recently it was shown that the PS agent performs well in a number of simple task environments, also when compared to standard models of reinforcement learning (RL). In this paper we study the performance of the PS agent further in more complicated scenarios. To that end we chose two well-studied benchmarking problems, namely the "grid-world" and the "mountain-car" problem, which challenge the model with large and continuous input space. We compare the performance of the PS agent model with those of existing models and show that the PS agent exhibits competitive performance also in such scenarios.

cs.AI↗

arXiv2014

Implementing quantum control for unknown subroutines

We present setups for the practical realization of adding control to unknown subroutines, supplementing the existing quantum optical scheme for black-box control with a counterpart for the quantum control of the ordering of sequences of operations. We also provide schemes to realize either task using trapped ions. These practical circumventions of recent no-go theorems are based on existing technologies. We argue that the possibility to add control to unknown operations in practice is a common feature of many physical systems. Based on the proposed implementations we discuss the apparent contradictions between theory and practice.

quant-ph↗

arXiv2014

Demonstrating an element of measurement-based quantum error correction

In measurement-based quantum computing an algorithm is performed by measurements on highly-entangled resource states. To date, several implementations were demonstrated, all of them assuming perfect noise-free environments. Here we consider measurement-based information processing in the presence of noise and demonstrate quantum error detection. We implement the protocol using a four-qubit photonic cluster state, where we first encode a general qubit non-locally such that phase errors can be detected. We then read out the error syndrome and analyze the output states after decoding. Our demonstration shows a building block for measurement-based quantum computing which is crucial for realistic scenarios.

quant-ph↗

arXiv2013

Multiple re-encounter approach to radical pair reactions and the role of nonlinear master equations

We formulate a multiple-encounter model of the radical pair mechanism that is based on a random coupling of the radical pair to a minimal model environment. These occasional pulse-like couplings correspond to the radical encounters and give rise to both dephasing and recombination. While this is in agreement with the original model of Haberkorn and its extensions that assume additional dephasing, we show how a nonlinear master equation may be constructed to describe the conditional evolution of the radical pairs prior to the detection of their recombination. We propose a nonlinear master equation for the evolution of an ensemble of independently evolving radical pairs whose nonlinearity depends on the record of the fluorescence signal. We also reformulate Haberkorn's original argument on the physicality of reaction operators using the terminology of quantum optics/open quantum systems. Our model allows one to describe multiple encounters within the exponential model and connects this with the master equation approach. We include hitherto neglected effects of the encounters, such as a separate dephasing in the triplet subspace, and predict potential new effects, such as Grover reflections of radical spins, that may be observed if the strength and time of the encounters can be experimentally controlled.

quant-ph↗

arXiv2013

Quantum walks on embedded hypercubes

It has been proved by Kempe that discrete quantum walks on the hypercube (HC) hit exponentially faster than the classical analog. The same was also observed numerically by Krovi and Brun for a slightly different property, namely, the expected hitting time. Yet, to what extent this striking result survives in more general graphs, is to date an open question. Here we tackle this question by studying the expected hitting time for quantum walks on HCs that are embedded into larger symmetric structures. By performing numerical simulations of the discrete quantum walk and deriving a general expression for the classical hitting time, we observe an exponentially increasing gap between the expected classical and quantum hitting times, not only for walks on the bare HC, but also for a large family of embedded HCs. This suggests that the quantum speedup is stable with respect to such embeddings.

quant-ph↗

arXiv2013

Projective simulation for artificial intelligence

We propose a model of a learning agent whose interaction with the environment is governed by a simulation-based projection, which allows the agent to project itself into future situations before it takes real action. Projective simulation is based on a random walk through a network of clips, which are elementary patches of episodic memory. The network of clips changes dynamically, both due to new perceptual input and due to certain compositional principles of the simulation process. During simulation, the clips are screened for specific features which trigger factual action of the agent. The scheme is different from other, computational, notions of simulation, and it provides a new element in an embodied cognitive science approach to intelligent action and learning. Our model provides a natural route for generalization to quantum-mechanical operation and connects the fields of reinforcement learning and quantum computation.

nlin.AO↗

arXiv2012

Quantum transport efficiency and Fourier's law

We analyze the steady-state energy transfer in a chain of coupled two-level systems connecting two thermal reservoirs. Through an analytic treatment we find that the energy current is independent of the system size, hence violating Fourier's law of heat conduction. The classical diffusive behavior in Fourier's law of heat conduction can be recovered by introducing decoherence to the quantum systems constituting the chain. Implications of these results on energy transfer in biological light harvesting systems, and the role of quantum coherences and entanglement are discussed.

quant-ph↗

arXiv2012

Reducing Spacetime to Binary Information

We present a new description of discrete space-time in 1+1 dimensions in terms of a set of elementary geometrical units that represent its independent classical degrees of freedom. This is achieved by means of a binary encoding that is ergodic in the class of space-time manifolds respecting coordinate invariance of general relativity. Space-time fluctuations can be represented in a classical lattice gas model whose Boltzmann weights are constructed with the discretized form of the Einstein-Hilbert action. Within this framework, it is possible to compute basic quantities such as the Ricci curvature tensor and the Einstein equations, and to evaluate the path integral of discrete gravity. The description as a lattice gas model also provides a novel way of quantization and, at the same time, to quantum simulation of fluctuating space-time.

gr-qc↗