Searcharxiv⌕ Search

arXiv subjects

Andrew Zhao

Publications and source records attributed to Andrew Zhao.

49 records · Page 3Linked to original sources
arXiv2023

Learning, Optimizing, and Simulating Fermions with Quantum Computers

Fermions are fundamental particles which obey seemingly bizarre quantum-mechanical principles, yet constitute all the ordinary matter that we inhabit. As such, their study is heavily motivated from both fundamental and practical incentives. In this dissertation, we will explore how the tools of quantum information and computation can assist us on both of these fronts. We primarily do so through the task of partial state learning: tomographic protocols for acquiring a reduced, but sufficient, classical description of a quantum system. Developing fast methods for partial tomography addresses a critical bottleneck in quantum simulation algorithms, which is a particularly pressing issue for currently available, imperfect quantum machines. At the same time, in the search for such protocols, we also refine our notion of what it means to learn quantum states. One important example is the ability to articulate, from a computational perspective, how the learning of fermions contrasts with other types of particles.

quant-ph↗
arXiv2023

Augmenting Unsupervised Reinforcement Learning with Self-Reference

Humans possess the ability to draw on past experiences explicitly when learning new tasks and applying them accordingly. We believe this capacity for self-referencing is especially advantageous for reinforcement learning agents in the unsupervised pretrain-then-finetune setting. During pretraining, an agent's past experiences can be explicitly utilized to mitigate the nonstationarity of intrinsic rewards. In the finetuning phase, referencing historical trajectories prevents the unlearning of valuable exploratory behaviors. Motivated by these benefits, we propose the Self-Reference (SR) approach, an add-on module explicitly designed to leverage historical information and enhance agent performance within the pretrain-finetune paradigm. Our approach achieves state-of-the-art results in terms of Interquartile Mean (IQM) performance and Optimality Gap reduction on the Unsupervised Reinforcement Learning Benchmark for model-free methods, recording an 86% IQM and a 16% Optimality Gap. Additionally, it improves current algorithms by up to 17% IQM and reduces the Optimality Gap by 31%. Beyond performance enhancement, the Self-Reference add-on also increases sample efficiency, a crucial attribute for real-world applications.

cs.LG↗
arXiv2023

Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplation

Recent breakthroughs in large language models (LLMs) have brought remarkable success in the field of LLM-as-Agent. Nevertheless, a prevalent assumption is that the information processed by LLMs is consistently honest, neglecting the pervasive deceptive or misleading information in human society and AI-generated content. This oversight makes LLMs susceptible to malicious manipulations, potentially resulting in detrimental outcomes. This study utilizes the intricate Avalon game as a testbed to explore LLMs' potential in deceptive environments. Avalon, full of misinformation and requiring sophisticated logic, manifests as a "Game-of-Thoughts". Inspired by the efficacy of humans' recursive thinking and perspective-taking in the Avalon game, we introduce a novel framework, Recursive Contemplation (ReCon), to enhance LLMs' ability to identify and counteract deceptive information. ReCon combines formulation and refinement contemplation processes; formulation contemplation produces initial thoughts and speech, while refinement contemplation further polishes them. Additionally, we incorporate first-order and second-order perspective transitions into these processes respectively. Specifically, the first-order allows an LLM agent to infer others' mental states, and the second-order involves understanding how others perceive the agent's mental state. After integrating ReCon with different LLMs, extensive experiment results from the Avalon game indicate its efficacy in aiding LLMs to discern and maneuver around deceptive information without extra fine-tuning and data. Finally, we offer a possible explanation for the efficacy of ReCon and explore the current limitations of LLMs in terms of safety, reasoning, speaking style, and format, potentially furnishing insights for subsequent research.

cs.AI↗
arXiv2023

Quantum simulation of exact electron dynamics can be more efficient than classical mean-field methods

Quantum algorithms for simulating electronic ground states are slower than popular classical mean-field algorithms such as Hartree-Fock and density functional theory, but offer higher accuracy. Accordingly, quantum computers have been predominantly regarded as competitors to only the most accurate and costly classical methods for treating electron correlation. However, here we tighten bounds showing that certain first quantized quantum algorithms enable exact time evolution of electronic systems with exponentially less space and polynomially fewer operations in basis set size than conventional real-time time-dependent Hartree-Fock and density functional theory. Although the need to sample observables in the quantum algorithm reduces the speedup, we show that one can estimate all elements of the $k$-particle reduced density matrix with a number of samples scaling only polylogarithmically in basis set size. We also introduce a more efficient quantum algorithm for first quantized mean-field state preparation that is likely cheaper than the cost of time evolution. We conclude that quantum speedup is most pronounced for finite temperature simulations and suggest several practically important electron dynamics problems with potential quantum advantage.

quant-ph↗
arXiv2022

Provable General Function Class Representation Learning in Multitask Bandits and MDPs

While multitask representation learning has become a popular approach in reinforcement learning (RL) to boost the sample efficiency, the theoretical understanding of why and how it works is still limited. Most previous analytical works could only assume that the representation function is already known to the agent or from linear function class, since analyzing general function class representation encounters non-trivial technical obstacles such as generalization guarantee, formulation of confidence bound in abstract function space, etc. However, linear-case analysis heavily relies on the particularity of linear function class, while real-world practice usually adopts general non-linear representation functions like neural networks. This significantly reduces its applicability. In this work, we extend the analysis to general function class representations. Specifically, we consider an agent playing $M$ contextual bandits (or MDPs) concurrently and extracting a shared representation function $ϕ$ from a specific function class $Φ$ using our proposed Generalized Functional Upper Confidence Bound algorithm (GFUCB). We theoretically validate the benefit of multitask representation learning within general function class for bandits and linear MDP for the first time. Lastly, we conduct experiments to demonstrate the effectiveness of our algorithm with neural net representation.

cs.LG↗
arXiv2022

A Mixture of Surprises for Unsupervised Reinforcement Learning

Unsupervised reinforcement learning aims at learning a generalist policy in a reward-free manner for fast adaptation to downstream tasks. Most of the existing methods propose to provide an intrinsic reward based on surprise. Maximizing or minimizing surprise drives the agent to either explore or gain control over its environment. However, both strategies rely on a strong assumption: the entropy of the environment's dynamics is either high or low. This assumption may not always hold in real-world scenarios, where the entropy of the environment's dynamics may be unknown. Hence, choosing between the two objectives is a dilemma. We propose a novel yet simple mixture of policies to address this concern, allowing us to optimize an objective that simultaneously maximizes and minimizes the surprise. Concretely, we train one mixture component whose objective is to maximize the surprise and another whose objective is to minimize the surprise. Hence, our method does not make assumptions about the entropy of the environment's dynamics. We call our method a $\textbf{M}\text{ixture }\textbf{O}\text{f }\textbf{S}\text{urprise}\textbf{S}$ (MOSS) for unsupervised reinforcement learning. Experimental results show that our simple method achieves state-of-the-art performance on the URLB benchmark, outperforming previous pure surprise maximization-based objectives. Our code is available at: https://github.com/LeapLabTHU/MOSS.

cs.LG↗
arXiv2022

Fermionic partial tomography via classical shadows

We propose a tomographic protocol for estimating any $ k $-body reduced density matrix ($ k $-RDM) of an $ n $-mode fermionic state, a ubiquitous step in near-term quantum algorithms for simulating many-body physics, chemistry, and materials. Our approach extends the framework of classical shadows, a randomized approach to learning a collection of quantum-state properties, to the fermionic setting. Our sampling protocol uses randomized measurement settings generated by a discrete group of fermionic Gaussian unitaries, implementable with linear-depth circuits. We prove that estimating all $ k $-RDM elements to additive precision $ \varepsilon $ requires on the order of $ \binom{n}{k} k^{3/2} \log(n) / \varepsilon^2 $ repeated state preparations, which is optimal up to the logarithmic factor. Furthermore, numerical calculations show that our protocol offers a substantial improvement in constant overheads for $ k \geq 2 $, as compared to prior deterministic strategies. We also adapt our method to particle-number symmetry, wherein the additional circuit depth may be halved at the cost of roughly 2-5 times more repetitions.

quant-ph↗
arXiv2022

Quantum computational advantage attested by nonlocal games with the cyclic cluster state

We propose a set of Bell-type nonlocal games that can be used to prove an unconditional quantum advantage in an objective and hardware-agnostic manner. In these games, the circuit depth needed to prepare a cyclic cluster state and measure a subset of its Pauli stabilizers on a quantum computer is compared to that of classical Boolean circuits with the same, nearest-neighboring gate connectivity. Using a circuit-based trapped-ion quantum computer, we prepare and measure a six-qubit cyclic cluster state with an overall fidelity of 60.6% and 66.4%, before and after correcting for measurement-readout errors, respectively. Our experimental results indicate that while this fidelity readily passes conventional (or depth-0) Bell bounds for local hidden-variable models, it is on the cusp of demonstrating a higher probability of success than what is possible by depth-1 classical circuits. Our games offer a practical and scalable set of quantitative benchmarks for quantum computers in the pre-fault-tolerant regime as the number of qubits available increases.

quant-ph↗
arXiv2021

DMotion: Robotic Visuomotor Control with Unsupervised Forward Model Learned from Videos

Learning an accurate model of the environment is essential for model-based control tasks. Existing methods in robotic visuomotor control usually learn from data with heavily labelled actions, object entities or locations, which can be demanding in many cases. To cope with this limitation, we propose a method, dubbed DMotion, that trains a forward model from video data only, via disentangling the motion of controllable agent to model the transition dynamics. An object extractor and an interaction learner are trained in an end-to-end manner without supervision. The agent's motions are explicitly represented using spatial transformation matrices containing physical meanings. In the experiments, DMotion achieves superior performance on learning an accurate forward model in a Grid World environment, as well as a more realistic robot control environment in simulation. With the accurate learned forward models, we further demonstrate their usage in model predictive control as an effective approach for robotic manipulations.

cs.RO↗
arXiv2020

Measurement reduction in variational quantum algorithms

Variational quantum algorithms are promising applications of noisy intermediate-scale quantum (NISQ) computers. These algorithms consist of a number of separate prepare-and-measure experiments that estimate terms in a Hamiltonian. The number of terms can become overwhelmingly large for problems at the scale of NISQ hardware that may soon be available. We approach this problem from the perspective of contextuality, and use unitary partitioning (developed independently by Izmaylov et al. [J. Chem. Theory Comput. 16, 190 (2020)]) to define variational quantum eigensolver procedures in which additional unitary operations are appended to the ansatz preparation to reduce the number of terms. This approach may be scaled to use all coherent resources available after ansatz preparation. We also study the use of asymmetric qubitization to implement the additional coherent operations with lower circuit depth. We investigate this technique for lattice Hamiltonians, random Pauli Hamiltonians, and electronic structure Hamiltonians. Using this technique, we find a constant factor speedup for lattice and random Pauli Hamiltonians. For electronic structure Hamiltonians, we prove that linear term reduction with respect to the number of orbitals, which has been previously observed in numerical studies, is always achievable. For systems represented on 10--30 qubits, we find that there is a reduction in the number of terms by approximately an order of magnitude. Applied to the plane-wave dual basis representation of fermionic Hamiltonians, however, unitary partitioning offers only a constant factor reduction. Finally, we show that noncontextual Hamiltonians may be reduced to effective commuting Hamiltonians using unitary partitioning.

quant-ph↗
arXiv2020

The No-Flippancy Game

We analyze a coin-based game with two players where, before starting the game, each player selects a string of length $n$ comprised of coin tosses. They alternate turns, choosing the outcome of a coin toss according to specific rules. As a result, the game is deterministic. The player whose string appears first wins. If neither player's string occurs, then the game must be infinite. We study several aspects of this game. We show that if, after $4n-4$ turns, the game fails to cease, it must be infinite. Furthermore, we examine how a player may select their string to force a desired outcome. Finally, we describe the result of the game for particular cases.

math.CO↗
arXiv2015

Prevalence and recoverability of syntactic parameters in sparse distributed memories

We propose a new method, based on Sparse Distributed Memory (Kanerva Networks), for studying dependency relations between different syntactic parameters in the Principles and Parameters model of Syntax. We store data of syntactic parameters of world languages in a Kanerva Network and we check the recoverability of corrupted parameter data from the network. We find that different syntactic parameters have different degrees of recoverability. We identify two different effects: an overall underlying relation between the prevalence of parameters across languages and their degree of recoverability, and a finer effect that makes some parameters more easily recoverable beyond what their prevalence would indicate. We interpret a higher recoverability for a syntactic parameter as an indication of the existence of a dependency relation, through which the given parameter can be determined using the remaining uncorrupted data.

cs.CL↗