arXiv subjects

Yiheng Lin

Publications and source records attributed to Yiheng Lin.

At least 55 records · Page 3Linked to original sources

arXiv2023

Online Adaptive Policy Selection in Time-Varying Systems: No-Regret via Contractive Perturbations

We study online adaptive policy selection in systems with time-varying costs and dynamics. We develop the Gradient-based Adaptive Policy Selection (GAPS) algorithm together with a general analytical framework for online policy selection via online optimization. Under our proposed notion of contractive policy classes, we show that GAPS approximates the behavior of an ideal online gradient descent algorithm on the policy parameters while requiring less information and computation. When convexity holds, our algorithm is the first to achieve optimal policy regret. When convexity does not hold, we provide the first local regret bound for online policy selection. Our numerical experiments show that GAPS can adapt to changing environments more quickly than existing benchmarks.

math.OC↗

arXiv2023

Online switching control with stability and regret guarantees

This paper considers online switching control with a finite candidate controller pool, an unknown dynamical system, and unknown cost functions. The candidate controllers can be unstabilizing policies. We only require at least one candidate controller to satisfy certain stability properties, but we do not know which one is stabilizing. We design an online algorithm that guarantees finite-gain stability throughout the duration of its execution. We also provide a sublinear policy regret guarantee compared with the optimal stabilizing candidate controller. Lastly, we numerically test our algorithm on quadrotor planar flights and compare it with a classical switching control algorithm, falsification-based switching, and a classical multi-armed bandit algorithm, Exp3 with batches.

math.OC↗

arXiv2022

Global Convergence of Localized Policy Iteration in Networked Multi-Agent Reinforcement Learning

We study a multi-agent reinforcement learning (MARL) problem where the agents interact over a given network. The goal of the agents is to cooperatively maximize the average of their entropy-regularized long-term rewards. To overcome the curse of dimensionality and to reduce communication, we propose a Localized Policy Iteration (LPI) algorithm that provably learns a near-globally-optimal policy using only local information. In particular, we show that, despite restricting each agent's attention to only its $κ$-hop neighborhood, the agents are able to learn a policy with an optimality gap that decays polynomially in $κ$. In addition, we show the finite-sample convergence of LPI to the global optimal policy, which explicitly captures the trade-off between optimality and computational complexity in choosing $κ$. Numerical simulations demonstrate the effectiveness of LPI.

cs.LG↗

arXiv2022

Bounded-Regret MPC via Perturbation Analysis: Prediction Error, Constraints, and Nonlinearity

We study Model Predictive Control (MPC) and propose a general analysis pipeline to bound its dynamic regret. The pipeline first requires deriving a perturbation bound for a finite-time optimal control problem. Then, the perturbation bound is used to bound the per-step error of MPC, which leads to a bound on the dynamic regret. Thus, our pipeline reduces the study of MPC to the well-studied problem of perturbation analysis, enabling the derivation of regret bounds of MPC under a variety of settings. To demonstrate the power of our pipeline, we use it to generalize existing regret bounds on MPC in linear time-varying (LTV) systems to incorporate prediction errors on costs, dynamics, and disturbances. Further, our pipeline leads to regret bounds on MPC in systems with nonlinear dynamics and constraints.

math.OC↗

arXiv2022

Fast Ion Gates Outside the Lamb-Dicke Regime by Robust Quantum Optimal Control

We present a robust quantum optimal control framework for implementing fast entangling gates on ion-trap quantum processors. The framework leverages tailored laser pulses to drive the multiple vibrational sidebands of the ions to create phonon-mediated entangling gates and, unlike the state of the art, requires neither weak-coupling Lamb-Dicke approximation nor perturbation treatment. With the application of gradient-based optimal control, it enables finding amplitude- and phase-modulated laser control protocols that work beyond the Lamb-Dicke regime, promising gate speed at the order of microseconds comparable to the characteristic trap frequencies. Also, robustness requirements on the temperature of the ions and initial optical phase can be conveniently included to pursue high-quality fast gates against experimental imperfections. Our approach represents a step in speeding up quantum gates to achieve larger quantum circuits for quantum computation and simulation, and thus can find applications in near-future experiments.

quant-ph↗

arXiv2022

Near-Optimal Distributed Linear-Quadratic Regulator for Networked Systems

This paper studies the trade-off between the degree of decentralization and the performance of a distributed controller in a linear-quadratic control setting. We study a system of interconnected agents over a graph and a distributed controller, called $κ$-distributed control, which lets the agents make control decisions based on the state information within distance $κ$ on the underlying graph. This controller can tune its degree of decentralization using the parameter $κ$ and thus allows a characterization of the relationship between decentralization and performance. We show that under mild assumptions, including stabilizability, detectability, and a subexponentially growing graph condition, the performance difference between $κ$-distributed control and centralized optimal control becomes exponentially small in $κ$. This result reveals that distributed control can achieve near-optimal performance with a moderate degree of decentralization, and thus it is an effective controller architecture for large-scale networked systems.

math.OC↗

arXiv2022

Non-Hermitian skin effect in a single trapped ion

Non-Hermitian skin effect (NHSE) describes the exponential localization of all eigenstates toward boundaries in non-Hermitian systems, and has attracted intense research interest of late. Here we theoretically propose a scheme in which the NHSE significantly impacts the external motion of a single trapped ion through complex spin-motion dynamics. On the one hand, we show the competition between the NHSE and the coherent Bloch dynamics. On the other hand, since the NHSE manifests as a non-reciprocal flow in occupied phonon modes, we demonstrate that such dynamics can have potential applications in cooling and sensing. Our proposal can be readily implemented using existing experimental techniques, and offers a scalable (in terms of the available ions and phonon modes) simulation platform for relevant non-Hermitian physics.

quant-ph↗

arXiv2022

Decentralized Online Convex Optimization in Networked Systems

We study the problem of networked online convex optimization, where each agent individually decides on an action at every time step and agents cooperatively seek to minimize the total global cost over a finite horizon. The global cost is made up of three types of local costs: convex node costs, temporal interaction costs, and spatial interaction costs. In deciding their individual action at each time, an agent has access to predictions of local cost functions for the next $k$ time steps in an $r$-hop neighborhood. Our work proposes a novel online algorithm, Localized Predictive Control (LPC), which generalizes predictive control to multi-agent systems. We show that LPC achieves a competitive ratio of $1 + \tilde{O}(ρ_T^k) + \tilde{O}(ρ_S^r)$ in an adversarial setting, where $ρ_T$ and $ρ_S$ are constants in $(0, 1)$ that increase with the relative strength of temporal and spatial interaction costs, respectively. This is the first competitive ratio bound on decentralized predictive control for networked online convex optimization. Further, we show that the dependence on $k$ and $r$ in our results is near optimal by lower bounding the competitive ratio of any decentralized online algorithm.

math.OC↗

arXiv2022

Equipping Black-Box Policies with Model-Based Advice for Stable Nonlinear Control

Machine-learned black-box policies are ubiquitous for nonlinear control problems. Meanwhile, crude model information is often available for these problems from, e.g., linear approximations of nonlinear dynamics. We study the problem of equipping a black-box control policy with model-based advice for nonlinear control on a single trajectory. We first show a general negative result that a naive convex combination of a black-box policy and a linear model-based policy can lead to instability, even if the two policies are both stabilizing. We then propose an adaptive $λ$-confident policy, with a coefficient $λ$ indicating the confidence in a black-box policy, and prove its stability. With bounded nonlinearity, in addition, we show that the adaptive $λ$-confident policy achieves a bounded competitive ratio when a black-box policy is near-optimal. Finally, we propose an online learning approach to implement the adaptive $λ$-confident policy and verify its efficacy in case studies about the CartPole problem and a real-world electric vehicle (EV) charging problem with data bias due to COVID-19.

cs.LG↗

arXiv2022

Preserving multi-level quantum coherence by dynamical decoupling

Quantum information processing with multi-level systems (qudits) provides additional features and applications than the two-level systems. However, qudits are more prone to dephasing and dynamical decoupling for qudits has never been experimentally demonstrated. Here, as a proof-of-principle demonstration, we experimentally apply dynamical decoupling to protect superpositions with three levels of a trapped $^9\rm{Be}^+$ ion from ambient noisy magnetic field, prolonging coherence by up to approximately an order of magnitude. Our demonstration, straightforwardly scalable to more levels, may open up a path toward long coherence quantum memory, metrology and information processing with qudits.

quant-ph↗

arXiv2022

Observation of spin-tensor induced topological phase transitions of triply degenerate points with a trapped ion

Triply degenerate points (TDPs), which correspond to new types of topological semimetals, can support novel quasiparticles possessing effective integer spins while preserving Fermi statistics. Here by mapping the momentum space to the parameter space of a three-level system in a trapped ion, we experimentally explore the transitions between different types of TDPs driven by spin-tensor--momentum couplings. We observe the phase transitions between TDPs with different topological charges by measuring the Berry flux on a loop surrounding the gap-closing lines, and the jump of the Berry flux gives the jump of the topological charge (up to a $2π$ factor) across the transitions. For the Berry flux measurement, we employ a new method by examining the geometric rotations of both spin vectors and tensors, which lead to a generalized solid angle equal to the Berry flux. The controllability of multi-level ion offers a versatile platform to study high-spin physics and our work paves the way to explore novel topological phenomena therein.

quant-ph↗

arXiv2022

Experimental Demonstration of Swift Analytical Universal Control over Nearby Transitions

Along with the scaling of dimensions in quantum systems, transitions between the system's energy levels would become close in frequency, which are conventionally resolved by weak and lengthy pulses. Here, we extend and experimentally demonstrate analytically based swift quantum control techniques on a four-level trapped ion system, where we perform individual or simultaneous control over two pairs of spectrally nearby transitions with tailored time-varied drive, achieving operational fidelities ranging from 99.2(3)\% to 99.6(3)\%. Remarkably, we achieve approximately an order of magnitude speed up comparing with the case of weak square pulse for a general control. Therefore, our demonstration may be beneficial to a broad range of quantum systems with crowded spectrum, for spectroscopy, quantum information processing and quantum simulation.

quant-ph↗

arXiv2021

Multi-Agent Reinforcement Learning in Stochastic Networked Systems

We study multi-agent reinforcement learning (MARL) in a stochastic network of agents. The objective is to find localized policies that maximize the (discounted) global reward. In general, scalability is a challenge in this setting because the size of the global state/action space can be exponential in the number of agents. Scalable algorithms are only known in cases where dependencies are static, fixed and local, e.g., between neighbors in a fixed, time-invariant underlying graph. In this work, we propose a Scalable Actor Critic framework that applies in settings where the dependencies can be non-local and stochastic, and provide a finite-time error bound that shows how the convergence rate depends on the speed of information spread in the network. Additionally, as a byproduct of our analysis, we obtain novel finite-time convergence results for a general stochastic approximation scheme and for temporal difference learning with state aggregation, which apply beyond the setting of MARL in networked systems.

cs.LG↗

arXiv2021

Online Optimization with Feedback Delay and Nonlinear Switching Cost

We study a variant of online optimization in which the learner receives $k$-round $\textit{delayed feedback}$ about hitting cost and there is a multi-step nonlinear switching cost, i.e., costs depend on multiple previous actions in a nonlinear manner. Our main result shows that a novel Iterative Regularized Online Balanced Descent (iROBD) algorithm has a constant, dimension-free competitive ratio that is $O(L^{2k})$, where $L$ is the Lipschitz constant of the switching cost. Additionally, we provide lower bounds that illustrate the Lipschitz condition is required and the dependencies on $k$ and $L$ are tight. Finally, via reductions, we show that this setting is closely related to online control problems with delay, nonlinear dynamics, and adversarial disturbances, where iROBD directly offers constant-competitive online policies.

cs.LG↗

arXiv2021

Perturbation-based Regret Analysis of Predictive Control in Linear Time Varying Systems

We study predictive control in a setting where the dynamics are time-varying and linear, and the costs are time-varying and well-conditioned. At each time step, the controller receives the exact predictions of costs, dynamics, and disturbances for the future $k$ time steps. We show that when the prediction window $k$ is sufficiently large, predictive control is input-to-state stable and achieves a dynamic regret of $O(λ^k T)$, where $λ< 1$ is a positive constant. This is the first dynamic regret bound on the predictive control of linear time-varying systems. Under more assumptions on the terminal costs, we also show that predictive control obtains the first competitive bound for the control of linear time-varying systems: $1 + O(λ^k)$. Our results are derived using a novel proof framework based on a perturbation bound that characterizes how a small change to the system parameters impacts the optimal trajectory.

math.OC↗

arXiv2021

Online Optimization with Memory and Competitive Control

This paper presents competitive algorithms for a novel class of online optimization problems with memory. We consider a setting where the learner seeks to minimize the sum of a hitting cost and a switching cost that depends on the previous $p$ decisions. This setting generalizes Smoothed Online Convex Optimization. The proposed approach, Optimistic Regularized Online Balanced Descent, achieves a constant, dimension-free competitive ratio. Further, we show a connection between online optimization with memory and online control with adversarial disturbances. This connection, in turn, leads to a new constant-competitive policy for a rich class of online control problems.

cs.LG↗

arXiv2020

High-fidelity single-shot readout of single electron spin in diamond with spin-to-charge conversion

High fidelity single-shot readout of qubits is a crucial component for fault-tolerant quantum computing and scalable quantum networks. In recent years, the nitrogen-vacancy (NV) center in diamond has risen as a leading platform for the above applications. The current single-shot readout of the NV electron spin relies on resonance fluorescence method at cryogenic temperature. However, the the spin-flip process interrupts the optical cycling transition, therefore, limits the readout fidelity. Here, we introduce a spin-to-charge conversion method assisted by near-infrared (NIR) light to suppress the spin-flip error. This method leverages high spin-selectivity of cryogenic resonance excitation and high flexibility of photonionization. We achieve an overall fidelity $>$ 95% for the single-shot readout of an NV center electron spin in the presence of high strain and fast spin-flip process. With further improvements, this technique has the potential to achieve spin readout fidelity exceeding the fault-tolerant threshold, and may also find applications on integrated optoelectronic devices.

quant-ph↗

arXiv2020

Verifying the upper bound on the speed of scrambling with the analogue Hawking radiation of trapped ions

A general bound on the Lyapunov exponent of a quantum system is given by $λ_L\leq2π\,T/\hbar$, where $T$ is the system temperature, as established by Maldacena, Shenker, and Stanford (MSS). This upper bound is saturated when the system under consideration is the exact holographic dual of a black hole. It has also been shown that an inverted harmonic oscillator (IHO) may exhibit the behavior of thermal energy emission, in close analogy to the Hawking radiation emitted by black holes. We demonstrate that the Lyapunov exponent of the IHO indeed saturates the MSS bound, with an effective temperature equal to the analogue black hole radiation temperature, and propose using a trapped ion as a physical implementation of the IHO. We derive the corresponding out-of-time-ordered correlation function (OTOC) diagnosing quantum chaos, and theoretically show, for an experimentally realizable setup, that the effective temperature of the trapped-ion-IHO matches the upper MSS bound for the speed of scrambling.

quant-ph↗