arXiv subjects

Hongyi Zhou

Publications and source records attributed to Hongyi Zhou.

At least 55 records · Page 3Linked to original sources

arXiv2024

Continuous-Variable Source-Independent Quantum Random Number Generator with a Single Phase-Insensitive Detector

Quantum random number generators (QRNGs) harness quantum mechanical unpredictability to produce true randomness, which is crucial for cryptography and secure communications. Among various QRNGs, source-independent QRNGs (SI-QRNGs) relax the trust on the quantum source, allowing for flexible use of advanced detectors to achieve high randomness generation rates. Continuous-variable (CV) SI-QRNGs, in particular, hold promise for practical deployment due to their simplicity and randomness generation rates comparable to trusted-device QRNGs. In this work, we propose a novel CV-SI-QRNG scheme with a single phase-insensitive detector, and provide security proof based on semi-definite programming (SDP). We introduce a dimension reduction technique, which rigorously reduces an infinite-dimensional SDP problem to a finite-dimensional one, enabling efficient computation while maintaining valid randomness lower bound. We further validate our method through simulations. These results demonstrate the feasibility of our framework, paving the way for practical and simple SI-QRNG implementations.

quant-ph↗

arXiv2024

MaIL: Improving Imitation Learning with Mamba

This work presents Mamba Imitation Learning (MaIL), a novel imitation learning (IL) architecture that provides an alternative to state-of-the-art (SoTA) Transformer-based policies. MaIL leverages Mamba, a state-space model designed to selectively focus on key features of the data. While Transformers are highly effective in data-rich environments due to their dense attention mechanisms, they can struggle with smaller datasets, often leading to overfitting or suboptimal representation learning. In contrast, Mamba's architecture enhances representation learning efficiency by focusing on key features and reducing model complexity. This approach mitigates overfitting and enhances generalization, even when working with limited data. Extensive evaluations on the LIBERO benchmark demonstrate that MaIL consistently outperforms Transformers on all LIBERO tasks with limited data and matches their performance when the full dataset is available. Additionally, MaIL's effectiveness is validated through its superior performance in three real robot experiments. Our code is available at https://github.com/ALRhub/MaIL.

cs.LG↗

arXiv2024

A Retrospective on the Robot Air Hockey Challenge: Benchmarking Robust, Reliable, and Safe Learning Techniques for Real-world Robotics

Machine learning methods have a groundbreaking impact in many application domains, but their application on real robotic platforms is still limited. Despite the many challenges associated with combining machine learning technology with robotics, robot learning remains one of the most promising directions for enhancing the capabilities of robots. When deploying learning-based approaches on real robots, extra effort is required to address the challenges posed by various real-world factors. To investigate the key factors influencing real-world deployment and to encourage original solutions from different researchers, we organized the Robot Air Hockey Challenge at the NeurIPS 2023 conference. We selected the air hockey task as a benchmark, encompassing low-level robotics problems and high-level tactics. Different from other machine learning-centric benchmarks, participants need to tackle practical challenges in robotics, such as the sim-to-real gap, low-level control issues, safety problems, real-time requirements, and the limited availability of real-world data. Furthermore, we focus on a dynamic environment, removing the typical assumption of quasi-static motions of other real-world benchmarks. The competition's results show that solutions combining learning-based approaches with prior knowledge outperform those relying solely on data when real-world deployment is challenging. Our ablation study reveals which real-world factors may be overlooked when building a learning-based solution. The successful real-world air hockey deployment of best-performing agents sets the foundation for future competitions and follow-up research directions.

cs.RO↗

arXiv2024

HIRO: Heuristics Informed Robot Online Path Planning Using Pre-computed Deterministic Roadmaps

With the goal of efficiently computing collision-free robot motion trajectories in dynamically changing environments, we present results of a novel method for Heuristics Informed Robot Online Path Planning (HIRO). Dividing robot environments into static and dynamic elements, we use the static part for initializing a deterministic roadmap, which provides a lower bound of the final path cost as informed heuristics for fast path-finding. These heuristics guide a search tree to explore the roadmap during runtime. The search tree examines the edges using a fuzzy collision checking concerning the dynamic environment. Finally, the heuristics tree exploits knowledge fed back from the fuzzy collision checking module and updates the lower bound for the path cost. As we demonstrate in real-world experiments, the closed-loop formed by these three components significantly accelerates the planning procedure. An additional backtracking step ensures the feasibility of the resulting paths. Experiments in simulation and the real world show that HIRO can find collision-free paths considerably faster than baseline methods with and without prior knowledge of the environment.

cs.RO↗

arXiv2024

Variational Distillation of Diffusion Policies into Mixture of Experts

This work introduces Variational Diffusion Distillation (VDD), a novel method that distills denoising diffusion policies into Mixtures of Experts (MoE) through variational inference. Diffusion Models are the current state-of-the-art in generative modeling due to their exceptional ability to accurately learn and represent complex, multi-modal distributions. This ability allows Diffusion Models to replicate the inherent diversity in human behavior, making them the preferred models in behavior learning such as Learning from Human Demonstrations (LfD). However, diffusion models come with some drawbacks, including the intractability of likelihoods and long inference times due to their iterative sampling process. The inference times, in particular, pose a significant challenge to real-time applications such as robot control. In contrast, MoEs effectively address the aforementioned issues while retaining the ability to represent complex distributions but are notoriously difficult to train. VDD is the first method that distills pre-trained diffusion models into MoE models, and hence, combines the expressiveness of Diffusion Models with the benefits of Mixture Models. Specifically, VDD leverages a decompositional upper bound of the variational objective that allows the training of each expert separately, resulting in a robust optimization scheme for MoEs. VDD demonstrates across nine complex behavior learning tasks, that it is able to: i) accurately distill complex distributions learned by the diffusion model, ii) outperform existing state-of-the-art distillation methods, and iii) surpass conventional methods for training MoE.

cs.LG↗

arXiv2024

Error mitigated shadow estimation based on virtual distillation

Shadow estimation is a method for deducing numerous properties of an unknown quantum state through a limited set of measurements, which suffers from noises in quantum devices. In this paper, we introduce an error-mitigated shadow estimation approach based on virtual distillation, tailored for applications in near-term quantum devices. Our methodology leverages the qubit reset technique, thereby reducing the associated qubit overhead. Crucially, our approach ensures that the required qubit resources remain independent of the desired accuracy and avoid an exponential measurement overhead, marking a substantial advancement in practical applications. Furthermore, our technique accommodates a mixed Clifford and Pauli-type shadow, which can result in a reduction in the number of required measurements across various scenarios. We also study the trade-off between circuit depth and measurement overhead quantitatively. Through numerical simulations, we substantiate the efficacy of our error mitigation method, establishing its utility in enhancing the robustness of shadow estimations on near-term quantum devices.

quant-ph↗

arXiv2024

Measurement-device-independent quantum random number generation over 23 Mbps with imperfect single-photon sources

Quantum randomness relies heavily on the accurate characterization of the generator implementation, where the device imperfection or inaccurate characterization can lead to incorrect entropy estimation and practical bias, significantly affecting the reliability of the generated randomness. Measurement-device-independent (MDI) quantum random number generation (QRNG) endeavors to produce certified randomness, utilizing uncharacterized and untrusted measurement devices that are vulnerable to numerous attack schemes targeting measurement loopholes. However, existing implementations have shown insufficient performance thus far. Here, we propose a high-speed MDI-QRNG scheme based on a robust measurement tomography approach against the imperfection of single-photon sources. Compared with the conventional approach, the decoy-state method is introduced to obtain more accurate tomography results and a tighter lower bound of randomness. Finally, by using a high-speed time-bin encoding system, we experimentally demonstrated the scheme and obtained a reliable min-entropy lower bound of $7.37 \times 10^{-2}$ bits per pulse, corresponding to a generation rate over 23 Mbps, which substantially outperforms the existing realizations and makes a record in discrete-variable semi-device-independent QRNGs.

quant-ph↗

arXiv2024

Non-parametric estimates for graphon mean-field particle systems

We consider the graphon mean-field system introduced in the work of Bayraktar, Chakraborty, and Wu. It is the large-population limit of a heterogeneously interacting diffusive particle system, where the interaction is of mean-field type with weights characterized by an underlying graphon function. Through observation of continuous-time trajectories within the particle system, we construct plug-in estimators of the particle density, the drift coefficient, and thus the graphon interaction weights of the mean-field system. Our estimators for the density and drift are direct results of kernel interpolation on the empirical data, and a deconvolution method leads to an estimator of the underlying graphon function. We show that, as the number of particles increases, the graphon estimator converges to the true graphon function pointwisely, and as a consequence, in the cut metric. Besides, we conduct a minimax analysis within a particular class of particle systems to justify the pointwise optimality of the density and drift estimators.

math.ST↗

arXiv2024

Real-Time Seedless Post-Processing for Quantum Random Number Generators

Quantum-proof randomness extraction is essential for handling quantum side information possessed by a quantum adversary, which is widely applied in various quantum cryptography tasks. In this study, we introduce a real-time two-source quantum randomness extractor against quantum side information. Our extractor is tailored for forward block sources, a novel category of min-entropy sources introduced in this work. These sources retain the flexibility to accommodate a broad range of quantum random number generators. Our online algorithms demonstrate the extraction of a constant fraction of min-entropy from two infinitely long independent forward block sources. Moreover, our extractor is inherently block-wise parallelizable, presenting a practical and efficient solution for the timely extraction of high-quality randomness. Applying our extractors to the raw data of one of the most commonly used quantum random number generators, we achieve a simulated extraction speed as high as 64 Gbps.

quant-ph↗

arXiv2024

Open the Black Box: Step-based Policy Updates for Temporally-Correlated Episodic Reinforcement Learning

Current advancements in reinforcement learning (RL) have predominantly focused on learning step-based policies that generate actions for each perceived state. While these methods efficiently leverage step information from environmental interaction, they often ignore the temporal correlation between actions, resulting in inefficient exploration and unsmooth trajectories that are challenging to implement on real hardware. Episodic RL (ERL) seeks to overcome these challenges by exploring in parameters space that capture the correlation of actions. However, these approaches typically compromise data efficiency, as they treat trajectories as opaque \emph{black boxes}. In this work, we introduce a novel ERL algorithm, Temporally-Correlated Episodic RL (TCE), which effectively utilizes step information in episodic policy updates, opening the 'black box' in existing ERL methods while retaining the smooth and consistent exploration in parameter space. TCE synergistically combines the advantages of step-based and episodic RL, achieving comparable performance to recent ERL methods while maintaining data efficiency akin to state-of-the-art (SoTA) step-based RL.

cs.LG↗

arXiv2024

Improved Real-time Post-Processing for Quantum Random Number Generators

Randomness extraction is a key problem in cryptography and theoretical computer science. With the recent rapid development of quantum cryptography, quantum-proof randomness extraction has also been widely studied, addressing the security issues in the presence of a quantum adversary. In contrast with conventional quantum-proof randomness extractors characterizing the input raw data as min-entropy sources, we find that the input raw data generated by a large class of trusted-device quantum random number generators can be characterized as the so-called reverse block source. This fact enables us to design improved extractors. Specifically, we propose two novel quantum-proof randomness extractors for reverse block sources that realize real-time block-wise extraction. In comparison with the general min-entropy randomness extractors, our designs achieve a significantly higher extraction speed and a longer output data length with the same seed length. In addition, they enjoy the property of online algorithms, which process the raw data on the fly without waiting for the entire input raw data to be available. These features make our design an adequate choice for the real-time post-processing of practical quantum random number generators. Applying our extractors to the raw data generated by a widely used quantum random number generator, we achieve a simulated extraction speed as high as $300$ Gbps.

quant-ph↗

arXiv2023

A hybrid algorithm for quadratically constrained quadratic optimization problems

Quadratically Constrained Quadratic Programs (QCQPs) are an important class of optimization problems with diverse real-world applications. In this work, we propose a variational quantum algorithm for general QCQPs. By encoding the variables on the amplitude of a quantum state, the requirement of the qubit number scales logarithmically with the dimension of the variables, which makes our algorithm suitable for current quantum devices. Using the primal-dual interior-point method in classical optimization, we can deal with general quadratic constraints. Our numerical experiments on typical QCQP problems, including Max-Cut and optimal power flow problems, demonstrate a better performance of our hybrid algorithm over the classical counterparts.

quant-ph↗

arXiv2023

Hybrid algorithm simulating non-equilibrium steady states of an open quantum system

Non-equilibrium steady states are a focal point of research in the study of open quantum systems. Previous variational algorithms for searching these steady states have suffered from resource-intensive implementations due to vectorization or purification of the system density matrix, requiring large qubit resources and long-range coupling. In this work, we present a novel variational quantum algorithm that efficiently searches for non-equilibrium steady states by simulating the operator-sum form of the Lindblad equation. By introducing the technique of random measurement, we are able to estimate the nonlinear cost function while reducing the required qubit resources by half compared to previous methods. Additionally, we prove the existence of the parameter shift rule in our variational algorithm, enabling efficient updates of circuit parameters using gradient-based classical algorithms. To demonstrate the performance of our algorithm, we conduct simulations for dissipative quantum transverse Ising and Heisenberg models, achieving highly accurate results. Our approach offers a promising solution for effectively addressing non-equilibrium steady state problems while overcoming computational limitations and implementation challenges.

quant-ph↗

arXiv2023

Numerical Security Analysis of Three-State Quantum Key Distribution Protocol with Realistic Devices

Quantum key distribution (QKD) is a secure communication method that utilizes the principles of quantum mechanics to establish secret keys. The central task in the study of QKD is to prove security in the presence of an eavesdropper with unlimited computational power. In this work, we successfully solve a long-standing open question of the security analysis for the three-state QKD protocol with realistic devices, i,e, the weak coherent state source. We prove the existence of the squashing model for the measurement settings in the three-state protocol. This enables the reduction of measurement dimensionality, allowing for key rate computations using the numerical approach. We conduct numerical simulations to evaluate the key rate performance. The simulation results show that we achieve a communication distance of up to 200 km.

quant-ph↗

arXiv2023

MP3: Movement Primitive-Based (Re-)Planning Policy

We introduce a novel deep reinforcement learning (RL) approach called Movement Primitive-based Planning Policy (MP3). By integrating movement primitives (MPs) into the deep RL framework, MP3 enables the generation of smooth trajectories throughout the whole learning process while effectively learning from sparse and non-Markovian rewards. Additionally, MP3 maintains the capability to adapt to changes in the environment during execution. Although many early successes in robot RL have been achieved by combining RL with MPs, these approaches are often limited to learning single stroke-based motions, lacking the ability to adapt to task variations or adjust motions during execution. Building upon our previous work, which introduced an episode-based RL method for the non-linear adaptation of MP parameters to different task variations, this paper extends the approach to incorporating replanning strategies. This allows adaptation of the MP parameters throughout motion execution, addressing the lack of online motion adaptation in stochastic domains requiring feedback. We compared our approach against state-of-the-art deep RL and RL with MPs methods. The results demonstrated improved performance in sophisticated, sparse reward settings and in domains requiring replanning.

cs.LG↗

arXiv2023

Quantifying the dissipation enhancement of cellular flows

We study the dissipation enhancement by cellular flows. Previous work by Iyer, Xu, and Zlatoš produces a family of cellular flows that can enhance dissipation by an arbitrarily large amount. We improve this result by providing quantitative bounds on the dissipation enhancement in terms of the flow amplitude, cell size and diffusivity. Explicitly we show that the mixing time is bounded by the exit time from one cell when the flow amplitude is large enough, and by the reciprocal of the effective diffusivity when the flow amplitude is small. This agrees with the optimal heuristics. We also prove a general result relating the dissipation time of incompressible flows to the mixing time. The main idea behind the proof is to study the dynamics probabilistically and construct a successful coupling.

math.AP↗

arXiv2023

Quantum Network: Security Assessment and Key Management

As an extension of quantum key distribution, secure communication among multiple users is an essential task in a quantum network. When the quantum network structure becomes complicated with a large number of users, it is important to investigate network issues, including security, key management, latency, reliability, scalability, and cost. In this work, we utilize the classical and graph theories to address two critical issues in a quantum network: security and key management. First, we design a communication scheme with the highest security level that trusts a minimum number of intermediate nodes. Second, when the quantum key is a limited resource, we design key management and data scheduling schemes to optimize the utility of data transmission. Our results can be directly applied to the current metropolitan and free-space quantum network implementations and potentially be a standard approach for future quantum network designs.

quant-ph↗

arXiv2022

Deep Black-Box Reinforcement Learning with Movement Primitives

\Episode-based reinforcement learning (ERL) algorithms treat reinforcement learning (RL) as a black-box optimization problem where we learn to select a parameter vector of a controller, often represented as a movement primitive, for a given task descriptor called a context. ERL offers several distinct benefits in comparison to step-based RL. It generates smooth control trajectories, can handle non-Markovian reward definitions, and the resulting exploration in parameter space is well suited for solving sparse reward settings. Yet, the high dimensionality of the movement primitive parameters has so far hampered the effective use of deep RL methods. In this paper, we present a new algorithm for deep ERL. It is based on differentiable trust region layers, a successful on-policy deep RL algorithm. These layers allow us to specify trust regions for the policy update that are solved exactly for each state using convex optimization, which enables policies learning with the high precision required for the ERL. We compare our ERL algorithm to state-of-the-art step-based algorithms in many complex simulated robotic control tasks. In doing so, we investigate different reward formulations - dense, sparse, and non-Markovian. While step-based algorithms perform well only on dense rewards, ERL performs favorably on sparse and non-Markovian rewards. Moreover, our results show that the sparse and the non-Markovian rewards are also often better suited to define the desired behavior, allowing us to obtain considerably higher quality policies compared to step-based RL.

cs.LG↗