arXiv subjects

Haozhi Wang

Publications and source records attributed to Haozhi Wang.

31 records · Page 2Linked to original sources

arXiv2023

Meta Generative Flow Networks with Personalization for Task-Specific Adaptation

Multi-task reinforcement learning and meta-reinforcement learning have been developed to quickly adapt to new tasks, but they tend to focus on tasks with higher rewards and more frequent occurrences, leading to poor performance on tasks with sparse rewards. To address this issue, GFlowNets can be integrated into meta-learning algorithms (GFlowMeta) by leveraging the advantages of GFlowNets on tasks with sparse rewards. However, GFlowMeta suffers from performance degradation when encountering heterogeneous transitions from distinct tasks. To overcome this challenge, this paper proposes a personalized approach named pGFlowMeta, which combines task-specific personalized policies with a meta policy. Each personalized policy balances the loss on its personalized task and the difference from the meta policy, while the meta policy aims to minimize the average loss of all tasks. The theoretical analysis shows that the algorithm converges at a sublinear rate. Extensive experiments demonstrate that the proposed algorithm outperforms state-of-the-art reinforcement learning algorithms in discrete environments.

cs.LG↗

arXiv2023

Multi-agent Policy Reciprocity with Theoretical Guarantee

Modern multi-agent reinforcement learning (RL) algorithms hold great potential for solving a variety of real-world problems. However, they do not fully exploit cross-agent knowledge to reduce sample complexity and improve performance. Although transfer RL supports knowledge sharing, it is hyperparameter sensitive and complex. To solve this problem, we propose a novel multi-agent policy reciprocity (PR) framework, where each agent can fully exploit cross-agent policies even in mismatched states. We then define an adjacency space for mismatched states and design a plug-and-play module for value iteration, which enables agents to infer more precise returns. To improve the scalability of PR, deep PR is proposed for continuous control tasks. Moreover, theoretical analysis shows that agents can asymptotically reach consensus through individual perceived rewards and converge to an optimal value function, which implies the stability and effectiveness of PR, respectively. Experimental results on discrete and continuous environments demonstrate that PR outperforms various existing RL and transfer RL methods.

cs.AI↗

arXiv2023

CFlowNets: Continuous Control with Generative Flow Networks

Generative flow networks (GFlowNets), as an emerging technique, can be used as an alternative to reinforcement learning for exploratory control tasks. GFlowNet aims to generate distribution proportional to the rewards over terminating states, and to sample different candidates in an active learning fashion. GFlowNets need to form a DAG and compute the flow matching loss by traversing the inflows and outflows of each node in the trajectory. No experiments have yet concluded that GFlowNets can be used to handle continuous tasks. In this paper, we propose generative continuous flow networks (CFlowNets) that can be applied to continuous control tasks. First, we present the theoretical formulation of CFlowNets. Then, a training framework for CFlowNets is proposed, including the action selection process, the flow approximation algorithm, and the continuous flow matching loss function. Afterward, we theoretically prove the error bound of the flow approximation. The error decreases rapidly as the number of flow samples increases. Finally, experimental results on continuous control tasks demonstrate the performance advantages of CFlowNets compared to many reinforcement learning methods, especially regarding exploration ability.

cs.LG↗

arXiv2023

Characterization of Asymmetric Gap-Engineered Josephson Junctions and 3D Transmon Qubits

We have fabricated and characterized asymmetric gap-engineered junctions and transmon devices. To create Josephson junctions with asymmetric gaps, Ti was used to proximitize and lower the superconducting gap of the Al counter-electrode. DC IV measurements of these small, proximitized Josephson junctions show a reduced gap and larger excess current for voltage biases below the superconducting gap when compared to standard Al/AlOx/Al junctions. The energy relaxation time constant for an Al/AlOx/Al/Ti 3D transmon was T1 = 1 μs, over two orders of magnitude shorter than the measured T1 = 134 μs of a standard Al/AlOx/Al 3D transmon. Intentionally adding disorder between the Al and Ti layers reduces the proximity effect and subgap current while increasing the relaxation time to T1 = 32 μs.

quant-ph↗

arXiv2022

On the Convergence Theory of Meta Reinforcement Learning with Personalized Policies

Modern meta-reinforcement learning (Meta-RL) methods are mainly developed based on model-agnostic meta-learning, which performs policy gradient steps across tasks to maximize policy performance. However, the gradient conflict problem is still poorly understood in Meta-RL, which may lead to performance degradation when encountering distinct tasks. To tackle this challenge, this paper proposes a novel personalized Meta-RL (pMeta-RL) algorithm, which aggregates task-specific personalized policies to update a meta-policy used for all tasks, while maintaining personalized policies to maximize the average return of each task under the constraint of the meta-policy. We also provide the theoretical analysis under the tabular setting, which demonstrates the convergence of our pMeta-RL algorithm. Moreover, we extend the proposed pMeta-RL algorithm to a deep network version based on soft actor-critic, making it suitable for continuous control tasks. Experiment results show that the proposed algorithms outperform other previous Meta-RL algorithms on Gym and MuJoCo suites.

cs.AI↗

arXiv2022

A Low-speed Intruder Star in Hyades: A Temporary Residence

We hereby report a low-speed (about~21~km$\cdot$~s$^{-1}$ with respect to the Sun) intruder member in the Hyades cluster based on the data in the literature. The results show that the star is a non-native member star for the Hyades, with its radial velocity being smaller than the radial velocity of the Hyades cluster, even exceeding the standard deviation of the radial velocity of the cluster by a factor of 9. Furthermore, by analyzing and comparing the orbits of this star and its host, it may have intruded into its host in the past 2~Myr. If the star's current motion orbit remains unchanged, it may leave its host in the next 2~Myr. This implies that the intruder star may be temporarily residing in the cluster. This study presents the first observational evidence of a star intrusion into a cluster, which suggests that more evidence may be found.

astro-ph.SR↗

arXiv2022

Manifold Optimization Based Multi-user Rate Maximization Aided by Intelligent Reflecting Surface

In this work, two problems associated with a downlink multi-user system are considered with the aid of intelligent reflecting surface (IRS): weighted sum-rate maximization and weighted minimal-rate maximization. For the first problem, a novel DOuble Manifold ALternating Optimization (DOMALO) algorithm is proposed by exploiting the matrix manifold theory and introducing the beamforming matrix and reflection vector using complex sphere manifold and complex oblique manifold, respectively, which incorporate the inherent geometrical structure and the required constraint. A smooth double manifold alternating optimization (S-DOMALO) algorithm is then developed based on the Dinkelbach-type algorithm and smooth exponential penalty function for the second problem. Finally, possible cooperative beamforming gain between IRSs and the IRS phase shift with limited resolution is studied, providing a reference for practical implementation. Numerical results show that our proposed algorithms can significantly outperform the benchmark schemes.

cs.IT↗

arXiv2021

Multiple Intelligent Reflecting Surface aided Multi-user Weighted Sum-Rate Maximization using Manifold Optimization

Intelligent reflecting surface (IRS) are able to amend radio propagation condition tasks on account of its functional properties in phase shift optimizing. In fact, there exists geometry manifold in the base-station (BS) beamforming matrix and IRS reflection vector. Therefore, we propose a novel double manifold alternating optimization (DMAO) algorithm which makes use of differential geometry theory to improve optimization performance. First, we develop a multi-IRS multi-user system model to maximize the weighted sum-rate, which may lead to the non-convexity in our optimization procedure. Then in order to allocate an optimized coefficients to each BS antenna and IRS reflecting element, we present the beamforming matrix and reflection vector using complex sphere manifold and complex oblique manifold, respectively, which integrates the inner geometry structure and the constrains. By an innovative alternative iteration method, the system gradually converges an optimized stable state, which is associating with the maximized sum-rate. Furthermore, we quantize the IRS reflection coefficient considering the practical constrains. Experimental results demonstrated that our approach significantly outperforms the conventional methods in terms of weighted sum-rate.

eess.SP↗

arXiv2021

Perspective: Reproducible Coherence Characterization of Superconducting Quantum Devices

As the field of superconducting quantum computing approaches maturity, optimization of single-device performance is proving to be a promising avenue towards large-scale quantum computers. However, this optimization is possible only if performance metrics can be accurately compared among measurements, devices, and laboratories. Currently such comparisons are inaccurate or impossible due to understudied errors from a plethora of sources. In this Perspective, we outline the current state of error analysis for qubits and resonators in superconducting quantum circuits, and discuss what future investigations are required before superconducting quantum device optimization can be realized.

quant-ph↗

arXiv2021

Cryogenic single-port calibration for superconducting microwave resonator measurements

Superconducting circuit testing and materials loss characterization requires robust and reliable methods for the extraction of internal and coupling quality factors of microwave resonators. A common method, imposed by limitations on the device design or experimental configuration, is the single-port reflection geometry, i.e. reflection-mode. However, impedance mismatches in cryogenic systems must be accounted for through calibration of the measurement chain while it is at low temperatures. In this paper, we demonstrate a data-based, single-port calibration using commercial microwave standards and a vector network analyzer (VNA) with samples at millikelvin temperature in a dilution refrigerator, making this method useful for measurements of quantum phenomena. Finally, we cross reference our data-based, single-port calibration and reflection measurement with over-coupled 2D- and 3D-resonators against well established two-port techniques corroborating the validity of our method.

quant-ph↗

arXiv2020

Materials loss measurements using superconducting microwave resonators

The performance of superconducting circuits for quantum computing is limited by materials losses. In particular, coherence times are typically bounded by two-level system (TLS) losses at single photon powers and millikelvin temperatures. The identification of low loss fabrication techniques, materials, and thin film dielectrics is critical to achieving scalable architectures for superconducting quantum computing. Superconducting microwave resonators provide a convenient qubit proxy for assessing performance and studying TLS loss and other mechanisms relevant to superconducting circuits such as non-equilibrium quasiparticles and magnetic flux vortices. In this review article, we provide an overview of considerations for designing accurate resonator experiments to characterize loss, including applicable types of loss, cryogenic setup, device design, and methods for extracting material and interface losses, summarizing techniques that have been evolving for over two decades. Results from measurements of a wide variety of materials and processes are also summarized. Lastly, we present recommendations for the reporting of loss data from superconducting microwave resonators to facilitate materials comparisons across the field.

physics.app-ph↗

arXiv2020

Overlap junctions for superconducting quantum electronics and amplifiers

Due to their unique properties as lossless, nonlinear circuit elements, Josephson junctions lie at the heart of superconducting quantum information processing. Previously, we demonstrated a two-layer, submicrometer-scale overlap junction fabrication process suitable for qubits with long coherence times. Here, we extend the overlap junction fabrication process to micrometer-scale junctions. This allows us to fabricate other superconducting quantum devices. For example, we demonstrate an overlap-junction-based Josephson parametric amplifier that uses only 2 layers. This efficient fabrication process yields frequency-tunable devices with negligible insertion loss and a gain of ~ 30 dB. Compared to other processes, the overlap junction allows for fabrication with minimal infrastructure, high yield, and state-of-the-art device performance.

physics.app-ph↗

arXiv2015

Superconducting metamaterials and qubits

Superconducting thin-film metamaterial resonators can provide a dense microwave mode spectrum with potential applications in quantum information science. We report on the fabrication and low-temperature measurement of metamaterial transmission-line resonators patterned from Al thin films. We also describe multiple approaches for numerical simulations of the microwave properties of these structures, along with comparisons with the measured transmission spectra. The ability to predict the mode spectrum based on the chip layout provides a path towards future designs integrating metamaterial resonators with superconducting qubits.

cond-mat.supr-con↗