arXiv subjects

Huiliang Zhang

Publications and source records attributed to Huiliang Zhang.

17 recordsLinked to original sources

arXiv2025

Meta-Reinforcement Learning for Building Energy Management System

The building sector is one of the largest contributors to global energy consumption. Improving its energy efficiency is essential for reducing operational costs and greenhouse gas emissions. Energy management systems (EMS) play a key role in monitoring and controlling building appliances efficiently and reliably. With the increasing integration of renewable energy, intelligent EMS solutions have received growing attention. Reinforcement learning (RL) has recently been explored for this purpose and shows strong potential. However, most RL-based EMS methods require a large number of training steps to learn effective control policies, especially when adapting to unseen buildings, which limits their practical deployment. This paper introduces MetaEMS, a meta-reinforcement learning framework for EMS. MetaEMS improves learning efficiency by transferring knowledge from previously solved tasks to new ones through group-level and building-level adaptation, enabling fast adaptation and effective control across diverse building environments. Experimental results demonstrate that MetaEMS adapts more rapidly to unseen buildings and consistently outperforms baseline methods across various scenarios.

cs.AI↗

arXiv2025

STEMS: Spatial-Temporal Enhanced Safe Multi-Agent Coordination for Building Energy Management

Building energy management is essential for achieving carbon reduction goals, improving occupant comfort, and reducing energy costs. Coordinated building energy management faces critical challenges in exploiting spatial-temporal dependencies while ensuring operational safety across multi-building systems. Current multi-building energy systems face three key challenges: insufficient spatial-temporal information exploitation, lack of rigorous safety guarantees, and system complexity. This paper proposes Spatial-Temporal Enhanced Safe Multi-Agent Coordination (STEMS), a novel safety-constrained multi-agent reinforcement learning framework for coordinated building energy management. STEMS integrates two core components: (1) a spatial-temporal graph representation learning framework using a GCN-Transformer fusion architecture to capture inter-building relationships and temporal patterns, and (2) a safety-constrained multi-agent RL algorithm incorporating Control Barrier Functions to provide mathematical safety guarantees. Extensive experiments on real-world building datasets demonstrate STEMS's superior performance over existing methods, showing that STEMS achieves 21% cost reduction, 18% emission reduction, and dramatically reduces safety violations from 35.1% to 5.6% while maintaining optimal comfort with only 0.13 discomfort proportion. The framework also demonstrates strong robustness during extreme weather conditions and maintains effectiveness across different building types.

cs.AI↗

arXiv2025

Leveraging Multivariate Long-Term History Representation for Time Series Forecasting

Multivariate Time Series (MTS) forecasting has a wide range of applications in both industry and academia. Recent advances in Spatial-Temporal Graph Neural Network (STGNN) have achieved great progress in modelling spatial-temporal correlations. Limited by computational complexity, most STGNNs for MTS forecasting focus primarily on short-term and local spatial-temporal dependencies. Although some recent methods attempt to incorporate univariate history into modeling, they still overlook crucial long-term spatial-temporal similarities and correlations across MTS, which are essential for accurate forecasting. To fill this gap, we propose a framework called the Long-term Multivariate History Representation (LMHR) Enhanced STGNN for MTS forecasting. Specifically, a Long-term History Encoder (LHEncoder) is adopted to effectively encode the long-term history into segment-level contextual representations and reduce point-level noise. A non-parametric Hierarchical Representation Retriever (HRetriever) is designed to include the spatial information in the long-term spatial-temporal dependency modelling and pick out the most valuable representations with no additional training. A Transformer-based Aggregator (TAggregator) selectively fuses the sparsely retrieved contextual representations based on the ranking positional embedding efficiently. Experimental results demonstrate that LMHR outperforms typical STGNNs by 10.72% on the average prediction horizons and state-of-the-art methods by 4.12% on several real-world datasets. Additionally, it consistently improves prediction accuracy by 9.8% on the top 10% of rapidly changing patterns across the datasets.

cs.LG↗

arXiv2025

Nearest Neighbor Multivariate Time Series Forecasting

Multivariate time series (MTS) forecasting has a wide range of applications in both industry and academia. Recently, spatial-temporal graph neural networks (STGNNs) have gained popularity as MTS forecasting methods. However, current STGNNs can only use the finite length of MTS input data due to the computational complexity. Moreover, they lack the ability to identify similar patterns throughout the entire dataset and struggle with data that exhibit sparsely and discontinuously distributed correlations among variables over an extensive historical period, resulting in only marginal improvements. In this article, we introduce a simple yet effective k-nearest neighbor MTS forecasting ( kNN-MTS) framework, which forecasts with a nearest neighbor retrieval mechanism over a large datastore of cached series, using representations from the MTS model for similarity search. This approach requires no additional training and scales to give the MTS model direct access to the whole dataset at test time, resulting in a highly expressive model that consistently improves performance, and has the ability to extract sparse distributed but similar patterns spanning over multivariables from the entire dataset. Furthermore, a hybrid spatial-temporal encoder (HSTEncoder) is designed for kNN-MTS which can capture both long-term temporal and short-term spatial-temporal dependencies and is shown to provide accurate representation for kNN-MTSfor better forecasting. Experimental results on several real-world datasets show a significant improvement in the forecasting performance of kNN-MTS. The quantitative analysis also illustrates the interpretability and efficiency of kNN-MTS, showing better application prospects and opening up a new path for efficiently using the large dataset in MTS models.

cs.LG↗

arXiv2023

Adaptive Aggregation for Safety-Critical Control

Safety has been recognized as the central obstacle to preventing the use of reinforcement learning (RL) for real-world applications. Different methods have been developed to deal with safety concerns in RL. However, learning reliable RL-based solutions usually require a large number of interactions with the environment. Likewise, how to improve the learning efficiency, specifically, how to utilize transfer learning for safe reinforcement learning, has not been well studied. In this work, we propose an adaptive aggregation framework for safety-critical control. Our method comprises two key techniques: 1) we learn to transfer the safety knowledge by aggregating the multiple source tasks and a target task through the attention network; 2) we separate the goal of improving task performance and reducing constraint violations by utilizing a safeguard. Experiment results demonstrate that our algorithm can achieve fewer safety violations while showing better data efficiency compared with several baselines.

cs.LG↗

arXiv2022

Controllable tunability of a Chern number within the electronic-nuclear spin system in diamond

Chern numbers are gaining traction as they characterize topological phases in various physical systems. However, the resilience of the system topology to external perturbations makes it challenging to experimentally investigate transitions between different phases. In this study, we demonstrate the transitions of Chern number from 0 to 3, synthesized in an electronic-nuclear spin system associated with the nitrogen-vacancy (NV) centre in diamond. The Chern number is characterized by the number of degeneracies enclosed in a control Hamiltonian parameter sphere. The topological transitions between different phases are depicted by varying the radius and offset of the sphere. We show that the measured topological phase diagram is not only consistent with the numerical calculations but can also be mapped onto an interacting three-qubit system. The NV system may also allow access to even higher Chern numbers, which can be applied to exploring exotic topology or topological quantum information.

quant-ph↗

arXiv2022

Anticipating the Unseen Discrepancy for Vision and Language Navigation

Vision-Language Navigation requires the agent to follow natural language instructions to reach a specific target. The large discrepancy between seen and unseen environments makes it challenging for the agent to generalize well. Previous studies propose data augmentation methods to mitigate the data bias explicitly or implicitly and provide improvements in generalization. However, they try to memorize augmented trajectories and ignore the distribution shifts under unseen environments at test time. In this paper, we propose an Unseen Discrepancy Anticipating Vision and Language Navigation (DAVIS) that learns to generalize to unseen environments via encouraging test-time visual consistency. Specifically, we devise: 1) a semi-supervised framework DAVIS that leverages visual consistency signals across similar semantic observations. 2) a two-stage learning procedure that encourages adaptation to test-time distribution. The framework enhances the basic mixture of imitation and reinforcement learning with Momentum Contrast to encourage stable decision-making on similar observations under a joint training stage and a test-time adaptation stage. Extensive experiments show that DAVIS achieves model-agnostic improvement over previous state-of-the-art VLN baselines on R2R and RxR benchmarks. Our source code and data are in supplemental materials.

cs.CV↗

arXiv2021

Data-driven Model Predictive and Reinforcement Learning Based Control for Building Energy Management: a Survey

Building energy management is one of the core problems in modern power grids to reduce energy consumption while ensuring occupants' comfort. However, the building energy management system (BEMS) is now facing more challenges and uncertainties with the increasing penetration of renewables and complicated interactions between humans and buildings. Classical model predictive control (MPC) has shown its capacity to reduce building energy consumption, but it suffers from labor-intensive modelling and complex on-line control optimization. Recently, with the growing accessibility to the building control and automation data, data-driven solutions have attracted more research interest. This paper presents a compact review of the recent advances in data-driven MPC and reinforcement learning based control methods for BEMS. The main challenges in these approaches and insights on the selection of a control method are discussed.

eess.SY↗

arXiv2020

Imaging viscous flow of the Dirac fluid in graphene

The electron-hole plasma in charge-neutral graphene is predicted to realize a quantum critical system whose transport features a universal hydrodynamic description, even at room temperature. This quantum critical "Dirac fluid" is expected to have a shear viscosity close to a minimum bound, with an inter-particle scattering rate saturating at the Planckian time $\hbar/(k_B T)$. While electrical transport measurements at finite carrier density are consistent with hydrodynamic electron flow in graphene, a "smoking gun" of viscous behavior remains elusive. In this work, we directly image viscous Dirac fluid flow in graphene at room temperature via measurement of the associated stray magnetic field. Nanoscale magnetic imaging is performed using quantum spin magnetometers realized with nitrogen vacancy (NV) centers in diamond. Scanning single-spin and wide-field magnetometry reveals a parabolic Poiseuille profile for electron flow in a graphene channel near the charge neutrality point, establishing the viscous transport of the Dirac fluid. This measurement is in contrast to the conventional uniform flow profile imaged in an Ohmic conductor. Via combined imaging-transport measurements, we obtain viscosity and scattering rates, and observe that these quantities are comparable to the universal values expected at quantum criticality. This finding establishes a nearly-ideal electron fluid in neutral graphene at room temperature. Our results pave the way to study hydrodynamic transport in quantum critical fluids relevant to strongly-correlated electrons in high-$T_c$ superconductors. This work also highlights the capability of quantum spin magnetometers to probe correlated-electronic phenomena at the nanoscale.

cond-mat.mes-hall↗

arXiv2018

Geometric phase magnetometry using a solid-state spin

Magnetometry is a powerful technique for the non-invasive study of biological and physical systems. A key challenge lies in the simultaneous optimization of magnetic field sensitivity and maximum field range. In interferometry-based magnetometry, a quantum two-level system acquires a dynamic phase in response to an applied magnetic field. However, due to the 2π periodicity of the phase, increasing the coherent interrogation time to improve sensitivity results in reduced field range. Here we introduce a route towards both large magnetic field range and high sensitivity via measurements of the geometric phase acquired by a quantum two-level system. We experimentally demonstrate geometric-phase magnetometry using the optically addressable electronic spin associated with the nitrogen vacancy (NV) color center in diamond. Our approach enables unwrapping of the 2π phase ambiguity, decoupling of magnetic field range from sensitivity, and enhancement of the field range by about 400 times. We also find additional improvement in sensitivity in the nonadiabatic regime, and study how geometric-phase decoherence depends on adiabaticity. Our results show that the geometric phase can be a versatile tool for quantum sensing applications.

quant-ph↗

arXiv2016

Control and Local Measurement of the Spin Chemical Potential in a Magnetic Insulator

The spin chemical potential characterizes the tendency of spins to diffuse. Probing the spin chemical potential could provide insight into materials such as magnetic insulators and spin liquids and aid optimization of spintronic devices. Here, we introduce single-spin magnetometry as a generic platform for non-perturbative, nanoscale characterization of spin chemical potentials. We use this platform to investigate magnons in a magnetic insulator, surprisingly finding that the magnon chemical potential can be efficiently controlled by driving the system's ferromagnetic resonance. We introduce a symmetry-based two-fluid theory describing the underlying magnon processes, realize the first experimental determination of the local thermomagnonic torque, and illustrate the detection sensitivity using electrically controlled spin injection. Our results open the way for nanoscale control and imaging of spin transport in mesoscopic spin systems.

cond-mat.mes-hall↗

arXiv2016

Nanodiamond-enhanced MRI

Nanodiamonds are of interest as nontoxic substrates for targeted drug delivery and as highly biostable fluorescent markers for cellular tracking. Beyond optical techniques however, options for noninvasive imaging of nanodiamonds in vivo are severely limited. Here, we demonstrate that the Overhauser effect, a proton-electron double resonance technique developed to detect free radicals in vivo, can enable high contrast magnetic resonance imaging (MRI) of nanodiamonds in water at room temperature and ultra-low magnetic field. The technique transfers spin polarization from paramagnetic impurities at nanodiamond surfaces to $^1$H spins in the surrounding water solution, creating MRI contrast on-demand. We further examine the conditions required for maximum enhancement as well as the ultimate sensitivity of the technique. The ability to perform continuous hyperpolarization via the Overhauser mechanism, in combination with excellent in vivo stability, raises the possibility of performing noninvasive tracking of nanodiamonds over indefinitely long periods of time.

cond-mat.mes-hall↗

arXiv2016

Selective addressing of solid-state spins at the nanoscale via magnetic resonance frequency encoding

The nitrogen-vacancy (NV) centre in diamond is a leading platform for nanoscale sensing and imaging, as well as quantum information processing in the solid state. To date, individual control of two NV electronic spins at the nanoscale has been demonstrated. However, a key challenge is to scale up such control to arrays of NV spins. Here we apply nanoscale magnetic resonance frequency encoding to realize site-selective addressing and coherent control of a four-site array of NV spins. Sites in the array are separated by 100 nm, with each site containing multiple NVs separated by ~15 nm. Microcoils fabricated on the diamond chip provide electrically tuneable magnetic-field gradients ~0.1 G/nm. Tailored application of gradient fields and resonant microwaves allow site-selective NV spin manipulation and sensing applications, including Rabi oscillations, imaging, and nuclear magnetic resonance (NMR) spectroscopy with nanoscale resolution. Microcoil-based magnetic resonance of solid-state spins provides a practical platform for quantum-assisted sensing, quantum information processing, and the study of nanoscale spin networks.

quant-ph↗

arXiv2016

Efficiency of Cathodoluminescence Emission by Nitrogen-Vacancy Color Centers in Nanodiamond

Correlated electron microscopy and cathodoluminescence (CL) imaging using functionalized nanoparticles is a promising nanoscale probe of biological structure and function. Nanodiamonds (NDs) that contain CL-emitting color centers are particularly well suited for such applications. The intensity of CL emission from NDs is determined by a combination of factors, including: particle size; density of color centers; efficiency of energy deposition by electrons passing through the particle; and conversion efficiency from deposited energy to CL emission. We report experiments and numerical simulations that investigate the relative importance of each of these factors in determining CL emission intensity from NDs containing nitrogen-vacancy (NV) color centers. In particular, we find that CL can be detected from NV-doped NDs with dimensions as small as ~ 40 nm, although CL emission decreases significantly for smaller NDs.

cond-mat.mtrl-sci↗

arXiv2014

Nanoscale NMR Spectroscopy and Imaging of Multiple Nuclear Species

Nuclear magnetic resonance (NMR) and magnetic resonance imaging (MRI) are well-established techniques that provide valuable information in a diverse set of disciplines but are currently limited to macroscopic sample volumes. Here we demonstrate nanoscale NMR spectroscopy and imaging under ambient conditions of samples containing multiple nuclear species, using nitrogen-vacancy (NV) colour centres in diamond as sensors. With single, shallow NV centres in a diamond chip and samples placed on the diamond surface, we perform NMR spectroscopy and one-dimensional MRI on few-nanometre-sized samples containing $^1$H and $^{19}$F nuclei. Alternatively, we employ a high-density NV layer near the surface of a diamond chip to demonstrate wide-field optical NMR spectroscopy of nanoscale samples containing $^1$H, $^{19}$F, and $^{31}$P nuclei, as well as multi-species two-dimensional optical MRI with sub-micron resolution. For all diamond samples exposed to air, we identify a ubiquitous $^1$H NMR signal, consistent with a $\sim 1$ nm layer of adsorbed hydrocarbons or water on the diamond surface and below any sample placed on the diamond. This work lays the foundation for nanoscale NMR and MRI applications such as studies of single proteins and functional biological imaging with subcellular resolution, as well as characterization of thin films with sub-nanometre resolution.

quant-ph↗

arXiv2013

Silicon-Vacancy Color Centers in Nanodiamonds: Cathodoluminescence Imaging Marker in the Near Infrared

We demonstrate that nanodiamonds fabricated to incorporate silicon-vacancy (Si-V) color centers provide bright, spectrally narrow, and stable cathodoluminescence (CL) in the near-infrared. Si-V color centers containing nanodiamonds are promising as non-bleaching optical markers for correlated CL and secondary electron microscopy, including applications to nanoscale bioimaging.

cond-mat.mtrl-sci↗

arXiv2012

Multi-color correlative light and electron microscopy using nanoparticle cathodoluminescence

Correlative light and electron microscopy promises to combine molecular specificity with nanoscale imaging resolution. However, there are substantial technical challenges including reliable co-registration of optical and electron images, and rapid optical signal degradation under electron beam irradiation. Here, we introduce a new approach to solve these problems: multi-color imaging of stable optical cathodoluminescence emitted in a scanning electron microscope by nanoparticles with controllable surface chemistry. We demonstrate well-correlated cathodoluminescence and secondary electron images using three species of semiconductor nanoparticles that contain defects providing stable, spectrally-distinguishable cathodoluminescence. We also demonstrate reliable surface functionalization of the particles. The results pave the way for the use of such nanoparticles for targeted labeling of surfaces to provide nanoscale mapping of molecular composition, indicated by cathodoluminescence color, simultaneously acquired with structural electron images in a single instrument.

physics.ins-det↗