arXiv subjects

Na Li

Publications and source records attributed to Na Li.

At least 145 records · Page 8Linked to original sources

arXiv2024

MPC-Inspired Reinforcement Learning for Verifiable Model-Free Control

In this paper, we introduce a new class of parameterized controllers, drawing inspiration from Model Predictive Control (MPC). The controller resembles a Quadratic Programming (QP) solver of a linear MPC problem, with the parameters of the controller being trained via Deep Reinforcement Learning (DRL) rather than derived from system models. This approach addresses the limitations of common controllers with Multi-Layer Perceptron (MLP) or other general neural network architecture used in DRL, in terms of verifiability and performance guarantees, and the learned controllers possess verifiable properties like persistent feasibility and asymptotic stability akin to MPC. On the other hand, numerical examples illustrate that the proposed controller empirically matches MPC and MLP controllers in terms of control performance and has superior robustness against modeling uncertainty and noises. Furthermore, the proposed controller is significantly more computationally efficient compared to MPC and requires fewer parameters to learn than MLP controllers. Real-world experiments on vehicle drift maneuvering task demonstrate the potential of these controllers for robotics and other demanding control tasks.

eess.SY↗

arXiv2024

Multi-Agent Coverage Control with Transient Behavior Consideration

This paper studies the multi-agent coverage control (MAC) problem where agents must dynamically learn an unknown density function while performing coverage tasks. Unlike many current theoretical frameworks that concentrate solely on the regret occurring at specific targeted sensory locations, our approach additionally considers the regret caused by transient behavior - the path from one location and another. We propose the multi-agent coverage control with the doubling trick (MAC-DT) algorithm and demonstrate that it achieves (approximated) regret of $\widetilde{O}(\sqrt{T})$ even when accounting for the transient behavior. Our result is also supported by numerical experiments, showcasing that the proposed algorithm manages to match or even outperform the baseline algorithms in simulation environments. We also show how our algorithm can be modified to handle safety constraints and further implement the algorithm on a real-robotic testbed.

math.OC↗

arXiv2024

Skill Transfer and Discovery for Sim-to-Real Learning: A Representation-Based Viewpoint

We study sim-to-real skill transfer and discovery in the context of robotics control using representation learning. We draw inspiration from spectral decomposition of Markov decision processes. The spectral decomposition brings about representation that can linearly represent the state-action value function induced by any policies, thus can be regarded as skills. The skill representations are transferable across arbitrary tasks with the same transition dynamics. Moreover, to handle the sim-to-real gap in the dynamics, we propose a skill discovery algorithm that learns new skills caused by the sim-to-real gap from real-world data. We promote the discovery of new skills by enforcing orthogonal constraints between the skills to learn and the skills from simulators, and then synthesize the policy using the enlarged skill sets. We demonstrate our methodology by transferring quadrotor controllers from simulators to Crazyflie 2.1 quadrotors. We show that we can learn the skill representations from a single simulator task and transfer these to multiple different real-world tasks including hovering, taking off, landing and trajectory tracking. Our skill discovery approach helps narrow the sim-to-real gap and improve the real-world controller performance by up to 30.2%.

cs.LG↗

arXiv2024

Aggregate Peak EV Charging Demand: The Impact of Segmented Network Tariffs

Aggregate peak Electric Vehicle (EV) charging demand is a matter of growing concern for network operators as it severely limits the network's capacity, preventing its reliable operation. Various tariff schemes have been proposed to limit peak demand by incentivizing flexible asset users to shift their demand from peak periods. However, fewer studies quantify the effect of these tariff schemes on the aggregate level. In this paper, we compare the effect of a multi-level segmented network tariff with and without dynamic energy prices for individual EV users on the aggregate peak demand. Results based on real charging transactions from over 1200 public charging points in the Netherlands show that the segmented network tariff with flat energy prices results in more diverse load profiles with increasing aggregation, as compared to cost-optimized dispatch based on only dynamic day-ahead energy prices. When paired with dynamic energy prices, the segmented tariff still outperforms only dynamic energy price-based tariffs in reducing peaks. Results show that a balance between power thresholds and price per threshold is crucial in designing a suitable tariff, taking into account the needs of the power network. We also provide valuable insights to network operators by calculating the diversity factor for various peak demands per charging point.

eess.SY↗

arXiv2024

Ontology Completion with Natural Language Inference and Concept Embeddings: An Analysis

We consider the problem of finding plausible knowledge that is missing from a given ontology, as a generalisation of the well-studied taxonomy expansion task. One line of work treats this task as a Natural Language Inference (NLI) problem, thus relying on the knowledge captured by language models to identify the missing knowledge. Another line of work uses concept embeddings to identify what different concepts have in common, taking inspiration from cognitive models for category based induction. These two approaches are intuitively complementary, but their effectiveness has not yet been compared. In this paper, we introduce a benchmark for evaluating ontology completion methods and thoroughly analyse the strengths and weaknesses of both approaches. We find that both approaches are indeed complementary, with hybrid strategies achieving the best overall results. We also find that the task is highly challenging for Large Language Models, even after fine-tuning.

cs.CL↗

arXiv2024

Cooperative Multi-Agent Graph Bandits: UCB Algorithm and Regret Analysis

In this paper, we formulate the multi-agent graph bandit problem as a multi-agent extension of the graph bandit problem introduced by Zhang, Johansson, and Li [CISS 57, 1-6 (2023)]. In our formulation, $N$ cooperative agents travel on a connected graph $G$ with $K$ nodes. Upon arrival at each node, agents observe a random reward drawn from a node-dependent probability distribution. The reward of the system is modeled as a weighted sum of the rewards the agents observe, where the weights capture some transformation of the reward associated with multiple agents sampling the same node at the same time. We propose an Upper Confidence Bound (UCB)-based learning algorithm, Multi-G-UCB, and prove that its expected regret over $T$ steps is bounded by $O(γN\log(T)[\sqrt{KT} + DK])$, where $D$ is the diameter of graph $G$ and $γ$ a boundedness parameter associated with the weight functions. Lastly, we numerically test our algorithm by comparing it to alternative methods.

cs.LG↗

arXiv2024

An AI-Driven Approach to Wind Turbine Bearing Fault Diagnosis from Acoustic Signals

This study aimed to develop a deep learning model for the classification of bearing faults in wind turbine generators from acoustic signals. A convolutional LSTM model was successfully constructed and trained by using audio data from five predefined fault types for both training and validation. To create the dataset, raw audio signal data was collected and processed in frames to capture time and frequency domain information. The model exhibited outstanding accuracy on training samples and demonstrated excellent generalization ability during validation, indicating its proficiency of generalization capability. On the test samples, the model achieved remarkable classification performance, with an overall accuracy exceeding 99.5%, and a false positive rate of less than 1% for normal status. The findings of this study provide essential support for the diagnosis and maintenance of bearing faults in wind turbine generators, with the potential to enhance the reliability and efficiency of wind power generation.

cs.SD↗

arXiv2024

Scalable Reinforcement Learning for Linear-Quadratic Control of Networks

Distributed optimal control is known to be challenging and can become intractable even for linear-quadratic regulator problems. In this work, we study a special class of such problems where distributed state feedback controllers can give near-optimal performance. More specifically, we consider networked linear-quadratic controllers with decoupled costs and spatially exponentially decaying dynamics. We aim to exploit the structure in the problem to design a scalable reinforcement learning algorithm for learning a distributed controller. Recent work has shown that the optimal controller can be well approximated only using information from a $κ$-neighborhood of each agent. Motivated by these results, we show that similar results hold for the agents' individual value and Q-functions. We continue by designing an algorithm, based on the actor-critic framework, to learn distributed controllers only using local information. Specifically, the Q-function is estimated by modifying the Least Squares Temporal Difference for Q-functions method to only use local information. The algorithm then updates the policy using gradient descent. Finally, we evaluate the algorithm through simulations that indeed suggest near-optimal performance.

eess.SY↗

arXiv2024

Machine Unlearning: Taxonomy, Metrics, Applications, Challenges, and Prospects

Personal digital data is a critical asset, and governments worldwide have enforced laws and regulations to protect data privacy. Data users have been endowed with the right to be forgotten of their data. In the course of machine learning (ML), the forgotten right requires a model provider to delete user data and its subsequent impact on ML models upon user requests. Machine unlearning emerges to address this, which has garnered ever-increasing attention from both industry and academia. While the area has developed rapidly, there is a lack of comprehensive surveys to capture the latest advancements. Recognizing this shortage, we conduct an extensive exploration to map the landscape of machine unlearning including the (fine-grained) taxonomy of unlearning algorithms under centralized and distributed settings, debate on approximate unlearning, verification and evaluation metrics, challenges and solutions for unlearning under different applications, as well as attacks targeting machine unlearning. The survey concludes by outlining potential directions for future research, hoping to serve as a guide for interested scholars.

cs.LG↗

arXiv2024

Production of singly charm pentaquark from meson-baryon interaction in B decay

In the paper, we discuss the possible interpretation of the $J^P=1/2^-$ singly charm pentaquark as hadronic molecules. With the effective Lagrangian method, we further analyze the production properties of singly charm pentaquark from decays of $B$ meson, including strong coupling constants and production branching ratios of the charm pentaquark. Our numerical results show that the branching ratio of production of pentaquark from B meson can reach to order of $10^{-6}$.

hep-ph↗

arXiv2024

Sliding ferroelectric memories and synapses

Ferroelectric materials with switchable electric polarization hold great promise for a plethora of emergent applications, such as post-Moore's law nanoelectronics, beyond-Boltzmann transistors, non-volatile memories, and above-bandgap photovoltaic devices. Recent advances have uncovered an exotic sliding ferroelectric mechanism, which endows to design atomically thin ferroelectrics from non-ferroelectric parent monolayers. Although notable progress has been witnessed in understanding its fundamental properties, functional devices based on sliding ferroelectrics, the key touchstone toward applications, remain elusive. Here, we demonstrate the rewritable, non-volatile memory devices at room-temperature utilizing a two-dimensional (2D) sliding ferroelectric semiconductor of rhombohedral-stacked bilayer molybdenum disulfide. The 2D sliding ferroelectric memories (SFeMs) show superior performances with a large memory window of >8V, a high conductance ratio of above 106, a long retention time of >10 years, and a programming endurance greater than 104 cycles. Remarkably, flexible SFeMs are achieved with state-of-the-art performances competitive to their rigid counterparts and maintain their performances post bending over 103 cycles. Furthermore, synapse-specific Hebbian forms of plasticity and image recognition with a high accuracy of 97.81% are demonstrated based on flexible SFeMs. Our work demonstrates the sliding ferroelectric memories and synaptic plasticity on both rigid and flexible substrates, highlighting the great potential of sliding ferroelectrics for emerging technological applications in brain-inspired in-memory computing, edge intelligence and energy-efficient wearable electronics.

cond-mat.mes-hall↗

arXiv2023

Reduction on the congruences of partial sums of P-recursive sequences

Hou and Liu developed a telescoping method to prove the congruence of partial sums of P-recursive sequences. We release the requirement on the telescoper and utilize the congruence of the sequence. With this approach, we are able to confirm a conjecture of Sun and find a new congruence on the central trinomial coefficient.

math.CO↗

arXiv2023

Ocean Data Quality Assessment through Outlier Detection-enhanced Active Learning

Ocean and climate research benefits from global ocean observation initiatives such as Argo, GLOSS, and EMSO. The Argo network, dedicated to ocean profiling, generates a vast volume of observatory data. However, data quality issues from sensor malfunctions and transmission errors necessitate stringent quality assessment. Existing methods, including machine learning, fall short due to limited labeled data and imbalanced datasets. To address these challenges, we propose an ODEAL framework for ocean data quality assessment, employing AL to reduce human experts' workload in the quality assessment workflow and leveraging outlier detection algorithms for effective model initialization. We also conduct extensive experiments on five large-scale realistic Argo datasets to gain insights into our proposed method, including the effectiveness of AL query strategies and the initial set construction approach. The results suggest that our framework enhances quality assessment efficiency by up to 465.5% with the uncertainty-based query strategy compared to random sampling and minimizes overall annotation costs by up to 76.9% using the initial set built with outlier detectors.

cs.LG↗

arXiv2023

Gradient play in stochastic games: stationary points, convergence, and sample complexity

We study the performance of the gradient play algorithm for stochastic games (SGs), where each agent tries to maximize its own total discounted reward by making decisions independently based on current state information which is shared between agents. Policies are directly parameterized by the probability of choosing a certain action at a given state. We show that Nash equilibria (NEs) and first-order stationary policies are equivalent in this setting, and give a local convergence rate around strict NEs. Further, for a subclass of SGs called Markov potential games (which includes the setting with identical rewards as an important special case), we design a sample-based reinforcement learning algorithm and give a non-asymptotic global convergence rate analysis for both exact gradient play and our sample-based learning algorithm. Our result shows that the number of iterations to reach an $ε$-NE scales linearly, instead of exponentially, with the number of agents. Local geometry and local stability are also considered, where we prove that strict NEs are local maxima of the total potential function and fully-mixed NEs are saddle points.

cs.LG↗

arXiv2023

Towards Carbon-Free Electricity: A Flow-Based Framework for Power Grid Carbon Accounting and Decarbonization

This paper introduces a comprehensive framework aimed at advancing research and policy development in the realm of decarbonization within electric power systems. The framework focuses on three key aspects: carbon accounting, carbon-aware decision-making, and carbon-electricity market design. It addresses existing problems, methods, and proposes solutions. In contrast to traditional pool-based emissions models, our framework proposes a novel flow-based emissions model. This model incorporates the underlying physical power grid and power flows, allowing for accurate carbon accounting at both temporal and spatial scales. This, in turn, facilitates informed decision-making to achieve grid decarbonization goals. The framework is built on a flow-based accounting methodology and utilizes the carbon-aware optimal power flow (C-OPF) technique as a theoretical foundation for decarbonization decision-making. Additionally, the paper explores the potential design of carbon-electricity markets and pricing mechanisms to incentivize decentralized decarbonization actions. The critical issues of data availability, infrastructure development, and considerations of fairness and equity are also discussed. This paper seeks to advance scholarly understanding and foster progress toward achieving sustainable and carbon-free electric power systems.

eess.SY↗

arXiv2023

Optically controlling the competition between spin flips and intersite spin transfer in a Heusler half-metal on sub-100 fs timescales

The direct manipulation of spins via light may provide a path toward ultrafast energy-efficient devices. However, distinguishing the microscopic processes that can occur during ultrafast laser excitation in magnetic alloys is challenging. Here, we study the Heusler compound Co2MnGa, a material that exhibits very strong light-induced spin transfers across the entire M-edge. By combining the element-specificity of extreme ultraviolet high harmonic probes with time-dependent density functional theory, we disentangle the competition between three ultrafast light-induced processes that occur in Co2MnGa: same-site Co-Co spin transfer, intersite Co-Mn spin transfer, and ultrafast spin-flips mediated by spin-orbit coupling. By measuring the dynamic magnetic asymmetry across the entire M-edges of the two magnetic sublattices involved, we uncover the relative dominance of these processes at different probe energy regions and times during the laser pulse. Our combined approach enables a comprehensive microscopic interpretation of laser-induced magnetization dynamics on timescales shorter than 100 fs.

cond-mat.mtrl-sci↗

arXiv2023

What do Deck Chairs and Sun Hats Have in Common? Uncovering Shared Properties in Large Concept Vocabularies

Concepts play a central role in many applications. This includes settings where concepts have to be modelled in the absence of sentence context. Previous work has therefore focused on distilling decontextualised concept embeddings from language models. But concepts can be modelled from different perspectives, whereas concept embeddings typically mostly capture taxonomic structure. To address this issue, we propose a strategy for identifying what different concepts, from a potentially large concept vocabulary, have in common with others. We then represent concepts in terms of the properties they share with the other concepts. To demonstrate the practical usefulness of this way of modelling concepts, we consider the task of ultra-fine entity typing, which is a challenging multi-label classification problem. We show that by augmenting the label set with shared properties, we can improve the performance of the state-of-the-art models for this task.

cs.CL↗

arXiv2023

Extremely large anomalous Hall conductivity and unusual axial diamagnetism in a quasi-1D Dirac material La$_3$MgBi$_5$

Anomalous Hall effect (AHE), one of the most important electronic transport phenomena, generally appears in ferromagnetic materials but is rare in materials without magnetic elements. Here, we present a study of La$_3$MgBi$_5$, whose band structure carries multitype Dirac fermions. Although magnetic elements are absent in La$_3$MgBi$_5$, clear signals of AHE can be observed. In particular, the anomalous Hall conductivity is extremely large, reaching 42,356 $Ω^{-1}$ cm$^{-1}$ with an anomalous Hall angle of 8.8 %, the largest one that has been observed in the current AHE systems. The AHE is suggested to originate from the combination of skew scattering and Berry curvature. Another unique property discovered in La$_3$MgBi$_5$ is the axial diamagnetism. The diamagnetism is significantly enhanced and dominates the magnetization in the axial directions, which is the result of restricted motion of the Dirac fermion at Fermi level. Our findings not only establish La$_3$MgBi$_5$ as a suitable platform to study AHE and quantum transport, but also indicate the great potential of 315-type Bi-based materials for exploring novel physical properties.

cond-mat.mtrl-sci↗