arXiv subjects

Bing Liu

Publications and source records attributed to Bing Liu.

At least 109 records · Page 6Linked to original sources

arXiv2024

Generic Approach to Intrinsic Magnetic Second-order Topological Insulators via Inverted $p-d$ Orbitals

The integration of intrinsically magnetic and topologically nontrivial two-dimensional materials holds tantalizing prospects for the exotic quantum anomalous Hall insulators and magnetic second-order topological insulators (SOTIs). Compared with the well-studied nonmagnetic counterparts, the pursuit of intrinsic magnetic SOTIs remains limited. In this work, we address this gap by focusing on $p-d$ orbitals inversion, a fundamental but often overlooked phenomena in the construction of topological materials. We begin by developing a theoretical framework to elucidate $p-d$ orbitals inversion through a combined density-functional theory calculation and Wannier downfolding. Subsequently we showcase the generality of this concept in realizing ferromagnetism SOTIs by identifying two real materials with distinct lattices: 1$T$-VS$_2$ in a hexagonal lattice, and CrAs monolayer in a square lattice. We further compare it with other mechanisms requiring spin-orbit coupling and explore the similarities to topological Kondo insulators. Our findings establish a generic pathway towards intrinsic magnetic SOTIs.

cond-mat.mes-hall↗

arXiv2024

Cryptography-Based Privacy-Preserving Method for Distributed Optimization over Time-Varying Directed Graphs with Enhanced Efficiency

In this paper, we study the privacy-preserving distributed optimization problem, aiming to prevent attackers from stealing the private information of agents. For this purpose, we propose a novel privacy-preserving algorithm based on the Advanced Encryption Standard (AES), which is both secure and computationally efficient. By appropriately constructing the underlying weight matrices, our algorithm can be applied to time-varying directed networks. We show that the proposed algorithm can protect an agent's privacy if the agent has at least one legitimate neighbor at the initial iteration. Under the assumption that the objective function is strongly convex and Lipschitz smooth, we rigorously prove that the proposed algorithm has a linear convergence rate. Finally, the effectiveness of the proposed algorithm is demonstrated by numerical simulations of the canonical sensor fusion problem.

math.OC↗

arXiv2024

Modeling Low-Resource Health Coaching Dialogues via Neuro-Symbolic Goal Summarization and Text-Units-Text Generation

Health coaching helps patients achieve personalized and lifestyle-related goals, effectively managing chronic conditions and alleviating mental health issues. It is particularly beneficial, however cost-prohibitive, for low-socioeconomic status populations due to its highly personalized and labor-intensive nature. In this paper, we propose a neuro-symbolic goal summarizer to support health coaches in keeping track of the goals and a text-units-text dialogue generation model that converses with patients and helps them create and accomplish specific goals for physical activities. Our models outperform previous state-of-the-art while eliminating the need for predefined schema and corresponding annotation. We also propose a new health coaching dataset extending previous work and a metric to measure the unconventionality of the patient's response based on data difficulty, facilitating potential coach alerts during deployment.

cs.CL↗

arXiv2024

Towards Enhancing Health Coaching Dialogue in Low-Resource Settings

Health coaching helps patients identify and accomplish lifestyle-related goals, effectively improving the control of chronic diseases and mitigating mental health conditions. However, health coaching is cost-prohibitive due to its highly personalized and labor-intensive nature. In this paper, we propose to build a dialogue system that converses with the patients, helps them create and accomplish specific goals, and can address their emotions with empathy. However, building such a system is challenging since real-world health coaching datasets are limited and empathy is subtle. Thus, we propose a modularized health coaching dialogue system with simplified NLU and NLG frameworks combined with mechanism-conditioned empathetic response generation. Through automatic and human evaluation, we show that our system generates more empathetic, fluent, and coherent responses and outperforms the state-of-the-art in NLU tasks while requiring less annotation. We view our approach as a key step towards building automated and more accessible health coaching systems.

cs.CL↗

arXiv2024

A Theory for Length Generalization in Learning to Reason

Length generalization (LG) is a challenging problem in learning to reason. It refers to the phenomenon that when trained on reasoning problems of smaller lengths or sizes, the resulting model struggles with problems of larger sizes or lengths. Although LG has been studied by many researchers, the challenge remains. This paper proposes a theoretical study of LG for problems whose reasoning processes can be modeled as DAGs (directed acyclic graphs). The paper first identifies and proves the conditions under which LG can be achieved in learning to reason. It then designs problem representations based on the theory to learn to solve challenging reasoning problems like parity, addition, and multiplication, using a Transformer to achieve perfect LG.

cs.AI↗

arXiv2024

Continual Learning: Applications and the Road Forward

Continual learning is a subfield of machine learning, which aims to allow machine learning models to continuously learn on new data, by accumulating knowledge without forgetting what was learned in the past. In this work, we take a step back, and ask: "Why should one care about continual learning in the first place?". We set the stage by examining recent continual learning papers published at four major machine learning conferences, and show that memory-constrained settings dominate the field. Then, we discuss five open problems in machine learning, and even though they might seem unrelated to continual learning at first sight, we show that continual learning will inevitably be part of their solution. These problems are model editing, personalization and specialization, on-device learning, faster (re-)training and reinforcement learning. Finally, by comparing the desiderata from these unsolved problems and the current assumptions in continual learning, we highlight and discuss four future directions for continual learning research. We hope that this work offers an interesting perspective on the future of continual learning, while displaying its potential value and the paths we have to pursue in order to make it successful. This work is the result of the many discussions the authors had at the Dagstuhl seminar on Deep Continual Learning, in March 2023.

cs.LG↗

arXiv2024

Class Incremental Learning via Likelihood Ratio Based Task Prediction

Class incremental learning (CIL) is a challenging setting of continual learning, which learns a series of tasks sequentially. Each task consists of a set of unique classes. The key feature of CIL is that no task identifier (or task-id) is provided at test time. Predicting the task-id for each test sample is a challenging problem. An emerging theory-guided approach (called TIL+OOD) is to train a task-specific model for each task in a shared network for all tasks based on a task-incremental learning (TIL) method to deal with catastrophic forgetting. The model for each task is an out-of-distribution (OOD) detector rather than a conventional classifier. The OOD detector can perform both within-task (in-distribution (IND)) class prediction and OOD detection. The OOD detection capability is the key to task-id prediction during inference. However, this paper argues that using a traditional OOD detector for task-id prediction is sub-optimal because additional information (e.g., the replay data and the learned tasks) available in CIL can be exploited to design a better and principled method for task-id prediction. We call the new method TPL (Task-id Prediction based on Likelihood Ratio). TPL markedly outperforms strong CIL baselines and has negligible catastrophic forgetting. The code of TPL is publicly available at https://github.com/linhaowei1/TPL.

cs.LG↗

arXiv2024

Self-Initiated Open World Learning for Autonomous AI Agents

As more and more AI agents are used in practice, it is time to think about how to make these agents fully autonomous so that they can learn by themselves in a self-motivated and self-supervised manner rather than being retrained periodically on the initiation of human engineers using expanded training data. As the real-world is an open environment with unknowns or novelties, detecting novelties or unknowns, characterizing them, accommodating or adapting to them, gathering ground-truth training data, and incrementally learning the unknowns/novelties are critical to making the agent more and more knowledgeable and powerful over time. The key challenge is how to automate the process so that it is carried out on the agent's own initiative and through its own interactions with humans and the environment. Since an AI agent usually has a performance task, characterizing each novelty becomes critical and necessary so that the agent can formulate an appropriate response to adapt its behavior to accommodate the novelty and to learn from it to improve the agent's adaptation capability and task performance. The process goes continually without termination. This paper proposes a theoretic framework for this learning paradigm to promote the research of building Self-initiated Open world Learning (SOL) agents. An example SOL agent is also described.

cs.AI↗

arXiv2024

Retrieval Augmented Cross-Modal Tag Recommendation in Software Q&A Sites

Posts in software Q\&A sites often consist of three main parts: title, description and code, which are interconnected and jointly describe the question. Existing tag recommendation methods often treat different modalities as a whole or inadequately consider the interaction between different modalities. Additionally, they focus on extracting information directly from the post itself, neglecting the information from external knowledge sources. Therefore, we propose a Retrieval Augmented Cross-Modal (RACM) Tag Recommendation Model in Software Q\&A Sites. Specifically, we first use the input post as a query and enhance the representation of different modalities by retrieving information from external knowledge sources. For the retrieval-augmented representations, we employ a cross-modal context-aware attention to leverage the main modality description for targeted feature extraction across the submodalities title and code. In the fusion process, a gate mechanism is employed to achieve fine-grained feature selection, controlling the amount of information extracted from the submodalities. Finally, the fused information is used for tag recommendation. Experimental results on three real-world datasets demonstrate that our model outperforms the state-of-the-art counterparts.

cs.IR↗

arXiv2024

How to Forget Clients in Federated Online Learning to Rank?

Data protection legislation like the European Union's General Data Protection Regulation (GDPR) establishes the \textit{right to be forgotten}: a user (client) can request contributions made using their data to be removed from learned models. In this paper, we study how to remove the contributions made by a client participating in a Federated Online Learning to Rank (FOLTR) system. In a FOLTR system, a ranker is learned by aggregating local updates to the global ranking model. Local updates are learned in an online manner at a client-level using queries and implicit interactions that have occurred within that specific client. By doing so, each client's local data is not shared with other clients or with a centralised search service, while at the same time clients can benefit from an effective global ranking model learned from contributions of each client in the federation. In this paper, we study an effective and efficient unlearning method that can remove a client's contribution without compromising the overall ranker effectiveness and without needing to retrain the global ranker from scratch. A key challenge is how to measure whether the model has unlearned the contributions from the client $c^*$ that has requested removal. For this, we instruct $c^*$ to perform a poisoning attack (add noise to this client updates) and then we measure whether the impact of the attack is lessened when the unlearning process has taken place. Through experiments on four datasets, we demonstrate the effectiveness and efficiency of the unlearning strategy under different combinations of parameter settings.

cs.CR↗

arXiv2024

Limits on the Primordial Black Holes Dark Matter with future MeV detectors

Primordial black holes (PBHs) are a compelling candidate for Dark Matter (DM). There remain significant parameter spaces to be explored despite current astrophysical observations have set strong limits. Utilizing advanced MeV observation instruments, we have statistically established the upper limit of Hawking radiation emitted by PBHs in DM-dense systems, such as galaxy clusters or dwarf galaxies. These results can set a stringent upper limit on the ratio of PBH to DM, expressed as $f_{\rm PBH}$. Our results highlight the efficacy of MeV observations in DM-dense environments. The constraints on $f_{\rm PBH}$ for PBHs in the mass range of $10^{16}-10^{17} ~\rm g$ can be improved significantly compared with the current observations.

astro-ph.HE↗

arXiv2023

Grounding for Artificial Intelligence

A core function of intelligence is grounding, which is the process of connecting the natural language and abstract knowledge to the internal representation of the real world in an intelligent being, e.g., a human. Human cognition is grounded in our sensorimotor experiences in the external world and subjective feelings in our internal world. We use languages to communicate with each other and the languages are grounded on our shared sensorimotor experiences and feelings. Without this shard grounding, it is impossible for us to understand each other because all natural languages are highly abstract and are only able to describe a tiny portion of what has happened or is happening in the real world. Although grounding at high or abstract levels has been studied in different fields and applications, to our knowledge, limited systematic work at fine-grained levels has been done. With the rapid progress of large language models (LLMs), it is imperative that we have a sound understanding of grounding in order to move to the next level of intelligence. It is also believed that grounding is necessary for Artificial General Intelligence (AGI). This paper makes an attempt to systematically study this problem.

cs.AI↗

arXiv2023

Conditions for Length Generalization in Learning Reasoning Skills

Reasoning is a fundamental capability of AI agents. Recently, large language models (LLMs) have shown remarkable abilities to perform reasoning tasks. However, numerous evaluations of the reasoning capabilities of LLMs have also showed some limitations. An outstanding limitation is length generalization, meaning that when trained on reasoning problems of smaller lengths or sizes, the resulting models struggle with problems of larger sizes or lengths. This potentially indicates some theoretical limitations of generalization in learning reasoning skills. These evaluations and their observations motivated us to perform a theoretical study of the length generalization problem. This work focuses on reasoning tasks that can be formulated as Markov dynamic processes (MDPs) and/or directed acyclic graphs (DAGs). It identifies and proves conditions that decide whether the length generalization problem can be solved or not for a reasoning task in a particular representation. Experiments are also conducted to verify the theoretical results.

cs.AI↗

arXiv2023

Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Ranking

In the field of information retrieval, Query Likelihood Models (QLMs) rank documents based on the probability of generating the query given the content of a document. Recently, advanced large language models (LLMs) have emerged as effective QLMs, showcasing promising ranking capabilities. This paper focuses on investigating the genuine zero-shot ranking effectiveness of recent LLMs, which are solely pre-trained on unstructured text data without supervised instruction fine-tuning. Our findings reveal the robust zero-shot ranking ability of such LLMs, highlighting that additional instruction fine-tuning may hinder effectiveness unless a question generation task is present in the fine-tuning dataset. Furthermore, we introduce a novel state-of-the-art ranking system that integrates LLM-based QLMs with a hybrid zero-shot retriever, demonstrating exceptional effectiveness in both zero-shot and few-shot scenarios. We make our codebase publicly available at https://github.com/ielab/llm-qlm.

cs.IR↗

arXiv2023

Shining light on magnetic monopoles through high-energy muon colliders

The search for magnetic monopoles has been a longstanding concern of the physics community for nearly a century. However, up to now, the existence of elementary magnetic monopoles remains an open question. The electroweak 't Hooft-Polyakov monopoles have been predicted with mass at the order of 10 TeV. This mass scale is unreachable at current colliders. Recently, the muon colliders have gained much attention in the community due to technological developments. The advantages of the muon beam encourage us to raise the high-energy option and consider the high-energy muon collider as a unique opportunity to search for magnetic monopoles. This letter discusses the production of magnetic monopoles via the annihilation process and proposes the search for magnetic monopoles at future high-energy muon colliders.

hep-ph↗

arXiv2023

Sub-network Discovery and Soft-masking for Continual Learning of Mixed Tasks

Continual learning (CL) has two main objectives: preventing catastrophic forgetting (CF) and encouraging knowledge transfer (KT). The existing literature mainly focused on overcoming CF. Some work has also been done on KT when the tasks are similar. To our knowledge, only one method has been proposed to learn a sequence of mixed tasks. However, these techniques still suffer from CF and/or limited KT. This paper proposes a new CL method to achieve both. It overcomes CF by isolating the knowledge of each task via discovering a subnetwork for it. A soft-masking mechanism is also proposed to preserve the previous knowledge and to enable the new task to leverage the past knowledge to achieve KT. Experiments using classification, generation, information extraction, and their mixture (i.e., heterogeneous tasks) show that the proposed method consistently outperforms strong baselines.

cs.CL↗

arXiv2023

Sequential Tag Recommendation

With the development of Internet technology and the expansion of social networks, online platforms have become an important way for people to obtain information. The introduction of tags facilitates information categorization and retrieval. Meanwhile, the development of tag recommendation systems not only enables users to input tags more efficiently, but also improves the quality of tags. However, current tag recommendation methods only consider the content of the current post and do not take into account the influence of user preferences. Since the main body of tag recommendation is the user, it is very necessary to obtain the user's tagging habits. Therefore, this paper proposes a tag recommendation algorithm (MLP4STR) based on the dynamic preference of user's behavioral sequence, which models the user's historical post information and historical tag information to obtain the user's dynamic interest changes. A pure MLP structure across feature dimensions is used in sequence modeling to model the interaction between tag content and post content to fully extract the user's interests. Finally tag recommendation is performed.

cs.IR↗

arXiv2023

Diffuse gamma-ray emission around the Rosette Nebula

The Rosette Nebula is a young stellar cluster and molecular cloud complex, located at the edge of the southern shell of a middle-aged SNR Monoceros Loop (G205.5+0.5). We revisited the GeV gamma-ray emission towards the Rosette Nebula using more than 13 years of Fermi-LAT data. We tested several spatial models and found that compared to the result using the CO gas template only, the inclusion of the HII gas template can significantly improve the likelihood fit. We performed spectral analysis using the new spatial template. With both the gamma-ray observation and CO+HII gas data, we derived the cosmic ray spectrum of different components in the vicinity of the Rosette Nebula. We found the gamma-ray emissions from Rosette Nebula are substantially harder than previously reported, which may imply that Rosette Nebula is another example of a gamma-ray emitting young massive star cluster.

astro-ph.HE↗