arXiv subjects

Ling Feng

Publications and source records attributed to Ling Feng.

At least 37 records · Page 2Linked to original sources

arXiv2024

KB-Plugin: A Plug-and-play Framework for Large Language Models to Induce Programs over Low-resourced Knowledge Bases

Program induction (PI) has become a promising paradigm for using knowledge bases (KBs) to help large language models (LLMs) answer complex knowledge-intensive questions. Nonetheless, PI typically relies on a large number of parallel question-program pairs to make the LLM aware of the schema of the given KB, and is thus challenging for many low-resourced KBs that lack annotated data. To this end, we propose KB-Plugin, a plug-and-play framework that enables LLMs to induce programs over any low-resourced KB. Firstly, KB-Plugin adopts self-supervised learning to encode the detailed schema information of a given KB into a pluggable module, namely schema plugin. Secondly, KB-Plugin utilizes abundant annotated data from a rich-resourced KB to train another pluggable module, namely PI plugin, which can help the LLM extract question-relevant schema information from the schema plugin of any KB and utilize this information to induce programs over this KB. Experiments on five heterogeneous KBQA datasets show that KB-Plugin achieves better or comparable performance with 25$\times$ smaller backbone LLM compared to SoTA PI methods for low-resourced KBs, and even approaches the performance of supervised methods. Our code and data are available at https://github.com/THU-KEG/KB-Plugin.

cs.CL↗

arXiv2023

FedSSC: Shared Supervised-Contrastive Federated Learning

Federated learning is widely used to perform decentralized training of a global model on multiple devices while preserving the data privacy of each device. However, it suffers from heterogeneous local data on each training device which increases the difficulty to reach the same level of accuracy as the centralized training. Supervised Contrastive Learning which outperform cross-entropy tries to minimizes the difference between feature space of points belongs to the same class and pushes away points from different classes. We propose Supervised Contrastive Federated Learning in which devices can share the learned class-wise feature spaces with each other and add the supervised-contrastive learning loss as a regularization term to foster the feature space learning. The loss tries to minimize the cosine similarity distance between the feature map and the averaged feature map from another device in the same class and maximizes the distance between the feature map and that in a different class. This new regularization term when added on top of the moon regularization term is found to outperform the other state-of-the-art regularization terms in solving the heterogeneous data distribution problem.

cs.LG↗

arXiv2022

Interactive Contrastive Learning for Self-supervised Entity Alignment

Self-supervised entity alignment (EA) aims to link equivalent entities across different knowledge graphs (KGs) without seed alignments. The current SOTA self-supervised EA method draws inspiration from contrastive learning, originally designed in computer vision based on instance discrimination and contrastive loss, and suffers from two shortcomings. Firstly, it puts unidirectional emphasis on pushing sampled negative entities far away rather than pulling positively aligned pairs close, as is done in the well-established supervised EA. Secondly, KGs contain rich side information (e.g., entity description), and how to effectively leverage those information has not been adequately investigated in self-supervised EA. In this paper, we propose an interactive contrastive learning model for self-supervised EA. The model encodes not only structures and semantics of entities (including entity name, entity description, and entity neighborhood), but also conducts cross-KG contrastive learning by building pseudo-aligned entity pairs. Experimental results show that our approach outperforms previous best self-supervised results by a large margin (over 9% average improvement) and performs on par with previous SOTA supervised counterparts, demonstrating the effectiveness of the interactive contrastive learning for self-supervised EA.

cs.CL↗

arXiv2021

Edge of chaos as a guiding principle for modern neural network training

The success of deep neural networks in real-world problems has prompted many attempts to explain their training dynamics and generalization performance, but more guiding principles for the training of neural networks are still needed. Motivated by the edge of chaos principle behind the optimal performance of neural networks, we study the role of various hyperparameters in modern neural network training algorithms in terms of the order-chaos phase diagram. In particular, we study a fully analytical feedforward neural network trained on the widely adopted Fashion-MNIST dataset, and study the dynamics associated with the hyperparameters in back-propagation during the training process. We find that for the basic algorithm of stochastic gradient descent with momentum, in the range around the commonly used hyperparameter values, clear scaling relations are present with respect to the training time during the ordered phase in the phase diagram, and the model's optimal generalization power at the edge of chaos is similar across different training parameter combinations. In the chaotic phase, the same scaling no longer exists. The scaling allows us to choose the training parameters to achieve faster training without sacrificing performance. In addition, we find that the commonly used model regularization method - weight decay - effectively pushes the model towards the ordered phase to achieve better performance. Leveraging on this fact and the scaling relations in the other hyperparameters, we derived a principled guideline for hyperparameter determination, such that the model can achieve optimal performance by saturating it at the edge of chaos. Demonstrated on this simple neural network model and training algorithm, our work improves the understanding of neural network training dynamics, and can potentially be extended to guiding principles of more complex model architectures and algorithms.

cs.LG↗

arXiv2021

Identify Hidden Spreaders of Pandemic over Contact Tracing Networks

The COVID-19 infection cases have surged globally, causing devastations to both the society and economy. A key factor contributing to the sustained spreading is the presence of a large number of asymptomatic or hidden spreaders, who mix among the susceptible population without being detected or quarantined. Here we propose an effective non-pharmacological intervention method of detecting the asymptomatic spreaders in contact-tracing networks, and validated it on the empirical COVID-19 spreading network in Singapore. We find that using pure physical spreading equations, the hidden spreaders of COVID-19 can be identified with remarkable accuracy. Specifically, based on the unique characteristics of COVID-19 spreading dynamics, we propose a computational framework capturing the transition probabilities among different infectious states in a network, and extend it to an efficient algorithm to identify asymptotic individuals. Our simulation results indicate that a screening method using our prediction outperforms machine learning algorithms, e.g. graph neural networks, that are designed as baselines in this work, as well as random screening of infection's closest contacts widely used by China in its early outbreak. Furthermore, our method provides high precision even with incomplete information of the contract-tracing networks. Our work can be of critical importance to the non-pharmacological interventions of COVID-19, especially with increasing adoptions of contact tracing measures using various new technologies. Beyond COVID-19, our framework can be useful for other epidemic diseases that also feature asymptomatic spreading

physics.soc-ph↗

arXiv2020

Building and Using Personal Knowledge Graph to Improve Suicidal Ideation Detection on Social Media

A large number of individuals are suffering from suicidal ideation in the world. There are a number of causes behind why an individual might suffer from suicidal ideation. As the most popular platform for self-expression, emotion release, and personal interaction, individuals may exhibit a number of symptoms of suicidal ideation on social media. Nevertheless, challenges from both data and knowledge aspects remain as obstacles, constraining the social media-based detection performance. Data implicitness and sparsity make it difficult to discover the inner true intentions of individuals based on their posts. Inspired by psychological studies, we build and unify a high-level suicide-oriented knowledge graph with deep neural networks for suicidal ideation detection on social media. We further design a two-layered attention mechanism to explicitly reason and establish key risk factors to individual's suicidal ideation. The performance study on microblog and Reddit shows that: 1) with the constructed personal knowledge graph, the social media-based suicidal ideation detection can achieve over 93% accuracy; and 2) among the six categories of personal factors, post, personality, and experience are the top-3 key indicators. Under these categories, posted text, stress level, stress duration, posted image, and ruminant thinking contribute to one's suicidal ideation detection.

cs.CL↗

arXiv2020

Induced Percolation on Networked Systems

Percolation theory has been widely used to study phase transitions in complex networked systems. It has also successfully explained several macroscopic phenomena across different fields. Yet, the existent theoretical framework for percolation places the focus on the direct interactions among the system's components, while recent empirical observations have shown that indirect interactions are common in many systems like ecological and social networks, among others. Here, we propose a new percolation framework that accounts for indirect interactions, which allows to generalize the current theoretical body and understand the role of the underlying indirect influence of the components of a networked system on its macroscopic behavior. We report a rich phenomenology in which first-order, second-order or hybrid phase transitions are possible depending on whether the links of the substrate network are directed, undirected or a mix, respectively. We also present an analytical framework to characterize the proposed induced percolation, paving the way to further understand network dynamics with indirect interactions.

physics.soc-ph↗

arXiv2020

Optimal Machine Intelligence at the Edge of Chaos

It has long been suggested that the biological brain operates at some critical point between two different phases, possibly order and chaos. Despite many indirect empirical evidence from the brain and analytical indication on simple neural networks, the foundation of this hypothesis on generic non-linear systems remains unclear. Here we develop a general theory that reveals the exact edge of chaos is the boundary between the chaotic phase and the (pseudo)periodic phase arising from Neimark-Sacker bifurcation. This edge is analytically determined by the asymptotic Jacobian norm values of the non-linear operator and influenced by the dimensionality of the system. The optimality at the edge of chaos is associated with the highest information transfer between input and output at this point similar to that of the logistic map. As empirical validations, our experiments on the various deep learning models in computer vision demonstrate the optimality of the models near the edge of chaos, and we observe that the state-of-art training algorithms push the models towards such edge as they become more accurate. We further establishes the theoretical understanding of deep learning model generalization through asymptotic stability.

cs.LG↗

arXiv2020

Collaborative Inference for Efficient Remote Monitoring

While current machine learning models have impressive performance over a wide range of applications, their large size and complexity render them unsuitable for tasks such as remote monitoring on edge devices with limited storage and computational power. A naive approach to resolve this on the model level is to use simpler architectures, but this sacrifices prediction accuracy and is unsuitable for monitoring applications requiring accurate detection of the onset of adverse events. In this paper, we propose an alternative solution to this problem by decomposing the predictive model as the sum of a simple function which serves as a local monitoring tool, and a complex correction term to be evaluated on the server. A sign requirement is imposed on the latter to ensure that the local monitoring function is safe, in the sense that it can effectively serve as an early warning system. Our analysis quantifies the trade-offs between model complexity and performance, and serves as a guidance for architecture design. We validate our proposed framework on a series of monitoring experiments, where we succeed at learning monitoring models with significantly reduced complexity that minimally violate the safety requirement. More broadly, our framework is useful for learning classifiers in applications where false negatives are significantly more costly compared to false positives.

cs.LG↗

arXiv2019

Latent Suicide Risk Detection on Microblog via Suicide-Oriented Word Embeddings and Layered Attention

Despite detection of suicidal ideation on social media has made great progress in recent years, people's implicitly and anti-real contrarily expressed posts still remain as an obstacle, constraining the detectors to acquire higher satisfactory performance. Enlightened by the hidden "tree holes" phenomenon on microblog, where people at suicide risk tend to disclose their inner real feelings and thoughts to the microblog space whose authors have committed suicide, we explore the use of tree holes to enhance microblog-based suicide risk detection from the following two perspectives. (1) We build suicide-oriented word embeddings based on tree hole contents to strength the sensibility of suicide-related lexicons and context based on tree hole contents. (2) A two-layered attention mechanism is deployed to grasp intermittently changing points from individual's open blog streams, revealing one's inner emotional world more or less. Our experimental results show that with suicide-oriented word embeddings and attention, microblog-based suicide risk detection can achieve over 91\% accuracy. A large-scale well-labelled suicide data set is also reported in the paper.

cs.CL↗

arXiv2018

Local structure can identify and quantify influential global spreaders in large scale social networks

Measuring and optimizing the influence of nodes in big-data online social networks are important for many practical applications, such as the viral marketing and the adoption of new products. As the viral spreading on social network is a global process, it is commonly believed that measuring the influence of nodes inevitably requires the knowledge of the entire network. Employing percolation theory, we show that the spreading process displays a nucleation behavior: once a piece of information spread from the seeds to more than a small characteristic number of nodes, it reaches a point of no return and will quickly reach the percolation cluster, regardless of the entire network structure, otherwise the spreading will be contained locally. Thus, we find that, without the knowledge of entire network, any nodes' global influence can be accurately measured using this characteristic number, which is independent of the network size. This motivates an efficient algorithm with constant time complexity on the long standing problem of best seed spreaders selection, with performance remarkably close to the true optimum.

physics.soc-ph↗

arXiv2018

Non-trivial Resource Amount Requirement in the Early Stage for Containing Fatal Diseases

During an epidemic control, the containment of the disease is usually achieved through increasing devoted resource to shorten the duration of infectiousness. However, the impact of this resource expenditure has not been studied quantitatively. Using the well-documented cholera data, we observe empirically that the recovery rate which is related to the duration of infectiousness has a strong positive correlation with the average resource devoted to the infected individuals. By incorporating this relation we build a novel model and find that insufficient resource leads to an abrupt increase in the infected population size, which is in marked contrast with the continuous phase transitions believed previously. Counterintuitively, this abrupt phase transition is more pronounced in the less contagious diseases, which usually correspond to the most fatal ones. Furthermore, we find that even for a single infection source, public resource needs to meet a significant amount, which is proportional to the whole population size to ensure epidemic containment. Our findings provide a theoretical foundation for efficient epidemic containment strategies in the early stage.

physics.soc-ph↗

arXiv2016

A Method to Support Difficult Re-finding Tasks

Re-finding electronic documents from a personal computer is a frequent demand to users. In a simple re-finding task, people can use many methods to retrieve a document, such as navigating directly to the document's folder, searching with a desktop search engine, or checking the Recent Files List. However, when encountering a difficult re-finding task, people usually cannot remember the attributes used by conventional re-finding methods, such as file path, file name, keywords etc., the re-finding would fail. We propose a new method to support difficult re-finding tasks. When a user is reading a document, we collect all kinds of possible memory pieces of the user about the document, such as number of pages, number of images, number of math formulas, cumulative reading time, reading frequency, printing experiences etc. If the user wants to re-find a document later, we use these collected attributes to filter out the target document. To alleviate the user's cognitive burden, we use a question and answer wizard interface and provide recommendations to the answers for the user, the recommendations are generated by analyzing the collected attributes of each document and the user's experiences about them.

cs.IR↗

arXiv2015

A Simplified Self-Consistent Probabilities Framework to Characterize Percolation Phenomena on Interdependent Networks : An Overview

Interdependent networks are ubiquitous in our society, ranging from infrastructure to economics, and the study of their cascading behaviors using percolation theory has attracted much attention in the recent years. To analyze the percolation phenomena of these systems, different mathematical frameworks have been proposed including generating functions, eigenvalues among some others. These different frameworks approach the phase transition behaviors from different angles, and have been very successful in shaping the different quantities of interest including critical threshold, size of the giant component, order of phase transition and the dynamics of cascading. These methods also vary in their mathematical complexity in dealing with interdependent networks that have additional complexity in terms of the correlation among different layers of networks or links. In this work, we review a particular approach of simple self-consistent probability equations, and illustrate that it can greatly simplify the mathematical analysis for systems ranging from single layer network to various different interdependent networks. We give an overview on the detailed framework to study the nature of the critical phase transition, value of the critical threshold and size of the giant component for these different systems.

physics.soc-ph↗

arXiv2014

Competing for Attention in Social Media under Information Overload Conditions

Although the many forms of modern social media have become major channels for the dissemination of information, they are becoming overloaded because of the rapidly-expanding number of information feeds. We analyze the expanding user-generated content in Sina Weibo, the largest micro-blog site in China, and find evidence that popular messages often follow a mechanism that differs from that found in the spread of disease, in contrast to common believe. In this mechanism, an individual with more friends needs more repeated exposures to spread further the information. Moreover, our data suggest that in contrast to epidemics, for certain messages the chance of an individual to share the message is proportional to the fraction of its neighbours who shared it with him/her. Thus the greater the number of friends an individual has the greater the number of repeated contacts needed to spread the message, which is a result of competition for attention. We model this process using a fractional susceptible infected recovered (FSIR) model, where the infection probability of a node is proportional to its fraction of infected neighbors. Our findings have dramatic implications for information contagion. For example, using the FSIR model we find that real-world social networks have a finite epidemic threshold. This is in contrast to the zero threshold that conventional wisdom derives from disease epidemic models. This means that when individuals are overloaded with excess information feeds, the information either reaches out the population if it is above the critical epidemic threshold, or it would never be well received, leading to only a handful of information contents that can be widely spread throughout the population.

physics.soc-ph↗

arXiv2014

On the Real-time Prediction Problems of Bursting Hashtags in Twitter

Hundreds of thousands of hashtags are generated every day on Twitter. Only a few become bursting topics. Among the few, only some can be predicted in real-time. In this paper, we take the initiative to conduct a systematic study of a series of challenging real-time prediction problems of bursting hashtags. Which hashtags will become bursting? If they do, when will the burst happen? How long will they remain active? And how soon will they fade away? Based on empirical analysis of real data from Twitter, we provide insightful statistics to answer these questions, which span over the entire lifecycles of hashtags.

cs.SI↗

arXiv2014

Object Selection under Team Context

Context-aware database has drawn increasing attention from both industry and academia recently by taking users' current situation and environment into consideration. However, most of the literature focus on individual context, overlooking the team users. In this paper, we investigate how to integrate team context into database query process to help the users' get top-ranked database tuples and make the team more competitive. We introduce naive and optimized query algorithm to select the suitable records and show that they output the same results while the latter is more computational efficient. Extensive empirical studies are conducted to evaluate the query approaches and demonstrate their effectiveness and efficiency.

cs.DB↗

arXiv2014

A 2D based Partition Strategy for Solving Ranking under Team Context (RTP)

In this paper, we propose a 2D based partition method for solving the problem of Ranking under Team Context(RTC) on datasets without a priori. We first map the data into 2D space using its minimum and maximum value among all dimensions. Then we construct window queries with consideration of current team context. Besides, during the query mapping procedure, we can pre-prune some tuples which are not top ranked ones. This pre-classified step will defer processing those tuples and can save cost while providing solutions for the problem. Experiments show that our algorithm performs well especially on large datasets with correctness.

cs.DB↗