arXiv subjects

Yilun Liu

Publications and source records attributed to Yilun Liu.

53 records · Page 3Linked to original sources

arXiv2024

PUMA: Efficient Continual Graph Learning for Node Classification with Graph Condensation

When handling streaming graphs, existing graph representation learning models encounter a catastrophic forgetting problem, where previously learned knowledge of these models is easily overwritten when learning with newly incoming graphs. In response, Continual Graph Learning (CGL) emerges as a novel paradigm enabling graph representation learning from streaming graphs. Our prior work, Condense and Train (CaT) is a replay-based CGL framework with a balanced continual learning procedure, which designs a small yet effective memory bankn for replaying. Although the CaT alleviates the catastrophic forgetting problem, there exist three issues: (1) The graph condensation only focuses on labelled nodes while neglecting abundant information carried by unlabelled nodes; (2) The continual training scheme of the CaT overemphasises on the previously learned knowledge, limiting the model capacity to learn from newly added memories; (3) Both the condensation process and replaying process of the CaT are time-consuming. In this paper, we propose a PsUdo-label guided Memory bAnk (PUMA) CGL framework, extending from the CaT to enhance its efficiency and effectiveness by overcoming the above-mentioned weaknesses and limits. To fully exploit the information in a graph, PUMA expands the coverage of nodes during graph condensation with both labelled and unlabelled nodes. Furthermore, a training-from-scratch strategy is proposed to upgrade the previous continual learning scheme for a balanced training between the historical and the new graphs. Besides, PUMA uses a one-time prorogation and wide graph encoders to accelerate the graph condensation and the graph encoding process in the training stage to improve the efficiency of the whole framework. Extensive experiments on six datasets for the node classification task demonstrate the state-of-the-art performance and efficiency over existing methods.

cs.LG↗

arXiv2024

GCondenser: Benchmarking Graph Condensation

Large-scale graphs are valuable for graph representation learning, yet the abundant data in these graphs hinders the efficiency of the training process. Graph condensation (GC) alleviates this issue by compressing the large graph into a significantly smaller one that still supports effective model training. Although recent research has introduced various approaches to improve the effectiveness of the condensed graph, comprehensive and practical evaluations across different GC methods are neglected. This paper proposes the first large-scale graph condensation benchmark, GCondenser, to holistically evaluate and compare mainstream GC methods. GCondenser includes a standardised GC paradigm, consisting of condensation, validation, and evaluation procedures, as well as enabling extensions to new GC methods and datasets. With GCondenser, a comprehensive performance study is conducted, presenting the effectiveness of existing methods. GCondenser is open-sourced and available at https://github.com/superallen13/GCondenser.

cs.LG↗

arXiv2024

LogEval: A Comprehensive Benchmark Suite for Large Language Models In Log Analysis

Log analysis is crucial for ensuring the orderly and stable operation of information systems, particularly in the field of Artificial Intelligence for IT Operations (AIOps). Large Language Models (LLMs) have demonstrated significant potential in natural language processing tasks. In the AIOps domain, they excel in tasks such as anomaly detection, root cause analysis of faults, operations and maintenance script generation, and alert information summarization. However, the performance of current LLMs in log analysis tasks remains inadequately validated. To address this gap, we introduce LogEval, a comprehensive benchmark suite designed to evaluate the capabilities of LLMs in various log analysis tasks for the first time. This benchmark covers tasks such as log parsing, log anomaly detection, log fault diagnosis, and log summarization. LogEval evaluates each task using 4,000 publicly available log data entries and employs 15 different prompts for each task to ensure a thorough and fair assessment. By rigorously evaluating leading LLMs, we demonstrate the impact of various LLM technologies on log analysis performance, focusing on aspects such as self-consistency and few-shot contextual learning. We also discuss findings related to model quantification, Chinese-English question-answering evaluation, and prompt engineering. These findings provide insights into the strengths and weaknesses of LLMs in multilingual environments and the effectiveness of different prompt strategies. Various evaluation methods are employed for different tasks to accurately measure the performance of LLMs in log analysis, ensuring a comprehensive assessment. The insights gained from LogEvals evaluation reveal the strengths and limitations of LLMs in log analysis tasks, providing valuable guidance for researchers and practitioners.

cs.CL↗

arXiv2024

SPIN: Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification

Among the many tasks that Large Language Models (LLMs) have revolutionized is text classification. Current text classification paradigms, however, rely solely on the output of the final layer in the LLM, with the rich information contained in internal neurons largely untapped. In this study, we present SPIN: a model-agnostic framework that sparsifies and integrates internal neurons of intermediate layers of LLMs for text classification. Specifically, SPIN sparsifies internal neurons by linear probing-based salient neuron selection layer by layer, avoiding noise from unrelated neurons and ensuring efficiency. The cross-layer salient neurons are then integrated to serve as multi-layered features for the classification head. Extensive experimental results show our proposed SPIN significantly improves text classification accuracy, efficiency, and interpretability.

cs.LG↗

arXiv2024

CoachLM: Automatic Instruction Revisions Improve the Data Quality in LLM Instruction Tuning

Instruction tuning is crucial for enabling Language Learning Models (LLMs) in responding to human instructions. The quality of instruction pairs used for tuning greatly affects the performance of LLMs. However, the manual creation of high-quality instruction datasets is costly, leading to the adoption of automatic generation of instruction pairs by LLMs as a popular alternative. To ensure the high quality of LLM-generated instruction datasets, several approaches have been proposed. Nevertheless, existing methods either compromise dataset integrity by filtering a large proportion of samples, or are unsuitable for industrial applications. In this paper, instead of discarding low-quality samples, we propose CoachLM, a novel approach to enhance the quality of instruction datasets through automatic revisions on samples in the dataset. CoachLM is trained from the samples revised by human experts and significantly increases the proportion of high-quality samples in the dataset from 17.7% to 78.9%. The effectiveness of CoachLM is further assessed on various real-world instruction test sets. The results show that CoachLM improves the instruction-following capabilities of the instruction-tuned LLM by an average of 29.9%, which even surpasses larger LLMs with nearly twice the number of parameters. Furthermore, CoachLM is successfully deployed in a data management system for LLMs at Huawei, resulting in an efficiency improvement of up to 20% in the cleaning of 40k real-world instruction pairs. We release various assets of CoachLM, including the training data, code and test set (https://github.com/lunyiliu/CoachLM).

cs.CL↗

arXiv2024

Interpretable Online Log Analysis Using Large Language Models with Prompt Strategies

Automated log analysis is crucial in modern software-intensive systems for facilitating program comprehension throughout software maintenance and engineering life cycles. Existing methods perform tasks such as log parsing and log anomaly detection by providing a single prediction value without interpretation. However, given the increasing volume of system events, the limited interpretability of analysis results hinders analysts' comprehension of program status and their ability to take appropriate actions. Moreover, these methods require substantial in-domain training data, and their performance declines sharply (by up to 62.5%) in online scenarios involving unseen logs from new domains, a common occurrence due to rapid software updates. In this paper, we propose LogPrompt, a novel interpretable log analysis approach for online scenarios. LogPrompt employs large language models (LLMs) to perform online log analysis tasks via a suite of advanced prompt strategies tailored for log tasks, which enhances LLMs' performance by up to 380.7% compared with simple prompts. Experiments on nine publicly available evaluation datasets across two tasks demonstrate that LogPrompt, despite requiring no in-domain training, outperforms existing approaches trained on thousands of logs by up to 55.9%. We also conduct a human evaluation of LogPrompt's interpretability, with six practitioners possessing over 10 years of experience, who highly rated the generated content in terms of usefulness and readability (averagely 4.42/5). LogPrompt also exhibits remarkable compatibility with open-source and smaller-scale LLMs, making it flexible for practical deployment. Code of LogPrompt is available at https://github.com/lunyiliu/LogPrompt.

cs.SE↗

arXiv2023

CaseGNN: Graph Neural Networks for Legal Case Retrieval with Text-Attributed Graphs

Legal case retrieval is an information retrieval task in the legal domain, which aims to retrieve relevant cases with a given query case. Recent research of legal case retrieval mainly relies on traditional bag-of-words models and language models. Although these methods have achieved significant improvement in retrieval accuracy, there are still two challenges: (1) Legal structural information neglect. Previous neural legal case retrieval models mostly encode the unstructured raw text of case into a case representation, which causes the lack of important legal structural information in a case and leads to poor case representation; (2) Lengthy legal text limitation. When using the powerful BERT-based models, there is a limit of input text lengths, which inevitably requires to shorten the input via truncation or division with a loss of legal context information. In this paper, a graph neural networks-based legal case retrieval model, CaseGNN, is developed to tackle these challenges. To effectively utilise the legal structural information during encoding, a case is firstly converted into a Text-Attributed Case Graph (TACG), followed by a designed Edge Graph Attention Layer and a readout function to obtain the case graph representation. The CaseGNN model is optimised with a carefully designed contrastive loss with easy and hard negative sampling. Since the text attributes in the case graph come from individual sentences, the restriction of using language models is further avoided without losing the legal context. Extensive experiments have been conducted on two benchmarks from COLIEE 2022 and COLIEE 2023, which demonstrate that CaseGNN outperforms other state-of-the-art legal case retrieval methods. The code has been released on https://github.com/yanran-tang/CaseGNN.

cs.IR↗

arXiv2023

CaT: Balanced Continual Graph Learning with Graph Condensation

Continual graph learning (CGL) is purposed to continuously update a graph model with graph data being fed in a streaming manner. Since the model easily forgets previously learned knowledge when training with new-coming data, the catastrophic forgetting problem has been the major focus in CGL. Recent replay-based methods intend to solve this problem by updating the model using both (1) the entire new-coming data and (2) a sampling-based memory bank that stores replayed graphs to approximate the distribution of historical data. After updating the model, a new replayed graph sampled from the incoming graph will be added to the existing memory bank. Despite these methods are intuitive and effective for the CGL, two issues are identified in this paper. Firstly, most sampling-based methods struggle to fully capture the historical distribution when the storage budget is tight. Secondly, a significant data imbalance exists in terms of the scales of the complex new-coming graph data and the lightweight memory bank, resulting in unbalanced training. To solve these issues, a Condense and Train (CaT) framework is proposed in this paper. Prior to each model update, the new-coming graph is condensed to a small yet informative synthesised replayed graph, which is then stored in a Condensed Graph Memory with historical replay graphs. In the continual learning phase, a Training in Memory scheme is used to update the model directly with the Condensed Graph Memory rather than the whole new-coming graph, which alleviates the data imbalance problem. Extensive experiments conducted on four benchmark datasets successfully demonstrate superior performances of the proposed CaT framework in terms of effectiveness and efficiency. The code has been released on https://github.com/superallen13/CaT-CGL.

cs.LG↗

arXiv2022

Tunable ferroelectric topological defects on 2D topological surfaces: strain engineering skyrmion-like polar structures in 2D materials

Polar topological structures in ferroelectric thin films have recently drawn significant interest due to their fascinating physical behaviors and promising applications in high-density nonvolatile memories. However, most polar topological patterns are only observed in the perovskites superlattices. Here, we report the discovery of the tunable ferroelectric polar topological defective structures designed and achieved by strain engineering in two-dimensional PbX (X=S, Se, and Te) materials using multiscale computational simulations. First, the first-principles calculations demonstrate the strain-induced recoverable ferroelectric phase transition in such 2D materials. The unique polar topological vortex pattern is then induced by applied mechanical indentation, evidenced by molecular dynamics simulations based on a developed deep-learning potential. According to the strain phase diagram and applied complex strain loadings, the diverse polar topological structures, including antivortex structure and flux-closure structure, are predicted to be emergent through the finite-element simulations. We conclude that strain engineering is promising to tailor various designed reversible polar topologies in ultra-flexible 2D materials, which provide excellent opportunities for next-generation nanoelectronics and sensor devices.

cond-mat.mtrl-sci↗

arXiv2019

General Information Theory: Time and Information

This paper introduces time into information theory, gives a more accurate definition of information, and unifies the information in cognition and Shannon information theory. Specially, we consider time as a measure of information, giving a definition of time, event independence at the time frame, and definition of conditional probability. Further, we propose an analysis method of unified time measure, and find the law of information entropy reduction and increase, which indicates that the second law of thermodynamics is only the law at a certain time measure framework. We propose the concept of negative probability and information black hole to interpret the conservation of information in physics. After the introduction of time, we can give the definition of natural variation and artificial variation from the perspective of information, and point out that it is more reasonable to use the mutation to represent the neural network training process. Further, we point out the defects of the existing artificial intelligence.

cs.IT↗

arXiv2017

Effects of temperature and strain rate on mechanical behaviors of Stone-Wales defective monolayer black phosphorene

The mechanical behaviors of monolayer black phosphorene (MBP) are explored by molecular dynamics (MD) simulations using reactive force field. It is revealed that the temperature and strain rate have significant influence on mechanical behaviors of MBP, and they are further weakened by SW (Stone-Wales) defects. In general, the tensile strength for both of the pristine and SW defective MBP decreases with the increase of temperature or decreasing of strain rate. Surprisingly, for relatively high temperature and low strain rate, phase transition from the black phosphorene to a mixture of β-phase (β-P) and γ-phase (γ-P) is observed for the SW-2 defective MBP under armchair tension, while self-healing of the SW-2 defect is observed under zigzag tension. A deformation map of SW-2 defective MBP under armchair tension at different temperature and strain rate is established, which is useful for the design of phosphorene allotropes by strain. The results presented herein yield useful insights for designing and tuning the structure, and the mechanical and physical properties of phosphorene.

physics.comp-ph↗

arXiv2016

Architectures of Soft Robotic Locomotion Enabled by Simple Mechanical Principles

In nature, a variety of limbless locomotion patterns flourish from the small or basic life form (Escherichia coli, the amoeba, etc.) to the large or intelligent creatures (e.g., slugs, starfishes, earthworms, octopuses, jellyfishes, and snakes). Many bioinspired soft robots based on locomotion have been developed in the past decades. In this work, based on the kinematics and dynamics of two representative locomotion modes (i.e., worm-like crawling and snake-like slithering), we propose a broad set of innovative designs for soft mobile robots through simple mechanical principles. Inspired by and go beyond existing biological systems, these designs include 1-D (dimensional), 2-D, and 3-D robotic locomotion patterns enabled by simple actuation of continuous beams. We report herein over 20 locomotion modes achieving various locomotion functions, including crawling, rising, running, creeping, squirming, slithering, swimming, jumping, turning, turning over, helix rolling, wheeling, etc. Some of them are able to reach high speed, high efficiency, and overcome obstacles. All these locomotion strategies and functions can be integrated into a simple beam model. The proposed simple and robust models are adaptive for severe and complex environments. These elegant designs for diverse robotic locomotion patterns are expected to underpin future deployments of soft robots and to inspire series of advanced designs.

cond-mat.soft↗

arXiv2013

Enhancing Graphene-derived Materials through Multimodal and Self-healable Interfaces

Recent studies have shown that graphene-derived materials not only feature outstandingly multifunctional properties, but also act as model materials to implant nanoscale structural engineering insights into their macroscopic performance optimization. Functionalizing the interfaces between graphene sheets by interlayer crosslinks has been proven to be an effective route to tune the mechanical properties. Here we explore the graphene-derived material with a layer-by-layer structure and multiple crosslinking mechanisms. The effects of multimodal and self-healable crosslinks are assessed in terms of interlayer loading transfer capability. The results show that the brick-and-motar hierarchy and synergetic effects from different crosslinks enable synergetic enhancement in the strength and toughness. The findings here could shed light on the development of high-performance paper-, fiber- or film-like macroscopic materials from monolayer nanosheets with nanoengineerable interfaces.

cond-mat.mtrl-sci↗

arXiv2011

Mechanical Properties of Graphene Papers

Graphene-based papers attract particular interests recently owing to their outstanding properties, the key of which is their layer-by-layer hierarchical structures similar to the biomaterials such as bone, teeth and nacre, combining intralayer strong sp2 bonds and interlayer crosslinks for efficient load transfer. Here we firstly study the mechanical properties of various interlayer and intralayer crosslinks via first-principles calculations and then perform continuum model analysis for the overall mechanical properties of graphene-based papers. We find that there is a characteristic length scale l_{0}, defined as \Sqrt{Dh_{0}/4G}, where D is the stiffness of the graphene sheet, h_{0} and G are the height of interlayer crosslink and shear modulus respectively. When the size of the graphene sheets exceeds 3l_{0}, the tension-shear (TS) chain model that are widely used for nanocomposites fails to predict the overall mechanical properties of the graphene-based papers. Instead we proposed here a deformable tension-shear (DTS) model by considering the elastic deformation of the graphene sheets, also the interlayer and intralayer crosslinks. The DTS is then applied to predict the mechanics of graphene-based paper materials under tensile loading. According to the results we thus obtain, optimal design strategies are provided for designing graphene papers with ultrahigh stiffness, strength and toughness.

cond-mat.mtrl-sci↗

arXiv2011

Self-retracting motion of graphite micro-flakes: superlubricity in micrometer scale

Through experimental study, we reveal superlubricity as the mechanism of self-retracting motion of micrometer sized graphite flakes on graphite platforms by correlating respectively the lock-up or self-retraction states with the commensurate or incommensurate contacts. We show that the scale-dependent loss of self-retractability is caused by generation of contact interfacial defects. A HOPG structure is also proposed to understand our experimental observations, particularly in term of the polycrystal structure. The realisation of the superlubricity in micrometer scale in our experiments will have impact in the design and fabrication of micro/nanoelectromechanical systems based on graphitic materials.

cond-mat.mes-hall↗

arXiv2010

Effect of Interlayer Shear to Graphene Resonators

Graphene nanostrips with single or a few layers can be made into bending resonators with extremely high sensitivity to environment changes. In this work we study the effect of interlayer shear on resonant frequencies f of graphene nanostrips, via both molecular dynamics (MD) simulation and elastic model analysis incorporating interlayer shear. Contrary to the classical thin beam theory prediction f~nl^-2 (l is beam length and n layer number), MD simulation results reveal very different dependences, f~l^-1.36 and f - fmono~(n-1)/n (fmono is frequency of the monolayer beam). Interlayer shear modulus of multilayer graphene strips is much smaller than their intralayer Young's modulus, and the weak interlayer interaction can not maintain the registry between the carbon atoms in adjacent layers. Large shear deformation occurs during vibration of multilayer graphene nano-strips. Therefore we propose a multi-beam shear model (MBSM) with the interlayer shear energy of multilayer graphene nano-strips taken into account. It makes predictions consistent excellently with direct MD simulations without any fitting parameter required. The results are of importance for various applications of multi-layer graphene nano-strips, such as in nano-electromechanical devices including resonators, sensors and actuators, where interlayer shear has apparent impacts to deformation, vibration, and energy dissipation processes.

cond-mat.mes-hall↗

arXiv2010

Frictional scattering and frictional waveguides: achieving persistent superlubricity at high velocity on the nanoscale

Nanomechanical devices can operate at much higher speeds than their macroscopic analogues, due to low inertia. For example, peak speeds >100m/s have been predicted for carbon nanotube devices. This stimulates our interest in the atomic-scale physics of friction at high velocity. Here we study a model nanosystem consisting of a graphene flake moving freely on a graphite substrate at >100m/s. Using molecular dynamics we discover that ultra-low friction, or superlubricity, is punctuated by high-friction transients as the flake rotates through successive crystallographic alignments with the substrate. We term this phenomenon frictional scattering and show that it is mathematically analogous to Bragg scattering. We also show that frictional scattering can be eliminated by using graphitic nanoribbons as frictional waveguides to constrain the flake rotation, thus achieving persistent superlubricity. Finally, we propose an experimental method to study nanoscale high-velocity friction. These results may guide the design of efficient high-frequency nanomechanical devices.

cond-mat.mes-hall↗