arXiv subjects

Yixin Liu

Publications and source records attributed to Yixin Liu.

160 records · Page 9Linked to original sources

arXiv2022

Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detection

Anomaly detection from graph data has drawn much attention due to its practical significance in many critical applications including cybersecurity, finance, and social networks. Existing data mining and machine learning methods are either shallow methods that could not effectively capture the complex interdependency of graph data or graph autoencoder methods that could not fully exploit the contextual information as supervision signals for effective anomaly detection. To overcome these challenges, in this paper, we propose a novel method, Self-Supervised Learning for Graph Anomaly Detection (SL-GAD). Our method constructs different contextual subgraphs (views) based on a target node and employs two modules, generative attribute regression and multi-view contrastive learning for anomaly detection. While the generative attribute regression module allows us to capture the anomalies in the attribute space, the multi-view contrastive learning module can exploit richer structure information from multiple subgraphs, thus abling to capture the anomalies in the structure space, mixing of structure, and attribute information. We conduct extensive experiments on six benchmark datasets and the results demonstrate that our method outperforms state-of-the-art methods by a large margin.

cs.LG↗

arXiv2022

Towards Unsupervised Deep Graph Structure Learning

In recent years, graph neural networks (GNNs) have emerged as a successful tool in a variety of graph-related applications. However, the performance of GNNs can be deteriorated when noisy connections occur in the original graph structures; besides, the dependence on explicit structures prevents GNNs from being applied to general unstructured scenarios. To address these issues, recently emerged deep graph structure learning (GSL) methods propose to jointly optimize the graph structure along with GNN under the supervision of a node classification task. Nonetheless, these methods focus on a supervised learning scenario, which leads to several problems, i.e., the reliance on labels, the bias of edge distribution, and the limitation on application tasks. In this paper, we propose a more practical GSL paradigm, unsupervised graph structure learning, where the learned graph topology is optimized by data itself without any external guidance (i.e., labels). To solve the unsupervised GSL problem, we propose a novel StrUcture Bootstrapping contrastive LearnIng fraMEwork (SUBLIME for abbreviation) with the aid of self-supervised contrastive learning. Specifically, we generate a learning target from the original data as an "anchor graph", and use a contrastive loss to maximize the agreement between the anchor graph and the learned graph. To provide persistent guidance, we design a novel bootstrapping mechanism that upgrades the anchor graph with learned structures during model learning. We also design a series of graph learners and post-processing schemes to model the structures to learn. Extensive experiments on eight benchmark datasets demonstrate the significant effectiveness of our proposed SUBLIME and high quality of the optimized graphs.

cs.LG↗

arXiv2021

CLICKER: A Computational LInguistics Classification Scheme for Educational Resources

A classification scheme of a scientific subject gives an overview of its body of knowledge. It can also be used to facilitate access to research articles and other materials related to the subject. For example, the ACM Computing Classification System (CCS) is used in the ACM Digital Library search interface and also for indexing computer science papers. We observed that a comprehensive classification system like CCS or Mathematics Subject Classification (MSC) does not exist for Computational Linguistics (CL) and Natural Language Processing (NLP). We propose a classification scheme -- CLICKER for CL/NLP based on the analysis of online lectures from 77 university courses on this subject. The currently proposed taxonomy includes 334 topics and focuses on educational aspects of CL/NLP; it is based primarily, but not exclusively, on lecture notes from NLP courses. We discuss how such a taxonomy can help in various real-world applications, including tutoring platforms, resource retrieval, resource recommendation, prerequisite chain learning, and survey generation.

cs.CL↗

arXiv2021

Anomaly Detection in Dynamic Graphs via Transformer

Detecting anomalies for dynamic graphs has drawn increasing attention due to their wide applications in social networks, e-commerce, and cybersecurity. Recent deep learning-based approaches have shown promising results over shallow methods. However, they fail to address two core challenges of anomaly detection in dynamic graphs: the lack of informative encoding for unattributed nodes and the difficulty of learning discriminate knowledge from coupled spatial-temporal dynamic graphs. To overcome these challenges, in this paper, we present a novel Transformer-based Anomaly Detection framework for DYnamic graphs (TADDY). Our framework constructs a comprehensive node encoding strategy to better represent each node's structural and temporal roles in an evolving graphs stream. Meanwhile, TADDY captures informative representation from dynamic graphs with coupled spatial-temporal patterns via a dynamic graph transformer model. The extensive experimental results demonstrate that our proposed TADDY framework outperforms the state-of-the-art methods by a large margin on six real-world datasets.

cs.LG↗

arXiv2021

Distribution Locational Marginal Pricing Under Uncertainty Considering Coordination of Distribution and Wholesale Markets

An effective distribution electricity market (DEM) is required to manage the rapidly growing small-scale distributed energy resources (DERs) in distribution systems (DSs). This paper proposes a day-ahead DEM clearing and pricing mechanism to account for the uncertainty of DERs and the coordination with the wholesale electricity market (WEM) through a bi-level model. The upper-level model clears the WEM in the transmission system (TS) and forms the locational marginal price (LMP) and uncertainty LMP (ULMP) for energy and uncertainty/reserve, respectively. In the lower level, a robust scheduling model considering WEM-DEM coordination and uncertainties is proposed to clear the DEM. Accordingly, the distribution LMPs (DLMPs) for active power, reactive power and uncertainty/reserve are derived to reward the energy/reserve provision and charge uncertain resources in the DEM, which provide effective price signals for managing not only the voltage and congestion, but also the uncertainty in DSs. A heterogeneous decomposition (HGD) algorithm is utilized to solve the bi-level model in a decentralized manner with limited information interaction between TS and DSs, which guarantees the solution efficiency and information privacy. The effectiveness of the proposed method is verified via numerous case studies.

eess.SY↗

arXiv2021

ExplainaBoard: An Explainable Leaderboard for NLP

With the rapid development of NLP research, leaderboards have emerged as one tool to track the performance of various systems on various NLP tasks. They are effective in this goal to some extent, but generally present a rather simplistic one-dimensional view of the submitted systems, communicated only through holistic accuracy numbers. In this paper, we present a new conceptualization and implementation of NLP evaluation: the ExplainaBoard, which in addition to inheriting the functionality of the standard leaderboard, also allows researchers to (i) diagnose strengths and weaknesses of a single system (e.g.~what is the best-performing system bad at?) (ii) interpret relationships between multiple systems. (e.g.~where does system A outperform system B? What if we combine systems A, B, and C?) and (iii) examine prediction results closely (e.g.~what are common errors made by multiple systems, or in what contexts do particular errors occur?). So far, ExplainaBoard covers more than 400 systems, 50 datasets, 40 languages, and 12 tasks. ExplainaBoard keeps updated and is recently upgraded by supporting (1) multilingual multi-task benchmark, (2) meta-evaluation, and (3) more complicated task: machine translation, which reviewers also suggested.} We not only released an online platform on the website \url{http://explainaboard.nlpedia.ai/} but also make our evaluation tool an API with MIT Licence at Github \url{https://github.com/neulab/explainaBoard} and PyPi \url{https://pypi.org/project/interpret-eval/} that allows users to conveniently assess their models offline. We additionally release all output files from systems that we have run or collected to motivate "output-driven" research in the future.

cs.CL↗

arXiv2021

Priority prediction of Asian Hornet sighting report using machine learning methods

As infamous invaders to the North American ecosystem, the Asian giant hornet (Vespa mandarinia) is devastating not only to native bee colonies, but also to local apiculture. One of the most effective way to combat the harmful species is to locate and destroy their nests. By mobilizing the public to actively report possible sightings of the Asian giant hornet, the governmentcould timely send inspectors to confirm and possibly destroy the nests. However, such confirmation requires lab expertise, where manually checking the reports one by one is extremely consuming of human resources. Further given the limited knowledge of the public about the Asian giant hornet and the randomness of report submission, only few of the numerous reports proved positive, i.e. existing nests. How to classify or prioritize the reports efficiently and automatically, so as to determine the dispatch of personnel, is of great significance to the control of the Asian giant hornet. In this paper, we propose a method to predict the priority of sighting reports based on machine learning. We model the problem of optimal prioritization of sighting reports as a problem of classification and prediction. We extracted a variety of rich features in the report: location, time, image(s), and textual description. Based on these characteristics, we propose a classification model based on logistic regression to predict the credibility of a certain report. Furthermore, our model quantifies the impact between reports to get the priority ranking of the reports. Extensive experiments on the public dataset from the WSDA (the Washington State Department of Agriculture) have proved the effectiveness of our method.

cs.CL↗

arXiv2021

SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization

In this paper, we present a conceptually simple while empirically powerful framework for abstractive summarization, SimCLS, which can bridge the gap between the learning objective and evaluation metrics resulting from the currently dominated sequence-to-sequence learning framework by formulating text generation as a reference-free evaluation problem (i.e., quality estimation) assisted by contrastive learning. Experimental results show that, with minor modification over existing top-scoring systems, SimCLS can improve the performance of existing top-performing models by a large margin. Particularly, 2.51 absolute improvement against BART and 2.50 over PEGASUS w.r.t ROUGE-1 on the CNN/DailyMail dataset, driving the state-of-the-art performance to a new level. We have open-sourced our codes and results: https://github.com/yixinL7/SimCLS. Results of our proposed models have been deployed into ExplainaBoard platform, which allows researchers to understand our systems in a more fine-grained way.

cs.CL↗

arXiv2021

Enhancement of the superconductivity and quantum metallic state in the thin film of superconducting Kagome metal KV$_3$Sb$_5$

Recently V-based Kagome metal attracted intense attention due to the emergence of superconductivity in the low temperature. Here we report the fabrication and physical investigations of the high quality single-crystalline thin films of the Kagome metal KV$_3$Sb$_5$. For the sample with the thickness of about 15 nm, the temperature dependent resistance reveals a Berezinskii-Kosterlitz-Thouless (BKT) type behavior, indicating the presence of two-dimensional superconductivity. Compared with the bulk sample, the onset transition temperature $T^{onset}_{c}$ and the out-of-plane upper critical field $H_{c2}$ are enhanced by 15\% and more than 10 times respectively. Moreover, the zero-resistance state is destroyed by a magnetic field as low as 50 Oe. Meanwhile, the temperature-independent resistance is observed in a wide field region, which is the hallmark of quantum metallic state. Our results provide evidences for the existence of unconventional superconductivity in this material.

cond-mat.supr-con↗

arXiv2021

On Learning Text Style Transfer with Direct Rewards

In most cases, the lack of parallel corpora makes it impossible to directly train supervised models for the text style transfer task. In this paper, we explore training algorithms that instead optimize reward functions that explicitly consider different aspects of the style-transferred outputs. In particular, we leverage semantic similarity metrics originally used for fine-tuning neural machine translation models to explicitly assess the preservation of content between system outputs and input texts. We also investigate the potential weaknesses of the existing automatic metrics and propose efficient strategies of using these metrics for training. The experimental results show that our model provides significant gains in both automatic and human evaluation over strong baselines, indicating the effectiveness of our proposed methods and training strategies.

cs.CL↗

arXiv2021

Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning

Anomaly detection on attributed networks attracts considerable research interests due to wide applications of attributed networks in modeling a wide range of complex systems. Recently, the deep learning-based anomaly detection methods have shown promising results over shallow approaches, especially on networks with high-dimensional attributes and complex structures. However, existing approaches, which employ graph autoencoder as their backbone, do not fully exploit the rich information of the network, resulting in suboptimal performance. Furthermore, these methods do not directly target anomaly detection in their learning objective and fail to scale to large networks due to the full graph training mechanism. To overcome these limitations, in this paper, we present a novel contrastive self-supervised learning framework for anomaly detection on attributed networks. Our framework fully exploits the local information from network data by sampling a novel type of contrastive instance pair, which can capture the relationship between each node and its neighboring substructure in an unsupervised way. Meanwhile, a well-designed graph neural network-based contrastive learning model is proposed to learn informative embedding from high-dimensional attributes and local structure and measure the agreement of each instance pairs with its outputted scores. The multi-round predicted scores by the contrastive learning model are further used to evaluate the abnormality of each node with statistical estimation. In this way, the learning model is trained by a specific anomaly detection-aware target. Furthermore, since the input of the graph neural network module is batches of instance pairs instead of the full network, our framework can adapt to large networks flexibly. Experimental results show that our proposed framework outperforms the state-of-the-art baseline methods on all seven benchmark datasets.

cs.LG↗

arXiv2021

RefSum: Refactoring Neural Summarization

Although some recent works show potential complementarity among different state-of-the-art systems, few works try to investigate this problem in text summarization. Researchers in other areas commonly refer to the techniques of reranking or stacking to approach this problem. In this work, we highlight several limitations of previous methods, which motivates us to present a new framework Refactor that provides a unified view of text summarization and summaries combination. Experimentally, we perform a comprehensive evaluation that involves twenty-two base systems, four datasets, and three different application scenarios. Besides new state-of-the-art results on CNN/DailyMail dataset (46.18 ROUGE-1), we also elaborate on how our proposed method addresses the limitations of the traditional methods and the effectiveness of the Refactor model sheds light on insight for performance improvement. Our system can be directly used by other researchers as an off-the-shelf tool to achieve further performance improvements. We open-source all the code and provide a convenient interface to use it: https://github.com/yixinL7/Refactoring-Summarization. We have also made the demo of this work available at: http://explainaboard.nlpedia.ai/leaderboard/task-summ/index.php.

cs.CL↗

arXiv2020

A Locational Marginal Pricing Mechanism for Uncertainty Management Based on Improved Multi-Ellipsoidal Uncertainty Set

Large-scale integration of renewable energy sources (RES) brings huge challenges to the power system. A cost-effective reserve deployment and uncertainty pricing mechanism are critical to deal with the uncertainty and variability of RES. To this end, this paper proposes a novel locational marginal pricing mechanism in day-ahead market for managing uncertainties from RES. Firstly, an improved multi-ellipsoidal uncertainty set (IMEUS) considering the temporal correlation and conditional correlation of wind power forecast is formulated to better capture the uncertainty of wind power. The dimension of each ellipsoidal subset is optimized based on a comprehensive evaluation index to reduce the invalid region without large loss of modeling accuracy, so as to reduce the conservatism. Then, an IMEUS-based robust unit commitment (RUC) model and a robust economic dispatch (RED) model are established for the day-ahead market clearing. Both the reserve cost and ramping constraints are considered in the overall dispatch process. Furthermore, based on the Langrangian function of the RED model, a new locational marginal pricing mechanism is developed. The uncertainty locational marginal price (ULMP) is introduced to charge the RES for its uncertainties and reward the generators who provide reserve to mitigate uncertainties. The new pricing mechanism can provide effective price signals to incentivize the uncertainty management in the day-ahead market. Finally, the effectiveness of the proposed methods is verified via numerous simulations on the PJM 5-bus system and IEEE 118-bus system.

eess.SY↗

arXiv2020

Cyclic Label Propagation for Graph Semi-supervised Learning

Graph neural networks (GNNs) have emerged as effective approaches for graph analysis, especially in the scenario of semi-supervised learning. Despite its success, GNN often suffers from over-smoothing and over-fitting problems, which affects its performance on node classification tasks. We analyze that an alternative method, the label propagation algorithm (LPA), avoids the aforementioned problems thus it is a promising choice for graph semi-supervised learning. Nevertheless, the intrinsic limitations of LPA on feature exploitation and relation modeling make propagating labels become less effective. To overcome these limitations, we introduce a novel framework for graph semi-supervised learning termed as Cyclic Label Propagation (CycProp for abbreviation), which integrates GNNs into the process of label propagation in a cyclic and mutually reinforcing manner to exploit the advantages of both GNNs and LPA. In particular, our proposed CycProp updates the node embeddings learned by GNN module with the augmented information by label propagation, while fine-tunes the weighted graph of label propagation with the help of node embedding in turn. After the model converges, reliably predicted labels and informative node embeddings are obtained with the LPA and GNN modules respectively. Extensive experiments on various real-world datasets are conducted, and the experimental results empirically demonstrate that the proposed CycProp model can achieve relatively significant gains over the state-of-the-art methods.

cs.LG↗

arXiv2020

Conditional Automated Channel Pruning for Deep Neural Networks

Model compression aims to reduce the redundancy of deep networks to obtain compact models. Recently, channel pruning has become one of the predominant compression methods to deploy deep models on resource-constrained devices. Most channel pruning methods often use a fixed compression rate for all the layers of the model, which, however, may not be optimal. To address this issue, given a target compression rate for the whole model, one can search for the optimal compression rate for each layer. Nevertheless, these methods perform channel pruning for a specific target compression rate. When we consider multiple compression rates, they have to repeat the channel pruning process multiple times, which is very inefficient yet unnecessary. To address this issue, we propose a Conditional Automated Channel Pruning(CACP) method to obtain the compressed models with different compression rates through single channel pruning process. To this end, we develop a conditional model that takes an arbitrary compression rate as input and outputs the corresponding compressed model. In the experiments, the resultant models with different compression rates consistently outperform the models compressed by existing methods with a channel pruning process for each target compression rate.

cs.CV↗

arXiv2014

Design of an Encryption-Decryption Module Oriented for Internet Information Security SOC Design

In order to protect the security of network data, a high speed chip module for encrypting and decrypting of network data packet is designed. The chip module is oriented for internet information security SOC (System on Chip) design. During the design process, AES (Advanced Encryption Standard) and 3DES (Data Encryption Standard) encryption algorithm are adopted to protect the security of network data. The following points are focused: (1) The SOC (System on Chip) design methodology based on IP (Intellectual Property) core is used. AES (Advanced Encryption Standard) and 3DES (Data Encryption Standard) IP (Intellectual Property) cores are embedded in the chip module, peripheral control sub-modules are designed to control the encryption-decryption module, which is capable of shortening the design period of the chip module. (2) The implementation of encryption-decryption with hardware was presented, which improves the safety of data through the encryption-decryption chip and reduce the load of CPU. (3) In our hardware solution, two AES (Advanced Encryption Standard) cores are used to work in parallel, which improves the speed of the encryption module. Moreover, the key length of AES (Advanced Encryption Standard) encryption algorithm is designed with three optional configurations at 128 bits, 256 bits and 192 bits respectively and six optional encryption algorithm modes: CBC (Cipher Block Chaining) mode, ECB (Electronic Code Book) mode, GCM (Galois/Counter Mode) mode, XTS(cipherteXT Stealing) mode, CTR (CounTeR) mode and 3DES respectively, which adds the flexibility to its applications.

cs.AR↗