arXiv subjects

Yue Yu

Publications and source records attributed to Yue Yu.

At least 397 records · Page 22Linked to original sources

arXiv2021

DAGs with No Curl: An Efficient DAG Structure Learning Approach

Recently directed acyclic graph (DAG) structure learning is formulated as a constrained continuous optimization problem with continuous acyclicity constraints and was solved iteratively through subproblem optimization. To further improve efficiency, we propose a novel learning framework to model and learn the weighted adjacency matrices in the DAG space directly. Specifically, we first show that the set of weighted adjacency matrices of DAGs are equivalent to the set of weighted gradients of graph potential functions, and one may perform structure learning by searching in this equivalent set of DAGs. To instantiate this idea, we propose a new algorithm, DAG-NoCurl, which solves the optimization problem efficiently with a two-step procedure: 1) first we find an initial cyclic solution to the optimization problem, and 2) then we employ the Hodge decomposition of graphs and learn an acyclic graph by projecting the cyclic graph to the gradient of a potential function. Experimental studies on benchmark datasets demonstrate that our method provides comparable accuracy but better efficiency than baseline DAG structure learning methods on both linear and generalized structural equation models, often by more than one order of magnitude.

cs.LG↗

arXiv2021

Pull Request Decision Explained: An Empirical Overview

Context: Pull-based development model is widely used in open source, leading the trends in distributed software development. One aspect which has garnered significant attention is studies on pull request decision - identifying factors for explanation. Objective: This study builds on a decade long research on pull request decision to explain it. We empirically investigate how factors influence pull request decision and scenarios that change the influence of factors. Method: We identify factors influencing pull request decision on GitHub through a systematic literature review and infer it by mining archival data. We collect a total of 3,347,937 pull requests with 95 features from 11,230 diverse projects on GitHub. Using this data, we explore the relations of the factors to each other and build mixed-effect logistic regression models to empirically explain pull request decision. Results: Our study shows that a small number of factors explain pull request decision with the integrator same or different from the submitter as the most important factor. We also noted that some factors are important only in special cases e.g., the percentage of failed builds is important for pull request decision when continuous integration is used.

cs.SE↗

arXiv2021

SumGNN: Multi-typed Drug Interaction Prediction via Efficient Knowledge Graph Summarization

Thanks to the increasing availability of drug-drug interactions (DDI) datasets and large biomedical knowledge graphs (KGs), accurate detection of adverse DDI using machine learning models becomes possible. However, it remains largely an open problem how to effectively utilize large and noisy biomedical KG for DDI detection. Due to its sheer size and amount of noise in KGs, it is often less beneficial to directly integrate KGs with other smaller but higher quality data (e.g., experimental data). Most of the existing approaches ignore KGs altogether. Some try to directly integrate KGs with other data via graph neural networks with limited success. Furthermore, most previous works focus on binary DDI prediction whereas the multi-typed DDI pharmacological effect prediction is a more meaningful but harder task. To fill the gaps, we propose a new method SumGNN: knowledge summarization graph neural network, which is enabled by a subgraph extraction module that can efficiently anchor on relevant subgraphs from a KG, a self-attention based subgraph summarization scheme to generate a reasoning path within the subgraph, and a multi-channel knowledge and data integration module that utilizes massive external biomedical knowledge for significantly improved multi-typed DDI predictions. SumGNN outperforms the best baseline by up to 5.54\%, and the performance gain is particularly significant in low data relation types. In addition, SumGNN provides interpretable prediction via the generated reasoning paths for each prediction.

cs.LG↗

arXiv2021

PanGu-$α$: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation

Large-scale Pretrained Language Models (PLMs) have become the new paradigm for Natural Language Processing (NLP). PLMs with hundreds of billions parameters such as GPT-3 have demonstrated strong performances on natural language understanding and generation with \textit{few-shot in-context} learning. In this work, we present our practice on training large-scale autoregressive language models named PanGu-$α$, with up to 200 billion parameters. PanGu-$α$ is developed under the MindSpore and trained on a cluster of 2048 Ascend 910 AI processors. The training parallelism strategy is implemented based on MindSpore Auto-parallel, which composes five parallelism dimensions to scale the training task to 2048 processors efficiently, including data parallelism, op-level model parallelism, pipeline model parallelism, optimizer model parallelism and rematerialization. To enhance the generalization ability of PanGu-$α$, we collect 1.1TB high-quality Chinese data from a wide range of domains to pretrain the model. We empirically test the generation ability of PanGu-$α$ in various scenarios including text summarization, question answering, dialogue generation, etc. Moreover, we investigate the effect of model scales on the few-shot performances across a broad range of Chinese NLP tasks. The experimental results demonstrate the superior capabilities of PanGu-$α$ in performing various tasks under few-shot or zero-shot settings.

cs.CL↗

arXiv2021

Size-Sieving Separation of Hard-Sphere Mixtures through Cylindrical Pores

The collision dynamics of hard spheres and cylindrical pores is solved exactly, which is the minimal model for a regularly porous membrane. Nonequilibrium event-driven molecular dynamics simulations are used to show that the permeability $P$ of hard spheres of size $σ$ through cylinderical pores of size $d$ follow the hindered diffusion mechanism due to size exclusion as $P \propto (1-σ/d)^2$. Under this law, the separation of binary mixtures of large and small particles exhibits a linear relationship between $α^{-1/2}$ and $P^{-1/2}$, where $α$ and $P$ are the selectivity and permeability of the smaller particle, respectively. The mean permeability through polydisperse pores is the sum of permeabilities of individual pores, weighted by the fraction of the single pore area over the total pore area.

cond-mat.soft↗

arXiv2021

Threshold Changeable Secret Sharing Scheme and Its Application to Group Authentication

Group oriented applications are getting more and more popular in mobile Internet and call for secure and efficient secret sharing (SS) scheme to meet their requirements. A $(t,n)$ threshold SS scheme divides a secret into $n$ shares such that any $t$ or more than $t$ shares can recover the secret while less than $t$ shares cannot. However, an adversary, even without a valid share, may obtain the secret by impersonating a shareholder to recover the secret with $t$ or more legal shareholders. Therefore, this paper uses linear code to propose a threshold changeable secret sharing (TCSS) scheme, in which threshold should increase from $t$ to the exact number of all participants during secret reconstruction. The scheme does not depend on any computational assumption and realizes asymptotically perfect security. Furthermore, based on the proposed TCSS scheme, a group authentication scheme is constructed, which allows a group user to authenticate whether all users are legal group members at once and thus provides efficient and flexible m-to-m authentication for group oriented applications.

cs.CR↗

arXiv2021

On Controllability and Persistency of Excitation in Data-Driven Control: Extensions of Willems' Fundamental Lemma

Willems' fundamental lemma asserts that all trajectories of a linear time-invariant system can be obtained from a finite number of measured ones, assuming that controllability and a persistency of excitation condition hold. We show that these two conditions can be relaxed. First, we prove that the controllability condition can be replaced by a condition on the controllable subspace, unobservable subspace, and a certain subspace associated with the measured trajectories. Second, we prove that the persistency of excitation requirement can be relaxed if the degree of a certain minimal polynomial is tightly bounded. Our results show that data-driven predictive control using online data is equivalent to model predictive control, even for uncontrollable systems. Moreover, our results significantly reduce the amount of data needed in identifying homogeneous multi-agent systems.

eess.SY↗

arXiv2021

Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach

Fine-tuned pre-trained language models (LMs) have achieved enormous success in many natural language processing (NLP) tasks, but they still require excessive labeled data in the fine-tuning stage. We study the problem of fine-tuning pre-trained LMs using only weak supervision, without any labeled data. This problem is challenging because the high capacity of LMs makes them prone to overfitting the noisy labels generated by weak supervision. To address this problem, we develop a contrastive self-training framework, COSINE, to enable fine-tuning LMs with weak supervision. Underpinned by contrastive regularization and confidence-based reweighting, this contrastive self-training framework can gradually improve model fitting while effectively suppressing error propagation. Experiments on sequence, token, and sentence pair classification tasks show that our model outperforms the strongest baseline by large margins on 7 benchmarks in 6 tasks, and achieves competitive performance with fully-supervised fine-tuning methods.

cs.CL↗

arXiv2021

Implementation of Polygonal Mesh Refinement in MATLAB

We present a simple and efficient MATLAB implementation of the local refinement for polygonal meshes. The purpose of this implementation is primarily educational, especially on the teaching of adaptive virtual element methods.

math.NA↗

arXiv2021

An asymptotically compatible treatment of traction loading in linearly elastic peridynamic fracture

Meshfree discretizations of state-based peridynamic models are attractive due to their ability to naturally describe fracture of general materials. However, two factors conspire to prevent meshfree discretizations of state-based peridynamics from converging to corresponding local solutions as resolution is increased: quadrature error prevents an accurate prediction of bulk mechanics, and the lack of an explicit boundary representation presents challenges when applying traction loads. In this paper, we develop a reformulation of the linear peridynamic solid (LPS) model to address these shortcomings, using improved meshfree quadrature, a reformulation of the nonlocal dilitation, and a consistent handling of the nonlocal traction condition to construct a model with rigorous accuracy guarantees. In particular, these improvements are designed to enforce discrete consistency in the presence of evolving fractures, whose {\it a priori} unknown location render consistent treatment difficult. In the absence of fracture, when a corresponding classical continuum mechanics model exists, our improvements provide asymptotically compatible convergence to corresponding local solutions, eliminating surface effects and issues with traction loading which have historically plagued peridynamic discretizations. When fracture occurs, our formulation automatically provides a sharp representation of the fracture surface by breaking bonds, avoiding the loss of mass. We provide rigorous error analysis and demonstrate convergence for a number of benchmarks, including manufactured solutions, free-surface, nonhomogeneous traction loading, and composite material problems. Finally, we validate simulations of brittle fracture against a recent experiment of dynamic crack branching in soda-lime glass, providing evidence that the scheme yields accurate predictions for practical engineering problems.

math.NA↗

arXiv2020

Proportional-Integral Projected Gradient Method for Model Predictive Control

Recently there has been an increasing interest in primal-dual methods for model predictive control (MPC), which require minimizing the (augmented) Lagrangian at each iteration. We propose a novel first order primal-dual method, termed \emph{proportional-integral projected gradient method}, for MPC where the underlying finite horizon optimal control problem has both state and input constraints. Instead of minimizing the (augmented) Lagrangian, each iteration of our method only computes a single projection onto the state and input constraint set. Our method ensures that, along a sequence of averaged iterates, both the distance to optimum and the constraint violation converge to zero at a rate of $O(1/k)$ if the objective function is convex, where $k$ is the iteration number. If the objective function is strongly convex, this rate can be improved to $O(1/k^2)$ for the distance to optimum and $O(1/k^3)$ for the constraint violation. We compare our method against existing methods via a trajectory-planning example with convexified keep-out-zone constraints.

math.OC↗

arXiv2020

Data-driven learning of nonlocal models: from high-fidelity simulations to constitutive laws

We show that machine learning can improve the accuracy of simulations of stress waves in one-dimensional composite materials. We propose a data-driven technique to learn nonlocal constitutive laws for stress wave propagation models. The method is an optimization-based technique in which the nonlocal kernel function is approximated via Bernstein polynomials. The kernel, including both its functional form and parameters, is derived so that when used in a nonlocal solver, it generates solutions that closely match high-fidelity data. The optimal kernel therefore acts as a homogenized nonlocal continuum model that accurately reproduces wave motion in a smaller-scale, more detailed model that can include multiple materials. We apply this technique to wave propagation within a heterogeneous bar with a periodic microstructure. Several one-dimensional numerical tests illustrate the accuracy of our algorithm. The optimal kernel is demonstrated to reproduce high-fidelity data for a composite material in applications that are substantially different from the problems used as training data.

cs.LG↗

arXiv2020

DAGs with No Fears: A Closer Look at Continuous Optimization for Learning Bayesian Networks

This paper re-examines a continuous optimization framework dubbed NOTEARS for learning Bayesian networks. We first generalize existing algebraic characterizations of acyclicity to a class of matrix polynomials. Next, focusing on a one-parameter-per-edge setting, it is shown that the Karush-Kuhn-Tucker (KKT) optimality conditions for the NOTEARS formulation cannot be satisfied except in a trivial case, which explains a behavior of the associated algorithm. We then derive the KKT conditions for an equivalent reformulation, show that they are indeed necessary, and relate them to explicit constraints that certain edges be absent from the graph. If the score function is convex, these KKT conditions are also sufficient for local minimality despite the non-convexity of the constraint. Informed by the KKT conditions, a local search post-processing algorithm is proposed and shown to substantially and universally improve the structural Hamming distance of all tested algorithms, typically by a factor of 2 or more. Some combinations with local search are both more accurate and more efficient than the original NOTEARS.

cs.LG↗

arXiv2020

MCMH: Learning Multi-Chain Multi-Hop Rules for Knowledge Graph Reasoning

Multi-hop reasoning approaches over knowledge graphs infer a missing relationship between entities with a multi-hop rule, which corresponds to a chain of relationships. We extend existing works to consider a generalized form of multi-hop rules, where each rule is a set of relation chains. To learn such generalized rules efficiently, we propose a two-step approach that first selects a small set of relation chains as a rule and then evaluates the confidence of the target relationship by jointly scoring the selected chains. A game-theoretical framework is proposed to this end to simultaneously optimize the rule selection and prediction steps. Empirical results show that our multi-chain multi-hop (MCMH) rules result in superior results compared to the standard single-chain approaches, justifying both our formulation of generalized rules and the effectiveness of the proposed learning framework.

cs.CL↗

arXiv2020

SeqMix: Augmenting Active Sequence Labeling via Sequence Mixup

Active learning is an important technique for low-resource sequence labeling tasks. However, current active sequence labeling methods use the queried samples alone in each iteration, which is an inefficient way of leveraging human annotations. We propose a simple but effective data augmentation method to improve the label efficiency of active sequence labeling. Our method, SeqMix, simply augments the queried samples by generating extra labeled sequences in each iteration. The key difficulty is to generate plausible sequences along with token-level labels. In SeqMix, we address this challenge by performing mixup for both sequences and token-level labels of the queried samples. Furthermore, we design a discriminator during sequence mixup, which judges whether the generated sequences are plausible or not. Our experiments on Named Entity Recognition and Event Detection tasks show that SeqMix can improve the standard active sequence labeling method by $2.27\%$--$3.75\%$ in terms of $F_1$ scores. The code and data for SeqMix can be found at https://github.com/rz-zhang/SeqMix

cs.CL↗

arXiv2020

BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervision

We study the open-domain named entity recognition (NER) problem under distant supervision. The distant supervision, though does not require large amounts of manual annotations, yields highly incomplete and noisy distant labels via external knowledge bases. To address this challenge, we propose a new computational framework -- BOND, which leverages the power of pre-trained language models (e.g., BERT and RoBERTa) to improve the prediction performance of NER models. Specifically, we propose a two-stage training algorithm: In the first stage, we adapt the pre-trained language model to the NER tasks using the distant labels, which can significantly improve the recall and precision; In the second stage, we drop the distant labels, and propose a self-training approach to further improve the model performance. Thorough experiments on 5 benchmark datasets demonstrate the superiority of BOND over existing distantly supervised NER methods. The code and distantly labeled data have been released in https://github.com/cliang1453/BOND.

cs.CL↗

arXiv2020

STEAM: Self-Supervised Taxonomy Expansion with Mini-Paths

Taxonomies are important knowledge ontologies that underpin numerous applications on a daily basis, but many taxonomies used in practice suffer from the low coverage issue. We study the taxonomy expansion problem, which aims to expand existing taxonomies with new concept terms. We propose a self-supervised taxonomy expansion model named STEAM, which leverages natural supervision in the existing taxonomy for expansion. To generate natural self-supervision signals, STEAM samples mini-paths from the existing taxonomy, and formulates a node attachment prediction task between anchor mini-paths and query terms. To solve the node attachment task, it learns feature representations for query-anchor pairs from multiple views and performs multi-view co-training for prediction. Extensive experiments show that STEAM outperforms state-of-the-art methods for taxonomy expansion by 11.6\% in accuracy and 7.0\% in mean reciprocal rank on three public benchmarks. The implementation of STEAM can be found at \url{https://github.com/yueyu1030/STEAM}.

cs.CL↗

arXiv2020

$T_c$ and Pauli limited critical field of $\text{Sr}_2\text{Ru}\text{O}_4$: uniaxial strain dependence

Variations of critical temperature $T_c$ and in-plane critical field $H_{c2}$ of $\text{Sr}_2\text{Ru}\text{O}_4$ under uniaxial stress have recently been reported. We compare the strain dependence of $T_c$ and $H_{c2}$ in various pairing channels ($d$-wave, extended s-wave and $p$-wave) with the experimental observations, by studying a three-band tight-binding model that includes effects of spin-orbit and Zeeman couplings and a separable pairing interaction. Our study helps narrow down the possibility of pairing channels. The importance of the multi-band nature of $\text{Sr}_2\text{Ru}\text{O}_4$ is also highlighted.

cond-mat.supr-con↗