arXiv subjects

Veronika Thost

Publications and source records attributed to Veronika Thost.

26 records · Page 2Linked to original sources

arXiv2020

Exploring Software Naturalness through Neural Language Models

The Software Naturalness hypothesis argues that programming languages can be understood through the same techniques used in natural language processing. We explore this hypothesis through the use of a pre-trained transformer-based language model to perform code analysis tasks. Present approaches to code analysis depend heavily on features derived from the Abstract Syntax Tree (AST) while our transformer-based language models work on raw source code. This work is the first to investigate whether such language models can discover AST features automatically. To achieve this, we introduce a sequence labeling task that directly probes the language models understanding of AST. Our results show that transformer based language models achieve high accuracy in the AST tagging task. Furthermore, we evaluate our model on a software vulnerability identification task. Importantly, we show that our approach obtains vulnerability identification results comparable to graph based approaches that rely heavily on compilers for feature extraction.

cs.CL↗

arXiv2020

Improving Graph Neural Network Representations of Logical Formulae with Subgraph Pooling

Recent advances in the integration of deep learning with automated theorem proving have centered around the representation of logical formulae as inputs to deep learning systems. In particular, there has been a growing interest in adapting structure-aware neural methods to work with the underlying graph representations of logical expressions. While more effective than character and token-level approaches, graph-based methods have often made representational trade-offs that limited their ability to capture key structural properties of their inputs. In this work we propose a novel approach for embedding logical formulae that is designed to overcome the representational limitations of prior approaches. Our architecture works for logics of different expressivity; e.g., first-order and higher-order logic. We evaluate our approach on two standard datasets and show that the proposed architecture achieves state-of-the-art performance on both premise selection and proof step classification.

cs.AI↗

arXiv2020

Temporal Conjunctive Query Answering in the Extended DL-Lite Family

Ontology-based query answering (OBQA) augments classical query answering in databases by domain knowledge encoded in an ontology. Systems for OBQA use the ontological knowledge to infer new information that is not explicitly given in the data. Moreover, they usually employ the open-world assumption, which means that knowledge that is not stated explicitly in the data and that is not inferred is not assumed to be true or false. Classical OBQA however considers only a snapshot of the data, which means that information about the temporal evolution of the data is not used for reasoning and hence lost. We investigate temporal conjunctive queries (TCQs) that allow to access temporal data through classical ontologies. In particular, we study combined and data complexity of TCQ entailment for ontologies written in description logics from the extended DL-Lite family. Many of these logics allow for efficient reasoning in the atemporal setting and are successfully applied in practice. We show comprehensive complexity results for temporal reasoning with these logics.

cs.LO↗

arXiv2020

An Experimental Study of Formula Embeddings for Automated Theorem Proving in First-Order Logic

Automated theorem proving in first-order logic is an active research area which is successfully supported by machine learning. While there have been various proposals for encoding logical formulas into numerical vectors -- from simple strings to more involved graph-based embeddings -- little is known about how these different encodings compare. In this paper, we study and experimentally compare pattern-based embeddings that are applied in current systems with popular graph-based encodings, most of which have not been considered in the theorem proving context before. Our experiments show that the advantages of simpler encoding schemes in terms of runtime are outdone by more complex graph-based embeddings, which yield more efficient search strategies and simpler proofs. To support this, we present a detailed analysis across several dimensions of theorem prover performance beyond just proof completion rate, thus providing empirical evidence to help guide future research on neural-guided theorem proving towards the most promising directions.

cs.AI↗

arXiv2020

RuDaS: Synthetic Datasets for Rule Learning and Evaluation Tools

Logical rules are a popular knowledge representation language in many domains, representing background knowledge and encoding information that can be derived from given facts in a compact form. However, rule formulation is a complex process that requires deep domain expertise,and is further challenged by today's often large, heterogeneous, and incomplete knowledge graphs. Several approaches for learning rules automatically, given a set of input example facts,have been proposed over time, including, more recently, neural systems. Yet, the area is missing adequate datasets and evaluation approaches: existing datasets often resemble toy examples that neither cover the various kinds of dependencies between rules nor allow for testing scalability. We present a tool for generating different kinds of datasets and for evaluating rule learning systems, including new performance measures.

cs.AI↗

arXiv2019

Infusing Knowledge into the Textual Entailment Task Using Graph Convolutional Networks

Textual entailment is a fundamental task in natural language processing. Most approaches for solving the problem use only the textual content present in training data. A few approaches have shown that information from external knowledge sources like knowledge graphs (KGs) can add value, in addition to the textual content, by providing background knowledge that may be critical for a task. However, the proposed models do not fully exploit the information in the usually large and noisy KGs, and it is not clear how it can be effectively encoded to be useful for entailment. We present an approach that complements text-based entailment models with information from KGs by (1) using Personalized PageR- ank to generate contextual subgraphs with reduced noise and (2) encoding these subgraphs using graph convolutional networks to capture KG structure. Our technique extends the capability of text models exploiting structural and semantic information found in KGs. We evaluate our approach on multiple textual entailment datasets and show that the use of external knowledge helps improve prediction accuracy. This is particularly evident in the challenging BreakingNLI dataset, where we see an absolute improvement of 5-20% over multiple text-based entailment models.

cs.CL↗

arXiv2018

Query Answering for Rough EL Ontologies (Extended Technical Report)

Querying large datasets with incomplete and vague data is still a challenge. Ontology-based query answering extends standard database query answering by background knowledge from an ontology to augment incomplete data. We focus on ontologies written in rough description logics (DLs), which allow to represent vague knowledge by partitioning the domain of discourse into classes of indiscernible elements. In this paper, we extend the combined approach for ontology-based query answering to a variant of the DL EL augmented with rough concept constructors. We show that this extension preserves the good computational properties of classical EL and can be implemented by standard database systems.

cs.LO↗

arXiv2018

Metric Temporal Extensions of DL-Lite and Interval-Rigid Names

The DL-Lite description logics allow for modeling domain knowledge on top of databases and for efficient reasoning. We focus on metric temporal extensions of DL-Lite_bool and its fragments, and study the complexity of satisfiability. In particular, we investigate the influence of rigid and interval-rigid symbols, which allow for modeling knowledge that remains valid over (some) time. We show that especially the latter add considerable expressive power in many logics, but they do not always increase complexity.

cs.LO↗