TSNLP - Test Suites for Natural Language Processing

Sabine Lehmann; Stephan Oepen; Sylvie Regnier-Prost; Klaus Netter; Veronika Lux; Judith Klein; Kirsten Falkedal; Frederik Fouvry; Dominique Estival; Eva Dauphin; Herve Compagnion; Judith Baur; Lorna Balkan; Doug Arnold

arXiv · cmp-lg/9607018

TSNLP - Test Suites for Natural Language Processing

Sabine Lehmann·Stephan Oepen·Sylvie Regnier-Prost·Klaus Netter·Veronika Lux·Judith Klein·Kirsten Falkedal·Frederik Fouvry·Dominique Estival·Eva Dauphin·Herve Compagnion·Judith Baur·Lorna Balkan·Doug Arnold

View original source ↗Read source document ↗Export metadata ↓

Abstract

The TSNLP project has investigated various aspects of the construction, maintenance and application of systematic test suites as diagnostic and evaluation tools for NLP applications. The paper summarizes the motivation and main results of the project: besides the solid methodological foundation, TSNLP has produced substantial multi-purpose and multi-user test suites for three European languages together with a set of specialized tools that facilitate the construction, extension, maintenance, retrieval, and customization of the test data. As TSNLP results, including the data and technology, are made publicly available, the project presents a valuable linguistic resourc e that has the potential of providing a wide-spread pre-standard diagnostic and evaluation tool for both developers and users of NLP applications.

Explore related subjects

cmp-lg cs.CL

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Sabine Lehmann, Stephan Oepen, Sylvie Regnier-Prost, Klaus Netter, Veronika Lux, Judith Klein, Kirsten Falkedal, Frederik Fouvry, Dominique Estival, Eva Dauphin, Herve Compagnion, Judith Baur, Lorna Balkan, Doug Arnold. 1996-07-15. TSNLP - Test Suites for Natural Language Processing. https://arxiv.org/abs/cmp-lg/9607018

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv1999

A Memory-Based Approach to Learning Shallow Natural Language Patterns

Recognizing shallow linguistic patterns, such as basic syntactic relationships between words, is a common task in applied natural language and text processing. The common practice for approaching this task is by tedious manual definition of possible pattern structures, often in the form of regular expressions or finite automata. This paper presents a novel memory-based learning method that recognizes shallow patterns in new text based on a bracketed training corpus. The training data are stored as-is, in efficient suffix-tree data structures. Generalization is performed on-line at recognition time by comparing subsequences of the new text to positive and negative evidence in the corpus. This way, no information in the training is lost, as can happen in other learning systems that construct a single generalized model at the time of training. The paper presents experimental results for recognizing noun phrase, subject-verb and verb-object patterns in English. Since the learning approach enables easy porting to new domains, we plan to apply it to syntactic patterns in other languages and to sub-language patterns for information extraction.

cmp-lg↗

arXiv1998

A Comparison of WordNet and Roget's Taxonomy for Measuring Semantic Similarity

This paper presents the results of using Roget's International Thesaurus as the taxonomy in a semantic similarity measurement task. Four similarity metrics were taken from the literature and applied to Roget's The experimental evaluation suggests that the traditional edge counting approach does surprisingly well (a correlation of r=0.88 with a benchmark set of human similarity judgements, with an upper bound of r=0.90 for human subjects performing the same task.)

cmp-lg↗

arXiv1998

Some Ontological Principles for Designing Upper Level Lexical Resources

The purpose of this paper is to explore some semantic problems related to the use of linguistic ontologies in information systems, and to suggest some organizing principles aimed to solve such problems. The taxonomic structure of current ontologies is unfortunately quite complicated and hard to understand, especially for what concerns the upper levels. I will focus here on the problem of ISA overloading, which I believe is the main responsible of these difficulties. To this purpose, I will carefully analyze the ontological nature of the categories used in current upper-level structures, considering the necessity of splitting them according to more subtle distinctions or the opportunity of excluding them because of their limited organizational role.

cmp-lg↗