CORPUS TOOLS FOR TYPE-LOGICAL GRAMMARS

INTRODUCTION

This is the first public release of my suite of corpus tools for type-logical grammars. At the moment, it is a considerably less polished (and more idiosyncratic) collection of utilities than Grail 3 which the collection is designed to work with. I hope people will find it useful nonetheless.

All source code is provided under the GNU lesser general public license and downloading any of these files constitutes an agreement to all the terms of this license.

UTILITIES

The different corpus utilities are classified below according to their function.

CORPUS DISPLAY

CORPUS TRANSFORMATION

LEXICON EXTRACTION

MAXIMUM ENTROPY

These scripts perform several steps for transform the extracted lexicon into a format suitable for the maximum entropy supertagger and for evaluation the supertagger on the test data.

SUPERTAGGER

A series of Java files based on and requiring an installation of the Edinburgh maximum entropy tools. Used in coordination with the scripts above for training and evaluating the supertagger.

CORPUS SEARCH

These scripts provide some search functions I needed in order to get an idea of some of the constructions found in the corpus and the number of occurrences.

ERROR TRACKING

These scripts compare a tagged file to a correct file in order to find instances of a specific error made by the supertagger. This may help improving the maximum entropy models either by adding new features or by improving the extraction algorithm.

MODELS

DOWNLOAD

All files in one big tarball.

OTHER CORPUS RESOURCES

Richard.Moot@labri.fr