A TEI XML Version of the Historical Thesaurus of English Database

My project during the Lexical Data Masterclass was to trial recreating a section of the Historical Thesaurus of English in TEI-compliant XML, to which I could potentially add further information not presently contained in the Thesaurus database. In particular, I’m interested in adding words which collocate with word senses throughout the lifetime of those senses.  For those unfamiliar with lexical collocation, the theory is that a word’s meaning is (at least in part) composed by the context in which it is used. This means that you form an idea of what a word means partly through the other words which are frequently used alongside it. This is not the whole story – if I say the word ‘penguin’ your brain will likely return to you a mental image of a penguin and perhaps its associated properties, not the list of words you’ve heard in sentences next to ‘penguin’! – nonetheless, collocates may well be part of how we build and understand meaning. Many linguists are busily engaged exploring whether collocation can be used to map the ways in which concepts are associated with each other.

For my project, the collocates rapidly became a secondary consideration after the challenge of taking a relational database and turning it into TEI-compliant XML. For some, this may be an hour’s work. However, my knowledge of TEI XML and XSLT was limited, so expanding it was the main purpose for me of attending LexMC.

The current TEI guidelines are exceptional in providing guidance for marking up dictionary data. They have to be twisted to some extent to fit the style of data contained in a thesaurus. Categories in the Historical Thesaurus are divided into semantic fields ranging from high-level concepts such as ‘The world’ and ‘Society’ to intricate detail such as ‘Animal body, general parts, consisting of segments’. These are arranged into a hierarchy by their relationship to each other, with the most fine-grained detail contained in ‘subcategories’ nested inside full categories. The data in each semantic field may be distributed across multiple ‘parts of speech’ (POS, i.e. noun, adjective, verb, adverb, etc.). Within each POS division are the word senses which have expressed that meaning, each accompanied by the date-range for which the sense has been attested as active. Subcategories (as opposed to the full, mainline categories) also belong to specific POS divisions.

My provisional method for encoding this data using TEI XML was to treat each semantic field, with its associated parts of speech, as an entry. These entries contained a <form> element with a ‘category’ type attribute, and this element contains the semantic field’s position in the hierarchy as defined by the category numbering system used in the Historical Thesaurus. Belonging to these fields are the POS divisions, each recorded using a <sense> element, containing a <def> for the category heading, a <gramGrp> for the POS, followed by a number of <form type=”lexeme”> entries for the word senses themselves. Each lexeme has an <orth> element giving its written form and then two <date> elements for the earliest and latest citation date. This stretches the TEI dictionary guidelines considerably, treating the category as equivalent to a dictionary headword and the lexemes inside it as ‘forms’ of the concept which the category represents. There is no objective base to a concept beyond the sequence of category headings which lead to it (which is, itself, not objective – a fascinating subject, but let’s not get into it right now!); instead, HT semantic fields exist as a positions in a hierarchy labelled with a positioning code, and so this code is perhaps the closest thing to a headword which exists for a category.

This structure could then be populated automatically using sample data from the database of the Historical Thesaurus, selected via an SQL query and output as XML. The intermediate XML form presented database rows as tables where each cell was given as a <column> element with the column heading as a name attribute. Two files were used here – one containing category details and the other lexeme details. With generous assistance from Laurent Romary, it was possible to create an XSLT script which read the tables for categories and lexemes into variables. Important details could then be plucked from these variables to create entries in the TEI XML output document. I spent more time than I would like to admit experimenting with reproducing the category numbering system via XSLT selection of the appropriate columns.

To retrieve some preliminary collocation detail for the sample categories, I used the version of Early English Books Online (EEBO-TCP, phase I) which was annotated with semantic labels by the SAMUELS project. The data is currently available in a corpus interface on Mark Davies’ corpus site and will be deposited in raw text form with the Oxford Text Archive. This was another proof-of-concept trial rather than a detailed exploration of the data, but it was successful enough to give me hope that a more complete and carefully considered version is an achievable goal.

Overall, the result of these efforts was a serviceable foundation for a TEI XML version of the Historical Thesaurus. What most interested me about the process was the requirement to think about and experiment with the structure of the data. I am still not satisfied with my representation of the resulting data layers in XML, although discussion at the LexMC closing symposium gave me further possibilities to test. Further work would also need to engage more extensively with the Thesaurus system for representing dates of lexeme activity. There is considerable complexity to this information, and for the purposes of the project I employed very basic (and thus, slightly misleading) ‘earliest’ and ‘latest’ attestation dates as a placeholder for complete activity data. There would still be a fair amount of work ahead to do this, but the Lexical Data Master Class really stretched the way I think about the data, how it can be structured, and the alternatives for its presentation.

I would like to thank the organisers and participants of the workshop for their feedback, and especially Laurent Romary, Martin Wynne, and Emrah Özcan for saving me from drowning in XSLT issues.

---

---