the Lexical Data Masterclass is back!

Co-organized by DARIAH-EU, the Berlin Brandenburg Academy of Sciences (BBAW), Inria and the Belgrade Center for Digital Humanities, with the support of the French Ministry of Higher Education and Research (MESRI), CLARIN and the European Lexicographic Infrastructure (ELEXIS), the 2018 edition of the Lexical Data Masterclass will take place in Berlin at the BBAW from 3 to 7 December.

LexMC2018 will bring together 20 advanced trainees together with experts to share experiences, methods and techniques for the creation, management and use of lexical data.

The masterclass will cover a number of topics ranging from general models for lexical content and TEI-based representation of lexical data to working efficiently with XML editors. The participants will have a chance to attend different sessions, consult with experts on their own dictionary projects and get to know and test TEI Lex-0, a newly proposed baseline encoding for lexicographic data.

The masterclass will also feature keynotes by Karlheinz Moerth, Director of the Austrian Centre for Digital Humanities at the Austrian Academy of Sciences and Benoit Sagot, head of the ALMAnaCH research team at Inria.

Potential applicants should submit a short proposal presenting their background and interest in the field together with a description of a concrete project involving lexical data that they would like to pursue during the masterclass.

Participation is free of charge. Travel costs and accommodation will be covered for all participants up to a maximum of 600€.

Applications should be made via the Lexical Master Class website: https://lexmc18.sciencesconf.org

Application deadline: 26th October 2018
Notification of acceptance: 5th November 2018

Further inquiries can be made to: lexmc18@sciencesconf.org

For an overview of the last year’s masterclass, see https://digilex.hypotheses.org/386.

The Lexical Data Masterclass – An Overview

From 4 to 8 December 2017, 21 participants met together with 8 trainers and 2 keynote speakers to work jointly and improve their digital dictionary projects.

The meeting, co-organized by the Centre Marc Bloch, DARIAH-EU, the Berlin Brandenburg Academy of Sciences (BBAW), Inria (Paris, France) and the Belgrade Center for Digital Humanities (BCDH, Serbia), with the support of the German Ministry of Education and Research (BMBF), CLARIN, DARIAH-DE and the EU H2020 project Humanities at Scale (HaS) has been construed as a master class, i.e. a series of training and working sessions where most of the knowledge transfer is issued through the concrete work on the participants’ projects. We want to reflect here on what everyone has obviously considered as a very successful meeting by providing an overview on the instructional sessions and the actual projects that been brought by the participants as reflected in the final symposium that took place on 8th December.  Continue reading The Lexical Data Masterclass – An Overview

Simple XSLT experiment: building up a concordance dictionary from a corpus

 

During the lexical master class in Berlin, we worked together with Simonetta Battista and Ellert Johannsson on the Dictionary of Old Norse Prose. One of the little exercises we have done was to use XSLT in order to generate lexical entries automatically from an existing annotated textual corpus of Old Norse.

The corpus

The starting point is a corpus encoded according to the TEI guidelines done to the level of token. The core unit in the corpus is the sentence (available under /TEI/text/body/div/p):

<s><w lemma="ljúka" type="sfg3en">Lýkur</w><w lemma="hér" type="aa">hér</w><w lemma="saga" type="nveo">sögu</w><w lemma="grettir" type="nkee-s">Grettis</w><w lemma="ásmundarsonar" type="nkee-s">Ásmundarsonar</w><pc>,</pc><w lemma="vor" type="fekee">vors</w><w lemma="samlandi" type="nkfe">samlanda</w><pc>.</pc></s>

where each token (<w>) is associated to a reference lemma (@lemma attribute).

The task at hand

The objective is, for each lemma occurring in the annotated corpus, to create a TEI lexical entry that groups together all sentences from the corpus. Going one step further, we have create a specific TEI document for each entry.

The XSLT stylesheet

Although quite simple, the stylesheet contains several XSLT techniques that we describe in the forthcoming sections.

Architecture of the stylesheet and first variables

The stylesheet relies on a a single template associate to the root element of our corpus:

<xsl:template match="/">…</xsl:template>

We first create the following variables (by mean of the XSLT element <xsl:variable/>):

  • theRoot: stores the root element of the source document so that it can be used later in the loop on lemmas, which looses track of the context node;
<xsl:variable name="theRoot" select="."/>
  • folderName: the name of the directory where we will store all the dictionary entry documents ;
  • allLemmas: that contains the sequence of all the different lemmas encountered in the corpus by means of a concise XPath instruction:
<xsl:variable name="allLemmas" select="distinct-values(descendant::w/@lemma)"/>

The XPath expression operates in two steps:

  • descendant::w/@lemma: extracts all lemma attributes from all words which are descendant of the current node (i.e. the root of our document in the current template) ;
  • the powerful XPath function distinct-values() that creates a sequence of all distinct values occurring in the argument sequence.

Creating each entry

The process of creating entries is based upon a main loop iterating on the different lemmas, within which we use <xsl:result-document/> to create an ouput document for each lemma.

<xsl:for-each select="$allLemmas">
   <xsl:sort/>
   <xsl:result-document href="{$folderName}/{.}.xml" method="xml">
   …
   </xsl :result-document>
</xsl:for-each>

We have left here a sorting instruction (<xsl:sort/>) as an illustration on how to use it, knowing that it is less useful when entries are created in separate files.

Note the use of the {} notation in attributes to resolve the value of specific XPath fragments which, put together, builds up the name of the output file.

For each entry, we generate:

  • a <form> element whose <orth> child element contains the lemma
  • a series of <cit> elements for each detected sentence in the corpus (see below)
<entry>
   <form type="lemma">
      <orth>
         <xsl:value-of select="."/>
      </orth>
   </form>
   <xsl:for-each select="$theRoot/descendant::w[@lemma = current()]">
      <cit>
         <quote>
            <xsl:copy-of select="parent::s"/>
         </quote>
      </cit>
   </xsl:for-each>
</entry>

The XPath expression “$theRoot/descendant::w[@lemma = current()] reads: find all the <w> elements descendent of the root element (of the corpus) whose @lemma attribute equals the current context node (i.e. the text value of the lemma from the encompassing <xsl:for-each/>). Note that the example sentence is simply obtained by looking for the parent <s> element of the current <w>.

The full XSLT

Here below, you can find the full XSLT, where you will notice how the header is built up in taking up some parts of the input corpus.

The full set of resources can be found in the DARIAH lexical working group GitHub space (note the @xpath-default-namespace attribute which ensures that all generated element are set in the TEI namespace):

<?xml version="1.0" encoding="UTF-8"?>

<xsl:stylesheet
   xmlns:xsl="http://www.w3.org/1999/XSL/Transform"    
   version="2.0" xmlns="http://www.tei-c.org/ns/1.0" 
   xpath-default-namespace="http://www.tei-c.org/ns/1.0">

<xsl:output method="xml" indent="yes"/>

<xsl:template match="/">
   <xsl:variable name="theRoot" select="."/>
   <xsl:variable name="folderName" select="'Entries'"/>
   <xsl:variable name="allLemmas"
      select="distinct-values(descendant::w/@lemma)"/>
   <xsl:for-each select="$allLemmas">
      <xsl:sort/>
      <xsl:result-document href="{$folderName}/{.}.xml" method="xml">
         <TEI>
            <teiHeader>
               <fileDesc>
                  <titleStmt>
                     <title>Concordance lexical entry for lemma: 
                        <xsl:value-of select="."/>
                     </title>
                  </titleStmt>
                  <publicationStmt>
                      <xsl:copy-of select="$theRoot/descendant::publicationStmt/distributor"/>
                      <xsl:copy-of select="$theRoot/descendant::publicationStmt/address"/>
                      <xsl:copy-of select="$theRoot/descendant::publicationStmt/availability"/>
                 </publicationStmt>
                 <sourceDesc>
                    <bibl>
                       <xsl:copy-of select="$theRoot/descendant::titleStmt/title"/>
                   </bibl>
                 </sourceDesc>
              </fileDesc>
            </teiHeader>
            <text>
               <body>
                  <entry>
                     <form type="lemma">
                        <orth>
                           <xsl:value-of select="."/>
                        </orth>
                     </form>
                     <xsl:for-each select="$theRoot/descendant::w[@lemma = current()]">
                        <cit>
                           <quote>
                              <xsl:copy-of select="parent::s"/>
                           </quote>
                        </cit>
                     </xsl:for-each>
                  </entry>
               </body>
            </text>
         </TEI>
      </xsl:result-document>
   </xsl:for-each>
</xsl:template>
</xsl:stylesheet>