When working on the Deutsches Wörterbuch, Jacob Grimm felt depressed by his life as a lexicographer. In the preface to the first volume, published in 1854, Jacob wrote:
“As if for days fine and tight flakes were falling down from the sky and soon the whole area is covered with vast snow I am quasi snowed in under the mass of words pressing towards me from all corners and crevices. Sometimes I want to get up and get rid of everything.”
Dorothea Grimm, his brother’s wife, observed Jacob and Wilhelm anxiously. They have to be liberated, otherwise they will get mouldy, she thought, watching them work on the dictionary from the early morning until late at night.
The Grimm brothers did not finish their dictionary. Between 1852 and 1961, generations of lexicographers contributed to the still largest dictionary of the German language, describing more than 300.000 headwords in 16 volumes in 32 parts, which weigh about 30 kilogrammes in total. The history of the so-called Grimm is not unique in the history of historical, legacy dictionaries. Many lexicographers produced them over a long period of time. Thus, they not only collect a great deal of knowledge about languages and the world inhabited by these languages, but they also reflect the lexicographic theories of their respective time of origin. They are – like all dictionaries – related to one another in various ways, and thus implicitly form networks of information, which are not easy to explore because of the heterogeneous dictionary structures.
When the internet era began in the early 1990s, it seemed that the new medium with its hypertextual structure would fit the needs of dictionaries like the Grimm much more so than print ever could. The question was, however, how to transfer the Grimm into digital format?
The mass of words, all over again
When a small team of digital humanists at Trier University started digitising the Grimm in 1998, they felt like Jacob Grimm must have felt about 150 years ago. 300 million printed characters with the weight of centuries seemed to weigh down on their shoulders. A mere image digitisation appeared to be the easiest way of publishing the dictionary online. That, however, would have made the dictionary simply visible and browsable online. But what about making the content accessible and searchable, what about making the existing information networks alive? Should the team try to convert images into machine-encoded text via optical character recognition (OCR)? The results of several trials were utterly disappointing. The quality of the print was not good enough. Faced with more than 67.000 columns that are densely populated with about 300.000.000 poorly printed characters, partly with diacritics, partly coming from different alphabets, the OCR software at the end of the 1990s was not up to the task. It produced so many mistakes that proofreading would have been extremely time consuming and expensive.
Thus, the team decided to produce a digital version of the dictionary via double keying: the process of having each dictionary segment manually and independently retyped by two different operators, then mutually compared for mistakes. The result satisfied the academic demand and made the encoding of the dictionary in XML much easier. Two years after the beginning of the digitisation process, the first volume of the “Grimm” was online and the dictionary gained new fans. The effort had been worthwhile. Since 2003, users can explore this treasure house of the German language online.
The digitisation of the Grimm can serve as a stimulus for retro-digitisers not to lose hope while confronted with legacy dictionary beasts made by humans and showing many inconsistencies. You can withstand the heavy snowstorm of digitising dictionaries if you are willing to take your time for the planning phase, if you never stop interdisciplinary communication, and if you study other dictionary digitisation projects and benefit from their experiences and from their mistakes.
The DigiLex blog will, among other things, provide you with the snow shovels necessary to face the challenges of retro-digitising paper dictionaries. So, let’s dig in, together!
Featured image: CC BY-NC-ND 2.0 by bored-now
One thought on “From Books to Bytes: Turning Paper Dictionaries into Digital Format”