Vision and decision: what should I do with an OCR?

As much as I like developing visions, they lead one way or another to pragmatical decisions that are not always so easy or satisfactory.

Here is the vision: you would have the list of books that Boeckh gave to the University Library in manuscript form, digitized on your screen. You would from there come through an XML/TEI-encoding to the scan of the book itself – and from there, through another XML/TEI-encoding, to the way Boeckh quoted or used this book in his work (let’s say for instance in his Encyklopädie-class or at the Philological Seminar, for which the concerning manuscripts would also be encoded and digitized). Smoothly surfing between genesis and reception: it is just beautiful.

And huge. There are massive encodings that are to be done (Julia Doborosky has been working on this, informing book entries by hundreds just as she had done with the manuscript entries last year), and there are texts to be scanned. Which ones? If Boeckh’s personal exemplar of the book listed by him is to be found (condition number one) and either if it contains interesting marginal notes by him or if there is no other scan of that book available in open-access worldwide (conditions number 2a and 2b), then the book should be scanned. At this point begins the project I am developing with the University Library at the Humboldt University. The idea is to virtually reconstruct Boeckh’s book collection. The information mass we are talking about there (thousands of books) is honestly a bit scary to me – I don’t usually work on that kind of scale -.

The first scans have come in a few weeks ago. They are beautiful (so I think anyway) as far as the resolution and the precision are concerned, but also concerning the subtle color shades. This is all very good, because it makes it possible to the eye to identify the elements on the page that are not printed, but handwritten, even if their color and texture make them look a lot like print. So far, so good. Now the question is: Do we want an OCR version of these scans?

The test versions of the OCR showed results that I was at first not able to interpret by myself. It turned out that they correspond to the actual state of the art. But what do you really expect from the OCR of a book with columns, two to three different types of script, ligatures, etc.? Inevitably, when I showed it to the Historians of Science and the Classical Philologists that were supposed to work with the OCR at the end of the day, they said they would rather have more books in a  good pdf than those OCR (somehow it all comes down to what money should be put in). Even when I argued that the OCR version would be helpful to realize full-text queries and that it could easily be improved by scholars, the damage was done and irreparable. Line numbers transformed into cabbalistic signs, diacritical elements completely lost, letters not recognized: of course, it is so – what else to expect?

I am still not convinced myself of the usefulness of OCRs that are so far away from – not even perfection, but just plain readability. What should these OCR really be there for? Large-scale queries? I am not sure that such a specialized corpus can contribute to that kind of project. Are they then supposed to be an experimenting field for OCR processes, and hence contribute to improve OCR? Or an experimenting field for OCR-to-TEI processes? I suppose that if I knew who is really going to work with these data, it would be much easier to decide whether or not they need/deserve OCR versions.

Hybridation Means

Although I had carefully avoided to confront myself with the question of how the digital version of our edition of Boeckh’s Encyklopädie (see here for more detail on that) was supposed to compare to the print version, I was compelled to do so this afternoon under the pressure of my co-grant applicants.

The fashionable way to edit is definitely to have both a digital and a print version: an hybrid edition. But even that leaves a wide range of options. One way is to have a book version shorter than the digital version is. Another way is to have a digital and a print version based on an identical source. In both cases, the decision concerns the text as well as the apparatus. Even if you replace the digital links with URLs that are written out, it is a major editorial decision to choose for instance to have a unique source for both versions, since such a scenario supposes that both the paper and the digital version rely and depend on one another. The whole conception has to systematically embrace both aspects.

So far, I thought it would make more sense to conceive differently the digital and the paper version. This is probably due to the fact that I did not consider actually making books out of what I edit digitally. The Encyklopädie, but also other corpora in our digital edition that are more likely to be considered readable (vs. the “searchable-only” corpora I edit myself and favor because they are so cool for network research, but that’s another subject), made Alex put the question of pdfs on the agenda of our next Digital Edition-meeting. I am curious to hear how this is supposed to work. With 6 different views of one document, which one is supposed to be turned into a pdf? The reading version, because the pdf is there to be read (vs. searched)? Or is the pdf of interest as a sum of information, regrouping all that you can get on one single document?

This question is relevant for the way we will conceive the print version of the Encyklopädie. If it is possible for instance to generate pdfs on the fly representing whichever version you like better (all text layers before 1811, only the text layer 1811, the original text of 1809,…), I don’t think we should trouble ourselves too much with a print version that should achieve what the digital edition is there for, e.g. representing the text layers (to be precise, the digital version is also there to compare the versions and to realize complex searches, which the print version will not be able to supply us with whatsoever).

We agreed this afternoon on a form of the print version that would basically use a double page to reproduce one manuscript page: 1809 text layer on the left side, each line being numbered, and on the right handside, all the ulterior text layers with a short description of where they are to be found on the manuscript page, peculiarities of ink and paper, datation of said layers – and at the bottom of all that, the minimalistic commentaries. In case that all the apparatus should not fit on one page, we would have to continue on the following page, e.g. on the following two pages in order to be able to get back to our left-right situation for the next manuscript page. I will have to make an exact calculation, but I don’t think this would be realizable on the scale of the whole Encyklopädie in less than 3 big books.

We will see on the next meeting if pdfs as Alex suggests to implement them actually are good hybridation means. And I am still  looking for arguments and examples pro and contra same source vs. not same source for digital and print version in order to make up my mind on that…

We want to see everything

As I mentioned in some earlier posts, I am working on a grant application for an edition project. What we want to edit is Boeckh’s Encyklopädie und Metholodogie der philologischen Wissenschaften. There exist several editions of this book, all of them being a) older and b) based on a cumulative view of the text. What we want to do in our edition is an extensive version. Let me explain why and how and what this is all about.

August Boeckh gave a class on this topic (Encyclopedy of the Philological Sciences) for the first time in 1809 when he was teaching at the Heidelberg University. He wrote down notes for the class in a notebook. In 1811, he moved to the Berlin University, where he gave the class again and again, every other summer term, almost until his death. During the course of time, he enriched his manuscript, added new bits, sometimes on little post-its he pasted into the notebook. All in all, he gave the class 26 times over a period of 56 years (between 1809 and 1865), which incidentally happen to be  the glorious time of German Classical Philology.

The first editor of the text, Ernst Bratuschek, was mostly inspired by the last sessions of the class, because he had attended said class. Several times. And to be precise, his edition of the Encyklopädie does not solely involve the Encyklopädie itself. Between 1862 and 1866, Bratuschek went not only to the Encyklopädie-class, but also to all of Boeckh’s other classes, and to the Philological Seminar, in order to have each and every point explained to him by the Master. The material he based his edition is thus composed of a) the knowledge of Boeckh’s system he had acquired by being taught by Boeckh, b) Boeckh’s own manuscript (in which Bratuschek didn’t hesitate to strike out all the “older” things that he considered were out of fashion),  c) several sets of clean students’ lecture notes, and d) Boeckh’s printed books. Needless to say, this edition is more a medley than anything, conceived as a handbook for the students of 1877.

Now we are not students of 1877, so what do we want from the Encyklopädie?

Because Boeckh was so influential and because Classical Philology was the leading Science by that time, studying the Encyklopädie  is not only a good way to reconstruct Boeckh’s conception of Philology (that would be the most obvious point), but also to put together important pieces of a background knowledge that influenced generations of scholars and school teachers. For my part (the co-authors of the grant application are in charge of other aspects that are just as interesting, but not so digitally interesting) I see two aspects of particular interest:

1) the Bibliography (bibliographical information gathered from the manuscript, about which Bratuschek explicitely writes that he only kept the most recent ones, can help reconstruct the scholarly horizon of Boeckhs and his disciples);

2) the evolution of the manuscript over time. The Archive of the Berlin-Brandenburg Academy of Science kindly agreed that I reproduce here one manuscript page:


As you can see, the additions have been done at different periods of time. Ink and paper are different. The manuscript as it presents itself is a sum of information, some overlapping others, some reaching out of the space of the page. I would not say that you don’t see anything, but for sure you don’t see everything.

What can we do to see everything?

The encoding, as fas as I can conceive it, should not be too complicated. Each block of text gets a time attributed: 1809 for the main text part for instance, or time spans for bits that we do not manage to date precisely. Each of these blocks of text is connected to the corresponding part of the scan of the manuscript. Possibly (if it makes sense) we can even attribute ink and paper quality to some layers we can date.

But then? Then, I imagine an interface in which you would see the scan (I always need to see the scan) and a whole range of visualizations: per period of time (for instance all the text layers of this page before 1830), per collation (all the differences between the 1809 and the 1832 version of this page), additionally to the categories we already do have in our digital edition (diplomatic transcription, easy-to-read version, entities, metadata). In a world in which we would all have overdimensioned monitors, I would also add a visualization of the differences with Bratuschek’s edition and a collation with the sets of lecture notes of the class we have from various students.

This way we would see everything! Or would we see nothing, having lost all kind of overview and being – yet again – overwhelmed by data overload?

This visualization question is crucial there, and I feel somewhat short of good ideas. So if you know any projects that have been working with several historical layers – be it of text, of painting, of whatever – and have had good ideas how to represent the dynamics, please let me know!


Voß, Boeckh, Rousseau…

… have probably nothing in common except for the cohabitation they are forced to in my schedule of this August: preparing the Voß volume (e.g. the contributions to the conference I organized together with Josefine Kitzbichler and Enrica Fantino in the Summer 2010, enriched with the Voß-Solger-correspondence), preparing the grant proposal for the edition of Boeckh’s Encyklopädie, and writing the long due paper on Communication for the Rousseau encyclopedia about to be edited in Potsdam by Stefanie Stockhorst and Iwan d’Aprile.

Communication strategies, one of my favorite refrains, certainly are the smallest common denominator between the three projects. Voß communicates – in his own dysfunctional way, bashing his best friends’ publications in newspapers reviews, then writing how much he loves them and appreciates their work in private letters – with the aim of being fully recognized in his scholarly skills (if not by his contemporary readers, then by the next generations: hope keeps alive). No wonder he comes to write about his new colleagues in 1810: “Böckh und Creuzer haben für Freundschaft keinen Sinn, als Gelehrte schäze ich beide außerordentlich” (loose translation: “Böckh and Creuzer have no sense for friendship, I value both a lot as scholars”). When I think of why both would not have wanted to engage into a friendship with Voß, I can only guess they considered  they were better off safe than sorry.

Boeckh didn’t stay long in Heidelberg, he came to Berlin in 1811. This takes us to the second point: Boeckh’ encyclopedia manuscript. It consists mainly of his own notes for a class he gave no less than 26 times within 56 years, every other summer term. The original text, dating back 1809 (given in Heidelberg), was enriched and actualized by Boeckh to reach its final state in 1865. The whole fun of this project is to find a nice (digital) way of representing the 26 text layers. But before we actually reach the fun part, we (Colin King and I, writing the grant application) need to concentrate on the novelty and actuality of this edition for today’s Classics, Hermeneutics and Science History. Finding the right balance between general ideas and convincing details is the whole science of writing applications – and here we are again with the communication strategies.

As for Rousseau… well. I am not a Rousseau specialist. I have just been under the spell of the introductory pages to the Confessions since I am 15 years old and found the idea of working on “communication” in his work interesting. I decided to see how Rousseau sets up communication at the beginning of his works in general. I read several of those, introductory pages to discourses, novels, etc.  – my original idea was that you find there a movement towards the reader at the same time as a movement of withdrawal from the reader, typical for the paradoxical position of Rousseau. Now that I have read more, I am surprised to see that the image that struck me at the beginning of the Confessions, the offering of oneself to a final judgement, is a topos in the introductory parts of his works – almost all of them. I am still not sure what I will do with that.

Two weeks of those, and then you will hear again of Beausobre and Chamisso in preparation for my two September talks, and then of Dorothea Tieck, whose biography I have to write for the NDB.

Although, to be honest, working is more of a fancy right now, since it is so hot that the fan is keeping me, the kids, and Toy Story 2 barely as cool as we can manage to survive.