Vision and decision: what should I do with an OCR?
As much as I like developing visions, they lead one way or another to pragmatical decisions that are not always so easy or satisfactory.
Here is the vision: you would have the list of books that Boeckh gave to the University Library in manuscript form, digitized on your screen. You would from there come through an XML/TEI-encoding to the scan of the book itself – and from there, through another XML/TEI-encoding, to the way Boeckh quoted or used this book in his work (let’s say for instance in his Encyklopädie-class or at the Philological Seminar, for which the concerning manuscripts would also be encoded and digitized). Smoothly surfing between genesis and reception: it is just beautiful.
And huge. There are massive encodings that are to be done (Julia Doborosky has been working on this, informing book entries by hundreds just as she had done with the manuscript entries last year), and there are texts to be scanned. Which ones? If Boeckh’s personal exemplar of the book listed by him is to be found (condition number one) and either if it contains interesting marginal notes by him or if there is no other scan of that book available in open-access worldwide (conditions number 2a and 2b), then the book should be scanned. At this point begins the project I am developing with the University Library at the Humboldt University. The idea is to virtually reconstruct Boeckh’s book collection. The information mass we are talking about there (thousands of books) is honestly a bit scary to me – I don’t usually work on that kind of scale -.
The first scans have come in a few weeks ago. They are beautiful (so I think anyway) as far as the resolution and the precision are concerned, but also concerning the subtle color shades. This is all very good, because it makes it possible to the eye to identify the elements on the page that are not printed, but handwritten, even if their color and texture make them look a lot like print. So far, so good. Now the question is: Do we want an OCR version of these scans?
The test versions of the OCR showed results that I was at first not able to interpret by myself. It turned out that they correspond to the actual state of the art. But what do you really expect from the OCR of a book with columns, two to three different types of script, ligatures, etc.? Inevitably, when I showed it to the Historians of Science and the Classical Philologists that were supposed to work with the OCR at the end of the day, they said they would rather have more books in a good pdf than those OCR (somehow it all comes down to what money should be put in). Even when I argued that the OCR version would be helpful to realize full-text queries and that it could easily be improved by scholars, the damage was done and irreparable. Line numbers transformed into cabbalistic signs, diacritical elements completely lost, letters not recognized: of course, it is so – what else to expect?
I am still not convinced myself of the usefulness of OCRs that are so far away from – not even perfection, but just plain readability. What should these OCR really be there for? Large-scale queries? I am not sure that such a specialized corpus can contribute to that kind of project. Are they then supposed to be an experimenting field for OCR processes, and hence contribute to improve OCR? Or an experimenting field for OCR-to-TEI processes? I suppose that if I knew who is really going to work with these data, it would be much easier to decide whether or not they need/deserve OCR versions.