Reading place codes out of 125 years of archive scans

Staatsbosbeheer, the Dutch forestry service, has collected information about its nature areas for 125 years. Much of it sits in a scientific archive that is hard to search: scanned reports, maps and forms, many with handwritten notes, behind an old catalogue system. Forest rangers work in GIS every day, but that archive doesn't connect to it.

Our product owner, Finn van der Ent, manages that archive. His question for the RangerGIS-Archive project was: how can AI make this information available to rangers in a modern, convenient way? Our team took the first, very practical step: find out which area a scanned document is about, so it can be classified and eventually shown on a map.

The key: place codes

Every Staatsbosbeheer area has a short place code (plaatscode), like HDWK for Westduinpark near The Hague. The organisation keeps a list of all objects with their codes, area, region and province. I converted that spreadsheet to JSON so the rest of the pipeline could use it.

If I could reliably read a place code from a scan, the document could be linked to an area automatically. The archive documents often have the code typed on a form, written by hand next to a word like Dossier, or both.

Step 1: get the text out

Scans arrive as PDFs. The first version used PyMuPDF to pull out embedded text and images, and Tesseract to OCR the images. For full-page scans I switched to pdf2image to render each page and ran Tesseract with the Dutch language model (lang='nld').

Raw OCR on old scans is noisy, so most of the work went into the next step.

Step 2: clean up the image first

OCR quality depends far more on the input image than on the OCR engine. I built up a preprocessing toolbox in OpenCV and tested each step on real archive pages:

1. Grayscale and binarisation, with Otsu's method choosing the threshold automatically.
2. Noise removal with a small dilation followed by erosion, which removes specks without eating thin strokes.
3. Thinning or thickening the font (erode or dilate) depending on whether letters bleed together or break apart.
4. Deskewing: blur, threshold and dilate until text lines become solid blocks, find the largest contour, take its angle with minAreaRect, and rotate the page straight.
5. Removing borders left by the scanner, by cropping to the largest contour.

Deskewing alone made a visible difference: Tesseract reads straight lines much better than lines tilted by a degree or two.

Step 3: segment before reading

Running OCR on a whole form mixes up columns, stamps and handwriting. So I segmented the page first:

• dilate the binary image so words merge into lines and lines into paragraphs,
• find the contours of those blocks with cv2.findContours,
• run Tesseract per paragraph (--psm 6) or even per word (--psm 7) on just that region.

Working region by region gave much cleaner text, and it made it possible to look specifically at the area around a keyword.

Step 4: match against the known codes

Even good OCR makes mistakes: 0 for O, a missing letter, an extra space. Exact matching would miss most codes. Since the list of valid codes is known, the better question is which known code is closest to what the OCR read?

I used fuzzy string matching (first fuzzywuzzy, later the faster rapidfuzz) to score every known place code against the extracted text, kept the top ten, and plotted them as a bar chart per document. When one code clearly beats the rest, that's the area the document belongs to. When several are close, the document goes to a human.

When OCR wasn't enough

Tesseract is built for print and struggles with handwriting. We tried EasyOCR, a deep-learning OCR library, with a regular expression that grabs the text after Dossier, where the code is often written by hand. I also built a small CRNN in TensorFlow (convolutional layers feeding a recurrent GRU layer, trained with CTC loss) on crops of handwritten codes I'd annotated in VoTT.

Even so, the honest conclusion halfway through was that the OCR route wasn't giving satisfying results on these documents. We'd also been working steps ahead of ourselves, polishing segmentation before we knew whether reading the text was the right approach at all.

Switching to a classifier

So we changed the question. Instead of read the text, then match it to a code, we asked: given an image of the code region, which place code is it? That's an image classification problem, with the place codes as the labels.

The pipeline became:

1. refine the image with OpenCV, using the same preprocessing toolbox,
2. let a CNN extract meaningful features,
3. classify the image into one of the place-code labels.

We compared frameworks and went with PyTorch, training on Google Colab for the GPU. The biggest problem we ran into was false positives: a classifier always picks some class, even for an image that doesn't contain a code it knows. That's where the fuzzy-matching idea still helps: combining the model's prediction with a confidence threshold, and sending unclear cases to a person. In our final comparison we put plain OCR side by side with OCR plus the CNN model.

Two small extras

Some documents contain aerial photos. A simple heuristic flags them: convert to HSV and count green and brown pixels; if they cover more than 30% of the image, it's probably a satellite or aerial photo. For photos with EXIF data, I also read the GPS coordinates, which could place an image on the map without any OCR at all.

What I learned

• Preprocessing beats model choice. Deskewing and binarisation improved results more than switching OCR engines.
• Use the data you already have. Knowing the full list of valid codes turned a hard reading problem into a ranking or classification problem.
• Be willing to reframe. Switching from "read the text" to "classify the image" was the biggest step forward in the project.
• Know when to hand over to a human. A confidence ranking that sends unclear documents to a person is more useful than a system that always guesses.

This is the groundwork for a better organised, searchable archive that rangers can eventually reach from their GIS tools.