What text recognition can and cannot recover from a scan
Last reviewed
A scanned PDF is a picture of a document wearing a document's file extension. There are no glyphs, no coordinates and no font metrics — just an image per page, and a converter walking the text layer finds nothing at all.
Text recognition is the only way through, and it is a genuinely different operation from conversion. It does not read structure badly; it reads characters from pixels, and everything after that is downstream of how well it did.
9 minute read
How a converter knows it is looking at a scan
There is no flag in the file that says so. The test is simply how much text came out: a document whose entire text layer yields fewer than twenty characters is treated as image-only.
That threshold is low on purpose. Many scans carry a handful of characters — a stamped page number, a digitally added footer, a form field — and a stricter test would classify those as text documents and hand back four characters of Markdown for a hundred-page file.
The consequence worth knowing: a document that is half scanned and half digital will not trip the test. It converts as a text document, and the scanned pages simply come back empty. If a conversion is suspiciously short, count the pages that produced nothing.
What recognition actually produces
It is easy to picture OCR as producing text. It does not. It produces a list of word boxes: for each word, the characters it thinks it saw, a rectangle on the page, and a confidence between zero and one.
That output shape is what makes structure possible afterwards. Positioned words are the same thing a digital PDF gives up, so once recognition is done, every rule that reconstructs headings, paragraphs, tables and columns can run unchanged — and a scan and its digital twin converge on nearly the same Markdown.
Why the boxes need adjusting first
There is one mismatch between what a recognition engine reports and what a spatial converter expects, and it is worth understanding because it explains a class of failure that looks like poor recognition but is not.
A digital PDF reports a font height: the size the type was set at. A recognition engine reports the bounding box around the inked pixels it found. Those are different numbers. The word sun and the word Ghjq, set in identical type, produce boxes that differ by roughly forty per cent, because one has ascenders and descenders and the other does not.
Feed those raw box heights into a heuristic that derives the body-text height from the most common height in the document, and the result is a noisy unit of measure and erratic heading detection. So words are grouped into lines first and every word inherits its line's height and baseline. A line almost always contains at least one ascender, which makes the line box a stable proxy for font size.
Confidence, and what gets thrown away
Every recognised word carries a confidence score, and two thresholds act on it.
| Below 0.25 | The word is discarded and counted, not guessed at |
|---|---|
| Page mean below 0.7 | The page is flagged as low confidence |
| Reported to you | Dropped word count, and which pages were flagged |
Why dropping is better than guessing
A discarded word leaves a hole. A low-confidence word that is kept leaves a plausible wrong word, and that is worse in every use the document has afterwards.
A hole is visible. You read the Markdown, you see a gap, you check the original. A confidently wrong word is invisible: it passes review, it gets indexed, it gets retrieved, and it is quoted back to you as fact by whatever you built on top of the corpus. Numbers are the sharpest case — a misread digit in a table is a number that is simply false and looks exactly as trustworthy as the true ones.
This is why the dropped word count is worth reading rather than dismissing. A page that dropped forty words did not lose forty words of meaning quietly; it told you where to look.
What recovers well, and what does not
The pattern is roughly what you would guess, with one exception at the bottom that surprises people.
| Recovers well | Flatbed scans at 300 DPI of printed text, straight and in focus |
|---|---|
| Recovers usably | Older photocopies, faxes, slightly grey originals |
| Recovers partly | Tables and forms — the characters survive, the alignment often does not |
| Recovers badly | Phone photographs, which are rotated, curved and unevenly lit |
| Does not recover | Handwriting, and text over busy background imagery |
Why phone photos are the weak case
Preprocessing before recognition is deliberately minimal: convert to greyscale, stretch the contrast so a faint scan uses the full range, encode losslessly. There is no deskew and no adaptive thresholding.
That is a considered choice rather than an omission. Every extra transform is another chance to destroy a faint glyph, and the documents that would benefit most from real geometric correction — photographs taken at an angle, of a page that is not flat — are exactly the ones where a naive rotation makes the result worse. Correcting them properly needs a different class of tool, and adding one should follow a measurement rather than a hope.
The practical consequence is simple. If you can put the page on a flatbed, do. A mediocre flatbed scan beats an excellent photograph.
What it costs, and what is kept
Recognition is the one operation on this site that cannot happen in your browser: it needs a real recognition engine and the CPU to run it. Pages are therefore sent to a server, which is why it is a separate page with a consent step that states the page count and the destination before anything moves.
Pages are rasterised at 300 DPI, which is the resolution the engine is tuned for and roughly where accuracy stops improving for printed text. Nothing is written to disk, and a finished result stays retrievable for ten minutes before it is gone. That window is the entire lifetime of any document-derived data on that machine.
Getting a better result before you start
- Scan at 300 DPI. Higher is slower without being better for printed text; lower loses strokes on small type.
- Scan in greyscale rather than colour or pure black and white. Hard thresholding at scan time destroys faint characters that contrast stretching could have recovered.
- Flatten the page. A book held open produces curved text near the spine, and curved baselines defeat line grouping even when every character is recognised.
- Straighten before you start. A page scanned two degrees off is a page where the line grouping has to fight the rotation.
- Check whether a text version exists at all. Many scanned documents are scans of something that was born digital, and half an hour of searching beats any amount of recognition.