How does AI translate a scanned document or image-based PDF? The system first uses optical character recognition (OCR) and document-layout analysis to convert page images into machine-readable text. It then translates that text and, in document-aware systems, rebuilds the translated page. Accuracy depends heavily on scan quality, handwriting, fonts, page layout and OCR performance. High-stakes scans should be reviewed against the original image before the translated version is approved.
A scan is not ordinary text.
For production multilingual workflows, see our AI Translation service for the delivery path behind the guidance in this article.
To a person, a scanned contract or historical letter may look like a page full of words. To a computer, it begins as an image: pixels arranged into shapes.
Before AI can translate the language, another system has to determine which pixels are letters, which letters form words, where paragraphs begin and what order the page should be read.
That is why scanned-document translation is fundamentally an OCR-plus-translation workflow.
Current Azure Document Translation documentation describes OCR-based processing for scanned PDFs, and Google Cloud supports scanned PDF translation while warning that formatting loss is more likely than with native PDFs.
How to tell whether a PDF is image-based
Try these tests:
- Can you select individual words?
- Can you search for a visible phrase?
- Does copy/paste produce readable text?
- Does zooming reveal pixelated letters?
If the answer is no to the first three, the PDF is likely image-based.
A searchable scan may pass the search test because it includes a hidden OCR layer. That OCR still needs validation.
The full scanned-document workflow
A high-quality workflow typically includes:
- page image ingestion;
- image cleanup;
- page orientation detection;
- layout analysis;
- OCR;
- language identification;
- OCR quality checks;
- translation;
- terminology checks;
- layout reconstruction;
- human review.
Skipping source verification means an OCR mistake can propagate invisibly through every later stage.
OCR is not translation
OCR asks:
What characters are visible on this page?
Translation asks:
What does this text mean in another language?
The difference matters.
If OCR reads:
The balance is 1,500
as:
The balance is 7,500
the translation engine has no reason to suspect the number is wrong.
The error occurred before translation.
What affects OCR accuracy?
Resolution
Very low-resolution text loses fine details.
Skew
A page photographed or scanned at an angle can reduce recognition accuracy.
Blur
Motion blur and poor focus merge character edges.
Contrast
Faded type, yellowed paper or shadows make segmentation harder.
Compression
Heavy JPEG compression creates block artifacts around characters.
Typeface
Decorative or historical fonts can be difficult.
Handwriting
Handwritten text is a different recognition problem from clean print.
Layout
Columns, sidebars, tables and stamps confuse reading order.
Improving the source image is often the single best way to improve the final translation.
Preprocessing scanned pages
Image preprocessing may include:
- deskewing;
- rotation;
- cropping;
- contrast adjustment;
- denoising;
- dewarping book pages;
- background removal;
- page separation.
Do not over-process.
Aggressive sharpening or thresholding can erase punctuation and diacritics.
Preserve the original scan so every processed version can be traced back.
Page orientation
A scan may contain:
- portrait pages;
- landscape exhibits;
- rotated inserts;
- upside-down pages.
OCR engines can detect orientation, but batch jobs should still flag low-confidence cases.
A page rotated 90 degrees may produce nonsense OCR that looks like random characters rather than a clear failure.
Layout analysis
Document AI needs to identify regions such as:
- title;
- paragraph;
- table;
- footer;
- page number;
- caption;
- marginal note;
- stamp;
- signature.
Layout is part of reading order.
A page with two newspaper columns must not be converted into alternating lines.
For forms, labels and values need to remain associated.
Language identification
OCR and translation both benefit from knowing the language.
Specifying the source language can improve recognition for:
- accented characters;
- script-specific punctuation;
- closely related alphabets.
Automatic detection is useful when the source is unknown, but short or multilingual pages may be difficult.
For archives, one file may contain correspondence in multiple languages. The workflow should support page-level or region-level language variation.
Old spelling and historical documents
Historical scans add problems beyond image quality.
The text may contain:
- obsolete spelling;
- archaic vocabulary;
- old typefaces;
- historical abbreviations;
- damaged characters.
A modern translation model may normalize or reinterpret language that should be preserved.
For research and archival uses, define whether the goal is:
- diplomatic transcription;
- normalized transcription;
- literal translation;
- modernized translation.
These are different deliverables.
Handwriting
Handwriting recognition remains more variable than printed-text OCR.
Difficulty increases with:
- cursive writing;
- signatures;
- corrections;
- overlapping notes;
- poor contrast;
- unfamiliar names.
A safe workflow marks uncertain handwriting instead of inventing text.
For evidentiary, historical or personal documents, a human should verify any passage where the system cannot read confidently.
Stamps, seals and signatures
These regions can contain important information but need special handling.
A stamp may include:
- organization;
- date;
- approval language;
- registration number.
A signature may identify a person but the pen stroke itself is not ordinary language.
Document translation should be able to represent non-text elements descriptively when required:
[signature]
[official seal]
rather than hallucinating words from graphic marks.
Tables in scanned documents
Tables require both OCR and structure recognition.
The system must determine:
- cell borders;
- row grouping;
- column grouping;
- merged cells;
- header relationships.
Poor scans can erase faint grid lines.
A useful QA approach compares each row against the source and verifies that translated labels still point to the correct numbers.
This is essential for:
- invoices;
- bank statements;
- laboratory results;
- technical specifications;
- financial reports.
Forms
Scanned forms combine printed templates with handwritten or typed entries.
Translate the static labels while deciding separately how to handle user-entered values.
A form field such as:
Name: John Smith
should not translate the person’s name unless the project uses transliteration rules.
Similarly:
- account numbers stay unchanged;
- dates may be localized in display but not altered semantically;
- addresses may need partial transliteration rather than translation.
Image-based PDFs with photographs
A PDF can contain photos of signs, equipment or screenshots.
Multimodal systems can increasingly detect and translate text inside images. Current Azure Translator updates include standalone image translation support in preview and expanded document image-processing capabilities.
Still, verify:
- whether image text was detected;
- whether it was actually translated;
- whether the replacement obscures the original image;
- whether the target text fits.
A translated paragraph beside an untranslated warning label is not a complete translation.
Mixed scan quality
One file can contain pristine pages and unreadable exhibits.
Do not calculate one quality assumption for the whole document.
Classify pages:
- clean;
- moderate;
- poor;
- handwriting;
- table-heavy;
- image-heavy.
Then route difficult pages to additional review.
This risk-based approach keeps automation efficient without pretending every page is equally easy.
Confidence thresholds
OCR confidence can be used for triage.
For example:
- high confidence → automated translation plus sampling;
- medium confidence → source-text review;
- low confidence → human transcription or re-scan.
The exact thresholds depend on the engine and domain.
Confidence should guide review, not serve as proof of accuracy.
Names and numbers
Names and numbers deserve special extraction checks.
OCR errors are especially damaging in:
- legal names;
- passport numbers;
- addresses;
- case numbers;
- dates;
- currency;
- dosage;
- serial numbers.
A translated sentence can hide the fact that the source value was misrecognized.
Consider running rule-based validation for expected number formats and comparing target numbers with source OCR.
Right-to-left scans
Scanned Arabic, Hebrew, Persian and other RTL documents require:
- script-aware OCR;
- correct reading order;
- proper bidi reconstruction;
- RTL paragraph direction;
- correct handling of embedded numbers.
A page can be recognized accurately but reconstructed in the wrong direction.
Visual QA is mandatory.
Choosing output format
The translated result can be returned as:
- PDF;
- DOCX;
- TXT;
- bilingual PDF;
- structured JSON;
- searchable text layer.
Choose based on the use case.
Best when visual correspondence matters.
DOCX
Best when reviewers need to edit or reformat.
TXT
Best for language analysis or plain-text archives.
Bilingual output
Useful for legal, research and review workflows where source-target comparison is important.
Human review strategy
A full human review should compare the translation with both:
- extracted source text;
- page image.
Why both?
The text view exposes the OCR the model received. The image reveals whether the OCR itself was wrong.
For difficult documents, a two-stage review is effective:
Stage 1: source verification
Correct OCR errors.
Stage 2: translation review
Review meaning and target-language quality.
Quality checklist
Image quality
- correct orientation;
- no missing page edges;
- sufficient contrast;
- no duplicate pages.
OCR
- names correct;
- numbers correct;
- reading order correct;
- punctuation reasonable;
- tables structured.
Translation
- meaning preserved;
- terminology correct;
- no omissions;
- no invented text.
Layout
- translated text visible;
- tables readable;
- page correspondence clear;
- images reviewed;
- RTL direction correct.
When to rescan instead of fixing OCR
Sometimes the cheapest solution is a better scan.
Rescan when:
- page is badly blurred;
- text is cut off;
- resolution is extremely low;
- shadows hide content;
- camera perspective distorts lines;
- color washout hides annotations.
No AI model can reliably recover information that is not visible in the source.
Security and privacy
Scanned documents often contain highly sensitive data:
- IDs;
- medical records;
- contracts;
- financial statements;
- signatures.
Before uploading, verify:
- access controls;
- encryption;
- retention;
- deletion process;
- subcontractor handling;
- regional processing requirements.
If a system uses customer data to improve models, determine whether that is acceptable for the project.
Keep source-image evidence available during review
A reviewer working only from OCR text cannot tell whether a suspicious word came from the source page or from recognition error. Keep the page image visible beside the extracted source and target translation. This three-way view is especially valuable for names, handwritten notes, dates, account numbers and table values. Where the scan is genuinely unreadable, mark uncertainty explicitly instead of forcing a confident reconstruction. That practice preserves the evidentiary value of the source and makes later corrections easier to audit.
Frequently asked questions
Can AI translate a photo of a document?
Yes, if the system includes OCR or multimodal image understanding. Image quality has a major effect on accuracy.
Can AI translate handwritten scans?
Sometimes, but handwriting recognition is more variable than printed-text OCR. Human verification is recommended for important content.
Why does scanned PDF translation lose formatting?
The software must reconstruct layout from page images rather than edit the original document structure. Complex pages are harder to rebuild.
What is OCR translation?
It is a two-stage process in which OCR first converts image text into machine-readable source text and a translation system then translates that text.
Should I correct OCR before translating?
For high-stakes or poor-quality scans, yes. Correcting the source prevents translation of recognition errors.
Conclusion
AI makes scanned-document translation dramatically more practical, but the workflow still depends on the quality of the pixels.
The most reliable process separates OCR from translation, validates names and numbers, reconstructs layout carefully and routes uncertain pages to human review. Clean scans may move almost automatically. Difficult historical, handwritten, legal or table-heavy pages deserve a more controlled path.
The objective is not merely to produce target-language text. It is to create a translation that can be traced back to what was actually visible on the source page.
Research references
- Microsoft Azure, Document Translation overview:
- Microsoft Azure Translator, What's new:
- Google Cloud, Translate documents:
