How to Detect Hallucinations and Omissions in AI Translation

For production multilingual workflows, see our AI Translation service for the delivery path behind the guidance in this article.

Two of the most serious translation failures are opposites.

A hallucination adds target-language information that is not supported by the source. An omission drops source information that should have been translated.

Both can be difficult to notice because modern systems are highly fluent. The target sentence may read perfectly even though it no longer represents the source.

The EMNLP 2023 HalOmi benchmark defines hallucinations as target content unrelated to the input and omissions as missing input information, and provides manually annotated examples across 18 translation directions.

Why fluency makes the problem dangerous

Older MT errors were often obvious.

Modern AI can produce:

• correct grammar;
• natural syntax;
• plausible terminology.

A hallucinated sentence can therefore look more professional than a faithful awkward sentence.

Quality review must explicitly test fidelity to the source.

Types of hallucination

Full hallucination

The target is largely unrelated to the source.

Partial hallucination

Part of the target is unsupported.

Elaboration

The model adds plausible detail.

Example:

Source:

“The device was tested.”

Target:

“The device was successfully tested and certified.”

“Certified” is unsupported.

Repetition hallucination

The system repeats phrases or creates loops.

These phenomena have been documented in neural machine translation research.

Types of omission

Full segment omission

An entire sentence or field is missing.

Partial omission

A qualification or clause disappears.

Entity omission

A name, date or number is dropped.

Discourse omission

A connecting idea such as “however” disappears, changing the relationship between statements.

Small omissions can be more dangerous than full missing sentences because they are harder to notice.

Why hallucinations occur

Potential causes include:

• uncertain source input;
• out-of-domain text;
• rare names;
• decoding behavior;
• noisy training;
• prompt over-instruction;
• long-context failures.

For document translation, upstream errors such as OCR can also cause apparent hallucination relative to the visual document even if the model faithfully translates the corrupted OCR.

Why omissions occur

Potential causes:

• long or complex sentences;
• repeated structure;
• segmentation;
• source truncation;
• token limits;
• attention failures;
• file extraction problems.

A translation pipeline must therefore test both model output and source extraction.

HalOmi benchmark

HalOmi is a manually annotated benchmark designed for multilingual hallucination and omission detection.

It covers:

• 18 translation directions;
• different resource levels;
• different scripts;
• partial/full hallucination;
• sentence/word-level omission.

The research also showed that conclusions from a single language pair do not always generalize.

This supports multilingual-specific QA rather than one global threshold.

Source-target length ratio

A very simple detector is length ratio.

If a source paragraph is 500 characters and the target is 40, something may be missing.

If the target is 1,500, there may be elaboration.

But languages expand differently.

Length is a flag, not proof.

Use language-pair baselines.

Segment alignment

Ensure every source segment maps to a target segment.

Detect:

• empty target;
• duplicated source ID;
• duplicated target;
• missing file object.

This catches pipeline omissions before semantic analysis.

Named-entity preservation

Extract:

• person names;
• organizations;
• locations;
• product names.

Check whether they appear in target form.

Be careful with:

• transliteration;
• conventional translated names.

Entity mismatch can reveal both omission and hallucination.

Number validation

Compare:

• dates;
• percentages;
• currency;
• quantities;
• IDs.

A target number absent from the source may be hallucinated.

A source number absent from the target may be omitted.

This is one of the most reliable automated checks.

Negation and polarity

Check:

• not;
• never;
• without;
• prohibit;
• allow.

Omitting negation can reverse meaning without large length changes.

Target sentiment/polarity differences can be used as a QA signal.

Semantic alignment

Embedding models or quality-estimation models can estimate whether source and target carry similar meaning.

Low semantic similarity can flag:

• hallucination;
• omission;
• mistranslation.

However, similarity metrics can miss a single critical detail inside a long segment.

Combine them with granular checks.

Word or phrase coverage

Alignment models can identify source words with no target correspondence.

This is useful for:

• nouns;
• numbers;
• technical terms.

Function words may legitimately have no direct target counterpart.

Use linguistic filtering.

Round-trip checks

Back translation can sometimes reveal missing meaning.

Workflow:

target → back to source → compare with original.

If major content disappears, investigate.

But back translation can also reconstruct meaning incorrectly and is weaker than supervised quality estimation in research.

Use it only as one signal.

LLM-based fidelity review

An LLM judge can be prompted to list:

1. source claims;
2. target claims;
3. source claims missing from target;
4. target claims unsupported by source.

This claim decomposition is more useful than asking:

“Is this translation accurate?”

The output should be validated against human examples before large-scale deployment.

Sentence splitting

Long sentences can hide omissions.

For QA, break semantic content into:

• clauses;
• propositions;
• list items.

Then compare coverage.

Do not necessarily change the final translation segmentation; use decomposition only for evaluation.

Tables and structured data

Omissions often happen outside prose.

Check:

• table cells;
• headers;
• footnotes;
• labels;
• captions.

A document translation can translate all paragraphs perfectly while missing an entire text box.

File-structure QA is critical.

Audio transcription translation

For translated audio, hallucination may originate in transcription.

Compare:

• audio;
• source transcript;
• target.

If the source transcript contains invented text, translation QA alone will not catch the true source error.

Risk scoring

Assign high risk to segments containing:

• numbers;
• medical instructions;
• legal obligations;
• financial disclosures;
• safety warnings.

Even a low statistical probability of hallucination should trigger human review in those segments.

Human verification

Humans should compare source and target.

Ask reviewers specifically:

• Is any source meaning missing?
• Is any target meaning unsupported?
• Are numbers/entities exact?
• Has uncertainty been strengthened?

Specific questions catch more fidelity errors than general proofreading.

Automated pipeline

A robust detector can combine:

1. structural completeness;
2. length ratio;
3. numbers/entities;
4. terminology;
5. semantic score;
6. LLM judge.

If several signals fail, escalate.

False positives

Legitimate translation may:

• paraphrase;
• split/merge sentences;
• omit redundant pronouns;
• add grammatical words.

Do not interpret every alignment gap as an error.

Human validation is essential while tuning thresholds.

Monitoring hallucination rate

Track:

• hallucinations per 1,000 segments;
• omissions per 1,000;
• critical error rate;
• language;
• model;
• domain.

A rising rate can indicate model or pipeline regression.

Example: a partial hallucination hidden inside good prose

Source:

“The study included 48 participants and lasted six weeks.”

Target:

“The six-week clinical study included 48 participants and demonstrated significant improvement.”

Most of the sentence is faithful, so broad semantic similarity can remain high. “Clinical” and “demonstrated significant improvement” are unsupported. A claim-level evaluator should identify each target proposition and map it to source evidence.

This kind of partial hallucination is more dangerous than nonsense output because readers are unlikely to question it.

Coverage matrices

For long documents, represent important source elements in a coverage table:

The matrix can be generated by an LLM and sampled by humans. It is particularly effective for reports, contracts and summaries where individual claims matter.

Detecting omission in lists

Bulleted and numbered lists are easy to check structurally. Compare list-item counts and important entities within each item. A model can accidentally merge two source items or drop a repetitive item that appears redundant.

For procedures and safety instructions, list coverage should be a hard validation gate.

Hallucinations after source preprocessing

Always preserve the extracted source used by the model. If OCR incorrectly reads “2025” as “2028,” the translation may be faithful to the extracted text but wrong relative to the document image. A reviewer needs to know whether the defect originated in extraction or translation.

This is why a document QA system should keep a chain:

original file → extracted source → target translation.

Incident response for hallucination

When a critical hallucination is found in production:

1. correct and remove the affected output;
2. search related translations generated by the same configuration;
3. identify the root cause;
4. add the case to a regression set;
5. update routing or prompts;
6. re-evaluate before restoring automation.

Treat recurring hallucination as a system defect, not a one-off editorial typo.

Precision and recall for hallucination detectors

A detector should be evaluated like any classifier.

Precision: of the segments it flags as hallucinations/omissions, how many truly contain those errors?

Recall: of the real hallucinations/omissions in the human-labeled test set, how many does it catch?

For high-risk translation, recall on critical fidelity errors is usually more important than avoiding every false alarm. A slightly noisy detector is acceptable if reviewers can clear false positives quickly and it rarely misses dangerous additions.

Create synthetic challenge errors

To benchmark detectors, take good human translations and deliberately inject controlled defects:

• delete a qualifier;
• change one number;
• add an unsupported claim;
• remove a negation;
• duplicate a sentence;
• swap a person name.

Because the location and error type are known, you can measure detector recall precisely. Mix these synthetic examples with naturally occurring model errors so the test set remains realistic.

Long-document hallucination detection

A target sentence can be supported by information elsewhere in a long source document. A narrow segment-level judge may incorrectly label it as hallucinated if the translation legitimately restructures content.

For document translation, provide the evaluator with the aligned paragraph or relevant retrieved source context. Conversely, do not give so much irrelevant context that almost any target claim can appear loosely supported.

High-risk unsupported specificity

Pay special attention when the target becomes more specific than the source:

• “may” → “will”;
• “several” → “five”;
• “treatment” → a named drug;
• “reviewed” → “approved.”

These are common forms of partial hallucination because the added detail sounds plausible. A QA rule can compare modality, quantities and named concepts explicitly.

Production threshold policy

A mature system can define:

• critical unsupported claim → block;
• likely omission in high-risk content → mandatory review;
• low-confidence stylistic mismatch → sample queue.

This converts hallucination detection from an interesting metric into a real publishing control.

Implementation takeaway

For production use, treat claim-level fidelity, critical-error recall and source-extraction traceability as measurable parts of the workflow rather than assumptions. Define the relevant policy before processing content, keep the model or data configuration versioned, and verify results on representative material from the actual language, locale and domain. If automation produces a result that conflicts with a deterministic rule, authoritative terminology or qualified human evidence, escalate the item instead of averaging the disagreement away. The most durable multilingual systems preserve source-to-target traceability, record exceptions and feed reviewed corrections back into benchmarks, glossaries or routing rules. That discipline makes quality improvements reproducible when models, providers, reviewers or source content change.

Frequently asked questions

Are AI translation hallucinations common?

They are not the dominant error in strong systems, but they can be catastrophic and deserve explicit monitoring.

What is the difference between hallucination and mistranslation?

A mistranslation represents source content incorrectly. A hallucination introduces content unsupported by the source.

Can automatic metrics detect omissions?

Some learned metrics can flag them, but targeted alignment and entity/number checks improve coverage.

Is back translation a reliable hallucination detector?

It is useful as an auxiliary signal but research shows backtranslation-based quality estimation underperforms supervised QE approaches.

What content should always be checked?

Numbers, legal obligations, medical instructions, safety content, names and financial facts.

Conclusion

Hallucinations and omissions are fidelity failures, and fluency can hide them.

A reliable QA stack checks file structure, segment coverage, entities, numbers, semantic alignment and unsupported target claims. Human reviewers then verify high-risk or flagged content.

The goal is simple: every important source idea should appear in the target, and every important target idea should come from the source.

Research references

• Dale et al., HalOmi (EMNLP 2023):
• Guerreiro et al., Hallucinations in Neural Machine Translation (EACL 2023):
• Agrawal et al., Backtranslation Quality Estimation (WMT 2022):