AI Translation Quality Assurance: A Step-by-Step Review Workflow

For production multilingual workflows, see our AI Translation service for the delivery path behind the guidance in this article.

AI translation quality assurance is not one proofreading pass at the end of a project. It is a sequence of controls designed to prevent, detect, classify and correct different kinds of errors before the translation reaches its audience.

The strongest workflows separate mechanical checks from linguistic judgment. Computers are excellent at verifying numbers, tags, placeholders and required terms. Qualified reviewers are better at evaluating meaning, naturalness, ambiguity and context. When these checks are ordered correctly, human reviewers spend less time on defects that software could have caught automatically.

For high-risk content, quality assurance should include a documented approval owner and zero tolerance for defined critical error types.

Step 1: define the quality requirement

Before translation begins, record:

• source language;
• target language and locale;
• audience;
• domain;
• publication channel;
• required turnaround;
• risk level;
• expected review depth.

“High-quality translation” is too vague to operate.

A customer support article, a legal contract and an internal research note can all have different acceptance criteria.

Step 2: validate the source

Translation cannot fix a broken source reliably.

Check for:

• missing pages;
• OCR errors;
• duplicated segments;
• malformed tables;
• tracked changes;
• ambiguous abbreviations;
• source-language inconsistencies.

For audio, validate:

• intelligibility;
• language;
• speaker structure.

For documents, verify that extraction includes text boxes, footnotes and other in-scope elements.

Step 3: classify content risk

A risk model helps allocate review.

Low risk

Internal understanding, draft research, transient chat.

Medium risk

Customer help, product documentation, ordinary marketing.

High risk

Legal, medical, financial, safety, regulatory, public claims.

Risk can be calculated at document or segment level.

A pricing table may be higher risk than the descriptive paragraph beside it.

Step 4: prepare terminology

Before generation, load:

• glossary;
• definitions;
• product names;
• entity list;
• do-not-translate terms.

WMT terminology research shows that terminology resources can improve term handling, but quality still needs direct evaluation.

Freeze the glossary version used for the job.

Step 5: prepare the style guide

The style guide should define:

• formal/informal address;
• brand voice;
• punctuation;
• capitalization;
• treatment of acronyms;
• locale conventions;
• acceptable translation freedom.

AI follows explicit operational rules better than vague instructions such as “make it professional.”

Step 6: translate with sufficient context

Avoid translating isolated strings when broader context is available.

Provide:

• paragraph context;
• document context;
• page/screen;
• domain;
• target audience.

For UI strings, include a description or screenshot identifier.

For dialogue, preserve speaker turns.

Context reduces ambiguity and inconsistency.

Step 7: structural completeness check

Before judging language, confirm that the output is complete.

Check:

• source and target segment counts;
• missing files;
• empty translations;
• duplicated paragraphs;
• missing tables;
• unprocessed images.

This catches pipeline failures early.

Step 8: placeholder and markup checks

Software and structured content may contain:

• {name};
• %s;
• HTML tags;
• XML;
• Markdown;
• ICU messages;
• URLs.

Validate exact preservation where required.

A translation can be linguistically perfect and still break production because {customer_name} became {customer name}.

Step 9: number and entity checks

Compare:

• dates;
• quantities;
• prices;
• percentages;
• IDs;
• phone numbers;
• names.

Automatically flag:

• missing values;
• new values;
• changed digits.

For locale formatting, distinguish intentional presentation changes from semantic changes.

Step 10: terminology QA

Check:

• required terms;
• prohibited variants;
• do-not-translate items;
• repeated consistency.

Use morphology-aware matching for inflected languages.

Route ambiguous term cases to a bilingual reviewer.

Step 11: hallucination and omission checks

Use:

• alignment;
• semantic quality estimation;
• length ratio;
• LLM fidelity review.

Ask whether:

• every important source claim appears;
• any target claim lacks source support.

HalOmi research demonstrates that hallucination and omission are distinct multilingual pathologies deserving targeted detection.

Step 12: automatic quality scoring

Optional automated metrics include:

• COMET;
• quality-estimation models;
• LLM judges.

Use them for:

• triage;
• trend monitoring;
• model comparison.

Do not convert the score directly into “percent accurate.”

Calibrate on your own reviewed data.

Step 13: bilingual human review

The reviewer compares source and target.

Review in this order:

1. meaning;
2. completeness;
3. numbers/entities;
4. terminology;
5. fluency;
6. style;
7. locale.

Starting with style can waste time polishing a sentence that is semantically wrong.

Step 14: severity classification

Classify errors:

Minor

Limited impact.

Major

Materially affects comprehension or requirement.

Critical

Can cause harm, legal/financial error, safety issue or invalid delivery.

A critical-error policy gives reviewers clear escalation rules.

Step 15: specialist review

Some content needs a second expertise layer.

Examples:

• attorney for legal concepts;
• clinician for medical terminology;
• engineer for safety procedures;
• finance specialist for disclosure.

The linguistic reviewer and subject expert can be different people.

Step 16: target-only proofreading

After semantic corrections, read the target without constantly looking at the source.

This helps detect:

• awkward flow;
• grammar;
• formatting;
• repeated wording;
• inconsistent headings.

Target-only review is useful, but it should come after bilingual accuracy review.

Step 17: visual and functional QA

For formatted content, inspect the final environment.

Documents:

• tables;
• fonts;
• page flow.

Web:

• navigation;
• forms;
• RTL;
• mobile.

Subtitles:

• timing;
• line breaks.

UI:

• truncation;
• buttons;
• dynamic values.

The final artifact, not raw translated text, is the deliverable.

Step 18: correct and re-run checks

Every correction can create a new issue.

After editing, rerun:

• numbers;
• glossary;
• placeholders;
• missing text.

Do not assume post-editing only improves quality.

Step 19: approval

Define who can mark:

• reviewed;
• approved;
• publishable;
• certified where applicable.

Keep raw AI drafts distinct from approved versions.

This prevents accidental use of unreviewed output.

Step 20: capture learning

Record recurring corrections in:

• glossary;
• style guide;
• translation memory;
• examples;
• automated rules.

QA should improve the next job, not merely fix the current one.

Quality gates

A simple enterprise pipeline can have:

Gate A: machine completeness

All files processed, no structural failures.

Gate B: automatic language QA

Numbers, terms, tags, semantic alerts.

Gate C: human linguistic acceptance

No unacceptable accuracy issues.

Gate D: business/specialist approval

Required for high-risk content.

Gate E: final artifact validation

Formatting and functionality pass.

This makes ownership visible.

Sampling for scale

Full bilingual review may be unnecessary for proven low-risk workflows.

Use:

• random sample;
• risk-weighted sample;
• low-confidence routing;
• new-language oversampling.

If error rates increase, expand review.

Sampling policy should be documented.

QA metrics

Track:

• critical errors;
• major errors per 1,000 words;
• terminology accuracy;
• omission rate;
• reviewer edit rate;
• pass rate;
• turnaround.

Do not chase one aggregate number at the expense of critical-error visibility.

Reviewer calibration

Use shared examples to align reviewers on:

• terminology;
• severity;
• acceptable style.

Recalibrate when:

• model changes;
• domain changes;
• new language launches.

Inconsistent human review produces noisy quality data.

Governance

Enterprise QA needs:

• owners;
• versions;
• logs;
• escalation.

NIST’s AI RMF organizes AI risk management around Govern, Map, Measure and Manage. That structure is useful for translation systems too: define ownership, understand use-case risk, measure quality and manage failures continuously.

Example QA flow for a translated contract

A contract workflow can make the abstract steps concrete. First, the intake process confirms the source version, governing-language clause, target locale and confidentiality requirement. Defined terms and legal entity names are extracted before translation. The AI draft is then checked automatically for clause count, numbers, defined terms and section references.

A bilingual legal-language reviewer compares every operative clause against the source, while a final formatting pass verifies numbering, annexes and signature blocks. Only after those gates does the file receive an approved status. If a critical error is found, the same model/glossary configuration is searched across related documents before the issue is closed.

This is slower than raw AI, but far faster than manually translating every word from scratch while providing a traceable quality process.

QA ownership matrix

A production workflow becomes clearer when each check has an owner:

This prevents the common assumption that “the reviewer” is responsible for every type of defect.

Acceptance evidence

For enterprise work, store a compact QA record:

• source version/hash;
• target locale;
• model and configuration;
• glossary/style version;
• automated checks passed;
• reviewer and review date;
• critical-error status;
• final artifact version.

This evidence is valuable when a client asks how a target was produced or when the source later changes.

Designing exception queues

Human review is most efficient when the system explains why a segment was routed. Examples:

• number mismatch;
• low semantic score;
• forbidden term;
• rare entity;
• high-risk content label;
• source ambiguity.

A reviewer then starts with the suspected issue rather than rereading every clean segment with equal attention. This is one of the most practical ways to turn AI QA into lower review cost rather than simply adding another automated report.

Regression after corrections

When a reviewer fixes a recurring issue, add a representative case to the regression suite. A model or prompt update should not be deployed until the system still handles:

• previous critical incidents;
• core terminology;
• known language-specific pitfalls.

A QA workflow is mature when past failures become permanent tests rather than institutional memory that disappears when staff change.

QA for rapidly changing source content

Continuous localization adds another failure mode: the translation may be perfectly reviewed but already stale. Every target segment should retain a source version or content hash. When the source changes, mark the target as:

• unchanged and still valid;
• automatically reusable;
• needs retranslation;
• needs human re-review.

This source-version gate belongs at the beginning and end of QA. It prevents teams from spending hours reviewing a translation of a paragraph that has already been rewritten upstream.

Service-level QA reporting

For managed translation services, a useful monthly report includes:

• words/pages/minutes processed;
• percentage auto-approved versus reviewed;
• major and critical errors found;
• terminology accuracy;
• top recurring error categories;
• languages below threshold;
• corrective actions.

The report should not expose sensitive source text unless necessary. The value is trend and accountability.

QA staffing model

A scalable team can separate:

• localization operations;
• native/bilingual reviewers;
• specialist reviewers;
• quality lead.

The quality lead owns guidelines, calibration and incident analysis rather than rereading every job. This creates leverage: one quality lead can improve thousands of translations by fixing the system and reviewer guidance.

Final publishing principle

No quality workflow should allow a target to become “approved” merely because every automated job returned HTTP 200. Approval represents a business decision that the artifact has passed the required gates for its use case.

Encode that status explicitly in the CMS, TMS or project database.

Frequently asked questions

Should every AI translation be human reviewed?

No. Review depth should reflect risk and measured system quality.

What automated checks are most valuable?

Completeness, numbers, placeholders, terminology and file structure because they are highly testable.

Which review should happen first: accuracy or fluency?

Accuracy. A fluent mistranslation remains wrong.

Can COMET or an LLM judge replace the QA workflow?

No. They are components within the workflow.

How do you reduce QA cost?

Improve source content, terminology, automated checks and risk-based routing so humans review the right segments.

Conclusion

AI translation QA works best as a controlled production pipeline rather than a final proofreading task.

Validate the source, guide generation, test deterministic constraints, evaluate semantic fidelity, use qualified human review where needed and verify the final artifact. Then capture every useful correction so the next translation starts from a stronger system.

That is how AI speed becomes repeatable translation quality.

Research references

• NIST AI Risk Management Framework:
• NIST AI RMF Playbook:
• WMT 2023 Terminology Shared Task:
• HalOmi (EMNLP 2023):
• WMT 2024 Metrics Shared Task: