Workflow guides · May 29, 2026 · Frank Wei

Why whole-document LLM translation hallucinates — and how to reduce it

Research on document-level machine translation shows fluent but unfaithful output, rising omissions at long context, and why segmenting plus a second-model QA pass helps.

LSP and language teams love the idea of uploading a 40-page client contract and getting back a perfect translation in one click. The output often reads well — and that is exactly what makes hallucinations dangerous. Recent research on document-level machine translation (DocMT) shows that large language models (LLMs) can produce fluent text that is not faithful to the source: invented sentences, dropped clauses, wrong idioms, and terminology that drifts from page 1 to page 30.

This article summarizes what the literature says, walks through annotated real-world examples, and explains how sensible segmentation plus a second, independent review model — the architecture behind BabelBee’s AI Reviewed and Team Verified tiers — maps to what actually works.

What researchers mean by “hallucination” in translation

In machine translation, “hallucination” is not science fiction. It means the model output is semantically unfaithful to the source while still looking grammatical and professional.

A 2025 survey in Information groups common failure modes as:

Fabrication — content in the translation that never appeared in the source
Omission — source material left out
Semantic drift — meaning shifts subtly; the sentence “works” but is wrong
Wrong substitutions — especially names, numbers, and terms
Domain mismatch — inserting content from the wrong domain (e.g. legal boilerplate where none existed)
Wrong language — output in a language the user did not ask for

LLMs can be more fluent and more paraphrastic than traditional neural MT, which helps readability but hurts traceability. For contracts, consent forms, and filings, traceability is the product requirement.

Real examples from human evaluation studies

The following patterns come from professional annotators evaluating GPT-style document translation (arXiv:2407.03658, WMT 2023 literary DocMT paper):

Example 1: Fluent but unrelated (classic hallucination)

Source (EN): You can send us a follow-up email at this address [EMAIL].

Machine translation (ES): Hágame saber si tiene alguna otra pregunta. (“Please let me know if you have any other questions.”)

The Spanish is grammatical. It is also not a translation of the source.

Example 2: Addition (over-translation)

Source (EN): That way you can be sure that you were the one who made the changes.

Machine translation (ES): …fuiste tú quien hizo todos los cambios. (“all” the changes)

The word all is not in the English. In policy or contract text, that is a material change.

Example 3: Idiom rendered literally (mistranslation)

Source (EN): It has to be done by the book.

Machine translation (FR): Il doit être fait par le livre. (“by the book” as a physical book)

The model missed the idiom by the book (follow the rules).

Example 4: Inconsistency across the same document

Source (NO): Please click on this link. … This link will expire in 24 hours.

Machine translation: …denne lenken … Denne linken utløper…

Both lenke and link are valid Norwegian for “link,” but one document should pick one. Whole-document single-pass translation often drifts because later paragraphs do not “see” earlier lexical choices in a controlled way.

Why “paste the whole document into ChatGPT” makes it worse

Counter-intuitively, longer context is not always better for translation quality.

Omissions and additions rise with document length

NVIDIA’s SEGALE study (EMNLP 2025) evaluated open-weight LLMs on book-length translation. Their main finding:

Many models cannot translate documents at their reported context length, because the number of under- and over-translation errors rises sharply as the input length grows.

They segmented books into 1k, 2k, 4k, and 8k-token chunks. For most models, quality degraded as chunks grew. Only a few models (e.g. Qwen2.5-72B-Instruct in their setup) stayed relatively stable up to about 4k tokens.

Takeaway: The context window on the spec sheet is not the same as “reliable at that length for translation.”

Paragraph-level context does not fix everything

WMT 2023 work on literary DocMT found that GPT-3.5 does use paragraph context and can beat sentence-by-sentence baselines — but omissions remain, especially at paragraph scope. GPT-4 helps but does not remove mistranslations and grammar issues.

So: document context helps coherence; it does not eliminate hallucination risk.

Single-pass “translate everything” encourages summarization

When input is very long and the model must produce a long target, pressure increases to compress, skip, or rephrase rather than translate segment by segment. That shows up in literary MT evaluations as high omission rates for some models (e.g. Gemini omitting large blocks in Chinese–English literary translation studies reported in 2026).

What the research says about fixing it

Two families of techniques show up repeatedly in ACL/EMNLP/NAACL papers from 2023–2025.

1. Segment intelligently — not “one prompt for 80 pages”

SubDocTrans (EMNLP 2025 Findings) proposes:

Topic-aware segmentation into sub-documents (not arbitrary 8k-token chunks)
Overlapping context so pronouns and discourse links survive boundaries
Multi-granularity prompts: bilingual summary, themes, proper nouns, transition hints
Sentence alignment after translation to catch omissions

They report better consistency, fewer omissions, and mitigated hallucinations versus strong DocMT baselines.

CAP / dynamic context (ACL 2024 Findings) makes a related point: feed the model only the context that matters for the current sentence, instead of filling the entire window with noise.

Engineering translation: Treat a Word/PDF contract as many aligned segments with structure preserved (headings, tables, list items), not one blob of text. Each segment gets focused attention; reviewers can see source ↔ target side by side.

2. Use a second model for review — but not “the same model fixing itself”

Pride and Prejudice (ACL 2024) measured self-bias in LLM self-refinement: when a model critiques its own output, it often prefers its own style and does not reliably improve translation quality. External feedback — a separate scorer or reviewer — reduces bias and can improve results.

Hallucination-focused preference optimization (NAACL 2025) shows that training with preference data targeting hallucinations can cut hallucination rates sharply — but that is model-training research. For product teams today, the practical pattern is:

Model A translates
Model B reviews (different model or different prompt role)
Humans sign off on flagged spans for high-stakes work

GPT-4 post-editing (EMNLP 2023 Findings) improved strong NMT outputs on WMT-22 pairs, but authors warn that post-editing can also hallucinate — so the reviewer should flag and constrain, not freely rewrite.

Best practice from the literature: second pass outputs structured findings (omission, number mismatch, terminology conflict, risky span) rather than a full hidden re-translation.

How this maps to BabelBee

BabelBee is built around the failure modes above — not around “bigger context = better.”

Research finding	BabelBee practice
Long single-pass → more under/over-translation	Documents translated as segments with layout preserved; each unit has a visible source/target pair
Terminology drifts across pages	Shared glossary + translation memory (TMX) recalled into each job; approved segments can write back to TM
Self-refine is weak	Second AI model runs QA on eligible tiers — not the same pass marking its own homework
Reviewers need traceability	Quality report with categorized issues; Team Verified requires approve/waive on flagged segments before export
High-stakes needs humans	Workbench editing, notes, roles (Owner / Reviewer / Admin); AI assists, does not replace sign-off

AI Fast vs AI Reviewed vs Team Verified

AI Fast — speed for internal understanding; accept higher hallucination risk
AI Reviewed — adds second-model QA + quality report (aligned with “external feedback” in the literature)
Team Verified — same QA signals plus mandatory human decisions on flagged segments (aligned with human-in-the-loop DocMT recommendations)

This is why “upload and pray” differs from “upload, QA, and ship.” The second path is slower; it is also what regulated client work and production QA workflows expect.

Frequently asked questions

Does a larger context window eliminate translation hallucinations?

No. SEGALE and multiple DocMT studies show more omissions and additions as effective input length grows for many models. Larger windows help coherence up to a point; they are not a substitute for segmentation and QA.

Is asking the same LLM to “check your translation” enough?

Often not. Pride and Prejudice (ACL 2024) shows self-bias in self-refinement loops. A separate review pass (different model or a dedicated QA role) is more reliable.

What should we check first in a machine-translated contract?

Prioritize omissions, numbers/dates/units, defined terms, and obligations (shall/must/may). Those match the error types annotators mark most often in GPT-4 document studies and the checks BabelBee surfaces in quality reports.

Can BabelBee replace a certified human translator?

No. BabelBee is an AI-native production platform for AI-assisted translation and team review. It does not issue stamped certified translations. For filings, confirm what the receiving authority accepts.