Building RAG Over Arabic Script Documents

Arabic RAG fails in three places that have nothing to do with each other: the text arrives in the wrong order, the same word is stored in several different encodings, and the chunker cuts on punctuation it does not recognise. Fixing them in the wrong order wastes the work, because a normalisation pass over scrambled text produces normalised scrambled text.

Three problems, in the order they bite

Arabic is written right to left, which is a rendering property and not a storage property. In a correctly produced file the characters are stored in logical order — the order you say them — and the renderer reverses the run for display. Nothing in a retrieval pipeline should ever reverse anything. If your extracted text needs reversing to look right, the extraction is wrong and reversing it is a second bug layered on the first.

So the order of operations is fixed:

1. Verify order. Confirm the extracted text is in logical order before you touch it. For PDFs this is the whole problem and it has its own page: building RAG over a right-to-left PDF.
2. Normalise. Collapse presentation forms, strip tatweel, decide on a diacritics policy, unify alef and yeh variants.
3. Segment and chunk. Split on Arabic punctuation, pack to a token budget.
4. Embed and index, with a language tag on every chunk.

Presentation forms and ligatures

Arabic letters change shape depending on position in the word: isolated, initial, medial, final. Unicode’s normal model is that you store the abstract letter and the renderer picks the shape. But Unicode also contains legacy presentation form blocks — Arabic Presentation Forms-A (U+FB50–U+FDFF) and Forms-B (U+FE70–U+FEFF) — that encode each shape separately, and text extracted from PDFs, from old databases and from some OCR engines arrives full of them.

The consequence is that the same word is several different strings. The letter beh has one abstract code point, U+0628, and four presentation forms in Forms-B. A document extracted as presentation forms and a query typed in abstract letters share no characters at all — lexical search finds nothing, and the embedding is computed over a sequence the model has barely seen.

Ligatures make it worse. Lam-alef is written as a single joined glyph and is encoded in Forms-B as one code point, so a naive character count is off by one per occurrence and a substring search for the alef fails. The mandatory Arabic ligatures also include the multi-character forms in Forms-A, such as the ones encoding whole words.

before (extracted from a PDF, presentation forms):
  U+FEE3 U+FE8E U+FEDF ...        # medial meem, final alef, initial lam
  len(text) == 43, no U+0645 anywhere

after NFKC:
  U+0645 U+0627 U+0644 ...        # meem, alef, lam
  len(text) == 47, matches a query typed on an Arabic keyboard

The length changing is the tell. Compatibility normalisation decomposes the lam-alef ligature into two letters, so a document whose character count grows under NFKC was carrying ligature code points.

The normalisation step

NFKC does the presentation-form and ligature work because those characters carry compatibility decompositions by design. The rest is orthographic policy you have to choose, and the choice must be applied identically to documents at index time and to queries at search time — an asymmetry here is the most common cause of “the exact phrase is in the corpus and search returns nothing”.

• Tatweel (U+0640). A kashida, inserted purely to stretch a word for justification. It carries no meaning and no pronunciation. Strip it unconditionally.
• Harakat (U+064B–U+0652). Short-vowel and gemination marks. Most modern prose omits them; religious, legal and pedagogical texts include them. If your corpus mixes both, strip them from the search representation or the marked and unmarked spellings of one word will never match.
• Alef variants. U+0623, U+0625, U+0622 and U+0671 all fold to U+0627 in a search-normalised form. Writers are inconsistent about hamza placement, so folding recovers real matches.
• Teh marbuta and yeh. U+0629 versus U+0647, and U+064A versus U+0649, differ by dialect and by keyboard. Folding them is standard practice in Arabic information retrieval.
• Arabic-Indic digits. U+0660–U+0669 are the same numbers as ASCII 0–9. Fold them for search so a query for 2024 matches a document written with ٢٠٢٤.

Chunking Arabic prose

Arabic uses its own punctuation code points, and a segmenter written for English matches none of them:

• ؟ (U+061F) — Arabic question mark, mirrored, not ASCII ?.
• ، (U+060C) — Arabic comma. The clause-level fallback.
• ؛ (U+061B) — Arabic semicolon.
• The ASCII full stop — Arabic does use . as its sentence terminator, which is the one piece of luck here. A regex of [.؟!؛] covers ordinary prose.

Beyond punctuation, Arabic prose is characteristically long-sentenced, with clauses coordinated by و (waw) rather than separated. A single sentence exceeding the chunk budget is normal, not exceptional, so the clause-level fallback fires often. Split on ، first and on a word-initial و only as a last resort — the waw is attached to the following word with no space, so a split there requires cutting inside a whitespace-delimited token, which is legal but easy to get wrong.

Do not set the budget in characters. Arabic tokenises poorly in many vocabularies, so a character budget tuned on English can produce token counts well beyond what you intended — the same unit mismatch worked through for CJK in setting chunk size in tokens, not characters.

The ingest order that works

import re, unicodedata

TATWEEL = "\u0640"
HARAKAT = re.compile("[\u064b-\u0652\u0670]")
ALEF = re.compile("[\u0622\u0623\u0625\u0671]")
DIGITS = {chr(0x0660 + i): str(i) for i in range(10)}

def fold_ar(s):
    s = unicodedata.normalize("NFKC", s)        # presentation forms, ligatures
    s = s.replace(TATWEEL, "")
    s = HARAKAT.sub("", s)
    s = ALEF.sub("\u0627", s)
    s = s.replace("\u0649", "\u064a").replace("\u0629", "\u0647")
    return "".join(DIGITS.get(c, c) for c in s)

SENT = re.compile(r"(?<=[.؟!؛])\s+")

def chunks_ar(text, budget, ntok):
    original = unicodedata.normalize("NFC", text)
    out, cur, n = [], [], 0
    for s in SENT.split(original):
        k = ntok(fold_ar(s))
        if cur and n + k > budget:
            out.append(" ".join(cur)); cur, n = [], 0
        cur.append(s); n += k
    if cur:
        out.append(" ".join(cur))
    # index the folded form, keep the original for display
    return [{"display": c, "search": fold_ar(c)} for c in out]

Two details in that function are load-bearing. The token budget is computed on the folded text because that is what gets embedded, while the chunk boundaries are computed on the original so the displayed text is intact. And the yeh and teh-marbuta folds are applied after the alef fold, because the alef pattern would otherwise not see characters produced by NFKC decomposition.

Finally, tag every chunk with its language and script at index time. An Arabic corpus almost always contains English technical terms, product names and citations, and a mixed index has a ranking problem of its own — see retrieval across a corpus that mixes several languages.