Rendering Mixed Arabic and English Text Without Bidi Bugs

Almost every mixed Arabic-English rendering complaint reduces to one question: a character that has no direction of its own — a full stop, a colon, a bracket, a space — sat between two runs that disagree, and the algorithm had to pick a side. The rules that pick are short, and once you can run them by hand the bugs stop being mysterious.

The two failures that account for most reports

The first is the wandering full stop. An Arabic sentence ends with an English product name, then a period. On screen the period appears at the far left of the line, apparently belonging to nothing, or in the middle of the sentence. Nothing is corrupted; the period is stored exactly where it was written.

The second is the split bracket pair. An Arabic phrase contains a parenthesised English aside, and the opening and closing parentheses end up on the same side of the aside, or facing the wrong way. Again the storage is fine.

Both come from the same place. Characters like ., ,, :, (, ), " and the space have no inherent direction — UAX #9 classes them as neutrals or as separators that later become neutral. The algorithm resolves them from context, and where the context is contradictory it falls back to the paragraph direction. If your paragraph direction is wrong, or absent, every neutral in it is decided by that wrong answer, and you will chase twenty symptoms that all have one cause.

How a neutral picks a side: N1 and N2

After the weak rules have run, every remaining neutral is resolved by two rules from Unicode Standard Annex #9:

• N1 — a run of neutrals between two strong characters of the same direction takes that direction. Numbers count as right-to-left for this purpose when the surrounding text is Arabic, which is why W2 mattered on the numbers page.
• N2 — anything left over takes the embedding direction, which for a top-level paragraph is the paragraph direction.

The start and end of the paragraph count as strong characters of the paragraph direction for N1’s purposes. Work the wandering full stop through it, uppercase for right-to-left:

paragraph direction: RTL
storage:  ARABIC WORDS Multigrid.
          [--- R ---] [-- L --][.]

The period sits between an L run and the end of paragraph (R).
Different directions, so N1 does not apply.
N2 gives it the paragraph direction: R, level 1.
The English run is at level 2.

display:  .SDROW CIBARA Multigrid
          ^ the period is now at the far left

That is correct behaviour. In an Arabic paragraph the sentence-final period belongs at the left end of the line, because that is where an Arabic sentence ends. The complaint usually arrives from someone reading the English word as the end of the sentence. If the sentence is genuinely English with an Arabic quotation in it, then the paragraph direction is wrong and that is the fix — not a control character.

Brackets get their own rule: N0

N1 and N2 treat each neutral independently, which produced the split bracket pair: one parenthesis had Arabic on both sides and became right-to-left, the other had English on one side and became left-to-right, and the pair no longer enclosed anything. Unicode 6.3 added rule N0 and the paired-bracket algorithm (BD16) to fix exactly this: matching bracket pairs are found first, and both members are resolved together, taking the direction of the strong text inside the pair if there is any, and the surrounding context otherwise.

Two consequences follow. First, a pair only resolves as a pair if it actually matches — an unbalanced parenthesis, common in model output that got truncated, falls back to N1/N2 and reintroduces the old bug. Second, the brackets are also mirrored at display time. The Bidi_Mirrored property, listed in BidiMirroring.txt, tells the renderer to draw U+0028 with the glyph of U+0029 when it is at an odd level. You always store the logical opening bracket. Never swap them yourself to “fix” the display, because the renderer will swap them again and you will have a file that is wrong everywhere except the one place you tested.

Isolates, embeddings, overrides and marks

There are four tools and they are not interchangeable. In descending order of how often you should reach for them:

• Isolates — U+2066 LRI, U+2067 RLI, U+2068 FSI, terminated by U+2069 PDI; in HTML, <bdi> or unicode-bidi: isolate. The enclosed run is opaque to the outside and the outside is opaque to it. This is what you want essentially always. FSI is the useful one for untrusted content: it picks direction from the first strong character inside, the same heuristic as HTML’s dir="auto".
• Embeddings — U+202A LRE, U+202B RLE, U+202C PDF; unicode-bidi: embed. Sets a direction but leaves the boundaries interacting with neighbours. Predates isolates and is deprecated in the HTML specification in their favour.
• Overrides — U+202D LRO, U+202E RLO; unicode-bidi: bidi-override. Forces a direction on every character regardless of its class. Almost always wrong for text; RLO in particular is the character used in filename-spoofing attacks, so plenty of systems strip it.
• Marks — U+200E LRM, U+200F RLM, U+061C ALM. Zero- width strong characters that give a neighbouring neutral something to attach to. Useful as a surgical last resort in plain-text formats with no markup. They are invisible characters in your data, so keep them out of anything you will later compare, hash or index.

What to ask the model to emit

A model asked for “Arabic text with English terms” will emit the characters in logical order, which is correct, and will not emit any direction metadata, which is also reasonable — the metadata belongs to the container, and the model does not know what the container is. Rendering is your job, and the instruction that reliably helps is to say what format you are rendering into.

Return Arabic prose. Any Latin-script term, product name, URL,
code identifier or number must be wrapped in <bdi> tags. Do not
insert Unicode bidi control characters (U+200E, U+200F, U+202A
through U+202E, U+2066 through U+2069) into the text. Use only
<bdi>. Return the fragment with no surrounding <html> or <body>.

Naming the control characters you do not want is worth the tokens. Some models will otherwise emit LRM and RLM directly, which renders correctly and leaves invisible characters in your database that will break an equality check months later. Wrap the whole response in a container that declares direction — dir="rtl" on the block, not on the page — and set the paragraph direction from the language you asked for, not from the response text.