Persian Punctuation Placement in Right-to-Left Text

“The question mark ended up on the wrong side” is nearly always a statement about the paragraph, not about the question mark. The way to see that clearly is to take one Persian sentence with an English term in it and run the algorithm by hand.

The Persian punctuation characters

Persian uses the Arabic script with its own additional letters, and it uses a punctuation set that is visually mirrored from the Latin one. The characters that matter here:

• U+060C ARABIC COMMA — the comma, drawn as a high-set mark curving the opposite way from the Latin one. Persian text should use this, not U+002C.
• U+061B ARABIC SEMICOLON and U+061F ARABIC QUESTION MARK — the semicolon and question mark, again mirrored.
• U+066B ARABIC DECIMAL SEPARATOR and U+066C ARABIC THOUSANDS SEPARATOR — the numeric separators, distinct from the comma and full stop.
• U+200C ZERO WIDTH NON-JOINER — not punctuation, but structurally essential in Persian for plural and verbal prefixes. A pipeline that strips zero-width characters as a hygiene measure silently changes Persian words.
• The full stop is U+002E, shared with Latin. There is no separate Arabic-script period.

One adjacent trap belongs here because it arrives in the same output and looks like a punctuation problem. Persian and Arabic use different code points for two letters that are drawn almost identically: Persian kaf is U+06A9 and Arabic kaf is U+0643; Persian yeh is U+06CC and Arabic yeh is U+064A. A model trained on far more Arabic than Persian will emit the Arabic forms inside otherwise correct Persian, and the text renders indistinguishably while failing every exact-match search, index lookup and deduplication against text typed on a Persian keyboard. Fold U+0643 to U+06A9 and U+064A to U+06CC on the way into any index, and keep the original for display if provenance matters.

Their bidi classes, which are not uniform

This is the part that explains the behaviour, and it surprises people who expect all Arabic-script punctuation to be classed together. The per-character assignments live in DerivedBidiClass.txt in the Unicode Character Database, and they differ:

• The Arabic question mark and semicolon are classed as strong Arabic letters, class AL. They carry direction themselves, so they do not need context to resolve — which is exactly why they exist as separate code points rather than being handled by mirroring.
• The Arabic comma is classed CS, a common separator, which is a weak class that only becomes numeric glue between two numbers and is otherwise resolved as a neutral.
• The ASCII full stop is CS too, and the ASCII question mark is ON, other neutral. Both take their direction entirely from context.

So swapping a Latin question mark for U+061F does not merely change the glyph — it changes a context-dependent neutral into a strong right-to-left character, and that changes how the characters around it resolve. That is the single most useful fact on this page.

One sentence, traced

Uppercase stands for right-to-left characters, following UAX #9’s own notation. The sentence is Persian, contains the English product name Redis, and ends with a Latin question mark.

storage (logical order):
  PERSIAN Redis PERSIAN?

classes:
  AL...AL  WS  L L L L L  WS  AL...AL  ON

P2/P3: first strong character is AL -> paragraph level 1 (RTL)

X rules: no explicit formatting characters, nothing to do
W rules: nothing numeric, nothing to do

N1: the WS between AL and L  -> strong types differ, N1 no
N2: -> paragraph level, R
N1: the WS between L and AL  -> differ, N2 -> R
N1: the final ON is between AL and end-of-paragraph (R)
    -> same direction, N1 applies -> R

I1/I2: at odd level 1, class L is raised to level 2
levels:
  1 1 1 1  1  2 2 2 2 2  1  1 1 1 1  1

L2: reverse each maximal run from the highest level down.
    level 2 run "Redis" stays as it is (even = LTR)
    the whole level-1 line is reversed

display (visual order, left to right):
  ?NAISREP Redis NAISREP

The question mark is at the far left of the display line, and that is correct: in a right-to-left line, the end of the sentence is the left end. The English word reads left-to-right, in place, because it was raised to an even level by I2. Nothing needed a control character.

Where output actually goes wrong

Run the same string with the paragraph level forced to 0 — which is what happens when the container is an LTR page, an untagged <div>, a plain-text field, or a PDF with no direction — and every resolution above changes. The final question mark now resolves by N2 to the paragraph direction, left-to-right, and lands at the right-hand end of the line, after the Persian rather than before it. The Persian runs are reversed relative to each other. The text is byte-identical. The container was the variable.

Two more genuine failure modes, both worth checking before you reach for control characters:

• The model emitted Latin punctuation. A comma at U+002C in Persian prose is a typographic error before it is a bidi one — it will resolve acceptably in a correctly-directed paragraph and it is still the wrong character. Asking for U+060C, U+061B and U+061F by code point in the prompt fixes it, and the check is a regular expression over the response.
• The sentence starts with the Latin term. Persian word order regularly puts a borrowed product name first. With dir="auto" or FSI, the first strong character is Latin, the paragraph is detected as left-to-right, and the whole sentence lays out wrongly. Set direction from the language you requested rather than from the first glyph; the reasoning is worked through in formatting RTL text in a chat interface.

Persian digits behave unlike Arabic ones

Persian writes its numerals with the Extended Arabic-Indic digits U+06F0 to U+06F9, which are visually similar to but distinct from the Arabic-Indic digits U+0660 to U+0669 used for Arabic. Four and six differ in shape between the two sets, and they are separate code points, so a naive string comparison between a Persian and an Arabic rendering of the same number fails.

They also differ in bidi class. The Arabic-Indic digits are class AN. The Extended Arabic-Indic digits are class EN, the same class as ASCII digits — which then gets converted to AN by rule W2 whenever the preceding strong character is an Arabic letter. In ordinary Persian prose the two paths converge, so you will not notice. They diverge in a paragraph whose preceding strong character is Latin, where Persian digits stay EN and interact with a following % or currency sign as European numbers do. It is an edge case, it is real, and it is why a number formatter should be given a locale rather than a hand-written digit mapping. The equivalent question for which digit shapes to emit at all is Arabic-Indic numerals in AI output.