The Turkish Dotted and Dotless I Problem in AI Text

“TITLE”.toLowerCase() returns “tıtle” on a machine configured for Turkish, and the comparison against “title” that worked everywhere else now fails. The same mechanism, running the other way, turns Turkish words into misspellings in AI-generated text.

Turkish has four I letters

English has two forms of one letter. Turkish has two distinct letters, each with two forms, and the dot is a phonemic distinction rather than a typographic flourish:

dotted    : İ  U+0130    i  U+0069     front vowel
dotless   : I  U+0049    ı  U+0131     back vowel

so:  upper("i") = "İ"      lower("İ") = "i"
     upper("ı") = "I"      lower("I") = "ı"

Compare that with the default Unicode case mapping, which is what every runtime uses unless told otherwise: upper(“i”) = “I” and lower(“I”) = “i”. The default rules and the Turkish rules disagree on both mappings, in both directions. This is one of the few places where Unicode explicitly ships a language-conditional case mapping, recorded in the Unicode character database’s SpecialCasing.txt under the tr and az conditions.

Azerbaijani uses the same pair and has the same behaviour, so any fix keyed on tr alone is half a fix.

The bug, in both directions

Turkish rules applied to protocol strings

This is the famous one. A runtime whose default locale is Turkish applies Turkish casing to every string, including strings that are not Turkish:

// Java, on a machine whose default locale is tr-TR

"TITLE".toLowerCase()                        -> "tıtle"
"ID".toLowerCase()                           -> "ıd"
"IMAGE/PNG".toLowerCase()                    -> "ımage/png"
"INFO".toLowerCase().equals("info")          -> false

// and the same rule fires the other way on uppercase:
"image/png".toUpperCase()                    -> "İMAGE/PNG"
"image/png".toUpperCase().equals("IMAGE/PNG") -> false

// every one of these is a header, a MIME type or a key lookup
// that resolves correctly on every machine except a Turkish one.

Any code that lowercases an identifier before comparing it — an HTTP header name, a config key, a file extension, a CSS property, an enum name parsed from JSON, a SQL keyword — is a candidate. The symptom is a system that works perfectly in testing and fails only for users in Turkey, which makes it one of the hardest classes of bug to reproduce from a report.

Default rules applied to Turkish text

The reverse is the one that shows up in AI output and in text pipelines. Take the Turkish word ışık (light). Uppercase it with the default mapping and you get IŞIK — which happens to be right. Now take iyi (good): the default mapping gives IYI, and the correct Turkish uppercase is İYİ. Lowercase ISTANBUL with the default mapping and you get istanbul; the correct Turkish result is ıstanbul, which is not a word — the city is İstanbul and only the correct uppercase form round-trips.

Language models produce this constantly, because title-casing and uppercasing patterns in training data are overwhelmingly English-shaped. It also appears wherever a pipeline lowercases text before embedding or indexing it: a Turkish corpus lowercased with default rules has every İ mangled before the model ever sees it.

Lowercasing one character can produce two

Here is the detail that breaks code written by people who already know about the Turkish I. Under the default Unicode mapping, İ (U+0130) does not lowercase to i. It lowercases to two code points:

"İ"                      -> U+0130                        length 1
"İ".lower()   (Python)   -> U+0069 U+0307                 length 2
                            i + COMBINING DOT ABOVE

"İ".lower() == "i"       -> False
len("İ".lower())         -> 2

"İ".casefold()           -> U+0069 U+0307                 length 2
"İ".lower(tr)            -> U+0069                        length 1  (locale-aware)

The default mapping preserves the dot as a combining mark, because discarding it would lose information for languages that need it. The consequence is that a string can get longer when lowercased, which violates an assumption buried in a great deal of string-handling code: fixed-size buffers, offset arithmetic in a search highlighter, a VARCHAR width, an assertion that case conversion preserves length. And the resulting i plus combining dot compares unequal to a plain i unless you also normalize.

The same expansion shows up in slug generation, where it is particularly hard to spot. A pipeline that lowercases a Turkish article title and then strips everything outside [a-z0-9-] will silently delete the combining dot, turning İstanbul into istanbul on one machine and into stanbul on another depending on whether the strip runs before or after normalization. Two servers in the same fleet then generate two different URLs for one article. Normalize to NFC after case folding and before the strip, and the ambiguity disappears.

It matters in identifiers for a second reason as well. ı and i are visually confusable in many fonts, especially at small sizes, and internationalised domain names and usernames are precisely where a confusable pair is a security problem rather than a cosmetic one. That is one of the arguments for comparing identifiers under compatibility normalization and a confusable-skeleton check rather than under a bare lowercase — see NFC and NFKC normalization for which form belongs in an identifier comparison.

The fix in each runtime

There are two operations and they must not be confused. Case conversion for display is locale-dependent and must be told which locale. Case conversion for comparison must be locale-independent, and preferably is not case conversion at all but case folding.

• Java. toLowerCase() with no argument uses the default locale, which is the entire bug. Use toLowerCase(Locale.ROOT) for protocol strings and toLowerCase(new Locale(“tr”)) for Turkish display text. Better still, use equalsIgnoreCase for comparisons and avoid producing the lowercased string at all.
• .NET. ToLower() is culture-sensitive. Use ToLowerInvariant() for identifiers, and compare with StringComparison.OrdinalIgnoreCase rather than lowercasing first.
• JavaScript. toLowerCase() is locale-independent by specification, so the protocol direction of this bug does not occur — but for that same reason it is wrong for Turkish display text. Use toLocaleLowerCase(“tr”) there.
• Python. str.lower() and str.casefold() are not locale-aware, so the same split applies: they are safe for identifiers and wrong for Turkish. Use PyICU or an explicit pre-substitution of İ to i and I to ı before folding.
• SQL and search engines. Check the collation. A Turkish collation applied to a column of enum values reproduces the Java bug inside the database, where it is even harder to see.

The rule that prevents it

Never use a case conversion whose behaviour depends on ambient configuration. Every call site is one of two things, and both are explicit:

1. Comparing machine strings — header names, keys, extensions, enum values. Use an invariant or ordinal comparison, or Unicode case folding. Never the default locale, and ideally never a lowercased intermediate string at all.
2. Presenting text to a human — a title, a heading, a name. Pass the content’s language explicitly, which means you must know it; if you do not, that is a language-resolution problem and not a string-handling one.
3. Normalize after folding, not before. Because folding can emit combining marks, the NFC pass belongs at the end of the chain. The ordering rules are in NFC and NFKC normalization.
4. Add a Turkish string to the test suite. One assertion that “IDs” still lowercases to “ids” with the default locale forced to tr-TR catches every instance of this bug in your codebase in one run.