Why Greek Text Costs More Tokens Than English

Greek has an alphabet of twenty-four letters, spaces between words, no combining marks in normal use and morphology no more complex than German’s. It still costs roughly two to three times English, and the reason is not in the language at all.

The arithmetic on a labelled sentence

Αυτή η λειτουργία χρησιμοποιεί πολλή μνήμη.
"This function uses a lot of memory."

Thirty-seven Greek letters, five spaces, one full stop. Greek and Coptic occupies U+0370–U+03FF, inside the two-byte UTF-8 range, so that is 74 bytes of letters plus 6 of ASCII: exactly 80 bytes. The English is 35 bytes, near 9 tokens. The byte floor caps the Greek at 80 tokens.

Derive the middle with the assumption stated. Assume merges exist for the most frequent Greek letter pairs and for a handful of very common short words — και, το, της — but that longer content words break into two to four pieces each. Six words of that shape gives something in the region of 18 to 30 tokens against the English 9, a derived multiplier of roughly 2× to 3×.

Now look at what produced that. Greek used 37 letters where English used 29; that is a mild penalty, and Greek words are longer than English ones but not dramatically so. Two bytes per letter doubles the byte count. And then the tokenizer roughly doubles it again, because it can turn function into one token, with its leading space attached, and cannot do the same for λειτουργία. The script is fine. The vocabulary is not.

The cost is the vocabulary, not the alphabet

This page exists to isolate that claim, because Greek is the cleanest case for it in the whole cluster. Every other language here has a structural excuse: three-byte code points, combining marks, missing spaces, agglutination, decomposition. Greek has none of them. It is a left-to-right alphabetic script with spaces and precomposed letters, which is to say it is structurally the same kind of thing as Spanish.

What Greek does not have is corpus share. A merge table has a fixed number of slots — roughly a hundred thousand in cl100k_base, roughly two hundred thousand in o200k_base, per OpenAI’s tiktoken encodings — and those slots are allocated by frequency in a training corpus that is overwhelmingly English. Greek is spoken by roughly thirteen million people and its share of that corpus is correspondingly small, so it earns letter-pair merges and not word merges.

There is a curious wrinkle that flatters the individual letters without helping the language: Greek letters appear constantly in English-language mathematics, physics and code as symbols — α, β, π, μ, Σ, Δ, λ. Those uses are frequent enough that individual Greek letters are well covered as single tokens. It does nothing for Greek prose, because the merges that matter for prose are letter sequences, and a physics paper does not contain the sequence ουργία.

The practical implication is that Greek’s multiplier is the most model-dependent in this cluster. Changing model changes the number, sometimes substantially, in a way that changing model does not change the three-byte width of a Han character.

The nearest comparison is Russian, which is also alphabetic, also two bytes a letter, and also structurally simple — and which lands at a friendlier multiplier for one reason only: there is far more Russian than Greek in any web-scraped corpus. Set the two side by side and the vocabulary-share argument stops being a hypothesis and becomes the only remaining variable. The same holds within Cyrillic: Bulgarian shares Russian’s alphabet and not its corpus, and pays for it.

Final sigma and the accented vowels

Two places where the script does contribute. First, sigma has two lowercase forms: σ in any position, and ς word-finally. They are distinct code points, U+03C3 and U+03C2. Because a large share of Greek words end in sigma, this means the word-final letter of a great many words is a code point that appears nowhere else, splitting what would otherwise be a single set of word-ending merges into two.

Second, monotonic Greek marks stress with a tonos on the vowel, and the accented vowels are precomposed single code points — ά U+03AC, έ U+03AD, ή U+03AE, ί U+03AF, ό U+03CC, ύ U+03CD, ώ U+03CE — distinct from their unaccented forms. Stress falls on one of the last three syllables of almost every polysyllabic Greek word, so a large fraction of content words contain at least one of these. The tokenizer therefore sees the accented and unaccented spellings of the same stem as unrelated byte sequences, and has to earn merges for both.

The same effect explains why stripping accents is such a common preprocessing step in Greek search: it collapses two spellings into one and improves matching. It also changes the token count, so if you strip accents for indexing and not for generation, your two paths will disagree about cost. The transliteration case, where Greek is written in Latin letters entirely, is covered in Greeklish.

Polytonic Greek is a different problem

Everything above is modern monotonic Greek. Ancient, Byzantine and pre-1982 modern Greek use the polytonic system: acute, grave and circumflex accents, smooth and rough breathings, and the iota subscript, in combinations that Unicode encodes in the Greek Extended block U+1F00–U+1FFF.

That block is above U+0800, so every polytonic character takes three UTF-8 bytes rather than two — a 50% byte increase over monotonic Greek before any tokenization argument. And polytonic text is a vanishingly small share of any web-scraped corpus, so merge coverage is close to nothing and the fallback is close to the byte floor. If you are building anything over classical texts, the New Testament, or scholarly editions, the multiplier that applies to you is several times worse than the modern-Greek figure derived above, and the same text normalised to monotonic would cost substantially less. Whether that normalisation is acceptable is a philological question, not a technical one.

Measuring it

import unicodedata
import tiktoken

enc = tiktoken.get_encoding("o200k_base")

def strip_accents(s):
    d = unicodedata.normalize("NFD", s)
    return unicodedata.normalize("NFC", "".join(
        ch for ch in d if not unicodedata.combining(ch)))

el = "Αυτή η λειτουργία χρησιμοποιεί πολλή μνήμη."
en = "This function uses a lot of memory."

for label, s in (("greek", el),
                 ("greek, accents stripped", strip_accents(el)),
                 ("english", en)):
    print(label, "chars", len(s),
          "bytes", len(s.encode("utf-8")),
          "tokens", len(enc.encode(s)))

The gap between the first two rows is the accent penalty, and it is the one part of Greek’s cost you can change without changing model. The gap between the second and third rows is the vocabulary penalty, and you cannot change that at all except by choosing a different tokenizer.