Why AI-Generated Filenames Break on Accented Characters

A model produces the filename Résumé_Aoû.pdf. Your service writes it, stores the string in a database, and later calls open() with the stored name. ENOENT: no such file or directory. The file is listed in the directory, with what looks like exactly that name, and it will not open.

The symptom

The giveaway is that a byte-level listing disagrees with a visual one. Two commands, same directory:

$ ls
Résumé_Aoû.pdf

$ ls | hexdump -C | head -2
00000000  52 65 cc 81 73 75 6d cc  81 5f 41 6f cc 82 75 2e  |Re..sum.._Ao..u.|
00000010  70 64 66 0a                                       |pdf.|

# 52 65 CC 81 — that is "R", "e", then U+0301 COMBINING ACUTE.
# The name on disk is decomposed. The string in your database
# almost certainly is not:
#   "Résumé_Aoû.pdf" in NFC = 52 c3 a9 73 75 6d c3 a9 5f 41 6f c3 bb 2e ...

Fourteen bytes for Résumé_Aoû in NFC, seventeen in NFD. Three accents, one extra byte each. A filesystem that compares path components as byte strings sees two different names, and the fact that a terminal draws them identically is not its problem.

What each filesystem does with the bytes

• Linux (ext4, XFS, btrfs). A filename is an opaque byte string with exactly two forbidden bytes: 0x00 and 0x2F (the slash). No normalisation, no case folding. NFC and NFD forms of the same name are two separate files that can sit in one directory looking identical.
• Windows (NTFS). Filenames are UTF-16 code units, case-insensitive by default and normalisation-sensitive. Same outcome as Linux for this bug: two files, one apparent name.
• macOS, historically (HFS+). The filesystem normalised every name it was given to a decomposed form, close to NFD but with documented exceptions. You wrote NFC; you read back NFD. This is the origin of the entire class of bug and the reason NFD filenames circulate at all.
• macOS, currently (APFS). APFS stores what it is given rather than rewriting it, and the volume formats macOS creates by default are normalisation-insensitive: both forms resolve to the same file, and the first one written is the one you get back. So round-tripping works on a Mac and breaks the moment the name crosses to another machine.

The asymmetry is what makes this hard to catch. A team developing on macOS and deploying on Linux gets a system that works on every laptop and fails in production, and a team doing the reverse gets one that fails only for the designer.

Where the mismatch gets in

There is no single culprit, which is why the bug survives one fix. Every one of these is a boundary where the form can change:

• The model output itself. Nothing constrains a language model to emit NFC. It produces whatever its tokeniser decodes to, and both forms exist in training data, so a generated filename can arrive decomposed even on a Linux host.
• Browser form submission. HTML form submission is specified to normalise text to NFC before sending, so a name typed into a browser arrives composed. A name arriving through a JSON API has had no such treatment.
• The directory listing. readdir() returns what is on disk. If you build a name from a listing on one machine and use it on another, you have transported the form.
• Byte-length limits. ext4 caps a filename at 255 bytes, not characters. A 200-character French name that fits in NFC can exceed the cap in NFD, and the error is ENAMETOOLONG rather than anything mentioning encoding.

Git, zip and object storage

The three places the wrong form is most often preserved and shipped.

Git stores path bytes in the tree object, so a decomposed path committed on a Mac is a decomposed path everywhere. Git has a specific setting for this: core.precomposeunicode, which makes Git convert decomposed names it reads from the filesystem into precomposed form. It has defaulted to true on macOS since Git 1.8.5, which is why the problem is rarer than it was, and why a repository that predates that or was created with it off still carries decomposed paths. The visible symptom is a file that git status shows as both deleted and untracked, with the same name twice.

Zip archives store the name bytes plus, optionally, a flag saying they are UTF-8. Nothing in the format records a normalisation form. An archive built on a Mac from decomposed names extracts to decomposed names on Linux, and a manifest generated separately from composed strings then matches nothing.

Object storage keys are the least forgiving, because there is no directory to list visually and no filesystem to be insensitive on your behalf. An S3 key is a byte string. Upload with a decomposed key, request the composed one, and you get a 404 with no hint that the object exists. The same applies to the filename* parameter of a Content-Disposition header, which is percent-encoded UTF-8 under RFC 5987 and will faithfully transport whichever form you gave it.

The fix

1. Pick NFC and write it down as a rule. NFC is the correct default: it is what the Web Platform specifies for form submission, what most text on the web already is, and the shorter of the two in bytes.
2. Normalise where the name is created, not where it is used. The instant a model, an upload or a user hands you a filename, call normalize("NFC") on it and never handle the original again.
3. Separate the storage key from the display name. Store the composed display name in a column for showing to people, and generate the on-disk or in-bucket key from an identifier that has no interesting characters in it at all. This removes the whole class of problem rather than one instance of it, and it also disposes of path traversal, case-insensitivity collisions and length limits.
4. If you must keep human-readable keys, normalise on both the write and the read path, in the same function, and route every filesystem or bucket call through it.
5. When comparing a name you hold against a directory listing, normalise both sides at the point of comparison. Never assume the listing matches what you wrote, even if you wrote it a millisecond earlier on the same machine.
6. Check git config core.precomposeunicode on macOS workstations, and add a repository test that fails on any path containing a code point in U+0300–U+036F.
7. Serve downloads with a Content-Disposition header carrying both the ASCII filename fallback and the UTF-8 filename* form, and normalise the latter to NFC as well.

The underlying rule is the same one that governs every canonical-equivalence bug: normalise once at the edge, and let nothing behind that edge care. The filesystem case only feels different because the boundary is a system call rather than a database write.