Handling the Spanish Ñ in Text Processing Pipelines

Ñ is the fifteenth letter of the Spanish alphabet. It is not n with a tilde on it in the sense that é is e with an accent on it — it is a separate letter with its own name, eñe, its own dictionary section and its own position in the alphabet. Pipelines that assume otherwise produce two distinct and equally visible bugs.

Two different bugs, one character

They get reported together and they have nothing to do with each other, so fixing one and declaring victory is common.

• The sort bug. A list of Spanish words sorted by code point puts every word containing ñ (U+00F1) after every word containing z (U+007A), because 0xF1 is greater than 0x7A. The symptom is a name list whose ñ entries are all bunched at the bottom, after Zúñiga.
• The folding bug. An accent-stripping pass built for French runs ñ through NFD, sees a combining tilde, drops it, and produces n. The symptom is a search for año that returns documents about ano, which are different words and one of them is anatomical.

Both come from the same false premise — that ñ is a decorated n — but the fixes live in different parts of the system: one in the collation, one in the analyser.

Folding merges words that are not the same

Because ñ does have a canonical decomposition into n plus U+0303, the generic strip loop that works correctly for French destroys it silently. Labelled pairs it merges:

año      / ano       year         / anus
campaña  / campana   campaign     / bell
caña     / cana      cane, reed   / grey hair
peña     / pena      rock, club   / sorrow, penalty
moño     / mono      bun (hair)   / monkey
sueño    / sueno     dream        / (not a word)
uña      / una       fingernail   / a (feminine article)
cuñado   / cunado    brother-in-law / (not a word)

>>> unicodedata.normalize("NFD", "ñ")
'n\u0303'                     <- a combining mark, so the strip loop takes it

The first two rows are the ones that get an incident report written about them. A newsletter subject line, an auto-generated summary, or a search result snippet where año became ano is not a subtle internationalisation nit; it is a visible mistake in front of customers.

The rule is that the ASCII fold must have an exception list, and ñ is on it. Spanish has exactly one such letter — the accented vowels á é í ó ú and the ü in vergüenza are genuinely accent-marked variants and fold correctly — so the exception is one line, and its absence is the bug.

There is a third source of the same corruption that is not a fold at all. Text that has been through a lossy encoding round trip — UTF-8 bytes read as Latin-1, or a system that silently substituted unmappable characters — arrives with ñ already turned into ñ, ? or n. Once it is stored that way there is no reliable repair, because ano and año are both real words and nothing in the row says which one it was. This is the argument for validating the character set at ingest rather than at display: a mojibake check that rejects Ã followed by a continuation byte catches the whole class before it is written.

If you need recall for users who type ano meaning año, use the same pattern as French accent-insensitive search: index the folded form as an additional term and rank the exact form above it. Never replace the original.

Where ñ belongs in a sort

Spanish alphabetical order places ñ after all of n and before o. It is a full letter, so it sorts as a letter, and the consequence is that a word beginning ñ comes after every word beginning n — not interleaved with them.

Correct Spanish order:
  nube        n
  nunca       n
  ñandú       ñ    <- after every n-word
  ñoño        ñ
  oasis       o

Code-point order (the "C" collation, or a naive sort):
  nube
  nunca
  oasis
  ñandú            <- after o, and after z as well
  ñoño

One thing this page will not do is tell you what your collation currently produces, because that depends on the CLDR version, the locale tailoring and the library. Root collation treats the tilde as a secondary-level difference from n, which sorts ano before año correctly but can order año before anual — a primary-level comparison of a-n-o against a-n-u — where strict Spanish alphabetization puts every ñ word after every n word. Whether a given locale tailors this is a fact about your stack, and the only honest answer is to measure it.

It is also worth knowing what changed. The Association of Academies of the Spanish Language decided in 1994 to stop treating ch and ll as separate collation units, so they now alphabetize as c+h and l+l. Ñ was explicitly retained. Old software and old expectations still occasionally reflect the pre-1994 rules; the Real Academia Española is the authority to check against.

Checking what your collation actually does

Three lines, and it settles the question for your specific stack:

-- PostgreSQL: compare the collations available to you
SELECT w FROM (VALUES ('nube'),('nunca'),('ñandú'),('ñoño'),('oasis'),('zorro'))
  AS t(w) ORDER BY w COLLATE "C";              -- code points: ñ after z
SELECT w FROM (VALUES ('nube'),('nunca'),('ñandú'),('ñoño'),('oasis'),('zorro'))
  AS t(w) ORDER BY w COLLATE "es-ES-x-icu";    -- ICU: check the result

// JavaScript
const list = ["oasis", "ñandú", "nube", "anual", "año", "ano", "zorro"];
console.log([...list].sort());                              // code-point order
console.log([...list].sort(new Intl.Collator("es").compare));

Run both. If the second matches the expected Spanish order for your data, use that collation everywhere and stop. If it does not — in particular if año lands before anual and your users expect otherwise — you need an explicit collation tailoring, and ICU accepts rule strings that promote the tilde difference to the primary level for exactly this purpose. Do not attempt to fix it by rewriting the data.

What the pipeline should do

1. Normalize to NFC on ingest. A decomposed n+U+0303 must not survive into storage, or every downstream equality check and every collation comparison has two inputs to worry about instead of one.
2. Never strip ñ in place. Add it to the exception list of any ASCII fold, alongside the same treatment Polish gives its own non-variant letters. The fold may produce n as an additional index term; it may not replace the character in stored text.
3. Sort with an ICU collation, never with byte order. The C collation is correct for nothing that a human reads.
4. Pin the collation where it matters. Specify it on the query or the column rather than inheriting the server default, so the ordering is a decision in your code and not a property of the machine.
5. Watch the capital. Ñ is U+00D1 and needs the same treatment as the lowercase form; a fold table that lists only ñ mangles every all-caps heading.
6. Keep ñ out of generated identifiers. Slugs, file names and URLs need an ASCII form, and there the correct transliteration is n — but that is a deliberate one-way projection into a restricted character set, chosen at the point of generation, not a fold applied to the underlying text.