Yi Models’ Context Window and License Terms

Yi’s context window is either 4,096 tokens or 200,000, and which one you get depends on which repository you download. 01.AI released the long-context models as separate checkpoints with -200K in the name rather than as a configuration of the base ones, and almost every single-number answer to this question is quoting one of the two without saying which.

The releases, in order

• Yi-6B and Yi-34B (November 2023). The initial release from 01.AI, decoder-only transformers with a Llama-compatible architecture — which is why they dropped straight into existing tooling — trained on a bilingual English and Chinese corpus.
• Yi-6B-200K and Yi-34B-200K. Long-context variants released alongside, as their own repositories.
• Yi-9B (2024). A middle size, depth-upscaled from the 6B model.
• Yi-1.5 (May 2024). A retrained generation at 6B, 9B and 34B, released with 4K, 16K and 32K context variants and instruction-tuned counterparts.
• Yi-Coder and Yi-VL. Code and vision-language lines with their own cards and their own figures.
• Yi-Large. An API-only model. There are no weights, so the open-weight licence discussion below does not apply to it; its terms are the platform’s terms of service.

The Llama-compatible architecture is worth pausing on because it explains why Yi appeared in so much tooling so quickly, and because it caused a public argument at release: the initial checkpoints used Llama’s architecture with two tensors renamed, which drew criticism about attribution. 01.AI responded by restoring the original names and documenting the relationship. Nothing about the weights was in dispute — they were trained from scratch on 01.AI’s own corpus — but the episode is why some older loaders carry Yi-specific name mapping that is no longer needed. If you hit a key-mismatch error loading an early checkpoint, that history is the cause.

Context length, including the 200K variants

Yi-6B / Yi-34B                    4,096 tokens
Yi-6B-200K / Yi-34B-200K        200,000 tokens (approx. 262,144 configured)
Yi-9B                             4,096 tokens
Yi-1.5 (base variants)      4,096 / 16,384 / 32,768 tokens

The gap between the marketing figure of 200K and the configured max_position_embeddings in the repository is worth noting rather than treating as an error: the models were positioned as 200K-capable and configured to a power-of-two ceiling above it. If you are sizing buffers, read the config; if you are reading a comparison table, expect the round number.

The licence changed to Apache 2.0

The initial November 2023 release used the Yi Series Models Community License, which permitted research use freely but required registration with 01.AI before commercial use. That gate drew immediate criticism — a licence requiring you to ask permission is not something a downstream project can depend on — and 01.AI removed it: the Yi models were relicensed under Apache 2.0, and Yi-1.5 was released under Apache 2.0 from the start.

Two practical consequences. First, if you are reading documentation or a third-party comparison written in late 2023 or early 2024, its licence statement about Yi is stale and describes terms that no longer apply. Second, and more important for anything with a compliance process: Apache 2.0 here means Apache 2.0, with no acceptable-use policy bolted on. That puts Yi in a genuinely different category from the Apache-derived licences used by Falcon and by DBRX, and in the same category as Mistral’s Apache-2.0 releases and Qwen’s.

The licence file in each Hugging Face repository is authoritative, and because the change was applied by updating the repositories, a copy of the weights taken before the change carries the old text. If your organisation archived a mirror in 2023, the archive’s licence file is the one you accepted.

It is also worth being precise about what the relicensing did and did not cover. The model weights moved to Apache 2.0. The training data did not become public, and no corpus was released — Yi is an open-weight model in the narrow sense, not an open one in the sense described for OLMo. Those two axes are independent and conflating them is common: a model can be under the most permissive licence in existence and still ship nothing you can inspect.

What a 200K variant costs you in practice

The 200K checkpoints are ordinary transformers with a rescaled rotary position encoding. There is no architectural relief of the kind a hybrid stack provides — every layer still keeps a full KV cache that grows linearly with sequence length. So the costs are the ones you would predict:

The rescaling is worth naming precisely, because it is the standard move and you will meet it on every long-context transformer. Rotary embeddings rotate query and key vectors by an angle set by position and by a base frequency, conventionally exposed as rope_theta. Raising that base stretches the range of positions the same set of frequencies covers, which is what lets a model trained at one length be trained further at a much longer one without the angle pattern becoming unrecognisable. It is a training-time change with a config-time fingerprint: two checkpoints of the same architecture with different rope_theta values are different models, and that field is the reliable way to tell a genuine long-context checkpoint from one somebody edited a config on.

• Memory per request grows without bound up to the window. At 200K tokens the cache for a 34B model is a large multiple of what a typical request uses, and it is per concurrent request. Batch size collapses accordingly.
• Prefill dominates latency. Time to first token is set by processing the prompt, and attention over a 200K prompt is quadratic in the sequence length. This is not something streaming hides; the stream has not started yet.
• The base and 200K checkpoints are separate downloads. They were trained differently, not configured differently. You cannot get the long window by raising a config value on the base model. Raising max_position_embeddings past what the model was trained for produces fluent-looking degradation rather than an error, which is the worst failure mode available. The one architecture where extrapolating past the trained length is a documented design goal is ALiBi — see how MPT extrapolates without position fine-tuning.

Practically, this means the 200K variants are a specialist tool rather than a strictly better version of the base models. If your prompts are a few thousand tokens, the base checkpoint is the same model with the same quality and a far cheaper serving profile. Reach for the 200K one when you have a document that genuinely will not fit any other way, and expect to provision for it separately rather than running both from one pool.

What to verify

1. Confirm whether the repository name ends in -200K. That suffix, not the family name, is what determines the window.
2. Read config.json for max_position_embeddings and rope_theta, and note both. A community re-quantisation may have changed either.
3. Read LICENSE in the repository you are actually pulling, and record the commit hash. Do not rely on a comparison article’s licence column.
4. If you inherited a mirror from before 2024, check its licence file separately — it may predate the Apache 2.0 change.