The Open-Weights Wave and How It Started

The standard account has open-weight models beginning when Meta’s LLaMA weights appeared on BitTorrent in March 2023. That event is real and it mattered, but publishing model weights was the norm in NLP from 2018 and there is a continuous public record between the two. The thing that actually changed in 2023 was the licence.

Open was the default until 2019

Google published BERT’s weights in 2018 under the Apache 2.0 licence, and T5’s afterwards. Word vectors, machine translation models and the entire pretrained encoder generation were published as a matter of routine, because publishing weights was how you demonstrated a result and the weights were not commercially valuable on their own.

What made GPT-2’s staged release in February 2019 notable was that withholding was novel. The reaction at the time — considerable criticism from researchers who saw it as either overcautious or a publicity exercise — only makes sense against a background in which release was assumed. The full 1.5-billion-parameter model was published in November 2019 and nothing much happened, which became part of the argument in both directions afterwards.

2019 to 2022: the years the story skips

A continuous line of open releases runs from GPT-2 to LLaMA, and omitting it makes the 2023 wave look like a discontinuity when it was an acceleration.

March 2023: the leak

Meta published the LLaMA paper on 24 February 2023 and made the weights available to researchers on request under a non-commercial licence. Its stated result was that smaller models trained on far more tokens than was then usual could match much larger ones — a direct application of the compute-optimal finding described on the GPT model history page. That made the 7B and 13B versions runnable on hardware individuals actually own, which is the fact that made everything after it possible.

Within about a week the weights were circulating on BitTorrent, and a magnet link was submitted to a public repository as a pull request. Meta did not pursue takedowns aggressively, and by the time it might have mattered the weights were on thousands of machines.

The leak is not the interesting part. The interesting part is that a model with a licence forbidding commercial use, in the hands of everybody, produced a burst of tooling that no permissively licensed model had produced before it — because it was the first openly circulating model that was both good and small enough to run at home.

The eight weeks that followed

1. llama.cpp, from Georgi Gerganov in March 2023: a plain C and C++ implementation of inference with aggressive quantisation, which ran the 7B model on a laptop CPU. It removed the GPU requirement, which removed the last barrier to individual experimentation, and its quantised file format became a de facto standard for local models.
2. Stanford Alpaca, 13 March 2023: LLaMA-7B fine-tuned on 52,000 instruction-following examples that were themselves generated by a stronger commercial model. The team reported the total cost — data generation plus compute — as under $600. The demonstration that instruction-following could be added to a base model for the price of a laptop is what turned a leaked checkpoint into an ecosystem.
3. Vicuna and the conversation fine-tunes, later in March: the same idea applied to shared conversation logs, with claimed quality approaching commercial assistants on informal evaluations. The evaluations were weak — this is the period in which using a strong model to grade a weaker one became common and its problems were not yet understood — but the direction was right.
4. QLoRA, May 2023: fine-tuning a quantised model through low-rank adapters, bringing the memory requirement for fine-tuning a sizeable model within reach of a single consumer graphics card. This is the technique that made the long tail of community fine-tunes economically possible.

The compressed sequence is: a research licence in February, a leak in March, CPU inference and cheap instruction-tuning within a fortnight of each other, and single-GPU fine-tuning by May. Three months, from a gated research artefact to a technology an individual could adapt.

Llama 2, Mistral, and the licensing split

Meta released Llama 2 on 18 July 2023 with weights available for commercial use. This is the moment the norm changed, and the details of the licence are what the argument has been about ever since.

• The Llama 2 Community License is not an open-source licence. It restricts use by services above a stated monthly-active-user threshold, which required a separate licence from Meta, and it attaches an acceptable use policy. A licence that discriminates between users by size fails the Open Source Definition, and the Open Source Initiative said so publicly.
• Mistral 7B, in September 2023, was released under Apache 2.0. No use restrictions, no user threshold, no acceptable use policy attached to the licence. That is why it mattered disproportionately to its size: it demonstrated that a competitive model could be shipped under genuinely open terms.
• The distinction has a name and it is worth using. “Open weights” means you can download and run the parameters. “Open source” in its established sense means considerably more, and essentially nobody ships the training data, without which the result cannot be reproduced. The practical consequences are laid out in the licence-by-licence comparison.

The Open Source Initiative published version 1.0 of its Open Source AI Definition in October 2024, after a long and contested consultation. Its requirement for data information — enough detail about training data for a skilled person to build a substantially equivalent system — is stricter than what most self-described open models provide and looser than what some critics wanted, which is roughly what a negotiated definition looks like.

The release argument, and how its terms changed

Running alongside the chronology is an argument about whether to publish weights at all, and it is worth tracking separately because the positions moved in ways the model releases do not show.

1. 2019, the misuse case. GPT-2’s staged release was justified on the grounds that a model that generates fluent text cheaply could be used to produce misleading content at volume. The counter-argument at the time was that the capability was replicable — several groups did replicate it within months — so withholding imposed a cost on researchers without imposing one on a determined misuser.
2. 2022, structured access. Between GPT-2 and LLaMA, the middle position hardened into a practice: publish the paper, gate the weights behind an application, and serve the capability through an API where it can be monitored and revoked. OPT-175B and the original LLaMA are both this. The March 2023 leak is the demonstration that the gate is only as strong as the smallest number of people who need to be inside it.
3. 2023 onwards, the safety-of-fine-tuning case. The argument shifted to a more specific and better-evidenced concern: published research showed that the safety behaviour of an open-weight model can be removed by fine-tuning on a small number of examples for a trivial cost. That is a real and reproducible property, and it is a materially different objection from the 2019 one because it does not depend on predicting how anyone will behave.
4. The counter-case, also stronger than in 2019. Reproducible research, auditability by people outside the lab, no dependence on a vendor’s continued existence or pricing, the ability to run on infrastructure the user controls for regulatory reasons, and — the argument that has actually persuaded most organisations — cost. Almost the entire published literature on how these models work internally is done on open-weight models, because they are the only ones anybody outside a frontier lab can inspect.

The argument has not resolved and it has become more empirical, which is progress of a kind. Both sides now cite reproducible results rather than forecasts, which is roughly the opposite of how the debates on the wrong-predictions page were conducted.

Where it stands, and what to check

The period after 2023 is a widening of the same pattern rather than a new one: Llama 3 in 2024 including a 405-billion-parameter model with published weights, Google’s Gemma family under its own custom terms, Alibaba’s Qwen models mostly under Apache 2.0, and DeepSeek-V3 in December 2024 followed by DeepSeek-R1 in January 2025 under an MIT licence — the first widely used open-weight reasoning model, and the release that made the capability gap between open and closed models a subject of general rather than technical interest. Current state of that gap belongs on a page that is maintained for it; see the family tree of open model releases and the Chinese open models.

Three questions are worth asking of any model advertised as open, and all three have been necessary at some point in the history above:

1. What is the licence, by name? Apache 2.0 and MIT mean one thing. A bespoke community licence means read it, because the restrictions differ between models from the same vendor.
2. Are there use restrictions or user thresholds? The question that decides whether you can build a business on it, and the one most often skipped.
3. What is disclosed about the training data? Almost always the answer is very little, which bounds what anyone outside the lab can say about the model’s behaviour — the subject of what training data claims actually tell you.