Foundation Models for Robotics: Stating the Bet So It Can Be Settled
6 min read · updated August 3, 2026
“Foundation model for robotics” names a bet, not a result. The bet is specific enough to state as a claim with evidence that would confirm it and evidence that would refute it, and stating it that way is more useful than any survey of what is currently working.
The bet, stated precisely
Here is the claim, in one sentence: a single policy pretrained on diverse tasks and diverse robot bodies acquires representations that transfer, so that adding a new task or a new body costs a small amount of fine-tuning data rather than a full data-collection programme.
Everything follows from why that would matter. The data arithmetic says a task-specific policy costs on the order of operator-weeks per task per embodiment. If that is the permanent state of affairs, robot learning does not scale — every deployment is a project, and the field is a consultancy business. If pretraining amortises most of it, the marginal task becomes cheap and everything changes, including the economics in the payback calculation.
The bet is not that models get better. It is specifically about positive transfer across tasks and bodies, and that is an empirical claim which could be false. Transfer is not automatic: multi-task training can interfere as easily as it can help, and a model trained on twenty embodiments could plausibly end up worse on each than twenty specialists.
What each step contributed
The published line of work is worth reading as a sequence of separable contributions rather than as a leaderboard.
| Work | Description |
|---|---|
| RT-1 (2022) | Established that a transformer trained on a large, diverse single-robot dataset could absorb many tasks in one policy rather than requiring one policy per task. The contribution was the demonstration that scale and diversity in robot data behaved usefully at all — a precondition for everything after. |
| RT-2 (2023) | Put the policy on top of a large vision-language model and co-fine-tuned on web data and robot data together, with actions expressed as tokens in the text vocabulary. The contribution was the mechanism for semantic transfer: knowledge that only ever appeared in the web data showing up in how the policy handled instructions and objects. |
| Open X-Embodiment / RT-X (2023) | Pooled datasets contributed by roughly twenty institutions across more than twenty distinct robot embodiments into one collection, and trained policies on the union. The contribution was the dataset itself plus the first serious test of whether training across bodies helps or interferes. |
| Octo, OpenVLA (2024) | Open generalist policies trained on that pooled data, with weights and code released. The contribution is less architectural than infrastructural: an open baseline that other groups can fine-tune and compare against is what makes an empirical question answerable by more than one lab. |
| Flow-matching action experts (π0 and successors) | Replaced tokenised actions with a continuous generative action head attached to a vision-language backbone, described in its technical report as producing action chunks at 50 Hz. The contribution is rate and precision — the token route's quantisation and sequential decode were both real limits, as the VLA page describes. |
| Large-scale teleoperation datasets (BridgeData, DROID and similar) | Not architectures but the substrate: multi-scene, multi-task demonstration collections gathered on commodity hardware, deliberately diverse in scene and task rather than deep in one. Diversity is the property the bet requires, so these are as load-bearing as any model. |
The cross-embodiment problem
The single largest technical obstacle is that different robots do not share an action space. A seven-degree-of-freedom arm with a parallel gripper, a mobile manipulator, a bimanual setup and a dexterous hand have different numbers of joints, different limits, different kinematics and different control modes. A policy output that means something on one means nothing on another.
Three approaches exist and each concedes something:
- Normalise to a shared frame. Express actions as end-effector deltas in Cartesian space plus a gripper command, and let each robot’s own inverse kinematics turn that into joint commands. This is the common choice, and it is why so many datasets share a seven-dimensional action convention. It works for arms with a single end effector and loses everything about whole-body posture, redundancy resolution and multi-contact.
- Per-embodiment heads on a shared trunk. One backbone, one small output head per robot, each trained on that robot’s data. Clean, and it means a new robot costs a new head plus data — which is exactly the cost the bet was supposed to remove.
- Tokenise the action space itself. Include the robot’s description in the input so the model conditions on the body it is driving. Most general, least demonstrated, and it needs enough embodiments in training for the conditioning to mean anything.
Underneath all three is a physical fact that no representation fixes: two robots with different mass distribution, different gripper friction and different actuator bandwidth genuinely need different actions to achieve the same outcome. Some of the cross-embodiment gap is a representation problem and some of it is the same dynamics mismatch as sim-to-real, appearing between two real robots instead of between a simulator and one.
What would settle it
Because the bet is a claim about transfer, the observations that bear on it are claims about data efficiency rather than about peak performance. A policy that does something impressive tells you nothing; a policy that does something ordinary with a tenth of the data tells you a great deal.
Confirming observations
- A fine-tuning curve on a held-out embodiment — one whose data was excluded from pretraining — showing the pretrained model reaching a given success level with substantially fewer demonstrations than a model trained from scratch. The held-out part is the whole test; fine-tuning on a robot that was in the pretraining mix proves nothing about generalisation.
- A scaling curve: task performance improving predictably as pretraining data grows, with the relationship holding out of sample. The absence of established scaling laws for robot data is one of the clearest differences from language, and establishing one would be a substantial result on its own.
- Transfer of a skill rather than a task — a policy handling a manipulation strategy it saw only on other bodies or other objects, reproduced by independent groups on their own hardware.
- Negative transfer shrinking as embodiment count grows, which is what you would expect if the model is learning shared physics rather than memorising per-robot behaviour.
Refuting observations
- Fine-tuning data requirements on a new embodiment staying flat as pretraining scale increases. That would say the pretraining is not buying transferable structure.
- Specialists continuing to beat fine-tuned generalists by wide margins at equal data, across many independent groups.
- Gains that vanish under distribution shift — a policy that transfers across labs sharing a rig design but not across genuinely different setups, which would indicate the shared thing was the camera placement rather than the physics.
Why settling it is slow
Every observation above requires physical evaluation, and physical evaluation is the bottleneck on the entire research programme. You cannot run a robot policy ten thousand times overnight in CI. A serious evaluation is a person, a robot, a reset protocol and hours, which caps the number of trials and therefore the resolution of any comparison — see what twenty trials can and cannot distinguish.
This has a second-order effect worth noticing. When evaluation is expensive, the field’s iteration speed is set by evaluation rather than by training, and results become harder to reproduce because reproduction is as expensive as the original. Language modelling got its feedback loop nearly free: a held-out set, a number, in minutes. Robotics does not have that, and the question of whether robotics gets a moment like language’s probably turns on this as much as on any modelling advance.