Embodied AI: Why the Physical World Is Harder

Text models got good at the things people find hard and robots stayed bad at the things a toddler does without thinking. That inversion has a name and a forty-year-old explanation, and the explanation is more useful than the name.

Moravec’s paradox, restated

Hans Moravec put it in Mind Children in 1988: it is comparatively easy to make computers perform well on intelligence tests or play checkers, and difficult or impossible to give them the perception and mobility of a one-year-old. Steven Pinker offered the same observation from the other end — the hard problems are easy and the easy problems are hard.

The usual gloss is evolutionary: sensorimotor skill has had hundreds of millions of years of optimisation behind it and abstract reasoning has had a few thousand, so the former is deeply engineered and subconscious while the latter is shallow and available to introspection. That is probably true and it is not actionable. The actionable restatement is about supervision.

Every capability a large model has came from a corpus somebody could collect. Text about chess exists in enormous quantity because people wrote it down. Text about how much force to apply to a doorknob does not exist in any quantity at all, because nobody has ever written it down and there is no natural process that produces it as a byproduct. The paradox is not that motor control is intrinsically deeper than reasoning. It is that reasoning left a written trace and motor control left none, so one had a corpus waiting and the other has to be paid for a trajectory at a time.

The deadline is the first difference

A control loop is a piece of code that runs at a fixed period: read the sensors, compute a command, write it to the actuators, wait for the next tick. Joint-level control on an industrial manipulator commonly runs at 1 kHz — one millisecond per iteration. A whole-body controller on a legged robot commonly runs somewhere between 100 Hz and 1 kHz. A quadrotor’s attitude loop runs faster still, because a quadrotor is open-loop unstable and falls over in a fraction of a second if nothing is correcting it.

Now put a large model in that loop. Suppose the model answers in 200 ms, which would be a good time to first token for a hosted model and is optimistic for one producing a structured action. Against a 100 Hz loop, whose period is 10 ms, that is twenty consecutive control cycles during which the loop must emit a command and the model has not answered. Against a 1 kHz joint loop it is two hundred.

There is no version of “wait for the model” here. The actuators are being commanded at 1 kHz regardless; the only question is what they are being commanded with. That is why essentially every working architecture in this field is two systems — something slow that decides what to do, and something fast that keeps the machine stable and on trajectory while the slow thing thinks. Getting specific about what fills the twenty cycles is most of what a robotics latency budget is.

The second: nobody scraped the actions

Language model pretraining corpora are measured in trillions of tokens and were assembled from material that already existed. The equivalent for manipulation would be a corpus of paired observations and actions — what the robot saw, and what command was issued — and it does not exist, because issuing a command requires a robot and a person or a policy driving it.

Video is the tempting substitute and it is missing the label. A clip of somebody opening a jar records the outcome, not the joint torques, the grip force, or the proprioceptive state that produced it. Recovering the action from the observation is an inverse dynamics problem that is under-determined in general: many force profiles produce visually identical motion, and the difference between them is exactly what you needed. The two things that do produce labelled actions — teleoperated demonstration and simulation — each cost something specific and each fails in a specific way.

The third: failure is physical

A language model that produces a wrong token produces a bad sentence. Someone reads it, notices or does not, and the world is otherwise unchanged. The output is cheap to inspect, cheap to discard and cheap to retry, which is why so much of the tooling around models is built on the assumption that you can simply run it again.

A wrong torque moves mass. Consider what that means with the simplest possible arithmetic: a two-kilogram payload moving at one metre per second carries one joule of kinetic energy, and stopping it inside a centimetre requires a hundred newtons of average force. Those are small numbers by industrial standards and they are already well beyond what you would want applied to a hand. The safety standards for human-robot collaboration exist precisely because the relationship between mass, speed and injury is calculable, and the certified layer that enforces it is deliberately not the learned part.

Three consequences follow, and they are what people mean when they say robotics is unforgiving. Retries are not free — a failed grasp may have knocked the object somewhere the policy has never seen. Evaluation is not free — you cannot run ten thousand trials overnight in CI, so the statistics are always thin. And exploration is not free, which is the single largest obstacle to reinforcement learning on real hardware.

What this cluster is about

• The rate. Why architectures split into a slow planner and a fast controller, what the fast half is actually doing, and what a system does when the slow half is late — which is a design decision, not an exception handler.
• The data. Where robot training data comes from, what an hour of teleoperation buys, why simulation is the only source that scales, and what simulation quietly leaves out.
• The gap. Sim-to-real named part by part: contact, friction, actuator dynamics, latency, sensor noise. Some of those transfer nearly free and some do not transfer at all, and knowing which is which is the difference between a policy that works on hardware and one that does not.
• The failure. Why safety here is an architecture with a certified, deterministic layer in it, and why no amount of prompt engineering is a substitute for a torque limit that a stochastic policy cannot reach past.

Almost none of it is about whether models are clever enough. The interesting constraints are rates, joules, trajectories per hour and the difference between a simulator’s friction coefficient and the floor’s.