Edge Compute for Robots: What Fits On Board and What Must Go Remote

The question is never whether a model could run on a robot. It is what fits inside a power budget shared with the motors, a mass budget that the motors have to carry, and a memory bandwidth figure that caps generation rate no matter how fast the arithmetic units are.

The budget is watts, grams and bandwidth

On a mobile robot the compute draws from the same battery as everything else, and every watt spent thinking is a watt not spent moving. The framing that keeps people honest is to express compute power as a fraction of the energy budget rather than in absolute terms.

assume a 1.5 kWh pack and an 8-hour shift target
average allowable draw = 1500 Wh / 8 h = 187 W total

drive motors, average           ~100 W   (duty-cycle dependent)
sensors, comms, controllers      ~25 W
                                -------
left for perception + policy     ~62 W

a 60 W compute module is the ENTIRE remainder. a 150 W one
turns an 8-hour shift into roughly 4, which changes the
fleet size you need -- a capital decision, not a spec choice.

On a flying platform it is worse still, because mass is charged superlinearly through hover power — the derivation is on the aerial autonomy page and it is the sharpest version of this constraint anywhere in robotics.

Thermal is the third hidden term. A module rated at some peak draw will throttle if it cannot reject the heat, and a sealed robot enclosure in a warm building is a poor heatsink. A thermally throttled accelerator does not fail; it silently halves its throughput, which turns into jitter in a control loop and is miserable to diagnose after the fact.

The calculation that sets the ceiling

Here is the one piece of arithmetic worth memorising, because it does not go out of date when the hardware does. Autoregressive generation is memory-bandwidth bound rather than compute bound: producing one token requires streaming the model’s weights from memory through the arithmetic units, and at batch size one — which is what a single robot is — there is nothing to amortise that read across. So the upper bound on tokens per second is memory bandwidth divided by the bytes that must be read per token.

tokens/s  <=  memory_bandwidth (GB/s) / model_bytes (GB)

module bandwidth  100 GB/s   (an assumption; substitute yours)

  7B weights at int8   = 7.0 GB  ->  <= 14 tok/s
  7B weights at int4   = 3.5 GB  ->  <= 28 tok/s
  3B weights at int8   = 3.0 GB  ->  <= 33 tok/s
  1B weights at int8   = 1.0 GB  ->  <= 100 tok/s

this is a CEILING. real throughput is lower once
attention, KV cache traffic and preprocessing are counted.

Read that against the rate requirement. If the policy must emit an action every 20 ms and an action is eight tokens, you need 400 tokens per second, and none of the rows above reach it. That is not a hardware shortfall to be waited out; it is the reason action chunking and continuous action heads exist — they change the numerator of the requirement rather than the denominator of the supply. The general mechanism is the same one behind why bandwidth rather than FLOPs sets decode speed.

Three ways to fit under it

Quantise

Halving the bytes per parameter roughly doubles the ceiling above, and it does so linearly and predictably — which makes it the highest-value single change available. The cost is quality, and the shape of that cost is the subject of what quantisation does to a model. For a policy rather than a chat model there is a specific extra caution: quantisation error in an action head is error in a physical command, so the evaluation has to be on task success, not on a perplexity proxy.

Distil into something smaller

A large model in the lab producing training targets for a small model on the robot is the standard pattern, and it fits robotics unusually well because the deployed task distribution is narrow. A general policy must handle everything; the robot in front of you has to handle one cell. Distillation converts that narrowness into parameters you no longer have to stream.

Split the model, not just the stack

Run the vision encoder on board at frame rate and the language reasoning remotely at whatever rate it manages. Encoders are feed- forward, run once per observation rather than once per token, and are not bandwidth-bound in the same way. This also cuts uplink cost dramatically: sending a feature vector instead of a compressed image stream is orders of magnitude less data, with the privacy benefit that raw camera frames never leave the machine.

Where to draw the line

A blunt rule that survives most arguments: if the answer arriving 500 ms late would be merely annoying, it can be remote. If it would be dangerous, it cannot, and no amount of network engineering changes that, because the failure you are designing against is the network being absent rather than slow.

Designing for the link that will drop

Wireless links on moving machines fail. Not occasionally — predictably, behind racking, at range, in the corner of the building where the access points overlap badly. Any architecture in which a dropped packet produces undefined behaviour will produce undefined behaviour on a schedule.

• Make the timeout a state transition. Not an exception, not a retry loop: a defined move to a degraded mode with its own behaviour, its own indicator, and its own exit condition. The same discipline as a circuit breaker around any remote dependency, with the difference that the fallback here has to hold a deadline.
• Budget the round trip honestly. Wireless latency is heavy-tailed. A p50 that looks fine and a p99 that is ten times worse describes a system that will surprise you, and the p99 is the number the design has to survive.
• Send less, less often. Features rather than frames, events rather than streams, deltas rather than full state. Every byte not sent is latency and energy not spent.
• Keep the robot able to finish or abort alone. The test is simple: unplug the network mid-task and watch. If the answer is not obvious in advance, the split is in the wrong place.