Latency in Robotics: When 100 ms Is Too Slow

In most software, latency is a quality-of-service number: worse is worse, and eventually users complain. In a control loop it is a stability parameter. Past a threshold the system does not get sluggish — it oscillates and then destroys itself.

A control loop has a deadline, not a target

A digital controller runs at a fixed period T: sample the sensors, compute, write the actuator command, wait for the next tick. Between ticks the command is held constant — a zero-order hold — so the actuator is doing whatever it was last told for the whole period, whether or not that is still appropriate.

Two rules set T. Sampling theory says you must sample at more than twice the highest frequency present, or aliasing turns fast dynamics into slow ones that the controller responds to wrongly. And practical control design says the sample rate should be roughly ten to twenty times the closed-loop bandwidth you want — sampling at exactly the Nyquist limit is a theoretical minimum, not an engineering target. A system with a 10 Hz closed-loop bandwidth therefore wants sampling somewhere around 100 to 200 Hz, and joint-level torque control on a stiff manipulator wants 1 kHz for the same reason applied to a much faster plant.

The deadline part is what distinguishes this from ordinary software: the actuator is commanded every T regardless. Missing the deadline does not delay the command; it means last cycle’s command is applied again. A few of those is a glitch. A run of them is an open-loop system.

What a delay costs, in phase margin

Here is the calculation that makes “100 ms is too slow” a statement about physics rather than taste. A pure time delay of τ seconds does not attenuate a signal at all; it adds a phase lag that grows linearly with frequency, φ = ωτ radians. Stability margins are eaten directly by that lag: whatever phase margin the controller was designed with, the delay subtracts ω_c τ at the crossover frequency ω_c.

phase lost to delay:  phi = omega_c * tau   (radians)
                          = omega_c * tau * 180/pi  (degrees)

crossover omega_c = 20 rad/s  (~3.2 Hz closed-loop bandwidth)

  tau =  5 ms  ->  0.10 rad  =  5.7 deg
  tau = 20 ms  ->  0.40 rad  = 22.9 deg
  tau = 50 ms  ->  1.00 rad  = 57.3 deg
  tau = 100 ms ->  2.00 rad  = 114.6 deg   -- unstable for any
                                              sane design

a design starting with 60 deg of phase margin is
marginal at 50 ms and gone well before 100 ms.

the delay margin -- how much delay a loop can absorb
before it goes unstable -- is  PM(rad) / omega_c :
  60 deg = 1.047 rad,  1.047 / 20 = 52 ms.

And ω_c in that example is low. A stiff manipulator joint may cross over at 100 rad/s or more, at which point the delay margin is around 10 ms and a network round trip is not merely unhelpful but disqualifying. This is the whole reason the fast loop is local: there is no engineering that recovers a phase margin you spent on transport.

Where the milliseconds actually go

Latency in a robot is not one number, and the model call is often not the largest contributor. An honest budget looks something like this — all figures are placeholders to be replaced with measurements from your own stack.

The number that matters for the phase-margin calculation is the total from physical event to physical response, and the number that matters for control quality is its variance: a loop with 20 ms of jitter behaves worse than one with 40 ms of constant delay, because constant delay can be modelled and compensated and jitter cannot. Reporting a mean hides both. Percentiles are the right summary here for the same reason they are the right summary for inference latency.

The twenty-cycle gap

Now the arithmetic this cluster keeps returning to. Take a model that answers in 200 ms — a reasonable figure for a hosted call and an optimistic one for a vision-language-action forward pass — and put it against a 100 Hz controller.

model response      200 ms
control period       10 ms   (100 Hz)
cycles per response  200 / 10 = 20

against a 1 kHz joint loop:  200 / 1 = 200 cycles

during those cycles the actuators ARE being commanded.
the only question is by what.

Everything in modern embodied AI architecture is an answer to that question, and there are only four of them:

• A chunk of precomputed actions. The model emitted a sequence covering the next several hundred milliseconds and the controller is playing it back. This is action chunking, and it is the dominant answer.
• A trajectory being tracked. The model emitted a waypoint or a spline, and a conventional tracker is following it at full rate, correcting for disturbance without asking anyone.
• A local optimisation. A model-predictive controller re-solves at a middle rate against a cost function whose parameters the slow layer set. The fast layer is still doing real work; it just is not doing semantics.
• A reflex. Impedance control, force limits, collision reaction. This layer runs at the highest rate and is the one that must never be waiting for anything.

The split, and what the fast half does

The two-system architecture — a slow, general, semantic planner and a fast, narrow, deterministic controller — is not a convention borrowed from cognitive science. It is what the phase-margin calculation requires. Published architectures state their rates explicitly: the π₀ report describes action chunks generated at 50 Hz from a vision-language backbone, and NVIDIA’s GR00T N1 describes a two-system design in which a vision-language module reasons slowly while a diffusion transformer produces actions at 120 Hz. Whatever else differs, the rate stratification is the same.

The division of labour that follows is worth stating plainly. The slow layer decides what: which object, which approach, when the subtask is done, what to do about an instruction. The fast layer decides how much: this torque, this correction, this deceleration, right now, given the last millisecond of sensor data. The slow layer may be a large model on a network. The fast layer may not be, ever.

When the slow half is late

This is the part that separates a design from a prototype. “What happens when the planner does not answer in time” is a specification, not an exception handler, and it has to be written down before the first deployment rather than discovered during one.

• Continue the current chunk. Safe if the chunk was short and the world is slow. This is the normal case and it is why chunk horizon is a safety parameter as much as a performance one.
• Decelerate to a stop along the planned path. The standard degraded behaviour: keep the trajectory, drop the speed, come to rest somewhere known.
• Hold position under impedance control. Appropriate when stopping mid-task is safe and the robot may be in contact — compliant hold rather than stiff hold, so a person can push it away.
• Trigger a defined stop category. The formal option, and the one the certified safety layer owns. Note that stopping is not universally the safe action — a vehicle in traffic and an arm holding something heavy both have better fallbacks than immediate power removal — which is why the fallback is designed per system.

The discipline is the same one that applies to any dependency that can be slow: decide what the degraded behaviour is and implement it deliberately. The difference is that here the degraded behaviour has to hold a deadline too.