Teleoperation as a Bridge Technology: The Ratio That Decides It

A remote human driving a robot is usually described as a stopgap on the way to autonomy. It is better understood as an architecture with its own economics, its own physics, and the useful property that operating it produces exactly the data needed to reduce it.

Two different jobs with one name

Teleoperation covers two activities that share an interface and share nothing else.

Data collection is a human driving a robot in order to produce demonstrations. The output is a dataset; the robot doing anything useful is incidental. Throughput, coverage and label fidelity are the metrics, and the choice of rig determines what the data contains.

Service delivery is a human intervening so that a deployed robot completes real work. The output is the work; the data is the byproduct. The metrics are intervention rate, handling time and how many machines one person can cover.

Confusing the two produces bad decisions. A rig optimised for collection — bimanual, high fidelity, one operator glued to one robot — is the wrong shape for service, where an operator must supervise many machines and take over occasionally. Conversely, a supervisory interface built for service produces sparse, interventional data that is excellent for correcting a policy and poor for training one from scratch.

The ratio that decides the business

Here is the calculation that determines whether a human-in-the-loop deployment is a business or a subsidy. It has two inputs: how often a robot needs help, and how long help takes.

r = interventions per robot-hour
h = mean handling time per intervention (hours)

operator load from one robot  = r * h
robots per operator (raw)     = 1 / (r * h)

but interventions arrive irregularly, and two at once means
one robot waits. target a utilisation rho below 1 so the
queue does not blow up:

  N = rho / (r * h),   with rho ~ 0.6-0.7 in practice

WORKED (all inputs assumed):
  r = 4 /h, h = 2 min = 0.0333 h  ->  r*h = 0.133
    raw  1/0.133 = 7.5 robots     ->  N = 0.65/0.133 = 4.9

  halve the intervention rate: r = 2 /h
    raw  15 robots                ->  N = 9.8

  halve handling time instead: h = 1 min
    raw  15 robots                ->  N = 9.8

  both:  r = 2 /h, h = 1 min      ->  N = 19.5

Two things fall out that are not obvious from the slogan. First, the ratio is hyperbolic in the product r·h, so improvements compound: halving both the intervention rate and the handling time quadruples the fleet one person can cover. Second — and this is the part that gets missed — handling time is as valuable as autonomy. Making the robot ask for help better, showing the operator context faster, letting them resolve a case in one click rather than by driving the arm manually: all of that moves h, and moving h is usually far cheaper engineering than moving r.

The queueing term is not a detail either. Interventions are not evenly spaced, and a fleet sized at the raw ratio will have robots idle waiting for an operator a substantial fraction of the time. Idle robots are the availability term in the payback calculation, which is where over-optimistic staffing shows up as a failed business case rather than as a queueing problem.

What delay does to an operator

Not all teleoperation tolerates delay equally, and the distinction is the same one that runs through this whole cluster: whether the human is inside a control loop or above it.

Direct, force-reflecting control

A bilateral haptic system, where the operator feels the forces the robot experiences, is a closed loop through the human, the network and the robot. Such loops want to run at around a kilohertz for the rendered contact to feel stiff, and they are destabilised by delay for exactly the reason any loop is — the phase lag argument in the latency page applies unchanged, with the human as one of the elements. Round-trip delays of even tens of milliseconds degrade the achievable stiffness, and the classical remedies are control-theoretic rather than engineering: the wave-variable formulation from Niemeyer and Slotine makes the communication channel passive so the loop stays stable for any delay, at the cost of distorting the force the operator feels.

Supervisory control

The operator specifies a goal — this object, that place, this action — and an onboard controller executes it. The human is above the loop, not in it, so delay costs responsiveness rather than stability. Sheridan formalised this as supervisory control decades ago, and it is what makes teleoperation over long, variable-latency links tractable at all. It is also the reason interfaces converge on click-to-grasp and click-to-navigate rather than on joysticks: raising the abstraction level buys delay tolerance and cuts handling time at the same moment.

There is a middle case worth naming: shared autonomy, where operator input and an autonomous policy are blended, with the policy handling the fast corrections and the human providing intent. It has the delay tolerance of supervision and much of the flexibility of direct control, and it is the natural interface once a policy is good enough to be worth blending with.

Why every intervention is worth more than the task

This is what makes teleoperation a bridge rather than a crutch. When an operator takes over, three things are recorded at once: the state the policy had reached, the fact that it was a failure state, and the correct action from there.

That triple is precisely what the compounding-error result asks for. Behaviour cloning degrades because the policy is trained on the expert’s state distribution and deployed on its own; DAgger’s answer is to label the states the policy actually visits. A production intervention log is a DAgger dataset generated as a side effect of doing business, in the exact distribution that matters, weighted by frequency of occurrence — the failures that happen most get the most labels.

The flywheel is therefore real and not merely rhetorical: interventions fund the operation and reduce r; a lower r raises the fleet-per-operator ratio; a higher ratio lowers unit cost, which funds more robots, which generate more interventions in more situations. There is no step in that loop requiring a research breakthrough, which is unusual in this field. The caveats are that the data is sparse and skewed towards failure — you will have very few examples of the smooth nominal case — and that the interface must actually be recording state and action at sufficient rate, which is an engineering decision made early and expensive to retrofit.

What it does not solve

• Safety is not delegated by adding a human. A remote operator with a laggy video feed has worse situational awareness than a person in the room, and cannot see what is behind the robot. The certified limits still have to hold regardless of who is commanding, and an operator command should be treated as untrusted input to the same envelope a policy’s output goes through.
• Attention does not scale linearly. The ratio maths assumes an operator can hold context on many machines. Vigilance decrements are well documented in human-factors research, and an operator monitoring twenty feeds for a rare event is in the worst possible task design. Interfaces that summon attention rather than requiring it are doing real safety work.
• Link quality bounds the deployment. Everything above assumes a connection. What happens when it drops is a specification the robot must implement locally, exactly as in the on-board/remote split.
• The privacy surface is real. Teleoperation means a person somewhere sees a camera feed from wherever the robot is — somebody’s workplace, somebody’s home. That is a policy and consent question before it is a technical one, and it does not resolve itself by being unmentioned.