SLAM Explained: How a Robot Builds a Map While Finding Itself in It

To know where you are you need a map. To build a map you need to know where you are. SLAM solves both at once by treating them as one estimation problem over poses and landmarks, and every design decision in the field is about making that problem small enough to solve in real time.

The circular dependency

Dead reckoning alone — integrating wheel encoders or an IMU — gives a pose estimate whose error accumulates without bound. Every step adds a small error and nothing ever removes one, so a 1% odometry error over 100 m of travel is roughly a metre of position error, and after a kilometre the estimate is useless as an absolute position while still being excellent over any short interval.

Observing the world fixes this, provided you know what you are looking at. Seeing a landmark whose position you already know constrains your pose. But the landmark’s position was itself estimated from an earlier pose that had its own error, so the two uncertainties are correlated and cannot be treated independently. That correlation is not a nuisance to be approximated away; it is where the information lives. Two landmarks observed from the same pose have correlated errors, and later observing them both from a new position tightens the estimate of everything.

Worked: two steps and one landmark

Strip it to one dimension. A robot starts at a known origin, moves, and observes one landmark from each of two positions. Uncertainties combine by inverse variance, which is the whole of the estimation machinery in this simplified case.

known:
  x0 = 0.0 m exactly (the origin defines the frame)

observation 1, from x0:
  landmark seen at 5.0 m, sigma 0.2 m
  -> L = 5.0, sigma_L = 0.2

motion:
  odometry reports +10.0 m, sigma 0.5 m
  -> x1 = 10.0 from odometry alone, sigma 0.5

observation 2, from x1:
  landmark seen 4.5 m BEHIND the robot, sigma 0.2 m
  -> this implies x1 = L + 4.5 = 9.5
     with variance = sigma_L^2 + 0.2^2
                   = 0.04 + 0.04 = 0.08
     sigma = 0.283 m

two independent estimates of x1:
  A: 10.0, variance 0.25   -> weight 1/0.25 = 4.0
  B:  9.5, variance 0.08   -> weight 1/0.08 = 12.5

fused estimate (inverse-variance weighted mean):
  x1 = (4.0 x 10.0 + 12.5 x 9.5) / (4.0 + 12.5)
     = (40.0 + 118.75) / 16.5
     = 158.75 / 16.5
     = 9.62 m

fused uncertainty:
  variance = 1 / (4.0 + 12.5) = 0.0606
  sigma    = 0.246 m

Three things in that arithmetic are the whole of SLAM in miniature. The fused estimate is pulled toward the more confident measurement rather than being averaged, so a precise landmark observation dominates noisy odometry. The fused uncertainty, 0.246 m, is smaller than either input — the robot is better localised after the observation than before, which is the thing dead reckoning can never do. And the landmark estimate should now be updated too, using the improved pose; leaving it at 5.0 m is the approximation that separates a filter that works from one that drifts.

The front end

The front end turns raw sensor data into constraints, and it is where almost all the failures originate. For a camera it detects and matches features between frames and estimates relative pose from the matches. For a LiDAR it registers consecutive sweeps, which is exactly the problem on the registration page — and the reason ICP variants sit at the centre of LiDAR SLAM is that the motion prediction from odometry supplies the initial guess ICP needs, and ICP returns the correction.

Two details make a real system work. Sweeps must be deskewed before matching, because a spinning sensor measures over 100 ms during which the robot moved — the same timing arithmetic as in LiDAR and camera fusion. And feature extraction should prefer geometrically stable structure: LOAM, from Ji Zhang and Sanjiv Singh at RSS 2014, split points into edge and planar features by local curvature and matched each to the corresponding structure in the previous sweep, which is both faster and more stable than matching raw points.

Data association is the fragile step. Deciding that this feature is the same physical thing as that one is a discrete choice, and unlike the continuous estimation behind it, a wrong discrete choice is not averaged out by more data.

The back end: filters and graphs

Two formulations, and the field moved decisively from the first to the second.

The filtering approach keeps a single current state — robot pose plus every landmark — and its full covariance matrix, updated with an extended Kalman filter as each measurement arrives. It is elegant and it does not scale: with n landmarks the covariance has O(n²) entries and each update touches all of them, so a few hundred landmarks is the practical ceiling. It also cannot revise the past. A linearisation made when the estimate was poor is baked in permanently, and the inconsistency it introduces cannot be undone.

The graph approach keeps everything. Each pose and landmark is a node, each measurement an edge carrying a relative constraint and an information matrix saying how much to trust it, and the estimate is the configuration minimising the total weighted squared error over all edges. This is a nonlinear least squares problem of the same family as bundle adjustment in structure from motion, and it is tractable for the same reason: the graph is sparse, since each pose connects only to its neighbours and to the few landmarks it saw. Sparse Cholesky factorisation solves systems with tens of thousands of poses in well under a second, and because the whole history is retained, a new measurement can correct an estimate made minutes ago.

Loop closure, and why a wrong one is fatal

Loop closure is recognising a previously visited place and adding an edge between two poses that are far apart in time. It is what converts accumulated drift into a bounded error.

a robot drives a 200 m loop with 1 % odometry drift

  accumulated position error on return = 2.0 m
  the map now shows a corridor that does not join up

adding one loop-closure edge asserting
"pose 1 and pose 900 are the same place" lets the
optimiser distribute that 2.0 m over the 900 intermediate
poses — roughly 2.2 mm of correction each, which is well
inside their individual uncertainties

Detection is a place-recognition problem: bag-of-visual-words over image features, learned global descriptors, or geometric descriptors for LiDAR. Whatever proposes the loop, the candidate must be verified geometrically before being accepted — register the two scans and check the residual and the inlier count.

The reason for that insistence is that a false loop closure is unrecoverable in a way no other error is. Every other constraint is approximately right and the optimiser reconciles them. A false loop asserts that two genuinely different places are one, and the optimiser will fold the map in half to satisfy it, corrupting every pose in between. Perceptual aliasing makes this a live risk in exactly the environments SLAM is deployed in: two identical office corridors, two rows of a warehouse, two floors of a car park. Robust cost functions — Huber or Cauchy kernels, or switchable constraints that let the optimiser downweight an edge it cannot satisfy — reduce the damage but do not remove the need for verification.

Where SLAM fails

• Featureless geometry. A long uniform corridor or a large empty warehouse floor is unobservable along one axis. Scan matching converges with a low residual at any position along the corridor — the sliding failure from registration — and the pose drifts while every diagnostic looks healthy.
• Dynamic scenes. Constraints derived from people, vehicles or a moving forklift are true at the instant of measurement and false a second later. They enter the graph as ordinary edges and pull the solution. Detecting and rejecting dynamic objects before the back end sees them is a required stage in any real deployment, not an optimisation.
• Pure rotation, for monocular systems. Turning on the spot gives no baseline, so no new landmark can be triangulated and existing ones cannot be refined. Fast rotation is a common cause of monocular tracking loss, and it is why visual-inertial systems, which get rotation directly from a gyroscope, are far more robust.
• Scale, for monocular systems. A single camera recovers the map up to an unknown global scale, for the same reason structure from motion does. Stereo, LiDAR, or an IMU with observable acceleration fixes it; nothing in the images can.
• Long sessions with no loops. A robot travelling in a straight line for a kilometre has no opportunity to close anything, and its error grows as if there were no SLAM at all. This is not a failure of the algorithm but a limit of the information available, and the answer is an external absolute reference — GNSS, a surveyed marker, a prior map — rather than a better back end.