Voxel Grids and Point Clouds: Why the Representation You Pick Changes the Model

A point cloud stores what the sensor measured. A voxel grid stores every place the sensor could have measured, whether it did or not. That difference decides your memory bill, your neighbourhood query, and whether a convolution is available to you at all.

Two ways to store the same measurements

A point cloud is a list. Each entry is a coordinate triple plus whatever attributes came with it — intensity, return number, RGB — and the file grows with the number of measurements, not with the size of the space. Nothing is stored for empty regions because empty regions are not represented at all.

A voxel grid is an array. Space is divided into cubes of side s and each cube holds a value: occupied or not, a count, a mean intensity, a learned feature vector. The array has an entry for every cube whether anything is there or not, so its size is set by the volume and the resolution and is completely independent of how many points you measured.

The consequence people meet first is that a voxel grid destroys precision. Snapping a point to its cube centre introduces an error up to s/2 in each axis; for uniformly distributed points the root-mean-square error per axis is s/√12, about 0.289 s. At a 5 cm voxel that is 1.4 cm of RMS error added to data your scanner may have measured to 3 mm. The precision is gone and no later stage recovers it.

Worked: the memory of one 10-metre room

Take a cube 10 m on a side containing roughly 300 m² of surface — floor, walls, some furniture — scanned into 5 million points with three float32 coordinates and one float32 intensity.

point cloud
  5,000,000 points x 16 bytes = 80,000,000 bytes = 76 MiB
  independent of the resolution you later choose

dense voxel grid over the same 10 x 10 x 10 m cube
  s = 0.20 m   50 x 50 x 50       =        125,000 cells
  s = 0.05 m   200 x 200 x 200    =      8,000,000 cells
  s = 0.01 m   1000 x 1000 x 1000 =  1,000,000,000 cells

at 1 byte per cell (occupancy only)
  s = 0.20 m   125 KB
  s = 0.05 m   8 MB
  s = 0.01 m   1 GB

at a 16-channel float32 feature per cell (a network's first layer)
  s = 0.05 m   8,000,000 x 16 x 4 = 512 MB
  s = 0.01 m   1,000,000,000 x 16 x 4 = 64 GB

Halving the voxel size multiplies the cell count by eight. That is the whole story of why dense 3D convolution did not survive contact with real scenes: the resolution you want for a 3 cm feature makes the tensor larger than any accelerator’s memory, and the point cloud it came from was 76 MiB.

Occupancy scales with the voxel size

The grid is mostly empty, and how empty is worth deriving because it is not intuitive. Points from a scanner lie on surfaces, and a surface is two-dimensional. The number of voxels a surface of area A touches is approximately A / s², while the total number of voxels in a volume V is V / s³. The occupied fraction is therefore:

occupied / total = (A / s^2) / (V / s^3) = A * s / V

with A = 300 m^2 and V = 1000 m^3:

  s = 0.20 m  ->  300 * 0.20 / 1000 = 6.0 %
  s = 0.05 m  ->  300 * 0.05 / 1000 = 1.5 %
  s = 0.01 m  ->  300 * 0.01 / 1000 = 0.3 %

occupied voxel counts
  s = 0.20 m  ->     7,500
  s = 0.05 m  ->   120,000
  s = 0.01 m  -> 3,000,000

Refining the grid makes it emptier, not fuller. The occupied count grows quadratically while the total grows cubically, so every step toward the resolution you actually want makes the dense representation proportionally more wasteful. At 1 cm, 99.7% of a billion cells hold nothing and the network spends 99.7% of its arithmetic on them.

What each representation makes computable

• Neighbourhood lookup. In a grid, the neighbours of a cell are its index ± 1 in each axis: constant time, no data structure. In a point cloud, finding the points within a radius means building and querying a k-d tree or an octree, and the query cost is in the inner loop of normal estimation, registration and clustering alike.
• Convolution. A grid is a tensor, so ordinary convolution applies with weight sharing and translation equivariance for free. On raw points there is no convolution; you build an analogue out of neighbourhood aggregation and a symmetric function, as on the classification page.
• Free space. This is the one that decides robotics arguments. A point cloud can say “there is a surface here” and cannot say “there is nothing here” — an absence of points means unmeasured, occluded, or empty, and the three are indistinguishable. A voxel grid can carry three states, and ray casting from the sensor origin marks every cell along the beam as free. Path planning needs that distinction; classification does not.
• Fixed-shape batching. Training wants tensors of equal shape. A grid over a fixed volume is always the same shape; a point cloud is not, and every framework needs either padding, resampling to a fixed count, or ragged-batch machinery.
• Exact measurement. Only the point cloud has it. If the deliverable is a distance between two surfaces to the millimetre, quantising to a grid has already thrown the answer away.

Sparse structures, which are the actual answer

Nobody with a real workload picks one of the two extremes. The practical structures store the grid’s addressing scheme without the grid’s memory.

An octree subdivides only where there is content, so a large empty region costs one node. This is the structure behind the OctoMap occupancy mapping used in robotics and behind the geometry coder in MPEG’s G-PCC — see point cloud compression, where the octree is the compression rather than merely the index. A spatial hash map keyed on integer cell coordinates gives constant-time lookup with storage proportional to occupied cells only, which is what sparse convolution libraries build on: they keep a table of active cells and compute outputs only where inputs exist, so a network over the 1 cm grid above touches 3 million cells rather than a billion.

The pillar trick used in LiDAR detection is a different reduction of the same problem — collapse the vertical axis entirely so the grid is 2D and the cell count drops by the depth factor. It works because outdoor scenes are effectively height fields from a driving perspective, and it fails in a multi-storey car park for exactly that reason. See 3D object detection from LiDAR for the arithmetic of that trade.

Choosing

Keep raw points when the output is a measurement, when precision is the deliverable, when the data is going into an archive that will be reprocessed with unknown future tools, or when the density varies so much across the scene that no single voxel size is right for all of it. Voxelise when the output is a decision — a class, a box, an occupancy map — when you need convolution, or when downstream code requires a fixed-shape tensor.

The practical pattern is to voxelise late and never in place. Keep the archival cloud at full precision, derive a grid at whatever resolution the current model wants, and treat that grid as a cache. Voxelising on ingest, before you know what will be asked of the data, is the version of this decision that cannot be undone: the 1.4 cm of RMS error from a 5 cm grid is now in your source of truth, and re-deriving a finer grid from a coarser one recovers nothing.