Gaussian Splatting, Explained
11 min read · updated August 5, 2026
A Gaussian splat scene is a few million translucent ellipsoids floating in space, each carrying a position, a shape, an opacity and a colour that changes with viewing angle. There is no network to evaluate at render time and no ray to march: the ellipsoids are projected to the screen, sorted by depth and blended, which is the operation graphics hardware has been built around for thirty years. Everything else about the method — its speed, its file sizes, its failure modes — follows from that one choice of representation.
What one Gaussian is, as data
A 3D Gaussian is not a mesh vertex and not a voxel. It is a probability density in space, an ellipsoidal blob with soft edges, stored as a fixed-length record of floating-point numbers. The whole scene is an array of these records and nothing else.
one Gaussian, spherical-harmonic degree 3:
position mu 3 floats (x, y, z) in world space
scale s 3 stored as log s, exp() applied on use
rotation q 4 quaternion, normalised on use
opacity o 1 stored as a logit, sigmoid() applied on use
colour SH 48 16 coefficients x 3 colour channels
----
59 floatsThe activations in that listing are not decoration. Scale is stored in log space because a scale must stay positive and gradient descent on a raw number will happily push it negative. Opacity is stored as a logit for the same reason, bounded to (0, 1) by a sigmoid. The quaternion is normalised before use because only a unit quaternion is a rotation. These three constraints are enforced by the parameterisation rather than by clipping, which is why the optimiser can be a plain Adam over unconstrained numbers.
Why scale and rotation instead of a covariance matrix
The shape of the blob is its covariance matrix, and a 3×3 symmetric matrix has six free numbers — one fewer than the seven stored above. Storing those six directly does not work. A covariance must be positive semi-definite to describe an ellipsoid at all, and nothing in gradient descent preserves that property; a few steps in and you have a matrix that describes no shape. Factorising it removes the problem by construction:
Sigma = R S S^T R^T S = diag( exp(s_x), exp(s_y), exp(s_z) ) scale R = rotation matrix built from q / ||q|| orientation R S S^T R^T is positive semi-definite for ANY q and ANY s, so no gradient step can produce an invalid Gaussian.
Read the factorisation physically: S stretches a unit sphere into an axis-aligned ellipsoid and R turns it. A splat that has converged onto a flat wall will have one very small scale component and two large ones — it has become a disc. Most Gaussians in a finished scene are close to discs, because most of a captured scene is surfaces.
Colour that depends on where you stand
The 48 numbers for colour are the coefficients of a spherical-harmonic expansion: a function on the sphere of viewing directions, evaluated per frame to give this splat its colour from this camera. The count is fixed by the degree.
coefficients per channel at degree d = (d + 1)^2 d = 0 1 x 3 channels = 3 flat colour, no view dependence d = 1 4 x 3 = 12 d = 2 9 x 3 = 27 d = 3 16 x 3 = 48 the common default
Degree 0 gives a splat one colour from every angle, which renders a matte world. Each degree added lets the colour vary more sharply as the camera moves, which is what puts the sheen on a table top and the glint on a door handle. It is also, as the next section shows, where almost the entire file goes.
Count the bytes: one splat, then one scene
Every claim about splatting being memory-hungry is this arithmetic, and it is short enough to do in front of you. Assume fp32 storage and degree 3, the defaults in the reference implementation.
per Gaussian:
59 floats x 4 bytes = 236 bytes
of which spherical harmonics: 48 x 4 = 192 bytes = 81%
whole scene, uncompressed:
1,000,000 Gaussians x 236 B = 236 MB
3,000,000 x 236 B = 708 MB
5,000,000 x 236 B = 1,180 MB
same scenes at degree 0 (14 floats = 56 bytes):
3,000,000 x 56 B = 168 MB -- 4.2x smallerFour fifths of a splat scene is view-dependent colour. That single ratio explains why every compression paper starts there, and why a scene that looks fine on a fixed camera path can often be shipped at a lower degree without anyone noticing.
The count of Gaussians is not a setting. It is an outcome of the optimisation, so scene size measures how complicated the room was. A blank wall converges to a handful of large discs; a bookshelf does not.
The bill during training is four times larger
The number that catches people out is not the file on disk, it is the GPU during fitting. Adam keeps two moment buffers per parameter, and the backward pass needs somewhere to put gradients:
per Gaussian, while optimising:
parameters 59 floats x 4 B = 236 B
gradients 59 x 4 B = 236 B
Adam m and v 118 x 4 B = 472 B
------
944 B
3,000,000 Gaussians -> 2.83 GB
...before the training images, the framebuffer, the per-tile
instance lists or the sort keys, all of which scale with the
number of Gaussians visible in a frame.This is why densification has to be bounded in practice and why a scene that grows past a few million primitives starts failing on consumer cards rather than merely getting slower. The same parameters-plus-optimiser-state arithmetic that decides how much VRAM a model needs applies here unchanged; only the meaning of a parameter differs.
Why projecting and sorting is fast
The representation would be of no interest if drawing it were slow. The reason it is not slow is that a 3D Gaussian projects to a 2D Gaussian under an affine camera, and a 2D Gaussian’s integral along a ray has a closed form. There is no quadrature, no sampling along the ray, no per-sample function to evaluate.
project (Zwicker and colleagues, EWA splatting, 2001):
Sigma_2D = J W Sigma W^T J^T then drop the third row and column
W = world-to-camera rotation
J = Jacobian of the affine approximation to the projective
transform, taken at this Gaussian's centre
shade one pixel at offset d from the projected centre:
alpha_i = o_i * exp( -0.5 * d^T Sigma_2D^-1 d )
C = sum_i c_i * alpha_i * prod_{j<i} (1 - alpha_j)That blending formula is the same front-to-back compositing a volumetric renderer performs, which is the point: splatting does not approximate a different integral, it evaluates the same one analytically per primitive instead of numerically per sample. The radiance-field page counts what the sampled version costs — tens of millions of network evaluations for one frame. Here is the other side of that comparison.
1920 x 1080, 16 x 16 pixel tiles: tiles = ceil(1920/16) x ceil(1080/16) = 120 x 68 = 8,160 pixels = 2,073,600 once per visible Gaussian (assume 1.5M survive frustum culling): project mean and covariance tens of FLOPs evaluate 16 SH coefficients x 3 a few hundred FLOPs -> order 10^8 to 10^9 FLOPs for the whole frame per pixel, per overlapping Gaussian: one exponential, one multiply for alpha, three multiply-adds at B = 50 blended splats per pixel: 2,073,600 x 50 = 1.04 x 10^8 blend operations
Assumptions, stated: the 1.5 million visible primitives and B = 50 are illustrative, not measured. B is the number this arithmetic turns on and it is entirely scene-dependent — foliage and smoke run far higher than an empty corridor. Substitute your own and the shape of the answer does not change: the total lands around 109 arithmetic operations, several orders below what per-sample network evaluation demands for the same frame.
Three implementation details do the rest of the work, and all three are why the method arrived in 2023 rather than 2003.
- Tiling. The screen is cut into 16×16 tiles. Each projected Gaussian is duplicated once per tile it touches, so a tile’s pixels share one list of candidates in fast shared memory instead of every pixel searching the scene.
- A single radix sort. The duplicated instances are given a 64-bit key of tile index in the high bits and quantised view depth in the low bits, then sorted once, globally, with a GPU radix sort. Radix sorting is linear in the number of keys, not
n log n, so ordering several million primitives is a fixed handful of passes rather than the bottleneck people assume it is. - Early termination. Blending front to back means accumulated transmittance only falls. Once it drops below about 10-4 the remaining splats behind cannot change the pixel, and the thread stops. Opaque scenes therefore blend far fewer than the full candidate list; translucent ones do not, which is exactly why smoke and leaves are the slow cases.
The sort is per-primitive, not per-fragment: every pixel in a tile uses the same depth order, taken from each Gaussian’s centre. That approximation is nearly free and almost always right, and when it is wrong — two large splats interpenetrating — their order can flip as the camera moves, which is the origin of the popping artefact splat scenes are known for.
The population grows itself
Nothing in the representation says how many Gaussians a scene should have, and no one sets it. The optimiser adds and removes primitives as it goes, which is the part of the method with no counterpart in ordinary network training.
- Initialise from the sparse point cloud. Structure-from-motion has already produced camera poses and, as a by-product, a sparse cloud of scene points. Each point becomes one small, roughly spherical Gaussian carrying that point’s colour. Typically this is on the order of a hundred thousand primitives — one or two per cent of what the finished scene will hold.
- Render, compare, backpropagate. Rasterise from a training camera, take the loss against the real photograph, and push gradients into positions, scales, quaternions, opacities and every spherical-harmonic coefficient. All of them are ordinary leaf parameters. The rasteriser is written to be differentiable, which is the only reason any of this is possible.
- Densify where the gradient says detail is missing. Every hundred iterations or so, look at the magnitude of each Gaussian’s positional gradient in view space, averaged over recent views. A large value means the primitive is being pulled in different directions by different photographs — it is trying to explain more detail than one blob can hold.
- Clone or split, according to size. If such a Gaussian is small, the region is under-reconstructed: copy it and offset the copy along the gradient. If it is large, the region is over-reconstructed: split it into two, divide the scale by a constant of roughly 1.6, and draw the two new centres from the original Gaussian’s own distribution so the pair occupies the volume the parent did.
- Prune, and periodically reset opacity. Any Gaussian whose opacity falls below a small threshold is deleted, as are those that have grown implausibly large. Every few thousand iterations all opacities are pushed back down to near zero, so each primitive has to re-earn its opacity from the photographs; whatever fails to is pruned on the next pass.
The opacity reset deserves the attention it rarely gets. Without it, the optimiser has a cheap way to reduce loss near the training cameras: park a faint blob just in front of the lens and let it act as a coloured filter. These “floaters” cost nothing to keep and look catastrophic from any other viewpoint. Forcing every primitive back to near-transparency periodically is what stops the scene filling with them.
What actually shrinks a scene
Because the byte count decomposes cleanly, so does the compression problem. None of these techniques touch the rasteriser; they change only what is handed to it.
| Technique | Description |
|---|---|
| Lower the SH degree | Degree 3 to degree 1 removes 36 of 48 colour floats, taking a splat from 236 to 92 bytes — a 2.6× reduction from one line of config. The cost is specular response: surfaces go flatter as the camera moves. |
| Half precision on the coefficients | The SH coefficients tolerate fp16 far better than positions do, because a colour error of one part in a thousand is invisible while a position error of the same size is not. Degree 1 with fp16 harmonics gives 44 + 24 = 68 bytes, 3.5× below the fp32 default. |
| Codebooks over the attributes | Scales, rotations and harmonics repeat heavily across a scene, so a learned codebook plus a per-splat index beats storing each value. This is ordinary vector quantisation, with the same recall-for-memory trade it has everywhere else. |
| Prune by contribution | Score each Gaussian by how much it actually changes rendered pixels across the training views, then delete the tail. Scenes routinely tolerate losing a large fraction of their primitives before any visible difference appears, because densification is deliberately generous. |
Degree 1 with fp16 harmonics alone takes a three-million-Gaussian scene from 708 MB to 204 MB, before a codebook or a single Gaussian is pruned. That is the difference between a scene that streams to a phone and one that does not, which is why compression is the most active part of the area.
What splatting cannot do
The representation buys speed with memory and with a strong assumption about what the world is made of. Both bills come due in specific places.
| Limit | Description |
|---|---|
| Memory is the binding constraint | Hundreds of megabytes per room, and roughly four times that in GPU memory while fitting. A mesh of the same room is single-digit megabytes. Anything involving a network, a phone or many scenes resident at once runs into this before it runs into frame time. |
| Reflections are painted on | A mirror is geometry seen through a surface; view-dependent colour on a splat at that surface can only approximate it as paint that changes with angle. Sixteen coefficients per channel is a very low-frequency function on the sphere, so a sharp highlight is not representable — the optimiser instead inserts primitives behind the glass that look correct from the training views and wrong from anywhere else. |
| Thin structures fight the primitive | A wire, a leaf edge or a railing needs extremely anisotropic Gaussians. Those project to near-degenerate ellipses at grazing angles, so they flicker as the camera turns, and densification answers the residual error by spending thousands of primitives on a structure a mesh would describe with four vertices. |
| Editing has nothing to grab | There is no object, no part and no semantics — only a few million unlabelled blobs. Moving a chair means segmenting it out of that soup first, and rotating what you extract requires rotating each spherical-harmonic basis with it, or the lighting comes out wrong. Deformation and animation are open work for the same reason. |
| It reconstructs, it does not invent | A fitted scene is a compression of photographs that were actually taken. Regions no camera saw are filled with whatever reduced the loss on the views that exist, which is usually floaters. This is the opposite of a generative model: no prior over scenes, no ability to complete an unobserved corner, and no transfer at all from one scene to the next. |
The last row is the misunderstanding worth correcting most often. Splatting sits beside photogrammetry, not beside image generation. The optimisation is gradient descent, the pipeline is differentiable and the vocabulary is borrowed from deep learning, but the artefact produced is a measurement of one room, and it knows nothing whatsoever about any other.