Detecting Building Footprints From Satellite Imagery
10 min read · updated August 11, 2026
A segmentation model gives you a raster of building-or-not. Turning that into polygons a surveyor would recognise is three more steps, and each one has a documented way of going wrong.
The resolution floor
Ground sample distance decides whether the task is possible at all, and the arithmetic is short.
GSD area per pixel a 130 m^2 house is... 10 m 100 m^2 1.3 pixels 3 m 9 m^2 14 pixels 0.5 m 0.25 m^2 520 pixels 0.3 m 0.09 m^2 1,444 pixels
At Sentinel-2’s 10 m a detached house is barely more than one pixel, so no model recovers its outline — the information is not in the image. Sub-metre imagery is the floor for footprints, and 0.3 m is where the corners become crisp enough for the vectorisation step to produce clean right angles.
Going the other way, converting a mask back to area is the same multiplication:
blob of 1,480 connected pixels at 0.3 m GSD area = 1,480 x 0.3 x 0.3 = 1,480 x 0.09 = 133.2 m^2
One caveat on that conversion: it is only valid if the raster is in a projected CRS whose units are metres and whose scale is locally correct. Doing it on imagery in geographic coordinates, where a pixel is a fraction of a degree, gives an area that varies with latitude — and doing it in Web Mercator gives an area inflated by the square of the local scale factor, which at 60° latitude is a factor of four.
A mask is not a set of buildings
Semantic segmentation — a U-Net or a transformer variant predicting building-or-not per pixel — produces connected blobs. Where buildings are detached, one blob is one building and the distinction does not matter. Where they are not, it matters completely.
A terrace of twelve houses shares party walls. There is no building-free pixel between them, so semantic segmentation produces a single blob, and vectorising it produces one polygon of roughly 1,600 m² where there should be twelve of roughly 130 m². Every count is wrong by a factor of twelve and every area is wrong by the same factor in the other direction. This is the single largest error source in footprint extraction over European and Asian cities, and it is invisible in an aggregate area metric because the total area is correct.
Two standard treatments. Predict an extra boundary channel alongside the interior channel, then take interior-minus-boundary as seeds and run a watershed to split the blob — this keeps a fast semantic architecture and adds one output. Or use instance segmentation, where the model proposes objects and predicts a mask per object, which handles the split natively and costs considerably more per tile. Either way, evaluate on instance counts and not only on pixels, or the failure does not appear in your numbers.
From mask to polygon
Three steps, and the middle one has a parameter worth understanding.
- Contour extraction. Marching squares traces the boundary of the binary mask, producing a polygon whose vertices sit on pixel corners. The output is a staircase: every edge is axis-aligned and one pixel long.
- Simplification. Douglas-Peucker removes vertices whose perpendicular distance from the chord between their neighbours is below a tolerance. The tolerance is in ground units and it trades two failures against each other. At 0.3 m GSD, a tolerance of 0.15 m (half a pixel) removes nothing and leaves the staircase; 1 m gives clean straight walls; 3 m starts cutting corners off the building itself. The right value scales with GSD, not with building size.
- Regularisation. Buildings are overwhelmingly rectilinear, and simplified contours are not. Estimate the dominant edge orientation of each polygon, then snap edges to that orientation and its perpendicular within a tolerance. This is what makes output look like a cadastre rather than like a traced photograph, and it is a strong prior that is simply wrong for circular and curved structures — stadiums, silos, some churches — which come back as polygons with a few long straight sides.
Regularisation improves human judgement of the output more than it improves overlap metrics, and occasionally lowers them. That gap between what scores well and what looks right is worth knowing about before you tune against a single number.
The roof is not the footprint
Satellite imagery is rarely acquired straight down. At an off-nadir view angle, a building’s roof is displaced in the image relative to its base by an amount that depends on height:
displacement = height x tan(view angle from nadir) 20 m building, 5 deg off-nadir: 20 x 0.0875 = 1.75 m 20 m building, 20 deg off-nadir: 20 x 0.3640 = 7.28 m 20 m building, 30 deg off-nadir: 20 x 0.5774 = 11.55 m 60 m building, 20 deg off-nadir: 60 x 0.3640 = 21.8 m
What a segmentation model outlines is the roof, because that is what is visible. So the polygon you extract is the footprint translated by several metres in the direction away from the sensor, and the taller the building the larger the offset. In a dense city the roof of one tower overlaps the ground position of its neighbour, which is why footprints extracted from high off-nadir imagery of a downtown look plausible individually and are collectively sheared.
Correcting it requires the view geometry — carried in the image metadata as rational polynomial coefficients — and a height estimate, from stereo pairs, LiDAR, or a model that predicts height directly. Where none of that is available, the honest response is to record the view angle with the output and treat the polygons as roof outlines, which for counting buildings is fine and for property boundaries is not.
The same displacement is a hazard when comparing two dates: two scenes acquired at different view angles show the same unchanged building at different positions, which a naive difference reads as change. That interaction is covered in change detection over time.
Metrics that mean something
Pixel accuracy is useless here and it is worth seeing why in numbers. Suppose buildings cover 8% of a scene’s pixels:
predict "no building" everywhere pixel accuracy = 92% buildings found = 0
Intersection over union is the standard replacement. For a single instance it is the overlap area divided by the union area, and instance-level F1 at an IoU threshold — usually 0.5 — is the metric the SpaceNet challenges established for this task. Report both: IoU alone rewards getting the big buildings right, instance F1 alone rewards finding them without regard to shape, and the terrace-merging failure above only shows up in the instance count.
Boundary-focused metrics matter more than they appear to. A polygon can reach IoU 0.9 while having a ragged, non-orthogonal outline that no cadastral system would accept, because IoU is an area measure and the raggedness is confined to a thin band along the perimeter. If the output feeds a mapping product rather than a statistic, measure the boundary directly.
On training data: SpaceNet, Microsoft’s Building Footprints releases and Google’s Open Buildings are the standard public starting points, and they differ substantially in geographic coverage, licence and vintage. Check all three of those properties against your use before building on any of them, particularly the licence, which is the one that cannot be fixed later.