Why a SageMaker Endpoint Costs More Than Expected

Almost every surprising SageMaker bill has the same cause, and it is not the instance type. A real-time endpoint bills for its instances from the moment it reaches InService until you delete it, whether or not anything invokes it. There is no idle rate and no scale-to-zero. Everything below is about proving that is what happened and deciding what to do.

What you are actually being billed for

Three separate things, and conflating them is what makes the bill confusing.

• Instance hours. The dominant term. InstanceType × InitialInstanceCount × wall-clock hours the endpoint exists. Invocations do not enter this calculation at all.
• Storage. The EBS volume attached to each hosting instance, billed per GB-month for as long as the instance exists.
• Data processed. Bytes in and out of the endpoint. Normally a rounding error, and worth checking only if you are moving large payloads.

Because the first term has no traffic component, a development endpoint left up over a weekend costs the same as a production endpoint serving continuously on the same hardware. Check current per-hour rates on Amazon SageMaker AI pricing; they vary by instance family and region and this page deliberately quotes none of them.

Confirming it in Cost Explorer

Before changing anything, establish that hosting is the line item. Filter Cost Explorer to the SageMaker service and group by usage type. Hosting usage types contain Host and name the instance class, so a line like USW2-Host:ml.g5.2xlarge is a hosting instance-hour charge and is what you are looking for. Training, processing and notebook usage types are distinct and are a different problem.

Then group by resource or by tag to attribute it. This is the argument for tagging endpoints at creation: without a tag, a bill with six hosting lines tells you the shape of the spend but not which team owns it. Divide the monthly hosting figure by 730 hours and by your instance count, and you have an effective hourly rate you can compare to the published one — if they match, the endpoint was up the whole month, and that is your answer.

Read utilization before resizing

The instinct on seeing a large hosting bill is to move to a smaller instance. Do the measurement first, because the fix depends on which way the endpoint is wrong, and a resize that ignores the data can make latency worse for no saving.

Four metrics, from the two SageMaker namespaces. Invocations and ModelLatency live in AWS/SageMaker; CPUUtilization, MemoryUtilization, GPUUtilization and DiskUtilization live in /aws/sagemaker/Endpoints. Both are dimensioned on EndpointName and VariantName.

aws cloudwatch get-metric-statistics \
  --namespace AWS/SageMaker \
  --metric-name Invocations \
  --dimensions Name=EndpointName,Value=my-endpoint \
               Name=VariantName,Value=AllTraffic \
  --start-time 2026-07-01T00:00:00Z \
  --end-time   2026-08-01T00:00:00Z \
  --period 3600 --statistics Sum

Sum Invocations over a month at hourly resolution and count how many hours were zero. That single number usually settles it. An endpoint with 600 idle hours out of 730 is not an instance-sizing problem.

Read CPUUtilization carefully. AWS documents it as the sum of each core’s utilisation, so on a four-core instance the range is 0–400% and a reading of 90% means the instance is about 22% busy, not 90%. People have downsized on that misreading and then wondered why latency doubled. MemoryUtilization and DiskUtilization are ordinary 0–100% percentages; GPUUtilization is multiplied by the GPU count in the same way as CPU.

Three causes that look identical

All three produce one large hosting number, and the fixes have nothing in common.

• An idle endpoint. Low invocations, low utilisation, continuous instance hours. Usually something deployed for an evaluation and never deleted, or an endpoint behind a feature that was turned off. This is the most common cause by a distance.
• An over-provisioned endpoint. Real traffic, but utilisation well under what the instance offers — frequently because InitialInstanceCount was set for a launch peak and never revisited, or because autoscaling has a MinCapacity set as a comfort blanket rather than a capacity decision.
• Endpoint sprawl. Each endpoint individually defensible; the total not. This is the one that hides from a per-endpoint investigation, and it is what ListEndpoints is for: list every endpoint in every region and check each one has an owner.

aws sagemaker list-endpoints \
  --query 'Endpoints[].{Name:EndpointName,Status:EndpointStatus,Created:CreationTime}' \
  --output table

What to change

Match the fix to the cause. In rough order of how much they save:

1. Delete what nothing calls. Deleting the endpoint stops the instance-hour charge; the model and endpoint config are metadata and cost nothing, so a deleted endpoint can be recreated from the same config in one call. This makes deletion far less drastic than it feels, and it is the correct action for anything with a month of zero invocations.
2. Move intermittent traffic off real-time hosting. Serverless inference bills per request rather than per hour. Asynchronous inference can genuinely scale its instances to zero — see async inference on SageMaker, including the second scaling policy it needs to wake up again. Both trade latency for the removal of the idle-hours term.
3. Autoscale, with an honest floor. A target-tracking policy removes the gap between peak provisioning and average load. Its value is capped by MinCapacity, so a policy from 3 to 10 saves far less than one from 1 to 10.
4. Right-size, last. Once utilisation is understood and correctly read, choose the smallest instance that holds your p95 latency. Do this after the other three, because resizing an endpoint that should not exist saves a fraction of what deleting it does.

Whichever you choose, add a budget alarm so the next surprise is a notification rather than an invoice, and tag every endpoint at creation so the next investigation starts with an owner instead of a usage type.