GPU cost attribution

Commitment purchased and not consumed: $1,216/mo of Reserved Instance capacity and $760/mo of Savings Plan commitment

PROVENvalidation: synthetic fixture

What to do.Sell unused Standard RIs on the Reserved Instance Marketplace where the term allows, or move workloads onto the committed instance families so the commitment is consumed. Savings Plan commitments cannot be sold; the recovery is to shift eligible usage onto them before the term ends.

23 of 63 provisioned GPUs had no pod requesting them when the cluster was read, $60,432/mo

Assumption.The cluster was read once, at 2026-09-01T00:00:00+00:00, and this figure applies that single reading to the whole billing period. One reading proves what was allocated at that instant and nothing about the hours either side of it. Sample allocation hourly for the period and whatever is observed becomes proven; until then this is an extrapolation and is rated as one.

What to do.Scale the affected node groups down to the number of GPUs actually requested. Where a node group exists for burst capacity, set its floor to zero and let the cluster autoscaler bring it up on a pending pod.

14 production GPU instances run continuously at on-demand rates, $153,949/mo

Counted once.Taken on its own this finding identifies $33k per month. The figure above is what survives de-duplication and is what the recoverable total uses. The difference, $8,626 per month, sits on machines already claimed by gpu zero allocation, which are counted first because PROVEN findings go before LIKELY ones and the order of work decides within a rating. A machine is counted once however many findings describe it.

Assumption.The recoverable figure applies the one-year no-upfront Compute Savings Plan rate AWS publishes for each family (g5 21.25%, g6e 24.40%, p3 18.62%, p4d 23.92%, p5 21.25%), which comes to 21.48% across this set, and assumes the capacity is still wanted twelve months from now. The rates were read from the AWS Savings Plans and EC2 price lists for us-east-1, both published 2026-09-10; re-read them for another region, because they differ by region. If the workload is being migrated or retired inside the term, this finding does not apply.

What to do.Size a Compute Savings Plan to the observed floor of GPU usage, not to the peak. Commit to the level that has been running every day of the period and leave the rest on demand.

9 untagged instances read as non-production and run continuously, $11,916/mo

Assumption.These resources carry no Environment tag. Each is classified from its account, its name, or its instance family, and the signals are listed per resource below. The finding holds if the client confirms the classification; strike any row that is wrong and the total falls by that row's cost. The recovery then carries the same working-week assumption as the tagged finding above. It treats a 50-hour working week as the time these machines are needed, which leaves 118 of the 168 hours in a week recoverable by scheduling, or 70.2% of the line. A different working pattern moves the figure in proportion, so confirm the hours and rescale. A machine genuinely needed around the clock recovers nothing here, whatever its tag says.

What to do.Confirm the classification, tag the resources so this stops being an inference, then schedule them off outside working hours.

10 tagged non-production instances run continuously, $6,495/mo, $4,706/mo of it GPU

Assumption.The cost is read straight from the bill and the Environment tag is the client's own, so neither is in question. The recovery is. It treats a 50-hour working week as the time these machines are needed, which leaves 118 of the 168 hours in a week recoverable by scheduling, or 70.2% of the line. A different working pattern moves the figure in proportion, so confirm the hours and rescale. A machine genuinely needed around the clock recovers nothing here, whatever its tag says.

What to do.Stop non-production instances outside working hours with an EventBridge schedule or an autoscaler floor of zero. GPU instances should be stopped rather than resized, because the accelerator is the entire cost.

4 EBS volumes and snapshots billed for the full period with no associated compute, $320/mo

Assumption.A CUR does not carry EBS attachment state, so this is inferred from a volume billing every day of the period while nothing in the account consumed compute hours against it. Confirm with `aws ec2 describe-volumes --filters Name=status,Values=available` before deleting anything. Snapshots that are the only copy of a dataset are not waste regardless of what the billing says.

What to do.Snapshot then delete volumes confirmed as available. For snapshots, apply a lifecycle policy rather than deleting by hand.

$22,326/mo on NVIDIA V100 (p3), superseded silicon

What is missing.Whether a newer family is cheaper for these workloads depends on model size, precision, memory footprint and batch size. None of that is in a Cost and Usage Report. Closing this gap needs one benchmark run of the real workload on a current-generation instance, measuring throughput per dollar rather than utilization. That is a day of work per distinct workload and it is the only thing that will settle it.

What to do.Benchmark the largest single workload on a current-generation instance before changing anything. Do not migrate a fleet on the strength of a spec sheet.

$309,670/mo of GPU spend has no utilization telemetry (100% of GPU spend)

What is missing.No per-GPU utilization data exists for these instances, so no statement can be made about whether the silicon did work. Closing this gap means running the NVIDIA DCGM exporter with profiling metrics enabled and scraping DCGM_FI_PROF_SM_ACTIVE and DCGM_FI_PROF_PIPE_TENSOR_ACTIVE into Prometheus. Note that DCGM_FI_DEV_GPU_UTIL, the metric on most default dashboards, does not close this gap: it reports whether a kernel was resident, not how much of the chip it used. In our own lab measurement on one H100 rented on Lambda, published at nameplateanalytics.com/method, two configurations of one training job read 99.5% and 99.3% GPU utilization while one produced 26.6 times the real tokens per second of the other. That is one GPU and a configuration built for the test, so it shows what the reading misses, not how often. Export the result with its sample count and the span the samples cover; a mean on its own cannot be tied to any part of the billing period.

What to do.Deploy the NVIDIA DCGM exporter, add DCGM_FI_PROF_SM_ACTIVE to its counters file, which the shipped default comments out, and give the container SYS_ADMIN so the profiling fields can be read. Pod and namespace mapping comes from --kubernetes, which the published manifests already set; pod labels need --kubernetes-enable-pod-labels, which they do not. Two weeks of data is enough to size the next engagement.

Showing the 25 largest of 40 resources. The full list is in the machine-readable findings file that accompanies this report.

$97,207/mo of GPU capacity is claimed by a pod but has no utilization data

What is missing.A pod holding a GPU proves the accelerator is reserved, not that it is busy. Whether these GPUs are doing work needs DCGM_FI_PROF_SM_ACTIVE, and for tensor workloads DCGM_FI_PROF_PIPE_TENSOR_ACTIVE, scraped per pod. Two things are usually in the way. DCGM_FI_PROF_SM_ACTIVE is commented out of the exporter's default counters file, so it has to be added and the exporter needs SYS_ADMIN to read it. And pod labels are only attached when the exporter runs with --kubernetes-enable-pod-labels, which the published Helm chart leaves off; pod and namespace mapping itself comes from --kubernetes, which the shipped manifests do set.

What to do.Add DCGM_FI_PROF_SM_ACTIVE to the exporter's counters file, give the container SYS_ADMIN, and turn on --kubernetes-enable-pod-labels so the metrics carry the pod they belong to. Scrape for two weeks, then re-run this audit. Nothing should be resized before that data exists.