Compute Cost Curve

Problem solved

Hyperscaler procurement teams, neocloud capex committees, and PE infrastructure investors need an apples-to-apples cost per effective FLOP across chip generations and deployment archetypes. Spec sheets do not produce this number. The framework produces it under named utilization, useful-life, and energy assumptions, with the sensitivity ranking that matters for procurement.

Inputs

• Acquisition price per accelerator (H200, B200, GB200, MI300X, TPU v5p, custom), with 12 month forward curve where observable
• MLPerf training and inference results, normalized to dense FP8 throughput
• Power draw at sustained load and idle, including NVLink and host CPU overhead
• Cooling architecture (air, direct-to-chip, immersion) with PUE assumption by site type
• Industrial electricity tariff at the site (EIA 861, ISO/RTO LMP, named PPA)
• Network fabric (NVLink, NDR Infiniband, RoCE) and the implied scaling efficiency from MLPerf strong-scaling tables
• Useful-life and depreciation schedule (24, 36, or 48 month base cases)
• Utilization profile by workload (training-heavy, inference-heavy, mixed) with idle fraction

Outputs

• All-in dollars per effective FP8 PFLOP-hour at named utilization
• Channel decomposition: capex per hour, power per hour, cooling per hour, network amortization per hour, idle drag
• Tornado chart of TCO sensitivity to electricity price, utilization, useful life, and PUE
• Dollars per million tokens for inference workloads at named batch sizes and quantization
• Crossover utilization: the point at which a newer chip undercuts an older chip on TCO terms
• Scenario tree for forward chip prices and electricity prices

Method

1. Step 1. Normalize throughput. Take MLPerf dense FP8 training and inference results, scale to the workload mix in the engagement scope, and convert to effective PFLOP-hours. NVLink, Infiniband, and weak-scaling versus strong-scaling deltas are kept separate.
2. Step 2. Build the capex line. Acquisition price per accelerator plus the network fabric, host servers, and rack-level integration. Spread across the useful-life base case (default 36 months for training, 48 months for inference). Salvage value defaults to 5 percent.
3. Step 3. Build the power line. Sustained load times utilization plus idle draw times idle fraction, multiplied by the site electricity tariff. Behind-the-meter PPA prices override grid tariffs where contractually applicable.
4. Step 4. Build the cooling line. Apply PUE by cooling architecture: 1.40 for legacy air, 1.20 for direct-to-chip, 1.05 for immersion. Multiply IT load by PUE minus one to get cooling overhead. Water-side cooling adds a separate water-cost line for jurisdictions where water is constrained.
5. Step 5. Build the network line. NVLink within the rack is amortized into capex; cross-rack NDR Infiniband or RoCE has its own capex amortization plus power. Scaling efficiency loss above 1,024 GPUs gets a tax that lowers effective FLOPs.
6. Step 6. Sum, divide, sensitivity. Add all lines, divide by effective FLOP-hours at named utilization, run the tornado on electricity price, utilization, useful life, and PUE. Report the crossover utilization where the candidate undercuts the incumbent.

Assumptions

• MLPerf dense FP8 throughput is the comparison currency. Sparse and reduced-precision throughput is reported separately as a sensitivity case.
• Useful life defaults to 36 months for training and 48 months for inference. Hyperscalers are increasingly running 60-month inference deployments; the framework reports both.
• Idle draw is a real cost, not a rounding error. The framework requires an explicit idle-fraction input.
• Electricity price is annualized and held constant within the depreciation window for the base case. A separate scenario tree handles forward-curve sensitivity.
• NVLink and Infiniband scaling losses follow the published MLPerf strong-scaling tables. Custom fabrics require an engagement-specific calibration.

Limitations

• Real-world utilization at hyperscaler scale is rarely the spec-sheet utilization. The framework requires the operator's own utilization data; in the absence of it, results carry an explicit utilization-band warning.
• Custom accelerator pricing is often opaque. The framework runs three pricing scenarios (low, mid, high) where contracts are confidential.
• Software stack maturity (CUDA, ROCm, custom) does not appear directly in the cost stack but is captured implicitly through MLPerf throughput.
• Inference token economics are workload-specific. Generic dollar per million tokens numbers are not portable across model class, context length, or batching strategy.

Example application

Applied to a 2026 hyperscaler procurement decision: H200 versus B200 versus GB200 NVL72 for a 50,000 accelerator training cluster on PJM with a 120 dollar per MWh blended power tariff and 36 month depreciation. The framework runs the six steps, produces the dollar per effective FP8 PFLOP-hour for each candidate at 65 percent and 80 percent utilization, and identifies the utilization crossover where GB200 economics dominate H200 even at the higher acquisition price. See Hyperscaler GPU procurement 2026.