The scope shift is not a consolidation story
The expectation for most of the last decade was that on-premise infrastructure would shrink toward irrelevance and the interesting cost problems would all be cloud problems.
That is not what happened. The estate went hybrid and stayed hybrid. Regulated workloads did not move. Data gravity kept large datasets where they were. And the arrival of accelerated computing pushed a great deal of new capital expenditure back into owned hardware, because at sustained high utilization, buying accelerators is frequently cheaper than renting them.
So the discipline that learned to manage the cloud half is now being handed the other half, and the other half is often larger.
Where teams actually are with it
Here is the finding worth sitting with, because the obvious interpretation is wrong.
The obvious interpretation is that organizations have started optimizing their on-premise estates. They have not.
When the 2026 survey asked which capabilities practitioners are prioritizing for data center spend, the answer came back in this order: allocation, then planning and estimating, then forecasting and budgeting, then data ingestion and reporting. Workload optimization ranks fifth.
Compare that to how cloud went. In cloud, optimization was the opening move, because the invoice made waste impossible to ignore and there was an obvious lever to pull. On premise there is no invoice, so there is no forcing function, so teams are a full maturity cycle back. They are trying to establish what they own, what it cost, and which part of the business should carry it.
That sequencing is correct, incidentally. You cannot optimize what you cannot see, and a report handed to a team with no mechanism to act on it does not change anything either. But it does mean that most organizations reporting that they now manage data center spend are, in practice, at the accounting stage rather than the efficiency stage.
Four assumptions that fail at the boundary
Point a cloud cost tool at an on-premise cluster and it will produce output. The output will be confidently wrong, which is more dangerous than no output, because somebody will put it in a board pack.
Four things break.
There is no invoice
The entire cloud cost feedback loop starts with a monthly bill that attributes spend to resources. On premise, the money was spent years ago as capital expenditure, and it is now being recognized as depreciation on a schedule that has no relationship to what the hardware is doing this week.
There is no event that says "this node was idle in August and it cost you." The signal that drives every cloud cost workflow simply does not fire.
Capacity is not elastic
In cloud, the remedy for over-provisioning is to provision less. The saving is immediate and real, because you stop paying for the capacity the moment you stop using it.
On premise, provisioning less does not save anything. The hardware is already bought, already racked, already drawing power. Freeing up capacity produces a different asset: headroom. That headroom is only worth money if you can do something with it, which means either deferring the next purchase, absorbing a workload that would otherwise have gone to cloud, or making it available to another team or tenant.
This is not a smaller problem than the cloud version. It is a different one, and it has a different economic case that has to be argued rather than shown on a bill.
The tool often cannot act
In cloud, a third-party controller with write access is a manageable conversation for most commercial organizations. In the environments where a great deal of on-premise compute actually lives, banking, healthcare, defense, government and research, it frequently is not grantable at all.
This is the constraint that eliminates whole categories of tooling before an evaluation begins. It is covered in more depth in what national and regulated infrastructure runs into, which is the extreme version of the same problem.
The unit of account is wrong
Cloud cost tooling prices in hours of a named instance type. That works because the provider publishes a rate and bills against it.
A depreciating asset does not have an hourly rate. It has an acquisition cost, a useful life, a power draw, a cooling overhead, a share of facility cost, a maintenance contract, and a utilization profile. Turning that into a defensible per-hour number is possible, and we will get to how, but it is a modeling exercise rather than a lookup.
What to measure instead
The useful question is identical in both environments. What was allocated, what was consumed, and what is the gap? Only the instrumentation differs.
Measure capacity, not currency, at least to begin with. Node hours and GPU hours are comparable across cloud and on-premise. Currency is not, because one side is billed monthly and the other is a depreciating asset. Establish the capacity picture first, then apply cost models to it. Comparing a cloud invoice directly against depreciated capital expenditure produces a number that means nothing and will not survive its first challenge from finance.
Use device telemetry as the source of truth. Scheduler records tell you what somebody asked for. They do not tell you what the hardware did. On the accelerator side specifically, this gap is enormous, and a cluster reporting high utilization can still be substantially idle precisely because the number being reported measures allocation rather than work. Use device telemetry rather than scheduler records wherever the hardware exposes it.
Group by workload class, not by platform. A batch training job has the same economics whether it runs under a batch scheduler or as a container. A long-running service has the same economics either way. It is more useful to group by workload class rather than by scheduler than to maintain two separate cost programs that never reconcile.
Account for the boundary honestly. Data movement between environments has real cost, and in hybrid estates it is routinely larger than the compute savings from any placement decision. Egress is the item most often left out of a cloud-versus-on-premise comparison, and leaving it out is usually what makes the comparison come out the way the author wanted.
Building a defensible internal rate
At some point somebody will ask what an hour on your own hardware costs, and the answer needs to survive scrutiny.
The components are not controversial. Acquisition cost amortized over the useful life you actually use, not the accounting life. Power, at your real tariff rather than a national average. Cooling, usually expressed as a facility efficiency multiplier on power. Facility cost, whether that is rack space, colocation or a share of a building. Support and maintenance contracts. And the personnel cost of operating the thing, which is left out far more often than it should be.
Divide the annual total by the hours the asset is actually available, not by the hours in a year, and you have a rate.
Two cautions. First, be explicit about whether your rate is per available hour or per consumed hour, because the two differ by exactly the amount of the problem you are trying to measure, and quoting one while meaning the other is the most common error in these models. Second, agree the model with finance before you publish a number that depends on it. The same argument applies here as applies to cloud savings: a figure finance did not agree to in advance is not a figure, it is a slide.
The FOCUS gap
The FinOps Open Cost and Usage Specification exists to normalize cost and usage data so that spend can be compared across providers rather than reconciled by hand.
It is worth knowing where it currently stands for this problem. In the 2026 survey, the areas practitioners most want FOCUS to expand into are AI workloads and data center, which is a direct signal that hybrid estates cannot currently be normalized. Adoption is growing but the coverage that would let you put cloud and on-premise into one comparable dataset is not there yet.
Practically, that means anyone doing this today is building the normalization layer themselves. Worth knowing before you scope the project.
Three ways this goes wrong
Comparing capex against a cloud invoice. The single most common error, and it produces whatever answer the person doing the comparison already wanted. Compare capacity first. Apply cost models second, with the model written down.
Treating a high utilization number as the answer. Utilization figures on on-premise clusters are usually honest and usually measuring the wrong thing, because they report allocation rather than work performed. A cluster can report utilization in the nineties with a large share of its accelerators doing nothing.
Assuming your cloud tooling will follow you. It will connect, it will produce charts, and the charts will be wrong for the four reasons above. This is worse than having no tooling, because the output looks authoritative.
A ninety-day sequence
If you are starting from nothing on the on-premise side, the order matters more than the tooling.
First, inventory. What hardware exists, what it cost, when it was bought, and what it is nominally for. This is frequently harder than it sounds and it is frequently where the project stalls.
Second, instrument. Get device-level telemetry flowing for CPU, memory and accelerators. Read-only. No changes to anything yet.
Third, establish the allocated-versus-consumed gap by workload class. Do not act on it. Just publish it, because the number itself usually changes the conversation.
Fourth, agree the cost model with finance and turn the capacity gap into a currency figure they will stand behind.
Only then optimize, and when you do, apply the same safety discipline you would in cloud: changes reviewed by a human, scoped and reversible, with an audit trail. A cost program that causes one incident does not get corrected. It gets canceled.
The underlying point
Kubernetes cost lives in three distinct layers, each with its own waste pattern. The on-premise estate has the same structure and one crucial difference: no layer of it generates a monthly bill.
The mechanism producing the waste is identical in both places. Somebody declares what a workload needs before it runs. The declaration is padded, because under-requesting fails loudly and over-requesting fails silently. And then it is padding that never gets revisited, because nothing prompts a review.
In cloud, the invoice eventually prompts the review. On premise, nothing does. That is the entire difference, and it is why the half of the estate nobody bills for is usually the half with the most waste in it.
About OptOps. OptOps is the optimization layer for enterprise infrastructure, built to work across substrates rather than within one. The same engine reads real workload behavior on Kubernetes in the cloud and on HPC and GPU compute on premise, producing sized recommendations with cost and confidence attached. Read-only by default, so it deploys where write-access tools cannot.
References
- FinOps Foundation. State of FinOps 2026. Sixth annual practitioner survey, 1,192 respondents representing more than US$83 billion in annual cloud spend. Published under Attribution 4.0 International.
- FinOps Foundation. FinOps Open Cost and Usage Specification (FOCUS).
- FinOps Foundation. FinOps for Data Center: Practical Cost Modeling and FOCUS Alignment.
- FinOps Foundation. Mission update, 2026.