Before you ask what AI costs, ask where it should run

Reading Time: 7 minutes

AI’s infrastructure footprint is becoming harder to ignore. According to the International Energy Agency, data centers consumed approximately 485 terawatt-hours of electricity globally in 2025, an increase of 17% in a single year, with demand expected to roughly double by 2030. That does not make AI infrastructure irresponsible by default but it does mean organizations need a better way to decide where AI should run.

Most organizations are not developing new data center campuses themselves. They are consuming cloud services, reserving accelerated compute, placing owned hardware in colocation facilities, modernizing existing environments, or using AI already embedded in applications.

But choosing cloud or colocation does not make the footprint disappear. Organizations still consume power. The grid connection is still in a real place. Cooling still depends on local conditions. What changes is the lever: organizations decide what capacity to buy to support processing, where workloads should run, and what evidence it expects from providers.

Infrastructure leaders need to ask whether each workload is running in the right place, for the right reason, with the right level of accountability.

How should organizations decide where AI workloads run?

Workload placement sounds like an infrastructure decision. In reality, it brings together cost, control, risk, latency, data residency and security, resilience, and resource use. Get it right and the rest of the AI operating model becomes easier to manage. Get it wrong and the organization must rely on governance controls to manage risks created by infrastructure that was not designed for the workload.

Some use cases belong in public clouds. Others are better suited to private cloud architecture, hybrid environments, edge deployment, colocation, dedicated accelerated compute, or sovereign architectures. When IT managers factor in proper workload placement one environment isn’t always the most “responsible.” Instead, the strategy should be to match each workload to the environment that suits it and to be able to explain the trade offs—whether they involve cost, performance, or data security.

What are the main AI infrastructure deployment models?

Most enterprise AI estates are mixed, often using public clouds, private clouds, or hosted data center services. A single organization may use several of these models at once, sometimes by design and sometimes because different teams moved at different speeds.

Model	What it means	Who typically uses it
Public cloud AI	Consuming AI capabilities from cloud platforms or AI features embedded in existing software	The majority of organizations, particularly for experimentation and general productivity
Dedicated AI cloud	Reserving or buying access to large-scale accelerated compute from specialist providers	Organizations with sustained, high-volume AI workloads
On-premises infrastructure	Buying and operating AI hardware in an owned or managed environment	Large enterprises with predictable production workloads and suitable facilities
Colocated architecture	Placing owned hardware in a third-party facility that provides space, power, cooling, and connectivity	Organizations wanting control without constructing a facility
Private cloud and Sovereign  AI  architecture	Controlled environments with defined residency, access, and governance	Government, defense, healthcare, financial services, and other regulated sectors

Data sovereignty is best treated as a requirement, not a destination. It may rule out some deployment options or add obligations around residency, access, and control, but it does not remove the need to decide which infrastructure model best fits the workload.

What can infrastructure leaders control when deploying AI?

Organizations that buy data center capacity have less influence over facility design than those that build or operate their own environments. But they still influence demand, consumption, supplier behavior, and reporting discipline.

The levers available to them include region selection, utilization, model right-sizing, workload scheduling, capacity commitments, supplier due diligence, and contract terms. Individually, none of these factors changes what a hyperscaler builds tomorrow. Collectively, consistent buyer expectations change what providers disclose, design, and prioritize.

What decisions shape AI’s infrastructure footprint?

These decisions apply whether an organization owns infrastructure, buys cloud services, uses colocation, or combines all three. The levers change by model, but the discipline is the same: make the trade-offs visible before scale makes them expensive.

1. How can organizations improve AI resource utilization and capacity planning?

Address resource waste. Idle accelerators still draw power. Unused reserved capacity still signals demand. Overbuying is both a financial problem and a footprint problem, which makes utilization one of the clearest responsible AI infrastructure metrics. Capacity planning tools, and real-time observability monitoring, help organizations rebalance workloads and gain continuous insight into GPU, CPU, memory, storage, and network usage, helping teams distinguish between temporary underutilization and chronic waste.

2. How should organizations choose where AI workloads run?

Training, fine-tuning, and inference have different cost, performance, and data security/ control profiles. Bursty experimentation may fit public cloud. Predictable, high-volume inference may justify dedicated capacity, private infrastructure, or colocation. Sensitive workloads may need sovereign or private AI. The responsible answer depends on the workload, not the label attached to the platform.

Placement also has a local impact. A workload running in one region may draw from a cleaner, less constrained grid than the same workload running somewhere else. For many teams, region selection is still treated as a default setting. It should be treated as an architectural choice.

3. How do power, cooling, and local resources affect AI infrastructure?

International Energy Agency on AI growth. While hardware supply shortages persist, chips can be ordered faster than utilities can approve new interconnections, expand transmission capacity, or bring new generation online. That is why responsible AI infrastructure has to look beyond the rack and consider what the surrounding grid can support.

Data center cooling is more nuanced than most public debate allows. Different designs shift the balance between energy use and water use, and the right approach depends on climate, rack density, workload pattern, and local water source. Blanket claims about data center water use usually miss those trade-offs.

4. What should organizations ask AI infrastructure suppliers?

For organizations buying capacity, supplier due diligence is the most practical control point. Procurement should ask the questions communities ask of developers: where the capacity sits, how power and cooling are managed, what data is reported, and what happens during local grid or water stress.

• Location-specific energy and water data, including peak and annual figures
• Cooling design, operating assumptions, and efficiency methodology
• Drought, grid-stress, or demand-response plans
• Emissions reporting boundaries and backup generation approach
• Hardware refresh, reuse, recycling, and disposal pathways

Real answers state methods, boundaries, and peaks as well as averages. Averages can hide the moments that matter most to communities and utilities.

5. How can organizations manage AI cost, consumption, and business value?

AI cost does not sit neatly in one bill. It accumulates across infrastructure, models routing choices, data, applications, software, labor, governance, security, and change. Counting pilots, licenses, or tokens consumed shows activity. It does not prove value.

A better model connects inputs, efficiency, outputs, and outcomes. Infrastructure leaders need to know what capacity was committed, how efficiently it was used, what work it produced, and whether that work delivered a business result worth the cost and resource consumption.

That is the bridge between “prove the value of AI” and “prove the footprint.” If the business outcome is weak, the infrastructure case is weak too.

Why responsible AI infrastructure is a business decision

The public debate about AI data centers often focuses on developers, utilities, and local officials. That debate will continue, and it should. But enterprise buyers also shape demand. They decide which workloads to scale, which capacity to reserve, which regions to use, which providers to trust, and which claims they are prepared to stand behind.

Buying capacity instead of building facilities does not transfer responsibility. It changes the questions: region selection instead of site selection, utilization instead of facility design, procurement evidence instead of utility negotiation, and workload value instead of AI activity.

None of this asks organizations to choose between innovation and responsibility. It asks them to know where their AI runs, what it costs, what it consumes, who owns the outcome, and whether the result is worth the footprint.

How can SHI helps organizations build responsible AI infrastructure?

SHI helps organizations decide where AI should run, what it will cost, and how to scale it responsibly. That work can span readiness assessment, use case prioritization, validation in SHI’s AI and Cyber Labs, workload placement, infrastructure readiness, sovereign and private AI design, governance, and AI cost and value management.

The answer will not be the same for every workload. Sometimes public cloud architecture is right. Sometimes dedicated capacity, colocation, private infrastructure, or a sovereign architecture makes more sense. The value is in reaching a decision the organization can justify to its board, its regulators, and the communities that supply the power.

 

NEXT STEPS: Speak to an SHI expert

Check out our responsible AI series:
Responsible AI adoption: 5 questions every board, CEO, and CIO should ask
Why AI literacy is becoming a board-level AI governance issue
Speed to power: Why behind-the-meter power generation provides competitive edge in the AI era