Why VMware Private AI Cloud signals a new era for the IT infrastructure layer

Reading Time: 9 minutes

For many organizations, the AI experimentation phase is over; bringing AI into production use is well underway. An overwhelming 88% of organizations report regular AI use in at least one business function, according to McKinsey’s “State of AI in 2025” survey.

As AI becomes integrated into enterprise workflows, private cloud for AI architecture is gaining traction as a more secure, governed, and more cost-efficient foundation to support AI workloads.

If you doubt the increasing role of private cloud for AI, consider that a staggering 83% of enterprises are considering repatriating workloads from public to private clouds, and 50% have already done so, according to Broadcom.

“As organizations are moving from pilot deployments … to more production deployments of AI done at scale, they are facing major challenges around cost and security when it comes to data privacy, resiliency, availability of the infrastructure,” said Prashanth Shenoy, vice president of marketing for Broadcom’s VMware Cloud Foundation Division, in “VMware Intros Private AI Cloud, AI Factory As Workloads Shift to On-Prem.”

Eighty-three percent of enterprises are considering repatriating workloads from public to private clouds.

From private cloud to VMware Private AI Cloud

Building private cloud architecture for AI was also the key theme at VMware Explore, taking place Aug. 31 to Sept. 3, 2026, at the Venetian Convention and Expo Center in Las Vegas. The virtualization platform has long been an enduring technology for private cloud infrastructure.

Building on its VMware Cloud Foundation 9 (VCF 9) that was introduced two years ago, Broadcom announced its VMware Private AI Cloud and – at its foundation – the VMware AI Factory model-as-a-service offering. VMware Private AI Cloud aims to give organizations a production-ready path to securely build, run, and govern inference workloads, agentic applications, and traditional enterprise workloads together on a single private cloud platform.

Broadcom’s VMware AI Factory, its software-defined foundation of VMware Private AI Cloud, provides a simplified path to AI in production, with new automation innovations for deploying AI-ready infrastructure. With VMware AI Factory, customers can achieve faster time to first model deployment and better manage AI tokenomics (a discipline that ensures the financial cost of AI is justified by real business outcomes).

The recent announcements reflect the company’s efforts to build all the foundational elements of a private cloud environment for AI. The company “has the ecosystem that you need for the next generation of AI ” said David Linthicum, founder and lead researcher at Linthicum Research, noted that during an interview with theCUBE. “Whether it’s governance whether it’s security, whether it’s operations. What it’s going to take to run AI on-premises, which is really going to be the destination for [AI] given how much it costs to run in the cloud.”

Industry watchers believe Broadcom has the ecosystem that organizations need for the next generation of AI.

Reducing costs with private cloud infrastructure

One of the first key vectors that VMware’s new private cloud addresses is skyrocketing infrastructure costs. Gartner estimates that that dynamic random access memory (DRAM) pricing will increase. Some VMware Cloud Foundation. And VMware Private AI Cloud offers the opportunity to reduce memory and storage costs, which have loomed large over the past couple of years given hardware shortages.

Broadcom executives noted that the hardware shortage won’t resolve anytime soon, and the VMware platform is designed as a workaround.  “There is a significant change in how hardware costs are manifesting themselves,” Velaga said. This is not a temporary phenomenon. You are going to see that hardware is going to be a scarce resource,” Velaga said.

According to Broadcom, organizations can experience a 51 percent cost reduction over the course of three years with VMware Private AI Cloud. Reduced TCO stems from more virtual machines (or VMs) per host reduced power and cooling, reduced staffing, and more.

Velaga noted that hardware costs are driving organizations to assess whether their infrastructure gives them the greatest bang for their buck. And according to “Advanced Memory Tiering Now Available with VMware cloud Foundation 9.0,” memory tiering strategies can reduce memory TCO by some 40%– significant cost relief during the persistent hardware shortage. Similarly, vSAN global deduplication can deliver up to 34% lower storage TCO, compared with other storage offerings.

VMware Private AI Cloud addresses skyrocketing infrastructure costs.

Data security

Organizations are also increasingly focused on AI security. As organizations adopt AI and feed their models proprietary and sensitive data, organizations have mounting concern about keeping data secure, governed, and private. “They would rather bring the compute to the data they have on-prem rather than taking the data to a compute that is off-prem,” said Ram Velaga, president of the Infrastructure Software Group at Broadcom.

According to Private Cloud Outlook 2026, 51 percent of organizations are moving workloads back to private clouds given their concerns about data security. AI-enabled security threats now move at machine speed. Indeed, breakout time— how long it takes for an adversary to start moving laterally across an organization’s network — reached an all-time low in in 2026, according to CrowdStrike: The average fell to 29 minutes, and the fastest breakout time observed dropped to a mere 27 seconds

“You also need to be a little concerned about AI,” said Paul Turner, chief product officer, Broadcom during the keynote. “The risks of AI and autonomous attacks into your supply chain, into your environment, into your infrastructure. As part of the Private AI Cloud, you must also deliver a secure infrastructure. Every attack stage is now automated at machine speed.”

VMware Private AI Cloud protects against AI-accelerated threats by minimizing the attack surface and enabling continuous compliance. Automated, non-disruptive updates keep systems current.

“It’s time to rethink security,” Turner continued. “Using software to stop software—it’s the only way forward. Because of agents, because of applications, because of frontier AI and the risk and threat from those, we have to move security to be built in. It has to become part of the application perimeter.”

VMware Private AI Cloud protects against AI-accelerated threats by minimizing the attack surface and enabling continuous compliance.

AI sovereignty

AI costs and security concerns have prompted organizations to consider private clouds to protect data captured in AI models from leaking data to public models—which threatens data security and organizations’ competitive advantage. VMware Private AI Cloud is designed to enable AI sovereignty to give organizations greater control and governance over their data. “Organizations see now these use cases emerge with these frontier models and the clouds. But the counterbalance to that is, ‘Hey wait a minute. I’m concerned about data leakage,” said Chris Wolf, Broadcom’s global head of AI and Advanced Services, in a CUBE interview.

“‘I’m concerned about somebody else leveraging an AI model’s expertise to disrupt my business. … You have sovereignty considerations. It doesn’t mean don’t use frontier models. It means be practical. Use frontier models where they make sense. Use specialized models where they make sense. And now IT operations caught in the middle of all this.”

What VMware Private AI Cloud means for the infrastructure layer and AI

VMware’s announcement comes at a time when organizations are clamoring for help to solve these persistent infrastructure challenges. In his keynote, Velaga noted that organizations are increasingly turning to the data center infrastructure layer as the way to address the challenges of rising costs, data security, AI sovereignty, and lack of visibility into cost per outcomes (that is, tokenomics).

“How actually do you get your infrastructure to be able to deal with all of these issues? Scaling, on-prem, being able to provide sovereignty, dealing with hardware costs, and also having this one platform that brings all of this together,” Velaga noted. “This is exactly what VMware has been trying to solve—which is abstracting away your application and your workload requirements from the underlying hardware.”

So one of the central questions that has arisen from VMware Explore supersedes the conference itself: Could VMware–and the infrastructure layer itself–be the part of the stack that enables the next generation of AI?

“[VMware has] all the piece parts,” Linthicum noted in the CUBE interview. “They have security, they have governance, they have agentic platforms, they have generative AI development platforms, they are able to provide model production and model-tuning capabilities—everything you need to drive AI into the next generation that allow people to leverage this technology to be a true force multiplier but at a fraction of the cost.”

Could VMware–and the infrastructure layer itself—be the part of the stack that enables the next generation of AI?

Treat VMware Private AI Cloud as a strategic evaluation, not a default

VMware Private AI Cloud arrives at the right moment for many organizations. AI is moving out of experimentation and into production, and that shift is exposing hard questions about infrastructure cost, data control, security, governance, and operational readiness.

But the announcements should not prompt every VMware customer to immediately build a private AI cloud. The smarter move is to treat VMware Private AI Cloud and other announcements as a strategic evaluation point: which AI workloads belong in a private cloud environment, which can stay in public cloud or software-as-a-service (SaaS) platforms, and what foundation is required to run them securely and economically?

For organizations with significant VMware investments, sensitive data, regulatory requirements, or production AI use cases that need tighter control, VMware Private AI Cloud may be a strong fit. It gives IT teams a path to bring AI closer to enterprise data, apply familiar operational models, and manage AI workloads alongside traditional applications. That could be especially valuable where data residency, latency, compliance, or cost predictability matter.

But private AI cloud is not automatically the right answer. If an organization is still defining AI use cases, relying mainly on SaaS-based AI tools, or experimenting with small-scale pilots, a major platform commitment may be premature. In those cases, customers should first validate the business case, workload requirements, model strategy, and cost profile before investing in dedicated private AI infrastructure.

A private AI cloud is not automatically the right answer.

SHI’s view is that customers should ask three questions before moving forward.

1. Is private AI the right fit for the workload? Not every AI workload needs to run on private infrastructure. Organizations should assess where the data lives, how sensitive it is, what governance controls are required, how much latency matters, and whether the workload depends on frontier models, specialized models, or a mix of both.
2. Is the VMware estate ready? VMware Private AI Cloud should be considered alongside broader VMware planning, including VMware Cloud Foundation maturity, licensing and renewal strategy, operational skills, infrastructure modernization, GPU capacity, storage, networking, and power and cooling requirements. For some customers, the first step may not be deploying private AI cloud. It may be optimizing their existing VMware environment so it can support AI later.
3. Can the organization prove value before scaling? AI infrastructure decisions should not be made on technical capability alone. Customers need a clear view of AI economics, including utilization, cost per outcome, model performance, governance overhead, and the business value generated by each workload. Lower infrastructure cost or lower token consumption only matters if the AI workload delivers measurable value.

SHI works with customers to assess where AI workloads should run, compare private cloud with public cloud and hybrid alternatives, evaluate VMware readiness, and build phased roadmaps that align infrastructure investments with business outcomes. The goal is not simply to adopt VMware Private AI Cloud because it exists. The goal is to determine whether it is the right foundation for the customer’s AI strategy — and if so, how to implement it in a way that is secure, governed, cost-aware, and built for production.

NEXT STEPS:

Learn more about SHI and VMware.

Learn more about migrating to VMware Cloud Foundation.

Get the SHI VMware Strategic Assessment.