Sovereign AI: Take Control of Your AI Stack

Summary

Sovereign AI cannot be separated from energy sovereignty, according to five infrastructure leaders who joined a panel on building an AI stack that stays under an organization's control.

Steven Armstrong moderated the conversation between Chris Wolf, Lawrence Snapp, Simon Irish, and Suman Debnath, who argued that sovereignty means more than running a model inside a national border. It requires owning the control plane and the data plane: encryption keys, identity systems, and policy enforcement, with the ability to air-gap the whole stack. The panel turned next to the physical constraints behind that ambition. Panelists cited small modular nuclear reactors, grid curtailment risk, and the wattage gap between a legacy server rack and a GPU rack as evidence that energy, not chips, has become the real bottleneck. They also described techniques for getting more out of existing infrastructure: virtualization that cut one manufacturer's compute footprint in half, and AI gateways that route simple prompts to local open-source models while reserving expensive frontier models for harder reasoning. The panel closed on regulation. Lawrence Snapp, fresh from a week at the UN's AI for Good summit in Geneva, argued that regulators could accelerate AI adoption rather than slow it down.

Key Takeaways

• True sovereignty means owning the control plane, encryption keys, identity, and policy enforcement, not just running a model inside a particular region.
• Small modular nuclear reactors are becoming central to AI infrastructure plans because few other technologies can deliver gigawatt-scale, reliable power from a compact footprint.
• One large manufacturer in Korea cut its total compute capacity in half after applying virtualization and workload orchestration to its GPU and HPC clusters.
• Organizations are using AI gateways to route simple prompts to local open-source models and save costly cloud or closed models for harder reasoning, offloading up to 90% of token volume in some cases.
• Efficiency gains in AI infrastructure increase overall energy demand instead of reducing it: cheaper, faster service pulls in more users, which panelists said cancels out the savings.
• Regulated industries such as banking and insurance are adopting AI faster than less-regulated ones because they already have compliance frameworks and expertise to build on.
• During power shortages, utility regulators prioritize households over data centers, which is pushing AI infrastructure operators to build their own energy resilience instead of depending on the grid.

Deep Dives

What Is Sovereign AI, Really?

Sovereign AI gets defined too loosely, the panel argued from the start. Chris Wolf, who leads AI infrastructure strategy at Broadcom's Glasswing, said recent industry disruptions have made the case for the idea impossible to ignore. "I cannot take dependencies on external entities that could be just, disabled via kill switch at any time," he said, treating that risk as the clearest argument yet for building AI systems that don't depend on any single outside provider staying available.

Wolf drew a sharper line than most vendors do. An organization's data sitting inside its own region or cloud account doesn't make it sovereign, he said. Sovereignty requires control over the systems that manage that data: "there's lots of claims around Sovereign AI where it's, essentially an organization has control of the data plane. And that doesn't mean anything if you're, if you don't have full control of your encryption keys." The same goes for identity and policy enforcement, both routinely outsourced to external providers even in deployments marketed as sovereign.

His conclusion was blunt: "when you think about a design point for sovereign AI, if you don't own the control plane, then it isn't truly a sovereign solution." In practice, an organization should be able to disconnect an edge site or data center from the outside world entirely and keep running, including model updates and software lifecycle management. Wolf pushed back on vendors who propose keeping the control plane cloud-hosted for convenience, since that reintroduces the dependency sovereignty is supposed to remove: the cloud connection drops, and so does the organization's ability to operate.

Lawrence Snapp, fresh from the UN's AI for Good conference in Geneva, tied sovereignty to trust and flagged model drift as a complication for both. Reinforcement tuning can shift a model's behavior in ways engineers don't fully understand, he said. A system working well today can behave differently tomorrow, which undercuts the guarantees sovereign infrastructure is supposed to provide.

Why AI Data Centers Need So Much Energy

Simon Irish, whose company builds small modular nuclear reactors, argued that AI has pulled the tech industry into territory it has never occupied before. "Energy is the primary means of production," he said. "So it's almost as if the tech industry has joined the ranks of the traditional industries of steelmaking and car manufacturing." Without power, there's no output, regardless of how advanced the model is.

Irish framed the challenge around two questions: where the power is needed, and when. On location, gigawatt-scale AI campuses now demand more electricity than aging grids in most advanced economies can deliver to a single site, he said. He argued small modular reactors are the only technology that can produce that much reliable power from roughly 25 acres of land. On timing, AI workloads need power on demand, at any hour, without interruption, which rules out intermittent renewable sources that "do not tick the firm box," in his words. Nuclear, he said, "produces... come rain, come wind, come whatever."

One data point concerned him: annual power demand growth in the U.S. has jumped from a historical 0.5% to somewhere between 2.5% and 3%. He called that a warning sign for utilities, regulators, and enterprises still planning power needs on old assumptions. Data center operators, he added, get pushed to the front of the curtailment line when supply tightens, since utility regulators answer to the households who vote them into office, not the companies running server racks.

Suman Debnath, who works on infrastructure at Crusoe, backed this up with numbers from his own career. A decade ago, a full rack of storage, networking, and compute drew 10 to 15 kilowatts, he said. A modern rack built around an accelerator like Nvidia's GB200 draws roughly 120 kilowatts or more. "Time to power is the new time to market," he said, describing his company's approach as going "from electron to token": source the power first, then build compute and cloud services on top of it.

Getting More From the Hardware Already in Place

New power plants aren't the only answer. Wolf pointed to virtualization as an underused lever for AI inference workloads: pooling and scheduling GPU resources intelligently cuts waste without much hit to performance. He cited one of the largest manufacturers in Korea, which applied virtualization to its AI and HPC clusters and "wound up reducing their overall compute capacity by 50%," with knock-on drops in networking, server nodes, and software licensing.

Wolf also described a tiered routing system his own team built inside Broadcom: an AI gateway that inspects each prompt and decides where it runs. A simple task, like translating a document, goes to a local open-source model at no token cost. A request that needs deep reasoning can still go to the cloud, but only when it has to. "In some use cases, you might be able to offload 90% of your tokens to local open source models that you're running," he said, which cuts spend while keeping cloud capacity free for harder problems.

Lawrence Snapp raised a different kind of waste: how much spend goes nowhere useful at all. "Right now, a lot of people are basically taking a private jet to the corner store to get a gallon of milk," he said, describing the gap between the compute a task needs and what organizations throw at it by default. A meaningful share of tokens spent today produce outputs that are simply wrong, he said: "we're spending a hundred tokens when 20 of them are hallucinations." He called this "consumption efficiency" one of the biggest open opportunities in AI, ahead of supply.

Simon Irish complicated the optimism. Efficiency gains, he said, tend to increase demand rather than reduce it. Services get cheaper and faster, more people use them, and total energy consumption keeps climbing even as per-task efficiency improves. Lawrence Snapp agreed, predicting that removing adoption friction will make demand grow "exponentially."

Open Source vs. Proprietary AI Models: What Enterprises Are Choosing

Suman Debnath said the math around open-weight models has shifted over the past two years, especially since DeepSeek's release. Enterprises that once defaulted to closed, API-based frontier models for a quick proof of concept now route 80 to 90% of workloads to open-source or fine-tuned models hosted on their own infrastructure, he said, driven largely by cost. For regulated sectors like healthcare, finance, and government, there's often no choice: data has to stay within a defined territory, whether that's a VPC, a private cloud, or a sovereign cloud.

He shared an example that surprised his own team: the first serious request for a fine-tuned, self-hosted model at his company came from legal, not engineering. The legal team wanted a model trained on contracts and legal documents that would never leave the organization's control. That pattern is showing up across enterprises, he said, as non-technical teams start asking how to apply AI to their own specialized work.

Wolf cared less about which model to pick than how committed a vendor is to open standards. He pointed to Ethernet-based networking as the safer choice for AI back-ends, since it preserves supplier choice compared with proprietary fabrics, and praised projects like VLLM and PyTorch for giving organizations a common entry point across GPU vendors. Both AMD and Nvidia have upstreamed drivers into VLLM. Llama.cpp does the same work for CPU-based inference.

He flagged a risk that gets less attention than it deserves: how fast silicon vendors patch vulnerabilities once they surface. "There's some silicon suppliers that are really one and done," he said. "Here you go. It worked. We certified it in our factory. It's your problem now." Exploit chains increasingly run from software down into silicon, he said, which makes patch velocity, not just raw performance, a core criterion for picking a hardware partner. Enterprises need to close the gap between nine-to-eighteen-month upgrade cycles and the near-daily patch pace hyperscalers already run.

Regulators as an Unlikely Accelerant

Lawrence Snapp's closing point came straight from a week at the UN's AI for Good summit in Geneva, where he said sovereignty dominated conversations involving heads of state and religious leaders alike. His takeaway ran against instinct: "I think regulators might be the heroes of AI," he said, arguing that adoption friction, not overregulation, is the bigger risk to the global economy right now.

Regulated industries such as banking and insurance have adopted AI faster than less-regulated sectors, he said, because they already have compliance frameworks and specialists to lean on. Groups like the UN's ITU are building global standards, alongside independent efforts from regulatory experts, because organizations are asking for a path to trustworthy deployment rather than resisting oversight. "If we can remove AI adoption friction, we will unleash AI's value," he said, adding that doing so might mean current valuations aren't a bubble at all, but an undervaluation.

Snapp urged organizations to act before any governance framework is perfect: "start with some framework. Who cares if it's right? I guarantee it's not wrong to start with something." Chris Wolf closed with career advice: engineers should resist becoming generic "AI experts" and instead use AI to deepen the domain expertise they already have.

Simon Irish gave the panel its final line: this AI wave differs from prior tech cycles because "AI sovereignty is energy sovereignty," a link he said can't be separated from AI policy going forward.