Scaling AI Without Breaking the Budget

Summary

Four AI leaders spent a DataCamp webinar picking apart how their organizations overspent on AI, and what changed once the bill arrived.

Earlier this year, companies pushed employees to use AI for everything, some even ranking staff by token consumption. Then leaders saw the invoices, and the goal shifted from using AI constantly to using it where it pays off. In this session, host Richie Cotton asked four guests to trace exactly where that money went: Rob Ferguson, VP of technology and strategy at inference platform Fireworks; Meryem Arik, CEO of UK inference lab Doubleword; Nirmal Jingar, a senior engineering leader in supply chain AI; and Daria Fedechkina, CFO at LexisNexis France.

Their stories ranged from agents that burned twelve hours generating nonsense to a bank chatbot doing customers' physics homework on the side. But the more useful part of the conversation was how each of them now controls AI costs: separating an AI budget into experiment, production and scale stages; matching model size to task difficulty instead of defaulting to the biggest model available; and treating cost per token as almost meaningless next to cost per business outcome. The panel also covered training, since several of the costliest mistakes traced back to employees who never learned how the pricing model worked. What follows is a practical guide to reducing AI costs and planning an AI budget that survives contact with real usage, drawn from people who watch this problem happen at scale every day.

Key Takeaways

• Long-running AI agents can burn many hours of pure inference generating unusable output if they go off track early and nobody catches it.
• Consumption-based AI pricing flips the old cost problem: instead of paying for a subscription nobody uses, companies now pay for heavy use that creates no value.
• Hidden AI costs, such as data preparation, employee training and human review of outputs, often exceed the cost of the model itself.
• Annual AI budgets set in January are close to guesswork, because model prices and token volumes both shift on a monthly basis.
• Production AI costs cannot be estimated by scaling up a prototype's costs. Evaluation, observability, retrieval infrastructure and GPU costs all compound once an agent goes live.
• Return on AI investment is more useful measured against business outcomes than against tokens spent, since a cheap workflow nobody uses is a worse deal than an expensive one that works.
• Routing tasks through an intelligence ladder, cheapest capable model first and escalating only when needed, is one of the most reliable ways to cut inference cost.
• Working directly with an inference provider, rather than through a generic model router, can raise cache hit rates enough to cut costs by roughly four times.
• Training and cost awareness need to be ongoing, not a single onboarding session, because employees keep discovering new and expensive ways to misuse AI tools.

Deep Dives

When AI Spending Goes Wrong

Every guest had a version of the same story: AI spending goes wrong quietly, then all at once. Arik, whose company provides inference for long-running agents, described watching costs spiral on tasks that run eight to twelve hours unsupervised. "It's very easy for cost to go out of hand in these kinds of workloads," she said, "because what we've noticed has happened is if the agent's gonna go off track, it can do so really early in the process and then generate twelve hours of nonsense and garbage." The company only rearchitects its agent setup after the expensive failure has already happened once.

Ferguson split his examples into two categories: AI for productivity and AI for production. On the productivity side, he described chief AI officers who tracked employee AI usage on leaderboards, rewarding whoever burned the most tokens, even when that meant "summarizing the same document over and over." On the production side, his favorite disaster involved a major bank's public support chatbot. High schoolers discovered it would happily solve their physics homework, effectively handing out free access to a frontier model through a customer service window with no guardrails.

Fedechkina described a subtler, two-sided version of the same waste. Some teams buy licenses that barely get used, so the company pays for access nobody touches. Others adopt a tool and burn through it immediately: one of her team members exhausted a month's token allowance in two days simply by running everything through the most powerful reasoning mode in a single, ever-growing chat thread. "Without adoption, you waste licenses," she said. "Without literacy, you waste tokens." Jingar's version was structural rather than individual: companies negotiate a cheaper rate with a frontier model provider, then discover their own agents are making 10 to 50 times more calls than necessary, wiping out any savings from the discount.

The Hidden Costs Nobody Budgets For

Ask a CFO where AI money actually goes, and the answer is rarely the invoice. Fedechkina argued the real spend sits in three layers most budgets never capture: getting data and infrastructure ready, since "AI doesn't work in isolation" and needs clean, structured, access-controlled data before a model ever sees it; the people and process cost of training staff and building human review into workflows, because "you cannot delegate your responsibility to AI"; and, most expensive of all, applying AI to a process that never needed fixing. "If you are to make chaos, eventually, you will get automated chaos," she said, warning that a faster analysis nobody acts on creates technical efficiency without economic value.

Arik's view of hidden cost comes from the inference side. A small number of frontier models, not the full catalog of options, drive most of a company's bill, which she said mirrors the instinct many teams have to reach for the flagship model, whether that means Fable or Opus, for every task. She estimated a smaller model like DeepSeek Flash can cost a fraction of a frontier model's price while remaining highly capable for many workloads. She also pointed to a less obvious lever: most inference runs in real time, which is expensive to provision because it has to handle unpredictable spikes. Workloads that can tolerate a delay, like document processing or long-running agents, can move to batch endpoints that run roughly 70% cheaper.

Ferguson named a different hidden cost entirely: rework. As organizations shift from occasional chat use to constant agentic work, the ratio of tokens processed per useful output has exploded. "Agentic task burn is thousands of times more tokens than chat," he said, describing teams where multiple agents independently review the same repository, doing the same work over and over without anyone realizing it.

AI Budget Planning for a Cost That Won't Sit Still

Every panelist agreed on one thing: a fixed annual AI budget set months in advance is close to fiction. Arik put it bluntly. In January, she could not have predicted the number of tokens her customers would use by August, and any dollar figure she attached to the cost of those tokens would have been wrong in the other direction, since prices for a given level of intelligence keep falling. Her customers now avoid committing to a specific model more than a month out, uncertain whether it will still be relevant. She cited Uber's public budget overrun as the predictable result of trying to lock in a number for a market moving that fast.

Jingar's answer is to stop budgeting like it is one number at all. He recommends an AI economic budget split into experiment, production and scale, because a $500 prototype that looks impressive in a demo creates entirely different economics once 50,000 people use it. "Never extrapolate production economics from prototype economics," he said, pointing out that evaluation, observability, retrieval infrastructure, GPUs and data pipelines all compound once agents move from a proof of concept into production.

Fedechkina takes a similar view from the finance side, arguing that a strategy slide listing giving staff AI tools, transforming an existing process, and building AI into the product all look the same on a slide, but hide three completely different cost structures underneath. She separates spending into run, transform and reinvent categories, each with its own funding logic, and funds new initiatives in stages: a small budget for learning, released only once there is clear evidence something works, with full funding reserved for the scale phase. She also budgets a reserve for usage growth, since successful adoption almost always consumes more than a pilot predicted, "not because we did something wrong."

Ferguson went further, arguing that budget itself may be the wrong frame for 2026. Rigid governance that requires evaluating every use case before approving it risks holding back the teams already doing valuable work. His advice is to route between multiple providers and build a real understanding of the cost and performance mix rather than committing blindly to one vendor at a fixed rate per seat.

Choosing the Right AI Model for the Job

Once budgets are set, the next lever is picking the right AI model for each task, and the panel's advice converged quickly: build evaluations, don't guess. Arik acknowledged that most teams skip this step. Her rough substitute is the Artificial Analysis leaderboard, comparing a candidate model's intelligence score against known models like Opus or Fable, and checking its release date, since "it won't have been released this month unless it was a good model." Her broader point was about discipline, not tools: "Most people are not data driven," she said. "It's very vibe space... a surprising number of people are making very big decisions off of very little data."

Jingar's method is what he calls an intelligence ladder: start with the cheapest thing capable of reliably solving the problem, whether that is a simple rule, an API call, or a small model, and escalate to a frontier model only when necessary. "Frontier intelligence should be an escalation," he said, "not a default setting." He used password resets as an example of a task that needs no language model at all, arguing that some of the most expensive AI mistakes come from applying the biggest available tool to the smallest available problem.

Ferguson described the gap between what he calls AI-dependent and AI-native organizations. AI-dependent teams wait for the next model release and hope it works better and cheaper. AI-native teams instead build small portfolios of three or four specialized models, learn how to route between them, and continuously rebase their workflows onto new models as they appear. His core metric for whether AI training investment is working is simple: do the agents finish their tasks reliably? An agent that fails constantly wastes money regardless of which model sits underneath it.

Arik added a technical lever specific to inference providers: speculative decoding, where a small model accelerates a larger one without changing its output. One customer saw a threefold speed increase and a 50% cost cut from the technique, but only because they worked directly with their provider rather than through a generic router. She also warned that routers optimized purely for the cheapest per-token price can wreck cache hit rates without anyone realizing it: the difference between a cached and uncached price can run 80 to 90%, turning an apparently cheap choice into an expensive one.

Building a Team That Uses AI Well

Every technical fix in this conversation eventually traced back to a training gap. Fedechkina's approach starts before any tool is introduced: employees need to understand what task they are trying to automate and what result they want, before they ever pick a tool for it. From there, her organization builds practical cost awareness (a growing, single chat thread can eat through a month's budget without anyone noticing) and pairs formal training with a network of AI champions who share what works on a weekly basis. "One training is not enough," she said. "It's just a lifelong learning capability."

Ferguson's focus was cultural rather than procedural: what he calls multiplayer AI. Teams that hide how they use AI, rather than sharing prompts and outputs openly, create what he described as a "phantom second culture," where mistakes spread unchecked and colleagues start to distrust each other's work without saying so. Strict, inflexible budgets make this worse, since visibility into someone's AI usage can read as evidence they are cutting corners rather than working efficiently. His fix starts with leadership modeling open conversation about when and how AI is genuinely useful.

Jingar's closing advice reframed the entire session: stop optimizing for cheapness and start optimizing for economic intelligence. Every workflow, in his framing, carries an intelligence budget, a latency budget and a dollar budget, and the goal is a system that can judge which one matters most for a given task. Simple problems deserve cheap intelligence; high-value decisions justify a frontier model and deeper reasoning. "The future isn't just about one model answering everything," he said. "It's a system that knows when intelligence is worth paying for."

Arik's final point doubled as a hiring tip for anyone listening: ask a prospective employer what your token budget will be before you join. The answer, she said, reveals how seriously that company has actually thought about the problem this entire panel spent an hour dissecting.