More AI, Lower Costs: How the Equation Changes

Thursday, 27 August 2026 at 06:00

Meer AI gebruiken en toch de kosten drukken zo verandert de rekensom

Companies can scale their use of artificial intelligence dramatically without letting costs rise at the same pace. The winning play is no longer finding a single cheap model, but combining smart model routing, tighter context, specialized models, and better utilization of pricey AI chips. Prosus shows how big the impact can be: the tech group says it cut AI costs by about 90 percent while token usage grew 52x in a year.

That example hits a crucial question for organizations moving from pilots to large-scale AI. A simple employee chatbot is manageable. Hundreds or thousands of continuously running AI agents—processing documents, helping customers, running analyses, and completing tasks autonomously—change the economics entirely. AI can be expensive, but often doesn’t have to be.

More AI doesn’t have to mean proportionally higher bills

AI costs spike when systems run nonstop. Every request burns tokens and compute, and long-running agents tend to carry ever-growing context.

At small volumes, inefficiencies hide. A few extra cents per request barely matter at a thousand calls. At millions or billions, the same inefficiency becomes a line item.

So a second optimization challenge now rivals model quality: how much compute is truly required to do the job well?

Prosus offers an extreme scale case. CEO Fabricio Bloisi and Ioannis Zempekakis write that the company now processes 8.2 trillion tokens per month. Token usage, they say, grew 52x last year. Across its ecosystem, AI agents work alongside roughly 40,000 employees and five million partners, serving services that collectively reach about one billion customers.

Despite that surge, Prosus says it cut costs by around 90 percent—not by using less AI, but by changing how its AI capacity is deployed.

The priciest model shouldn’t answer every query

According to Prosus, model routing drives the biggest savings. Software decides per task which model has enough capability to get it done.

The idea is simple. A complex analysis, unfamiliar problem, or multi-step plan may need a powerful reasoning model. Summarizing a standard document, extracting an invoice number, or rewriting a short text probably doesn’t.

Yet many AI systems still send all tasks to the same heavyweight model. Easier to build, but ruinous at scale.

Prosus estimates about 50 percent of its total savings come from routing workloads across models. The company evaluates options like Luna, DeepSeek, and Kimi on its own workloads.

Price per million tokens alone won’t reveal the cheapest choice. A model can be five times cheaper per token but need more tokens—or multiple attempts—to finish the job.

A better metric: cost per successfully completed task.

That also changes procurement and comparisons. Benchmarks and price sheets are a starting point, but in-house evaluation is ultimately more valuable. Organizations need to know which model delivers the best cost-performance ratio on their own documents, processes, and quality bars.

Long context windows can quietly drain your budget

A second saving lever: how much information a model receives with each request.

Modern models can ingest huge amounts of text. That tempts teams to resend full chat histories, documents, instructions, and prior outputs every time.

For agents, this adds up fast. An agent active for hours or days accumulates more and more context.

Prosus examined a production workload where the average request carried roughly 115,000 context tokens. By trimming stale or irrelevant data, making file transfers more efficient, and fetching information only when needed, it cut that average to about 60,000 tokens.

The same infrastructure could then handle nearly twice the traffic with similar response times, Prosus says.

The lesson goes beyond Prosus. A model’s maximum context length is a capability, not a target. A system that supports one million tokens shouldn’t process hundreds of thousands on every call.

In other words: smart context management is cost management.

AI memory needs to get more selective

That challenge will only grow as companies deploy more AI agents. Traditional chatbots handle short exchanges; agents run longer tasks and stitch together data from multiple systems.

Developers therefore have to decide what an agent should truly remember.

Instead of keeping everything in active context, data can be stored externally and fetched only when relevant. Other details can be summarized—or deleted entirely—once a subtask is done.

This creates a split between available memory and active context. An agent can retain access to vast company data without forcing the model to reprocess it on every single request.

The upside isn’t just lower costs. Trimming irrelevant context also reduces the risk of the model being distracted by information that no longer matters to the task at hand.

Smaller AI models move to the front line

A third path to lower costs is specialization. A large general model can do thousands of things, but many business processes repeat the same steps millions of times.

For that, a smaller specialized model can be far more economical.

One technique is distillation: using the knowledge or behavior of a powerful model to train a smaller one for a specific goal. The large model effectively acts as a teacher.

Prosus says an initial distillation step for a specific task cut inference costs by roughly a factor of ten. With further optimization, that advantage rose to about forty times for the same task.

The company also used quantization, which lowers numerical precision to reduce a model’s memory and compute needs.

Prosus credits about 20 percent of its total savings to specialized models.

It points to a likely blueprint for large-scale enterprise AI: not one model to rule them all, but a network of models with distinct jobs. A powerful model handles edge cases and complexity, while smaller models chew through the predictable volume.

Idle GPUs still burn money

Companies running open models on their own infrastructure face another issue: unused capacity.

With a third-party model via API, you mostly pay per use. On your own stack, GPUs must be available before demand hits.

AI traffic also spikes hard. Prosus describes a workload averaging roughly one request per second, with peaks nearly six times higher.

If you size solely for the peak, you’ll sit on idle GPUs for much of the day.

Not every AI job needs to run instantly. Reports, evaluations, data processing, and scheduled agent tasks can be shifted to windows when human-driven demand is low.

The same GPU can power interactive apps by day and background jobs at night.

Prosus estimates about 10 percent of its total savings come from better GPU utilization and load balancing.

The cheapest AI stack is likely multi-layered

The Prosus example shows the AI cost debate is shifting. Companies don’t have to chase the cheapest model—they need to stop using expensive capacity where cheaper options deliver the same outcome.

Prosus breaks down its estimated savings as follows:

50 percent by routing tasks to the right model;
20 percent by cutting context per request;
20 percent by using specialized models;
10 percent by improving GPU utilization.

The exact mix will vary. Companies using mostly external APIs have less control over GPU usage. Firms with millions of repetitive transactions can gain more from specialized models.

The core principle is the same: optimize not just AI price, but the total compute needed to finish a business process successfully.

Cheaper AI usually means a lot more AI

Lower unit costs don’t necessarily shrink AI budgets. They make AI viable in far more places.

An agent used sporadically might add limited value. Make it cheap enough to monitor processes continuously, and the usage model flips.

AI can then check for changing conditions, process documents on arrival, flag anomalies, and handle routine work—without a human prompt.

It echoes cloud and storage trends: cheaper units rarely cut total consumption. They unlock use cases that weren’t economical before.

For the AI market, that could be the next phase. The first race was about who had the most powerful model. Now that multiple strong models are available, the surrounding infrastructure matters more.

Companies that precisely match the “intelligence” each task needs can build a lasting edge.

The standout number from Prosus might not be the 90 percent cost reduction, but that it came alongside 52 times higher token usage. The next era of enterprise AI may not be about using fewer tokens—but about using far more AI, while driving down the cost of every task.