From Access to Architecture: Five Operating Standards for the Efficiency Era of Enterprise AI

In recent months, some of the most sophisticated technology companies in the world got a surprising bill. Amazon quietly removed its internal token-usage leaderboard. Microsoft cancelled a set of AI coding subscriptions. Across the market, there have been repeated reports of unexpectedly large token invoices arriving at firms that considered themselves ahead of the curve. Citadel Securities, tracking the aggregate price and mix of large-language-model usage, now sees the beginnings of a bifurcation between frontier and everyday AI — a market quietly rationing its most expensive capability toward the work that actually justifies it.¹

The lesson is not that AI is too costly. It is that AI adoption outran AI architecture. For the past eighteen to twenty-four months, the enterprise imperative was speed: get the tools into people’s hands, drive usage, prove the technology works. That mission largely succeeded — and it left no time for standards, no time for best practices, no time to design how the work should actually be done. The next eighteen to twenty-four months will reward a different discipline. The advantage will not go to the firm that uses AI most, but to the firm that has standardized how.

The pressure is real, and it will not ease

It would be convenient to conclude that the answer is simply to slow down and spend less. It is not. The demand that drove the adoption era has not softened; it has intensified. In the most recent Work Trend Index, 53% of leaders say productivity must increase, while 80% of the global workforce — employees and leaders alike — report they lack the time or energy to do their work.² The telemetry of the modern workday is telling: interruptions arrive on average every two minutes, edits to a single presentation spike 122% in the ten minutes before a meeting and roughly 60% of meetings are now called ad hoc.² This is the environment into which AI was dropped, without any operating standards of its own.

And pressure does not merely coexist with undisciplined usage — it manufactures it. When every hour is already spoken for, people reach for whatever produces output now: the most powerful model, the whole document pasted in, a fresh prompt written from scratch. No one has the slack to build the reusable version, so the same context gets rebuilt session after session and the waste compounds. Speed without design is not a temporary cost of adoption; left alone, it becomes the operating model.

So the posture that matters here is the disciplined middle. Now is not the time to turn away from the technology, nor is this the time to break the glass and go all in. The firms that retreat will forfeit the capacity gains they cannot get from headcount. The firms that spend without design will keep paying for undisciplined usage — and calling it progress.

Why firms are leaving value on the table

The waste is not random. It follows three structural drivers in how these systems price and behave — and each has a standard that answers it.

Cost is input-dominated, and more context is not safer. In most enterprise workflows, the tokens sent into the model — pasted documents, re-explained context, entire uploaded corpora — dominate the bill. Worse, oversized context degrades quality rather than protecting it. In controlled testing, model accuracy fell roughly 25 points — from answering the large majority of questions correctly to missing nearly a third — as the input grew from a short brief to a modest stack of documents.³ The intuition that a bigger context window is a safer context window is precisely backwards.

Work is redone and resent — the retry tax. Cost does not only enter at the start of a task; it accumulates as the task runs. Every step in an agentic workflow resends the full history, and stable material that never changes — system prompts, reference files, prior turns — is re-billed at full price on every call. The compounding is severe: in one analysis of agentic work, the review-and-refinement stage alone consumed 59.4% of all tokens, an automated quality layer that quietly became the dominant cost center.⁴ Much of the enterprise AI bill is not new work. It is the same work, sent again.

Capability is over-provisioned. Left to habit, users reach for the most powerful available model regardless of task — the equivalent of routing every routine request to a PhD-level specialist, or a senior executive, when a capable generalist would do the job as well. Pointing a frontier reasoning model at summarizing a set of meeting notes is not a rounding error when it is the default across thousands of daily requests. By one estimate, a smaller model is the optimal choice for roughly half of all prompts — which means frontier-tier spend on that half is pure over-provisioning.⁵

These are not teething problems that scale away on their own. A useful maturity model runs from Hierarchical to AI-Augmented to AI-Native to Outcome-Native — and the AI-Augmented stage, where everyone has access but no one has standards, is empirically the least efficient of all.⁶ Costs rise not only because more people use AI, but because early usage is poorly engineered. That is independent corroboration of the core point: the cure for the cost of adoption is not less AI, but more discipline.

Context capital: the asset most firms are not yet building

Here is the reframe that changes the economics. Model consumption is not a fixed cost per user. It is an operating-model variable the enterprise can engineer. Repeated instructions, preferred formats, institutional knowledge and task procedures can be converted into reusable assets that require upfront investment but lower the marginal AI cost of every future task.

We call this context capital — the skills, templates, taxonomies, retrieval structures and routing rules that accumulate as genuine intellectual property, alongside data and talent. Its logic is a flywheel: upfront workflow engineering leads to higher reuse and first-pass acceptance, which lowers consumption per outcome, which improves gross margin. Two firms running the same models on the same processes can incur materially different costs, because one rebuilds context every session while the other has banked it.

Standard operating procedures are how context capital gets built. The following five are where we would start.

Standard 1 — Architect the project before the first prompt. Most work begins with a blank chat and a document dump. It should begin with a structure: what lives in persistent project knowledge, what is retrieved per task and what is never pasted at all. Retrieval, not re-pasting, becomes the default. The payoff is real and double-sided — selective, on-demand loading of context has been shown to cut context by roughly 85% and raise accuracy, and hybrid retrieval approaches deliver 39% to 65% cost reduction at quality comparable to loading everything.⁷⁸ Less context, better answers.

Standard 2 — Route the model to the task, not the task to the model. A documented routing rule — by task type, owned by whoever sets AI policy — captures the asymmetry that makes over-provisioning expensive: output tokens carry roughly a five-to-eight-times premium over input tokens across the major providers, so sending simple work to lighter models saves far more than its share of volume implies.⁹ Learned routing has demonstrated better than 85% cost reduction while retaining 95% of frontier-model quality, and capping reasoning budgets on tasks that do not need deliberation cut reasoning tokens by an average of 67% at under 3% accuracy cost.¹⁰¹¹

Standard 3 — Cache what repeats. Stable instructions, tool schemas and reference material get resent from scratch, session after session, when they could be cached once and reused at a fraction of the price. Prompt caching can reduce cost up to 90% and latency up to 85% on long, stable prefixes, with cache reads priced at roughly a tenth of standard input.¹² But the discipline is not “turn caching on.” Caching only pays off with reuse: a short-lived cached prefix costs slightly more to write and a longer-lived one costs more still, so the standard is an owned inventory of what is stable enough to cache — and a schedule for refreshing it before it goes stale.

Standard 4 — Prune the session, not just the setup. Even a well-architected session accumulates dead weight as it runs: exploratory turns, superseded drafts, resolved tangents. This is a distinct lever from how the session starts, and most firms have not yet named it. Periodically stripping stale context mid-session has been shown to cut total compute cost by 21% to 36%, independent of what entered at the outset.¹³ The research proves pruning works; it does not hand you the trigger. A workable default: after roughly ten exchanges, or when a thread has outlived the task that started it, prompt the user to checkpoint the essentials and start clean — so the tool enforces the discipline rather than relying on the user to remember it.

Standard 5 — Measure accepted output, not tokens saved. Without a shared metric, cost and quality quietly work against each other. A team can look efficient by producing cheap output that then demands heavy human correction — or, in the pattern that gives this era its signature waste, generating an AI output, dropping it into an email and having the recipient run it through a second AI to summarize it back down. The right denominator is the accepted task, not the token. The path is staged. First, instrument: tag every call by user, project and task type, and advance only when you can attribute more than 80% of spend to specific workflows — a configuration decision, not an engineering project. Then optimize: target 40% to 70% cost reduction with no measured quality drop, watching that router misclassification does not push retries — and cost per accepted task — back up.¹⁴ The roll-up metric, inference yield — accepted business outputs divided by AI consumption cost — keeps the whole system honest: a cheap answer that has to be redone has poor inference yield, however low its token count. One discipline protects that metric: keep two ledgers, machine cost and human time, and never let a token saving be reported as a productivity gain. They have different owners and must be proven separately.

At the executive level, these operating metrics translate into the outcomes a function actually manages — a faster close and tighter forecast accuracy in finance, shorter resolution times in service, quicker turnaround in legal. Track that translation, but do not claim the standards caused it without isolating the effect; the honest sequence is attribution first, business impact second.

How to move forward

The efficiency era is not a cost-cutting exercise dressed up as strategy. It is the point at which AI usage stops being an expense line and starts becoming an asset the firm owns. The standards above are the mechanism; context capital is the asset; margin is the result.

For leaders deciding where to begin, five moves:

• Instrument first. Tag spend by workflow and measure cost per accepted task before optimizing anything. You cannot manage what you have not attributed.
• Set a routing policy. Make model selection a documented rule, not a matter of individual habit.
• Bank the caching and retrieval wins. These are the best-evidenced, lowest-risk levers available — capture them early.
• Name an owner for context capital. Skills, templates and routing rules are assets; assets that no one maintains decay.
• Report inference yield. One shared metric, tracked at the team level, keeps efficiency and quality on the same side.

The adoption era proved AI works. The efficiency era will prove which firms can make it pay. The gap between the two is not access to the technology — nearly everyone now has that. It is the discipline to build the architecture underneath it. That window is open now, and it will not stay open for those who mistake usage for advantage.

Notes and sources