When an AI budget grows faster than expected, the first response is often procurement: find a cheaper model, negotiate a lower rate, or impose a smaller token limit. Those moves can help, but they begin too late in the story.
An agent does not buy tokens in isolation. It performs a job. It reads context, calls tools, waits for services, retries failures, reformats results, asks another model to check the first one, and sometimes creates work for a human to repair. The model invoice is only one view of that system.
The useful economic question is not, “How much did this model call cost?” It is, “How much did it cost to complete an acceptable unit of work?”
Measure the whole job
Choose a unit people already understand: one ticket resolved, one account reviewed, one report produced, one product record updated, or one research brief approved.
For that unit, measure:
- model and tool costs;
- elapsed time;
- number of attempts and retries;
- failure and timeout rates;
- human review time;
- rework caused by incomplete or incorrect output;
- repeated work already performed on similar requests.
This wider view prevents false savings. A smaller model that costs half as much but doubles the review burden is not cheaper. A cached answer that ignores a changed account state is not efficient. A fast route that causes a user-visible mistake transfers cost rather than removing it.
The Punk cost optimization guide recommends measuring frequency, observed cost and latency, verifiability, safety, and freshness together. That sequence is more useful than ranking providers by list price because it identifies the actual operating pattern.
Repetition is often the first economic signal
Many agents repeatedly solve work that is similar enough to learn from but not identical enough for a simple response cache. A support workflow may classify the same families of requests; a finance assistant may produce the same summary structure from changing numbers. Repetition matters economically because it concentrates spend in a pattern the team can measure.
Do not assume the whole answer should be reused. Different patterns call for different interventions: a declared or instrumented read-only tool result might be cached briefly, a deterministic transformation might stop requiring model judgment, and a qualified plan might perform current reads without paying to rediscover its sequence. How to Reuse Agent Work Without Serving Stale Data covers that plan-reuse decision in depth.
For tools visible to Punk, the tool observability and caching documentation makes the safety boundary explicit: pure computation and declared read-only tools can be candidates for caching with a defined lifetime, while writes are not cached. Identity and subject context also matter. The same lookup result may be appropriate for one account and entirely wrong for another.
Match the control to the waste
“Optimize the agent” is too vague to be useful. Diagnose the waste and apply the narrowest control.
If identical safe reads repeat
Use a short-lived tool-result cache with clear identity, permission, and freshness boundaries. The question is not only how long the data remains useful. It is also who is allowed to receive it.
If the plan repeats but the data changes
Treat repeated planning as its own candidate and keep current records current. Do not count the opportunity as savings until the alternative route has been qualified.
If a transformation is stable and checkable
Consider a deterministic path. A fixed calculation, field mapping, or rules-based classification should not require open-ended reasoning forever if the organization can define and test the expected behavior.
If quality varies by task
Route by the job. Some work may be suitable for a smaller model with a clear fallback. Ambiguous or high-impact work may continue to require the stronger live path.
If failures cause repeated spending
Fix reliability first. Timeouts, duplicate actions, and unbounded retries can erase gains from model selection. Add limits, idempotency where appropriate, and visible failure ownership.
Demand proof in the same economic unit
An estimated saving is not the same as a realized one. Before changing a production route, compare the candidate with the current path on representative work.
Useful questions include:
- Did the result meet the same acceptance criteria?
- Did it fetch information that needed to be current?
- Did it preserve authorization and policy boundaries?
- Did it avoid unintended actions?
- Did it reduce total cost per acceptable result?
- Did latency improve without shifting work to a reviewer?
Compare before and after in the same unit. If the baseline is cost per accepted account review, the result cannot be defended with a lower cost per model call alone. Include the calls that moved elsewhere, the tool fees that remained, the reviewer time that changed, and the failures that required another attempt.
The same discipline applies to projected savings. Separate observed cost from modeled opportunity. State the eligible share of traffic, the acceptance rate, and the costs left outside the estimate. Punk's safe model routing guide treats an apparently cheaper route as a candidate that still needs evidence, policy, and fallback—not as realized savings.
A modest role for Punk
For supported model traffic, Punk sits in the request path so teams can observe cost, latency, routes, and repeated work together. Tool cost and action context enter that view when tools are declared or instrumented. For eligible patterns visible to the runtime, Punk can help test alternatives and route narrowly after the required evidence and policy conditions are met.
It cannot make an inherently rare, ambiguous, or poorly measured task economical. It also should not force reuse where freshness, risk, or open-ended judgment makes the live model the honest choice.
The product fit is strongest when work repeats, the output can be checked, and the organization wants an exit path when a candidate is not eligible.
Build a one-page agent income statement
For one agent and one recurring job, create a simple weekly view:
- jobs attempted and jobs accepted;
- total model and tool cost;
- retries and failures;
- human review and rework time;
- the three most common repeated patterns;
- one proposed change, with a measurable acceptance condition.
Do not start with a portfolio-wide model migration. Start with one costly, repeated job. If the workflow is wasteful, make it less wasteful. If the model is genuinely the largest remaining lever, you will then have the evidence to change it for the right reason.
Questions readers ask
Should we switch to a cheaper model first?
Usually not. Establish a workload-level baseline first so you know whether model selection, repeated work, retries, tool usage, or operational cleanup is driving the cost.
Can every repeated request be cached?
No. Safe reuse depends on identity, authorization, freshness, policy, side effects, and whether the result can be checked.