Why the price of one model call is misleading

A proposal often shows a price per million input and output tokens. But a business is not buying generated text; it is buying a resolved task: a correctly classified request, a prepared response, a checked document, or a CRM record filled in. Between a user request and that result, an agent may call the model several times, use tools, retry a failed step, and hand part of the work to an employee.

Comparing systems by the cost of one call is like evaluating delivery by the cost of one turn of the steering wheel. The useful unit is the cost of a completed task at an acceptable quality level.

Define success before you count

The unit of measurement should map to a business process, not a technical event. For example, “request processed” might mean that the required fields were extracted, a CRM record was created, validations passed, and any exception was routed to the responsible employee.

For a pilot, record:
- the number of processing attempts;
- the number of results accepted against predefined quality criteria;
- error, retry, and human-handoff rates;
- employee time spent on routine checks and corrections;
- spending on models, tools, infrastructure, and support.

The denominator is the number of accepted successful tasks. If an agent made 1,000 attempts but only 920 results passed review, do not divide the costs by 1,000.

What belongs in the full cost

Add the costs for each use case across several buckets.

  • **Model.** Count every call in the chain, including classification, planning, data extraction, validation, and retries. Input, output, cached, and reasoning tokens may have different prices. Inspect the actual usage fields in API responses and reconcile totals with the provider dashboard.
  • **Tools and data.** OCR, speech recognition, search, vector storage, external APIs, and paid connectors remain part of the bill even when the agent invoked them.
  • **Human work.** Include initial review, exception handling, correction of bad outputs, and process support. Measure minutes, not only the number of “escalations”: a short correction across dozens of records still adds up to hours.
  • **Operations.** Allocate the scenario’s share of servers or cloud, logs, monitoring, redundancy, prompt and integration updates. For local inference, include accelerator depreciation, electricity, spare capacity, and administrator time.
  • **Development and quality control.** Include regular sample reviews, test datasets, model-change checks, and engineering time. One-time work can be allocated across the expected life of the solution, but show that calculation separately.

A management formula is:

**Cost per successful task = (model + tools + human work + operations + allocated implementation) / accepted tasks.**

Show p50 and p95 cost as well. An average hides rare but expensive loops in which an agent repeats actions or waits on an external service.

A modeled example: processing an inbound request

Consider a pilot processing 1,000 requests a month. Every figure below is an illustrative assumption used to demonstrate the method—not a specific provider tariff or a measured company result.

Assume five model calls per attempt on average. Each call uses 3,000 input tokens and 500 output tokens. That is 15,000 input tokens and 2,500 output tokens per attempt. For arithmetic only, assume a price of $2 per million input tokens and $10 per million output tokens:
- model: 0.015 × $2 + 0.0025 × $10 = $0.055 per attempt;
- at an assumed exchange rate of 90 rubles per dollar, about RUB 4.95.

Now add the process. Suppose 10% of requests require four minutes of review, while 8% fail acceptance criteria and need six minutes of manual handling. At an assumed fully loaded employee cost of RUB 900 per hour:
- routine review: 100 × 4/60 × 900 = RUB 6,000;
- exception handling: 80 × 6/60 × 900 = RUB 7,200;
- model for 1,000 attempts: about RUB 4,950;
- assumed monthly integration, logging, and four hours of support: RUB 8,500.

If 92% of attempts are accepted, the total is approximately (6,000 + 7,200 + 4,950 + 8,500) / 920 = **RUB 29 per accepted task**. Under these assumptions, manually processing every request at six minutes each would cost RUB 90 per request. This is not a savings forecast: quality, employee time, and all costs must be measured on your own data before making a rollout decision. Paid OCR or overnight support, for example, is not included and would change the result.

Collecting data in a pilot

Start with a small flow that can be checked by a person. Give every task an end-to-end identifier and record its related events: model calls, token usage by category, tool calls, retries, latency, employee decisions, and the final status in the business system. Do not put more customer data in logs than is needed for diagnosis; set retention periods and access controls.

With a hosted API, usage fields may be returned with each call, while aggregate spending appears in the provider dashboard. A local server needs its own metrics. For example, vLLM exposes metrics through a Prometheus /metrics endpoint, which can show request and token load, latency, and queue behavior. An observability system such as Langfuse can collect token and cost data, but for a self-hosted model you must define the unit price and accounting rules yourself. Monitoring helps collect evidence; it does not define the business success criterion for you.

Reconcile three levels:
1. **Provider bill:** what was actually charged over the period.
2. **Task trace:** which stages and retries caused the spending.
3. **Process outcome:** whether the result was accepted and how many employee minutes it required.

If the totals do not match, investigate duplicate logging, hidden calls, caching, or double-counting a parent and child operation.

How a local model changes the calculation

Local inference does not make generation free. It changes the cost structure: instead of a per-token tariff, you have GPU purchase or rental, idle capacity, electricity, maintenance, and reserve capacity. If the model and hardware serve several processes, costs can be allocated across them—but you cannot set the entire server cost to zero and then claim the local option is free.

Compare options on the same test set and at comparable quality:
- accepted tasks per hour on the available configuration;
- p95 latency and retry rate under realistic load;
- the number of people needed to operate it;
- behavior when a GPU or network fails;
- whether data may be sent to an external API.

Local deployment is more compelling when data control, predictable internal infrastructure, or substantial steady demand matters. An external API may be cheaper for an irregular pilot when the company does not need to buy and maintain its own compute. The answer depends on workload and requirements, not on the slogan “local is cheaper.”

Risks that distort the result

  • **Counting calls instead of completed tasks.** An agent can look cheap per call and expensive across a long tail of failed attempts.
  • **Looking only at the average.** p95 and maximum cost expose the cycles that consume the budget.
  • **Ignoring quality review.** If more tasks need correction, a low token price does not offset the manual work.
  • **Forgetting context growth.** Re-sending conversation history, retrieved documents, and instructions increases input use. Caching can change the price, but its effect should be confirmed against provider data.
  • **Comparing a pilot with production under different conditions.** A pilot may have little concurrency, no traffic peaks, and no redundancy.
  • **Mixing implementation cost with variable cost.** Separate monthly operating spend from the cost of launching the solution, then show payback at the expected volume.

A practical one-week plan

1. Choose one recurring process with a clear input and verifiable output.
2. Record success criteria and the manual cost of the current process before the model demo.
3. On 100–300 real, anonymized, or otherwise authorized examples, measure calls, retries, tools, and human minutes.
4. Calculate cost per accepted task, acceptance rate, p50 and p95 cost, and employee time saved.
5. Compare the result with the manual process and set a threshold at which automation stops being worthwhile.
6. Only then decide whether you need local deployment, a different tariff, a simpler workflow, or ordinary automation instead of an agent.

The main point is simple: budget for the result the business accepts, not for tokens or message count. When you can connect cost to each completed task and explain retries and human minutes, a pilot becomes a testable financial decision.