Compare equal outcomes

Comparing a GPU price with an API tariff is almost always incomplete. Buying a server shifts some costs to capital expenditure, but does not eliminate power, cooling, depreciation, updates, support, or downtime. API cost depends on volume, context, model, and retries.

Compare the full cost per accepted result at the same quality and similar data and availability requirements. Use the real workload, not peak performance from a specification sheet.

An illustrative model

Assume a server costs RUB 600,000 and is depreciated over 36 months: RUB 16,667 per month. At average node power of 1.5 kW, 720 operating hours, and an assumed electricity rate of RUB 8 per kWh, electricity is about RUB 8,640, before a detailed cooling estimate.

Add an assumed RUB 16,000 per month for administration, maintenance, and answer review, plus RUB 4,893 for networking, space, and overhead. The total is about RUB 46,200. This is an illustration, not a market quote: replace each number with actual rates and operating schedules. A failover node, downtime, licenses, and cost of capital can change the result.

Divide monthly costs by answers that pass acceptance criteria. If the system produces 3,000 drafts but 2,100 are accepted, use 2,100 as the denominator.

What to measure before buying

For one or two weeks, collect task volume and hourly load, input and answer lengths, retry share, percentage accepted without correction, latency, availability, and staff maintenance time. For a GPU, track utilization, memory, power, and errors. Monitoring shows actual load but does not replace depreciation and labor calculations.

When a local deployment fits

A local model may make sense with steady workloads, restrictions on sending data externally, or predictable-latency needs—provided the stack meets quality requirements. With infrequent requests, equipment sits idle and cost per task rises. Staff are also needed for updates, monitoring, and recovery.

An API may be more practical for seasonal demand and rapid experimentation. Calculate its cost from actual input and output tokens, model choice, tools, retries, and retention limits. Include redundancy and quality control for either option.

A practical decision sequence

Choose one internal workflow and an acceptance criterion. Compare API and local inference on the same anonymized sample; measure quality, latency, and review time, then enter actual prices and demand into the full TCO model.

Buying equipment before a pilot is justified only when data or infrastructure constraints already rule out other options. Otherwise, measure monthly volume and cost per accepted result first. DCGM helps monitor GPUs, but the purchasing decision must include the whole process.