Why a cheap training run does not mean a cheap project

LoRA and QLoRA have made large-language-model adaptation substantially more accessible. Instead of changing all weights, LoRA trains small low-rank matrices while the base model stays frozen. QLoRA additionally loads the base weights in 4-bit form and trains adapters through the quantised model. In the original QLoRA work, the authors demonstrated fine-tuning a 65-billion-parameter model on a single GPU with 48 GB of memory.

For a small business, this is an important technical shift: experimenting with a 7–9-billion-parameter model no longer necessarily requires an owned cluster. Yet it is easy to draw the wrong financial conclusion and count only GPU hours as the cost of the project.

The training run can be cheap. A working system still requires the team to:

  • define a measurable task and a baseline without fine-tuning;
  • collect, clean, and reconcile examples;
  • separate training data from independent evaluation;
  • run several controlled experiments;
  • integrate the adapter into the inference environment;
  • monitor quality after data or base-model changes.

It is therefore more useful to calculate the cost of one verified improvement to a business process over its entire lifecycle, rather than the “price of LoRA.”

When fine-tuning solves the right problem

Fine-tuning is appropriate when a company needs a stable change in model behaviour: response format, classification, analytical style, domain terminology, or reasoning pattern for a recurring operation. A good candidate has a clear input, a verifiable output, and enough high-quality examples.

Typical tasks include:

  • classifying enquiries under an internal taxonomy;
  • extracting fields from repetitive documents;
  • producing answers in a corporate format;
  • drafting under a stable template and terminology;
  • selecting the next action in a constrained process.

Fine-tuning is not a convenient knowledge base. If policies, prices, products, or instructions change frequently, current facts are better supplied through RAG or a direct query to the system of record. Weights are harder to correct quickly, to purge of one specific record with confidence, and to tie to a cited source.

A practical order of operations is:

1. Build a baseline with a strong prompt and structured output.
2. Add RAG if the problem is missing or changing facts.
3. Measure the remaining error on a fixed set of examples.
4. Fine-tune only when the error concerns stable behaviour rather than knowledge or integration.

This sequence can save more than choosing the least expensive GPU because it may show that training is unnecessary.

The components of a complete budget

1. Task definition and baseline

The team must define success before training starts. For classification, that may mean precision and recall for every important class. For document generation, it may be the proportion of outputs that pass the schema, business rules, and expert review. For support, it may be time to an approved answer and the number of high-cost errors.

Without a baseline, the team cannot know whether fine-tuning paid off. Attractive demonstration examples do not show average quality and do not expose rare but costly failures.

2. Data

In practice, this is often the most expensive component. The team must select real cases, remove duplicates and personal data, reconcile labelling rules, correct contradictions, and preserve example provenance. If two experts judge the same answer differently, the model will not remove the process ambiguity; it will learn it.

The dataset should be split at least into training, tuning, and final evaluation partitions. Near-duplicate variants of one document must not be distributed across these partitions, or data leakage will inflate the metric.

3. Experiments and compute

LoRA reduces the number of trainable parameters, while QLoRA lowers the memory requirement for the base model. This makes multiple runs affordable, but it does not guarantee quality. The team still has to choose the base model, chat template, context length, adapter rank, learning rate, and data mixture.

The best way to economise here is not to eliminate repeats, but to enforce experimental discipline: one hypothesis per run, fixed data and configuration versions, and the same evaluation set.

4. Evaluation and acceptance

An aggregate metric is not enough. Evaluation should be broken down by segments: rare categories, long documents, Russian and mixed-language inputs, missing fields, conflicting instructions, and personal data. Refusals and cases that must be escalated to a human should be tested separately.

For a business workflow, the “share of operations accepted without edits” is often more useful than textual similarity to a reference answer. It has a direct relationship to labour cost.

5. Deployment and maintenance

An adapter depends on a base model, tokenizer, and dialogue template. Those versions need to be pinned. Replacing the base without regression testing is not a safe upgrade. Every release needs an artifact hash, a data card, evaluation results, and a clear rollback path.

A LoRA adapter can be merged into the base model, avoiding separate inference latency. Dynamically loading multiple adapters provides flexibility, but increases the number of combinations that must be tested and supported.

A model calculation for a small company

Consider a pilot for classification and draft preparation on a local instruction-tuned model of roughly eight billion parameters. This is not a market quotation; it is a way to expose the cost structure.

Assumptions:

  • 2,000 selected examples;
  • 48 hours of domain-specialist work on rules, cleaning, and labelling at RUB 2,500 per hour;
  • 32 hours of ML-engineering work on the pipeline, experiments, and deployment at RUB 3,500 per hour;
  • 24 hours of independent result review at RUB 2,500 per hour;
  • 20 GPU rental hours at an illustrative RUB 250 per hour;
  • 12 hours of engineering support during the first month at RUB 3,500 per hour.

The result is:

  • data and rules — RUB 120,000;
  • engineering — RUB 112,000;
  • independent evaluation — RUB 60,000;
  • GPU — RUB 5,000;
  • first month of observation — RUB 42,000;
  • total — RUB 339,000.

Compute accounts for roughly 1.5% of this illustrative budget. If GPU rental doubles in price, the total grows by only RUB 5,000. If the team underestimates data cleaning by twenty hours, the project grows by RUB 50,000. The main financial risk is not where the graphics cards glow.

Calculating payback without self-deception

Fine-tuning should not be justified by the number of generated tokens. It needs an operational effect:

`monthly value = operation volume × minutes saved × loaded hourly labour cost / 60 − monthly maintenance`.

Continue the illustrative example. Suppose the system handles 3,000 cases each month, fine-tuning saves an average of 1.5 review minutes for every accepted result, and the loaded labour cost is RUB 1,200 per hour. Gross savings are RUB 90,000 per month. With RUB 25,000 in maintenance, the net effect is RUB 65,000, giving a simple payback period of about 5.2 months on the RUB 339,000 pilot.

The calculation is valid only if the 1.5 minutes are measured and quality does not deteriorate. If an employee still rechecks every draft in full, there may be no saving. If automation increases costly errors, the effect becomes negative.

Before launch, record:

  • operation time before the pilot;
  • the share of results accepted without edits;
  • the cost of false acceptance and false rejection;
  • the volume of exceptions routed to people;
  • recurring monitoring and retraining costs.

Architecture that preserves the economics

A minimum production environment includes:

  • a versioned store of examples and labelling rules;
  • a separate, protected evaluation set;
  • a reproducible adapter-training script;
  • a registry for the base model, tokenizer, template, and adapter;
  • automatic format and business-rule checks;
  • shadow or limited release to a fraction of traffic;
  • an exception queue and human approval;
  • a log of model version, output, and employee decision.

This setup helps distinguish model degradation from a change in input data or business rules. It also shows whether the share of results accepted without edits is truly increasing.

Risks that cannot be excluded from the calculation

A company should not train on data for which its rights are unclear, or on secrets it is not prepared to preserve inside an artifact. Personal-data minimisation, access control, and a retention policy for datasets and adapters are required before training.

Another risk is dependency on the base model. A new release may be better on average and worse on the company’s format. An upgrade is therefore a new candidate, not a minor technical change. It must pass the same protected test and limited rollout.

Finally, a narrow adapter can improve the target operation while degrading general answers. It should not be mixed into a general-purpose assistant without routing and evaluation.

A concrete next step

Before buying a GPU, choose one recurring operation and collect 150–300 real examples. Remove duplicates, create a protected test set, and measure the base model with a prompt and RAG. Then classify residual errors into three groups: knowledge, behaviour, and integration.

If knowledge errors dominate, improve retrieval and sources. If integration errors dominate, fix schemas and business rules. If a stable behavioural gap remains, estimate a LoRA pilot using five budget lines: data, engineering, compute, evaluation, and maintenance.

The management conclusion is straightforward: QLoRA makes training technically accessible, but only a measurable reduction in manual work or errors makes it economically justified. Vnutrik can start the GPU cheaply; the company still pays for an experiment that solves the wrong problem.