Start with the task, then choose a model

An AI pilot often starts with a model demo, even though value depends on the workflow. Choose an operation that occurs often enough and has a clear input and measurable output: classify requests, extract document fields, or draft a response for an employee.

Describe the current process: who receives the task, which sources they open, how much time it takes, where errors occur, and who approves the result. This is the baseline. Without it, you cannot tell whether AI made work faster or added another review step.

Prepare a small but useful sample

Collect real examples, removing unnecessary personal and commercially sensitive information. For each, record the correct answer, acceptable variants, and reasons to escalate the task to a person. Include routine cases and exceptions: empty fields, poor scans, ambiguous wording, and missing sources.

Check that documents are current, have an owner, and are visible to pilot users. If the workflow needs a CRM, file store, or service desk, restrict test access first and do not write results automatically into production data.

Define acceptance criteria before launch

Choose several measures tied to the business task:

  • share of results accepted without correction;
  • employee time per task, including review;
  • errors weighted by their consequences;
  • share of cases correctly escalated to a specialist;
  • cost per accepted result and service availability.

Set the minimum acceptance threshold and prohibited errors in advance. For document extraction, evaluate field accuracy; for search, check whether the needed passage appears among the top results. Do not reduce everything to one average score: a rare but expensive error may matter more than a few seconds saved.

Shadow mode reduces risk

At first, the model produces a recommendation or draft while people continue the normal process. Compare suggestions with employee decisions, record corrections, and retain reasons for disagreements. Only after stable results should you permit a safe automatic action with confirmation or rollback.

Assign a pilot owner who makes decisions about data and metrics. IT owns access and logging, the process owner defines quality, and employee users review real cases. This is a small team, but each person needs a clear responsibility.

How long to run and when to stop

The first cycle does not require months of development. A limited-sample pilot may take one or two weeks if the process and data are available. Stop or redesign it if the model routinely invents evidence, misses the minimum quality threshold, requires too much review, or loses its economic benefit after support costs are counted.

OECD reviews emphasize skills, data access, and risk management as important for SMEs, but international research is not a ready-made forecast for every company or for Russia. Use it as a checklist of questions, not a promise of results.

Pre-purchase checklist

  • There is one process owner and one pilot owner.
  • Typical and difficult examples have been collected and anonymized.
  • Baseline time, error rate, and manual cost are known.
  • Data sources, access rights, and retention period are documented.
  • Acceptance criteria, prohibited errors, and escalation rules are defined.
  • Shadow mode can compare outputs without writing to production systems.

With these items in place, model and vendor comparisons become concrete: the required quality and cost are visible. If they are missing, prepare the process and data first. That is usually cheaper than buying a system before deciding which work it should improve.