What the case showed—and what it did not prove yet

Adria P.A., a small Croatian vehicle dealership, approached EDIH Adria with a familiar service-business problem: sales and service inquiries arrive in waves, while customers still expect answers about availability, prices, or service appointments outside office hours. The company has 10–49 employees and had no practical AI implementation experience before the project.

The team followed a test-before-invest approach. It compared a ready-made chatbot platform with a more flexible solution based on open technologies, then assembled a RAG assistant. Instead of relying only on information encoded in a language model, a RAG system retrieves relevant material from a company-controlled knowledge base before producing an answer. The Adria P.A. knowledge base combined employee interviews, vehicle price lists, sales-process information, and digitised vehicle manuals.

The management lesson is not that “the bot replaced support.” The published case explicitly says implementation was still in progress, the follow-up T1 digital-maturity assessment had not been completed, and quantitative benefits were projections. The test nevertheless exposed three valuable constraints: complex manuals were weak as the only knowledge source, Croatian voice interaction underperformed text, and non-routine cases still required human expertise.

Separate projections from measured outcomes

The case page offers attractive targets: a projected 10–20% increase in customer satisfaction, a possible 20–30% reduction in customer-service labour cost, and a potential 10–15% improvement in lead conversion. It also anticipates that answers to frequent questions could fall from hours or days to seconds. These are not verified production outcomes; they are projections for the future implementation.

That distinction matters. A percentage without a baseline, observation period, inquiry volume, and control group should not be copied into another company's budget. Even an instant answer has negative value if it quotes an expired price, promises the wrong repair date, or directs the customer to the wrong branch.

The case is therefore more useful as a map for testing a hypothesis:

  • routine questions are credible automation candidates;
  • employee knowledge must be collected explicitly instead of relying on a PDF archive;
  • a text channel should be proven before voice;
  • CRM and ERP integration matters, but write privileges should follow quality validation;
  • financial value exists only when released staff time produces a measurable operational or commercial result.

Why uploading manuals was not enough

A long manual is written for a human who already understands the context. It may contain tables, exceptions, references to other sections, configuration variants, and conditions that apply only to a particular market or model year. Splitting the manual into chunks does not guarantee that retrieval will return all related conditions together.

A customer assistant works better when the business exposes managed entities such as:

  • product or service, variant, and applicability;
  • price, currency, effective date, and expiry date;
  • branch, opening hours, available slot, and responsible owner;
  • required documents and restrictions;
  • approved answer wording;
  • data owner and next review date.

Manuals can remain a source for rare reference questions, while prices, stock, booking rules, and commercial terms should preferably live in structured storage or arrive through a controlled API. This reduces the risk of assembling individually true fragments into a false answer and makes it possible to retire stale records quickly.

Employee interviews mattered at Adria P.A. because the real process is usually broader than the formal procedure. A salesperson knows which clarifying question to ask, when a promise cannot be made without a check, and when the conversation must go to a person. That knowledge should become test scenarios and escalation criteria, not remain a free-form interview transcript.

A practical architecture for a small or midsize business

The first system does not need to be an autonomous agent with access to every corporate application. A safer starting point is an assistant that reads approved information, drafts an answer, and hands complex cases to an employee.

A practical flow is:

1. An inquiry arrives from a website, messenger, or contact form.
2. A classifier identifies intent, branch, product, and request sensitivity.
3. Retrieval selects documents and records within the permissions of the user and channel.
4. The model drafts a concise answer with internal citations and a confidence indicator.
5. Deterministic checks block promises absent from the source: final price, confirmed stock, warranty decision, or repair date.
6. Low confidence, conflicting sources, or a non-standard intent routes the conversation to an employee together with the retrieved context.
7. The CRM initially receives a draft or queued event, not an irreversible action executed in the model's name.

If data is sensitive or cannot be sent to an external provider, generation, vector retrieval, and logs can be deployed in a local environment. “Local,” however, does not replace access control. A sales assistant does not need every customer's warranty history, and a public chat does not need internal procurement prices.

Prepare data before choosing a model

Pilot quality is often constrained by the evaluation set, not model size. A company can begin with 100–200 anonymised inquiries from recent months. For each example, record:

  • the customer's intent and expected route;
  • a reference answer approved by the process owner;
  • the mandatory source and its current version;
  • whether an automated answer is permitted;
  • signals that require a human handoff;
  • actual handling time and the number of repeat contacts.

Split this collection into a development set and a closed evaluation set. Measure routing accuracy, retrieved-context coverage, answer grounding, correct refusal rate, and handoff quality rather than a vague notion of “intelligence.” RAG evaluation tools expose measures such as context precision and recall, response relevancy, and faithfulness—the degree to which an answer is supported by retrieved context. Automated evaluation should be complemented by review from an experienced employee.

A four-week shadow pilot

The first pilot can run without speaking to a customer directly. The assistant receives a copy of each real inquiry and proposes an answer; the employee follows the normal process and scores the model's output. The company learns where the system helps and fails without risking an incorrect customer promise.

A compact protocol is:

  • week 1: inventory sources, owners, permissions, and expiry rules;
  • week 2: build the reference set and configure retrieval;
  • week 3: shadow-test new inquiries and review errors and escalation rules;
  • week 4: expose one narrow class of frequent questions in a limited text channel, still with employee approval.

Five groups of indicators are sufficient for a continuation decision: the share of inquiries the system can safely resolve; answer accuracy and grounding; median time to a useful response; the rate and quality of escalations; and cost per handled inquiry. Track employee corrections, complaints, repeat contacts, and the conversion of leads in which the assistant actually participated as separate measures.

Build the economics without promising “30% less”

Use your own operating data. Consider a model scenario with 4,000 monthly inquiries, six minutes of active employee time per inquiry, and a fully loaded hourly cost equivalent to RUB 900. Baseline handling cost is RUB 360,000 per month. If the assistant can safely close 25% of inquiries while review and escalations consume half of the saved time, the net labour-time value is about RUB 45,000 per month.

This is an illustrative model, not an Adria P.A. result. The business case must also include infrastructure, integration, knowledge preparation, quality control, data maintenance, and staff training. Conversely, value may come from faster response, fewer lost leads, and the ability to survive a demand spike without temporary hiring rather than from reducing headcount.

Risks before connecting CRM and ERP

RAG reduces some unsupported answers, but it does not eliminate them. OWASP explicitly notes that RAG and fine-tuning do not fully mitigate prompt injection. A retrieved file or user message can contain an instruction designed to alter model behaviour.

The first release therefore needs straightforward boundaries:

  • sources are reviewed and have named owners;
  • external content is treated as untrusted and separated from system rules;
  • retrieval enforces access rights;
  • deterministic code validates outputs;
  • the model cannot change prices, orders, appointments, or customer records without approval;
  • logs avoid unnecessary personal data;
  • the interface provides a clear human-handoff and refusal path.

Voice adds another error layer: speech recognition, names, numbers, accents, and background noise. Adria P.A.'s experience suggests a sensible order—prove text first, separately measure transcription quality on representative calls, and only then combine the channels.

The manager's next step

Do not begin by selecting the “strongest” model. Choose one narrow stream of frequent inquiries, appoint an owner for every source, and assemble 100–200 real examples. For four weeks, compare assistant drafts with employee decisions in shadow mode. If the system grounds answers in approved sources, escalates complex cases correctly, and produces value at your volumes, open one external text channel. Keep CRM or ERP write access for the next stage.