Count useful answers, not conversations
A chatbot can answer a thousand questions and barely reduce a team’s workload. Some topics are not covered by the corpus, some require a precise source, and some answers will be corrected by a specialist. RAG economics should track answers accepted by employees and time actually released.
RAG retrieves passages from selected documents and generates an answer grounded in that context. It can fit recurring questions about procedures, policies, or catalogues that change more often than model retraining makes sense. But the system does not automatically become a source of truth: results depend on document quality, retrieval, and source presentation.
An illustrative estimate for 1,000 inquiries
Assume 600 of 1,000 inquiries concern topics covered by the knowledge base. If 80% of those cases produce answers that need no specialist correction, that is 480 useful answers. At six minutes per manual response, the gross estimate is 2,880 minutes, or 48 hours.
This is a ceiling before review, configuration, operations, and answers an employee still needs to read. It does not imply automatic headcount reduction: value may come from a shorter queue or more time for complex work.
Useful answers = inquiries × share of topics covered × share accepted without correction. Gross time = useful answers × baseline manual response time. Subtract review time and calculate full costs for documents, retrieval, models, infrastructure, and support.
Evaluate before expanding the corpus
Start with 50–100 anonymized questions: routine, rare, and difficult. Keep a correct answer or supporting document for each. Check separately whether retrieval found the right passage, whether outdated information appeared, whether the answer is grounded, and how long review took.
Retrieval quality and final-answer quality are different stages. Search can be evaluated with labeled documents and ranking metrics; answers should be separately assessed for groundedness, relevance, and completeness. Automated evaluation helps compare versions but still needs expert spot checks.
Costs and security
Include document preparation, scan recognition, embeddings, storage, indexing, backups, monitoring, and the knowledge-base owner’s time. Add specialist review and correction. Do not index every shared folder at once: identify owners, freshness, and access rules. A passage the user may not access must not enter that user’s context.
Begin in recommendation mode, preserve source links, and route uncertain answers to a person. If retrieval is weak, improve the corpus and chunking; if context is good but the answer is poor, adjust instructions and output format. Recalculate accepted-answer share and net time savings after a few weeks. Expand the corpus or connect a CRM only after that.
