What should actually be automated

Local meeting transcription is useful not as an “AI recorder” but as a controlled pipeline: ingest audio, recognize speech, separate turns, extract decisions and actions, let a person verify the result, and only then write it to CRM or project systems.

This architecture fits meetings containing commercial terms, internal plans, personal data, and technical details that should not be sent to an external API. Local deployment controls the data path, but recognition errors, incorrect speaker labels, retention, and permissions remain process and architecture responsibilities.

whisper.cpp provides a local C/C++ runtime for Whisper models with CPU, Apple Silicon, NVIDIA, AMD, Vulkan, OpenVINO, and integer quantization support. It is a practical ASR layer for an existing server or workstation. “Who said what” needs a separate diarization component; more precise word timing needs alignment.

A seven-stage pipeline

1. **Recording and notice.** The interface clearly shows that recording is active, who owns it, and why a transcript is created. The company approves meeting categories, retention, and access before the pilot.
2. **Audio ingestion.** Files enter protected storage through a one-time link or approved folder. Checksum, source, duration, and meeting ID are retained.
3. **Normalization.** ffmpeg converts audio to a consistent format, sample rate, and channel layout. The original is preserved and the derivative is logged.
4. **Recognition.** whisper.cpp or another local ASR creates segments, text, and timestamps. Model, quantization, and parameter versions are pinned.
5. **Alignment and speakers.** WhisperX can refine word timestamps and use pyannote for diarization. Output contains provisional Speaker 1/2 labels, not proven identities.
6. **Structuring.** A language model receives the transcript and drafts decisions, tasks, deadlines, risks, and open questions. Missing fields must not be invented.
7. **Review and export.** A meeting secretary or project owner listens to uncertain passages, fixes names and numbers, and approves the minutes. Only confirmed actions reach external systems.

Separating stages makes each component replaceable and errors traceable. A monolithic “make minutes” button hides whether distortion came from audio, ASR, diarization, summarization, or export.

Diarization is not identity recognition

Speaker diarization answers “which segments share one voice,” not “what is this person's name.” Assigning real identities requires meeting rosters, introductions, manual mapping, or a separate biometric process with its own risks and legal basis.

pyannote.audio is an open Python toolkit for diarization. Its community pipeline can operate offline after accepting model conditions and downloading required files. The repository also documents optional telemetry; production settings should be reviewed and recorded.

WhisperX lists important limitations: overlapping speech is handled poorly, diarization is imperfect, alignment needs a language-specific phoneme model, and some numbers or symbols may lack accurate timestamps. Russian deployments therefore need explicit tests for surnames, abbreviations, amounts, dates, and mixed-language terminology.

Data and infrastructure

A pilot needs an isolated server, object or file storage, a job queue, and a small metadata database. GPU acceleration helps large models and batch processing, but whisper.cpp supports CPU. Size infrastructure for the required turnaround time, not a demo's peak speed.

Store separately:

  • original audio for the shortest justified period;
  • normalized audio as a temporary artifact;
  • draft transcript and confidence information;
  • human-corrected minutes;
  • access, model-version, and export logs.

Permissions inherit from the meeting. A user who cannot access the recording must not discover its passages through search or RAG. Use short-lived audio links, disable outbound network access where unnecessary, and download and verify model artifacts in advance.

Measuring quality

A single “accuracy percentage” is not enough. Use four metric groups:

  • text: word error rate plus separate accuracy for names, numbers, dates, and domain terms;
  • speakers: diarization error rate and the share of turns with correct labels;
  • minutes: decision and action recall, no invented commitments, correct owner and due date;
  • operations: turnaround time, manual correction share, cost per audio hour, and failed jobs.

Build a golden set from 20–30 representative meetings: a quiet room, one laptop microphone, remote participants, poor audio, interruptions, and Russian mixed with English terms. Replace sensitive data or label it entirely inside the protected environment.

Critical fields deserve stricter review than general prose. A minor introductory error rarely changes business outcomes; a wrong amount, date, or owner changes an obligation. Link uncertain text to the exact audio passage for fast verification.

Security and limitations

A transcript is a searchable derivative copy of a conversation and is easier to copy than audio. Transcript access must therefore be no broader than recording access. Add approved minutes to corporate RAG only with the same ACLs.

Treat automatic summaries as drafts. A model can merge discussions, assign a decision to the wrong person, or turn a possibility into a commitment. CRM, calendar, and task-tracker exports require a strict schema and owner approval.

Do not promise perfect speaker separation with noise and overlapping speech. Legal evidence, employment decisions, or other high-impact uses require a separate procedure and expert review; a general meeting assistant is insufficient.

A modeled economic case

Assume 20 employees conduct 120 meeting hours per month. Manual minutes require another 15 minutes per audio hour, or 30 hours. Reviewing drafts takes six minutes per hour, or 12 hours, releasing 18 hours.

At RUB 1,500 fully loaded per hour, the resource effect is RUB 27,000. Suppose preventing two missed actions adds RUB 40,000, while server support and quality control cost RUB 25,000. The modeled net effect is RUB 42,000 per month. A RUB 420,000 implementation has a simple ten-month payback.

This is an example. Economics fail when meetings are short, minutes are unused, or staff still replay every recording. Measure actual review time and the value of confirmed actions.

A two-week pilot

In week one, select one meeting type and ten recordings. Define recording notice and retention, deploy ASR without external access, compare two models or quantizations, and label errors in names, numbers, and speakers.

In week two, add draft minutes but prohibit automatic writes to external systems. The meeting owner approves decisions and actions while the team measures review time, completeness, and dangerous errors.

Scale only when real audio is accurate enough, responsible reviewers genuinely save time, and permissions and deletion schedules are verified. Local transcription becomes a business tool not when every word is recognized, but when confirmed decisions reliably reach the people who need them.