The short answer

n8n, Ollama, Qdrant and PostgreSQL can be assembled quickly into a private AI environment that tests a business hypothesis without sending company documents to an external API. A working Docker Compose stack, however, is not yet a production system.

n8n's official Self-hosted AI Starter Kit combines workflow orchestration, local model inference, a vector database and an operational database. Its own documentation positions the kit primarily as a starting point and proof of concept rather than a fully optimised production deployment. That distinction matters. A pilot answers “can AI solve this task?” Production must also answer who can access the data, what happens when a component fails, how the system is restored and who approves a consequential action.

What the stack gives a business

Each component solves a different part of the workflow:

  • **n8n** connects email, CRM, file storage and internal APIs, runs workflows and records execution history;
  • **Ollama** runs a language model inside the selected security perimeter;
  • **Qdrant** stores the vector index used to retrieve document passages for RAG;
  • **PostgreSQL** stores workflow state and operational data.

For a small or medium-sized company, this is a practical way to test one narrow use case: classifying incoming requests, searching policies, drafting a response, extracting contract fields or checking an application against internal rules.

Start with a process whose input, expected outcome and responsible owner are known. Starting with a “universal corporate agent” often produces an attractive chat interface without a measurable business result.

What to measure before discussing production

A representative sample and a manual result review are enough for the first pilot. At minimum, record:

  • the share of documents or requests handled correctly;
  • the share of answers that cite the right source;
  • how often an employee must correct the output;
  • processing time before and after automation;
  • response latency and peak load;
  • cases in which the system should refuse to answer;
  • the cost of one successfully completed operation, including infrastructure and staff time.

Measure the quality of the final action, not “model accuracy” in the abstract. A model may summarise a document well and still select an obsolete policy version, leaving the business outcome wrong.

Why the quick Compose file cannot simply stay in production

Demo environments commonly place every component on one network, keep secrets in a shared environment file and update by pulling the newest image. That is convenient for experimentation but creates several classes of risk.

Mutable versions

An image tagged `latest` can change between restarts. Production needs pinned versions of n8n, Ollama, Qdrant and PostgreSQL, a change log and compatibility tests on a separate environment. Rollback must be designed before an upgrade, not improvised after one.

Secrets and network boundaries

Integration passwords, API keys and the n8n encryption key should not live in a shared file available to every project participant. Use separate secret storage, least-privilege read access, a rotation procedure and protected backups of keys required for recovery.

Do not expose internal services directly to the internet. External access should pass through TLS and an authenticated reverse proxy or corporate gateway. Internal ports should be reachable only by the components that require them.

An unprotected vector database

Qdrant documentation warns that a self-hosted open-source instance listens on network interfaces without authentication or encryption by default. Before real documents are ingested, bind it to a private network and enable TLS and API keys. Search-only consumers should receive read-only or collection-scoped rights, not an administrative key.

Backups that were never restored

A Docker volume is not a backup strategy. PostgreSQL backups, Qdrant snapshots, workflow configuration and encryption keys need schedules, retention and a tested restore procedure. The useful metric is not that a backup job ran, but the measured time required to return the system to service.

A practical first production architecture

The first production use case does not automatically require Kubernetes. One or two managed nodes can be sufficient when responsibilities and operating procedures are explicit.

1. **Ingress.** A corporate gateway authenticates users, rate-limits requests and writes a technical audit log.
2. **Orchestration.** n8n runs approved workflows; experimental and production workflows are separated.
3. **Inference.** Ollama or another private model server receives only the context needed for the request, not unrestricted access to the archive.
4. **RAG.** Qdrant stores vectors with access metadata, document version, department and validity period.
5. **Operational data.** PostgreSQL runs on protected storage, is backed up on schedule and is isolated from temporary experiments.
6. **Actions.** Read operations are separated from writes. Sending email, changing CRM data, publishing or deleting requires deterministic validation and, for consequential operations, human approval.

Queue mode, Redis and additional workers become useful when measured workload demands them. They should be introduced to solve an observed capacity problem, not merely to make an architecture diagram look mature.

Data and RAG: the model is rarely the main risk

RAG quality often fails because of the index contents rather than the selected LLM. Before ingestion, define the owner of each dataset, permitted user groups, the current document version, expiry date, deletion rules, treatment of personal or contractual data and the method for removing drafts and duplicates.

Permissions must be enforced before retrieval, or inside collection filters, rather than after text generation. A language model is not an authorisation system and must never decide whether a user is entitled to see a retrieved passage.

Agents need fewer permissions than employees

OWASP describes excessive agency as a combination of unnecessary functionality, permissions and autonomy. In practice:

  • a read tool should not be able to delete or transmit data;
  • integrations should act as a specific user or a narrowly scoped technical role;
  • financial, legal or public actions need human approval or deterministic business checks.

A system-prompt instruction is useful guidance, but it does not replace database permissions, OAuth scopes, network policy or validation in the target system.

Count the economics of the whole environment

Private inference removes variable external API charges but adds hardware, electricity, updates, monitoring and responsibility for availability. Compare the total cost of one reliably completed process, not the price of a token with the price of a GPU.

Include servers, implementation and support time, backups, logs, downtime during upgrades, manual review, the cost of errors and component licensing constraints. An external API may remain more economical for an infrequent process with no sensitive data. Private AI is most compelling when data control, predictable workload, autonomy or integration with closed systems dominates the decision.

A transition plan

1. Fix one process, owner and success metric.
2. Build the pilot on anonymised or limited data.
3. Validate it on real examples and catalogue recurring errors.
4. Map data, integrations and minimum permissions.
5. Pin component versions and create a separate test environment.
6. Configure TLS, secrets, network restrictions and roles.
7. Add logs, limits, resilience and backups.
8. Prove restore and safe shutdown procedures.
9. Keep human approval for irreversible actions.
10. Expand only after these controls work.

Executive takeaway

The Self-hosted AI Starter Kit is an effective, inexpensive way to prove the value of private AI. It is not “enterprise AI in one command.” A sensible pilot ends with metrics, a data map, a risk register and a production cost estimate—not only a demonstration.

If nobody can explain who owns the data, how the environment is restored and who approves a write to an external system, the project is not ready for production, even if the robot is already blinking happily.