Guide 7 min read

AI assistant over company data: how it works, what it needs and where its limits are

I explain how a RAG assistant finds answers in your documents, what you need to prepare and where its limits lie, without technical jargon.

AI assistant over company data: how it works, what it needs and where its limits are
Author: Maksym Kamenchuk, Nexflow — Banská Bystrica Published Updated

An AI assistant over company data is usually built with the RAG technique (Retrieval-Augmented Generation). It is a technique that improves AI answers by connecting a language model to an external data source. The n8n documentation defines it this way.

When asked a question, the assistant first finds the relevant parts of your documents, and only then does the model compose an answer from them. It does not rely only on its own training data, but also on your own materials.

So you need documents in a usable form, a vector database and a language model. The quality of the answers depends on the source materials and on how the texts are split.

How a RAG assistant works

An ordinary language model relies on its training data. It generally does not know your internal policies, price lists or contracts. RAG solves this by searching for relevant documents before answering and basing the answer on them (n8n documentation).

  1. Preparing documents. The documents are uploaded to a vector database, which splits them into parts (chunks).
  2. Conversion to vectors. Each part is converted by a so-called embedding model into a vector, i.e. a numerical representation of the text.
  3. User question. The system finds the parts with the most similar meaning.
  4. Answer. The language model receives the retrieved parts and creates an answer backed by your documents.

A vector database is a special database designed for storing and searching vectors. It searches by meaning, not by keyword matches (n8n documentation). A customer can therefore ask in different words than those in the document, and the right passage can still be found.

How a RAG assistant answersHow a RAG assistant answers1 · Preparing documents• Upload to a vector database• Split into parts (chunks)2 · Conversion to vectors• Embedding model• Numerical representation of text3 · User question• Parts with the most similar meaning are found4 · Answer• Language model receives retrieved parts• Answer backed by your documents
Diagram. Process illustration: from documents to an answer.

What you need for it

Source documents

The texts that are meant to be the source of answers: manuals, internal procedures, product sheets. I recommend keeping them clear and up to date, because the assistant relies on them.

Embedding model

Converts texts into vectors. It can be, for example, a model from OpenAI (example workflow in n8n).

Vector database

Stores the vectors and searches them by similarity of meaning.

Language model

Composes an answer in natural language from the retrieved parts.

The embedding model and the language model do not have to come from the same provider. For example, Anthropic does not offer its own embedding model (Anthropic documentation).

Why text splitting matters

The assistant does not work with a whole document at once, but with its parts. According to n8n, small chunks (for example 200 to 500 tokens) are suitable for precise retrieval. Large chunks carry more context, but they can be diluted or noisy (n8n documentation). Put simply, a token is a piece of text, the unit in which language models count text.

Overlap between neighbouring chunks is also important, so that the AI understands the context of each one. Splitting by structure, for example Markdown, can help (n8n documentation).

What it means for you: the right splitting setup depends on your data. There is no universal number, and I recommend testing the answers on real questions after the setup.

How to keep knowledge up to date

Documents change, so it is worth considering connecting the assistant to the source instead of filling it manually. In the sample n8n workflow, documents from Google Drive are synchronised with the vector database automatically, without manual intervention (workflow in n8n). With a different document source, such a connection has to be set up separately.

The answer then comes from the retrieved parts of documents, not only from the model's knowledge, so it is grounded in sources (workflow in n8n).

Data protection and GDPR

With RAG, your texts travel to several services: the embedding model, the vector database and the language model. When designing the system you therefore need to know where each part is processed and with whom you sign contracts.

A community workflow for n8n describes the option of routing AI calls through the European infrastructure of the Requesty service. According to the workflow's author, this ensures privacy protection (workflow in n8n). This is the workflow author's claim, not a verified fact, and certainly not a guarantee of compliance.

GDPR compliance is not determined by the tool, but by the lawfulness of processing, informing the individuals concerned and contracts with processors. Before uploading documents containing personal data, I recommend consulting a lawyer or a data protection officer. Nexflow does not provide tax or legal advice.

I write about running automations on your own server in the article n8n self-hosted for a company.

Limits of a RAG assistant

When it is not worth it

My work starts at €390. It is described in more detail in the article How much does AI automation cost for a small business.

Frequently asked questions

What is RAG in simple terms?

It is a technique in which the AI, before answering, searches for the relevant parts of your documents and composes the answer from them. According to the n8n documentation, the model thus does not rely only on its training data, but also on the company's own knowledge.

Do I have to train the model on my data?

With RAG the model does not rely only on its training data; it retrieves the relevant parts of your documents and uses them as the basis for the answer. How the chosen service handles the data specifically has to be checked in its terms.

Which documents are suitable?

Texts in which an answer can be found: manuals, internal procedures, product information. I recommend keeping them clear and up to date. The way of splitting into chunks is adapted to the specific data.

Can the assistant's knowledge be kept up to date without manual work?

In the sample n8n workflow, changes to documents in Google Drive are transferred to the vector database automatically. With a different document source, such a connection has to be set up separately.

Is it GDPR compliant?

It depends on what data you upload, where it travels and what contracts you have with processors. A tool alone does not guarantee compliance. Nexflow does not provide tax or legal advice, so I recommend an assessment with a lawyer or a data protection officer.

When is a RAG assistant not enough?

When source materials are missing, when the AI is supposed to perform actions, or when every answer must be flawless without checking. In that case you need a different solution or a human check added.

Sources

Related services

ServiceAI assistant over company data ServiceAI chatbot for business

Related articles

DecidingAI agent vs chatbot vs automation: what you need TechnicalSelf-hosted n8n: VPS cost, backups and GDPR PricingHow much does AI automation cost for a small business?

Thinking about an assistant for your documents?

Describe to me what documents you have and what questions the assistant should answer. I will tell you honestly whether RAG is worth it or a simpler solution is enough.

I want a free consultation →