AI assistant over company data: how it works, what it needs and where its limits are
I explain how a RAG assistant finds answers in your documents, what you need to prepare and where its limits lie, without technical jargon.
An AI assistant over company data is usually built with the RAG technique (Retrieval-Augmented Generation). It is a technique that improves AI answers by connecting a language model to an external data source. The n8n documentation defines it this way.
When asked a question, the assistant first finds the relevant parts of your documents, and only then does the model compose an answer from them. It does not rely only on its own training data, but also on your own materials.
So you need documents in a usable form, a vector database and a language model. The quality of the answers depends on the source materials and on how the texts are split.
How a RAG assistant works
An ordinary language model relies on its training data. It generally does not know your internal policies, price lists or contracts. RAG solves this by searching for relevant documents before answering and basing the answer on them (n8n documentation).
- Preparing documents. The documents are uploaded to a vector database, which splits them into parts (chunks).
- Conversion to vectors. Each part is converted by a so-called embedding model into a vector, i.e. a numerical representation of the text.
- User question. The system finds the parts with the most similar meaning.
- Answer. The language model receives the retrieved parts and creates an answer backed by your documents.
A vector database is a special database designed for storing and searching vectors. It searches by meaning, not by keyword matches (n8n documentation). A customer can therefore ask in different words than those in the document, and the right passage can still be found.
What you need for it
Source documents
The texts that are meant to be the source of answers: manuals, internal procedures, product sheets. I recommend keeping them clear and up to date, because the assistant relies on them.
Embedding model
Converts texts into vectors. It can be, for example, a model from OpenAI (example workflow in n8n).
Vector database
Stores the vectors and searches them by similarity of meaning.
Language model
Composes an answer in natural language from the retrieved parts.
The embedding model and the language model do not have to come from the same provider. For example, Anthropic does not offer its own embedding model (Anthropic documentation).
Why text splitting matters
The assistant does not work with a whole document at once, but with its parts. According to n8n, small chunks (for example 200 to 500 tokens) are suitable for precise retrieval. Large chunks carry more context, but they can be diluted or noisy (n8n documentation). Put simply, a token is a piece of text, the unit in which language models count text.
Overlap between neighbouring chunks is also important, so that the AI understands the context of each one. Splitting by structure, for example Markdown, can help (n8n documentation).
What it means for you: the right splitting setup depends on your data. There is no universal number, and I recommend testing the answers on real questions after the setup.
How to keep knowledge up to date
Documents change, so it is worth considering connecting the assistant to the source instead of filling it manually. In the sample n8n workflow, documents from Google Drive are synchronised with the vector database automatically, without manual intervention (workflow in n8n). With a different document source, such a connection has to be set up separately.
The answer then comes from the retrieved parts of documents, not only from the model's knowledge, so it is grounded in sources (workflow in n8n).
Data protection and GDPR
With RAG, your texts travel to several services: the embedding model, the vector database and the language model. When designing the system you therefore need to know where each part is processed and with whom you sign contracts.
A community workflow for n8n describes the option of routing AI calls through the European infrastructure of the Requesty service. According to the workflow's author, this ensures privacy protection (workflow in n8n). This is the workflow author's claim, not a verified fact, and certainly not a guarantee of compliance.
GDPR compliance is not determined by the tool, but by the lawfulness of processing, informing the individuals concerned and contracts with processors. Before uploading documents containing personal data, I recommend consulting a lawyer or a data protection officer. Nexflow does not provide tax or legal advice.
I write about running automations on your own server in the article n8n self-hosted for a company.
Limits of a RAG assistant
- Answer quality depends on the source materials. If documents are outdated or contradictory, the assistant will rely on exactly those. This is my practical conclusion, not a statement from the documentation.
- An answer is not guaranteed. The system finds the most similar parts by meaning, but it may not always find the right one. For important topics I therefore recommend checking the answers.
- Choosing a model is a trade-off. Smaller embedding models are faster and cheaper and suit short general documents. Larger ones understand meaning better and suit long documents or complex topics (n8n documentation).
- The setup needs tuning. Chunk size, overlap and the type of model all affect the result and vary with the data.
When it is not worth it
- You have nothing to search. If the knowledge exists only in people's heads and not in documents, it first has to be written down. The assistant will not invent it.
- You have few documents. For a few pages, a well-organised FAQ page or a simple chatbot is enough.
- You need the AI to act, not answer. That is more a case for an AI agent or automation.
- The answer must always be flawless (for example a legally binding opinion) and nobody will check it. RAG is not enough for that.
My work starts at €390. It is described in more detail in the article How much does AI automation cost for a small business.
Frequently asked questions
What is RAG in simple terms?
It is a technique in which the AI, before answering, searches for the relevant parts of your documents and composes the answer from them. According to the n8n documentation, the model thus does not rely only on its training data, but also on the company's own knowledge.
Do I have to train the model on my data?
With RAG the model does not rely only on its training data; it retrieves the relevant parts of your documents and uses them as the basis for the answer. How the chosen service handles the data specifically has to be checked in its terms.
Which documents are suitable?
Texts in which an answer can be found: manuals, internal procedures, product information. I recommend keeping them clear and up to date. The way of splitting into chunks is adapted to the specific data.
Can the assistant's knowledge be kept up to date without manual work?
In the sample n8n workflow, changes to documents in Google Drive are transferred to the vector database automatically. With a different document source, such a connection has to be set up separately.
Is it GDPR compliant?
It depends on what data you upload, where it travels and what contracts you have with processors. A tool alone does not guarantee compliance. Nexflow does not provide tax or legal advice, so I recommend an assessment with a lawyer or a data protection officer.
When is a RAG assistant not enough?
When source materials are missing, when the AI is supposed to perform actions, or when every answer must be flawless without checking. In that case you need a different solution or a human check added.
Sources
- Retrieve relevant context | Build — checked 5 October 2026
- Embeddings - Claude Platform Docs — checked 5 October 2026
- Build a self-updating RAG system with OpenAI, Qdrant and ... — checked 5 October 2026
- Dynamic AI model selector with GDPR compliance via ... — checked 5 October 2026
Related services
Related articles
Thinking about an assistant for your documents?
Describe to me what documents you have and what questions the assistant should answer. I will tell you honestly whether RAG is worth it or a simpler solution is enough.
I want a free consultation →