AI on the company's own documents (RAG), in Hungarian too

According to Mesterséges Intelligencia Központ Kft., a software development company based in Budapest, Hungary, RAG gives verifiable answers from a company's own documents: the system first finds the relevant passages, and the model answers from them with sources, in Hungarian too.

In short: There is no need to train the model on your own data: RAG hands it the relevant passages at the moment of the question, and the answer cites its source. Quality depends on how well the documents are organised, how accurate the search is and on continuous measurement; it works in Hungarian too, but it has to be measured on your own Hungarian questions.

What is RAG, and where does the term come from?

RAG stands for retrieval-augmented generation. The term comes from a paper published in 2020 by Patrick Lewis and co-authors, titled “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”. The authors combined the so-called parametric memory of a pre-trained language model with a non-parametric memory: a dense vector index of Wikipedia, accessed with a neural retriever.

Today the term more broadly means a solution in which, for each question, the system first finds the necessary passages in a collection of documents, and the language model formulates the answer from them. The model's own knowledge does not change in the process: it receives the company's material at the moment of the question. This gives RAG three practical advantages: knowledge can be updated immediately, the answer can cite its source, and the search can be filtered to what the person asking is allowed to see.

How does it work? Six steps

Collecting: we bring the sources together, such as the document store, the wiki, the ticketing system and selected parts of the mailbox.
Splitting: long documents are broken into smaller parts (chunks) of a few paragraphs. OpenAI's retrieval service, for example, by default creates chunks of up to 800 tokens with an overlap of 400 tokens.
Embedding: an embedding model turns each chunk into a list of numbers, a vector. According to OpenAI's description, a small distance between vectors suggests high relatedness and a large distance low relatedness.
Searching: the same model turns the question into a vector, and the system finds the chunks closest to it. In its description of Contextual Retrieval, Anthropic complements search by meaning with literal keyword search (BM25), because it finds exact terms, such as identifiers, better, and then a reranking step passes only the most relevant chunks to the model.
Filtering and assembling: of the chunks found, only those the person asking may see are kept, and they are given to the model together with the question.
Answering with sources: the model answers from the passages it received and indicates where each statement comes from. Anthropic's citations feature points to page numbers for PDFs and to character positions for plain text; OpenAI's file search adds annotations in the answer that reference the source file.

A common problem with splitting is that a chunk loses its context; Anthropic warns about this in the same description. A sentence such as “The deadline runs from the date of receipt” does not say on its own which contract or which deadline it refers to. That is why it is worth adding the document title, the section heading and the date to every chunk, so that both the search and the model can see its context.

RAG or fine-tuning?

Fine-tuning means training the model further on task-specific data. According to OpenAI's guide to LLM accuracy, RAG provides the domain-specific context a task needs and is called for when the model's knowledge is out of date or the task requires the company's proprietary information; fine-tuning helps when the model's output is inconsistent or its format or tone is wrong. According to the guide, the two methods complement each other and can be used together.

Aspect	RAG	Fine-tuning
What it is for	giving the model the company's knowledge at the moment of the question	making the model's behaviour, format and tone consistent
Fresh knowledge	valid once the index is updated after the document changes	requires new training
Cited sources	the answer can point to the passage found	knowledge built into the model's weights comes with no source reference
Access rights	can be filtered per document	the trained knowledge is the same for every user
Withdrawing a document	the document is removed from the index	its effect only disappears with new training
When it is worth it	almost always the starting point for company documents	when consistency of format or tone is the problem, alongside RAG

For company documents, RAG is therefore the natural starting point. The question “do we have to train the model?” usually really means: how does the model get the company's knowledge? And the answer to that is RAG.

In Hungarian too? Language support and mixed-language material

The answering model

The models of the major providers understand and write Hungarian well, but their published language benchmarks do not always cover Hungarian. Anthropic's multilingual support page publishes results for 14 languages other than English, and Hungarian is not among them (checked on 17 October 2026); the page also stresses, however, that Claude is capable in many languages beyond those in the table, recommends testing with the languages relevant to the application, and advises stating the response language in the system prompt. This matches our own experience: the Claude-based AI assistant on our homepage talks in Hungarian and 13 other languages, but we measure every new use on Hungarian questions.

The embedding model

Search quality depends at least as much on the embedding model as on the answering model. Among the major providers (checked on 17 October 2026):

according to its documentation, Google's Gemini Embedding 2 model works across over 100 languages;
Anthropic does not offer its own embedding model but points to Voyage AI's models, of which voyage-4 is optimised for general-purpose and multilingual retrieval;
OpenAI's embeddings guide presents the text-embedding-3 models but does not list the supported languages.

We therefore choose the embedding model on the company's own Hungarian question–document pairs, not on the provider's general benchmarks.

Hungarian specifics

Inflection: “szerződés”, “szerződésben” and “szerződéseinkről” (contract, in the contract, about our contracts) are about the same thing, but literal keyword search does not treat them as a match without stemming. Search by meaning helps here, and hybrid search combines the two.
Technical terms and internal names: the company's abbreviations, product codes and internal names are not part of the model's general knowledge; a short glossary improves the search a great deal.
Mixed-language material: if the question is in Hungarian and the manual in English, a multilingual embedding model is needed and the response language must be fixed; quoted sources stay in their original language.
Everyday phrasing: typos, questions typed without accents and colloquial expressions must also be included among the test questions.

Which documents can it work from?

Source	What to watch
Text-based PDF	processes well; source citations can point to page numbers
Scanned PDF	text recognition (OCR) is needed first: according to Anthropic's documentation, scanned PDFs without extractable text cannot be cited
Word and Excel	Anthropic recommends converting DOCX and XLSX files to text, while OpenAI's file search also accepts DOCX directly; the structure of tables must be preserved
Wiki and intranet	the current version should count, and old pages should be archived
Tickets, help desk	they contain a lot of personal data and outdated solutions, so they must be filtered
Emails	only a selected, organised set; the most sensitive source in terms of personal data and access rights

Access rights per document

The most sensitive point of RAG is that the system must not reveal a passage that the person asking could not open in the source system either. We do not leave this to the model but build it into the search:

every chunk carries the access rights of its source document as metadata;
filtering happens before or during the search (OpenAI's retrieval service, for example, offers filtering by file attributes before semantic search), so whatever the person asking may not see never even reaches the model;
changes in access rights are synchronised from the source system rather than maintained by hand;
we log who asked what and from which documents they received an answer.

Keeping the index fresh

RAG is only as good as its index is fresh. What we pay attention to:

Change detection: the system processes new and modified documents automatically.
Deletion: a document deleted at the source is also removed from the index; according to OpenAI's documentation, removal is eventually consistent, so a deleted file may still appear in results for a short time.
Versions: distinguishing the current version from a withdrawn one, with dates in the metadata.
Source owner: every source has someone responsible who flags when something is out of date.
Expiry: an expiry rule for temporary material; OpenAI's vector stores, for example, can be given an expiry.

Fewer wrong answers: sources, “I don't know”, measurement

A language model can be wrong even with the material provided. Anthropic's guide on reducing hallucinations recommends explicitly allowing the model to say “I don't know”, grounding it in direct quotes, having it find a supporting quote for each claim (and retract the claim if it cannot), and restricting it to the documents provided. The guide adds that these techniques reduce but do not eliminate wrong answers entirely, so critical information must be validated. We build on these:

Cited sources: every answer shows which part of which document it comes from.
“I don't know” and handover: if the answer is not in the material, the system says so and hands the question to a member of staff.
Only from the material provided: the model does not use its general knowledge about the company.
Evaluation set: real questions with the expected answer and the correct source. We measure separately whether the search found the right passage and whether the answer is faithful to it; according to OpenAI's evaluation guide, test data should include typical, edge and adversarial cases, and the evaluation should be rerun on every change.
Feedback: users can flag a bad answer, and the log produces new test cases.

Data protection: where are the documents stored?

In a RAG system the company's material passes through three places:

The source system: the documents stay where they were, in the document store, the wiki or the mailbox.
The index: the chunks and vectors in a search database. It can run in the company's own environment or at a provider. With OpenAI's file search, for example, uploaded files remain stored until they are deleted or the vector store expires, according to its data controls documentation and retrieval guide.
The model call: only the chunks selected for the question reach the model. The provider's data processing terms apply here. What they say about training, retention and the place of processing is summarised in the article on connecting business systems with AI.

If the documents contain personal data, the processing falls under the GDPR, and a contract under Article 28 of the General Data Protection Regulation is needed with the provider acting as processor. We cover the legal questions (transparency, data transfers, impact assessment) in the article AI Act and GDPR for a business chatbot. This is information, not legal advice.

The steps of a RAG project

Goal and scope of questions: who asks, about what, and what counts as a good answer.
Inventory and clean-up of sources: filtering out duplicates and contradictory or outdated versions.
Access model: who may receive answers from which document.
Evaluation set: real Hungarian questions with the expected answer and source.
Prototype: tuning the splitting, the embedding model and the search based on measurement.
Connection: the interface, cited sources, handover to staff and logging.
Pilot: with a narrow group of users, collecting feedback.
Operation: updating the index, continuous measurement and testing model switches.

Typical mistakes

The document store “as it is”: if three versions of the same policy exist, the system may quote any of them.
Access rights as an afterthought: in the first demo everything is visible; in production this can be a data breach.
Answers without sources: users cannot check them, and do not trust them either.
Testing with made-up questions: the demo questions work nicely, the real Hungarian questions with typos do not.
A one-off load: the knowledge base soon goes out of date without updates.
Scanned PDFs without text recognition: the system simply cannot see their content.

Frequently asked questions

What is RAG?

RAG (retrieval-augmented generation) is a solution in which, for each question, the system first finds the relevant passages in the company's documents, and the language model answers from them, citing the source. The term comes from a research paper published in 2020.

Does it answer in Hungarian from our own documents?

Yes. Today's large language models understand and write Hungarian well, and they also work from Hungarian documents; the AI assistant on our homepage talks in Hungarian too. Because the providers' published benchmarks do not always cover Hungarian, we measure both the answering and the embedding model on the company's own Hungarian questions before going live.

Do we have to train the model on our own data?

As a rule, no. RAG hands the model the passages it needs at the moment of the question, so knowledge can be updated, the source can be cited and access can be filtered per document. Fine-tuning may come into play if the format or tone of the answers is not consistent enough, and even then alongside RAG, not instead of it.

What happens if it does not know the answer?

We build it to say so and to hand the question to a member of staff instead of making something up. The model is explicitly allowed to answer “I don't know”, may only work from the material provided, and unanswered questions are fed back from the log to extend the knowledge base.

Where are our documents stored?

In the source system, as before; alongside them in the search index, which can run in the company's own environment or at a provider; and for each question only the selected passages reach the model, under the provider's data processing terms. Which setup suits a company depends on how sensitive the documents are and on its data protection requirements.

How we can help

Mesterséges Intelligencia Központ Kft. develops AI chatbots that answer from the company's own documents (in Hungarian too) using RAG with cited sources, built on the language models of OpenAI (ChatGPT), Anthropic (Claude) and Google (Gemini). The first consultation is free of charge; we measure the size of the development in COSMIC function points, and the offer and the schedule are based on it. We sign a non-disclosure agreement (NDA) on request, and after handover we also keep the knowledge base up to date and monitor how the system performs. The contract governs source code and usage rights.

The details are on the AI chatbots and agents and AI integration pages; the article Chatbot or AI agent? helps decide which of the two you need. If you are planning a system that answers from your company's documents, write to us using the details on the Contact page.