RAG on company documents: why answer quality depends on file order

Before you start the assistant

If your repository contains old versions of policies, scans without OCR and duplicates of the same file, RAG will start answering from the same faulty foundations. According to AZONE-AI, in five deployments the most work required was not the model or the prompt, but data loading, metadata and versioning; a similar conclusion appears in SkillHint's material, where RAG quality depends on document order, how content is chunked, search, metadata and permissions.

In practice the problem is not whether the model "knows" the answer, but whether it can find it at all. If a document does not get into the index or is indexed in the wrong form, the user will receive an answer confident in tone but weak in substance.

What degrades the answer most

According to RedAI, low-quality scans, tables converted to images and unstructured files reduce the quality of search and the generated answer. ClickShot emphasizes that text from PDFs must be extracted together with headings, and that scans require separate OCR checks, because tables, footnotes and attachments often lose their structure.

The most common sources of errors are very simple:

• Old versions of documents get mixed with current ones.
• Duplicates and copies blur search results.
• Poor chunking severs context, for example splitting a condition from its exception.
• Missing metadata makes it hard to distinguish version, owner and scope.
• Weak OCR hides content the model cannot recover.

How to organize files before deployment

Praxio states plainly that an assistant is only as good as the materials it receives, and inconsistent documents produce inconsistent answers. In the AZONE-AI article and SkillHint's material the same direction appears: first organize the corpus, then tune search and generation.

A good minimal process looks like this:

• Designate a single set of sources for answers, not the entire network of company folders.
• Mark owner, status and date of currency for each document.
• Remove duplicates and clearly indicate the canonical document.
• Run scans through OCR and check read quality.
• Split long materials into logical sections, not mechanically every few pages.
• Test questions that rely on tables, footnotes and attachments.

How to recognise that RAG works well

Polygant points out that a high-quality RAG does not need the maximum number of files, but a managed knowledge corpus. The same material distinguishes three evaluation levels: search, generation and impact on workflow. This matters because sometimes the model crafts a good-looking answer but bases it on the wrong fragment.

The simplest business test is practical: does the system find the correct document, does it show the source of the answer, and does the quoted fragment match the question. If not, the problem usually lies not in the model but in the state of the files and how they were prepared.

Lub System helps B2B companies implement AI, automation and IT solutions end-to-end - from strategy to deployment. See our services or get in touch to discuss your case.