RAG: how AI answers from your documents
Updated:
RAG (retrieval-augmented generation) means the AI first finds the relevant pieces of your documents and only then writes an answer from them. The model does not need to be retrained, and you can see which document each answer came from.
The three steps
Split documents into short pieces. Turn each piece into a vector, a list of numbers that captures its meaning, and store them. When a question comes in, find the closest pieces and give them to the model together with the question.
Why answers go wrong
Usually the search found the wrong pieces, not the model invented things. Pieces that are too long, tables flattened into text and scanned PDFs without a text layer are the common causes.
Keep a small set of real questions with known answers and rerun it after every change. That is the only honest way to know RAG got better.