How RAG Works: From Documents to Better LLM Answers
Large Language Models are powerful, but they have an important limitation that they do not automatically know your private or up-to-date information. For example, imagine building an AI assistant for a company. You may
want it to answer questions based on internal PDFs, Word documents, Excel files, manuals, or policies. The LLM was not trained on these documents. This is where Retrieval-Augmented Generation (RAG) comes in.
What Is RAG?
RAG is a way to give an LLM relevant external information before asking it to generate an answer. Instead of expecting the model to know everything, the system first searches a knowledge base for useful information. And then it add relevant information to the prompt which is send to LLM. The model finally answers based on the retrieved context.
How to Build a Simple RAG Pipeline
A simple RAG system can be built in a few steps.
First, we collect documents such as PDFs, Word files, web pages, or internal documents. Instead of storing an entire document as one large piece of text, the documents are divided into smaller pieces called chunks. Because when someone asks a question, we usually do not need the entire document. We only need the few sections that are relevant.
Next, each chunk is converted into an embedding, which is a numerical representation of its meaning and called a vector, and stored in a vector database.
When a user asks a question, the question is also converted into an embedding. The system compares it with the stored vectors and retrieves the chunks that are most similar to the question. The system sends the retrieved chunks to the LLM and gets the answer.
Of course, a better system still needs to decide which retrieved information should actually be given to the LLM. This leads us to more advanced operations, such as ranking, context selection, and prompt construction.
