Imagine your company has 10,000 documents. They might include product manuals, company policies, research reports, customer support documents and other internal knowledge.
Now a user asks "Can a customer return this product after 30 days?". The answer may already exist somewhere in those documents. But we cannot send all 10,000 documents to the LLM every time someone asks a question. Instead, we first find the information related to the question. This is the basic idea behind RAG.
What Is RAG?
RAG stands for Retrieval-Augmented Generation, which means Search first. Generate second. Before asking the LLM to answer, the system searches a knowledge base and finds the most relevant information. Then it adds that information to the prompt. Finally the augmented prompt is sent to LLM.
What Is an Embedding?
To resolve how the system knows which documents are relevant, this is where embeddings come in. An embedding converts text into numbers.
For example, How can I return my order? might be converted into something like [0.21, -0.37, 0.84, 0.12, ...]. This numerical representation is called a vector. These numbers represent the meaning of the text. This allows the system to understand that How can I return my order? and What is your refund process? are related, even though they use different words.
What Is a Vector Database?
These embeddings can be stored in a vector database. When a user asks a question, the system converts the question into an embedding and searches for similar vectors. The LLM now receives information that is actually related to the user's question.
