AI Systems / POV

RAG Systems: An Innovative Approach to Question-Answering

RAG tackles two limits of LLMs, hallucination and outdated knowledge, without retraining. How retrieval and generation fit together and the design choices involved.

RAG addresses two weaknesses of large language models: they can answer confidently and be wrong, and what they know can be out of date. By supplying relevant retrieved context along with the question, the model has source material to use in its answer. This can reduce unsupported answers, though it does not prevent them. Because the information lives in a separate vector database, you can update it without retraining the model, and that is why I would consider RAG when answers need to reflect changing information.

I break the system into two parts: a retrieval component that searches the vector database much like a search engine, and a generation component, such as GPT-3.5 or Llama2, that writes the answer from the retrieved context. I list use cases from question answering over documents to chatbots and recommendations, and eight design decisions, including chunk size, embeddings, the choice of database, the matching algorithm and how the prompt is built.

Summary of an essay originally published on Medium. Read the original ↗