AI Systems / POV
Fine-Tuning vs RAG: Choosing the Right Approach for Your Question-Answering System
Fine-tuning or RAG for a question-answering system? Eight factors I use to decide, from task goals and latency to how often your information changes.
Fine-tuning and RAG are two ways to build a question-answering system with an LLM. Fine-tuning changes the model’s weights using training examples. RAG brings in information from external sources when a question is asked, so those sources can be updated without retraining the model. I mention LoRA, QLoRA and adapter-based methods, and open models like Llama2 that make both routes practical.
I compare eight factors. Fine-tuning can help with specific tasks such as classification and summarisation, and may allow a smaller, faster model, though it needs suitable training data. RAG is useful when information changes often, when answers depend on many documents, or when we need to control which sources the model uses. Which would I choose? I would start with what the system needs to answer and how often that information changes.
Summary of an essay originally published on Medium. Read the original ↗