Retrieval-augmented generation, or RAG, has become a common way to make language models answer from your own data. That makes it a popular interview topic for AI roles. Interviewers want to hear that you understand the pipeline end to end and know where it fails.
Here are the core concept questions to prepare.
Core RAG questions
1. What is RAG, and why use it? RAG retrieves relevant documents at query time and gives them to the model as context. It helps the model answer from current or private data and reduces made-up answers, without retraining.
2. Walk me through a basic RAG pipeline. Ingest and split documents into chunks, create embeddings, store them in a vector database, retrieve the most relevant chunks for a query, then pass them to the model with the question.
3. What are embeddings? Numeric vectors that capture meaning, so similar text sits close together. Explain how similarity search uses them.
4. When would you use RAG instead of fine-tuning? RAG suits changing knowledge and source citations. Fine-tuning suits changing style or behaviour. They can also be combined.
5. What is a vector database, and what does it do? It stores embeddings and finds nearest neighbours quickly. Mention approximate search and the trade-off between speed and accuracy.
Practise explaining RAG in your own words
Understanding RAG is one thing. Explaining it clearly to an interviewer, including where it breaks, is another. Careerscan gives you a place to practise.
Careerscan reads your resume against the AI role you want, then interviews you on it. An AI and ML specialist interviewer asks about the skills the job needs and any RAG or LLM projects on your resume, and follows up when your answer stays at surface level.
- Questions built from your resume, the job’s skills and what the company is known to ask
- Voice or text answers, one question at a time
- A report that scores each answer and shows what a stronger answer would include
- A short lesson and practice round for each gap
Ready to practise? Start free on Careerscan (15 free credits, no card needed)
Chunking, retrieval quality and evaluation questions
6. How do you decide how to chunk documents? Chunks that are too small lose context and chunks that are too large add noise. Mention overlap, splitting at natural boundaries, and testing sizes against real questions.
7. What can go wrong in retrieval, and how do you improve it? The right chunk may not be retrieved. Mention hybrid search that combines keyword and vector search, re-ranking, query rewriting and metadata filters.
8. How do you reduce hallucinations in a RAG system? Retrieve better context, tell the model to answer only from the sources, require citations, and say “I don’t know” when evidence is missing.
9. How do you evaluate a RAG system? Evaluate retrieval and generation separately: whether the right chunks were found, and whether the answer is correct and grounded in them. Build a set of test questions.
10. Tell me about a RAG project you built. Describe the data, your design choices, what failed, and how you measured improvement.
How to answer well
- Walk through the pipeline in order.
- Name the failure points at each stage.
- Explain trade-offs such as cost, speed and quality.
- Say “I” for what you built, not only “we”.
- Show how you measured improvement, not just that you built it.
The bottom line
RAG interviews reward people who know where the system breaks and how to fix it. Learn the pipeline, then practise explaining it out loud with your own project.
Be ready for the follow-up questions. Start your free Careerscan interview and practise before the real one.