Efficient, faithful RAG
Does compressing the context destroy the evidence you'd need to explain a fact-check? To find out, I trained a REFRAG-inspired classifier for adaptive retrieval on FEVER. It reached 88.2% accuracy while retrieving 1.5 sentences on average, compared with a full-context baseline at 85.6% that used 11.1 sentences and took about five times longer.
- Role
- Solo course project, EECS 595 NLP
- Timeline
- Fall 2025
- Stack
- PyTorch, Qwen-3 4B, FEVER
- Status
- Paper, poster and code below
The problem
RAG grounds a model in evidence, but the cost grows with the context, and so does the noise. The open question for me was whether aggressive compression keeps the sentences a human would need to check the answer, or only the ones the model needs to guess it.
What I built
An end-to-end retrieval, compression and evaluation pipeline on FEVER using Qwen-3 4B. I compared keyword top-k, semantic top-k and compression-based selection on accuracy, evidence recall and context efficiency.
Results
Adaptive selection at a threshold of 0.7 reached 88.2% accuracy and 90.1% evidence recall while using 87% fewer context sentences than the full-context baseline. Aggressive compression improved both efficiency and explainability, up to a point. Past that threshold, recall falls off quickly.
What I'd do differently
The interesting finding for me was that less context was more faithful, not just cheaper. The model simply stopped getting distracted. I'd want to test it on a domain where the evidence isn't a single Wikipedia sentence, which is exactly the setting Insora lived in.