A retrieval-based question answering system over private documents
Context-aware retrieval over a document corpus, so answers are grounded in the source material instead of generated from a model's memory.
- Design and implementation
- University project
- LangChain · Retrieval · GPT-Index
What it is.
A natural language interface over a document set: ask a question, get an answer drawn from the documents rather than from the model's general knowledge.
Built during college, before retrieval-augmented generation became a standard pattern. No large language model provider offered grounded, document-aware answering off the shelf at the time — what is a single API call today had to be assembled piece by piece.
Why it's hard.
- A language model asked about private documents will answer confidently from memory instead of from the source.
- Naive keyword search returns passages that share words with the question but not meaning.
Constraints
- Answers must be traceable to retrieved passages.
- The corpus is private, so nothing can rely on the model having seen it during training.
What I built.
- Built the indexing and retrieval pipeline and the answer composition step.
- Built the web interface used to query the corpus.
How it fits together.
Document corpus
Source material, chunked for retrieval
Index
Embedding-based lookup over chunks
Retriever
Selects the passages most relevant to the question
Answer composition
Language model answers from retrieved context only
Streamlit UI
Query interface
Engineering choices.
Ground every answer in retrieved text
The model is only allowed the retrieved passages as context, which turns a hallucination problem into a retrieval-quality problem — measurable and fixable.
Chunking as a first-class concern
Chunk size and overlap changed answer quality more than model choice did.
Building it.
- Indexing and retrieval with GPT-Index and LangChain, with a Streamlit front end for querying.
What fought back.
No off-the-shelf path to grounded answers
Models were far weaker and much less steerable, context windows were small, and embedding and retrieval tooling was immature and changing week to week. Chunking, ranking and prompt construction all had to be built and tuned by hand, with no established recipe to follow.
Questions that span several documents
Single-passage retrieval answers narrow questions well and synthesis questions badly; retrieving a wider, ranked set of passages was the practical fix.
Where it stands.
- A working system that answers questions from a private corpus with answers grounded in the retrieved source passages.
What I took from it
- In a retrieval system, most of the perceived model quality is actually retrieval quality.
- Building this before the pattern was common taught me the mechanics underneath today's RAG stacks — which is why I read new AI tooling as engineering rather than magic.
Tools and technology.
Retrieval
- LangChain
- GPT-Index
- Embeddings
ML
- TensorFlow
- Python
Interface
- Streamlit