- RAG
- FastAPI
- Streamlit
- Ollama
- Python
WealthTech RAG Assistant
A RAG system that knows which of four disagreeing documents is telling the truth.
Status: Running locally. Fully built and verified against a real fund's actual regulatory documents.
Problem
A relationship manager meeting a client needs a precise answer about a fund — exit load, expense ratio, holdings, strategy — and those answers live across four separate regulatory documents: a monthly Factsheet, a Key Information Memorandum (KIM), a Scheme Information Document (SID), and a Statement of Additional Information (SAI). Individually these documents run long — a SID alone is often 70+ pages, an SAI can run past 180.
The harder problem isn't length. It's that these documents disagree with each other on purpose. A factsheet reports the expense ratio actually being charged this month. The SID states the maximum the scheme is permitted to charge under SEBI regulation. Both numbers are correct. They are not the same number. A system that retrieves "the most similar passage" and presents it as the answer has a roughly fifty-fifty chance of handing an RM the wrong figure — with a citation that looks entirely legitimate.
A second, quieter version of the same problem: a single AMC's monthly factsheet often covers five to ten schemes in one PDF. A naive system that chunks the document without tracking which pages belong to which scheme can retrieve Scheme A's number for a question about Scheme B — a real, correctly formatted number, attributed to the wrong fund.
Zoom out and the scale makes manual lookup impractical to begin with. India has roughly 40 AMCs, each running upwards of ten schemes, each scheme carrying its own Factsheet, KIM, SID, and SAI — thousands of pages an RM cannot reasonably keep open, let alone cross-reference, in the middle of a client meeting. Today that gap is closed by memory, a guess, or "let me get back to you." What's actually needed is the ability to just ask, in plain language, and get back the exact answer — with the exact document, page, or line it came from, so the RM can show the client the source in the same conversation.
Solution
A retrieval-augmented question-answering system built around a document-authority routing layer — code that decides which of the four document types is allowed to answer a given question, before any similarity search runs. This sits ahead of standard RAG retrieval, not instead of it.
In practice, this produces three specific behaviours, each verified against a real fund's actual documents:
- A question about permitted ceilings excludes the factsheet entirely. "What is the maximum permissible expense ratio" retrieves only from the SID — the factsheet is not merely ranked lower, it is not in the candidate set at all.
- A question where documents genuinely disagree retrieves both sides deliberately. Expense ratio, exit load, and portfolio turnover are three attributes where the factsheet and the KIM/SID diverge by design. These trigger a guaranteed retrieval floor from each side, so the answer presents both figures — labelled by document type and date — rather than silently picking one.
- An ambiguous or unanswerable question is refused, not guessed. A scheme-specific question with no scheme named, or a question the corpus genuinely has no document to answer, returns a stated refusal rather than a fabricated response.
Every one of these is a deterministic decision — pattern matching and explicit weight tables — made in code, not by asking a language model to judge. The model's only job is turning retrieved, verified passages into readable prose with citations that map back to a real page number.
Architecture
Streamlit UI
| HTTPS + shared secret
v
FastAPI service
|-- Document-authority router deterministic -- which doc type(s) apply?
|-- Compliance classifier deterministic -- permitted? which scheme?
|-- Hybrid retriever cosine + BM25 + reciprocal rank fusion
|-- Generator local LLM -- language only
v
Local index (chunks, embeddings, document + scheme metadata)
No vector database — at the corpus sizes this system is built for, a brute-force similarity search over an in-memory array completes in single-digit milliseconds. No hosted LLM API — embeddings and generation both run locally via Ollama, with no external call, no API key, and no usage cost.
Where the LLM Fits (and Where It Doesn't)
Almost every decision that determines the answer happens before the model is ever called — and it's all deterministic code, not a language model's judgment. Whether the question is even allowed, which of the four document types is permitted to answer it, which scheme it's about, which passages get retrieved, and whether there's enough evidence to answer at all: every one of those is a rule or a score computed in code. The model's only job is the last, narrow step — turning the passages that code already selected into a naturally worded answer.
Citations work the same way. The model never sees a real page number — it
only refers to a retrieved passage by a numbered index, like [1] or
[2]. The application maps those indices back to the actual page numbers
after the model finishes generating. A fabricated citation isn't caught
and filtered afterward — it's structurally impossible for the model to
produce one, since it never has a real page number to invent in the first
place.
Measured on a real question against the live index: classification, routing, and retrieval together completed in under half a second. Generation — the only step that touches the model — took the remaining time. The deterministic backbone is fast and cheap; the one genuinely slow step is exactly the one narrow task the model was scoped to.
Tech Stack
Python, FastAPI, Streamlit, PyMuPDF for PDF extraction, rank-bm25 and NumPy
for hybrid retrieval. Ollama running locally for both embeddings
(nomic-embed-text) and generation (gemma4:12b) — no external API
dependency anywhere in the pipeline.
Business Impact
This closes a real-time knowledge gap that shows up in almost every client meeting. Instead of parking a question — "let me check and get back to you" — an RM can pull out a phone or laptop, ask in plain language, and get a precise, cited answer from any fund's own documents on the spot, in the room, while the client is still asking.
That immediacy is the actual business case. An RM no longer has to rely on a product specialist or the AMC's own team to confirm a number — the answer, with its exact source, is already in hand. There's no ambiguity left for the client to sit with and no delay for a competing advisor to fill. A question answered live, with proof, is a conversation that moves toward a decision instead of stalling on "we'll come back to this later" — and every one of those stalls is a real chance at losing the client's attention, if not the client.
Lessons Learned
A single word-boundary bug shipped three times before systematic testing caught it. A regex pattern written to match "exit load" didn't match "exit loads" — the plural defeated a word-boundary check. The same class of mistake, written independently at three different points during development, was only caught by building a test set with natural plural phrasing and running it against everything at once.
A local model's internal reasoning trace can silently consume its entire output budget. One model returned a separate "thinking" field alongside its visible answer — and with a long system prompt, occasionally spent its whole token allowance reasoning and never wrote a visible answer at all. No error, no warning — just an empty response. Diagnosed by inspecting every field the model actually returned, not just the one the code was reading for.
Retrieval can correctly find the right answer and still miss the cutoff. A passage containing an exact, correctly extracted figure ranked sixth in one real query; the system retrieves the top five. Not a chunking or embedding failure — a close scoring race. Logged as a known limitation rather than quietly patched, because the honest fix is a measured evaluation of how often this happens, not a guess at a new cutoff number.
Two local models, given identical retrieved context, disagreed in a way that revealed a genuine trade-off rather than a clear winner. One model was more concise and more explicit about cross-document disagreement; the other attempted more answers but occasionally hedged with soft, interpretive language on uncertain questions. Neither dominated on the small sample tested — which is itself the finding: model choice here belongs to a measured evaluation, not a handful of manual comparisons.
The App, In Action
Real screenshots from the running system, answering real questions against a real fund's actual regulatory documents.
