RAG Assistant for Law Firms with Page-Level Citations
An SDH in-house prototype: a legal back office with an AI assistant that answers only from case documents and cites the page.
Legal RAG Assistant answers from case documents only and shows the page behind every statement.
Partner's Profile
Background
Law firms, in-house legal departments and notaries need to find facts across case materials, and every answer must hold up against the primary source: an invented date or amount is unacceptable. Many of them also cannot send client documents to external clouds. SDH built this in-house R&D prototype to work out how a legal AI assistant should behave when correctness and control over data come first. There is no client behind it; it is our own engineering reference.
Objective
Build a working case-management back office with an AI assistant that answers only from the firm's own documents and shows where each statement comes from. The same retrieval had to power a client-facing chat limited to what a client may see. The architecture had to support both cloud and local models, so a firm can decide where its documents are processed, and it had to say plainly when the documents contain no answer.
Solution
As an R&D project of our AI software development team, we built a legal back office covering clients, matters, documents, drafts, search, analytics and audit, with an AI assistant on top. Documents in PDF, DOCX and TXT are processed in a Celery queue: split into overlapping fragments of 1,200 characters with a 200-character overlap, tagged with page numbers and indexed in Qdrant. Staff get a search page and an assistant filtered by matter. Clients get a public chat that searches only documents marked public plus their own uploads. Contract analysis, version comparison, draft generation, timelines and streaming answers complete the set. As in any custom RAG application, the value is in retrieval and verification rather than in the model.
Search is hybrid: vector and full-text branches are merged with Reciprocal Rank Fusion (k=60). Qdrant returns full-text hits unranked, so we first rank them by how many query words they cover. Results can be filtered by matter, document type, status and date added.
Verifiability is built into the answer flow:
- fragments below a relevance threshold are dropped;
- if nothing passes, the answer is “Not found in the documents”, and the model is not called at all;
- the model must put a citation [n] after every statement;
- only sources that are actually cited are shown, with document, page and a confidence level.
A single facade covers LLMs and embeddings: OpenAI or local Ollama models (bge-m3 embeddings, qwen2.5 7B), switched by one setting. The default configuration uses OpenAI; the architecture can run fully on-premise with local models. Switching the embedding provider changes vector dimensions, so the collection is recreated and fully re-indexed. For Cyrillic texts, a character-based token estimate under-counted tokens almost twofold and broke the embeddings request limit; exact counting with tiktoken fixed it. One honest limitation: nothing automatically checks that a cited fragment supports the statement. The citation UI exists so the lawyer can verify it.
Technologies
Backend
Python 3.11, Django, Celery, uWSGI
Data & Search
PostgreSQL 14, Redis 7, Qdrant
AI / ML
OpenAI API, Ollama (bge-m3, qwen2.5), tiktoken
Projects
RAG Assistant for Law Firms with Page-Level Citations
An SDH in-house prototype: a legal back office with an AI assistant that answers only from case documents and cites the page.
Need a project estimate?
Drop us a line, and we provide you with a qualified consultation.