Private RAG Knowledge System
On-premise retrieval-augmented generation system for a professional services firm handling sensitive client documents.
The problem
A professional services firm needed internal AI search across its client documents but couldn't use cloud AI tools due to data sensitivity and compliance constraints.
The system
Deployed a private RAG stack: Ollama serving a local model, LangChain for document chunking and retrieval, PostgreSQL with pgvector for embeddings, and a React interface for staff queries. All running on the firm's existing server infrastructure.
Architecture
- Private DocumentsInternal client files
- Document ProcessingChunking and ingestion via LangChain
- Embedding & IndexingVector embeddings generated locally
- Vector StoragePostgreSQL + pgvector
- RetrievalRelevant passages selected for the query
- Local Model ServingOllama
- Grounded ResponseAnswer assembled from retrieved passages
- Staff InterfaceReact query UI
Designed for private infrastructure: the retrieval and generation path does not depend on an external model provider.
Key technical decisions
Assessed infrastructure constraints. Designed a private architecture using local models and vector storage. Chose Ollama for model serving and pgvector for embeddings storage.
What this demonstrates
Building retrieval-augmented AI under strict data-sensitivity constraints, with model serving, vector storage and the staff interface all inside the client's own infrastructure.
Technology
Other client work.
A similar problem — or a clear requirement?
Facing a similar business problem? Start with Diagnose — technology isn't presumed. Already know what you need? Email the requirement and discuss scope.