RAGBuilder

The problem

Most RAG projects treat the knowledge base as a by-product: a script chunks some PDFs, pushes vectors somewhere, and nobody can say afterwards which version of which paragraph an answer came from. In a regulated domain such as insurance regimes, contracts or compliance rules, that is not acceptable.

RAGBuilder treats retrieval knowledge as a versioned data product. Each collection (one set of documents, e.g. one insurance regime) gets its own chunk table in PostgreSQL + pgvector, with history, audit trail and evaluation built in.

What it does

• Sources & ingestion: folder status, chunk previews without writing, manifest editing (dates, levels, tags), file replacement, new sources by upload.
• Versions: every ingestion builds a version; activate it, restore an archived one, compare.
• Chunks: filter current / historical / building rows, edit text, dates, security level and tags, bulk-tag, see lineage and audit trail per chunk.
• Tags: a global vocabulary with keywords, optionally overridden per collection; used both to tag chunks (keywords + LLM) and to tag incoming questions.
• Playground: search or ask within a product, compare the four retrieval modes side by side, preview the exact prompt, see model, tokens and latency.
• Evaluation: golden questions, hit@k, MRR, per-question ranks; runs are stored with model and versions so two runs can be compared honestly.
• Audit log: every change, with author and comment.

How it is built

The core idea is the profile: a plain folder (profiles/<name>/) holding everything specific to a family of collections: profile.yaml (chunking, schema, retrieval settings), models.yaml, ingestion hooks in Python, ranking queries in SQL (-- [vector], -- [text bm25]…, fused in Python), a post-processing hook, the table DDL with row-level security, and the prompt block template.

The engine (reading, chunking, tagging, ingestion, search, answers, model clients) is self-contained: it is copied as-is into downloadable kits and exported apps, so what you evaluate is exactly what runs in production.

Models come from Azure AI Foundry or any OpenAI-compatible API (OpenAI, Ollama, vLLM, Mistral…). Vectors are stored per provider:model:dimension, so embeddings from different models are never silently mixed.

Why it matters

Retrieval quality is a data-quality problem. Making chunks editable, versioned and measurable turns RAG from a demo into something a business owner can sign off on.