Blueprint for an Enterprise AI Assistant
An enterprise assistant has a different job from a consumer chatbot: it must answer from the company’s own documents, refuse questions it shouldn’t touch, and never leak data on the way through. This blueprint meets those constraints with a fully self-contained, open-source Retrieval-Augmented Generation (RAG) stack — no data leaves the environment, and every step is auditable.
The stack
Four parts, each replaceable with a comparable open model:
| Role | Example model / library | What it does |
|---|---|---|
| Generation | google/flan-t5-base |
A sequence-to-sequence model that writes the answer from a supplied prompt. |
| Embedding | sentence-transformers/all-MiniLM-L6-v2 |
Turns document chunks and queries into vectors for similarity comparison. |
| Vector search | FAISS | Stores the document vectors and returns the closest matches to a query fast. |
| Guardrails | Custom regex / policy rules | Blocks queries that violate policy and redacts PII before anything reaches the model. |
How it works
Build the index once. Load the generation and embedding models. Split the internal corpus — policies, runbooks, SOPs — into small overlapping chunks so retrieval lands on the right passage instead of a whole document. Embed every chunk and store the vectors in the FAISS index.
Then, per query, run the pipeline:
- Screen. Check the incoming query against policy rules. If it tries to exfiltrate data or otherwise breaches policy, reject it immediately with a reason — before any retrieval or generation happens.
- Retrieve. Embed the query and pull the k nearest chunks from the index. That retrieved text becomes the answer’s grounding context.
- Augment. Assemble a structured prompt: system instructions (“answer strictly from the provided context”), the retrieved chunks, the user’s question with PII redacted, and formatting rules (cite sources inline, stay concise).
- Generate. Pass the assembled prompt to the generator. Because it’s told to answer only from context, the response stays grounded and the redactions hold.
Why screen before retrieval
Ordering matters. Screening the query first means a disallowed request never touches the index or the model — cheaper, safer, and easier to audit than filtering the output after the fact. Redacting PII inside the prompt, rather than trusting the model to withhold it, keeps sensitive values out of the generation path entirely.
Evaluating it
Run representative queries and judge two things separately: whether retrieval surfaced the right context, and whether the generated answer stayed faithful to it. Most quality problems are retrieval problems — a wrong or missing chunk — not generation problems, so measure them independently.
The result is a scalable, auditable, fully self-hosted assistant: answers grounded in internal knowledge, policy enforced at the door, and no dependency on an external API.
