RAG
Retrieval-augmented generation
What it is
Retrieval-augmented generation answers a question by first finding the relevant passages in a customer's own content, then giving only those passages to a language model to write the answer. Content is split into chunks, turned into embeddings and stored in a vector index. Done well, answers cite their sources and stay inside what the customer's documents actually say.
Enterprise buyers want answers grounded in their data, not the model's general knowledge. They also need to know one customer's documents can never surface in another customer's answers.
What it takes
About 6–12 engineer-weeks to build in-house, or 3–6 using Pinecone, Vectara, LlamaIndex.
Answer these first
- Product: What questions should it answer well, and which should it refuse?
- Engineering: How does retrieval respect each user's permissions on the source documents?
- Engineering: How will we know when a change to chunking or embeddings made answers worse?
- Security: How are each tenant's indexes kept apart?
- Legal: Which model providers see customer content, and do they retain it?
- Sales: Do buyers need citations before they'll trust the answers?
This page works best with JavaScript on. Every answer also has its own address, like /what/scim.