Skip to content
Nubent

/Glossary · AI

What is RAG?

RAG (Retrieval-Augmented Generation) is a technique that improves a language model's answers by retrieving relevant information from your own sources and passing it into the prompt as context. The model can then answer with current, citable data without being retrained.

How RAG works

First, documents are processed: they are split into chunks, converted into vectors with an embedding model and stored in a vector database. When a question comes in, it is also converted into a vector, the most similar chunks are retrieved, and they are sent to the model along with the query so it can generate the answer.

On AWS, Amazon Bedrock Knowledge Bases automates that pipeline: it connects sources like S3, Confluence or SharePoint, generates the embeddings and stores the vectors in OpenSearch Serverless, Aurora PostgreSQL with pgvector, Amazon S3 Vectors or other supported engines. Answers can include citations to the source documents.

When RAG makes sense

RAG fits best when the model needs private knowledge or knowledge that changes often: internal manuals, policies, contracts, support tickets or product documentation. It also fits when you need traceability, because every answer can cite its source and users can check it.

If what you want is to change the style or format of the answers, fine-tuning or a good prompt usually works better. If the data is structured, like sales or inventory, it often makes more sense for the model to generate SQL queries instead of searching for similar text.

Best practices and common mistakes

RAG quality depends more on retrieval than on the model. Test different chunking strategies, combine semantic and lexical search such as BM25, rerank results, and filter by metadata like department or date. Measure retrieval relevance and answer faithfulness against your own evaluation set.

Common mistakes: indexing documents without cleaning them, ignoring permissions and exposing information to users who should not see it, and not refreshing the index when sources change. Another one is sending too many chunks, which makes every call more expensive and less accurate.

/FAQ

Questions about RAG

Another question? Write to hello@nubent.com and an engineer will answer.

What is the difference between RAG and fine-tuning?

RAG adds knowledge at query time without changing the model, and you update it by simply reindexing documents. Fine-tuning adjusts the model's weights to change its behavior or style. For knowledge that changes often, RAG is usually simpler and cheaper.

Which vector database should I use on AWS?

It depends on volume and on what you already run. OpenSearch Serverless combines vector and lexical search, Aurora PostgreSQL with pgvector builds on an existing relational database, and Amazon S3 Vectors lowers cost for large volumes with looser latency requirements.

Does RAG eliminate hallucinations?

It reduces them a lot, but it does not eliminate them. If retrieval returns irrelevant or incomplete chunks, the model can still make things up. That is why you should require citations, instruct the model to answer only from the context, use Guardrails contextual grounding checks and evaluate answers continuously.

Next step

What should we build next on AWS?

30 minutes with an engineer. Concrete next steps, whether or not you work with us.