/Glossary · AI
What is RAG?
RAG (Retrieval-Augmented Generation) is a technique that improves a language model's answers by retrieving relevant information from your own sources and passing it into the prompt as context. The model can then answer with current, citable data without being retrained.
How RAG works
First, documents are processed: they are split into chunks, converted into vectors with an embedding model and stored in a vector database. When a question comes in, it is also converted into a vector, the most similar chunks are retrieved, and they are sent to the model along with the query so it can generate the answer.
On AWS, Amazon Bedrock Knowledge Bases automates that pipeline: it connects sources like S3, Confluence or SharePoint, generates the embeddings and stores the vectors in OpenSearch Serverless, Aurora PostgreSQL with pgvector, Amazon S3 Vectors or other supported engines. Answers can include citations to the source documents.
When RAG makes sense
RAG fits best when the model needs private knowledge or knowledge that changes often: internal manuals, policies, contracts, support tickets or product documentation. It also fits when you need traceability, because every answer can cite its source and users can check it.
If what you want is to change the style or format of the answers, fine-tuning or a good prompt usually works better. If the data is structured, like sales or inventory, it often makes more sense for the model to generate SQL queries instead of searching for similar text.
Best practices and common mistakes
RAG quality depends more on retrieval than on the model. Test different chunking strategies, combine semantic and lexical search such as BM25, rerank results, and filter by metadata like department or date. Measure retrieval relevance and answer faithfulness against your own evaluation set.
Common mistakes: indexing documents without cleaning them, ignoring permissions and exposing information to users who should not see it, and not refreshing the index when sources change. Another one is sending too many chunks, which makes every call more expensive and less accurate.
/FAQ
Questions about RAG
Another question? Write to hello@nubent.com and an engineer will answer.
What is the difference between RAG and fine-tuning?
RAG adds knowledge at query time without changing the model, and you update it by simply reindexing documents. Fine-tuning adjusts the model's weights to change its behavior or style. For knowledge that changes often, RAG is usually simpler and cheaper.
Which vector database should I use on AWS?
It depends on volume and on what you already run. OpenSearch Serverless combines vector and lexical search, Aurora PostgreSQL with pgvector builds on an existing relational database, and Amazon S3 Vectors lowers cost for large volumes with looser latency requirements.
Does RAG eliminate hallucinations?
It reduces them a lot, but it does not eliminate them. If retrieval returns irrelevant or incomplete chunks, the model can still make things up. That is why you should require citations, instruct the model to answer only from the context, use Guardrails contextual grounding checks and evaluate answers continuously.
How we help with RAG
Generative AI on AWS
Generative AI on AWS with Amazon Bedrock: assistants, RAG over your documents and GenAI features in your product, with evaluations, guardrails and cost control.
Learn moreAI-native on AWS
Products and platforms with generative AI at the core: Amazon Bedrock, agents, RAG, evaluation, guardrails, LLMOps and inference costs under control.
Learn moreData on AWS
A data platform on AWS: automated ingestion, an S3 data lake, Glue transformations, Athena and Redshift for queries, and Lake Formation for governance.
Learn moreClaude on Amazon Bedrock
Claude model integration through Amazon Bedrock: model selection, prompt caching, tool use, guardrails, evaluation and quotas, with your data staying in your AWS account.
Learn moreNext step
What should we build next on AWS?
30 minutes with an engineer. Concrete next steps, whether or not you work with us.