Learning path · Level 6 · Tutorial 1

How Retrieval-Augmented Generation Uses Documents

Explain how an assistant retrieves document passages to support an answer, and distinguish this process from retraining a model or simply pasting more text into a conversation.

  • Level: Intermediate
  • Duration: 13 min
  • 8 steps
Start tutorial Back to the path
How Retrieval-Augmented Generation Uses Documents

What you will go through

  1. What Documents Add to an Answer See how selected passages, rather than an entire document collection, can support an assistant’s answer. 1 min
  2. Chunks Keep—or Lose—Meaning Learn how separately retrievable document passages can preserve—or separate—the conditions needed to interpret a rule. 2 min
  3. Embeddings Compare Meaning Learn how embeddings represent questions and document chunks in a compatible way so a system can compare meaning beyond exact keyword matches. 2 min
  4. Relevant Is Not the Same as Sufficient Learn why semantic relevance is not enough: a retrieved passage must also apply to the right resource and include the condition needed to support an answer. 2 min
  5. Follow the Question Through RAG Trace how prepared document passages become retrieved context and support an answer at question time. 2 min
  6. Retrieval, Pasted Context, or Retraining? Compare retrieval-augmented generation (RAG), manually pasted context, and retraining by following how document information reaches the model. 2 min
  7. Where a Document-Grounded Answer Can Fail Inspect visible collection, retrieval, and answer stages to distinguish a retrieval miss, unsupported generation, and evidence absent from the collection. 2 min
  8. Trace and Explain a Document-Grounded Answer Trace a Cedar Studio booking question through chunks, embeddings, semantic search, retrieved context, and a supported answer. 2 min

What you will learn

  • Trace a question through document chunks, semantic search, retrieved context, and answer generation.
  • Explain the roles of chunks and embeddings using a plain-language example.
  • Distinguish retrieval-augmented generation from model retraining and direct-context prompting.
  • Identify why a relevant-looking retrieved passage may still be insufficient evidence.

Before you start

  • How Language Models Generate Text
  • Researching with Verifiable Sources

Step 1 of 8

What Documents Add to an Answer

See how selected passages, rather than an entire document collection, can support an assistant’s answer.

From documents to usable context

A collection is larger than an answer’s evidence

retrieval-augmented generation (RAG) helps an assistant answer with selected document passages.

A document collection can contain many guides. For one question, the assistant is given only selected passages as retrieved context. It then uses that context while generating its answer.

So, access to a collection does not mean every document—or every fact in every document—is available in the answer.

A left-to-right diagram showing a large document collection, one selected Room Booking Guide passage as retrieved context, and a generated answer.

Only selected passages move from the document collection into retrieved context for this answer.

Evidence must fit the question

Use the rule that actually applies

For Cedar Studio, the question needs the rule for practice rooms. A general-sounding answer is not enough; the answer needs the documented rule that applies to that resource.

A passage from the right guide can support the conclusion. A passage about another resource cannot establish the practice-room rule, even if it also mentions 3 hours.

Example

Worked Cedar Studio example

Question: What is the rule for a member who wants a 3-hour practice room booking?

Selected passage from “Room Booking Guide”: Members may book practice rooms for up to 2 hours without staff approval. Bookings longer than 2 hours require staff approval.

Generated answer: A 3-hour practice room booking requires staff approval.

The passage supports that requirement. It does not guarantee that approval will be granted or that a room is available.

Tip

Keep resource rules separate

“Equipment Guide” says members may borrow projectors for up to 3 hours. That rule is about projectors, not practice rooms. A collection may contain related material without containing support for the specific conclusion you need.

Check your understanding

Identify the three roles

Match each description to its role in the Cedar Studio example.

Tap an item and then its matching pair.

Step 2 of 8

Chunks Keep—or Lose—Meaning

Learn how separately retrievable document passages can preserve—or separate—the conditions needed to interpret a rule.

Passages can be retrieved separately

A chunk is a retrievable passage

A chunk is a document passage small enough to retrieve separately. One document can provide many chunks, so a retrieved chunk is not necessarily a whole document—or even a whole page.

For Cedar Studio, the assistant needs a passage from the "Room Booking Guide" that keeps the booking rule connected to the condition that changes it. A passage that sounds relevant but leaves out that condition can mislead the answer.

A boundary can separate related meaning

A comparison between an intact document passage containing two connected rule statements and the same statements divided into separate passages.

A chunk boundary can leave a condition outside the passage that was retrieved.

A split can remove the needed condition

Example

Keep the rule connected

Consider this complete passage from the "Room Booking Guide":

Members may book practice rooms for up to 2 hours without staff approval. Bookings longer than 2 hours require staff approval.

As one chunk, the passage connects the limit, the exception, and the required action. It can support the conclusion for a member requesting a 3-hour practice room booking: staff approval is required. It does not guarantee that approval or room availability.

What can be lost at the boundary

Now imagine a boundary divides the two sentences. A retrieved chunk containing only the first sentence tells you what happens up to 2 hours, but it does not state the rule for a longer booking.

Shorter is not automatically better or worse. What matters here is whether the separately retrievable passage retains the meaning needed for the question. If related conditions are split apart, the system may need to retrieve multiple chunks to recover the full rule.

Tip

A useful check

Ask: “If this passage were retrieved by itself, would it still include the condition needed to interpret the rule?”

Check the idea

Quick check

Which description best fits a chunk in a document collection?

Choose the best answer.

Apply the chunk boundary test

Choose a grouping and its reasoning

For the question, “What is the rule for a member who wants a 3-hour practice room booking?”, which grouping best preserves the booking rule and its approval condition within one retrievable passage?

Select the grouping with the strongest retained meaning.

Step 3 of 8

Embeddings Compare Meaning

Learn how embeddings represent questions and document chunks in a compatible way so a system can compare meaning beyond exact keyword matches.

Embeddings make comparison possible

A representation for comparison

An embedding is a numerical representation of text that a system can use to compare aspects of meaning. You do not need to read the numbers yourself.

A question and a document chunk can be turned into compatible embeddings. The system can then compare those representations to find text that appears meaningfully related, even when the wording is not identical.

In retrieval-augmented generation (RAG), this comparison can help connect a question to useful document passages.

A conceptual view of similarity

A simplified map with a question point and a document-passage point close together, while unrelated points are farther away.

Illustrative similarity map: nearby points represent text that the system considers more similar in meaning.

Tip

Keep the metaphor limited

Treat the map as a metaphor. Real embeddings are not readable summaries, and they are not literally arranged on a flat page.

A plain-language Cedar Studio example

Example

Different words, related meaning

Consider the central question:

What is the rule for a member who wants a 3-hour practice room booking?

A chunk from the "Room Booking Guide" says that members may book practice rooms for up to 2 hours without staff approval, while bookings longer than 2 hours require staff approval.

The question says “wants a 3-hour practice room booking.” The chunk says “book practice rooms” and “bookings longer than 2 hours.” The wording is not an exact copy, but both texts concern the same kind of request and duration rule. Compatible embeddings can capture enough of that relationship for comparison.

Similarity is not proof

Embedding proximity means the representations appear similar for the comparison task. It does not prove that a passage is true, authoritative, complete, or the right basis for a final answer.

Think of proximity as a clue about related meaning—not a verdict about correctness.

Apply the distinction

Check your understanding

True or false: A question and a passage can have similar embeddings even when their wording differs; however, their proximity alone does not prove the passage is correct.

Step 4 of 8

Relevant Is Not the Same as Sufficient

Learn why semantic relevance is not enough: a retrieved passage must also apply to the right resource and include the condition needed to support an answer.

Similarity does not prove support

A rank is only a starting point

Semantic search can rank chunks as candidates for a question by meaning-related similarity. In this simplified example, the ranking is a useful shortlist—not proof that a passage is true, authoritative, complete, or applicable.

For Cedar Studio’s question, a passage about a 3-hour projector loan may resemble a 3-hour practice room booking because both involve a member, a duration, and access to a resource. But projectors and practice rooms are different resources. Similar wording does not make the projector rule evidence for a room-booking answer.

Similarity can surface the wrong evidence

An illustrative similarity map ranks a projector passage, a short practice-room passage, and a longer-booking approval passage as candidates for a practice-room question.

Illustrative similarity map: semantic similarity can rank a passage highly even when it addresses the wrong resource or lacks a needed condition.

Evaluate the candidate, not just its rank

Check scope and condition

To support the requested conclusion, inspect more than rank. Ask:

  1. Does the passage apply to the same resource: a practice room?
  2. Does it cover the condition that matters: a booking longer than 2 hours?
  3. Does it establish the requested conclusion: staff approval is required?

A candidate can be on-topic yet still fail one of these checks. In particular, a rule for bookings up to 2 hours does not tell you what happens for a 3-hour booking.

Candidate passages compared

The ranks below are illustrative. A lower-ranked candidate can provide stronger support than a higher-ranked one.

Rank measures semantic similarity in this simplified example; it does not decide whether a passage is sufficient evidence.
RankCandidate passageSupport checkSufficient?
Rank 1“Equipment Guide”: Members may borrow projectors for up to 3 hours.Wrong resource: projectors are not practice rooms.No
Rank 2“Room Booking Guide”: Members may book practice rooms for up to 2 hours without staff approval.Right resource, but it omits the rule for bookings longer than 2 hours.No
Rank 3“Room Booking Guide”: Practice room bookings longer than 2 hours require staff approval.Covers the relevant duration condition and staff approval.Yes

Apply the evidence check

Choose sufficient evidence

For the question “What is the rule for a member who wants a 3-hour practice room booking?”, which assessment is correct?

Step 5 of 8

Follow the Question Through RAG

Trace how prepared document passages become retrieved context and support an answer at question time.

Prepare the collection once

Document preparation makes passages searchable

A representative retrieval-augmented generation (RAG) flow has two phases. During Document preparation, the system divides the Document collection into retrievable Chunks and creates Passage embeddings for those chunks. This makes passages available for later search.

For Cedar Studio, the "Room Booking Guide" can provide chunks containing the practice-room duration rule and its staff-approval condition. Preparation is done ahead of questions; it is not restarted from scratch every time a member asks about a booking.

The two-phase RAG flow

A left-to-right diagram showing Document preparation: Document collection, Chunks, and Passage embeddings; then Question time: Question embedding, Semantic search, Retrieved context, and Generated answer.

Document preparation makes chunks searchable; Question time selects passage text and uses it to generate an answer.

Question time: search, select, generate

What happens when a question arrives

At Question time, the system represents the new question as a Question embedding. Semantic search compares that representation with the available Passage embeddings and ranks candidate chunks by meaning-related similarity.

The system then selects passages to form Retrieved context. For generation, the model receives the question, applicable instructions, and the selected passage text. It does not receive only embedding numbers. In this simplified flow, chunks that were not selected are not automatically available to the model for this answer.

Retrieval selects material; generation writes an answer from the supplied context. They are separate operations.

Tip

Keep the stages separate

A RAG flow that uses embeddings is common, but it is not the only possible implementation. The key distinction here is the sequence: prepared passages can be searched at question time, selected passage text becomes retrieved context, and then a model generates an answer using that context.

Trace one question through the flow

Example

A complete Cedar Studio trace

Question: What is the rule for a member who wants a 3-hour practice room booking?

  1. The question becomes a Question embedding.
  2. Semantic search compares it with the prepared Passage embeddings and ranks candidate chunks.
  3. The system selects these passages as Retrieved context:
  • Rank 1, “Room Booking Guide”: “Bookings longer than 2 hours require staff approval.”
  • Rank 2, “Room Booking Guide”: “Members may book practice rooms for up to 2 hours without staff approval.”
  1. The model receives the question, instructions, and those selected passage texts. It generates: A 3-hour practice room booking requires staff approval.

A projector passage from "Equipment Guide" is not selected and is not automatically supplied to the model. The answer does not guarantee staff approval or room availability.

Separate preparation from a new question

Follow the route from question to answer

The motion emphasizes selection before generation.

Check the two phases

Which action belongs to Document preparation, rather than being repeated for each new question?

Preparation or Question time?

Choose the preparation activity.

Apply the full Question time sequence

Sequence the Question time flow

Put the Question time stages in order for the Cedar Studio booking question.

  1. Represent the question as a Question embedding.
  2. Place the question, instructions, and selected passage text in the model's context.
  3. Select relevant chunks as Retrieved context.
  4. Use Semantic search to rank candidate chunks against the prepared Passage embeddings.
  5. Generate an answer from the supplied context.

Step 6 of 8

Retrieval, Pasted Context, or Retraining?

Compare retrieval-augmented generation (RAG), manually pasted context, and retraining by following how document information reaches the model.

Compare the routes

Three different routes

Three routes for document information

The same rule can influence an answer in different ways. The key question is how the information reaches the model.

  • In retrieval-augmented generation (RAG), the system automatically selects passages from a document collection at question time, then supplies those passages as retrieved context.
  • In direct-context prompting, a person manually puts selected document text into the conversation. The model can use that text, but no retrieval step selected it.
  • In retraining, a training process changes learned model parameters. It is not a question-time lookup of a passage.

All three can support useful answers, but they are different mechanisms.

How information reaches the model

A three-lane comparison: automatic document search supplies selected passages in RAG; a person inserts text into a conversation in direct-context prompting; training examples alter an abstract model in retraining.

RAG selects passage text automatically at question time. Direct-context prompting receives text chosen manually. Retraining changes the model rather than selecting a passage for this question.

Use the mechanism, not the label

What changes in each approach

Use these distinctions when you hear that an assistant “uses documents.”

The mechanism—not merely the presence of document text—determines the classification.
ApproachHow document information arrivesAt question timeChanges learned parameters?
RAGAutomatic selection from a collectionSelected passages become retrieved contextNo
Direct-context promptingA person inserts chosen text into the conversationInserted text is already in the promptNo
RetrainingA training process adjusts the modelThe answer can draw on changed parametersYes

Tip

Important distinction

These mechanisms are not mutually exclusive. For example, a system can use a retrained model and also use RAG for a particular question. But pasting a longer excerpt into a conversation, by itself, does not turn that interaction into RAG.

Practice: identify the route

Classify the scenarios

Match each scenario to its document-use mechanism.

Tap an item and then its matching pair.

Step 7 of 8

Where a Document-Grounded Answer Can Fail

Inspect visible collection, retrieval, and answer stages to distinguish a retrieval miss, unsupported generation, and evidence absent from the collection.

Locate the failure layer

Make the stages visible

A wrong answer alone does not reveal where a document-grounded process failed. Compare three visible layers:

  1. Document collection — what the available sources contain.
  2. Retrieved context — what was selected for this question.
  3. Generated answer — what the model says.

A retrieval miss occurs when needed evidence exists in the collection but is absent from retrieved context. Unsupported generation occurs when needed evidence is present in retrieved context, but the answer changes it or adds a claim it does not support. Evidence absent from the collection means the visible sources never state the fact the question requires.

These issues can occur separately or together. When a case shows more than one issue, name all that apply rather than forcing one label.

Compare the visible layers

A left-to-right diagram showing a needed rule present in the document collection but missing from the retrieved context before a cautious answer is generated.

Compare the collection with retrieved context before deciding whether retrieval or generation is the visible problem.

Example

Example: needed evidence was missed

Question: What is the rule for a member who wants a 3-hour practice room booking?

Document collection: “Room Booking Guide” states: “Members may book practice rooms for up to 2 hours without staff approval. Bookings longer than 2 hours require staff approval.”

Retrieved context: “Members may book practice rooms for up to 2 hours without staff approval.”

Generated answer: “The supplied passage does not state the rule for a 3-hour practice room booking.”

The needed approval rule exists in the collection, but it was not selected. This is a retrieval miss. The answer is cautious; it does not create an unsupported claim.

Separate absence from misuse

Inspect what the answer does with evidence

Keep the layers distinct. A visible passage can be enough for the requested conclusion, yet the generated answer can still go beyond it. Conversely, a system can correctly acknowledge that the collection lacks a requested fact.

The examples below intentionally isolate one classification each. In a real RAG case, more than one failure can be visible at once, so an evaluator should identify each applicable issue.

Example

Example: retrieved evidence was distorted

Question: What is the rule for a member who wants a 3-hour practice room booking?

Retrieved context from “Room Booking Guide”: “Members may book practice rooms for up to 2 hours without staff approval. Bookings longer than 2 hours require staff approval.”

Generated answer: “Staff approval is guaranteed for a 3-hour practice room booking.”

The retrieved passage supports that staff approval is required. It does not say approval will be granted. Retrieval supplied the needed rule, but the answer distorted it. This is unsupported generation.

Example

Example: the collection lacks the requested fact

Question: Will a practice room definitely be available for a member’s 3-hour booking?

Document collection: “Room Booking Guide” states the booking-duration and staff-approval rules. “Equipment Guide” states that members may borrow projectors for up to 3 hours.

Generated answer: “The visible collection does not state whether a practice room will be available.”

No visible source states an availability rule. This is evidence absent from the collection. The cautious answer accurately identifies the gap instead of inventing an availability claim.

Practice the distinction

Classify the isolated cases

Match each transparent case to its most direct classification.

Tap an item and then its matching pair.

Apply the full check

Trace three transparent cases

For each case, name the classification and point to the visible information that supports it. Use this format: A — classification; evidence: ...

A. The collection includes “Bookings longer than 2 hours require staff approval” in “Room Booking Guide.” Retrieved context contains only “Members may book practice rooms for up to 2 hours without staff approval.” The answer says, “The supplied passage does not state the rule for a 3-hour practice room booking.”

B. Retrieved context includes “Bookings longer than 2 hours require staff approval.” The answer says, “Staff approval is guaranteed for a 3-hour practice room booking.”

C. The visible collection contains the practice-room duration rules and the projector borrowing rule, but no statement about practice-room availability. Asked whether a room will definitely be available, the answer says, “The visible collection does not state whether a practice room will be available.”

Write at least 180 characters (0/180).

Step 8 of 8

Trace and Explain a Document-Grounded Answer

Trace a Cedar Studio booking question through chunks, embeddings, semantic search, retrieved context, and a supported answer.

Inspect the miniature case

A small collection, not one undivided document

Use this complete miniature collection for one question.

“Room Booking Guide”

  • Chunk A: Members may book practice rooms for up to 2 hours without staff approval.
  • Chunk B: Bookings longer than 2 hours require staff approval.

“Equipment Guide”

  • Chunk C: Members may borrow projectors for up to 3 hours.

A chunk is separately retrievable. The collection contains all three chunks, but only selected chunks become retrieved context for a particular question.

The Cedar Studio RAG trace

A left-to-right diagram showing document preparation and question time, with room-rule chunks selected as retrieved context while a projector chunk is not selected.

The retrieved context contains selected passage text, not every chunk in the Document collection.

Example

Visible selection for the question

Question: What is the rule for a member who wants a 3-hour practice room booking?

Retrieved context

  • Rank 1: “Room Booking Guide” — Chunk B: Bookings longer than 2 hours require staff approval.
  • Rank 2: “Room Booking Guide” — Chunk A: Members may book practice rooms for up to 2 hours without staff approval.

Not selected: “Equipment Guide” — Chunk C concerns projectors, not practice rooms. Its 3-hour limit cannot answer the booking question.

Connect selection to a supported answer

Interpret the stages

During Document preparation, the system separates the Document collection into chunks and represents each chunk with a passage embedding. At Question time, it represents the question with a question embedding and uses semantic search to rank chunks by similarity.

The selected passage text—not embedding numbers—is supplied as retrieved context. Here, Chunk B is the evidence needed for the approval condition. Chunk A adds the contrast with bookings of up to 2 hours. A chunk boundary matters because separating a condition from its rule can leave a relevant passage insufficient to support an answer.

Example

Check the evidence and mechanism

The supported answer is: A 3-hour practice room booking requires staff approval.

This does not mean approval is guaranteed, and it does not say a room is available.

This is retrieval-augmented generation (RAG) because the system selected chunks from a collection at question time. If a person manually copied chosen passages into a conversation, that would be direct-context prompting. Neither action changes learned model parameters; changing those parameters would be retraining.

Summary

Key checks

  • Chunks are separately retrievable passages, so their boundaries can preserve or remove needed conditions.
  • Embeddings support similarity comparison; retrieved context contains the selected passage text used by the generator.
  • A high-ranked passage must still apply to the requested resource and include the needed rule.
  • RAG selects context from a collection at question time. Manual pasting is direct-context prompting, while retraining changes model parameters.

Apply the end-to-end sequence

Sequence the flow

Put the simplified Cedar Studio RAG flow in order.

  1. Split the Document collection into chunks.
  2. Use Semantic search to rank candidate chunks.
  3. Produce a Generated answer from the question and Retrieved context.
  4. Create a Question embedding for the booking question.
  5. Create Passage embeddings for the chunks.
  6. Place selected passage text in Retrieved context.

Final application and recap

Explain the document-grounded answer

Write 4–6 sentences about the Cedar Studio case. Explain what chunks do and what embeddings do, then trace the question through semantic search, retrieved context, and answer generation. Identify the exact “Room Booking Guide” evidence needed for the 3-hour practice room conclusion, including what the conclusion does not guarantee. Finally, explain why this is RAG rather than retraining or direct-context prompting.

Write at least 350 characters (0/350).

Tutorial complete

Congratulations! You completed: How Retrieval-Augmented Generation Uses Documents

You have completed the tutorial. You can now trace how document passages become retrieved context, check whether that context supports an answer, and distinguish RAG from manual context and retraining.

Download the App now to have access to + 5000 free courses, exercises, certificates and lots of content without paying anything!

  • 100% free online courses from start to finish

    Thousands of online courses in video, ebooks and audiobooks.

  • More than 60 thousand free exercises

    To test your knowledge during online courses

  • Valid free Digital Certificate with QR Code

    Generated directly from your cell phone's photo gallery and sent to your email

Cursa app on the ebook screen, the video course screen and the course exercises screen, plus the course completion certificate