Learning path · Level 6 · Tutorial 4

Building an Evidence-Based Answer Test Set

Create a small, repeatable test set and a clear rubric for document-grounded answers. Evaluate evidence and meaning rather than relying on whether an answer sounds convincing or matches one preferred wording.

  • Level: Intermediate
  • Duration: 18 min
  • 8 steps
Start tutorial Back to the path
Building an Evidence-Based Answer Test Set

Step 1 of 8

Start with an Evidence-First Test Case

Create a test case from source-based expectations before evaluating an assistant answer.

Set expectations from the sources

An evidence-first test

Build expectations before you look at an answer

A test case links one question to four things:

  • the source-based expected evidence;
  • the separate required answer points;
  • concrete unacceptable claims; and
  • the answer you will later evaluate.

Start with the documents, not with a convincing-sounding response. This keeps the test focused on what the Cedar Learning Hub sources actually support.

A visual workflow moves from source passages to a test-case checklist, then to an assistant answer, and finally to an evidence comparison.

The workflow protects the evaluation from being shaped by the answer's wording.

Record the reference before the response

What the fields are for

Test meaning, not a preferred sentence

A required answer point is a meaning that must be present. It is not a script the assistant must repeat.

For example, an answer can use different wording and still pass if it preserves the supported limit. Conversely, polished wording does not pass if it adds a claim the source does not establish.

Write short, observable unacceptable claims. They tell a reviewer what would materially change the answer.

Example

Example: a compact test case

Question type: Direct lookup

Question: How far in advance may a registered member reserve a room?

Expected evidence: Room Booking Guide > Booking window — “Reservations may be made up to 7 days in advance.”

Required answer points:

  • States that a reservation may be made up to 7 days in advance.
  • Keeps the limit as a maximum, not a minimum or guarantee.

Unacceptable claims:

  • Says reservations can be made any time in the future.
  • Changes the limit to 2 hours.

An answer such as “A registered member can reserve a room up to 7 days ahead” preserves the required meaning. It need not copy the excerpt word for word.

Apply the evidence-first sequence

Practice: sequence the workflow

Put these actions in the order that keeps expectations independent of the assistant answer.

  1. Collect the assistant answer
  2. Inspect the labeled source passages
  3. Judge the answer against the source-based expectations
  4. Write expected evidence, required answer points, and unacceptable claims

Step 2 of 8

Choose a Representative Question Mix

Select four complementary questions that reflect the collection's intended uses and ordinary evidence limits.

Build a balanced starter set

Aim for variety, not volume

A small test set is useful when its questions cover different ways people will use the collection. Four easy questions about booking time may all look successful while leaving important behavior untested.

For Cedar Learning Hub, begin with four complementary purposes:

  • direct lookup
  • conditional application of a rule
  • synthesis across sources
  • a routine question containing a partly unsupported request

This is a bounded quality check for these intended uses—not a statistical claim about every possible question.

Four visually distinct question purposes: timing lookup, rule condition, cross-document synthesis, and an unanswered equipment question.

Each question type probes a different ordinary use or evidence limit in the collection.

Map questions to coverage

Four complementary candidate questions

Use these four questions as the starter mix. Do not write full expected evidence or scoring rules yet; first make sure the questions are meaningfully different.

Question-type and source-coverage matrix

The matrix reveals what each case is designed to exercise. A row should add a distinct purpose, source combination, or evidence limit—not just reword an existing question.

A small, deliberately varied set for Cedar Learning Hub.
CaseQuestion typeQuestion focusSource coverage
1Direct lookupHow far in advance can a registered member reserve a room?Room Booking Guide > Booking window
2Conditional ruleA registered member already has one active booking. Can they make another booking?Room Booking Guide > Booking limits
3SynthesisWhich room has both a display and a whiteboard, and what is the maximum duration for one booking?Room Facilities Guide > Equipment; Room Booking Guide > Booking limits
4Partly unsupported requestDoes Birch Room have a whiteboard and microphones?Room Facilities Guide > Equipment; microphone availability is Not stated in sources

Check the question purposes

Practice: classify the four questions

Match each candidate question to the testing purpose it represents.

Match each question to its question type.

Tap an item and then its matching pair.

Select a non-redundant set

Practice: spot redundancy and a missing purpose

Which proposed set is the representative four-case starter mix for Cedar Learning Hub?

Step 3 of 8

Write Expectations from the Sources

Read the evidence boundary

Source snapshot for this exercise

Build expectations before you look at an assistant answer. For this exercise, the complete source snapshot is below—no external files or tool access are needed.

Both sources are active, version 1.0, updated 2026-02-01. Treat the excerpts as the evidence boundary: do not add likely details from general knowledge. No passage in this snapshot addresses microphone availability.

Cedar Learning Hub source passages

Use each reference together with its short excerpt when you write expected evidence.

Active sources, version 1.0, updated 2026-02-01
Passage referenceShort source excerpt
Room Booking Guide > EligibilityRegistered members may book.
Room Booking Guide > Booking limitsEach member can hold one active booking. Each booking can last up to 2 hours.
Room Booking Guide > Booking windowReservations may be made up to 7 days in advance.
Room Facilities Guide > EquipmentCedar Room has a display and a whiteboard.
Room Facilities Guide > EquipmentBirch Room has a whiteboard but no display.

Specify evidence, meanings, and limits

Turn a question into observable expectations

A test case records the meaning an acceptable answer must preserve—not one preferred sentence.

For every question, fill the same fields:

  • Question type and Question identify what is being tested.
  • Expected evidence gives a passage reference plus the short rule, fact, or qualification it supplies.
  • Required answer points are separate bullets for the meanings an answer must include.
  • Unacceptable claims name specific statements that would overstate, invent, or change the source meaning.

A document title alone is too broad. Include the relevant section and excerpt, such as Room Booking Guide > Booking limits with the applicable rule.

Evidence can come from more than one source

A diagram showing two source passages feeding separate evidence points in one test case.

For a synthesis question, preserve each needed meaning and its own evidence.

Example

Worked test case: connect both passages

Question type: Synthesis across sources
Question: “I need a room with a display for a 2-hour session. Which room can I book?”

Expected evidence

  • Room Facilities Guide > Equipment — “Cedar Room has a display and a whiteboard.”
  • Room Booking Guide > Booking limits — “Each booking can last up to 2 hours.”

Required answer points

  • Cedar Room meets the display requirement.
  • A booking may last up to 2 hours.

Unacceptable claims

  • Birch Room has a display.
  • A booking may last longer than 2 hours.

This case can accept a concise paraphrase. It does not require these exact words, but it does require both meanings.

Separate evidence from silence

Keep an evidence gap visible

A partly unsupported request has two parts: what the sources establish and what they do not establish.

Do not turn a gap into a negative fact. If neither source mentions microphones, write Not stated in sources; that does not mean microphones are unavailable.

For the selected room-equipment question, expected evidence should retain the documented equipment passage and also record the microphone gap. Its required answer points should separately cover the supported equipment fact and the unresolved microphone information. Its unacceptable claims should include invented positive or negative microphone statements.

This structure lets an answer be useful where evidence exists while remaining precise about its limit.

Tip

Use the same boundary across the test case

A source qualification belongs in more than one place:

  • In Expected evidence, record the relevant passage and Not stated in sources where needed.
  • In Required answer points, require the response to distinguish the supported part from the gap.
  • In Unacceptable claims, name concrete invented details, including a confident negative claim based only on silence.

Do not write “the answer must use this exact sentence.” Evaluate whether the required meanings survive paraphrasing.

Apply the structure to all four questions

Draft the four-case starter set

In your own text document, draft the four-case Cedar Learning Hub starter set. Use these questions:

  1. “How far in advance may a registered member reserve a room?”
  2. “A registered member already has an active booking. Can they make another booking?”
  3. “I need a room with a display for a 2-hour session. Which room can I book?”
  4. “Does Birch Room have a whiteboard and microphones?”

For each test case, use exactly these labels: Question type, Question, Expected evidence, Required answer points, and Unacceptable claims.

Include passage references and short excerpts in expected evidence. Write required answer points as separate bullets. For the room-equipment question, preserve the source qualification and identify at least one unsupported claim. Paste your complete four-case draft here.

Write at least 500 characters (0/500).

Step 4 of 8

Define Observable Scoring Anchors

Apply a consistent 0–2 rubric for accuracy, grounding, coverage, and uncertainty.

Score dimensions ask different questions

Four independent checks

A rubric keeps evaluation focused on observable evidence, not on whether an answer sounds polished. Score each dimension separately using the same scale: 0 = Not met, 1 = Partly met, 2 = Met.

  • Accuracy: Is the answer factually correct?
  • Grounding: Do correctly identified passages support its substantive claims?
  • Coverage: Does it include every required answer point?
  • Uncertainty: Does it clearly distinguish documented facts from genuine information gaps?

These are independent questions. Do not replace them with one overall impression.

What each score examines

A document-grounded answer is reviewed through four separate lenses: factual correctness, source support, required-point completeness, and handling of missing evidence.

Each lens can receive a different score; fluent wording does not automatically pass all four.

Use observable anchors, not a general impression

The 0–2 scoring anchors

Use these anchors consistently. Unverified means the sources do not establish a claim; it does not mean the claim has been proven false.

Apply the anchor that matches observable answer behavior, not its style or length.
Dimension0 = Not met1 = Partly met2 = Met
AccuracyIncorrect or unverified core assertionMinor error or unverified ancillary assertion; core remains intactVerified correct
GroundingSubstantive claim lacks support, or cited passage does not support itSupport exists, but reference is missing or too broadSubstantive claims are supported by correctly identified passages
CoverageNo required answer points presentSome required answer points presentAll required answer points present
UncertaintyGuesses a gap or refuses answerable contentSeparation is vague or unnecessarily cautiousClearly separates support from genuine gaps without needless hedging

Independent scores can reveal different defects

A test case before scoring

The expectations are set before an answer is examined.

Question type: Routine question with a partly unsupported request

Question: According to the sources, who may book, what is the maximum duration of one booking, and are microphones available?

Expected evidence:

  • Room Booking Guide > Eligibility — “Registered members may book.”
  • Room Booking Guide > Booking limits — “Each booking can last up to 2 hours.”
  • Not stated in sources — microphone availability.

Required answer points:

  • Registered members may book.
  • One booking may last up to 2 hours.
  • Microphone availability is Not stated in sources.

Unacceptable claims:

  • Microphones are available.
  • Microphones are unavailable.

Example

Example answer to score

Actual answer

“Registered members may book. Each booking can last up to 2 hours. Microphones are not available. [Room Booking Guide > Eligibility; Room Booking Guide > Booking limits]”

Criterion-level notes

The first two claims are supported, but microphone availability is an explicit part of the question. Turning that evidence gap into a definite negative is an unverified core assertion, not an ancillary detail.

Keep scores separate: one unsupported core claim can affect several dimensions.
CriterionScoreEvidence note
Accuracy0 = Not met“Microphones are not available” -> Not stated in sources -> An unverified core assertion answers an explicit part of the question.
Grounding0 = Not met“Microphones are not available” -> cited Booking Guide passages do not address microphones -> Substantive claim lacks support.
Coverage1 = Partly metEligibility and 2-hour duration are present -> Required gap statement is absent -> Some required answer points are present.
Uncertainty0 = Not metDefinite microphone claim -> Not stated in sources -> The answer guesses instead of identifying the gap.

Practice: classify the evidence behavior

Practice: distinguish adjacent anchors

Choose the score set that correctly evaluates all four separate observations.

A. “Cedar Room has a display.” No passage reference is supplied.

B. “Cedar Room has microphones.” It cites Room Facilities Guide > Equipment.

C. “Microphone availability is Not stated in sources.”

D. “I cannot determine whether one booking can last up to 2 hours.”

Step 5 of 8

Calibrate Your Scores on Sample Answers

Score contrasting Cedar Learning Hub answers with independent criterion ratings and evidence-based notes.

A repeatable scoring sequence

A repeatable scoring sequence

Score claims, not style

Calibration checks whether the rubric rewards evidence-based meaning rather than polished wording. For each answer:

  1. Identify its factual assertions.
  2. Compare each assertion with the expected evidence.
  3. Check every required answer point.
  4. Assign Accuracy, Grounding, Coverage, and Uncertainty independently.

A concise paraphrase can meet the rubric without copying a reference sentence. A fluent addition can lower several scores if the sources do not establish it. When reviewers disagree, return to the passage or evidence gap and the relevant scoring anchor—not the answer’s confidence or style.

Trace each claim before rating

A flow from answer claims to source evidence, required points, and four separate criterion ratings.

Compare claims with evidence and required points before assigning independent ratings.

Set the comparison basis first

Example

Test case A: supported synthesis

Question type: Synthesis across sources

Question: What equipment does Cedar Room have, and what is the maximum duration of one booking?

Expected evidence

  • Room Facilities Guide > Equipment — “Cedar Room has a display and a whiteboard.”
  • Room Booking Guide > Booking limits — “Each booking can last up to 2 hours.”

Required answer points

  • Cedar Room has a display.
  • Cedar Room has a whiteboard.
  • One booking may last up to 2 hours.

Unacceptable claims

  • Cedar Room has no display.
  • A booking may last longer than 2 hours.
  • Microphone availability is asserted as a fact.

These expectations test meanings, not a preferred answer sentence.

Example

Test case B: documented fact and evidence gap

Question type: Routine question with a partly unsupported request

Question: Does Birch Room have a whiteboard and microphones?

Expected evidence

  • Room Facilities Guide > Equipment — “Birch Room has a whiteboard but no display.”
  • Not stated in sources — microphone availability.

Required answer points

  • Birch Room has a whiteboard.
  • Microphone availability is Not stated in sources.

Unacceptable claims

  • Birch Room has no microphones.
  • Birch Room has microphones.
  • A missing microphone statement proves either availability or absence.

The documented equipment and the information gap must remain separate.

Score sample answer A

Practice scoring: concise paraphrase

Score this sample answer against test case A from the previous screen.

Actual answer: “Cedar Room has a screen and a whiteboard. A booking can last no more than two hours. [Room Facilities Guide > Equipment; Room Booking Guide > Booking limits]”

Provide four separate scores using 0 = Not met, 1 = Partly met, or 2 = Met. Then write evidence notes in the form Claim -> Evidence -> Rating.

Write at least 100 characters (0/100).

Score sample answer B

Practice scoring: plausible but unsupported

Score this sample answer against test case B from the earlier screen.

Actual answer: “Birch Room has a whiteboard, but it has no microphones. [Room Facilities Guide > Equipment]”

Provide four separate scores using 0 = Not met, 1 = Partly met, or 2 = Met. Then write evidence notes in the form Claim -> Evidence -> Rating.

Write at least 100 characters (0/100).

Step 6 of 8

Record a Repeatable Baseline

Create a run record that preserves the evaluation inputs, actual response, scores, and evidence notes.

Preserve a comparison, not just a score

Capture the conditions, not just the scores

A repeatable baseline preserves what was available and what was observed at one point in time. It does not promise that a later assistant will use identical wording.

Before judging the result, retain the source snapshot and relevant passages, the complete test case, the test set and rubric versions, and the exact instructions. Then preserve the Actual answer verbatim, followed by separate Scores and Evidence notes.

This order matters: an edited rewrite is not the answer that was scored. Without the sources, instructions, context mode, and original response, a later comparison is not fair.

What a baseline preserves

A flow from versioned source documents and a test case through instructions and an answer to separate scoring notes.

A baseline connects the available evidence and instructions to the unedited answer and its criterion-level evaluation.

Tip

Keep changes and unknowns visible

Do not silently replace a corrected expectation, source snapshot, or rubric. Preserve the earlier record and assign a new version to the corrected artifact. A comparison is meaningful only when you can see which version was used.

Likewise, record unavailable metadata as Unknown. Do not infer a model name, retrieval setting, or hidden configuration from an answer.

Organize setup, case, and result

A lightweight run-record template

Use a text document or spreadsheet with setup, case, and result sections. The run-record labels below preserve the conditions that can affect an answer or its evaluation.

Retain these labels for each evaluated response.
SectionRun-record labelWhat to retain
SetupSource snapshotActive source titles, versions, update dates, and relevant passages
SetupTest set versionVersion of the test cases and their expectations
SetupRubric versionVersion of the four scoring anchors
SetupInstructionsComplete, exact answer instructions used for the run
SetupTool/modelVisible tool and model name, or Unknown
SetupContext modeDocument retrieval, Direct context, or Provided sample
ResultRun dateDate in YYYY-MM-DD format
ResultActual answerUnedited response, including source references
ResultScoresAccuracy, Grounding, Coverage, Uncertainty shown separately
ResultEvidence notesCriterion notes in Claim -> Evidence -> Rating form

Keep the case and the visible setup

Attach the complete test case to the record rather than relying on a short nickname. Preserve its Question type, Question, Expected evidence, Required answer points, and Unacceptable claims.

For Cedar Learning Hub, the source snapshot identifies both active sources: Room Booking Guide, version 1.0, updated 2026-02-01; and Room Facilities Guide, version 1.0, updated 2026-02-01. Retain the relevant passage excerpts alongside those identifiers.

A provided sample is a valid no-tool route. An optional local run requires the full source text and exact instructions that you supplied to your own assistant.

Record a sample answer faithfully

Example

Example: preserve the setup and test case

Source snapshot: Active sources: Room Booking Guide, version 1.0, updated 2026-02-01; Room Facilities Guide, version 1.0, updated 2026-02-01.

Test set version: 1.0

Rubric version: 1.0

Instructions: “Answer using only the active Cedar Learning Hub sources. Cite factual claims with source title plus section. When requested information is absent, say Not stated in sources; do not infer or add outside facts.”

Tool/model: Unknown

Context mode: Provided sample

Run date: 2026-02-01

Question type: Routine question with a partly unsupported request

Question: Does Birch Room have a whiteboard and microphones?

Expected evidence: Room Facilities Guide > Equipment — “Birch Room has a whiteboard but no display.” Not stated in sources — microphone availability.

Required answer points

  • Birch Room has a whiteboard.
  • Microphone availability is Not stated in sources.

Unacceptable claims

  • Birch Room has microphones.
  • Birch Room has no microphones.

Example

Example: preserve the result separately

Actual answer: “Birch Room has a whiteboard. Microphone availability is Not stated in sources. [Room Facilities Guide > Equipment]”

Scores

  • Accuracy: 2 = Met
  • Grounding: 2 = Met
  • Coverage: 2 = Met
  • Uncertainty: 2 = Met

Evidence notes

  • “Birch Room has a whiteboard” -> Room Facilities Guide > Equipment -> Accuracy 2.
  • Factual room-equipment claim cites its matching section -> Room Facilities Guide > Equipment -> Grounding 2.
  • Whiteboard fact and microphone evidence gap are both present -> Required answer points -> Coverage 2.
  • “Not stated in sources” -> microphone availability is not addressed by either source -> Uncertainty 2.

The Actual answer remains an observation. Scores and Evidence notes are the later evaluation of that observation.

Practice: restore the missing baseline fields

Repair an incomplete record

A colleague retained only these parts of a Cedar Learning Hub record: Run date, Test set version, Rubric version, and four separate Scores.

Write the missing record content needed for a fair repeat comparison. Include:

  1. Every missing run-record label and what it must retain.
  2. The complete test case, using all five required test-case field labels.
  3. Tool/model: Unknown if no visible tool or model information was saved.

Do not invent the old response, instructions, or metadata. State that the original item must be retained verbatim or recorded as Unknown where appropriate.

Write at least 500 characters (0/500).

Step 7 of 8

Report Failures Without Hiding Them

Show profiles, not a single verdict

Keep the criterion profile visible

Report every evaluated test case as a four-part profile, in this order: Accuracy, Grounding, Coverage, and Uncertainty. Keep the four scores separate; do not replace them with an overall average or a label such as “passed.”

A positive-looking profile can still contain an important failure. An answer may include all required answer points while adding a claim that the sources do not support. Separate scores keep that grounding and uncertainty problem visible.

One weak criterion should not disappear

Two case report cards show four separate criterion rows, with one grounding-related failure visibly distinct from otherwise stronger scores.

A case-level profile preserves the evidence-based detail that a single overall result could hide.

Make failures and gaps explicit

Separate observed results from guesses

Use a case status before interpreting results:

  • Scored means an Actual answer was collected and evaluated.
  • Not run means no answer was evaluated. It is not a pass, a failure, or a score of zero.

For a scored case, make each failure note specific: Claim -> Evidence -> Rating. Identify the exact unsupported claim or omitted required answer point, then connect it to the relevant passage or to Not stated in sources.

Report what the evidence demonstrates. An unsupported statement is observable. Saying that retrieval failed, a model ignored instructions, or a source was missed is a hidden-cause explanation that this record does not establish.

Example

An evidence-specific failure note

Actual answer: “Reservations may be made up to 7 days in advance. Bookings are confirmed by email. [Room Booking Guide > Booking window]”

Evidence notes

  • “Reservations may be made up to 7 days in advance” -> Room Booking Guide > Booking window — “Reservations may be made up to 7 days in advance.” -> Supported.
  • “Bookings are confirmed by email” -> Not stated in sources -> An unverified ancillary assertion; Accuracy 1 = Partly met for the case.
  • “Bookings are confirmed by email” -> cited Room Booking Guide > Booking window does not address confirmation methods -> Grounding 0 = Not met.
  • Definite email-confirmation claim -> Not stated in sources -> Uncertainty 0 = Not met.

The note reports the unsupported addition without claiming why it appeared.

Case status and criterion scores

Keep an unrun case in the report. A partial baseline can be useful, but it must not be presented as a completed four-case result.

Separate scores expose the unsupported email-confirmation claim. The unrun case has no inferred scores.
Test caseStatusAccuracyGroundingCoverageUncertainty
Case 1: booking timingScored1020
Case 4: equipment and microphonesNot run————

Practice transparent reporting

Choose the transparent report

Case 1 is Scored. Its answer correctly says reservations may be made up to 7 days in advance and cites Room Booking Guide > Booking window. It also states, “Bookings are confirmed by email.” Neither active Cedar Learning Hub source addresses booking confirmation methods.

Case 4, about Birch Room equipment and microphone availability, is Not run.

Which report is evidence-specific and does not hide either result?

Step 8 of 8

Assemble and Review Your Test Set

Finish a four-case evidence-based test set with a reusable rubric, two scored responses, and a traceable evaluation record.

Four test cases, starting with expectations

Assemble one evidence-first package

A baseline-ready test set is a connected package, not merely a list of questions. Keep each test case linked to expectations written from the sources. Then keep collected answers, scores, and evidence notes separate from those expectations.

For Cedar Learning Hub, preserve the four complementary cases already established: direct lookup, conditional application of a rule, synthesis across sources, and a routine question with a partly unsupported request. This package supports a later comparison, but it establishes quality only for the cases that are actually scored.

What the finished package connects

A diagram showing four test cases connected to source evidence, a shared rubric, two scored answers, and a preserved evaluation record.

Each test case begins with source-based expectations; answers are collected and scored afterward.

Test case 1: booking timing

Question type
Direct lookup

Question
How far in advance may a registered member reserve a room?

Expected evidence
Room Booking Guide > Booking window — “Reservations may be made up to 7 days in advance.”

Required answer points

  • States that a reservation may be made up to 7 days in advance.
  • Keeps the limit as a maximum, not a minimum or guarantee.

Unacceptable claims

  • Says reservations can be made any time in the future.
  • Changes the limit to 2 hours.

Test case 2: active-booking limit

Question type
Conditional application of a rule

Question
A registered member already has an active booking. Can they make another booking?

Expected evidence
Room Booking Guide > Booking limits — “Each member can hold one active booking. Each booking can last up to 2 hours.”

Required answer points

  • Explains that the member cannot hold another active booking while one is already active.

Unacceptable claims

  • Says a member may hold two active bookings.
  • Treats eligibility to book as permission to hold multiple active bookings.

Attach the rubric and repeatable setup

Test case 3: display and duration

Question type
Synthesis across sources

Question
I need a room with a display for a 2-hour session. Which room can I book?

Expected evidence

  • Room Facilities Guide > Equipment — “Cedar Room has a display and a whiteboard.”
  • Room Booking Guide > Booking limits — “Each member can hold one active booking. Each booking can last up to 2 hours.”

Required answer points

  • Cedar Room meets the display requirement.
  • A booking may last up to 2 hours.

Unacceptable claims

  • Birch Room has a display.
  • A booking may last longer than 2 hours.

Test case 4: equipment and evidence gap

Question type
Routine question with a partly unsupported request

Question
Does Birch Room have a whiteboard and microphones?

Expected evidence

  • Room Facilities Guide > Equipment — “Birch Room has a whiteboard but no display.”
  • Not stated in sources — microphone availability.

Required answer points

  • Birch Room has a whiteboard.
  • Microphone availability is Not stated in sources.

Unacceptable claims

  • Birch Room has no microphones.
  • Birch Room has microphones.
  • A missing microphone statement proves either availability or absence.

Reusable rubric 1.0

Attach this shared rubric to every test case. Keep the four ratings separate; do not replace them with one combined total.

Use 0 = Not met, 1 = Partly met, and 2 = Met.
Dimension2 = Met1 = Partly met0 = Not met
AccuracyVerified correctnessMinor error or unverified ancillary assertionIncorrect or unverified core assertion
GroundingSubstantive claims supported by correctly identified passagesSupport exists, but reference is missing or too broadSubstantive claim lacks support or cited passage does not support it
CoverageAll required answer points presentSome required answer points presentNo required answer points present
UncertaintyClearly separates support from genuine gapsSeparation is vague or unnecessarily cautiousGuesses a gap or refuses clearly answerable content

Shared source snapshot and run setup

Add this setup block before collecting answers. It preserves the exact conditions of test set version 1.0.

  • Source snapshot: Active sources: Room Booking Guide, version 1.0, updated 2026-02-01; Room Facilities Guide, version 1.0, updated 2026-02-01.
  • Test set version: 1.0
  • Rubric version: 1.0
  • Instructions: “Answer using only the active Cedar Learning Hub sources. Cite factual claims with source title plus section. When requested information is absent, say Not stated in sources; do not infer or add outside facts.”
  • Tool/model: Unknown
  • Context mode: Provided sample
  • Run date: 2026-02-01

For a local run, replace only metadata you can actually see. Keep unavailable metadata as Unknown.

Preserve two contrasting scored responses

Record outcomes by case status

Two cases are scored below; the other two remain visibly untested. A concise answer can pass when it preserves the required meaning and identifies supporting evidence. A fluent unsupported addition remains visible as a failure rather than being hidden by other strengths.

Case status overview

Cases 1 and 2 belong to the test set, but no Actual answer has been collected for them. They must remain Not run, not be treated as a pass or a score of zero.

Use only `Scored` and `Not run` as case statuses.
Test caseStatusRecord state
Case 1: booking timingNot runNo Actual answer, Scores, or Evidence notes
Case 2: active-booking limitNot runNo Actual answer, Scores, or Evidence notes
Case 3: display and durationScoredAnswer and criterion-level notes preserved
Case 4: equipment and microphonesScoredUnsupported microphone assertion remains visible

Scored record: acceptable paraphrase

Status: Scored

Actual answer: “Cedar Room is the option with a display, and a booking can last up to 2 hours. [Room Facilities Guide > Equipment; Room Booking Guide > Booking limits]”

Scores

  • Accuracy: 2 = Met
  • Grounding: 2 = Met
  • Coverage: 2 = Met
  • Uncertainty: 2 = Met

Evidence notes

  • “Cedar Room is the option with a display” -> Room Facilities Guide > Equipment — “Cedar Room has a display and a whiteboard.” -> Accuracy 2.
  • The display and duration claims cite their matching sections -> correctly identified passages -> Grounding 2.
  • Cedar Room’s display and the 2 hours maximum are both present -> Required answer points -> Coverage 2.
  • The answer gives only supported information for this question -> Expected evidence -> Uncertainty 2.

Scored record: unsupported addition

Status: Scored

Actual answer: “Birch Room has a whiteboard, but it has no microphones. [Room Facilities Guide > Equipment]”

Scores

  • Accuracy: 0 = Not met
  • Grounding: 0 = Not met
  • Coverage: 1 = Partly met
  • Uncertainty: 0 = Not met

Evidence notes

  • “It has no microphones” -> Not stated in sources -> Accuracy 0 because an unverified core assertion changes the answer.
  • “It has no microphones” -> Room Facilities Guide > Equipment does not address microphone availability -> Grounding 0.
  • “Birch Room has a whiteboard” is present, but the required Not stated in sources qualification is absent -> Required answer points -> Coverage 1.
  • Definite microphone absence -> Not stated in sources -> Uncertainty 0 because the response converts a gap into certainty.

Review, apply, and preserve

Final review checklist

Before calling the package ready, check:

  • All four test cases use the five required field labels.
  • Each Expected evidence entry contains a passage reference and short excerpt, or Not stated in sources for a genuine gap.
  • Required answer points are separate bullets, not one preferred sentence.
  • The rubric lists Accuracy, Grounding, Coverage, Uncertainty in that order and preserves four separate scores.
  • At least two cases are Scored; every other case is explicitly Not run.
  • Every scored case retains its Actual answer and Evidence notes in Claim -> Evidence -> Rating form.
  • The source snapshot, exact Instructions, versions, Context mode, and available run metadata are preserved.

Summary

Baseline review

Use this sequence whenever you create or repeat a document-grounded answer check.

  • Establish expectations from sources before examining an answer.
  • Collect answers while preserving the exact setup and Actual answer.
  • Score Accuracy, Grounding, Coverage, and Uncertainty against evidence.
  • Preserve the record for later comparison.
  • A baseline supports conclusions only about the test cases that were actually evaluated.

Final application: review one case

Review the checklist, then choose one test case from the on-screen set or your local copy. Submit a concise review containing:

  1. Its Question type, Question, Expected evidence, Required answer points, and Unacceptable claims.
  2. One criterion result from a scored answer, naming the criterion and using Claim -> Evidence -> Rating with a score of 0, 1, or 2.
  3. One sentence explaining why this package is useful for a later comparison but does not prove quality beyond the cases evaluated.

No file upload is needed.

Write at least 180 characters (0/180).

Download the App now to have access to + 5000 free courses, exercises, certificates and lots of content without paying anything!

  • 100% free online courses from start to finish

    Thousands of online courses in video, ebooks and audiobooks.

  • More than 60 thousand free exercises

    To test your knowledge during online courses

  • Valid free Digital Certificate with QR Code

    Generated directly from your cell phone's photo gallery and sent to your email

Cursa app on the ebook screen, the video course screen and the course exercises screen, plus the course completion certificate