AI Literacy Exercise: Check Two Flawed Sample Answers

Assess the reasoning you can observe

A learner may choose the more confident of two AI-style answers without checking either one. To assess their judgement, ask them to identify the claims, show the supporting evidence, and defend the corrections they make.

This is an original, low-stakes assessment method for adult workplace learning. It includes a complete fictional exercise that a facilitator can adapt. The outputs below were written for the exercise; they are not results from a tested model or examples of learner performance.

Use it after participants have practised source checking. It is intended to guide feedback and further practice, not to make employment decisions or award a professional qualification. No learning outcome is guaranteed.

Give everyone the same evidence sheet

The fictional business lends display stands for events. The task is to review a proposed reply to: “Can I collect a wide stand on Saturday, and is a carry bag included?” Give every participant this short evidence sheet:

• E1, collection guide: collection is available Monday to Friday, 10 a.m. to 4 p.m. No weekend arrangement is described.

• E2, equipment guide: narrow stands include a carry bag; wide stands do not include one.

• E3, exception process: the operations coordinator can investigate requests outside the published arrangements. Investigation does not confirm availability.

For this exercise, these statements are the complete supplied evidence. All services, hours, and arrangements are invented. Participants do not need an account, a live AI tool, customer messages, or external research to complete the task.

Ask them to label each claim “supported,” “contradicted,” or “not established.” A missing weekend rule leaves weekend availability unestablished; it does not prove that an exception is impossible.

Present two differently flawed outputs

Fictional output A: “Absolutely. You can collect a wide stand on Saturday, and every stand comes with a carry bag. We will have it ready for you.”

Fictional output B: “Collection is Monday to Friday, 10 a.m. to 4 p.m. Wide stands do not include a carry bag, so Saturday collection is impossible.”

Ask participants to assess both. Output A is friendly and confidently answers the request. Output B preserves two supplied facts, then adds a conclusion the evidence does not establish. Choosing B as “better” leaves an important checking task unfinished.

The assignment is to produce one corrected reply, an evidence note for each material correction, and the next staff action. Tell learners that attractive wording earns no credit unless the meaning is supported.

Use a claim-by-claim response sheet

Give learners these fields for each claim: output and phrase; classification; evidence reference; explanation; corrected wording; remaining uncertainty. Encourage short explanations that another person could check.

Then ask two defence questions: “Which change matters most, and why?” and “What information would allow you to give a more definite answer?” These expose whether the learner can explain the boundary rather than merely replace a sentence.

For an independent assessment attempt, ask learners to work without new AI-generated critique. If you allow assistance during practice, record that condition. A correction supplied by a tool does not itself show that the learner can justify it.

Practise one claim at a time

Put E1–E3 beside the two answers. Separate the Saturday promise, the bag claim, the preparation promise, the weekday hours, the wide-stand bag fact and the conclusion that Saturday is impossible. Mark all six before reading the key. When no supplied statement confirms a reservation or preparation, write “no confirming evidence” rather than attaching an unrelated reference.

Then explain your judgement aloud or in writing. The key can check classifications and evidence references; a facilitator still needs to review the reasoning. Read the sample correction only after writing your own. Seeing it first is useful guided practice, but it changes what the attempt demonstrates. If the claim-checking activity is available on this page, select a classification and relevant evidence, then use “Check this claim”; your reasoning and corrected reply are not scored. The response sheet also works on paper.

Review against an instructor key

Keep this review key separate until the initial response is complete:

• A, Saturday collection: not established. E1 gives weekday arrangements and E3 permits investigation, without confirming an exception.

• A, every stand includes a bag: contradicted by E2. The requested wide stand has no included bag.

• A, ready for collection: not established. The evidence contains no confirmed preparation or reservation.

• B, weekday hours and no included bag: supported by E1 and E2.

• B, Saturday is impossible: not established. E3 leaves a request for investigation open.

One acceptable correction: “Published collection hours are Monday to Friday, 10 a.m. to 4 p.m. Wide stands do not include a carry bag. Saturday collection is not confirmed; the operations coordinator would need to investigate the request.”

The next staff action is to arrange that investigation through the ordinary workflow. Equivalent wording is acceptable if it preserves the facts and uncertainty. Do not mark an answer wrong merely because it differs from the sample sentence.

Use the same criteria for spoken and written work

Carnegie Mellon's Eberly Center describes rubrics through criteria, descriptors, and performance levels. For this original exercise, record “demonstrated,” “partly demonstrated,” or “not yet demonstrated” against four criteria:

• Evidence: links each material judgement to the relevant supplied statement.

• Distinctions: separates a contradiction from something not established.

• Correction: removes unsupported promises while retaining useful supported facts.

• Next action: identifies what remains to be checked without claiming it has happened.

A partially demonstrated correction might fix the bag claim but retain the Saturday promise. Record that exact gap instead of giving only a total score.

Offer a written explanation or a spoken explanation with the same evidence sheet and criteria. CAST's UDL guidelines encourage varied media for expression when a specific medium is not essential to the goal. Here the proposed goal is evidence-based judgement, so spelling speed or presentation polish should not determine the result.

For spoken responses, the facilitator can note the learner's evidence references and read back the recorded reasoning for confirmation. Check individual access needs in advance; offering two modes alone does not establish that the activity is accessible to everyone.

Turn the result into specific practice

Have facilitators compare their judgement of one fictional response before assessing participants. Resolve disagreements about the evidence or rubric rather than changing expectations after a learner finishes.

Give feedback tied to one observable gap: “You corrected the bag statement; now explain why missing weekend information cannot support a definite refusal.” Then use a fresh fictional case with the same reasoning demands to see what the learner can do independently.

Keep the outcome modest: evidence demonstrated in this exercise, plus a next practice step.

Sources rechecked 4 October 2026

• Carnegie Mellon University: Creating and Using Rubrics
• CAST UDL Guidelines: Use multiple media for communication

A practical next step

Explore a practical team AI workshop.

Related reading: Plan a workshop your team can use the next day.