Audit an AI Summary One Claim at a Time

Research Notes

A fluent summary can hide a changed number, a lost condition or a conclusion that the source never made. Review claims in small units before sharing the result.

Three-step citation audit: locate the passage, compare the claim and preserve qualifications
Original editorial diagram by Prompt Task Journal. Illustrative worksheet, not an application screenshot.

Split dense sentences into claims

A sentence saying a tool is cheaper, faster and easier contains at least three claims. Copy those claims into a review table. Start with the ones that affect a decision: prices, deadlines, eligibility conditions and promised outcomes. A useful audit can be selective, but its scope should be explicit.

Compare the source and the wording

Open the original passage rather than asking the same model whether it was right. Check units, dates and the population being described. “Some participants reported” must not become “users achieve”. When the source is a product page, label a performance assertion as the publisher’s claim unless independent evidence supports more.

Use three review states

Mark each claim as supported, needs qualification or unresolved. Supported means the passage substantiates the wording within the scope you checked; it does not mean the source itself is beyond challenge. Needs qualification covers statements that lose a condition. Unresolved means you need more evidence or should remove the claim.

Rewrite only after the evidence pass

Ask for a revision based on your marked table. Require the model to preserve the qualifications and omit unresolved assertions. Compare the revised summary with the table again, especially if it combines sentences. Keep the original source links with the finished summary so the next reader can inspect the basis rather than inherit your confidence.

An example to adapt

Break this draft into independently checkable factual claims. Return claim | supplied source reference | qualification to verify. Do not search for or invent citations. Mark any claim without a supplied reference “unresolved”. I will check the originals before asking for a revision.
Draft and source references: [Paste here.]

Illustrative starting point. Adapt it to your inputs, permissions and review process.

Before you use it

  • Numbers retain their units and dates.
  • Conditions and exceptions remain visible.
  • Publisher claims are attributed.
  • Unresolved claims are omitted or clearly marked.