How to fact-check an AI answer in five minutes

To fact-check an AI answer, split it into separate claims, find each important claim in a source you can open yourself, and confirm it against an official or primary source. Then ask the assistant for the strongest argument against its own answer. For an everyday answer with a few claims, that can take about five minutes.

An assistant often writes in the same fluent, confident tone whether it is right or wrong, so a convincing answer tells you little about whether it is true. For the reasons behind the errors, see why AI assistants make things up.

What are the steps to fact-check an AI answer?

• Pull out the checkable claims. Separate facts, numbers, names, dates, quotations and cited sources from explanation and opinion.
• Ask for sources, and treat them as leads. A source an assistant names is a place to look, not proof.
• Confirm each source exists and says what is claimed. Open it and find the passage yourself.
• Cross-check key claims against a primary or official source. Go to the people who produce or decide the information.
• Ask for the strongest counter-argument. See what the best case against the answer looks like.

How do I pull out the claims and ask for sources?

Read the answer once and mark anything that could be proven wrong: a number, a date, a name, a quotation, a law, a study.

You can ask the assistant to do the sorting: "List every factual claim in your answer as a separate line." Then ask where each one comes from: "For each claim, give the title, the publisher, the year and the exact passage that supports it."

Some assistants can search the web and show real links; others answer from memory. Either can produce a reference with a believable author, title and web address that does not exist, or point to a real page that does not say what it claims.

How do I check a source, and what counts as a primary one?

Open the link. If it leads to an error page, or to a real page about something else, the source fails. If there is no link, search for the exact title and publisher.

Then find the passage. Search the page for the key words of the claim and check that the numbers, conditions and strength of wording match. Typical failures: a source that says "may be linked to" where the answer says "causes", or a study of a different group than the one you asked about.

Suppose an assistant tells you a tenant must get a deposit back within a set number of days and cites "Section 7 of the state housing code". Search for that code on the government's own website and read the section. You might find that Section 7 covers something else, that the rule belongs to a different state, or that the time limit differs from the answer's. Any of those means the claim is not confirmed.

Even a source that checks out may be second-hand. A primary source is where the information originates: the original study rather than a news story about it, the text of the law rather than a blog explaining it, the company's own documentation, the agency that publishes the guidance. If two pages agree, check whether one copied the other, because a copy is not a second source.

How do I stress-test an AI answer?

Once a claim has survived the checks, ask the assistant to attack it: "What is the strongest argument that this answer is wrong? What would a knowledgeable skeptic say, and what evidence would change the conclusion?" Keep the wording neutral. If you write "I'm sure this is right, aren't I?", an assistant can lean toward agreeing with you.

Treat the counter-argument as a list of things to check, not a verdict. A model can write a persuasive case against almost anything, so use it to find weak spots, such as a newer rule or a group the evidence did not cover.

Two things are not checks. Asking "are you sure?" is one, because the assistant may drop a correct answer or defend a wrong one. Asking a second assistant is the other, because two models can make the same mistake. Their disagreement shows you where to look, but their agreement is not confirmation.

Which AI claims are most likely to be wrong?

Some details go wrong more often than others. Check these first:

• Very precise numbers with no source. A figure to the decimal point with no named origin may just sound right.
• Quotations. Words attributed to a person or a document may be paraphrased, merged or invented. Find the original text.
• Legal cases. Court filings drafted with an AI tool have cited cases that do not exist. Look the case up in the court's own records.
• URLs. Links can be made up or mangled. Open them.
• Recent events. A model without search knows only what it learned before some cutoff date, and even with search it can misread a page. Check the source's date.

Health, money, law and safety need checking every time, because a mistake there can hurt. Use the assistant to understand terms and prepare questions, and base the decision on official guidance and a qualified professional. For a longer project, see a method for researching with AI.

How can I prompt for answers that are easier to verify?

Shape the answer so that checking is faster:

• Ask it to quote the passage. "Quote the exact sentence that supports each claim and say where it appears." This works best when you have pasted the document in, as in summarizing long documents with AI. A quote produced from memory can be wrong too, so still look it up.
• Separate what it knows from what it infers. "Label each statement as stated in the text I gave you, general knowledge, or your own inference." Inferences deserve the closest look.
• Allow "I do not know." "If you cannot support a claim, say so instead of guessing."

The better AI prompts checklist has more.

What if I cannot verify an AI answer?

Sometimes five minutes is not enough. Then treat the claim as a hypothesis, not a fact.

• Do not repeat it as settled in an email, a report or an essay. If you must mention it, say it is unverified.
• Mark it in your notes so you do not later remember it as confirmed.
• Weigh the stakes. For a casual question, a hypothesis may be enough. For health, money, law or safety, wait for an official source or a qualified person.

The same habit applies when you learn with an AI: a claim you cannot trace does not belong in your essay. How to study with AI without cheating yourself covers that side.

Where NAMELESS stands

NAMELESS is an AI model that is still in training and not yet available. It is built on our own architecture, with unlimited context, no rereading of the conversation, and stronger logic and reasoning. Stronger reasoning is not the same as verified facts, though: no model is immune to mistakes, and this routine applies to ours as much as to any other. To follow progress, join the waitlist.