When to Keep a Human in the Loop With AI

· 6 min read

Keep a human in the loop whenever a mistake would be costly, hard to undo, or would land on someone else, and treat money, health, legal matters and private data as high-stakes by default. Let AI work alone only on low-stakes, easily reversed tasks that you can spot-check afterwards.

"Human in the loop" means a human looks at the AI's work, or approves its action, before the result counts. The skill is deciding how much looking a task deserves and making sure it is real.

Which kinds of work should not run without a human?

Four kinds:

  • Costly mistakes. A wrong figure in a quote, or a message to a client sent to the wrong address.
  • Hard to undo. Deleting files with no backup, sending an email, paying a bill. Once it is read or the money has moved, it is hard or impossible to take back.
  • Work that affects someone else. A reply, a grade, a hiring shortlist, a decision about a customer. The people affected did not agree to carry the AI's mistakes. In some places the law also limits decisions about people made only by software, so check the rules for your field.
  • Money, health, legal or private data. Errors here are expensive and often invisible until later.

What four questions decide how much oversight a task needs?

Ask these before handing a task over, and write the answers down for anything that repeats.

1. What is the worst realistic outcome? Think of the worst plausible result, not the worst imaginable one. "The summary misses a point" is mild. "The wrong client gets the contract" is not. 2. Can it be undone, and how fast? A draft can be deleted in a second. An email sent to a long list usually cannot be recalled. 3. Who is affected? Only you, or also a customer, a colleague, a child, a patient? 4. How would I notice a mistake? If a wrong total can be checked against your own records, you can catch it in seconds. If wrong output looks fine, such as a plausible but invented fact, someone has to check before it leaves your hands.

If the first three answers are mild and the fourth is "quickly", little oversight is needed. A mistake you would not notice calls for a check even when the stakes look modest, because it can repeat unseen.

What are the three levels of oversight?

  • Level one: suggest only. The AI drafts or proposes and you do the action yourself. Use it for anything high-stakes or new.
  • Level two: act after approval. The AI prepares the action and waits. You see exactly what will happen and approve or reject it. Approving the exact action beats a broad permission like "allow the terminal", and why an AI agent should ask before it acts explains why. Use it for actions that matter and come up often.
  • Level three: act alone, with a log you read. The AI proceeds and records what it did, so you check afterwards instead of beforehand. Use it only for routine, low-stakes, reversible work. The log is part of the deal, and it only counts if someone reads it.

Start new work at level one, move to level two once you have watched it go right several times, and to level three only after it has been reliable on that exact kind of task. Go back toward level one after any surprise, and whenever the tool, its settings or the kind of task changes, because past reliability does not carry over. A chatbot that only answers sits at level one; an agent that acts forces the choice, as AI agents vs. chatbots explains.

What do the levels look like in practice?

Three invented examples, each landing at a different level.

Suppose a freelance designer asks an assistant to draft polite reminders for late invoices. A client will read each one and money is involved, so it stays at level one: the designer checks the name and amount against their records and sends it themselves.

In a second case, a home cook asks an assistant to tag saved recipes as quick or slow. The worst outcome is a wrong tag, only the cook is affected, and fixing one takes a second. This can run at level three, with a glance at the tags each week.

In a third, a small dental practice asks an assistant to word a reminder template, with a placeholder for the patient's name. Staff approve the template before anyone uses it (level two). Anything about a patient's condition or medication stays at level one, written by staff, because it is health data.

Likewise, a summary only you will skim can run alone, while one you will repeat to a client needs the checks in how to fact-check an AI answer.

How does a human check turn into a rubber stamp?

A human in the loop is only a safeguard while they actually check, and a few habits wear that away.

The first is rubber-stamping: approvals you no longer read. A telltale sign is that you cannot say what you approved last time. It often follows approval fatigue, because when every step asks, the one that matters looks like all the others.

The second is checking only the AI's account of its work. If it reports "updated the records, no problems" and that is all you see, you are reading its summary, not the work, and a summary can be wrong, incomplete or silent about the one odd item.

Two quieter traps remain. People tend to trust a machine's output, especially when it is usually right, which is called automation bias. And a second assistant checking the first can catch some errors but can share its blind spots, so it is not a human check.

How do I keep a human check real?

  • Show the real thing. Look at the actual email, the exact command or the changed file, not a description of it.
  • Save approvals for what matters. Let low-risk steps, such as reading your own notes, proceed so each prompt gets attention.
  • Spot-check level three. Each week, trace a few random items back to the source and read the log.
  • Decide in advance who checks what. Name the checker, what they look at, and what they do when something is wrong. For a team, write it down; an AI use policy for small teams shows how.
  • Make saying no cheap. If rejecting causes friction, people approve instead.

A human check lowers the risk without removing it, because the human can miss an error too. Whoever approves answers for the result, so do not approve what you have not read. For the highest-stakes work, choose a checker who could have done the task unaided.

Where NAMELESS stands

NAMELESS is an AI model that is still in training and not yet available, and there is no launch date, so nothing above can be tried on it today. Our goal is a model on our own architecture, with unlimited context and no rereading of the conversation, extremely fast, with stronger logic and reasoning. That does not remove the need for the checks above: costly, hard-to-undo or private work still needs a human, and an approval only helps if it is read. To follow progress, join the waitlist.

Keep reading

All posts