Human-In-The-Loop Isn't The Safety
A human approval step only contains risk when the reviewer can reliably detect a bad recommendation.
A few weeks ago I posted about blast radius control in agentic AI. One pattern keeps coming back to me: human-in-the-loop is useful only when the human can meaningfully review the work.
Fixing the underlying authorization stack is the right long-term answer, but many enterprise systems were not built around agents and will not change at the same speed as the models. Teams still need a practical way to contain risk while those controls evolve.
A practical boundary while the authorization stack catches up
One approach is to separate preparation from execution. Let the agent research, cross-reference systems, surface the relevant context, and produce a recommendation. Make the evidence, signals, trade-offs, and proposed action visible. Then require a person to decide whether execution should proceed.
Visual summary
Separate preparation from authority
The agent can compress the work, but the human remains accountable for whether execution proceeds.
Agent prepares
Research + recommendation
Cross-reference systems, surface context, expose signals and trade-offs, and explain the proposed action.
Human decides
Authorize execution
Interrogate the evidence, reject or change the recommendation, and decide whether the action should run.
Why it matters: transparency helps, but it does not prove that the recommendation is reviewable at the required volume and pace.
This pattern is pragmatic. It lets the agent compress a large amount of work without giving it final authority over the consequence.
It also feels safer than it may be.
The same review pattern can create very different consequences
AI-generated code provides a familiar example. A pull request can look clean, pass automated checks, and read coherently. The reviewer skims it, approves it, and moves on without stress-testing the logic. A human was in the loop, but the approval did not add much safety.
The same review pattern becomes more consequential when the recommendation affects payroll, performance, or employment.
| Reviewed action | If the review fails |
|---|---|
| AI-generated code | A bad merge reaches the system |
| Payroll updates | A paycheck may not clear correctly |
| Performance evaluation | An inaccurate record may follow someone’s career |
| Workforce action | A PIP, promotion, or headcount decision may be difficult to undo |
These are scenarios, not measured incident rates. The point is that the quality of the approval step has to match the consequence of the action.
The reviewer must be able to catch the bad recommendation
The real bar for human-in-the-loop is not “did a human see it?” The bar is: can the human reliably catch a bad recommendation before it executes?
Visual summary
The real human-in-the-loop safety test
A human seeing a recommendation is not the same as a human being able to validate it.
01
Agent recommendation
The system surfaces context, evidence, trade-offs, and the proposed action.
02
Human review
A person decides whether the action should proceed.
The test
Can the reviewer catch a bad recommendation?
The answer depends on the review unit, signal quality, volume, and pace.
Why it matters: the approval step only contains risk when the reviewer can reliably identify the recommendation that should not proceed.
Review stops being meaningful when one or more of these conditions are present:
- The unit is too large. A batch contains more changes than one person can inspect.
- The signal is buried. The reviewer sees a polished explanation but cannot find the assumption that changes the result.
- The context is missing. Validation requires reopening systems and recreating the agent's work.
- The pace is too high. Recommendations arrive faster than people can examine them.
- The consequence is hard to reverse. A quick approval can create a long-lived human or operational impact.
In those conditions, the workflow has not added a safety layer. It has added a rubber stamp with extra steps.
Human oversight is also a UX problem
This is one of the most important user-experience problems in enterprise AI. The goal is not only to make agents capable. The interface and workflow have to make the critical evidence reviewable at the speed and scale at which the agent operates.
The design questions I am using are:
- Unit: Should the person review one action, a plan, or a batch?
- Signal: What evidence and exception should the interface surface first?
- Capacity: At what volume or pace does reliable review stop being practical?
- Authority: Which actions require approval, and which should never be available to the agent?
- Recovery: What can be staged, reversed, or compensated if the decision is wrong?
I do not think there is one safe batch size or review threshold for every system. The answer depends on the action and its consequence. The design requirement is to prove that the reviewer can interrogate the recommendation rather than assume that an approval button creates control.