Dr.Hani Tiếng Việt
Applied Knowledge Technology

Designing Human-in-the-Loop Workflows for AI

5 min readAssoc. Prof. Nguyen Hai Ninh
Nhóm nhân sự cùng trao đổi trước máy tính trong không gian làm việc

Many teams begin using AI with a simple request: let the tool make a slow task faster. That can produce a quicker first draft, a neater summary, or an earlier customer response. Yet when a tool enters work without defining who checks it, what they check, and when the process must stop, speed can bring risk: incomplete data enters a report, an unapproved promise reaches a customer, or a small error is repeated at scale.

Human-in-the-loop does not mean that someone must reread every sentence generated by AI. It is a way to design a workflow so that people intervene where professional judgement, accountability, and real-world context are genuinely needed. When designed well, AI reduces the work of searching, organizing, and preparing; people retain the authority to confirm, decide, and take responsibility for the result.

Do not start with the tool; start with the work output

A workflow is often suitable for AI when its inputs are reasonably clear, its output can be checked, and the task recurs. A customer-service team might use AI to classify requests and draft replies; an HR team might use it to summarize feedback after training; a consulting team might use it to organize interview notes by theme. Each can create value when the organization defines in advance what a good output looks like.

By contrast, a broad request such as “use AI to do strategy” is not yet enough to design a workflow. Strategy requires choices, trade-offs, and an understanding of resources, customers, and competitors. AI can help assemble information or pose challenging questions, but it cannot replace the manager who is accountable for direction. The starting question should be: in one specific task, which part can be prepared more quickly and which part still requires judgement from an authorized person?

Workflow point AI can support People need to retain
Receiving inputs Extracting, classifying, and detecting missing information. Deciding which data may be used and which must be excluded.
Preparing options Summarizing, comparing, drafting, or suggesting questions. Checking sources, assumptions, and fit with the context.
Making a decision Displaying options and possible consequences. Choosing an option, approving commitments, and accepting trade-offs.
Sending or implementing Standardizing forms, tracking status, and prompting action. Confirming final content, handling exceptions, and remaining accountable to stakeholders.

Define stop points before deployment

A stop point is a condition that requires the workflow to return to a person. It should be tied to the level of risk, not to a vague feeling that “important matters should be escalated.” An AI assistant might classify support tickets and draft replies to questions that already have guidance. If a ticket contains payment information, a public complaint, a refund request above a threshold, or signs of a data incident, however, the system should stop and route it to the responsible employee.

These stop points keep teams out of two extremes. One is allowing AI to send everything in the name of saving time. The other is requiring people to inspect every simple task, leaving the tool unable to create meaningful benefit. Organizations need to learn from exceptions: which exceptions recur, which errors could be prevented with better data, and which conditions should become new rules.

Example: preparing a weekly sales report

AI can consolidate CRM data, group opportunities by stage, and draft a summary of changes that need attention. A sales manager should not merely read the summary and forward it. The manager needs to check major accounts, the reasons an opportunity has fallen in probability, and commitments that are coming due. The weekly report then becomes a conversation about action: who needs to meet which customer, which obstacle must be removed, and which decision needs escalation.

Design review roles by the consequences involved

Not every output needs the same reviewer. An internal item may be confirmed by the responsible employee using a checklist. A proposal sent to a customer needs the relationship owner to check commitments, pricing, and tone. A recommendation involving legal, financial, or sensitive data issues must go to a person with the appropriate expertise. When review roles are unclear, people close to the work may hesitate to decide, while senior managers are pulled into detail that does not need their attention.

The checklist should be short and linked to errors with real consequences: does the figure have a source; does the reply promise something the company cannot deliver; has customer data entered a place where it is not permitted; does another person need to approve it? A checklist that merely repeats “check carefully” does not help a person find errors more effectively.

Measure quality as well as speed

AI experimentation is easily called successful when the time to create a first draft falls. That is only one input metric. Teams should also track rework, errors found after sending, time spent handling exceptions, and feedback from recipients. If time falls while revision rounds increase, the workflow may have shifted cost from drafting to checking.

A simple improvement cycle is to choose one recurring situation; define the output and stop points; test it at small scale; record errors and exceptions; adjust data, guidance, or access rights; and only then expand. This approach helps technology become an operating capability rather than remain an appealing demonstration.

Conclusion

A strong human-in-the-loop workflow neither leaves people outside AI nor turns them into editors of every machine-written sentence. It allocates preparation, checking, and decision-making sensibly. Start with a specific output, define clear stop points, assign the right confirmation role, and measure both quality and speed. Then AI can help teams work faster without obscuring accountability.

Continue reading

Related insights