The Who Why and How of Human in the Loop

Key Points: 

• Human-in-the-Loop (HITL) positions employees as guides of their own AI systems   
  

• AI failures reveal system limitations but also offer opportunities for refinement, turning errors into a pathway for improvement   

  
  

• A balanced HITL approach is essential to integrate human values, prevent biases, and ensure AI evolves responsibly and in alignment with an organization’s values 

57% of American workers have tried ChatGPT at work. AI adoption at work is on the rise. As more and more employees interact regularly with tools like ChatGPT, they becoming increasingly more familiar with them. Relationships are developing between your employees and Natural Language Processors like ChatGPT. Understanding that relationship and how to leverage it to improve upon them is crucial to using AI successfully in your organization. 

As the relationship continues to grow, your employee is becoming a kind of AI supervisor. Over the last year, the concept of “Human-in-the-Loop” has become a commonplace, fundamental concept of AI. It refers it a form of human oversight of an AI system, such as an engineer adjusting a large AI system behind closed doors.  

However, if your employees are using systems like ChatGPT regularly, they are positioned to participate in HITL in ways that can be tremendously valuable. When an employee interacts with AI, they are not merely a passive user. They can actively become a part of its learning process. Every prompt, correction, and piece of feedback they provide it refines the AI, guiding it to better align itself with the employee’s goals. More simply, the employee becomes their own Human-in-the-Loop. 

Take my recent experience as an example. I asked ChatGPT to help me refresh a jazz guitar lesson plan. I’ve been studying jazz guitar for a couple of years now, and one of my goals is to get more comfortable with inverted chord voicings (for non-musicians, if a chord is the sum of its notes, changing the order of those notes can enrich your writing and playing). Initially, ChatGPT suggested I spend an extra 15 minutes a week drilling scales (individual notes, not chords). I was confused. That is not my goal here.

So, I told ChatGPT, “Thanks for the suggestion, but I’m not interested in scales right now. Let’s make sure we’re focusing on the goals I explicitly share so we can build a plan that better uses my time.” ChatGPT apologized. It then provided me with a set of inverted chord exercises that will keep me busy through April 2025. 

My interaction is a common one among ChatGPT users and it is important. As individuals, employees have the power to shape and improve AI outputs directly. AI is bound to make mistakes, like misunderstanding intentions and goals. The key is recognizing that limitations are not setbacks. They are opportunities for growth.  

When employees see themselves as in-the-loop with AI, they can take an active role in recognizing its limitations and pushing it to improve. They can be essential to refining AI and making it more aligned with their own or an organization’s needs. Each interaction, correction, and bit of guidance we provide helps the AI learn more effectively. 

Recognizing AI failures are sources of insight and growth is an important capability for any organization. Not only do failures reveal how systems are built, their tendencies, and their biases exist – it reveals a pathway for encouraging the system to develop and perform in a more efficient and more accurate manner in the future. 

Leveraging Failures as Opportunities 

AI failures provide critical insight into the dynamics of how humans develop relationships with AI. More specifically, the dynamics at play between systems that learn and human intention. Human decision-making is complex. It involves values, context, empathy, historical biases, and so on. AI can struggle in its inability to understand these subtle human complexities. This is part of what makes a Human-in-the-Loop so important. It ensures that human judgement and intent are integrated into an AI system’s learning process so that it becomes better at replicating human behaviours. 

For example, AI models used for job recruitment have been known to be biased against certain minority groups. It is crucial to identify why this bias exists and address it directly. A Human-in-the-Loop plays a critical role here. They can analyze data and algorithms to determine where the issue occurred, what parameters led to the unintended outcome and begin designing a solution.  

How You and Your Employees Can Be Their Your Own Human-in-the-Loop 

Here are three actionable tips that you, your colleagues, or any employee can follow to encourage tools like ChatGPT to improve, especially when it makes a mistake: 

Explain: Clearly point out what went wrong and why. Provide the rationale behind your thinking. This helps the system learn more effectively and align with your specific needs. 

Coach: If AI seems to misunderstand your request, guide it by rephrasing or breaking down your request into smaller components. This makes it easier for the tool to understand your intentions and learn from them. 

Validate: Positive reinforcement can help. When AI gets it right, acknowledge it. This validation encourages AI to replicate the improved behavior in the future. 

HITL as a Balanced Partnership, When Kept in Check 

Building a relationship with AI is a dynamic process. It is about fostering growth, accountability, and understanding. While employees can be their own Human-in-the-Loop, it is essential that there is at least an awareness and intention to align an employee’s guidance of AI with the organization's goals and priorities. Without this alignment, individuals adjusting and guiding AI may inadvertently trigger old or create new biases that may diverge from strategic objectives. Human-in-the-Loop is certainly a bridge that connects human values, expertise, and context with the computational power of AI – but it must be implemented thoughtfully.  

As organizations become more comfortable integrating AI, remember that Human-in-the-Loop is about creating synergy between human insight and machine learning. By maintaining human involvement, we ensure that AI evolves in a direction that benefits not only the organization but also its employees, customers, and broader stakeholders.