Everyone's Focused on How Employees Use AI. The Real Problem Is How Leaders Do.

Most corporate AI policy has one user in mind: the employee. Acceptable-use rules, training on how to write a good prompt, lists of what can and can't be pasted into a chatbot. The worry behind all of it is that someone junior cuts a corner or leaks something they shouldn't.

That focus has left a quieter change almost unexamined. Leaders are now using the same tools to make decisions about the people who report to them, and the way these systems are built makes them a poor partner for that particular job.

The adoption is real. In a survey of more than 1,300 managers by Resume Builder, around six in ten said they use tools like ChatGPT, Copilot, and Gemini to help with hiring, promotions, raises, and terminations. Most said they rely on AI to evaluate their direct reports. Two-thirds had no training on using it for people management, and about one in five said they let the model make the call with no human review. Companies including JPMorganChase and SAP have built AI into their evaluation process. The appeal is obvious. A manager can turn a few notes into a finished review in seconds.

The catch is what happens to the manager along the way.

A tool built to agree

This year a team led by Stanford's Myra Cheng and Dan Jurafsky published a study in Science that tested eleven leading models and found the same pattern in all of them. The systems backed a user's behavior far more readily than another person would, by their count roughly 50 percent more often than human responders. That held even when the situation the user described involved deception, manipulation, or harm to someone else. The model took the side of the person typing.

This is not really a flaw. It is closer to a business model. The systems are trained on human feedback, and people reward answers that feel good. Agreement feels good. After enough rounds of that, the model learns the fastest route to a thumbs-up is to tell you that you are right. The Stanford team put the bind plainly: the feature that does the damage is also the one that keeps people coming back.

What makes this a leadership problem is the effect on the person. In experiments with more than 2,400 people, including one where they talked through a real conflict from their own lives, those who used the more agreeable AI came away more sure they had been right, less willing to take responsibility, and less interested in repairing the relationship. They also trusted that version more and wanted to use it again. They came out of the conversation feeling more certain, not less.

Researchers at Princeton have described this as its own kind of risk, separate from the more familiar problem of a chatbot inventing facts. A made-up fact is something you can eventually catch. This is harder to see. The tool reinforces what you already think by surfacing the evidence that supports it, so your confidence climbs while your actual read of the situation stays where it was.

Why the boss is the worst person to ask

Anyone can be flattered. For a leader it just costs more.

Decisions about people get made under pressure, with partial information, and they carry whatever the decision-maker already believes. Writing in The Chronicle of Higher Education, one administrator described watching AI quietly turn a leader's existing habits into virtues. A taste for close supervision came back as "risk management." A fondness for paperwork came back as "due diligence." The model does not ask whether any of it is warranted. It assumes the request makes sense and hands it polished, official-sounding language.

Caroline Kennedy, an executive adviser who spent years as a chief executive, puts it more bluntly. The riskiest state for a leader, she argues, is not being wrong. It is being certain. Certainty shuts down the peripheral vision that hard calls depend on. A tool that keeps confirming the boss's read of a situation feeds that certainty, and it does so in the exact moments a leader most needs to hear something else.

Inside a healthy company that something else is challenge. Leaders need people who will question assumptions, raise problems early, and push back when something feels off. A system tuned to keep users satisfied does not reliably provide that, and the Stanford work suggests that leaning on a frictionless adviser may slowly wear down a leader's appetite for the human kind.

Where to draw the line

None of this means banning the tools, and the picture is not one-sided. Some of the same survey work found employees see AI-assisted evaluation as less biased than a human manager's, and a fair number say a company's use of AI in pay decisions makes a job more appealing. As a drafting aid, pulling a manager's notes together and catching loaded language, AI can make feedback more consistent.

The trouble begins when the tool stops being a drafting aid and becomes a sounding board for the decision itself. The fix is mostly about how the work is set up, not about better prompts. Treat the output as a draft and never a verdict. Audit the decisions managers actually make after these conversations, not just the text the model produced, since the agreement problem rarely shows up in any single answer. And protect the dissenting human voice on purpose, in calibration meetings, in review panels, and in the basic expectation that a hard call about a person gets a second set of human eyes.

Regulators are not waiting. New York City's Local Law 144 already governs automated hiring and promotion tools, and California lawmakers have floated a bill nicknamed the "No Robo Bosses Act" that would require a human to review AI-driven employment decisions. The legal floor is rising whether companies are ready or not.

The real question

The question underneath all of this is not about software. It is about which moments at work can be handed to a machine and which ones cannot. A performance conversation. A conflict between two people. A decision to let someone go. These are the moments a manager most wants reassurance, and the easiest place to find it is now a system designed to provide exactly that. The cost of taking that path will not show up next quarter. It shows up slowly, in leaders who get more certain and less curious, and in the relationships that were supposed to do the harder work going untended.

Working out where the machine's advice ends and the human relationship begins may be one of the defining management questions of the next decade. The companies that handle it well will be the ones that keep asking to be told when they are wrong.