The High Cost of Low-Level AI Hacks
People are outsmarting AI to rig reservations or write code for free. Did firms focus too much on high-level AI errors?
A car dealer’s AI-driven program sells a car… for $1. An AI agent gets its user a spot in a Pilates class by hacking the gym’s website (and kicking out another member). And then there’s the fast-food outlet’s chatbot that customers are using not to order hamburgers but to debug sophisticated computer code.
These situations—all of them documented—are of course not nearly as catastrophic as starting a nuclear war, breaking the world’s financial system, or another of the AI-related doomsday scenarios many people fear. But experts say that users’ ability to evade safeguards is frustrating the non-technology organizations that are on pace to lay out $280 billion on AI this year alone. Low-level though they may be, these snafus can still be damaging to both a firm’s reputation and its bottom line. Worse, they could become an even bigger problem as AI tools become increasingly powerful and ubiquitous. “These are somewhat funny examples, but it’s deadly serious,” says Bryan Ackermann, Korn Ferry’s head of AI strategy and transformation.
These low-stakes misuses of the tech have happened to firms that are both new to and experienced with AI. A common hiccup is an AI agent oversharing—essentially, forwarding content the recipient was never entitled to see. Others, like the car-sales example, revolve around a concept called “helpful scope creep,” in which an AI does the thing it’s been asked to do—followed by four other unsanctioned tasks.
Ironically enough, the problems are human related, experts say. Some leaders have become entranced by AI’s potential capabilities. Becoming infatuated with a shiny new tool, AI-infused or not, rather than examining its potential flaws, is a very human response, says Julia Maddox, Korn Ferry’s global lead of AI consulting transformation. “When we get a first result that looks professional and final, we intuitively trust it,” she says. It’s why organizations should ask people from multiple perspectives to vet AI tools.
Often these mistakes occur when the AI agent is told to do something, but not given guardrails. Take the gym-class situation: One could argue that the tech did its job, says Mirka Kowalczuk, Korn Ferry’s senior vice president of global change enablement; the only problem was that the AI hadn’t been told that hacking into another company was not OK. “We cannot assume an agent knows what it cannot do, or that it has built-in ethics guardrails in the way we would expect from people,” Kowalczuk says. Experts say it’s critical that before an AI is deployed, internally or externally, that it knows—and follows—the security and ethics rules of the organization itself.
A good tip, Ackermann, is that organizations should instruct AI agents to not act “weird”—and define what that means. The company should be able to flag and review any AI actions that come close to that definition. It’s also imperative that firms catch these issues early, says Shanda Mints, Korn Ferry’s vice president of AI strategy and transformation. “AI can scale very quickly, but if you make a mistake in the process, that mistake also scales very quickly,” she says.
Learn more about Korn Ferry’s AI in the Workplace capabilities.
