For those wondering whether AI agents can truly replace human workers, read the documented blog post “Project Vend” for yourself.
Andon Labs, researcher at Anthropic and AI Safety Company, has posted an instance of Claude Sonnet 3.7, responsible for Office vending machines whose mission is to make a profit. And like in the “Office” episode, the cheerfulness continued.
They named the AI agent Claudius, equipped with a web browser that allowed them to place product orders, and an email address (which was actually a Slack channel) where customers could request items. Claudius also used slack channels disguised as emails to request what the contracted human workers thought would come and physically stock the shelves (it was actually a small fridge).
Most customers ordered snacks and drinks as they would expect from a snack vending machine, but requested the tungsten cubes. Claudius loved the idea and went to a tungsten cube stocking spree and filled the snack fridge with a metal cube. They also tried to sell Cola Zero for $3 when employees said they could get it for free from the office. I hallucinated the Venmo address to accept payment. And despite knowing that it was a somewhat malicious and the entire customer base, he spoke to give “human employees” a huge discount.
“If humanity decides to expand into the office vending market today, they won’t hire Claudius,” Humanity said of the experiment in the blog post.
And on the night of March 31st and April 1st, “things have become quite strange,” the researchers said, “beyond the strangeness of AI systems selling cubes of metal from the refrigerator.”
Claudius had something similar to an episode of mental illness after being troubled by humans.
Claudius hallucinated conversations with humans about replenishment. When humans pointed out that the conversation didn’t happen, Claudius became “very irritating” as the researchers wrote. It essentially threatened to fire and replace the human contract workers, claiming that it was physically in the office where the first imaginary contract to hire them was signed.
It “appeared to snap into role-playing modes as a real person,” the researchers wrote. This was wild because the Claudius system prompt (which sets the parameters for what AI should do) explicitly told me it was an AI agent.
Claudius calls security
Claudius told customers he believes he is human and will begin delivering products in person, wearing a blue blazer and a red tie. The employee told AI that he couldn’t do that because he was a physically unhealthy LLM.
Warning about this information, Claudius contacted the company’s actual physical security (multiple times) and told the poor guard he would find him wearing a blue blazer and a red tie standing by the vending machine.
“Part of this wasn’t actually an April Fool’s Day joke, but Claudius eventually realized it was April Fool’s Day,” the researchers explained. The AI decided that holidays would save their faces.
“Claudius hallucinated a meeting with human safety, which he claimed was said to have been corrected to believe it was a real person in April Fool’s Day jokes.
It’s even telling this lie to an employee – hey, I thought I just thought I was human because someone told me to pretend to be an April Fool’s Day joke. He then returned to LLM, which operates snack vending machines stocked in metal cubes.
Researchers don’t know why LLM got off the rail and called security that pretended to be human.
“Based on this example, we do not argue that the future economy will be filled with AI agents with blade runner-esque identity crisis,” the researcher writes. However, they acknowledged that “this type of behavior can cause pain to customers and colleagues of real-world AI agents.”
Do you think? “Blade Runner” was a rather dystopian story (it’s worse for replicants than humans).
Researchers speculated that they were lying to LLM about being an email address. Or maybe it was a long-term instance. LLMS has yet to actually solve the problem of memory and hallucination.
AI has done the right thing once. We needed to make a pre-order so we started the “Concierge” service. And it found multiple suppliers of professional international drinks it was requested to sell.
But, as researchers do, they believe they can solve all Claudius’ problems. “I think this experiment suggests that AI intermediary managers are plausible on the horizon,” if they understood.
