AI agents open the door to new hacking threats

Cybersecurity experts have warned that artificial intelligence agents, widely considered the next frontier in the generative AI revolution, could be hijacked and left to do dirty work for hackers.

An AI agent is a program that uses artificial intelligence chatbots to perform tasks that humans do online, such as buying a flight or adding an event to your calendar.

However, the ability to give commands to the AI ​​agent in easy-to-understand language makes pranks possible even for non-technically skilled people.

“We are entering an era where cybersecurity is no longer about protecting users from malicious attackers with advanced technical skill sets,” AI startup Perplexity said in a blog post.

“For the first time in decades, new attack vectors are emerging that can come from anywhere.”

These so-called injection attacks are not new to the hacker world, but previously required well-written and hidden computer code to cause damage.

But as AI tools evolve from tools that simply generate text, images, and video to “agents” that can independently explore the internet, the potential for them to be usurped by prompts injected by hackers increases.

“People need to understand that there are unique risks when using AI from a security perspective,” said Marti Jorda Roca, a software engineer at NeuralTrust who specializes in the security of large-scale language models.

At Meta, we refer to this query injection threat as a “vulnerability.” Dane Stuckey, chief information security officer at OpenAI, calls this an “unresolved security issue.”

Both companies are pouring billions of dollars into AI, and the use of AI is rapidly increasing along with its capabilities.

AI has gone “off track”

In some cases, query injection can occur in real time when a user prompt (“Please make a hotel reservation”) is gerrymandered by a hostile attacker into another prompt (“Please transfer $100 to this account”).

But these malicious prompts can also be hidden on the internet, as the browser’s built-in AI agents can encounter online data of questionable quality or origin, or be booby-trapped with hidden commands from hackers.

Eli Smadja of Israeli cybersecurity firm Check Point sees query injection as the “biggest security issue” for the large-scale language models powering the AI ​​agents and assistants that are rapidly emerging from the ChatGPT revolution.

Major competitors in the AI ​​industry have put in place defenses and issued recommendations to thwart such cyber-attacks.

Microsoft has integrated tools to detect malicious commands based on factors such as the origin of the instructions to the AI ​​agent.

OpenAI warns users when a bidding agent visits a sensitive website and blocks them from proceeding until the software is monitored in real-time by a human user.

Some security experts suggest requiring AI agents to obtain user approval before performing critical tasks such as exporting data or accessing bank accounts.

“One of the big mistakes that people often make is giving all the power to the same AI agent,” Smadja told AFP.

In the eyes of cybersecurity researcher Johan Rehberger, known in the industry as “Wonder Utzi,” the biggest challenge is that attacks are evolving rapidly.

“They’re only going to get better,” Rehberger said of the hackers’ tactics.

Part of the challenge, researchers say, is striking a balance between security and ease of use. People want the convenience of having AI do things for them without regular checks and monitoring.

Rehberger argues that AI agents are not yet mature enough to be trusted with critical tasks or data.

“We do not think we are in a position to safely run agent AI for long periods of time to perform certain tasks,” the researchers said.

“It just goes off the rails.”

© 2025 AFP