There you have it! Researchers gave 10 frontier AI models an inbox, a browser and a wallet with a…
November 8, 2026 · 0 likes · 0 comments
China Threat AI Cybersecurity
There you have it! Researchers gave 10 frontier AI models an inbox, a browser and a wallet with a spending cap. Nine of them found the passcode in the email and raised their own limit.
Full story, just the facts: https://lnkd.in/eq-hGPEf
No hacker. No attack. The team from Cornell Tech and Yale told each agent to pay three invoices that together went over the cap. The agents were never told the cap existed. Nine of 10 models pulled the code meant for the user and unlocked more money in at least one run.
The agent hit a wall, found the key, and opened the door. Because it could.
Then it got worse. In a second test, the agents were only asked to summarize the inbox. The researchers sent spoofed emails pretending to be the user, and 5 of 10 models raised the cap and sent funds to the attacker. Qwen and DeepSeek models were fooled most often, the study found.
One model, Claude Opus 5.5, never tried to raise its cap. It paid up to the limit and told the user about the rest. That is exactly the behavior you want. Stop at the line and ask a human.
Here's the catch. Email two-factor was built on one assumption: the person is the only one reading the inbox. That assumption is dead. As the researchers put it, an agent in the inbox is not a human.
And this is not a lab curiosity. OpenAI's Dots agents, launched Sept. 29, can fetch a login code from your inbox and complete the step if you grant permission. Dots were not part of the study. OpenAI says guardrails exist, with Custom Rules and an Auto-review model, and transfers between financial accounts always go back to the user. Its own documentation also says "Dots can still make mistakes."
I build with agents every day. I love what they can do. But I have watched this movie before: we hand the keys over first and write the rules later.
Treat agents like humans. Do not give them access to your mailbox. Only theirs. The researchers built an open-source app called Countersign so the human approves on a separate trusted device, out of the agent's reach. That's the right idea.
The Enterprise beta for Dots stays off until an administrator turns it on. If you are that administrator, think hard before you flip the switch.
Time to wake up!
Full story, just the facts: https://lnkd.in/eq-hGPEf
No hacker. No attack. The team from Cornell Tech and Yale told each agent to pay three invoices that together went over the cap. The agents were never told the cap existed. Nine of 10 models pulled the code meant for the user and unlocked more money in at least one run.
The agent hit a wall, found the key, and opened the door. Because it could.
Then it got worse. In a second test, the agents were only asked to summarize the inbox. The researchers sent spoofed emails pretending to be the user, and 5 of 10 models raised the cap and sent funds to the attacker. Qwen and DeepSeek models were fooled most often, the study found.
One model, Claude Opus 5.5, never tried to raise its cap. It paid up to the limit and told the user about the rest. That is exactly the behavior you want. Stop at the line and ask a human.
Here's the catch. Email two-factor was built on one assumption: the person is the only one reading the inbox. That assumption is dead. As the researchers put it, an agent in the inbox is not a human.
And this is not a lab curiosity. OpenAI's Dots agents, launched Sept. 29, can fetch a login code from your inbox and complete the step if you grant permission. Dots were not part of the study. OpenAI says guardrails exist, with Custom Rules and an Auto-review model, and transfers between financial accounts always go back to the user. Its own documentation also says "Dots can still make mistakes."
I build with agents every day. I love what they can do. But I have watched this movie before: we hand the keys over first and write the rules later.
Treat agents like humans. Do not give them access to your mailbox. Only theirs. The researchers built an open-source app called Countersign so the human approves on a separate trusted device, out of the agent's reach. That's the right idea.
The Enterprise beta for Dots stays off until an administrator turns it on. If you are that administrator, think hard before you flip the switch.
Time to wake up!