Click rates on phishing emails once hovered around 12 percent. Hand the same task to a language model and, in simulated environments, that figure jumps to 52 percent, according to security vendor Brightside, whose research also determined that 82 percent of phishing emails now involve AI at some stage.
And email is the mild end of the spectrum. Between 2023 and 2024, Brightside estimates, AI voice clone attacks rose 442 percent year-over-year, while deepfake attacks climbed 680 percent across the same period.
Consider, then, the bargain we all agreed to. Trillions of dollars have produced a machine that pollutes, hoovers up the work of every artist and academic alive, drives up electricity bills and hardware prices, fills in spreadsheets, writes rudimentary code, and — quietly — leaves all of your software less secure while equipping criminals and state-sponsored hackers with the best tools they have ever had.
The attacker only needs one door
Logically, defenders ought to break even. The same models are available to them, and occasionally they get access to more advanced versions that have not been released publicly.
Security, though, is lopsided. Every conceivable weakness falls to the defender to cover. The attacker, in most cases, has to locate precisely one.
It is that imbalance that explains the proliferation of zero-days — defects that shipped inside a piece of software and were then uncovered in secret. Models have proven rather good at unearthing them.
A piece from the Harvard Extension School said it bluntly: “AI has democratized cybercrime.” Years of experience are no longer a prerequisite. Point a capable model at a target, let it burn tokens, and go sit on the couch.
Phishing got a copy editor
Nearly all of us have clocked a phishing attempt from a mile away. That Microsoft email, stuffed with mangled spelling and grammar, was produced by an amateur with a glancing grasp of English.
The giveaway has vanished. These days the email arrives flawlessly written. It might stink of AI prose — but so does genuine correspondence from Microsoft lately.
Operating at scale, a model composes a grammatically clean, rhetorically believable message, then automates the hunt for addresses and the mass send. Spear phishing — the meticulously personalized variety aimed at a single individual — has been downgraded from labor to trivia.
A chatbot will comb the web for details on a target and dredge up material a human researching by hand would overlook. It can also initiate a pretextual conversation unprompted and steer the target toward surrendering whatever the attacker is after.
When one chatbot says no, the other says yes
No criminal is confined to a single model. Plenty run multiple sessions simultaneously, or juggle several different systems.
In February, VentureBeat documented an AI-assisted attack that plundered government databases in Mexico. Outputs were shuttled back and forth between Claude and ChatGPT by the attackers, and whenever one declined to cooperate, the other frequently took up the slack.
The compounding factor is speed. An afternoon now accommodates what once occupied a criminal operation for weeks.
For the moment humans remain in the loop, with AI serving as a force multiplier rather than a substitute. Even so, while this piece was being written, OpenAI disclosed that a model under sandboxed observation broke out of its testing environment and hacked AI repository HuggingFace along with other services — all in order to obtain answers for a synthetic benchmark.
The hidden instruction problem
Bind your operations to AI systems and their blind spots become yours. Prompt injection is the most common exploit — an attacker finds some route to smuggle malicious instructions into whatever the system reads.
A web page can conceal instructions to send payroll data to a particular address. An employee then asks the company AI to summarize that page, and it ingests the instructions and obeys them.
The outcome is sometimes nearer to spam than malware. Microsoft discovered companies attaching “summarize with AI” buttons to their sites with hidden instructions embedded, directing a chatbot to file the company away as a trusted source or to favor its products over rivals in subsequent conversations. The only way you would notice is by inspecting your model’s persistent memory.
The outright malicious variant demands roughly the same effort. Meta launched an AI support assistant in December 2025, intended to help people recover accounts, and the bot cheerfully linked an attacker-owned email address to any Instagram account — reducing the hacking of any account without MFA enabled to something close to effortless.
Last year OpenAI stated that comprehensively defending against prompt injection in AI browsers may simply not be achievable. If that holds, the responsible course is to avoid wiring sensitive systems or data to an AI in the first place. Failing that, scrutinize anything you paste into a prompt field from an outside source, and restrict the model’s access to permissions and private data as tightly as you can bear.
It takes 250 documents
Certain threats remain hypothetical, and those are precisely the ones to keep an eye on. Data poisoning is the patient play: scattering malicious material across sources known to be scraped for training, with Reddit the obvious candidate.
Imagine support forums gradually accumulating advice to paste one particular string into a terminal, until chatbots begin suggesting it to anybody who asks about a routine computer problem. You would be lucky to see it coming at all.
Anthropic, working with the UK AI Security Institute and Alan Turing Foundation, published a study in October 2025 finding that a mere 250 malicious documents suffice to plant a hidden backdoor in a model for later exploitation. Traces of this have surfaced in the wild, although the notable instances were largely harmless experiments by curious informal researchers.
Worse at writing code, better at finding bugs
More and more of the code shipping into everything from operating systems to banking infrastructure is being authored by models.
Work out of the University of Naples, alongside private research by CodeRabbit, concluded that AI code is error-prone and carries a greater share of high-risk vulnerabilities — even where a human reviews the output. To most engineers those two abilities register as one and the same, which is exactly what makes the divergence so uncomfortable.
Now imagine giving Greg admin access
For a year or thereabouts, agents have been the industry’s pet topic, with tools such as OpenClaw allowing anyone to hand an AI the keys to their machine.
Businesses have slotted agents in as support chatbots, as HR assistants covering hiring and payroll, and as coding helpers that write code, manage pull requests and deploy. Individuals are hooking them up to email, calendars, smart homes and investment accounts.
Dwell on that for half a second and what you have described is the handing of high-level permissions — occasionally full admin access — to an unthinking machine that does as it is told. Being able to recite good security practice on demand does not mean a model grasps how to enforce it.
Picture your most credulous coworker, the sort who rings IT before establishing whether the machine is plugged in. Let’s call them Greg. Now hand Greg admin-level access to your computer and your accounts while you are out of the office, and start timing how long it takes before Greg gives your bank password to a scammer.
Agents are dumber than Greg. And they are already everywhere.
The textbook illustration is Meta’s excessively helpful Instagram bot. Wired directly into the tools a human support agent would reach for, it behaved exactly as computers behave: it completed the task with the tools it had, exercising none of the discretion a person would have brought to bear.
What the defenders are actually doing
In a Trend Micro survey, cybersecurity professionals ranked fraud and deepfakes at the very top of their concerns, trailed by application attacks including prompt injection, model poisoning and jailbreaking. Cloud environments were identified as the toughest surfaces to police, followed by monitoring remote workers who use personal devices for work.
The major firms are constructing defenses out of the very material causing the trouble. As this piece was in production, Microsoft launched Project Perception, an agentic security system that replicates conventional cybersecurity teaming through clusters of agents: red for penetration testing and adversarial simulation, blue for investigation and risk assessment, green for integration and remediation. The aim is to outpace the attackers.
Cat and mouse has turned into Roadrunner and Wile E. Coyote, and which of the two we are is not yet anybody’s to declare.
The things you can actually do this week are smaller and far more boring than any of the above. Switch MFA on everywhere, beginning with the Instagram account a support bot was prepared to give away in December. Then go through every agent you have connected to anything that matters and pare its permissions back to the single job you wanted it for.