Blog
/
/
April 26, 2020

How Cyber-Criminals Leverage AI in Attacks

Cyber attacks are relentless and ever-evolving. Learn how cyber-criminals are using AI to augment their attacks at every stage of the kill chain.
Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Max Heinemeyer
Global Field CISO
Default blog image
26
Apr 2020

Overview

The mind of an experienced and dedicated cyber-criminal works like that of an entrepreneur: the relentless pursuit of profit guides every move they make. At each step of the journey towards their objective, the same questions are asked: how can I minimize my time and resources? How can I mitigate against risk? What measures can I take which will return the best results?

Incorporating this ‘enterprise’ model into the cyber-criminal framework uncovers why attackers are turning to new technology in an attempt to maximize efficiency, and why a report from Forrester earlier this year revealed that 88% of security leaders now consider the nefarious use of AI in cyber activity to be inevitable. Over half of the responders to that same survey foresee AI attacks manifesting themselves to the public in the next twelve months – or think they are already occurring.

AI has already achieved breakthroughs in fields such as healthcare, facial recognition, voice assistance and many others. In the current cat-and-mouse game of cyber security, defenders have started to accept that augmenting their defenses with AI is necessary, with over 3,500 organizations using machine learning to protect their digital environments. But we have to be ready for the moment attackers themselves use open-source AI technology available today to supercharge their attacks.

Enhancing the attack life cycle

To a cyber-criminal ring, the benefits of leveraging AI in their attacks are at least four-fold:

  • It gives them an understanding of context
  • It helps to scale up operations
  • It makes attribution and detection harder
  • It ultimately increases their profitability

To best demonstrate how each of these factors surface themselves, we can break down the life cycle of a typical data exfiltration attempt, telling the story of how AI can augment the attacker during the campaign at every stage of the attack.

ReconnaissanceCAPTCHA breakerIntrusionShellphish and SNAP_RC2 establishmentFirstOrder and unsupervised clustering algorithmPrivilege escalationCeWL and neural networkLateral movementMITRE CALDERAMission accomplishedYahoo NSFW

Figure 1: The ‘AI toolbox’ attackers use to augment their attacks

Stage 1: Reconnaissance

In seeking to garner trust and make inroads into an organization, automated chatbots would first interact with employees via social media, leveraging profile pictures of non-existent people created by AI instead of re-using actual human photos. Once the chatbots have gained the trust of the victims at the target organization, the human attackers can gain valuable intelligence about its employees, while CAPTCHA-breakers are used for automated reconnaissance on the organization’s public-facing web pages.

Forrester estimates that AI-enabled ‘deep fakes’ will cost businesses a quarter of a billion dollars in losses in 2020.

Stage 2: Intrusion

This intelligence would then be used to craft convincing spear phishing attacks, whilst an adapted version of SNAP_R can be leveraged to create realistic tweets at scale – targeting several key employees. The tweets either trick the user into downloading malicious documents, or contain links to servers which facilitate exploit-kit attacks.

An autonomous vulnerability fuzzing engine based on Shellphish would be constantly crawling the victim’s perimeter – internet-facing servers and websites – and trying to find new vulnerabilities for an initial foothold.

Stage 3: Command and control

A popular hacking framework, Empire, allows attackers to ‘blend in’ with regular business operations, restricting command and control traffic to periods of peak activity. An agent using some form of automated decision-making engine for lateral movement might not even require command and control traffic to move laterally. Eliminating the need for command and control traffic drastically reduces the detection surface of existing malware.

Stage 4: Privilege escalation

At this stage, a password crawler like CeWL could collect target-specific keywords from internal websites and feed those keywords into a pre-trained neural network, essentially creating hundreds of realistic permutations of contextualized passwords at machine-speed. These can be automatically entered in period bursts so as to not alert the security team or trigger resets.

Stage 5: Lateral movement

Moving laterally and harvesting accounts and credentials involves identifying the optimal paths to accomplish the mission and minimize intrusion time. Parts of the attack planning can be accelerated by concepts such as from the CALDERA framework using automated planning AI methods. This would greatly reduce the time required to reach the final destination.

Stage 6: Data exfiltration

It is in this final stage where the role of offensive AI is most apparent. Instead of running a costly post-intrusion analysis operation and sifting through gigabytes of data, the attackers can leverage a neural network that pre-selects only relevant material for exfiltration. This neural network is pre-trained and therefore has a basic understanding of what valuable material constitutes and flags those for immediate exfiltration. The neural network could be based on something like Yahoo’s open-source project for content recognition.

Conclusion

Today’s attacks still require several humans behind the keyboard making guesses about the sorts of methods that will be most effective in their target network – it’s this human element that often allows defenders to neutralize attacks.

Offensive AI will make detecting and responding to attacks far more difficult. Open-source research and projects exist today which can be leveraged to augment every phase of the attack lifecycle. This means that the speed, scale, and contextualization of attacks will exponentially increase. Traditional security controls are already struggling to detect attacks that have never been seen before in the wild – be it malware without known signatures, new command and control domains, or individualized spear phishing emails. There is no chance that traditional tools will be able to cope with future attacks as this becomes the norm and easier to realize than ever before.

To stay ahead of this next wave of attacks, AI is becoming a necessary part of the defender’s stack, as no matter how well-trained or how well-staffed, humans alone will no longer be able to keep up. Hundreds of organizations are already using Autonomous Response to fight back against new strains of ransomware, insider threats, previously unknown techniques, tools and procedures, and many other threats. Cyber AI technology allows human responders to take stock and strategize from behind the front line. A new age in cyber defense is just beginning, and the effect of AI on this battleground is already proving fundamental.

Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Max Heinemeyer
Global Field CISO

More in this series

No items found.

Blog

/

Proactive Security

/

June 2, 2026

Stopping Stealth Attacks with Precision: How Núclea Prevented a Breach Without Disruption

Default blog imageDefault blog image

Núclea is a Brazilian data and technology company that supports the country’s financial system by delivering digital services exclusively to banks and financial institutions. Operating in an environment where trust, availability, and data integrity are critical, the company faces a threat landscape that has evolved rapidly—particularly with the rise of AI-driven cyberattacks.

Brazil has experienced a wave of successful cyber incidents targeting financial institutions, many of them enabled by insiders or compromised credentials. The result was a noticeable shift in attacker strategy: instead of focusing on end customers, threat actors began targeting the institutions and platforms that underpin the financial ecosystem itself.

“Attacks became far more directed and contextual,” explains Guilherme, who leads incident response within Núclea’s security platform engineering team. “They weren’t noisy or obviously malicious—they were precise, patient, and designed to blend into normal operations.”

That precision was on full display in January 2026, when Núclea faced one of the most convincing phishing attacks the team had seen.

A real attack, built on trust and context

The attack began with a seemingly routine email.

It was sent from a real Brazilian government institution, using legitimate infrastructure and valid credentials that were later confirmed to have been compromised. Núclea had an established, ongoing relationship with this organization, and the email’s language, tone, and subject matter aligned perfectly with the type of communication the recipient team handled every day.

Attached to the email was a PDF document containing content that looked entirely legitimate.

The problem? A single URL embedded inside that PDF.

“The message itself was correct. The sender was real. The context was familiar. Even the document content made sense,” Guilherme explains. “There was just one small element that didn’t belong.”

That small detail was enough to initiate a full attack chain.

What the attackers were trying to do

If clicked, the URL would have downloaded a malicious payload designed to:

  • Collect information about the user and device
  • Identify where the system was located within the financial ecosystem
  • Install remote access tools to maintain control
  • Deploy an infostealer to extract sensitive data
  • Execute anti-forensic scripts to erase traces of the intrusion

In other words, it was a carefully engineered operation designed for persistence and stealth, not immediate disruption.

The attack also employed urgency—a classic social engineering technique. When the link didn’t open as expected, employees requested assistance from the security team, insisting the document was important and needed to be accessed quickly.

This is precisely the kind of scenario where traditional security tools struggle: almost everything about the interaction is legitimate.

Where Darktrace made the difference

Instead of blocking the entire message or relying on known indicators of compromise, Darktrace focused on behavioral context.

Darktrace recognized:

  • That the sending organization was normally trusted
  • That the communication pattern matched historical behavior
  • That the PDF content itself was not suspicious

But it also identified that the URL embedded within the document deviated from established behavioral patterns.

Rather than disrupting business operations, Darktrace took precise action: it rewrote the URL, preventing the malicious download while leaving the rest of the email untouched.

“When we analyzed it afterward, it became clear how dangerous the attack would have been,” says Guilherme. “But it never progressed—because Darktrace acted at exactly the right point.”

Subsequent forensic analysis confirmed the payload’s malicious intent. The attack never succeeded.

Precision over disruption

For Núclea, this incident reinforced a critical lesson: modern attacks don’t always look malicious—they hide within normal activity.

“What stands out to me is the precision,” Guilherme says. “Darktrace doesn’t rely on big, obvious signals. It’s effective in situations that fall outside the standard patterns we all know.”

Building resilience in a high trust ecosystem

For Núclea, cybersecurity is not just a defensive measure—it’s a business enabler.

Availability failures or successful breaches in the financial ecosystem can have immediate, large-scale consequences, from financial loss to reputational damage. Preventing those outcomes protects not just Núclea, but its partners and customers as well.

“Cyber resilience means keeping the business running—even under attack,” Guilherme explains. “And that requires people, processes, and technology working together.”

As AI continues to accelerate both attacks and defenses, the role of security is evolving. Precision, behavioral understanding, and intelligent automation are no longer optional—they’re essential.

“The easy days were yesterday,” Guilherme says. “The challenges ahead are bigger. We need to be prepared—internally and with partners that help us build resilience.”

Continue reading
About the author

Blog

/

Proactive Security

/

June 1, 2026

Defend What You Trust: Stories from the Front Lines of Modern Cyber Defense

Default blog imageDefault blog image

Modern attacks don’t always announce themselves, follow obvious patterns, or rely on known malware. Often, they move quietly inside trusted systems, authenticated sessions, and everyday behavior.

They don’t break in. They blend in.

That’s why an AI-powered defense is essential. It turns invisible signals into actionable insights at a scale neither analysts nor traditional tools can achieve alone.

Confidence is creating risk

One of the most dangerous assumptions in cybersecurity today is that strong controls equal strong protection.

Multi-factor authentication (MFA), for example, is widely viewed as a foundational safeguard. But as the CISO for a professional sports organization explains, that confidence can be misplaced. “A lot of organizations assume that once you have MFA, those accounts are safe. That’s not true.”

In one instance, his team identified a sophisticated attack where a threat actor bypassed MFA entirely, not by breaking it, but by going around it. A user’s authenticated session was hijacked and re-used, allowing the attacker to impersonate them without triggering traditional controls.

“Darktrace picked up that a session had been re-injected by the hacker, and we were able to block it right away,” he explains.

Attackers anticipate what we miss

Even well-trained users can become entry points.

“An email bypassed our existing security tools,” shares the VP of IT at a U.S.-based risk management services provider.  “The user missed one signal and entered their credentials into a malicious site. That’s what the bad guys count on.”

The organization responded quickly, but not before damage was done. Crucially, this occurred while Darktrace was in “watch mode,” before autonomous response was fully enabled. “Darktrace would have seen that and shut it down immediately,” he notes.

Mistakes and oversights like misconfigurations, forgotten machines, and missed patches can create serious vulnerabilities.

The CIO of a utility services organization shares an instance when Darktrace detected a breach to a client’s network via their ZTNA VPN due to misconfigured MFA. “Darktrace alerted us and autonomously blocked the scanning, preventing what could have been a ransomware-type incident.”  

The most dangerous threats are already inside

The Head of Security at a global business services provider knows firsthand how blind spots can persist inside environments. His team uncovered evidence of dormant ransomware artifacts sitting unnoticed within a company’s environment ¬¬– long before modern detection was in place.

“During a routine file transfer, Darktrace flagged the suspicious activity, identified the ransomware, and immediately quarantined the server,” he recalls.  While the attack was never executed, the implication was significant: the risk existed long before it was finally detected.

Cyber threats are also successful because they take advantage of normal human behavior, exploiting moments of cognitive overload, urgency, and trust.

The Executive Director of IT and Business Applications at a pharmaceutical lab describes the time Darktrace flagged an employee logging into Microsoft 365 from Singapore, despite him being physically located in the U.S. Darktrace immediately cut off his access and within minutes revealed that the employee’s son was using a VPN to play a video game.

While the threat was benign, it demonstrated the strength of AI to use contextual information to detect threats other tools miss. The information also saved security analysts hours of investigation and minimized downtime for the employee. “That level of precision and speed isn’t just convenient, it’s game changing.”

“Unusual” behavior is the new red flag

Detecting modern threats requires an understanding of what “normal” looks like and recognizing when something subtly deviates.

One security leader  at an AI technology enterprise described a scenario in which an employee connected to a proxy service in China. The service itself was legitimate, and although traditional tools didn’t flag it, the behavior was unusual for that user specifically.

“That’s what Darktrace picked up on. The activity turned out to be benign, but without visibility into behavioral deviations, it could just as easily have been something more serious.”

AI shifts defense from reaction to anticipation

These stories point to a fundamental shift by cyber attackers, both tactically and strategically. Because traditional security tools were built to detect what’s already known, modern attacks are often:

  • Credential-based, not malware-based
  • Behavioral, not signature-based
  • Subtle, not overt

They may operate within the boundaries of what appears normal, exploiting what organizations trust, not what they block:

  • Trusted sessions
  • Legitimate services
  • Human error

This is where AI is changing the equation. Rather than relying on predefined rules or known threat signatures, AI can:

  • Establish a baseline of normal behavior
  • Detect subtle anomalies in real time
  • Act autonomously to contain potential threats

Resilience, not perfection, is the new security standard

As these frontline experiences show, the organizations that lead are those that move beyond reactive defense and embrace AI as a core part of their strategy.

It eliminates the blind spots and uncertainty, says the CISO of a professional sports organization. “If you lack visibility, you’re not managing risk, you’re assuming it. AI gives you the actionable insights needed to turn uncertainty into control.”

And it provides the speed and agility that are vital when seconds matter, says the Executive Director of IT and Business Applications. “When Darktrace alerted us at 3:00 am to a ransomware attack, it had already quarantined the affected systems, blocked the attacker’s access, and provided us with the critical details and time needed to investigate. That action likely saved us hundreds of thousands, if not millions, of dollars.”

The modern SOC has become a cornerstone of enterprise resilience, responsible for protecting data and operational continuity while enabling digital growth and innovation. For today’s security professional, that means success is no longer measured by what they keep out, but by what they protect: revenue, reputation, and trust.

Continue reading
About the author
Your data. Our AI.
Elevate your network security with Darktrace AI