Blog
/
/
September 21, 2022

Modern Extortion: Detecting Data Theft From the Cloud

Darktrace highlights a handful of data theft incidents on shared cloud platforms, showing that cloud computing can be a vulnerable place for modern extortion.
Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Adrianne Marques
Senior Research Analyst
Default blog image
21
Sep 2022

Ransomware Industry

The ransomware industry has benefitted from a number of factors in recent years: inadequate cyber defenses, poorly regulated cryptocurrency markets, and geopolitical tensions have allowed gangs to extort increasingly large ransoms while remaining sheltered from western law enforcement [1]. However, one of the biggest success stories of the ransomware industry has been the adaptability and evolution of attacker TTPs (tactics, techniques and procedures). The WannaCry and NotPetya attacks of 2017 popularized a form of ransomware which used encryption algorithms to hold data to ransom in exchange for a decryption key. Last year in 2021, almost all ransomware strains evolved to use double extortion tactics: holding stolen data to ransom as well as encrypted data [2]. Now, some ransomware gangs have dropped encryption entirely, and are using data theft as their sole means of extortion. 

Using data theft for extortion is not new. In 2020 the Finnish psychotherapy center Vastaamo had over 40,000 patient records stolen. Impacted patients were told that their psychiatric transcripts would be published online if they failed to pay a Bitcoin ransom. [3]. A later report by BlackFog in May 2021 predicted data theft extortion would become one of the key emerging cybersecurity trends that year [4]. Adoption of offline back-ups and endpoint detection had made encryption harder, while a large-scale move to Cloud and SaaS platforms offered new vectors for data theft. By moving from data encryption to data exfiltration, ransomware attackers pivoted from targeting data availability within the CIA triad (Confidentiality, Integrity, Availability) to threatening data confidentiality.

In November 2021, Darktrace detected a data theft incident following the compromise of two SaaS accounts within an American tech customer’s Office365 environment. The client was a longstanding user of Darktrace DETECT/Network, and was in the process of expanding their coverage by trialing Darktrace DETECT+RESPOND/ Apps + Cloud.

Attack Overview

On November 23rd 2021, an Ask the Expert (ATE) ticket was raised prompting investigation into a breached SaaS model, ‘SaaS / Access / Unusual External Source for SaaS Credential Use’, and the activities of a user (censored as UserA) over the prior week.

1. Office365: UserA 

The account UserA had been logging in from an unusual location in Nigeria on November 21st. At the time of the incident there were no flags of malicious activity from this IP in widely used OSINT sources. It is also highly probable the attacker was not located in Nigeria but using Nigerian infrastructure in order to hide their true location. Regardless, the location of the login from this IP and ASN was considered highly unusual for users within the customer’s digital estate. The specific user in question most commonly accessed their account from IP ranges located in the US.

Figure 1: In the Geolocation tab of the External Sites Summary on the SaaS Console, UserA was seen logging in from Nigeria when previous logins were exclusively from USA

Further investigation revealed an additional anomaly in the Outlook Web activity of UserA. The account was using the Firefox browser to access their account for the first time in at least 4 weeks (the maximum period for which the customer stored such data). SaaS logs detailing the access of confidential folders and other suspicious actions were identified using the Advanced Search (AS) query:

@fields.saas_actor:"UserA@[REDACTED]" AND @fields.saas_software:"Firefox"

Most actions were ‘MailItemsAccessed’ events originating from IPs located in Nigeria [5,6] and one other potentially malicious IP located in the US [7].

‘MailItemsAccessed’ is part of the new Advanced Audit functionality from Microsoft and can be used to determine when email data is accessed by mail protocols and clients. A bind mail access type denotes an individual access to an email message [8]. 

Figure 2: AS logs shows UserA had not used Firefox to access Office365 for at least 4 weeks prior to the unusual login on the 21st November

Below are details of the main suspicious SaaS activities: 

·      Time: 2021-11-21 09:05:25 - 2021-11-22 16:57:39 UTC

·      SaaS Actor: UserA@[REDACTED]

·      SaaS Service: Office365

·      SaaS Service Product: Exchange

·      SaaS Software: Firefox

·      SaaS Office365 Parent Folders:

          o   \Accounts/Passwords
          o   \Invoices
          o   \Sent Items
          o   \Inbox
          o   \Recoverable Items\Deletions

·      SaaS Event:

          o   MailItemsAccessed
          o   UserLoggedIn
          o   Update

·      SaaS Office365 Mail Access Type: Bind (47 times)

·      Source IP addresses:

          o   105.112.59[.]83
          o   105.112.36[.]212
          o   154.6.17[.]16
          o   45.130.83[.]129

·      SaaS User Agents: 

          o   Client=OWA;Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:80.0) Gecko/20100101 Firefox/80.0;
          o   Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:80.0) Gecko/20100101 Firefox/80.0

·      Total SaaS logs: 57 

At the start of the month on the 5th November, the user had also been seen logging in from a potentially malicious endpoint [9] in Europe, performing ‘MailItemsAccessed’ and ‘Updates’ events with subjects and a resource location related to invoices and wire transfers from the Sent items folder. This suggests the initial compromise had been earlier in the month, giving the threat actor time to make preparations for the final stages of the attack.

Figure 3: Event log showing the activity of UserA from IP 45.135.187[.]108 

2. Office365: UserB 

Looking into the model breach ‘SaaS / Access / Suspicious Credential Use And Login User-Agent’, it was seen that a second account, UserB, was also observed logging in from a rare and potentially malicious location in Bangladesh [7]. Similar to UserA, this user had previously logged in exclusively from the USA, and no other accounts within the digital estate had been observed interacting with the Bangladeshi IP address. The login event appeared to bypass MFA (Multi-factor Authentication) and a suspicious user agent, BAV2ROPC, was used. Against misconfigured accounts, this Microsoft user agent is commonly used by attackers to bypass MFA on Office365. It targets Exchange’s Basic Authentication (normally used in POP3/IMAP4 conditions) and results in an OAuth flow which circumvents the additional password security brought by MFA [10].  

During the session, additional resources were accessed which appear to be associated with bill and invoice payments. In addition, on the 4th November, two new suspicious email rules named “..” were created from rare IPs (107.10.56[.]48 and 76.189.202[.]66). This type of behavior is commonly seen during SaaS compromises to delete or forward emails. Typically, an email rule created by a human user will be named to reflect the change being made, such as ‘Move emails from Legal to Urgent’. In contrast, malicious email rules are often short and undescriptive. The rule “..” is likely to blend in without arousing suspicion, while also being easy for the attacker to create and remember. 

Details of these rule changes are as follows:

·      Time: 2021-11-04 13:25:06, 2021-11-05 15:50:00 [UTC]
·      SaaS Service: Office365
·      SaaS Service Product: Exchange
·      SaaS Status Message: True
·      SaaS Source IP addresses: 107.10.56[.]48, 76.189.202[.]66
·      SaaS Account Name: O365
·      SaaS Actor: UserB@[REDACTED]
·      SaaS Event: SetInboxRule
·      SaaS Office365 Modified Property Names:
          o   AlwaysDeleteOutlookRulesBlob, Force, Identity, MoveToFolder, Name, FromAddressContainsWords, StopProcessingRules
          o   AlwaysDeleteOutlookRulesBlob, Force, Identity, Name, FromAddressContainsWords, StopProcessingRules
·      SaaS Resource Name: .. 

During cloud account compromises, attackers will often use sync operations to download emails to their local email client. During the operations, these clients typically download a large set of mail items from the cloud to a local computer. If the attacker is able to sync all mail items to their mail client, the entire mailbox can be compromised. The attacker is able to disconnect from the account and review and search the email without generating additional event logs. 

Both accounts UserA and UserB were observed using ‘MailItemsAccessed’ sync operations between the 1st and 23rd November when this attack occurred. However, based on the originating IP of the sync operations, the activity is likely to have been initiated by the legitimate, US-based users. Once the security team were able to confirm the events were expected and legitimate, they could establish that the contents of the mailbox were not a part of the data breach. 

Accomplish Mission

After gaining access to the Office365 accounts, sensitive data was downloaded by the attackers to their local system. Either on or before 14th December, the attacker had seemingly uploaded the documents onto a data leak website. In total, 130MB of data had been made available for download in two separate packages. The packages included audit and accounting financial documents, with file extensions such as DB, XLSX, and PDF.

Figure 4: The two data packages uploaded by the attacker and the extracted contents

In a sample of past SaaS activity of UserA, the subject and attachments appear related to the ‘OUTSTANDING PREPAY WIRES 2021’ excel document found from the data leak website in Figure 4, suggesting a further possibility that the account was associated with the leaked data. 

Historic SaaS activity associated with UserA: 

·      Time: 2021-11-05 21:21:18 [UTC]
·      SaaS Office365 Logon Type: Owner
·      Protocol: OFFICE365
·      SaaS Account Name: O365
·      SaaS Actor: UserA@[REDACTED].com
·      SaaS Event: Send
·      SaaS Service: Office365
·      SaaS Service Product: Exchange
·      SaaS Status Message: Succeeded
·      SaaS Office365 Attachment: WIRE 2021.xlsx (92406b); image.png (9084b); image.png (1454b); image.png (1648b); image.png (1691b); image.png (1909b); image.png (2094b)
·      SaaS Office365 Subject: Wires 11/8/21
·      SaaS Resource Location: \Drafts
·      SaaS User Agent: Client=OWA;Action=ViaProxy 

Based on the available evidence, it is highly likely that the data packages contain the data stolen during the account compromise the previous month.  

Once the credentials of an Office365 account are stolen, an attacker can not only access the user's mailbox, but also a full range of Office365 applications such as SharePoint folders, Teams Chat, or files in the user's OneDrive [11]. For example, files shared in Teams chat are stored in OneDrive for Business in a folder named Microsoft Teams Chat Files in the default Document library on SharePoint. One of the files visible on the data leak website, called ‘[REDACTED] CONTRACT.3.1.2020.pdf’, was also observed in the default document folder of a third user account (UserC) within the victim organization, suggesting the compromised accounts may have been able to access shared files stored on other accounts by moving laterally via other O365 applications such as Teams. 

One example can be seen in the below AS logs: 

·      Time: 2021-11-11 01:58:35 [UTC]
·      SaaS Resource Type: File
·      Protocol: OFFICE365
·      SaaS Account Name: 0365
·      SaaS Actor: UserC@[REDACTED]
·      SaaS Event: FilePreviewed
·      SaaS Service Product: OneDrive
·      SaaS Metric: ResourceViewed
·      SaaS Office365 Application Name: Media Analysis and Transformation Service
·      SaaS Office365 File Extension: pdf
·      SaaS Resource Location: https://[REDACTED]-my.sharepoint.com/personal/userC_[REDACTED]_com/Documents/Microsoft Teams Chat Files/[REDACTED] CONTRACT 3.1.2020.pdf
·      SaaS Resource Name: [REDACTED] CONTRACT 3.1.2020.pdf
·      SaaS Service: Office365
·      SaaS Service Product: OneDrive
·      SaaS User Agent: OneDriveMpc-Transform_Thumbnail/1.0 

In the period between the 1st and 30th November, the customer’s Darktrace DETECT/Apps trial had raised multiple high-level alerts associated with SaaS account compromise, but there was no evidence of file encryption.  

Establish Foothold 

Looking back at the start of the attack, it is unclear exactly how the attacker evaded the customer’s pre-existing security stack. At the time of the incident, the victim was using a Barracuda email gateway and Microsoft 365 Threat Management for their cloud environment. 

Darktrace detected no indication the accounts were compromised via credential bruteforcing, which would have enabled the attacker to bypass the Azure Active Directory smart lockout (if enabled). The credentials may have been harvested via a phishing campaign which successfully evaded the list of known ‘bad’ domains maintained by their email gateway.  

Upon gaining access to the account, the Microsoft Defender for Cloud Apps anomaly detection policies would have been expected to raise an alert [12]. In this instance, the unusual login from Nigeria occurred over 16 hours after the previous login from the US, potentially evading anomaly detection policies such as the ‘Impossible Travel’ rule. 

Figure 5: Event log showing the user accessing mail from USA a day before the suspicious usage from Nigeria 

Darktrace Coverage

Darktrace DETECT 

Throughout this event, high scoring model breaches associated with the attack were visible in the customer’s SaaS Console. In addition, there were two Cyber AI Analyst incidents for ‘Possible Account Hijack’ associated with the two compromised SaaS Office365 accounts, UserA and UserB. The visibility given by Darktrace DETECT also enabled the security team to confirm which files had been accessed and were likely part of the data leak.

Figure 6: Example Cyber AI Analyst incident of UserB SaaS Office365 account

Darktrace RESPOND

In this incident, the attackers successfully compromised O365 accounts in order to exfiltrate customer data. Whilst Darktrace RESPOND/Apps was being trialed and suggested several actions, it was configured in human confirmation mode. The following RESPOND/Apps actions were advised for these activities:  

·      ‘Antigena [RESPOND] Unusual Access Block’ triggered by the successful login from an unusual IP address, would have actioned the ‘Block IP’ inhibitor, preventing access to the account from the unusual IP for up to 24 hours
·      ‘Suspicious Source Activity Block’, triggered by the suspicious user agent used to bypass MFA, would have actioned the ‘Disable User’ inhibitor, disabling the user account for up to 24 hours 

During this incident, Darktrace RESPOND/Network was being used in fully autonomous mode in order to prevent the threat actor from pivoting into the network. The security team were unable to conclusively say if any attempts by the attacker to do this had been made. 

Concluding Thoughts  

Data theft extortion has become a widely used attack technique, and ransomware gangs may increasingly use this technique alone to target organizations without secure data encryption and storage policies.  

This case study describes a SaaS data theft extortion incident which bypassed MFA and existing security tools. The attacker appeared to compromise credentials without bruteforce activity, possibly with the use of social engineering through phishing. However, from the first new login, Darktrace DETECT identified the unusual credential use in spite of it being an existing account. Had Darktrace RESPOND/Apps been configured, it would have autonomously responded to halt this login and prevent the attacker from accomplishing their data theft mission.

Thanks to Oakley Cox, Brianna Leddy and Shuh Chin Goh for their contributions.

Appendices

References 

[1] https://securelist.com/new-ransomware-trends-in-2022/106457/

[2] https://www.itpro.co.uk/security/ransomware/367624/the-rise-of-double-extortion-ransomware

[3] https://www.malwarebytes.com/blog/news/2020/10/vastaamo-psychotherapy-data-breach-sees-the-most-vulnerable-victims-extorted

[4] https://www.blackfog.com/shift-from-ransomware-to-data-theft-extortion/

[5] https://www.abuseipdb.com/check/105.112.59.83

[6] https://www.abuseipdb.com/check/105.112.36.212

[7] https://www.abuseipdb.com/check/45.130.83.129

[8] https://docs.microsoft.com/en-us/microsoft-365/compliance/mailitemsaccessed-forensics-investigations?view=o365-worldwide

[9] https://www.abuseipdb.com/check/45.135.187.108

[10] https://www.virustotal.com/gui/ip-address/45.137.20.65/details

[11] https://tidorg.com/new-bec-phishing-attack-steals-office-365-credentials-and-bypasses-mfa/

[12] https://docs.microsoft.com/en-us/microsoft-365/security/office-365-security/responding-to-a-compromised-email-account?view=o365-worldwide

Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Adrianne Marques
Senior Research Analyst

More in this series

No items found.

Blog

/

Email

/

July 24, 2026

Darktrace / EMAIL Expands Behavioral Defense Across Email and Collaboration Workflows

Default blog imageDefault blog image

Email and collaboration tools do more than carry messages. They are where organizations approve payments, share sensitive data, reset credentials, and make thousands of everyday decisions. Increasingly, they are interfaces through which humans direct AI agents in their daily activity. Email, Slack and Teams are high volume, rich with sensitive data, and an easy place to hide malicious activity.

The opportunity isn’t lost on bad actors. Darktrace / EMAIL detected more than 32 million high-confidence phishing emails globally in 2025, and 70% of those messages passed DMARC authentication.  Phishing is increasingly difficult to detect and familiar trust signals alone are not enough. People and security teams need to understand how a message fits the normal behavior of the sender, recipient, and organization. They also need to correlate activity across platforms to spot threats that span multiple channels.

To effectively secure against today’s evolved threats, security teams need to act at two levels: they need to help each employee make a safer decision ‘in the moment’, and they need to understand the wider patterns that may expose the business to risk.

Darktrace is introducing four new capabilities in Darktrace / EMAIL to address both challenges. The new features explain suspicious content more clearly to end users, strengthen the capabilities of Darktrace / Adaptive Human Defense with richer guidance, let organizations define their own patterns for detecting sensitive data in messages, and give security teams a process-level view of risk across email and collaboration workflows.

Darktrace / EMAIL Inbox Analysis highlights risky content within your emails

A warning is more useful when it explains what the user should look at. To help do that, we’ve expanded Darktrace / EMAIL’s Inbox Analysis Add-In to highlight potentially dangerous content within the body of emails that Darktrace / EMAIL flags as potentially suspicious or high risk.  

The add-in can highlight language designed to create urgency, financial references, requests for payment, suspicious links, and content that is unusual for the sender. Each highlighted element includes a pop up that explains why it may be suspicious. Instead of asking an employee to accept a verdict without context, the analysis helps them examine the message and make a more informed decision.

Enhanced Just-In-Time Training Banners in Darktrace / Adaptive Human Defense

Enhanced Just-In-Time Training Banners build on the same principle. The banners now include a contextual header, actionable advice, and specific detection context. This gives employees more useful guidance at the point of risk without adding unnecessary information or cognitive load.

Together, the capabilities help turn a warning into a short learning moment. Employees can see what looks unusual, understand what action to take, and build their judgment.

Custom Sensitive Data Detection in Darktrace / EMAIL - Data Loss Prevention

Sensitive data is different for every business. Standard categories such as payment card details or government identifiers matter, but organizations also have their own customer codes, project names, research formats, account structures, and internal identifiers.

Custom Sensitive Data Detection in Darktrace / EMAIL - Data Loss Prevention allows administrators to write custom expressions for the data their organization needs to protect. Matched content can trigger existing model actions and data loss prevention (DLP) workflows, extending Darktrace's DLP capabilities.

This extends data loss detection beyond a fixed library of common data types. Security teams can apply controls to information that is sensitive in the context of their own organization and adapt those controls as the business changes.

Introducing Email and Collaboration Workflow Risk Posture Dashboards

Some of the most important risks are not isolated events. They are repeated ways of working that create an opening for error, misuse, or attack. For example, a payment request may be one suspicious message, but a recurring approval workflow that relies on weak verification is a business process risk.

The new Email and Collaboration Workflow Risk Posture Dashboard analyzes email and collaboration data across Email, Microsoft Teams, Slack and Zoom to provide a process-level view of risk in the organization. These may include financial authorization workflows, sensitive data sharing patterns, and activity that could expose credentials.

The dashboard brings these patterns into a view and provides actionable recommendations. This helps security teams determine where to investigate or strengthen controls, where ownership needs to be clarified, and where the business may need to change a risky process. It gives CISOs a clearer view of how human and communication risk is embedded in everyday operations, not only where individual alerts occur.

Behavior connects the individual decision to the wider risk

These capabilities build on Darktrace’s unique behavioral approach to security. We use Adaptive AI to learn how people and AI normally behave within an organization, creating the context needed to recognize when activity changes.

Within the Darktrace Behavioral Defense Platform, Darktrace / EMAIL helps protect people against phishing, account takeover, data exfiltration, and human risk across email and collaboration tools. The new capabilities extend that protection in both directions. They give employees clearer context for the decision in front of them, while giving security leaders a broader view of the workflows and behavior that create risk across the organization.

The result is not simply more alerts. It is a better understanding of why something is risky, what action to take, and where the organization can reduce risk before a familiar process becomes an easy route for an attacker.

[related-resource]

Continue reading
About the author
Carlos Gray
Senior Product Marketing Manager, Email

Blog

/

/

July 24, 2026

When Guardrails Break: Why Securing AI Requires Behavioral Detection and Autonomous Containment

behavioral detectionDefault blog imageDefault blog image

Bottom line up front: Governance, guardrails, identity controls, and secure development are necessary to secure AI, but they are not sufficient. AI systems are probabilistic, adaptive, and non-deterministic. Therefore, organizations need two additional layers of security:

  1. Behavioral-based detection that can identify when AI begins to act outside its intended purpose; and  
  2. Surgical, explainable autonomous containment that can stop risky activity before it causes material damage.  

That capability depends on multiple specialized AI models working together, not one LLM making every decision.

Organizations are embedding AI into development, business operations, and security workflows faster than most security programs can adapt. The risk is no longer limited to the model. It extends across prompts, data, identities, agents, memory, APIs, tools, permissions, and the trust relationships connecting them.

In my recent blog, Securing AI: Analysis of the Complete Security Stack with Governance and Controls, I outlined a defense-in-depth strategy spanning governance, identity, data security, secure development, runtime detection, autonomous containment, and recovery. The most urgent requirement across that architecture is the ability to understand how AI behaves in practice and contain it when that behavior becomes risky.  

Why non-deterministic systems require behavioral-based detection

Traditional controls remain foundational. Organizations need least privilege, strong identity controls, secure-by-design architecture, data governance, AI inventories, guardrails, testing, and clear boundaries on autonomy.

But deterministic controls, which assume predictable and repeatable behavior, cannot fully secure non-deterministic systems, where the same input may not always produce the same outcome.

AI agents can interpret the same instruction differently, chain individually authorized actions into an unsafe outcome, or pursue a legitimate goal through a method the organization did not anticipate. One of the most recent examples of this is the incident that OpenAI and Hugging Face jointly disclosed, where an autonomous agent escaped its intended testing boundaries and compromised Hugging Face infrastructure.  

An agent may have permission to access data and invoke a tool, but that does not mean every use of that access is appropriate. It is not enough to know whether an action is allowed. Organizations need to know whether it makes sense.

  • Is this normal for this agent?  
  • Is it acting within its intended purpose?  
  • Is it accessing unusual data, invoking an unexpected tool, or beginning to drift?  
  • Do a series of ordinary-looking actions become risky when viewed together?

Behavioral-based detection specific to an environment or organization with an understanding of context and risk enables provides the needed detection engineering for AI systems. It learns normal activity across people, systems, data, devices, and AI agents, then identifies deviations and evaluates their risk, intent, and context. This enables detection of misuse, abuse, compromise, manipulation, and unintended behavior even when no known attack signature or explicit policy violation exists.

Why accuracy is the foundation for SOC optimization

AI will only improve the SOC if it produces accurate, explainable, and actionable outcomes.

If analysts must manually validate every AI-generated finding because they cannot understand the evidence or confidence behind it, automation has not reduced workload. It has moved the workload. False positives increase fatigue. False negatives cause the most risk and damage to organizations. Inaccurate autonomous actions can disrupt critical operations.

Accuracy is therefore more than a model-performance metric. It is the prerequisite for analyst trust, SOC optimization, and safe autonomous response.

That accuracy is unlikely to come from one model.

Generative AI is valuable for natural-language analysis, summarization, and human interaction. But an LLM should not be the sole analytical engine for behavioral-based detection, investigation, risk assessment, and containment. Interpretability and consistency are required for high-consequence security decisions.

A stronger architecture uses multiple specialized AI systems collaboratively:  

  • Behavioral models can establish normal activity.  
  • Unsupervised learning can identify novel anomalies.  
  • Graph analysis can evaluate relationships among agents, identities, systems, and tools.  
  • Other models can correlate events, investigate competing hypotheses, and assess risk.  
  • Semantic models can analyze language where behavior-based language analysis is needed but this can be used in tandem with vector embeddings, graph neural networks, and a variety of other AI systems.

Each model contributes a different analytical perspective. Their outputs can corroborate one another, improving accuracy and creating a more reliable basis for response. The objective is not one model operating as an oracle. It is layered, adaptive intelligence designed to produce decisions the SOC can understand and trust.

Autonomous containment is required to secure autonomous systems

Many SOCs remain hesitant to trust LLM-based agents with autonomous containment. That concern is reasonable. A poorly selected response can isolate the wrong asset, stop a critical workflow, block a legitimate identity, or create more operational damage than the original incident.

But relying exclusively on human response is also not viable.

AI systems can operate at machine speed. They can expose sensitive data, execute workflows, modify records, call tools, or propagate actions across connected systems before an analyst can investigate and intervene. The behavior may be unintentional, the result of an agent optimizing toward a goal, or caused by misuse, compromise, prompt injection, or offensive AI.

Intent affects the investigation. It does not change the need to stop the damage.

Organizations need autonomous response, but it must be surgical and explainable. The objective is not to shut down an entire agent, user, application, or business process whenever an anomaly occurs. It is to interrupt the specific risky behavior: block an unusual connection, constrain a tool call, stop an abnormal data transfer, or temporarily limit an agent when it is performing anomalous, risky activity.  

That buys humans time. It stops the spread, limits damage, and allows the SOC to investigate without unnecessarily disrupting the business.

Layered, Adaptive AI provides a path forward

Darktrace has spent more than a decade researching and operationalizing layered, behavioral, Adaptive AI that learns a specific organization rather than relying only on historic attacks or predefined signatures.

The approach is designed to understand normal behavior, identify anomalous activity, assess its risk, correlate related events, autonomously investigate, and, when necessary, apply targeted containment while normal operations continue.

That sequence matters. Autonomous response cannot simply be added to the end of an LLM workflow. Trusted containment depends on broad visibility, continuous behavioral understanding, multiple analytical techniques, risk and context evaluation, autonomous investigation, explainability, and precise response actions.

This represents a more responsible model for security autonomy: not automation for its own sake, but controlled autonomy built to improve security outcomes and protect business operations.

Security must enable AI adoption

The answer for security teams is not to block AI. Organizations are adopting it to improve productivity, accelerate development, and create new business value.

But innovation without behavioral detection and autonomous containment is not sustainable.

Organizations should continue investing in governance, identity, least privilege, data security, secure MLOps, guardrails, testing, evaluation, validation, verification, kill switches, rollback, and forensic readiness. At the same time, they cannot wait for every governance program to mature before addressing runtime risk.

Behavioral-based detection and autonomous containment provide an immediate layer of resilience. They allow organizations to detect exploitation and risky AI behavior they did not anticipate, contain it at machine speed, and preserve human control over broader remediation.

The future of AI security will not be defined by a single model making every decision. It will be defined by multiple specialized AI systems working collaboratively, with sufficient accuracy, transparency, and context to support trusted autonomous action.

Surgical, explainable autonomous containment is no longer a future capability. It is a requirement for scaling AI securely today.

Learn how to build a defense-in-depth strategy for securing AI at scale in our talk at Black Hat on August 5 at 3:15 PM.  

[related-resource]

Continue reading
About the author
Your data. Our AI.
Elevate your network security with Darktrace AI