December 15, 2023

How Darktrace Halted A DarkGate in MS Teams

Discover how Darktrace thwarted DarkGate malware in Microsoft Teams. Stay informed on the latest cybersecurity measures and protect your business.

Written by

Natalia Sánchez Rocafort

Cyber Security Analyst

Inside the SOC

Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.

Written by

Natalia Sánchez Rocafort

Cyber Security Analyst

Dec 2023

Securing Microsoft Teams and SharePoint

Given the prevalence of the Microsoft Teams and Microsoft SharePoint platforms in the workplace in recent years, it is essential that organizations stay vigilant to the threat posed by applications vital to hybrid and remote work and prioritize the security and cyber hygiene of these services. For just as the use of these platforms has increased exponentially with the rise of remote and hybrid working, so too has the malicious use of them to deliver malware to unassuming users.

Researchers across the threat landscape have begun to observe these legitimate services being leveraged by malicious actors as an initial access method. Microsoft Teams can easily be exploited to send targeted phishing messages to individuals within an organization, while appearing legitimate and safe. Although the exact contents of these messages may vary, the messages frequently use social engineering techniques to lure users to click on a SharePoint link embedded into the message. Interacting with the malicious link will then download a payload [1].

Darktrace observed one such malicious attempt to use Microsoft Teams and SharePoint in September 2023, when a device was observed downloading DarkGate, a commercial trojan that is known to deploy other strains of malware, also referred to as a commodity loader [2], after clicking on SharePoint link. Fortunately for the customer, Darktrace’s suite of products was perfectly poised to identify the initial signs of suspicious activity and Darktrace RESPOND™ was able to immediately halt the advancement of the attack.

DarkGate Attack Overview

On September 8, 2023, Darktrace DETECT™ observed around 30 internal devices on a customer network making unusual SSL connections to an external SharePoint site which contained the name of a person, 'XXXXXXXX-my.sharepoint[.]com' (107.136[.]8, 13.107.138[.]8). The organization did not have any employees who went by this name and prior to this activity, no internal devices had been seen contacting the endpoint.

At first glance, this initial attack vector would have appeared subtle and seemingly trustworthy to users. Malicious actors likely sent various users a phishing message via Microsoft Teams that contained the spoofed SharePoint link to the personalized SharePoint link ''XXXXXXXX-my.sharepoint[.]com'.

Figure 1: Advanced Search query showing a sudden spike in connections to ''XXXXXXXX -my.sharepoint[.]com'.

Darktrace observed around 10 devices downloading approximately 1 MB of data during their connections to the Sharepoint endpoint. Darktrace DETECT observed some of the devices making subsequent HTTP GET requests to a range of anomalous URIs. The devices utilized multiple user-agents for these connections, including ‘curl’, a command line tool that allows individuals to request and transfer data from a specific URL. The connections were made to the IP 5.188.87[.]58, an endpoint that has been flagged as an indicator of compromise (IoC) for DarkGate malware by multiple open-source intelligence (OSINT) sources [3], commonly associated with HTTP GET requests:

GET request over port 2351 with the User-Agent header 'Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)' and the target URI '/bfyxraav' to 5.188.87[.]58
GET request over port 2351 with the user-agent header 'curl' and the target URI '/' to 5.188.87[.]58
GET request over port 2351 with the user-agent header 'curl/8.0.1' and the target URI '/msibfyxraav' to 5.188.87[.]58

The HTTP GET requests made with the user-agent header 'curl' and the target URI '/' to 5.188.87[.]58 were responded to with a filename called 'Autoit3.exe'. The other requests received script files with names ending in '.au3, such as 'xkwtvq.au3', 'otxynh.au3', and 'dcthbq.au3'. DarkGate malware has been known to make use of legitimate AutoIt files, and typically runs multiple AutoIt scripts (‘.au3’) [4].

Following these unusual file downloads, the devices proceeded to make hundreds of HTTP POST requests to the target URI '/' using the user-agent header 'Mozilla/4.0 (compatible; Synapse)' to 5.188.87[.]58. The contents of these requests, along with the contents of the responses, appear to be heavily obfuscated.

Figure 2: Example of obfuscated response, as shown in a packet capture downloaded from Darktrace.

While Microsoft’s Safe Attachments and Safe Links settings were unable to detect this camouflaged malicious activity, Darktrace DETECT observed the unusual over-the-network connectivity that occurred. While Darktrace DETECT identified multiple internal devices engaging in this anomalous behavior throughout the course of the compromise, the activity observed on one device in particular best showcases the overall kill chain of this attack.

The device in question was observed using two different user agents (curl/8.0.1 and Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)) when connecting to the endpoint 5.188.87[.]58 and target URI ‘/bfyxraav’. Additionally, Darktrace DETECT recognized that it was unusual for this device to be making these HTTP connections via destination port 2351.

As a result, Darktrace’s Cyber AI Analyst™ launched an autonomous investigation into the suspicious activity and was able to connect the unusual external connections together, viewing them as one beaconing incident as opposed to isolated series of connections.

Figure 3: Cyber AI Analyst investigation summarizing the unusual repeated connections made to 5.188.87[.]58 via destination port 2351.

Darktrace then observed the device downloading the ‘Autoit3.exe’ file. Darktrace RESPOND took swift mitigative action by blocking similar connections to this endpoint, preventing the device from downloading any additional suspicious files.

Figure 4: Suspicious ‘Autoit3.exe’ downloaded by the source device from the malicious external endpoint.

Just one millisecond later, Darktrace observed the device making suspicious HTTP GET requests to URIs including ‘/msibfyxraav’. Darktrace recognized that the device had carried out several suspicious actions within a relatively short period of time, breaching multiple DETECT models, indicating that it may have been compromised. As a result, RESPOND took action against the offending device by preventing it from communicating externally [blocking all outbound connections] for a period of one hour, allowing the customer’s security team precious time to address the issue.

It should be noted that, at this point, had the customer subscribed to Darktrace’s Proactive Threat Notification (PTN) service, the Darktrace Security Operations Center (SOC) would have investigated these incidents in greater detail, and likely would have sent a notification directly to the customer to inform them of the suspicious activity.

Additionally, AI Analyst collated various distinct events and suggested that these stages were linked as part of an attack. This type of augmented understanding of events calculated at machine speed is extremely valuable since it likely would have taken a human analyst hours to link all the facets of the incident together.

Figure 5: AI Analyst investigation showcasing the use of the ‘curl’ user agent to connect to the target URI ‘/msibfyxraav’.

Figure 6: Darktrace RESPOND moved to mitigate any following connections by blocking all outgoing traffic for 1 hour.

Following this, an automated investigation was launched by Microsoft Defender for Endpoint. Darktrace is designed to coordinate with multiple third-party security tools, allowing for information on ongoing incidents to be seamlessly exchanged between Darktrace and other security tools. In this instance, Microsoft Defender identified a ‘low severity’ incident on the device, this automatically triggered a corresponding alert within DETECT, presented on the Darktrace Threat Visuallizer.

The described activity occurred within milliseconds. At each step of the attack, Darktrace RESPOND took action either by enforcing expected patterns of life [normality] on the affected device, blocking connections to suspicious endpoints for a specified amount of time, and/or blocking all outgoing traffic from the device. All the relevant activity was detected and promptly stopped for this device, and other compromised devices, thus containing the compromise and providing the security team invaluable remediation time.

Figure 7: Overview of the compromise activity, all of which took place within a matter of miliseconds.

Darktrace identified similar activity on other devices in this customer’s network, as well as across Darktrace’s fleet around the same time in early September.

On a different customer environment, Darktrace DETECT observed more than 25 ‘.au3’ files being downloaded; this activity can be seen in Figure 9.

Figure 8: High volume of file downloads following GET request and 'curl' commands.

Figure 9 provides more details of this activity, including the source and destination IP addresses (5.188.87[.]58), the destination port, the HTTP method used and the MIME/content-type of the file

Figure 9: Additional information of the anomalous connections.

A compromised server in another customer deployment was seen establishing unusual connections to the external IP address 80.66.88[.]145 – an endpoint that has been associated with DarkGate by OSINT sources [5]. This activity was identified by Darktrace/DETECT as a new connection for the device via an unusual destination port, 2840. As the device in question was a critical server, Darktrace DETECT treated it with suspicion and generated an ‘Anomalous External Activity from Critical Network Device’ model breach.

Figure 10: Model breach and model breach event log for suspicious connections to additional endpoint.

Conclusion

While Microsoft Teams and SharePoint are extremely prominent tools that are essential to the business operations of many organizations, they can also be used to compromise via living off the land, even at initial intrusion. Any Microsoft Teams user within a corporate setting could be targeted by a malicious actor, as such SharePoint links from unknown senders should always be treated with caution and should not automatically be considered as secure or legitimate, even when operating within legitimate Microsoft infrastructure.

Malicious actors can leverage these commonly used platforms as a means to carry out their cyber-attacks, therefore organizations must take appropriate measures to protect and secure their digital environments. As demonstrated here, threat actors can attempt to deploy malware, like DarkGate, by targeting users with spoofed Microsoft Teams messages. By masking malicious links as legitimate SharePoint links, these attempts can easily convince targets and bypass traditional security tools and even Microsoft’s own Safe Links and Safe Attachments security capabilities.

When the chain of events of an attack escalates within milliseconds, organizations must rely on AI-driven tools that can quickly identify and automatically respond to suspicious events without latency. As such, the value of Darktrace DETECT and Darktrace RESPOND cannot be overstated. Given the efficacy and efficiency of Darktrace’s detection and autonomous response capabilities, a more severe network compromise in the form of the DarkGate commodity loader was ultimately averted.

Credit to Natalia Sánchez Rocafort, Cyber Security Analyst, Zoe Tilsiter.

Appendices

Darktrace DETECT Model Detections

[Model Breach: Device / Initial Breach Chain Compromise 100% –– Breach URI: /#modelbreach/114039 ] (Enhanced Monitoring)· [Model Breach: Device / Initial Breach Chain Compromise 100% –– Breach URI: /#modelbreach/114124 ] (Enhanced Monitoring)
[Model Breach: Device / New User Agent and New IP 62% –– Breach URI: /#modelbreach/114030 ]
[Model Breach: Anomalous Connection / Application Protocol on Uncommon Port 46% –– Breach URI: /#modelbreach/114031 ]
[Model Breach: Anomalous Connection / New User Agent to IP Without Hostname 62% –– Breach URI: /#modelbreach/114032 ]
[Model Breach: Device / New User Agent 32% –– Breach URI: /#modelbreach/114035 ]
[Model Breach: Device / Three Or More New User Agents 31% –– Breach URI: /#modelbreach/114036 ]
[Model Breach: Anomalous Server Activity / Anomalous External Activity from Critical Network Device 62% –– Breach URI: /#modelbreach/612173 ]
[Model Breach: Anomalous File / EXE from Rare External Location 61% –– Breach URI: /#modelbreach/114037 ]
[Model Breach: Anomalous Connection / Multiple Connections to New External TCP Port 61% –– Breach URI: /#modelbreach/114042 ]
[Model Breach: Security Integration / Integration Ransomware Detected 100% –– Breach URI: /#modelbreach/114049 ]
[Model Breach: Compromise / Beaconing Activity To External Rare 62% –– Breach URI: /#modelbreach/114059 ]
[Model Breach: Compromise / HTTP Beaconing to New Endpoint 30% –– Breach URI: /#modelbreach/114067 ]
[Model Breach: Security Integration / C2 Activity and Integration Detection 100% –– Breach URI: /#modelbreach/114069 ]
[Model Breach: Anomalous File / EXE from Rare External Location 55% –– Breach URI: /#modelbreach/114077 ]
[Model Breach: Compromise / High Volume of Connections with Beacon Score 66% –– Breach URI: /#modelbreach/114260 ]
[Model Breach: Security Integration / Low Severity Integration Detection 59% –– Breach URI: /#modelbreach/114293 ]
[Model Breach: Security Integration / Low Severity Integration Detection 33% –– Breach URI: /#modelbreach/114462 ]
[Model Breach: Security Integration / Integration Ransomware Detected 100% –– Breach URI: /#modelbreach/114109 ]· [Model Breach: Device / Three Or More New User Agents 31% –– Breach URI: /#modelbreach/114118 ]· [Model Breach: Anomalous Connection / Application Protocol on Uncommon Port 46% –– Breach URI: /#modelbreach/114113 ] · [Model Breach: Anomalous Connection / New User Agent to IP Without Hostname 62% –– Breach URI: /#modelbreach/114114 ]· [Model Breach: Device / New User Agent 32% –– Breach URI: /#modelbreach/114117 ]· [Model Breach: Anomalous File / EXE from Rare External Location 61% –– Breach URI: /#modelbreach/114122 ]· [Model Breach: Security Integration / Low Severity Integration Detection 54% –– Breach URI: /#modelbreach/114310 ]
[Model Breach: Security Integration / Integration Ransomware Detected 65% –– Breach URI: /#modelbreach/114662 ]Darktrace/Respond Model Breaches
[Model Breach: Antigena / Network::External Threat::Antigena Suspicious File Block 61% –– Breach URI: /#modelbreach/114033 ]
[Model Breach: Antigena / Network::External Threat::Antigena File then New Outbound Block 100% –– Breach URI: /#modelbreach/114038 ]
[Model Breach: Antigena / Network::Significant Anomaly::Antigena Enhanced Monitoring from Client Block 100% –– Breach URI: /#modelbreach/114040 ]
[Model Breach: Antigena / Network::Significant Anomaly::Antigena Significant Anomaly from Client Block 87% –– Breach URI: /#modelbreach/114041 ]
[Model Breach: Antigena / Network::Significant Anomaly::Antigena Controlled and Model Breach 87% –– Breach URI: /#modelbreach/114043 ]
[Model Breach: Antigena / Network::External Threat::Antigena Ransomware Block 100% –– Breach URI: /#modelbreach/114052 ]
[Model Breach: Antigena / Network::Significant Anomaly::Antigena Significant Security Integration and Network Activity Block 87% –– Breach URI: /#modelbreach/114070 ]
[Model Breach: Antigena / Network::Significant Anomaly::Antigena Breaches Over Time Block 87% –– Breach URI: /#modelbreach/114071 ]
[Model Breach: Antigena / Network::External Threat::Antigena Suspicious Activity Block 87% –– Breach URI: /#modelbreach/114072 ]
[Model Breach: Antigena / Network::External Threat::Antigena Suspicious File Block 53% –– Breach URI: /#modelbreach/114079 ]
[Model Breach: Antigena / Network::Significant Anomaly::Antigena Breaches Over Time Block 64% –– Breach URI: /#modelbreach/114539 ]
[Model Breach: Antigena / Network::External Threat::Antigena Ransomware Block 66% –– Breach URI: /#modelbreach/114667 ]
[Model Breach: Antigena / Network::External Threat::Antigena Suspicious Activity Block 79% –– Breach URI: /#modelbreach/114684 ]·
[Model Breach: Antigena / Network::External Threat::Antigena Ransomware Block 100% –– Breach URI: /#modelbreach/114110 ]·
[Model Breach: Antigena / Network::Significant Anomaly::Antigena Significant Anomaly from Client Block 87% –– Breach URI: /#modelbreach/114111 ]·
[Model Breach: Antigena / Network::Significant Anomaly::Antigena Controlled and Model Breach 87% –– Breach URI: /#modelbreach/114115 ]·
[Model Breach: Antigena / Network::Significant Anomaly::Antigena Breaches Over Time Block 87% –– Breach URI: /#modelbreach/114116 ]·
[Model Breach: Antigena / Network::External Threat::Antigena Suspicious File Block 61% –– Breach URI: /#modelbreach/114121 ]·
[Model Breach: Antigena / Network::External Threat::Antigena File then New Outbound Block 100% –– Breach URI: /#modelbreach/114123 ]·
[Model Breach: Antigena / Network::Significant Anomaly::Antigena Enhanced Monitoring from Client Block 100% –– Breach URI: /#modelbreach/114125 ]

List of IoCs

IoC - Type - Description + Confidence

5.188.87[.]58 - IP address - C2 endpoint

80.66.88[.]145 - IP address - C2 endpoint

/bfyxraav - URI - Possible C2 endpoint URI

/msibfyxraav - URI - Possible C2 endpoint URI

Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5) - User agent - Probable user agent leveraged

curl - User agent - Probable user agent leveraged

curl/8.0.1 - User agent - Probable user agent leveraged

Mozilla/4.0 (compatible; Synapse) - User agent - Probable user agent leveraged

Autoit3.exe - Filename - Exe file

CvUYLoTv.au3

eDVeqcCe.au3

FeLlcFRS.au3

FTEZlGhe.au3

HOrzcEWV.au3

rKlArXHH.au3

SjadeWUz.au3

ZgOLxJQy.au3

zSrxhagw.au3

ALOXitYE.au3

DKRcfZfV.au3

gQZVKzek.au3

JZrvmJXK.au3

kLECCtMw.au3

LEXCjXKl.au3

luqWdAzF.au3

mUBNrGpv.au3

OoCdHeJT.au3

PcEJXfIl.au3

ssElzrDV.au3

TcBwRRnp.au3

TFvAUIgu.au3

xkwtvq.au3

otxynh.au3

dcthbq.au3 - Filenames - Possible exe files delivered in response to curl/8.0.1 GET requests with Target URI '/msibfyxraav

f3a0a85fe2ea4a00b3710ef4833b07a5d766702b263fda88101e0cb804d8c699 - SHA256 file hash - Possible SHA256 hashes of 'Autoit3.exe' files

afa3feea5964846cd436b978faa7d31938e666288ffaa75d6ba75bfe6c12bf61 - SHA256 file hash - Possible SHA256 hashes of 'Autoit3.exe' files

63aeac3b007436fa8b7ea25298362330423b80a4cb9269fd2c3e6ab1b1289208 - SHA256 file hash - Possible SHA256 hashes of 'Autoit3.exe' files

ab6704e836a51555ec32d1ff009a79692fa2d11205f9b4962121bda88ba55486 - SHA256 file hash - Possible SHA256 hashes of 'Autoit3.exe' files

References

1. https://www.truesec.com/hub/blog/darkgate-loader-delivered-via-teams

2. https://feedit.cz/wp-content/uploads/2023/03/YiR2022_onepager_ransomware_loaders.pdf

‍3. https://www.virustotal.com/gui/ip-address/5.188.87[.]58

‍4. https://www.forescout.com/resources/darkgate-loader-malspam-campaign/

‍5. https://otx.alienvault.com/indicator/ip/80.66.88[.]145

Written by

Natalia Sánchez Rocafort

Cyber Security Analyst

Inside the SOC

Written by

Natalia Sánchez Rocafort

Cyber Security Analyst

Watch the NIS2 Webinar

Blog

July 7, 2026

Securing AI: Analysis of the Complete Security Stack with Governance and Controls

Why traditional cybersecurity approaches are not enough for AI

AI adoption outpaces most security programs’ ability to adapt. That gap is now one of the most consequential sources of cyber risk facing enterprises. As organizations embed generative and agentic AI into development workflows, business operations, and security tooling itself, the question is no longer whether AI will introduce risk. The question is whether organizations understand where that risk actually lives and how to manage it operationally.

Two recent pieces of guidance underscore this shift:

The upcoming Cybersecurity Framework Profile for AI from NIST

The Five Eyes government guidance on the careful adoption of agentic AI services

Taken together, they point to a critical conclusion. AI security cannot be reduced to model hardening or prompt filtering. It requires a defense in depth strategy that treats AI as both a new attack surface and a force multiplier for defense, while accounting for how AI fundamentally changes scale, speed, and autonomy.

Recent threat research suggests that today's cyber risk is driven less by initial compromise and more by an adversary's ability to blend into normal operations over time. AI systems create the same exposure in a new form: more autonomy, more scale, and more opportunities for risky behavior to blend into normal operations.

How NIST defines the three core pillars of AI security

The NIST profile organizes AI risk across three inseparable focus areas that span all cybersecurity functions, Secure, Defend and Thwart. These areas are not sequential. They exist simultaneously and must be addressed together.

Secure

This treats AI as an attack surface. It includes models, prompts, agents, pipelines, training and inference data, retrieval augmented generation corpora, and the AI supply chain itself. AI systems are opaque, probabilistic, and non-deterministic by design. Some vulnerabilities are inherent in how models are trained or how data is sourced. Traditional patching does not fully mitigate these risks. This is also where many enterprises are weakest today and, critically, where many security programs stop.

Defend

This is AI as a defensive force multiplier. AI can improve detection speed, scale, correlation, and response, but only if the right models are used and operationalized correctly. Machine-speed behavior-based detection, response and containment becomes critical in defending non-deterministic systems. Accuracy, explainability, governance, testing, validation, and integration into SOC workflows matter as much as capability. Without those controls, hallucination risk, over automation, and misplaced trust become security risks themselves.

Thwart

This treats AI as an adversarial accelerant. Threat actors are already using AI to generate targeted social engineering attacks, deepfakes, malware, and autonomous attack agents. Asymmetric warfare is highlighting faster vulnerability discovery and exploitation with a lag on patch development, testing and deployment.

How this looks in practice

Darktrace researchers observed scaled, automated exploitation of the React2Shell vulnerability within days of disclosure. A vulnerable cloud asset was exploited in under 120 seconds of being deployed. Darktrace research team observed an AI/LLM-generated malware sample used in exploitation activity tied to React2Shell. The significance isn't novelty. It is that AI lowers the barrier to producing usable offensive tooling and compresses the time between experimentation and deployment.

Tactics are getting more and more creative in order to string together steps of an attack kill chain. This creates a dependency on behavior-based detection, autonomous investigation, autonomous containment, training, resilience investment, and recovery planning across the entire enterprise.

Why agentic AI fundamentally changes enterprise cyber risk

The Five Eyes guidance on agentic AI highlights material changes to the cyber risk profile of an organization. Unlike generative AI systems that produce content for human consumption, agentic AI systems reason, plan, and act autonomously across tools, data, and environments. That autonomy, combined with access to real systems, amplifies the impact of traditional cyber failures and introduces new system level risks that are difficult to predict, observe, and contain.

Risk in agentic systems does not live in the model alone. It emerges from interactions between models, prompts, memory, tools, APIs, identities, privileges, inter-agent trust relationships, and human assumptions baked into design. Vulnerabilities are often introduced through data, connectors, natural language interfaces, protocols, and drift by design.

In supply-chain incidents, attackers did not need sophisticated exploits to scale impact. They abused trusted systems built for automation and implicit access. Agentic AI inherits that model. Once a system can act across tools, data, and workflows, compromise propagates through trust relationships that were never designed for machine autonomy.

The major agentic AI risk classes include the following:

The identity control for non-human identities or autonomous agents makes it difficult to mitigate over-permissioning, limiting access, scope, and duration, as well as access hygiene

Agents are frequently over permissioned

Compromised tools inherit agent authority

Static secrets enable impersonation

Implicit trust between agents enables lateral movement

Design and configuration risks compound this, including privileges evaluated once at startup, poor segmentation, unvetted third party tools, reused authorization decisions outside their original context, and guardrail limitations.

Behavioral risk

Agents can optimize for goals in unsafe ways, misinterpret ambiguous intent, chain actions into unintended sequences, change behavior during evaluation, and exhibit deceptive or sycophantic responses.

Structural risk

Structural risk follows from agentic systems that are tightly coupled, multicomponent ecosystems. Failures can propagate across agents. Hallucinations cascade downstream. Resource exhaustion becomes systemic. Tool misuse enables indirect prompt injection and command execution. Rogue agents can poison peer agents through trust relationships.

Accountability

Accountability becomes unclear as autonomy increases. Autonomous agents assume human identity permissions, and humans should have clear ownership of these agents, but they don’t, and this model is flawed. Decision paths are opaque and non-deterministic. Logs are fragmented and difficult to interpret. Reproducing an incident will be impossible without explicit design for observability and forensics. An agent compromise is functionally an insider threat, often with better access and fewer behavioral constraints than a human.

What does defense in depth look like for AI?

Agentic AI runs on software, networks, identities, and data. It must be governed using the same foundational principles that have proven resilient under uncertainty, including secure by design, defense in depth, zero trust, least privilege, continuous monitoring, behavior-based advanced threat detection and containment, and incident response and recovery.

Core components to a Defense in depth Strategy for Securing the use of AI:

Strong, precise identity control plane to include an identity per agent (cryptographic, non‑shared)
- Privilege monitoring and just‑in‑time access

Data Governance

Secure‑by‑default configurations
- Security Posture Management
- Zero Trust principles

Strong guardrails, deny‑by‑default policies, and isolation

Explicit instruction hierarchies and controlled context

Behavioral-based detection across entire enterprise to include inputs, tools, and outputs as well as AI used on the endpoint, across the network, cloud, SaaS, email, and OT
- Runtime anomaly detection and goal‑drift detection
- Autonomous containment to mitigate risk and minimize damage

Hard boundaries on autonomy and delegation

Testing, Evaluation, Validation and Verification
- Determine when autonomous action and when human in the loop
- Adversarial training and agent‑specific testing
- Simulation, red teaming, and chaos testing

Kill‑switches, rollback, and containment mechanisms
- Forensics data captures, interpretability, autonomous containment, and remediation/recovery plans

Until standards, tooling, and assurance methods mature, organizations should assume agentic AI systems will behave unexpectedly and design deployments around resilience, behavior-based detection, reversibility, and containment, not efficiency.

How security leaders should prepare for enterprise AI adoption

AI security is not model security alone. Data, pipelines, identities, and agents are first class assets. Many AI attacks succeed through standard cyber failures amplified by AI. Identity, data, and supply chain risk dominate. Behavior-based detection and response are critical, not optional. Logging, provenance, versioning, and forensics data capture of detections are mandatory because you cannot investigate or recover from AI incidents without them.

Risk will often be visible in behavior before it is clearly defined in policy or guidance. The same pattern has been seen in pre-CVE disclosure detection, where abnormal activity appears before the industry has named or described the vulnerability. AI systems introduce that uncertainty by design.

Security leaders should prioritize controls before AI is fully deployed, avoid generic AI security checklists, integrate AI risk into existing cyber programs, and mitigate the risk of non-deterministic technology with continuous oversight, monitoring, behavior analytics, anomaly detection, autonomous investigation, and autonomous containment.

Visibility has a different connotation with AI. Previously, audit logging worked for software/people, but with Generative AI-based systems, interpretability and explainability is difficult to understand, you cannot "undo" what has been done, or see the logic or control a chain of events. This is why behavioral-based detections and containment becomes critical.

What capabilities should every AI security program include?

If an organization asked “what must be in place before scaling AI?”:

AI Risk board and approval workflow

IAM + PAM for all AI services and agents

AI asset inventory

Prompt/output DLP with sanctioned AI access – This is not just pre- and post- filters, but behavior-based detections of semantic interface as well as behavior-based analysis of output with associated risk context.

Shadow AI identification

Secure MLOps – This is an entire paper itself

Runtime guardrails and tool restrictions
- Including AI Gateway/SASE/Zero trust/

Runtime security with behavior-based detections
- Complete visibility, monitoring, behavior analytics, anomaly detection, risk/intent/context evaluation of anomalies, autonomous investigation and autonomous containment of all AI assets across endpoint, network, SaaS, SASE, cloud, OT, email, and messaging platforms

Secure data pipelines and data governance

SOC workflow changes from malicious classification workflows to behavior-based detection workflows

Remediation plans for AI-related incidents

Layered Governance and Security Stack for Securing AI

The following outline considers governance and security tools that should be considered, well-integrated, deployed, tested, operationalized and embedded within security workflows. These tools and controls map to NIST’s CMF for AI.

These considerations do not need to be implemented in order. Runtime Detect and Respond will help mitigate risk while Governance, Visibility, and Identity mature.

Category	Tooling	Controls
Governance & Visibility	AI asset inventory / AI CMDB Shadow AI discovery SaaS discovery AI usage on non-endpoint managed systems via network or cloud telemetry MCP server/client usage via protocols Browser telemetry Gateway or SASE telemetry	Establish a risk board to set up controls Mandatory registration of AI systems Owner, data classification, intended use, and risk tier Supplier disclosure requirements Risk mitigation plan for AI adoption, innovation, or development
Identity, Access & Agent Control	Non-human autonomous agents should not have the full permissions associated with a human user. IAM with workload identities PAM for AI service accounts Secrets management with short-lived tokens Zero Trust principles Identity, permission, and token hygiene	Unique identities per model, agent, and pipeline Least privilege for tools, data, and APIs Explicit approval for autonomous actions
Data Security & Privacy	Data classification and labeling Enterprise DLP across endpoint, email, network, cloud, and SaaS Forensics data capture after risky detections	Prompt-level DLP through behavior-based semantic analysis with risk and intent context Input/interface analysis for risky data requests Output analysis for sensitive data Data integrity evaluation Retention and redaction policies for prompts and responses
Secure MLOps / LLMOps	Secure CI/CD with AI-specific gates Model registries with approval workflows Dependency, container, and artifact scanning SBOM/AIBOM generation IaC security scanning Security posture management Misconfiguration identification Hardening recommendations	Signed models and prompts Versioned datasets, configurations, logging, and controls Securing data pipelines Controlled promotion Quality assurance Adversarial testing
Runtime Security	Securing runtime goes beyond guardrails and model firewalls to include behavior-based detections, response, and containment. Detection, monitoring, and SOC integration Centralized visibility into prompts, outputs, and tool calls AI-specific detections Behavior-based detection for AI usage patterns Model drift and behavior monitoring Autonomous containment Behavior-based detection of model inputs and outputs Prompt injection detection Model manipulation, including jailbreaking, poisoning, and related attacks Sensitive data access attempts Behavior-based detection across low-code agents, high-code agents, MCP clients and servers, endpoint, network, cloud, email, SaaS, SASE, IoT, and OT	Policy enforcement between users, models, tools, agents, SaaS models/tools, and MCP servers/clients Risk, intent, and context evaluation for detections and response actions
Response & Recovery	Autonomous containment AI-assisted playbooks Forensics data capture for AI-related events Model rollback mechanisms Backup and restore for models and datasets	Kill switch for agents Autonomous response to agents performing risky behaviors Model and dataset rollback Remediation plans Tabletop exercises Supplier coordination plans Post-incident AI performance validation

AI security requires continuous visibility and behavioral detection

AI changes how fast systems move, how decisions are made, and how risk propagates. It does not change the fundamentals of security. Organizations that succeed will be the ones that apply those fundamentals rigorously, assume failure, and build systems that can detect, contain, and recover when AI behaves in ways they did not anticipate. Security is not what AI is allowed to do. It is whether the organization can understand, trust, and control what AI actually does in practice.

Take this guidance to understand different initiatives that organizations should be considering. Securing AI is the most critical component to AI safety. As organizations invest more in AI adoption, they should be investing in security in order to mitigate the risk of AI adoption. Organizations should be evaluating their governance and security stack to include well-integrated tools that are deployed, tested, operationalized and embedded within security workflows. While organizations mature in governance, visibility and identity access management, they should be investing in behavior-based detection and autonomous containment to mitigate AI risk.

About the author

Blog

July 6, 2026

NIST Just Proved It: AI Security Can’t Be Solved With Rules

Static AI guardrails are inherently limited

As organizations adopt generative AI, many still assume that the right set of guardrails will be enough. The problem is you can’t anticipate every way these systems might be misused, abused or attacked. What NIST has done is put a mathematical foundation under that intuition.

In recent research building on Gödel’s incompleteness theorems, which showed that any system built on a fixed set of rules will always have gaps, NIST demonstrates that there is no finite set of guardrails that can be universally robust against adversarial prompts. In plain terms, if your defense is based on a fixed set of rules, there will always be inputs that bypass them. Not because the rules are badly written, but because the problem space is bigger than static rules can ever cover.

This is not new in cybersecurity - detection rules have always had to live with this trade-off. What is different with GenAI is the scale and shape of that problem. These systems are built on human language, and human language is not bounded. It is fluid, contextual and deliberately ambiguous. The number of ways intent can be hidden is effectively limitless. You are not defending against a defined protocol or a fixed exploit chain. You are defending against the entire expressive capacity of people.

So attempting to create a complete set of rules is the wrong starting point. It assumes the problem can be deterministically described. NIST’s work shows that it cannot. Organizations still need a way to manage AI risk, but the traditional approach of defining allowed and disallowed patterns is always going to lag behind what is actually happening. The same input can be benign in one context and risky in another, and static rules struggle to capture that distinction.

The question then is what fills that gap?

AI security must shift from rules to behavior

What's required is a shift in what you are trying to understand. Rules try to describe what should and shouldn't happen. Behavior shows you what is happening. Or to put it another way, if inputs are unbounded and adversaries adapt, the only stable signal is behavior.

In a GenAI context, that means analyzing how an AI model is being used, how prompts evolve over time, how outputs are shaped, and where AI agent interactions start to drift from what is expected. It means moving from static definitions of bad to a more dynamic understanding of intent.

Instead of trying to predict every bad prompt, you focus on identifying when behavior starts to move outside expected norms. Instead of asking whether a single input matches a rule, you ask whether the overall pattern of activity makes sense for the system and how it’s being used.

Guardrails remain important but they are only one layer

This does not eliminate the need for guardrails. They still play a role. But they will never address the entire problem space and are simply one part of your defense in depth approach.

NIST’s proof is useful because it makes this explicit. It removes the assumption that with enough effort, a complete rule set is achievable. It isn’t.

Once you accept that, the shift becomes unavoidable. This is no longer a problem of writing better rules, but of understanding behavior in a space where the possible inputs are effectively unbounded.

For security leaders, that changes the nature of the problem. It is less about defining what should be allowed, and more about recognizing when something is no longer consistent with expected behavior.

That does not remove the need for guardrails, but it does change their role. They set boundaries, but they do not define understanding. The gap between the two is where risk now sits.

In the end, this is what “can’t be solved with rules” really means. Rules will always leave gaps, and those gaps are not theoretical. They show up in how systems actually behave Not what we expect them to do, or what we intended them to do, but what they are doing in practice. That is where the signal is, and increasingly, that is where the security problem sits.

References:

https://www.nist.gov/news-events/news/2026/06/nist-mathematical-proof-supports-transition-continuous-monitor-and-update

https://ieeexplore.ieee.org/document/11475847

About the author

Andrew Hollister

Principal Solutions Engineer, Cyber Technician

Your data. Our AI.

Elevate your network security with Darktrace AI

Get a demo

Check out this article by Darktrace: How Darktrace Halted A DarkGate in MS Teams

How Darktrace Halted A DarkGate in MS Teams

Securing Microsoft Teams and SharePoint

DarkGate Attack Overview

Conclusion

Appendices

Darktrace DETECT Model Detections

List of IoCs

References

From Click to Command: Behavioral Detection of AppleScript-Led MacOS Intrusions

Securing AI: Analysis of the Complete Security Stack with Governance and Controls

Enjoying the blog?

More in this series

From VPS to Phishing: How Darktrace Uncovered SaaS Hijacks through Virtual Infrastructure Abuse

Defending the Cloud: Stopping Cyber Threats in Azure and AWS with Darktrace

Top Eight Threats to SaaS Security and How to Combat Them

Blog

July 7, 2026

Securing AI: Analysis of the Complete Security Stack with Governance and Controls

Why traditional cybersecurity approaches are not enough for AI

How NIST defines the three core pillars of AI security

Secure

Defend

Thwart

How this looks in practice

Why agentic AI fundamentally changes enterprise cyber risk

Behavioral risk

Structural risk

Accountability

What does defense in depth look like for AI?

How security leaders should prepare for enterprise AI adoption

What capabilities should every AI security program include?

Layered Governance and Security Stack for Securing AI

AI security requires continuous visibility and behavioral detection

Blog

July 6, 2026

NIST Just Proved It: AI Security Can’t Be Solved With Rules

Static AI guardrails are inherently limited

AI security must shift from rules to behavior

Guardrails remain important but they are only one layer