ブログ
/
AI
/
November 3, 2024

AI and Cybersecurity: Predictions for 2025

Discover the role of AI in shaping cybersecurity predictions for 2025 and how organizations can prepare for emerging threats.
Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
The Darktrace Community
Default blog imageDefault blog imageDefault blog imageDefault blog imageDefault blog imageDefault blog image
03
Nov 2024

Introduction: AI cybersecurity predictions for 2025

Each year, Darktrace's AI and cybersecurity experts reflect on the events of the past 12 months and predict the trends we expect to shape the cybersecurity landscape in the year ahead. In 2024, we predicted that the global elections, fast-moving AI innovations, and increasingly cloud-based IT environments would be key factors shaping the cyber threat landscape.

Looking ahead to 2025, we expect the total addressable market of cybercrime to expand as attackers add more tactics to their toolkits. Threat actors will continue to take advantage of the volatile geopolitical environment and cybersecurity challenges will increasingly move to new frontiers like space. When it comes to AI, we anticipate the innovation in AI agents in 2024 to pave the way for the rise of multi-agent systems in 2025, creating new challenges and opportunities for cybersecurity professionals and attackers alike.

Here are ten trends to watch for in 2025:

1. The overall Total Addressable Market (TAM) of cybercrime gets bigger

Cybercrime is a global business, and an increasingly lucrative one, scaling through the adoption of AI and cybercrime-as-a-service. Annual revenue from cybercrime is already estimated to be over $8 trillion, which we’ve found is almost 5x greater than the revenue of the Magnificent Seven stocks. There are a few key factors driving this growth.

The ongoing growth of devices and systems means that existing malware families will continue to be successful. As of October 2024, it’s estimated that more than 5.52 billion people (~67%) have access to the internet and sources estimate 18.8 billion connected devices will be online by the end of 2024. The increasing adoption of AI is poised to drive even more interconnected systems as well as new data centers and infrastructure globally.

At the same time, more sophisticated capabilities are available for low-level attackers – we’ve already seen the trickle-down economic benefits of living off the land, edge infrastructure exploitation, and identity-focused exploitation. The availability of Ransomware-as-a-Service (RaaS) and Malware-as-a-Service (MaaS) make more advanced tactics the norm. The subscription income that these groups can generate enables more adversarial innovation, so attacks are getting faster and more effective with even bigger financial ramifications.

While there has also been an increasing trend in the last year of improved cross-border law enforcement, the efficacy of these efforts remains to be seen as cybercriminal gangs are also getting more resilient and professionalized. They are building better back-up systems and infrastructure as well as more multi-national networks and supply chains.

2. Security teams need to prepare for the rise of AI agents and multi-agent systems

Throughout 2024, we’ve seen major announcements about advancements in AI agents from the likes of OpenAI, Microsoft, Salesforce, and more. In 2025, we’ll see increasing innovation in and adoption of AI agents as well as the emergence of multi-agent systems (or “agent swarms”), where groups of autonomous agents work together to tackle complex tasks.

The rise of AI agents and multi-agent systems will introduce new challenges in cybersecurity, including new attack vectors and vulnerabilities. Security teams need to think about how to protect these systems to prevent data poisoning, prompt injection, or social engineering attacks.

One benefit of multi-agent systems is that agents can autonomously communicate, collaborate, and interact. However without clear and distinct boundaries and explicit permissions, this can also pose a major data privacy risk and avenue for manipulation. These issues cannot be addressed by traditional application testing alone. We must ensure these systems are secure by design, where robust protective mechanisms and data guardrails are built into the foundations.

3. Threat actors will be the earliest adopters of AI agents and multi-agent systems

We’ve already seen how quickly threat actors have been able to adopt generative AI for tasks like email phishing and reconnaissance. The next frontier for threat actors will be AI agents and multi-agent systems that are specialized in autonomous tasks like surveillance, initial access brokering, privilege escalation, vulnerability exploitation, data summarization for smart exfiltration, and more. Because they have no concern for safe, secure, accurate, and responsible use, adversaries will adopt these systems faster than cyber defenders.

We could also start to see use cases emerge for multi-agent systems in cyber defense – with potential for early use cases in incident response, application testing, and vulnerability discovery. On the whole, security teams will be slower to adopt these systems than adversaries because of the need to put in place proper security guardrails and build trust over time.

4. There is heightened supply chain risk for Large Language Models (LLMs)

Training LLMs requires a lot of data, and many experts have warned that world is running out of quality data for that training. As a result, there will be an increasing reliance on synthetic data, which can introduce new issues of accuracy and efficacy. Moreover, data supply chain risks will be an Achilles heel for organizations, with the potential interjection of vulnerabilities through the data and machine learning providers that they rely on. Poisoning one data set could have huge trickle-down impacts across many different systems. Data security will be paramount in 2025.

5. The race to identify software vulnerabilities intensifies

The time it takes for threat actors to exploit newly published CVEs is getting shorter, giving defenders an even smaller window to apply patches and remediations. A 2024 report from Cloudflare found that threat actors quickly weaponized proof of concept exploits in attacks as quickly as 22 minutes after the exploits were made public.

At the same time, 2024 also saw the first reports from researchers across academia and the tech industry using AI for vulnerability discovery in real-world code. With threat actors getting faster at exploiting vulnerabilities, defenders will need to use AI to identify vulnerabilities in their software stack and to help identify and prioritize remediations and patches.

6. Insider threat risks will force organizations to evolve zero trust strategies

In 2025, an increasingly volatile geopolitical situation and the intensity of the AI race will make insider threats an even bigger risk for businesses, forcing organizations to expand zero-trust strategies. The traditional zero-trust model provides protection from external threats to an organization’s network by requiring continuous verification of the devices and users attempting to access critical business systems, services, and information from multiple sources. However, as we have seen in the more recent Jack Teixeira case, malicious insiders can still do significant damage to an organization within their approved and authenticated boundary.

To circumvent the remaining security gaps in a zero-trust architecture and mitigate increasing risk of insider threats, organizations will need to integrate a behavioral understanding dimension to their zero-trust approaches. The zero-trust best practice of “never trust, always verify” needs to evolve to become “never trust, always verify, and continuously monitor.”

7. Identity remains an expensive problem for businesses

2024 saw some of the biggest and costliest attacks – all because the attacker had access to compromised credentials. Essentially, they had the key to the front door. Businesses still struggle with identity and access management (IAM), and it’s getting more complex now that we’re in the middle of a massive Software-as-a-Service (SaaS) migration driven by increasing rates of AI and cloud use across businesses.

This challenge is going to be exacerbated in 2025 by a few global and business factors. First, there is an increasing push for digital identities, such as the rollout of the EU Digital Identity Framework that is underway, which could introduce additional attack vectors. As they scale, businesses are turning more and more to centralized identity and access solutions with decentralized infrastructure and relying on SaaS and application-native security.

8. Increasing vulnerabilities at the edge

During the COVID-19 pandemic, many organizations had to stand-up remote access solutions quickly – in a matter of days or weeks – without the high level of due diligence that they require to be fully secured. In 2025, we expect to see continued fall-out as these quickly spun-up solutions start to present genuine vulnerability to businesses. We’ve already seen this start to play out in 2024 with the mass-exploitation of internet-edge devices like firewalls and VPN gateway products.

By July 2024, Darktrace’s threat research team observed that the most widely exploited edge infrastructure devices were those related to Ivanti Connect Secure, JetBrains TeamCity, FortiClient Enterprise Management Server, and Palo Alto Networks PAN-OS. Across the industry, we’ve already seen many zero days and vulnerabilities exploiting these internet-connected devices, which provide inroads into the network and store/cache credentials and passwords of other users that are highly valuable for threat actors.

9. Hacking Operational Technology (OT) gets easier

Hacking OT is notoriously complex – causing damage requires an intimate knowledge of the specific systems being targeted and historically was the reserve of nation states. But as OT has become more reliant and integrated with IT systems, attackers have stumbled on ways to cause disruption without having to rely on the sophisticated attack-craft normally associated with nation-state groups. That’s why some of the most disruptive attacks of the last year have come from hacktivist and financially-motivated criminal gangs – such as the hijacking of internet-exposed Programmable Logic Controllers (PLCs) by anti-Israel hacking groups and ransomware attacks resulting in the cancellation of hospital operations.  

In 2025, we expect to see an increase in cyber-physical disruption caused by threat groups motivated by political ideology or financial gain, bringing the OT threat landscape closer in complexity and scale to that of the IT landscape. The sectors most at risk are those with a strong reliance on IoT sensors, including healthcare, transportation, and manufacturing sectors.

10. Securing space infrastructure and systems becomes a critical imperative

The global space industry is growing at an incredibly fast pace, and 2025 is on track to be another record-breaking year for spaceflight with major missions and test flights planned by NASA, ESA, CNSA as well as the expected launch of the first commercial space station from Vast and programs from Blue Origin, Amazon and more. Research from Analysis Mason suggests that 38,000 additional satellites will be built and launched by 2033 and the global space industry revenue will reach $1.7 trillion by 2032. Space has also been identified as a focus area for the incoming US administration.

In 2025, we expect to see new levels of tension emerge as private and public infrastructure increasingly intersect in space, shining a light on the lack of agreed upon cyber norms and the increasing challenge of protecting complex and remote space systems against modern cyber threats.  Historically focused on securing earth-bound networks and environments, the space industry will face challenges as post-orbit threats rise, with satellites moving up the target list.

The EU’s NIS2 Directive now recognizes the space sector as an essential entity that is subject to its most strict cybersecurity requirements. Will other jurisdictions follow suit? We expect global debates about cyber vulnerabilities in space to come to the forefront as we become more reliant on space-based technology.

Conclusion: Preparing for the future

Whatever 2025 brings, Darktrace is committed to providing robust cybersecurity leadership and solutions to enterprises around the world. Our team of subject matter experts will continue to monitor emerging threat trends, advising both our customers and our product development teams.

And for day-to-day security, our multi-layered AI cybersecurity platform can protect against all types of threats, whether they are known, unknown, entirely novel, or powered by AI. It accomplishes this by learning what is normal for your unique organization, therefore identifying unusual and suspicious behavior at machine speed, regardless of existing rules and signatures. In this way, organizations with Darktrace can be ready for any developments in the cybersecurity threat landscape that the new year may bring.

Discover more about Darktrace's predictions on the AI and cybersecurity landscape for 2025 by watching the full recorded webinar here.

Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
The Darktrace Community

More in this series

No items found.

Blog

/

Network

/

February 3, 2026

Darktrace Malware Analysis: Unpacking SnappyBee

Default blog imageDefault blog image

Introduction

The aim of this blog is to be an educational resource, documenting how an analyst can perform malware analysis techniques such as unpacking. This blog will demonstrate the malware analysis process against well-known malware, in this case SnappyBee.

SnappyBee (also known as Deed RAT) is a modular backdoor that has been previously attributed to China-linked cyber espionage group Salt Typhoon, also known as Earth Estries [1] [2]. The malware was first publicly documented by TrendMicro in November 2024 as part of their investigation into long running campaigns targeting various industries and governments by China-linked threat groups.

In these campaigns, SnappyBee is deployed post-compromise, after the attacker has already obtained access to a customer's system, and is used to establish long-term persistence as well as deploying further malware such as Cobalt Strike and the Demodex rootkit.

To decrease the chance of detection, SnappyBee uses a custom packing routine. Packing is a common technique used by malware to obscure its true payload by hiding it and then stealthily loading and executing it at runtime. This hinders analysis and helps the malware evade detection, especially during static analysis by both human analysts and anti-malware services.

This blog is a practical guide on how an analyst can unpack and analyze SnappyBee, while also learning the necessary skills to triage other malware samples from advanced threat groups.

First principles

Packing is not a new technique, and threat actors have generally converged on a standard approach. Packed binaries typically feature two main components: the packed data and an unpacking stub, also called a loader, to unpack and run the data.

Typically, malware developers insert a large blob of unreadable data inside an executable, such as in the .rodata section. This data blob is the true payload of the malware, but it has been put through a process such as encryption, compression, or another form of manipulation to render it unreadable. Sometimes, this data blob is instead shipped in a different file, such as a .dat file, or a fake image. When this happens, the main loader has to read this using a syscall, which can be useful for analysis as syscalls can be easily identified, even in heavily obfuscated binaries.

In the main executable, malware developers will typically include an unpacking stub that takes the data blob, performs one or more operations on it, and then triggers its execution. In most samples, the decoded payload data is loaded into a newly allocated memory region, which will then be marked as executable and executed. In other cases, the decoded data is instead dropped into a new executable on disk and run, but this is less common as it increases the likelihood of detection.

Finding the unpacking routine

The first stage of analysis is uncovering the unpacking routine so it can be reverse engineered. There are several ways to approach this, but it is traditionally first triaged via static analysis on the initial stages available to the analyst.

SnappyBee consists of two components that can be analyzed:

  • A Dynamic-link Library (DLL) that acts as a loader, responsible for unpacking the malicious code
  • A data file shipped alongside the DLL, which contains the encrypted malicious code

Additionally, SnappyBee includes a legitimate signed executable that is vulnerable to DLL side-loading. This means that when the executable is run, it will inadvertently load SnappyBee’s DLL instead of the legitimate one it expects. This allows SnappyBee to appear more legitimate to antivirus solutions.

The first stage of analysis is performing static analysis of the DLL. This can be done by opening the DLL within a disassembler such as IDA Pro. Upon opening the DLL, IDA will display the DllMain function, which is the malware’s initial entry point and the first code executed when the DLL is loaded.

The DllMain function
Figure 1: The DllMain function

First, the function checks if the variable fdwReason is set to 1, and exits if it is not. This variable is set by Windows to indicate why the DLL was loaded. According to Microsoft Developer Network (MSDN), a value of 1 corresponds to DLL_PROCESS_ATTACH, meaning “The DLL is being loaded into the virtual address space of the current process as a result of the process starting up or as a result of a call to LoadLibrary” [3]. Since SnappyBee is known to use DLL sideloading for execution, DLL_PROCESS_ATTACH is the expected value when the legitimate executable loads the malicious DLL.

SnappyBee then uses the GetModule and GetProcAddress to dynamically resolve the address of the VirtualProtect in kernel32 and StartServiceCtrlDispatcherW in advapi32. Resolving these dynamically at runtime prevents them from showing up as a static import for the module, which can help evade detection by anti-malware solutions. Different regions of memory have different permissions to control what they can be used for, with the main ones being read, write, and execute. VirtualProtect is a function that changes the permissions of a given memory region.

SnappyBee then uses VirtualProtect to set the memory region containing the code for the StartServiceCtrlDispatcherW function as writable. It then inserts a jump instruction at the start of this function, redirecting the control flow to one of the SnappyBee DLL’s other functions, and then restores the old permissions.

In practice, this means when the legitimate executable calls StartServiceCtrlDispatcherW, it will immediately hand execution back to SnappyBee. Meanwhile, the call stack now appears more legitimate to outside observers such as antimalware solutions.

The hooked-in function then reads the data file that is shipped with SnappyBee and loads it into a new memory allocation. This pattern of loading the file into memory likely means it is responsible for unpacking the next stage.

The start of the unpacking routine that reads in dbindex.dat.
Figure 2: The start of the unpacking routine that reads in dbindex.dat.

SnappyBee then proceeds to decrypt the memory allocation and execute the code.

The memory decryption routine.
Figure 3: The memory decryption routine.

This section may look complex, however it is fairly straight forward. Firstly, it uses memset to zero out a stack variable, which will be used to store the decryption key. It then uses the first 16 bytes of the data file as a decryption key to initialize the context from.

SnappyBee then calls the mbed_tls_arc4_crypt function, which is a function from the mbedtls library. Documentation for this function can be found online and can be referenced to better understand what each of the arguments mean [4].

The documentation for mbedtls_arc4_crypt.
Figure 4: The documentation for mbedtls_arc4_ crypt.

Comparing the decompilation with the documentation, the arguments SnappyBee passes to the function can be decoded as:

  • The context derived from 16-byte key at the start of the data is passed in as the context in the first parameter
  • The file size minus 16 bytes (to account for the key at the start of the file) is the length of the data to be decrypted
  • A pointer to the file contents in memory, plus 16 bytes to skip the key, is used as the input
  • A pointer to a new memory allocation obtained from VirtualAlloc is used as the output

So, putting it all together, it can be concluded that SnappyBee uses the first 16 bytes as the key to decrypt the data that follows , writing the output into the allocated memory region.

SnappyBee then calls VirtualProtect to set the decrypted memory region as Read + Execute, and subsequently executes the code at the memory pointer. This is clearly where the unpacked code containing the next stage will be placed.

Unpacking the malware

Understanding how the unpacking routine works is the first step. The next step is obtaining the actual code, which cannot be achieved through static analysis alone.

There are two viable methods to retrieve the next stage. The first method is implementing the unpacking routine from scratch in a language like Python and running it against the data file.

This is straightforward in this case, as the unpacking routine in relatively simple and would not require much effort to re-implement. However, many unpacking routines are far more complex, which leads to the second method: allowing the malware to unpack itself by debugging it and then capturing the result. This is the approach many analysts take to unpacking, and the following will document this method to unpack SnappyBee.

As SnappyBee is 32-bit Windows malware, debugging can be performed using x86dbg in a Windows sandbox environment to debug SnappyBee. It is essential this sandbox is configured correctly, because any mistake during debugging could result in executing malicious code, which could have serious consequences.

Before debugging, it is necessary to disable the DYNAMIC_BASE flag on the DLL using a tool such as setdllcharacteristics. This will stop ASLR from randomizing the memory addresses each time the malware runs and ensures that it matches the addresses observed during static analysis.

The first place to set a breakpoint is DllMain, as this is the start of the malicious code and the logical place to pause before proceeding. Using IDA, the functions address can be determined; in this case, it is at offset 10002DB0. This can be used in the Goto (CTRL+G) dialog to jump to the offset and place a breakpoint. Note that the “Run to user code” button may need to be pressed if the DLL has not yet been loaded by x32dbg, as it spawns a small process to load the DLL as DLLs cannot be executed directly.

The program can then run until the breakpoint, at which point the program will pause and code recognizable from static analysis can be observed.

Figure 5: The x32dbg dissassembly listing forDllMain.

In the previous section, this function was noted as responsible for setting up a hook, and in the disassembly listing the hook address can be seen being loaded at offset 10002E1C. It is not necessary to go through the whole hooking process, because only the function that gets hooked in needs to be run. This function will not be naturally invoked as the DLL is being loaded directly rather than via sideloading as it expects. To work around this, the Extended Instruction Pointer (EIP) register can be manipulated to point to the start of the hook function instead, which will cause it to run instead of the DllMain function.

To update EIP, the CRTL+G dialog can again be used to jump to the hook function address (10002B50), and then the EIP register can be set to this address by right clicking the first instruction and selecting “Set EIP here”. This will make the hook function code run next.

Figure 6: The start of the hookedin-in function

Once in this function, there are a few addresses where breakpoints should be set in order to inspect the state of the program at critical points in the unpacking process. These are:

-              10002C93, which allocates the memory for the data file and final code

-              10002D2D, which decrypts the memory

-              10002D81, which runs the unpacked code

Setting these can be done by pressing the dot next to the instruction listing, or via the CTRL+G Goto menu.

At the first breakpoint, the call to VirtualAlloc will be executed. The function returns the memory address of the created memory region, which is stored in the EAX register. In this case, the region was allocated at address 00700000.

Figure 7: The result of the VirtualAlloc call.

It is possible to right click the address and press “Follow in dump” to pin the contents of the memory to the lower pane, which makes it easy to monitor the region as the unpacking process continues.

Figure 8: The allocated memory region shown in x32dbg’s dump.

Single-stepping through the application from this point eventually reaches the call to ReadFile, which loads the file into the memory region.

Figure 9: The allocated memory region after the file is read into it, showing high entropy data.

The program can then be allowed to run until the next breakpoint, which after single-stepping will execute the call to mbedtls_arc4_crypt to decrypt the memory. At this point, the data in the dump will have changed.

Figure 10: The same memory region after the decryption is run, showing lower entropy data.

Right-clicking in the dump and selecting "Disassembly” will disassemble the data. This yields valid shell code, indicating that the unpacking succeeded, whereas corrupt or random data would be expected if the unpacking had failed.

Figure 11: The disassembly view of the allocated memory.

Right-clicking and selecting “Follow in memory map” will show the memory allocation under the memory map view. Right-clicking this then provides an option to dump the entire memory block to file.

Figure 12: Saving the allocated memory region.

This dump can then be opened in IDA, enabling further static analysis of the shellcode. Reviewing the shellcode, it becomes clear that it performs another layer of unpacking.

As the debugger is already running, the sample can be allowed to execute up to the final breakpoint that was set on the call to the unpacked shellcode. Stepping into this call will then allow debugging of the new shellcode.

The simplest way to proceed is to single-step through the code, pausing on each call instruction to consider its purpose. Eventually, a call instruction that points to one of the memory regions that were assigned will be reached, which will contain the next layer of unpacked code. Using the same disassembly technique as before, it can be confirmed that this is more unpacked shellcode.

Figure 13: The unpacked shellcode’s call to RDI, which points to more unpacked shellcode. Note this screenshot depicts the 64-bit variant of SnappyBee instead of 32-bit, however the theory is the same.

Once again, this can be dumped out and analyzed further in IDA. In this case, it is the final payload used by the SnappyBee malware.

Conclusion

Unpacking remains one of the most common anti-analysis techniques and is a feature of most sophisticated malware from threat groups. This technique of in-memory decryption reduces the forensic “surface area” of the malware, helping it to evade detection from anti-malware solutions. This blog walks through one such example and provides practical knowledge on how to unpack malware for deeper analysis.

In addition, this blog has detailed several other techniques used by threat actors to evade analysis, such as DLL sideloading to execute code without arising suspicion, dynamic API resolving to bypass static heuristics, and multiple nested stages to make analysis challenging.

Malware such as SnappyBee demonstrates a continued shift towards highly modular and low-friction malware toolkits that can be reused across many intrusions and campaigns. It remains vital for security teams  to maintain the ability to combat the techniques seen in these toolkits when responding to infections.

While the technical details of these techniques are primarily important to analysts, the outcomes of this work directly affect how a Security Operations Centre (SOC) operates at scale. Without the technical capability to reliably unpack and observe these samples, organizations are forced to respond without the full picture.

The techniques demonstrated here help close that gap. This enables security teams to reduce dwell time by understanding the exact mechanisms of a sample earlier, improve detection quality with behavior-based indicators rather than relying on hash-based detections, and increase confidence in response decisions when determining impact.

Credit to Nathaniel Bill (Malware Research Engineer)
Edited by Ryan Traill (Analyst Content Lead)

Indicators of Compromise (IoCs)

SnappyBee Loader 1 - 25b9fdef3061c7dfea744830774ca0e289dba7c14be85f0d4695d382763b409b

SnappyBee Loader 2 - b2b617e62353a672626c13cc7ad81b27f23f91282aad7a3a0db471d84852a9ac          

SnappyBee Payload - 1a38303fb392ccc5a88d236b4f97ed404a89c1617f34b96ed826e7bb7257e296

References

[1] https://www.trendmicro.com/en_gb/research/24/k/earth-estries.html

[2] https://www.darktrace.com/blog/salty-much-darktraces-view-on-a-recent-salt-typhoon-intrusion

[3] https://learn.microsoft.com/en-us/windows/win32/dlls/dllmain#parameters

[4] https://mbed-tls.readthedocs.io/projects/api/en/v2.28.4/api/file/arc4_8h/#_CPPv418mbedtls_arc4_cryptP20mbedtls_arc4_context6size_tPKhPh

Continue reading
About the author
Nathaniel Bill
Malware Research Engineer

Blog

/

AI

/

February 3, 2026

Introducing Darktrace / SECURE AI: Complete AI Security Across Your Enterprise

Default blog imageDefault blog image

Why securing AI can’t wait

AI is entering the enterprise faster than IT and security teams can keep up, appearing in SaaS tools, embedded in core platforms, and spun up by teams eager to move faster.  

As this adoption accelerates, it introduces unpredictable behaviors and expands the attack surface in ways existing security tools can’t see or control, startup or platform, they all lack one trait. These new types of risks command the attention of security teams and boardrooms, touching everything from business integrity to regulatory exposure.

Securing AI demands a fundamentally different approach, one that understands how AI behaves, how it interacts with data and users, and how risk emerges in real time. That shift is at the core of how organizations should be thinking about securing AI across the enterprise.

What is the current state of securing AI?

In Darktrace’s latest State of AI in Cybersecurity Report research across 1,500 cybersecurity professionals shows that the percentage of organizations without an AI adoption policy grew from 55% last year to 63% this year.

More troubling, the percentage of organizations without any plan to create an AI policy nearly tripled from 3% to 8%. Without clear policies, businesses are effectively accelerating blindfolded.

When we analyzed activity across our own customer base, we saw the same patterns playing out in their environments. Last October alone, we saw a 39% month-over-month increase in anomalous data uploads to generative AI services, with the average upload being 75MB. Given the size and frequency of these uploads, it's almost certain that much of this data should never be leaving the enterprise.

Many security teams still lack visibility into how AI is being used across their business; how it’s behaving, what it’s accessing, and most importantly, whether it’s operating safely. This unsanctioned usage quietly expands, creating pockets of AI activity that fall completely outside established security controls. The result is real organizational exposure with almost no visibility, underscoring just how widespread AI use has already become desipite the existence of formal policies.

This challenge doesn’t stop internally. Shadow AI extends into third-party tools, vendor platforms, and partner systems, where AI features are embedded without clear oversight.

Meanwhile, attackers are now learning to exploit AI’s unique characteristics, compounding the risks organizations are already struggling to manage.

The leader in AI cybersecurity now secures AI

Darktrace brings more than a decade of behavioral AI expertise built on an enterprise‑wide platform designed to operate in the complex, ambiguous environments where today’s AI now lives.  

Other cybersecurity technologies try to predict each new attack based on historical attacks. The problem is AI operates like humans do. Every action introduces new information that changes how AI behaves, its unpredictable, and historical attack tactics are now only a small part of the equation, forcing vendors to retrofit unproven acquisitions to secure AI.  

Darktrace is fundamentally different. Our Self‑Learning AI learns what “normal” looks like for your unique business: how your users, systems, applications, and now AI agents behave, how they communicate, and how data flows. This allows us to spot even the smallest shifts when something changes in meaningful ways. Long before AI agents were introduced, our technology was already interpreting nuance, detecting drift, uncovering hidden relationships, and making sense of ambiguous activity across networks, cloud, SaaS, email, OT, identities, and endpoints.

As AI introduces new behaviors, unstructured interactions, invisible pathways, and the rise of Shadow AI, these challenges have only intensified. But this is exactly the environment our platform was built for. Securing AI isn’t a new direction for Darktrace — it’s the natural evolution of the behavioral intelligence we’ve delivered to thousands of organizations worldwide.

Introducing Darktrace / SECURE AI – Complete AI security across your enterprise

We are proud to introduce Darktrace / SECURE AI, the newest product in the Darktrace ActiveAI Security Platform designed to secure AI across the whole enterprise.

This marks the next chapter in our mission to secure organizations from cyber threats and emerging risks. By combining full visibility, intelligent behavioral oversight, and real-time control, Darktrace is enabling enterprises to safely adopt, manage, and build AI within their business. This ensures that AI usage, data access, and behavior remain aligned to security baselines, compliance, and business goals.

Darktrace / SECURE AI can bring every AI interaction into a single view, helping teams understand intent, assess risk, protect sensitive data, and enforce policy across both human and AI Agent activity. Now organizations can embrace AI with confidence, with visibility to ensure it is operating safely, responsibly, and in alignment with their security and compliance needs.  

Because securing AI spans multiple areas and layers of complexity, Darktrace / SECURE AI is built around four foundational use cases that ensure your whole enterprise and every AI use affecting your business, whether owned or through third parties, is protected, they are:

  • Monitoring the prompts driving GenAI agents and assistants
  • Securing business AI agent identities in real time
  • Evaluating AI risks in development and deployment
  • Discovering and controlling Shadow AI

Monitoring the prompts driving GenAI agents and assistants

For AI systems, prompts are one of the most active and sensitive points of interaction—spanning human‑AI exchanges where users express intent and AI‑AI interactions where agents generate internal prompts to reason and coordinate. Because prompt language effectively is behavior, and because it relies on natural language rather than a fixed, finite syntax, the attack surface is open‑ended. This makes prompt‑driven risks far more complex than traditional API‑based vulnerabilities tied to CVEs.

Whether an attacker is probing for weaknesses, an employee inadvertently exposes sensitive data, or agents generate their own sub‑tasks to drive complex workflows, security teams must understand how prompt behavior shapes model behavior—and where that behavior can go wrong. Without that behavioral understanding, organizations face heightened risks of exploitation, drift, and cascading failures within their AI systems.

Darktrace / SECURE AI brings together all prompt activity across enterprise AI systems, including Microsoft Copilot and ChatGPT Enterprise, low‑code environments like Microsoft Copilot Studio, SaaS providers like Salesforce and Microsoft 365, and high‑code platforms such as AWS Bedrock and SageMaker, into a single, unified layer of visibility.  

Beyond visibility, Darktrace applies behavioral analytics to understand whether a prompt is unusual or risky in the context of the user, their peers, and the broader organization. Because AI attacks are far more complex and conversational than traditional exploits against fixed APIs – sharing more in common with email and Teams/Slack interactions, —this behavioral understanding is essential. By treating prompts as behavioral signals, Darktrace can detect conversational attacks, malicious chaining, and subtle prompt‑injection attempts, and where integrations allow, intervene in real time to block unsafe prompts or prevent harmful model actions as they occur.

Securing business AI agent identities in real time

As organizations adopt more AI‑driven workflows, we’re seeing a rapid rise in autonomous and semi‑autonomous agents operating across the business. These agents operate within existing identities, with the capability to access systems, read and write data, and trigger actions across cloud platforms, internal infrastructure, applications, APIs, and third‑party services. Some identities are controlled, like users, others like the ones mentioned, can appear anywhere, with organizations having limited visibility into how they’re configured or how their permissions evolve over time.  

Darktrace / SECURE AI gives organizations a real‑time, identity‑centric understanding of what their AI agents are doing, not just what they were designed to do. It automatically discovers live agent identities operating across SaaS, cloud, network, endpoints, OT, and email, including those running inside third‑party environments.  

The platform maps how each agent is configured, what systems it accesses, and how it communicates, including activity such as MCP usage or interactions with storage services where sensitive data may reside.  

By continuously observing agent behavior across all domains, Darktrace / SECURE AI highlights when unnecessary or risky permissions are granted, when activity patterns deviate, or when agents begin chaining together actions in unintended ways. This real‑time audit trail allows organizations to evaluate whether agent actions align with intended operational parameters and catch anomalous or risky behavior early.    

Evaluating AI risks in development and deployment

In the build phase, new identities are created, entitlements accumulate, components are stitched together across SaaS, cloud, and internal environments, and logic starts taking shape through prompts and configurations.  

It’s a highly dynamic and often fragmented process, and even small missteps here, such as a misconfiguration in a created agent identity, can become major security issues once the system is deployed. This is why evaluating AI risk during development and deployment is critical.

Darktrace / SECURE AI brings clarity and control across this entire lifecycle — from the moment an AI system starts taking shape to the moment it goes live. It allows you to gain visibility into created identities and their access across hyperscalers, low‑code SaaS, and internal labs, supported by AI security posture management that surfaces misconfigurations, over‑entitlement, and anomalous building events. Darktrace/ SECURE AI then connects these development insights directly to prompt oversight, connecting how AI is being built to how it will behave once deployed.  The result is a safer, more predictable AI lifecycle where risks are discovered early, guardrails are applied consistently, and innovations move forward with confidence rather than guesswork.

Discovering and controlling Shadow AI

Shadow AI has now appeared across every corner of the enterprise. It’s not just an employee pasting internal data into an external chatbot; it includes unsanctioned agent builders, hidden MCP servers, rogue model deployments, and AI‑driven workflows running on devices or services no one expected to be using AI.  

Darktrace / SECURE AI brings this frontier into view by continuously analyzing interactions across cloud, networks, endpoints, OT, and SASE environments. It surfaces unapproved AI usage wherever it appears and distinguishes legitimate activity in sanctioned tools from misuse or high‑risk behavior. The system identifies hidden AI components and rogue agents, reveals unauthorized deployments and unexpected connections to external AI systems, and highlights risky data flows that deviate from business norms.

When the behavior warrants a response, Darktrace / SECURE AI enables policy enforcement that guides users back toward sanctioned options while containing unsafe or ungoverned adoption. This closes one of the fastest‑expanding security gaps in modern enterprises and significantly reduces the attack surface created by shadow AI.

Conclusion

What’s needed now along with policies and frameworks for AI adoption is the right tooling to detect threats based on AI behavior across shadow use, prompt risks, identity misuse, and AI development.  

Darktrace is uniquely positioned to secure AI, we’ve spent over a decade building AI that learns your business – understanding subtle behavior across the entire enterprise long before AI agents arrived. With over 10,000 customers relying on Darktrace as the last line of defense to capture threats others cannot, Securing AI isn’t a pivot for us, it's not an acquisition; it’s the natural extension of the behavioral expertise and enterprise‑wide intelligence our platform was built on from the start.  

To learn more about how to secure AI at your organization we curated a readiness program that brings together IT and security leaders navigating this responsibility, providing a forum to prepare for high-impact decisions, explore guardrails, and guide the business amid growing uncertainty and pressure.

Sign up for the Secure AI Readiness Program here: This gives you exclusive access to the latest news on the latest AI threats, updates on emerging approaches shaping AI security, and insights into the latest innovations, including Darktrace’s ongoing work in this area.

Ready to talk with a Darktrace expert on securing AI? Register here to receive practical guidance on the AI risks that matter most to your business, paired with clarity on where to focus first across governance, visibility, risk reduction, and long-term readiness.  

Continue reading
About the author
Brittany Woodsmall
Product Marketing Manager, AI
あなたのデータ × DarktraceのAI
唯一無二のDarktrace AIで、ネットワークセキュリティを次の次元へ