Blog
/
AI
/
January 30, 2025

Reimagining Your SOC: Overcoming Alert Fatigue with AI-Led Investigations  

Reimagining your SOC Part 2/3: This blog explores how the challenges facing the modern SOC can be addressed by transforming the investigation process, unlocking efficiency and scalability in SOC operations with AI.
Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Brittany Woodsmall
Product Marketing Manager, AI
Default blog imageDefault blog imageDefault blog imageDefault blog imageDefault blog imageDefault blog image
30
Jan 2025

The efficiency of a Security Operations Center (SOC) hinges on its ability to detect, analyze and respond to threats effectively. With advancements in AI and automation, key early SOC team metrics such as Mean Time to Detect (MTTD) have seen significant improvements:

  • 96% of defenders believing AI-powered solutions significantly boost the speed and efficiency of prevention, detection, response, and recovery.
  • Organizations leveraging AI and automation can shorten their breach lifecycle by an average of 108 days compared to those without these technologies.

While tool advances have improved performance and effectiveness in the detection phase, this has not been as beneficial to the next step of the process where initial alerts are investigated further to determine their relevance and how they relate to other activities. This is often measured with the metric Mean Time to Analysis (MTTA), although some SOC teams operate a two-level process with teams for initial triage to filter out more obviously uninteresting alerts and for more detailed analysis of the remainder. SOC teams continue to grapple with alert fatigue, overwhelmed analysts, and inefficient triage processes, preventing them from achieving the operational efficiency necessary for a high-performing SOC.

Addressing this core inefficiency requires extending AI's capabilities beyond detection to streamline and optimize the following investigative workflows that underpin effective analysis.

Challenges with SOC alert investigation

Detecting cyber threats is only the beginning of a much broader challenge of SOC efficiency. The real bottleneck often lies in the investigation process.

Detection tools and techniques have evolved significantly with the use of machine learning methods, improving early threat detection. However, after a detection pops up, human analysts still typically step in to evaluate the alert, gather context, and determine whether it’s a true threat or a false alarm and why. If it is a threat, further investigation must be performed to understand the full scope of what may be a much larger problem. This phase, measured by the mean time to analysis, is critical for swift incident response.

Challenges with manual alert investigation:

  • Too many alerts
  • Alerts lack context
  • Cognitive load sits with analysts
  • Insufficient talent in the industry
  • Fierce competition for experienced analysts

For many organizations, investigation is where the struggle of efficiency intensifies. Analysts face overwhelming volumes of alerts, a lack of consolidated context, and the mental strain of juggling multiple systems. With a worldwide shortage of 4 million experienced level two and three SOC analysts, the cognitive burden placed on teams is immense, often leading to alert fatigue and missed threats.

Even with advanced systems in place not all potential detections are investigated. In many cases, only a quarter of initial alerts are triaged (or analyzed). However, the issue runs deeper. Triaging occurs after detection engineering and alert tuning, which often disable many alerts that could potentially reveal true threats but are not accurate enough to justify the time and effort of the security team. This means some potential threats slip through unnoticed.

Understanding alerts in the SOC: Stopping cyber incidents is hard

Let’s take a look at the cyber-attack lifecycle and the steps involved in detecting and stopping an attack:

First we need a trace of an attack…

The attack will produce some sort of digital trace. Novel attacks, insider threats, and attacker techniques such as living-off-the-land can make attacker activities extremely hard to distinguish.

A detection is created…

Then we have to detect the trace, for example some beaconing to a rare domain. Initial detection alerts being raised underpin the MTTD (mean time to detection). Reducing this initial unseen duration is where we have seen significant improvement with modern threat detection tools.

When it comes to threat detection, the possibilities are vast. Your initial lead could come from anything: an alert about unusual network activity, a potential known malware detection, or an odd email. Once that lead comes in, it’s up to your security team to investigate further and determine if this is this a legitimate threat or a false alarm and what the context is behind the alert.

Investigation begins…

It doesn’t just stop at a detection. Typically, humans also need to look at the alert, investigate, understand, analyze, and conclude whether this is a genuine threat that needs a response. We normally measure this as MTTA (mean time to analyze).

Conducting the investigation effectively requires a high degree of skill and efficiency, as every second counts in mitigating potential damage. Security teams must analyze the available data, correlate it across multiple sources, and piece together the timeline of events to understand the full scope of the incident. This process involves navigating through vast amounts of information, identifying patterns, and discerning relevant details. All while managing the pressure of minimizing downtime and preventing further escalation.

Containment begins…

Once we confirm something as a threat, and the human team determines a response is required and understand the scope, we need to contain the incident. That's normally the MTTC (mean time to containment) and can be further split into immediate and more permanent measures.

For more about how AI-led solutions can help in the containment stage read here: Autonomous Response: Streamlining Cybersecurity and Business Operations

The challenge is not only in 1) detecting threats quickly, but also 2) triaging and investigating them rapidly and with precision, and 3) prioritizing the most critical findings to avoid missed opportunities. Effective investigation demands a combination of advanced tools, robust workflows, and the expertise to interpret and act on the insights they generate. Without these, organizations risk delaying critical containment and response efforts, leaving them vulnerable to greater impacts.

While there are further steps (remediation, and of course complete recovery) here we will focus on investigation.

Developing an AI analyst: How Darktrace replicates human investigation

Darktrace has been working on understanding the investigative process of a skilled analyst since 2017. By conducting internal research between Darktrace expert SOC analysts and machine learning engineers, we developed a formalized understanding of investigative processes. This understanding formed the basis of a multi-layered AI system that systematically investigates data, taking advantage of the speed and breadth afforded by machine systems.

With this research we found that the investigative process often revolves around iterating three key steps: hypothesis creation, data collection, and results evaluation.

All these details are crucial for an analyst to determine the nature of a potential threat. Similarly, they are integral components of our Cyber AI Analyst which is an integral component across our product suite. In doing so, Darktrace has been able to replicate the human-driven approach to investigating alerts using machine learning speed and scale.

Here’s how it works:

  • When an initial or third-party alert is triggered, the Cyber AI Analyst initiates a forensic investigation by building multiple hypotheses and gathering relevant data to confirm or refute the nature of suspicious activity, iterating as necessary, and continuously refining the original hypothesis as new data emerges throughout the investigation.
  • Using a combination of machine learning including supervised and unsupervised methods, NLP and graph theory to assess activity, this investigation engine conducts a deep analysis with incidents raised to the human team only when the behavior is deemed sufficiently concerning.
  • After classification, the incident information is organized and processed to generate the analysis summary, including the most important descriptive details, and priority classification, ensuring that critical alerts are prioritized for further action by the human-analyst team.
  • If the alert is deemed unimportant, the complete analysis process is made available to the human team so that they can see what investigation was performed and why this conclusion was drawn.
Darktrace cyber ai analyst workflow, how it works

To illustrate this via example, if a laptop is beaconing to a rare domain, the Cyber AI Analyst would create hypotheses including whether this could be command and control traffic, data exfiltration, or something else. The AI analyst then collects data, analyzes it, makes decisions, iterates, and ultimately raises a new high-level incident alert describing and detailing its findings for human analysts to review and follow up.

Learn more about Darktrace's Cyber AI Analyst

  • Cost savings: Equivalent to adding up to 30 full-time Level 2 analysts without increasing headcount
  • Minimize business risk: Takes on the busy work from human analysts and elevates a team’s overall decision making
  • Improve security outcomes: Identifies subtle, sophisticated threats through holistic investigations

Unlocking an efficient SOC

To create a mature and proactive SOC, addressing the inefficiencies in the alert investigation process is essential. By extending AI's capabilities beyond detection, SOC teams can streamline and optimize investigative workflows, reducing alert fatigue and enhancing analyst efficiency.

This holistic approach not only improves Mean Time to Analysis (MTTA) but also ensures that SOCs are well-equipped to handle the evolving threat landscape. Embracing AI augmentation and automation in every phase of threat management will pave the way for a more resilient and proactive security posture, ultimately leading to a high-performing SOC that can effectively safeguard organizational assets.

Every relevant alert is investigated

The Cyber AI Analyst is not a generative AI system, or an XDR or SEIM aggregator that simply prompts you on what to do next. It uses a multi-layered combination of many different specialized AI methods to investigate every relevant alert from across your enterprise, native, 3rd party, and manual triggers, operating at machine speed and scale. This also positively affects detection engineering and alert tuning, because it does not suffer from fatigue when presented with low accuracy but potentially valuable alerts.

Retain and improve analyst skills

Transferring most analysis processes to AI systems can risk team skills if they don't maintain or build them and if the AI doesn't explain its process. This can reduce the ability to challenge or build on AI results and cause issues if the AI is unavailable. The Cyber AI Analyst, by revealing its investigation process, data gathering, and decisions, promotes and improves these skills. Its deep understanding of cyber incidents can be used for skill training and incident response practice by simulating incidents for security teams to handle.

Create time for cyber risk reduction

Human cybersecurity professionals excel in areas that require critical thinking, strategic planning, and nuanced decision-making. With alert fatigue minimized and investigations streamlined, your analysts can avoid the tedious data collection and analysis stages and instead focus on critical decision-making tasks such as implementing recovery actions and performing threat hunting.

Stay tuned for part 3/3

Part 3/3 in the Reimagine your SOC series explores the preventative security solutions market and effective risk management strategies.

Coming soon!

Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Brittany Woodsmall
Product Marketing Manager, AI

More in this series

No items found.

Blog

/

AI

/

April 30, 2026

Mythos vs Ethos: Defending in an Era of AI‑Accelerated Vulnerability Discovery

mythos vulnerability discoveryDefault blog imageDefault blog image

Anthropic’s Mythos and what it means for security teams

Recent attention on systems such as Anthropic Mythos highlights a notable problem for defenders. Namely that disclosure’s role in coordinating defensive action is eroding.

As AI systems gain stronger reasoning and coding capability, their usefulness in analyzing complex software environments and identifying weaknesses naturally increases. What has changed is not attacker motivation, but the conditions under which defenders learn about and organize around risk. Vulnerability discovery and exploitation increasingly unfold in ways that turn disclosure into a retrospective signal rather than a reliable starting point for defense.

Faster discovery was inevitable and is already visible

The acceleration of vulnerability discovery was already observable across the ecosystem. Publicly disclosed vulnerabilities (CVEs) have grown at double-digit rates for the past two years, including a 32% increase in 2024 according to NIST, driven in part by AI even prior to Anthropic’s Mythos model. Most notably XBOW topped the HackerOne US bug bounty leaderboard, marking the first time an autonomous penetration tester had done so.  

The technical frontier for AI capabilities has been described elsewhere as jagged, and the implication is that Mythos is exceptional but not unique in this capability. While Mythos appears to make significant progress in complex vulnerability analysis, many other models are already able to find and exploit weaknesses to varying degrees.  

What matters here is not which model performs best, but the fact that vulnerability discovery is no longer a scarce or tightly bounded capability.

The consequence of this shift is not simply earlier discovery. It is a change in the defender-attacker race condition. Disclosure once acted as a rough synchronization point. While attackers sometimes had earlier knowledge, disclosure generally marked the moment when risk became visible and defensive action could be broadly coordinated. Increasingly, that coordination will no longer exist. Exploitation may be underway well before a CVE is published, if it is published at all.

Why patch velocity alone is not the answer

The instinctive response to this shift is to focus on patching faster, but treating patch velocity as the primary solution misunderstands the problem. Most organizations are already constrained in how quickly they can remediate vulnerabilities. Asset sprawl, operational risk, testing requirements, uptime commitments, and unclear ownership all limit response speed, even when vulnerabilities are well understood.

If discovery and exploitation now routinely precede disclosure, then patching cannot be the first line of defense. It becomes one necessary control applied within a timeline that has already shifted. This does not imply that organizations should patch less. It means that patching cannot serve as the organizing principle for defense.

Defense needs a more stable anchor

If disclosure no longer defines when defense begins, then defense needs a reference point that does not depend on knowing the vulnerability in advance.  

Every digital environment has a behavioral character. Systems authenticate, communicate, execute processes, and access resources in relatively consistent ways over time. These patterns are not static rules or signatures. They are learned behaviors that reflect how an organization operates.

When exploitation occurs, even via previously unknown vulnerabilities, those behavioral patterns change.

Attackers may use novel techniques, but they still need to gain access, create processes, move laterally, and will ultimately interact with systems in ways that diverge from what is expected. That deviation is observable regardless of whether the underlying weakness has been formally named.

In an environment where disclosure can no longer be relied on for timing or coordination, behavioral understanding is no longer an optional enhancement; it becomes the only consistently available defensive signal.

Detecting risk before disclosure

Darktrace’s threat research has consistently shown that malicious activity often becomes visible before public disclosure.

In multiple cases, including exploitation of Ivanti, SAP NetWeaver, and Trimble Cityworks, Darktrace detected anomalous behavior days or weeks ahead of CVE publication. These detections did not rely on signatures, threat intelligence feeds, or awareness of the vulnerability itself. They emerged because systems began behaving in ways that did not align with their established patterns.

This reflects a defensive approach grounded in ‘Ethos’, in contrast to the unbounded exploration represented by ‘Mythos’. Here, Mythos describes continuous vulnerability discovery at speed and scale. Ethos reflects an understanding of what is normal and expected within a specific environment, grounded in observed behavior.

Revisiting assume breach

These conditions reinforce a principle long embedded in Zero Trust thinking: assume breach.

If exploitation can occur before disclosure, patching vulnerabilities can no longer act as the organizing principle for defense. Instead, effective defense must focus on monitoring for misuse and constraining attacker activity once access is achieved. Behavioral monitoring allows organizations to identify early‑stage compromise and respond while uncertainty remains, rather than waiting for formal verification.

AI plays a critical role here, not by predicting every exploit, but by continuously learning what normal looks like within a specific environment and identifying meaningful deviation at machine speed. Identifying that deviation enables defenders to respond by constraining activity back towards normal patterns of behavior.

Not an arms race, but an asymmetry

AI is often framed as fueling an arms race between attackers and defenders. In practice, the more important dynamic is asymmetry.

Attackers operate broadly, scanning many environments for opportunities. Defenders operate deeply within their own systems, and it’s this business context which is so significant. Behavioral understanding gives defenders a durable advantage. Attackers may automate discovery, but they cannot easily reproduce what belonging looks like inside a particular organization.

A changed defensive model

AI‑accelerated vulnerability discovery does not mean defenders have lost. It does mean that disclosure‑driven, patch‑centric models no longer provide a sufficient foundation for resilience.

As vulnerability volumes grow and exploitation timelines compress, effective defense increasingly depends on continuous behavioral understanding, detection that does not rely on prior disclosure, and rapid containment to limit impact. In this model, CVEs confirm risk rather than define when defense begins.

The industry has already seen this approach work in practice. As AI continues to reshape both offense and defense, behavioral detection will move from being complementary to being essential.

Continue reading
About the author
Andrew Hollister
Principal Solutions Engineer, Cyber Technician

Blog

/

Network

/

April 29, 2026

Darktrace Malware Analysis: Jenkins Honeypot Reveals Emerging Botnet Targeting Online Games

botnetDefault blog imageDefault blog image

DDoS Botnet discovery

To observe adversary behavior in real time, Darktrace operates a global honeypot network known as “CloudyPots”, designed to capture malicious activity across a wide range of services, protocols, and cloud platforms. These honeypots provide valuable insights into the techniques, tools, and malware actively targeting internet‑facing infrastructure.

How attackers used a Jenkins honeypot to deploy the botnet

One such software honeypotted by Darktrace is Jenkins, a CI build system that allows developers to build code and run tests automatically. The instance of Jenkins in Darktrace’s honeypot is intentionally configured with a weak password, allowing attackers to obtain remote code execution on the service.

In one instance observed by Darktrace on March 18, 2026, a threat actor seemingly attempted to target Darktrace’s Jenkins honeypot to deploy a distributed denial-of-service (DDoS) botnet. Further analysis by Darktrace’s Threat Research team revealed the botnet was intended to specifically target video game servers.

How the Jenkins scriptText endpoint was used for remote code execution

The Jenkins build system features an endpoint named scriptText, which enables users to programmatically send new jobs, in the form of a Groovy script. Groovy is a programming language with similar syntax to Java and runs using the Java Virtual Machine (JVM). An attacker can abuse the scriptText endpoint to run a malicious script, achieving code execution on the victim host.

Request sent to the scriptText endpoint containing the malicious script.
Figure 1: Request sent to the scriptText endpoint containing the malicious script.

The malicious script is sent using the form-data content type, which results in the contents of the script being URL encoded. This encoding can be decoded to recover the original script, as shown in Figure 2, where Darktrace Analysts decoded the script using CyberChef,

The malicious script decoded using CyberChef.
Figure 2: The malicious script decoded using CyberChef.

What happens after Jenkins is compromised

As Jenkins can be deployed on both Microsoft Windows and Linux systems, the script includes separate branches to target each platform.

In the case of Windows, the script performs the following actions:

  • Downloads a payload from 103[.]177.110.202/w.exe and saves it to C:\Windows\Temp\update.dat.
  • Renames the “update.dat” file to “win_sys.exe” (within the same folder)
  • Runs the Unblock-File command is used to remove security restrictions typically applied to files downloaded from the internet.
  • Adds a firewall allow rule is added for TCP port 5444, which the payload uses for command-and-control (C2) communications.

On Linux systems, the script will instead use a Bash one-liner to download the payload from 103[.]177.110.202/bot_x64.exe to /tmp/bot and execute it.

Why this botnet uses a single IP for delivery and command and control

The IP 103[.]177.110.202 belongs to Webico Company Limited, specifically its Tino brand, a Vietnamese company that offers domain registrar services and server hosting. Geolocation data indicates that the IP is located in Ho Chi Minh City. Open-source intelligence (OSINT) analysis revealed multiple malicious associations tied to the IP [1].

Darktrace’s analysis found that the IP 103[.]177.110.202 is used for multiple stages of an attack, including spreading and initial access, delivering payloads, and C2 communication. This is an unusual combination, as many malware families separate their spreading servers from their C2 infrastructure. Typically, malware distribution activity results in a high volume of abuse complaints, which may result in server takedowns or service suspension by internet providers. Separate C2 infrastructure ensures that existing infections remain controllable even if the spreading server is disrupted.

How the malware evades detection and maintains persistence

Analysis of the Linux payload (bot _x64)

The sample begins by setting the environmental variables BUILD_ID and JENKINS_NODE_COOKIE to “dontKillMe”. By default, Jenkins terminates long-running scripts after a defined timeout period; however, setting these variables to “dontKillMe” bypasses this check, allowing the script to continue running uninterrupted.

The script then performs several stealth behaviors to evade detection. First, it deletes the original executable from disk and then renames itself to resemble the legitimate kernel processes “ksoftirqd/0” or “kworker”, which are found on Linux installations by default. It then uses a double fork to daemonize itself, enabling it to run in the background, before redirecting standard input, standard output, and standard error to /dev/null, hiding any logging from the malware. Finally, the script creates a signal handler for signals such as SIGTERM, causing them to be ignored and making it harder to stop the process.

Stealth component of the main function
Figure 3: Stealth component of the main function

How the botnet communicates with command and control (C2)

The sample then connects to the C2 server and sends the detected architecture of the system on which the agent was installed. The malware then enters a loop to handle incoming commands.

The sample features two types of commands, utility commands used to manage the malware, and commands to trigger attacks. Three special commands are defined: “PING” (which replies with PONG as a keep-alive mechanism), “!stop” which causes the malware to exit, and “!update”, which triggers the malware to download a new version from the C2 server and restart itself.

Initial connection to the C2 sever.
Figure 4: Initial connection to the C2 sever.

What DDoS attack techniques this botnet uses

The attack commands consist of the following:

Many of these commands invoke the same function despite appearing to be different attack techniques. For example, specialized attacks such as Cloudflare bypass (cfbypass, uam) use the exact same function as a standard HTTP attack. This may indicate the threat actor is attempting to make the botnet look like it has more capabilities than it actually has, or it could suggest that these commands are placeholders for future attack functionality that has yet to be implemented

All the commands take three arguments: IP, port to attack, and the duration of the attack.

attack_udp and attack_udp_pps

The attack_udp and attack_udp_pps functions both use a basic loop and sendto system call to send UDP packets to the victim’s IP, either targeting a predetermined port or a random port. The attack_udp function sends packets with 1,450 bytes of data, aimed at bandwidth saturation, while the attack_udp_pps function sends smaller 64-byte packets. In both cases, the data body of the packet consists of entirely random data.

Code for the UDP attack method
Figure 5: Code for the UDP attack method

attack_dayz

The attack_dayz function follows a similar structure to the attack_udp function; however, instead of sending random data, it will instead send a TSource Engine Query. This command is specific to Valve Source Engine servers and is designed to return a large volume of data about the targeted server. By repeatedly flooding this request, an attacker can exhaust the resources of a server using a comparatively small amount of data.

The Valve Source Engine server, also called Source Engine Dedicated server, is a server developed by video game company Valve that enables multiplayer gameplay for titles built using the Source game engine, which is also developed by Valve. The Source engine is used in games such as Counterstrike and Team Fortress 2. Curiously, the function attack_dayz, appears to be named after another popular online multiplayer game, DayZ; however, DayZ does not use the Valve Source Engine, making it unclear why this name was chosen.

The code for the “attack_dayz” attack function.
Figure 6: The code for the attack_dayz” attack function.

attack_tcp_push

The attack_tcp_push function establishes a TCP socket with the non-blocking flag set, allowing it to rapidly call functions such as connect() and send() without waiting for their completion. For the duration of the attack, it enters a while loop in which it repeatedly connects to the victim, sends 1,024 bytes of random data, and then closes the connection. This process repeats until the attack duration ends. If the mode flag is set to 1, the function also configures the socket with TCP no-delay enabled, allowing for packets to be sent immediately without buffering, resulting in a higher packet rate and a more effective attack.

The code for the TCP attack function.
Figure 7: The code for the TCP attack function.

attack_http

Similar to attach_tcp_push, attack_http configures a socket with no-delay enabled and non-blocking set. After establishing the connection, it sends 64 HTTP GET requests before closing the socket.

The code for the HTTP attack function.
Figure 8: The code for the HTTP attack function.

attack_special

The attack_special function creates a UDP socket and sets the port and payload based on the value of the mode flag:

  • Mode 0: Port 53 (DNS), sending a 10-byte malformed data packet.
  • Mode 1: Port 27015 (Valve Source Engine), sending the previously observed TSource Engine Query packet.
  • Mode 2: Port 123 (NTP), sending the start of an NTP control request.
The code for the attack_special function.
Figure 9: The code for the attack_special function.

What this botnet reveals about opportunistic attacks on internet-facing systems

Jenkins is one of the less frequently exploited services honeypotted by Darktrace, with only a handful campaigns observed. Nonetheless, the emergence of this new DDoS botnet demonstrates that attackers continue to opportunistically exploit any internet-facing misconfiguration at scale to grow the botnet strength.

While the hosts most commonly affected by these opportunistic attacks are usually “lower-value” systems, this distinction is largely irrelevant for botnets, where numbers alone are more important to overall effectiveness

The presence of game-specific DoS techniques further highlights that the gaming industry continues to be extensively targeted by cyber attackers, with Cloudflare reporting it as the fourth most targeted industry [2]. This botnet has likely already been used against game servers, serving as a reminder for server operators to ensure appropriate mitigations are in place.

Credit to Nathaniel Bill (Malware Research Engineer)
Edited by Ryan Traill (Content Manager)

Indicators of Compromise (IoCs)

103[.]177.110.202 - Attacker and command-and-control IP

F79d05065a2ba7937b8781e69b5859d78d5f65f01fb291ae27d28277a5e37f9b – bot_x64

References

[1] https://www.virustotal.com/gui/url/86db2530298e6335d3ecc66c2818cfbd0a6b11fcdfcb75f575b9fcce1faa00f1/detection

[2] - https://blog.cloudflare.com/ddos-threat-report-2025-q4/

Continue reading
About the author
Nathaniel Bill
Malware Research Engineer
Your data. Our AI.
Elevate your network security with Darktrace AI