Blog
/
/

Lessons Learned from the Florida Water Attack

Learn how the Florida water plant attack reshapes our approach to cybersecurity. It’s time to enhance defenses against modern digital threats.
Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Matthew Wainwright
CISO, Middletown Rhode Island
Default blog imageDefault blog imageDefault blog imageDefault blog imageDefault blog imageDefault blog image

United States Municipalities & Cyber Risk

Carried out in the shadows with ones and zeros, there’s a new age of attack against public authorities and critical public services up and down the United States. Municipalities are regularly infiltrated by criminals and hostile nation states – in many cases, they are held to ransom and blackmailed until they pay a hefty cost. And, the perpetrators will likely never be brought to justice.

Earlier this month, news broke that hackers had tampered with a water treatment facility in Florida. In doing so, the cyber-criminals were able to remotely control a computer to change the chemical levels of the water supply, increasing the amount of sodium hydroxide before a supervisor was able to catch the act in real time and revert the changes.

The attack on the City of Oldsmar is not the first of its kind. Last year, Israel’s water management facilities, specifically its agricultural water pumps, were hit by two cyber-attacks: one in Upper Galilee and one in the central province of Mateh Yehuda. The attacks were played down, but reports claimed that hackers did attempt to alter water chlorine levels before being detected and stopped.

Cyber-attacks exist in the digital space, but today they can have a real and tangible effect on the physical world. I worry that we face a ticking time bomb.

Defending Critical Infrastructure from Cyber Attacks

This isn’t because defenders of critical national infrastructure aren’t good at their jobs – it’s because the challenge we face is incredibly vast and complex; it is no longer a problem that humans can deal with alone.

It’s a rule of thumb in cyber security that the more sensitive your system, the less you want it to touch the Internet. However, ‘air-gapping’ – a security measure used to ensure a secure network is physically isolated from unsecured networks – is now widely accepted to be neither fool proof against today’s constantly-mutating hacker, nor a practical option.

Rapid digital transformation of critical national infrastructure in recent years means that there is now a path for attackers to run from spoof emails in an employee’s inbox right through to critical gas compressors and turbines. Importantly, this path doesn’t even need to be direct for the attack to succeed – in February last year a US Gas Pipeline was shut down for two days because of a ransomware attack that only managed to spread as far as the control panels used by operators to monitor the process. But by taking these systems offline, the attackers forced the operators to deliberately shut down the process to preserve safety.

In the case of the Florida attack, an obvious move gave the game away and, thankfully, none of the 15,000 residents were harmed. But incidents like this keep defenders of critical national infrastructure awake at night – I know, because they used to be my biggest concern too.

Critical Infrastructure Threat Trends

Over the last 12 months, we have seen a sharp rise in sophisticated, stealthy attackers that slip under the radar unnoticed. What will happen the next time an attacker breaks into critical systems, but there is no obvious move that gives them away, to warn security teams of foul play before it's too late?

We can’t put the brakes on digital transformation across heavy industry – nor would we want to since economies flounder if they are not constantly accelerating. But it’s time for a fundamental shift in how we think about the cyber challenge. No city or local government is immune to similar attacks and this is not a one-off.

The future of OT security for critical national infrastructure

The next generation of attacks we face requires next generation security solutions, which have the intelligence to act swiftly on behalf of humans. Over the last decade, I have learned that simply trying to stop attackers from getting in is futile – that only just about works for low-level attacks. Building taller walls is no longer enough against sophisticated attacks like those deployed against critical national infrastructure. Today, artificial intelligence is so advanced that it is capable of spotting the early warning signs of sophisticated attacks as they emerge and stopping them before they escalate – crucially, identifying the subtle indicators that are often imperceivable to the human eye.

Critical environments cannot afford to fail. An operational outage of even a few seconds when public safety is at stake is not tolerable. There isn’t the option of reverting to pen and paper and muddling along. We need to build in cyber resilience so these systems are able to resist and fight back against cyber-attacks.

Now that industrial environments cannot simply be air-gapped to keep them safe, we need to invest in artificial intelligence systems that can work in the background to automatically and dynamically block attacks that bleed from IT and defend critical systems 24/7.

Attackers, both criminal and state-sponsored, will continue to see critical national infrastructure as an attractive target – but today we have advanced technology available to us that can stop their attempts in their tracks. All we have to do is embrace it.

Find out more about the Industrial Immune System

Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Matthew Wainwright
CISO, Middletown Rhode Island

More in this series

No items found.

Blog

/

Network

/

August 8, 2025

Ivanti Under Siege: Investigating the Ivanti Endpoint Manager Mobile Vulnerabilities (CVE-2025-4427 & CVE-2025-4428)

ivanti cve exploitation edge infrastructure Default blog imageDefault blog image

Ivanti & Edge infrastructure exploitation

Edge infrastructure exploitations continue to prevail in today’s cyber threat landscape; therefore, it was no surprise that recent Ivanti Endpoint Manager Mobile (EPMM) vulnerabilities CVE-2025-4427 and CVE-2025-4428 were exploited targeting organizations in critical sectors such as healthcare, telecommunications, and finance across the globe, including across the Darktrace customer base in May 2025.

Exploiting these types of vulnerabilities remains a popular choice for threat actors seeking to enter an organization’s network to perform malicious activity such as cyber espionage, data exfiltration and ransomware detonation.

Vulnerabilities in Ivanti EPMM

Ivanti EPMM allows organizations to manage and configure enterprise mobile devices. On May 13, 2025, Ivanti published a security advisory [1] for their Ivanti Endpoint Manager Mobile (EPMM) devices addressing a medium and high severity vulnerability:

  • CVE-2025-4427, CVSS: 5.6: An authentication bypass vulnerability
  • CVE-2025-4428, CVSS: 7.2: Remote code execution vulnerability

Successfully exploiting both vulnerabilities at the same time could lead to unauthenticated remote code execution from an unauthenticated threat actor, which could allow them to control, manipulate, and compromise managed devices on a network [2].

Shortly after the disclosure of these vulnerabilities, external researchers uncovered evidence that they were being actively exploited in the wild and identified multiple indicators of compromise (IoCs) related to post-exploitation activities for these vulnerabilities [2] [3]. Research drew particular attention to the infrastructure utilized in ongoing exploitation activity, such as leveraging the two vulnerabilities to eventually deliver malware contained within ELF files from Amazon Web Services (AWS) S3 bucket endpoints and to deliver KrustyLoader malware for persistence. KrustyLoader is a Rust based malware that was discovered being downloaded in compromised Ivanti Connect Secure systems back in January 2024 when the zero-day critical vulnerabilities; CVE-2024-21887 and CVE-2023-46805 [10].

This suggests the involvement of the threat actor UNC5221, a suspected China-nexus espionage actor [3].

In addition to exploring the post-exploit tactics, techniques, and procedures (TTPs) observed for these vulnerabilities across Darktrace’s customer base, this blog will also examine the subtle changes and similarities in the exploitation of earlier Ivanti vulnerabilities—specifically Ivanti Connect Secure (CS) and Policy Secure (PS) vulnerabilities CVE-2023-46805 and CVE-2024-21887 in early 2024, as well as CVE-2025-0282 and CVE-2025-0283, which affected CS, PS, and Zero Trust Access (ZTA) in January 2025.

Darktrace Coverage

In May 2025, shortly after Ivanti disclosed vulnerabilities in their EPMM product, Darktrace’s Threat Research team identified attack patterns potentially linked to the exploitation of these vulnerabilities across multiple customer environments. The most noteworthy attack chain activity observed included exploit validation, payload delivery via AWS S3 bucket endpoints, subsequent delivery of script-based payloads, and connections to dpaste[.]com, possibly for dynamic payload retrieval. In a limited number of cases, connections were also made to an IP address associated with infrastructure linked to SAP NetWeaver vulnerability CVE-2025-31324, which has been investigated by Darktrace in an earlier case.

Exploit Validation

Darktrace observed devices within multiple customer environments making connections related to Out-of-Band Application Security Testing (OAST). These included a range of DNS requests and connections, most of which featured a user agent associated with the command-line tool cURL, directed toward associated endpoints. The hostnames of these endpoints consisted of a string of randomly generated characters followed by an OAST domain, such as 'oast[.]live', 'oast[.]pro', 'oast[.]fun', 'oast[.]site', 'oast[.]online', or 'oast[.]me'. OAST endpoints can be leveraged by malicious actors to trigger callbacks from targeted systems, such as for exploit validation. This activity, likely representing the initial phase of the attack chain observed across multiple environments, was also seen in the early stages of previous investigations into the exploitation of Ivanti vulnerabilities [4]. Darktrace also observed similar exploit validation activity during investigations conducted in January 2024 into the Ivanti CS vulnerabilities CVE-2023-46805 and CVE-2024-21887.

Payload Delivery via AWS

Devices across multiple customer environments were subsequently observed downloading malicious ELF files—often with randomly generated filenames such as 'NVGAoZDmEe'—from AWS S3 bucket endpoints like 's3[.]amazonaws[.]com'. These downloads occurred over HTTP connections, typically using wget or cURL user agents. Some of the ELF files were later identified to be KrustyLoader payloads using open-source intelligence (OSINT). External researchers have reported that the KrustyLoader malware is executed in cases of Ivanti EPMM exploitation to gain and maintain a foothold in target networks [2].

In one customer environment, after connections were made to the endpoint fconnect[.]s3[.]amazonaws[.]com, Darktrace observed the target system downloading the ELF file mnQDqysNrlg via the user agent Wget/1.14 (linux-gnu). Further investigation of the file’s SHA1 hash (1dec9191606f8fc86e4ae4fdf07f09822f8a94f2) linked it to the KrustyLoader malware [5]. In another customer environment, connections were instead made to tnegadge[.]s3[.]amazonaws[.]com using the same user agent, from which the ELF file “/dfuJ8t1uhG” was downloaded. This file was also linked to KrustyLoader through its SHA1 hash (c47abdb1651f9f6d96d34313872e68fb132f39f5) [6].

The pattern of activity observed so far closely mirrors previous exploits associated with the Ivanti vulnerabilities CVE-2023-46805 and CVE-2024-21887 [4]. As in those cases, Darktrace observed exploit validation using OAST domains and services, along with the use of AWS endpoints to deliver ELF file payloads. However, in this instance, the delivered payload was identified as KrustyLoader malware.

Later-stage script file payload delivery

In addition to the ELF file downloads, Darktrace also detected other file downloads across several customer environments, potentially representing the delivery of later-stage payloads.

The downloaded files included script files with the .sh extension, featuring randomly generated alphanumeric filenames. One such example is “4l4md4r.sh”, which was retrieved during a connection to the IP address 15.188.246[.]198 using a cURL-associated user agent. This IP address was also linked to infrastructure associated with the SAP NetWeaver remote code execution vulnerability CVE-2025-31324, which enables remote code execution on NetWeaver Visual Composer. External reporting has attributed this infrastructure to a China-nexus state actor [7][8][9].

In addition to the script file downloads, devices on some customer networks were also observed making connections to pastebin[.]com and dpaste[.]com, two sites commonly used to host or share malicious payloads or exploitation instructions [2]. Exploits, including those targeting Ivanti EPMM vulnerabilities, can dynamically fetch malicious commands from sites like dpaste[.]com, enabling threat actors to update payloads. Unlike the previously detailed activity, this behavior was not identified in any prior Darktrace investigations into Ivanti-related vulnerabilities, suggesting a potential shift in the tactics used in post-exploitation stages of Ivanti attacks.

Conclusion

Edge infrastructure vulnerabilities, such as those found in Ivanti EPMM and investigated across customer environments with Darktrace / NETWORK, have become a key tool in the arsenal of attackers in today’s threat landscape. As highlighted in this investigation, while many of the tactics employed by threat actors following successful exploitation of vulnerabilities remain the same, subtle shifts in their methods can also be seen.

These subtle and often overlooked changes enable threat actors to remain undetected within networks, highlighting the critical need for organizations to maintain continuous extended visibility, leverage anomaly based behavioral analysis, and deploy machine speed intervention across their environments.

Credit to Nahisha Nobregas (Senior Cyber Analyst) and Anna Gilbertson (Senior Cyber Analyst)

Appendices

Mid-High Confidence IoCs

(IoC – Type - Description)

-       trkbucket.s3.amazonaws[.]com – Hostname – C2 endpoint

-       trkbucket.s3.amazonaws[.]com/NVGAoZDmEe – URL – Payload

-       tnegadge.s3.amazonaws[.]com – Hostname – C2 endpoint

-       tnegadge.s3.amazonaws[.]com/dfuJ8t1uhG – URL – Payload

-       c47abdb1651f9f6d96d34313872e68fb132f39f5 - SHA1 File Hash – Payload

-       4abfaeadcd5ab5f2c3acfac6454d1176 - MD5 File Hash - Payload

-       fconnect.s3.amazonaws[.]com – Hostname – C2 endpoint

-       fconnect.s3.amazonaws[.]com/mnQDqysNrlg – URL - Payload

-       15.188.246[.]198 – IP address – C2 endpoint

-       15.188.246[.]198/4l4md4r.sh?grep – URL – Payload

-       185.193.125[.]65 – IP address – C2 endpoint

-       185.193.125[.]65/c4qDsztEW6/TIGHT_UNIVERSITY – URL – C2 endpoint

-       d8d6fe1a268374088fb6a5dc7e5cbb54 – MD5 File Hash – Payload

-       64.52.80[.]21 – IP address – C2 endpoint

-       0d8da2d1.digimg[.]store – Hostname – C2 endpoint

-       134.209.107[.]209 – IP address – C2 endpoint

Darktrace Model Detections

-       Compromise / High Priority Tunnelling to Bin Services (Enhanced Monitoring Model)

-       Compromise / Possible Tunnelling to Bin Services

-       Anomalous Server Activity / New User Agent from Internet Facing System

-       Compliance / Pastebin

-       Device / Internet Facing Device with High Priority Alert

-       Anomalous Connection / Callback on Web Facing Device

-       Anomalous File / Script from Rare External Location

-       Anomalous File / Incoming ELF File

-       Device / Suspicious Domain

-       Device / New User Agent

-       Anomalous Connection / Multiple Connections to New External TCP Port

-       Anomalous Connection / New User Agent to IP Without Hostname

-       Anomalous File / EXE from Rare External Location

-       Anomalous File / Internet Facing System File Download

-       Anomalous File / Multiple EXE from Rare External Locations

-       Compromise / Suspicious HTTP and Anomalous Activity

-       Device / Attack and Recon Tools

-       Device / Initial Attack Chain Activity

-       Device / Large Number of Model Alerts

-       Device / Large Number of Model Alerts from Critical Network Device

References

1.     https://forums.ivanti.com/s/article/Security-Advisory-Ivanti-Endpoint-Manager-Mobile-EPMM?language=en_US

2.     https://blog.eclecticiq.com/china-nexus-threat-actor-actively-exploiting-ivanti-endpoint-manager-mobile-cve-2025-4428-vulnerability

3.     https://www.wiz.io/blog/ivanti-epmm-rce-vulnerability-chain-cve-2025-4427-cve-2025-4428

4.     https://www.darktrace.com/blog/the-unknown-unknowns-post-exploitation-activities-of-ivanti-cs-ps-appliances

5.     https://www.virustotal.com/gui/file/ac91c2c777c9e8638ec1628a199e396907fbb7dcf9c430ca712ec64a6f1fcbc9/community

6.     https://www.virustotal.com/gui/file/f3e0147d359f217e2aa0a3060d166f12e68314da84a4ecb5cb205bd711c71998/community

7.     https://www.virustotal.com/gui/ip-address/15.188.246.198

8.     https://blog.eclecticiq.com/china-nexus-nation-state-actors-exploit-sap-netweaver-cve-2025-31324-to-target-critical-infrastructures

9.     https://www.darktrace.com/blog/tracking-cve-2025-31324-darktraces-detection-of-sap-netweaver-exploitation-before-and-after-disclosure

10.  https://www.synacktiv.com/en/publications/krustyloader-rust-malware-linked-to-ivanti-connectsecure-compromises

The content provided in this blog is published by Darktrace for general informational purposes only and reflects our understanding of cybersecurity topics, trends, incidents, and developments at the time of publication. While we strive to ensure accuracy and relevance, the information is provided “as is” without any representations or warranties, express or implied. Darktrace makes no guarantees regarding the completeness, accuracy, reliability, or timeliness of any information presented and expressly disclaims all warranties.

Nothing in this blog constitutes legal, technical, or professional advice, and readers should consult qualified professionals before acting on any information contained herein.

Any references to third-party organizations, technologies, threat actors, or incidents are for informational purposes only and do not imply affiliation, endorsement, or recommendation.

Darktrace, its affiliates, employees, or agents shall not be held liable for any loss, damage, or harm arising from the use of or reliance on the information in this blog.

The cybersecurity landscape evolves rapidly, and blog content may become outdated or superseded. We reserve the right to update, modify, or remove any content without notice.

Continue reading
About the author
Nahisha Nobregas
SOC Analyst

Blog

/

Cloud

/

August 7, 2025

How CDR & Automated Forensics Transform Cloud Incident Response

cloud security investigation guy on computer doing workDefault blog imageDefault blog image

Introduction: Cloud investigations

In cloud security, speed, automation and clarity are everything. However, for many SOC teams, responding to incidents in the cloud is often very difficult especially when attackers move fast, infrastructure is ephemeral, and forensic skills are scarce.

In this blog we will walk through an example that shows exactly how Darktrace Cloud Detection and Response (CDR) and automated cloud forensics together, solve these challenges, automating cloud detection, and deep forensic investigation in a way that’s fast, scalable, and deeply insightful.

The Problem: Cloud incidents are hard to investigate

Security teams often face three major hurdles when investigating cloud detections:

Lack of forensic expertise: Most SOCs and security teams aren’t natively staffed with forensics specialists.

Ephemeral infrastructure: Cloud assets spin up and down quickly, leaving little time to capture evidence.

Lack of existing automation: Gathering forensic-level data often requires manual effort and leaves teams scrambling around during incidents — accessing logs, snapshots, and system states before they disappear. This process is slow and often blocked by permissions, tooling gaps, or lack of visibility.

How Darktrace augments cloud investigations

1. Darktrace’s CDR finds anomalous activity in the cloud

An alert is generated for a large outbound data transfer from an externally facing EC2 instance to a rare external endpoint. It’s anomalous, unexpected, and potentially serious.

2. AI-led investigation stitches together the incident for a SOC analyst to look into

When a security incident unfolds, Darktrace’s Cyber AI Analyst TM is the first to surface it, automatically correlating behaviors, surfacing anomalies, and presenting a cohesive incident summary. It’s fast, detailed, and invaluable.

Once the incident is created, more questions are raised.

  • How were the impacted resources compromised?
  • How did the attack unfold over time – what tools and malware were used?
  • What data was accessed and exfiltrated?

What you’ll see as a SOC analyst: The incident begins in Darktrace’s Threat Visualizer, where a Cyber AI Analyst incident has been generated automatically highlighting large anomalous data transfer to a suspicious external IP. This isn’t just another alert, it’s a high-fidelity signal backed by Darktrace’s Self-Learning AI.

Cyber AI Analyst incident created for anomalous outbound data transfer
Figure 1: Cyber AI Analyst incident created for anomalous outbound data transfer

The analyst can then immediately pivot to Darktrace / CLOUD’s architecture view (see below), gaining context on the asset’s environment, ingress/egress points, connected systems, potential attack paths and whether there are any current misconfigurations detected on the asset.

Darktrace / CLOUD architecture view providing critical cloud context
Figure 2: Darktrace / CLOUD architecture view providing critical cloud context

3. Automated forensic capture — No expertise required

Then comes the game-changer, Darktrace’s recent acquisition of Cado enhances its cloud forensics capabilities. From the first alert triggered, Darktrace has already kicked in and automatically processed and analyzed a full volume capture of the EC2. Everything, past and present, is preserved. No need for manual snapshots, CLI commands, or specialist intervention.

Darktrace then provides a clear timeline highlighting the evidence and preserving it. In our example we identify:

  • A brute-force attempt on a file management app, followed by a successful login
  • A reverse shell used to gain unauthorized remote access to the EC2
  • A reverse TCP connection to the same suspicious IP flagged by Darktrace
  • Attacker commands showing how the data was split and prepared for exfiltration
  • A file (a.tar) created from two sensitive archives: product_plans.zip and research_data.zip

All of this is surfaced through the timeline view, ranked by significance using machine learning. The analyst can pivot through time, correlate events, and build a complete picture of the attack — without needing cloud forensics expertise.

Darktrace even gives the ability to:

  • Download and inspect gathered files in full detail, enabling teams to verify exactly what data was accessed or exfiltrated.
  • Interact with the file system as if it were live, allowing investigators to explore directories, uncover hidden artifacts, and understand attacker movement with precision.
Figure 3 Cado critical forensic investigation automated insights
Figure 3: Cado critical forensic investigation automated insights
Figure 4: Cado forensic file analysis of reverse shell and download option
Figure 5: a.tar created from two sensitive archives: product_plans.zip and research_data.zip
Figure 6: Traverse the full file system of the asset

Why this matters?

This workflow solves the hardest parts of cloud investigation:

  1. Capturing evidence before it disappears
  2. Understanding attacker behavior in detail - automatically
  3. Linking detections to impact with full incident visibility

This kind of insight is invaluable for organizations especially regulated industries, where knowing exactly what data was affected is critical for compliance and reporting. It’s also a powerful tool for detecting insider threats, not just external attackers.

Darktrace / CLOUD and Cado together acts as a force multiplier helping with:

  • Reducing investigation time from hours to minutes
  • Preserving ephemeral evidence automatically
  • Empowering analysts with forensic-level visibility

Cloud threats aren’t slowing down. Your response shouldn’t either. Darktrace / CLOUD + Cado gives your SOC the tools to detect, contain, and investigate cloud incidents — automatically, accurately, and at scale.

[related-resource]

Continue reading
About the author
Adam Stevens
Director of Product, Cloud Security
Your data. Our AI.
Elevate your network security with Darktrace AI