The State of AI in Cybersecurity: The Impact of AI on Cybersecurity Solutions
13
May 2024
Part 3: This blog discusses the impact of AI on cybersecurity solutions based on data from Darktrace’s State of AI Cybersecurity Report. Get the latest insights into the evolving challenges faced by organizations, the growing demand for skilled professionals, and the need for integrated security solutions by downloading the full report.
About the AI Cybersecurity Report
Darktrace surveyed 1,800 CISOs, security leaders, administrators, and practitioners from industries around the globe. Our research was conducted to understand how the adoption of new AI-powered offensive and defensive cybersecurity technologies are being managed by organizations.
Overwhelming alert volumes, high false positive rates, and endlessly innovative threat actors keep security teams scrambling. Defenders have been forced to take a reactive approach, struggling to keep pace with an ever-evolving threat landscape. It is hard to find time to address long-term objectives or revamp operational processes when you are always engaged in hand-to-hand combat.
The impact of AI on the threat landscape will soon make yesterday’s approaches untenable. Cybersecurity vendors are racing to capitalize on buyer interest in AI by supplying solutions that promise to meet the need. But not all AI is created equal, and not all these solutions live up to the widespread hype.
Do security professionals believe AI will impact their security operations?
Yes! 95% of cybersecurity professionals agree that AI-powered solutions will level up their organization’s defenses.
Not only is there strong agreement about the ability of AI-powered cybersecurity solutions to improve the speed and efficiency of prevention, detection, response, and recovery, but that agreement is nearly universal, with more than 95% alignment.
This AI-powered future is about much more than generative AI. While generative AI can help accelerate the data retrieval process within threat detection, create quick incident summaries, automate low-level tasks in security operations, and simulate phishing emails and other attack tactics, most of these use cases were ranked lower in their impact to security operations by survey participants.
There are many other types of AI, which can be applied to many other use cases:
Supervised machine learning: Applied more often than any other type of AI in cybersecurity. Trained on attack patterns and historical threat intelligence to recognize known attacks.
Natural language processing (NLP): Applies computational techniques to process and understand human language. It can be used in threat intelligence, incident investigation, and summarization.
Large language models (LLMs): Used in generative AI tools, this type of AI applies deep learning models trained on massively large data sets to understand, summarize, and generate new content. The integrity of the output depends upon the quality of the data on which the AI was trained.
Unsupervised machine learning: Continuously learns from raw, unstructured data to identify deviations that represent true anomalies. With the correct models, this AI can use anomaly-based detections to identify all kinds of cyber-attacks, including entirely unknown and novel ones.
What are the areas of cybersecurity AI will impact the most?
Improving threat detection is the #1 area within cybersecurity where AI is expected to have an impact.
The most frequent response to this question, improving threat detection capabilities in general, was top ranked by slightly more than half (57%) of respondents. This suggests security professionals hope that AI will rapidly analyze enormous numbers of validated threats within huge volumes of fast-flowing events and signals. And that it will ultimately prove a boon to front-line security analysts. They are not wrong.
Identifying exploitable vulnerabilities (mentioned by 50% of respondents) is also important. Strengthening vulnerability management by applying AI to continuously monitor the exposed attack surface for risks and high-impact vulnerabilities can give defenders an edge. If it prevents threats from ever reaching the network, AI will have a major downstream impact on incident prevalence and breach risk.
Where will defensive AI have the greatest impact on cybersecurity?
Cloud security (61%), data security (50%), and network security (46%) are the domains where defensive AI is expected to have the greatest impact.
Respondents selected broader domains over specific technologies. In particular, they chose the areas experiencing a renaissance. Cloud is the future for most organizations, and the effects of cloud adoption on data and networks are intertwined. All three domains are increasingly central to business operations, impacting everything everywhere.
Responses were remarkably consistent across demographics, geographies, and organization sizes, suggesting that nearly all survey participants are thinking about this similarly—that AI will likely have far-reaching applications across the broadest fields, as well as fewer, more specific applications within narrower categories.
Going forward, it will be paramount for organizations to augment their cloud and SaaS security with AI-powered anomaly detection, as threat actors sharpen their focus on these targets.
How will security teams stop AI-powered threats?
Most security stakeholders (71%) are confident that AI-powered security solutions are better able to block AI-powered threats than traditional tools.
There is strong agreement that AI-powered solutions will be better at stopping AI-powered threats (71% of respondents are confident in this), and there’s also agreement (66%) that AI-powered solutions will be able to do so automatically. This implies significant faith in the ability of AI to detect threats both precisely and accurately, and also orchestrate the correct response actions.
There is also a high degree of confidence in the ability of security teams to implement and operate AI-powered solutions, with only 30% of respondents expressing doubt. This bodes well for the acceptance of AI-powered solutions, with stakeholders saying they’re prepared for the shift.
On the one hand, it is positive that cybersecurity stakeholders are beginning to understand the terms of this contest—that is, that only AI can be used to fight AI. On the other hand, there are persistent misunderstandings about what AI is, what it can do, and why choosing the right type of AI is so important. Only when those popular misconceptions have become far less widespread can our industry advance its effectiveness.
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
These systems, which are exposed to the internet, are often targeted by threat actors to gain initial access to a network. They are constantly being scanned for vulnerabilities, known or unknown, by opportunistic actors hoping to exploit gaps in security. Unfortunately, this exposure remains a significant blind spot for many security teams, as monitoring edge infrastructure can be particularly challenging due to its distributed nature and the sheer volume of external traffic it processes.
In this blog, we discuss a vulnerability that was exploited in Fortinet’s FortiClient Endpoint Management Server (EMS) and the post-exploitation activity that Darktrace observed across multiple customer environments.
What is FortiClient EMS?
FortiClient is typically used for endpoint security, providing features such as virtual private networks (VPN), malware protection, and web filtering. The FortiClient EMS is a centralized platform used by administrators to enforce security policies and manage endpoint compliance. As endpoints are remote and distributed across various locations, the EMS needs to be accessible over the internet.
However, being exposed to the internet presents significant security risks, and exploiting vulnerabilities in the system may give an attacker unauthorized access. From there, they could conduct further malicious activities such as reconnaissance, establishing command-and-control (C2), moving laterally across the network, and accessing sensitive data.
CVE-2023-48788
CVE-2023-48788 is a critical SQL injection vulnerability in FortiClient EMS that can allow an attacker to gain unauthorized access to the system. It stems from improper neutralization of special elements used in SQL commands, which allows attackers to exploit the system through specially crafted requests, potentially leading to Remote Code Execution (RCE) [1]. This critical vulnerability was given a CVSS score of 9.8 and can be exploited without authentication.
The affected versions of FortiClient EMS include:
FortiClient EMS 7.2.0 to 7.2.2 (fixed in 7.2.3)
FortiClient EMS 7.0.1 to 7.0.10 (fixed in 7.0.11)
The vulnerability was publicly disclosed on March 12, 2024, and an exploit proof of concept was released by Horizon3.ai on March 21 [2]. Starting from March 24, almost two weeks after the initial disclosure, Darktrace began to observe at least six instances where the FortiClient EMS vulnerability had likely been exploited on customer networks. Seemingly exploited devices in multiple customer environments were observed performing anomalous activities, including the installation of Remote Monitoring and Management (RMM) tools, which was also reported by other security vendors around the same time [3].
Darktrace’s Coverage
Initial Access
To understand how the vulnerability can be exploited to gain initial access, we first need to explain some components of the FortiClient EMS:
The service FmcDaemon.exe is used for communication between the EMS and enrolled endpoint clients. It listens on port 8013 for incoming client connections.
Incoming requests are then sent to FCTDas.exe, which translates requests from other server components into SQL requests. This service interacts with the Microsoft SQL database.
Endpoint clients communicate with the FmcDaemon on the server on port 8013 by default.
Therefore, an SQL injection attack can be performed by crafting a malicious payload and sending it over port 8013 to the server. To carry out RCE, an attacker may send further SQL statements to enable and use the xp_cmdshell functionality of the Microsoft SQL server [2].
Shortly before post-exploitation activity began, Darktrace had observed incoming connections to some of the FortiClient EMS devices over port 8013 from the external IPs 77.246.103[.]110, 88.130.150[.]101, and 45.155.141[.]219. This likely represented the threat actors sending an SQL injection payload over port 8013 to the EMS device to validate the exploit.
Establish C2
After exploiting the vulnerability and gaining access to an EMS device on one customer network, two additional devices were seen with HTTP POST requests to 77.246.103[.]110 and 212.113.106[.]100 with a new PowerShell user agent.
Interestingly, the IP 212.113.106[.]100 has been observed in various other campaigns where threat actors have also targeted internet-facing systems and exploited other vulnerabilities. Open-source intelligence (OSINT) suggests that this indicator of compromise (IoC) is related to the Sliver C2 framework and has been used by threat actors such as APT28 (Fancy Bear) and APT29 (Cozy Bear) [4].
Unusual file downloads were also observed on four devices, including:
“SETUP.MSI” from 212.32.243[.]25 and 89.149.200[.]91 with a cURL user agent
“setup.msi” from 212.113.106[.]100 with a Windows Installer user agent
“run.zip” from 95.181.173[.]172 with a PowerShell user agent
The .msi files would typically contain the RMM tools Atera or ScreenConnect [5]. By installing RMM tools for C2, attackers can leverage their wide range of functionalities to carry out various tasks, such as file transfers, without the need to install additional tools. As RMM tools are designed to maintain a stable connection to remote systems, they may also allow the attackers to ensure persistent access to the compromised systems.
A scan of the endpoint 95.181.173[.]172 shows various other files such as “RunSchedulerTask.ps1” and “anydesk.exe” being hosted.
Shortly after these unusual file downloads, many of the devices were also seen with usage of RMM tools such as Splashtop, Atera, and AnyDesk. The devices were seen connecting to the following endpoints:
*[.]relay.splashtop[.]com
agent-api[.]atera[.]com
api[.]playanext[.]com with user agent AnyDesk/8.0.9
RMM tools have a wide range of legitimate capabilities that allow IT administrators to remotely manage endpoints. However, they can also be repurposed for malicious activities, allowing threat actors to maintain persistent access to systems, execute commands remotely, and even exfiltrate data. As the use of RMM tools can be legitimate, they offer threat actors a way to perform malicious activities while blending into normal business operations, which could evade detection by human analysts or traditional security tools.
One device was also seen making repeated SSL connections to a self-signed endpoint “azure-documents[.]com” (104.168.140[.]84) and further HTTP POSTs to “serv1[.]api[.]9hits[.]com/we/session” (128.199.207[.]131). Although the contents of these connections were encrypted, they were likely additional infrastructure used for C2 in addition to the RMM tools that were used. Self-signed certificates may also be used by an attacker to encrypt C2 communications.
Internal Reconnaissance
Following the exploit, two of the compromised devices then started to conduct internal reconnaissance activity. The following figure shows a spike in the number of internal connections made by one of the compromised devices on the customer’s environment, which typically indicates a network scan.
Reconnaissance tools such as Advanced Port Scanner (“www[.]advanced-port-scanner[.]com”) and Nmap were also seen being used by one of the devices to conduct scanning activities. Nmap is a network scanning tool commonly used by security teams for legitimate purposes like network diagnostics and vulnerability scanning. However, it can also be abused by threat actors to perform network reconnaissance, a technique known as Living off the Land (LotL). This not only reduces the need for custom or external tools but also reduces the risk of exposure, as the use of a legitimate tool in the network is unlikely to raise suspicion.
Privilege Escalation
In another affected customer network, the threat actor’s attempt to escalate their privileges was also observed, as a FortiClient EMS device was seen with an unusually large number of SMB/NTLM login failures, indicative of brute force activity. This attempt was successful, and the device was later seen authenticating with the credential “administrator”.
Lateral Movement
After escalating privileges, attempts to move laterally throughout the same network were seen. One device was seen transferring the file “PSEXESVC.exe” to another device over SMB. This file is associated with PsExec, a command-line tool that allows for remote execution on other systems.
The threat actor was also observed leveraging the DCE-RPC protocol to move laterally within the network. Devices were seen with activity such as an increase in new RPC services, unusual requests to the SVCCTL endpoint, and the execution of WMI commands. The DCE-RPC protocol is typically used to facilitate communication between services on different systems and can allow one system to request services or execute commands on another.
These are further examples of LotL techniques used by threat actors exploiting CVE-2023-48788, as PsExec and the DCE-RPC protocol are often also used for legitimate administrative operations.
Accomplish Mission
In most cases, the threat actor’s end goal was not clearly observed. However, Darktrace did detect one instance where an unusually large volume of data had been uploaded to “put[.]io”, a cloud storage service, indicating that the end goal of the threat actor had been to steal potentially sensitive data.
In a recent investigation of a Medusa ransomware incident that took place in July 2024, Darktrace’s Threat Research team found that initial access to the environment had likely been gained through a FortiClient EMS device. An incoming connection from 209.15.71[.]121 over port 8013 was seen, suggesting that CVE-2023-48788 had been exploited. The device had been compromised almost three weeks before the ransomware was actually deployed, eventually resulting in the encryption of files.
Conclusion
Threat actors have continued to exploit unpatched vulnerabilities in internet-facing systems to gain initial access to a network. This highlights the importance of addressing and patching vulnerabilities as soon as they are disclosed and a fix is released. However, due to the rapid nature of exploitation, this may not always be enough. Furthermore, threat actors may even be exploiting vulnerabilities that are not yet publicly known.
As the end goals for a threat actor can differ – from data exfiltration to deploying ransomware – the post-exploitation behavior can also vary from actor to actor. However, AI security tools such as Darktrace / NETWORK can help identify and alert for post-exploitation behavior based on abnormal activity seen in the network environment.
Despite CVE-2023-48788 having been publicly disclosed and fixed in March, it appears that multiple threat actors, such as the Medusa ransomware group, have continued to exploit the vulnerability on unpatched systems. With new vulnerabilities being disclosed almost every other day, security teams may find it challenging continuously patch their systems.
As such, Darktrace / Proactive Exposure Management could also alleviate the workload of security teams by helping them identify and prioritize the most critical vulnerabilities in their network.
Credit to Emily Megan Lim (Cyber Security Analyst) and Ryan Traill (Threat Content Lead)
Lifting the Fog: Darktrace’s Investigation into Fog Ransomware
Introduction to Fog Ransomware
As ransomware attacks continue to be launched at an alarming rate, Darktrace’s Threat Research team has identified that familiar strains like Akira, LockBit, and BlackBasta remain among the most prevalent threats impacting its customers, as reported in the First 6: Half-Year Threat Report 2024. Despite efforts by law agencies, like dismantling the infrastructure of cybercriminals and shutting down their operations [2], these groups continue to adapt and evolve.
As such, it is unsurprising that new ransomware variants are regularly being created and launched to get round law enforcement agencies and increasingly adept security teams. One recent example of this is Fog ransomware.
What is Fog ransomware?
Fog ransomware is strain that first appeared in the wild in early May 2024 and has been observed actively using compromised virtual private network (VPN) credentials to gain access to organization networks in the education sector in the United States.
Darktrace's detection of Fog Ransomware
In June 2024, Darktrace observed instances of Fog ransomware across multiple customer environments. The shortest time observed from initial access to file encryption in these attacks was just 2 hours, underscoring the alarming speed with which these threat actors can achieve their objectives.
Darktrace identified key activities typical of a ransomware kill chain, including enumeration, lateral movement, encryption, and data exfiltration. In most cases, Darktrace was able to successfully halt the progression Fog attacks in their early stages by applying Autonomous Response actions such as quarantining affected devices and blocking suspicious external connections.
To effectively illustrate the typical kill chain of Fog ransomware, this blog focuses on customer environments that did not have Darktrace’s Autonomous Response enabled. In these cases, the attack progressed unchecked and reached its intended objectives until the customer received Darktrace’s alerts and intervened.
Darktrace’s Coverage of Fog Ransomware
Initial Intrusion
After actors had successfully gained initial access into customer networks by exploiting compromised VPN credentials, Darktrace observed a series of suspicious activities, including file shares, enumeration and extensive scanning. In one case, a compromised domain controller was detected making outgoing NTLM authentication attempts to another internal device, which was subsequently used to establish RDP connections to a Windows server running Hyper-V.
Given that the source was a domain controller, the attacker could potentially relay the NTLM hash to obtain a domain admin Kerberos Ticket Granting Ticket (TGT). Additionally, incoming NTLM authentication attempts could be triggered by tools like Responder, and NTLM hashes used to encrypt challenge response authentication could be abused by offline brute-force attacks.
Darktrace also observed the use of a new administrative credential on one affected device, indicating that malicious actors were likely using compromised privileged credentials to conduct relay attacks.
Establish Command-and-Control Communication (C2)
In many instances of Fog ransomware investigated by Darktrace’s Threat Research team, devices were observed making regular connections to the remote access tool AnyDesk. This was exemplified by consistent communication with the endpoint “download[.]anydesk[.]com” via the URI “/AnyDesk.exe”. In other cases, Darktrace identified the use of another remote management tool, namely SplashTop, on customer servers.
In ransomware attacks, threat actors often use such legitimate remote access tools to establish command-and-control (C2) communication. The use of such services not only complicates the identification of malicious activities but also enables attackers to leverage existing infrastructure, rather than having to implement their own.
Internal Reconnaissance
Affected devices were subsequently observed making an unusual number of failed internal connections to other internal locations over ports such as 80 (HTTP), 3389 (RDP), 139 (NetBIOS) and 445 (SMB). This pattern of activity strongly indicated reconnaissance scanning behavior within affected networks. A further investigation into these HTTP connections revealed the URIs “/nice ports”/Trinity.txt.bak”, commonly associated with the use of the Nmap attack and reconnaissance tool.
Simultaneously, some devices were observed engaging in SMB actions targeting the IPC$ share and the named pipe “srvsvc” on internal devices. Such activity aligns with the typical SMB enumeration tactics, whereby attackers query the list of services running on a remote host using a NULL session, a method often employed to gather information on network resources and vulnerabilities.
Lateral Movement
As attackers attempted to move laterally through affected networks, Darktrace observed suspicious RDP activity between infected devices. Multiple RDP connections were established to new clients, using devices as pivots to propagate deeper into the networks, Following this, devices on multiple networks exhibited a high volume of SMB read and write activity, with internal share drive file names being appended with the “.flocked” extension – a clear sign of ransomware encryption. Around the same time, multiple “readme.txt” files were detected being distributed across affected networks, which were later identified as ransom notes.
Further analysis of the ransom note revealed that it contained an introduction to the Fog ransomware group, a summary of the encryption activity that had been carried out, and detailed instructions on how to communicate with the attackers and pay the ransom.
Data Exfiltration
In one of the cases of Fog ransomware, Darktrace’s Threat Research team observed potential data exfiltration involving the transfer of internal files to an unusual endpoint associated with the MEGA file storage service, “gfs302n515[.]userstorage[.]mega[.]co[.]nz”.
This exfiltration attempt suggests the use of double extortion tactics, where threat actors not only encrypt victim’s data but also exfiltrate it to threaten public exposure unless a ransom is paid. This often increases pressure on organizations as they face the risk of both data loss and reputational damage caused by the release of sensitive information.
Darktrace’s Cyber AI Analyst autonomously investigated what initially appeared to be unrelated events, linking them together to build a full picture of the Fog ransomware attack for customers’ security teams. Specifically, on affected networks Cyber AI Analyst identified and correlated unusual scanning activities, SMB writes, and file appendages that ultimately suggested file encryption.
Conclusion
As novel and fast-moving ransomware variants like Fog persist across the threat landscape, the time taken for from initial compromise to encryption has significantly decreased due to the enhanced skill craft and advanced malware of threat actors. This trend particularly impacts organizations in the education sector, who often have less robust cyber defenses and significant periods of time during which infrastructure is left unmanned, and are therefore more vulnerable to quick-profit attacks.
Traditional security methods may fall short against these sophisticated attacks, where stealthy actors evade detection by human-managed teams and tools. In these scenarios Darktrace’s AI-driven product suite is able to quickly detect and respond to the initial signs of compromise through autonomous analysis of any unusual emerging activity.
When Darktrace’s Autonomous Response capability was active, it swiftly mitigated emerging Fog ransomware threats by quarantining devices exhibiting malicious behavior to contain the attack and blocking the exfiltration of sensitive data, thus preventing customers from falling victim to double extortion attempts.
Credit to Qing Hong Kwa (Senior Cyber Analyst and Deputy Analyst Team Lead, Singapore) and Ryan Traill (Threat Content Lead)
Appendices
Darktrace Model Detections:
- Anomalous Server Activity::Anomalous External Activity from Critical Network Device
- Anomalous Connection::SMB Enumeration
- Anomalous Connection::Suspicious Read Write Ratio and Unusual SMB
- Anomalous Connection::Uncommon 1 GiB Outbound
- Anomalous File::Internal::Additional Extension Appended to SMB File