Blog

Thought Leadership

Generative AI: How Darktrace AI protects 8,400 customers from security and privacy risks

Default blog imageDefault blog imageDefault blog imageDefault blog imageDefault blog imageDefault blog image
12
Jun 2023
12
Jun 2023
This blog describes how Darktrace DETECT and RESPOND can help organizations reduce privacy and security risks related to generative AI.

Generative AI and Large Language Model (LLM) tools have entered the mainstream of public consciousness this year, with people using the likes of OpenAI’s ChatGPT and Google Bard for everything from helping web searches to using the AI capabilities to drive efficiency in the workplace.

At Darktrace, we have long understood the potential for AI to be one of the most transformative technological opportunities of our time. Our Darktrace Cyber AI Research Centre in Cambridge has been researching and developing AI tools for over a decade – tools like Darktrace DETECT™ and RESPOND™ which use a variety of AI technology to keep 8,400 customers around the world safe from cyber disruption. 

As pioneers of AI and understanding its potential to change the world, we recognize that in 2023, the AI genie is out of the bottle. AI tools are rapidly becoming part of our day to day lives. 

74% of active customer deployments have employees using generative AI tools in the workplace [1]

While generative AI tools have the power to increase productivity and augment human creativity, businesses need to move quickly to keep up with the pace of innovation. These tools carry potential privacy and security risks if used incorrectly or without proper policies in place that match the unique needs of the business – creating challenges for CISOs.

Privacy and Security Risks with Generative AI 

Government agencies like the UK’s National Cyber Security Centre (NCSC) have already issued guidance about the need to manage risk when using generative AI tools and other LLMs in the workplace. In the United States, the Cybersecurity and Infrastructure Agency (CISA) has also expressed concerns about the security implications of generative AI.

One of the reasons for this is because LLMs can learn from your prompts, storing information entered and using it to train datasets. With that data in the system, it is possible that if someone enters the right prompt, the LLM could potentially use your company’s data in response to a query.

And if the information you entered contains sensitive files or data such as intellectual property or know-how, financial reports, confidential internal documents, or sales numbers, it could become part of the third-party AI model and potentially available to others, creating privacy, intellectual property, and security risks if the appropriate guardrails are not in place. 

How Darktrace Helps Manage Generative AI Use 

In response to the growing use of generative AI tools, Darktrace has announced new risk and compliance models to help Darktrace customers address concerns around the risk of IP loss and data leakage.

We’re excited about how immensely powerful these generative AI tools are, with the capability to help people and businesses work efficiently– but like any other technology, there’s the risk that they could be inadvertently misused if not managed or monitored correctly. That’s why the new risk and compliance models for Darktrace DETECT™ and RESPOND™ make it easier for customers to put guardrails in place to monitor, and when necessary, respond to activity and connections to generative AI and LLM tools such as AutoGPT, ChatGPT, Stable Diffusion, Claude, and more. 

Each business will have its own distinct policies and needs related to generative AI tools, so we’ve also made it easier for customers to add their own list of tools to monitor for. 

Darktrace’s Self-Learning AI makes it possible to detect generative AI activity that may deviate from company policies or best practices. We bring our AI to each customer’s data, and it learns the day-to-day workings of every user, asset, and device – building an understanding of your business’s unique ‘pattern of life’.  That’s why it can detect even subtle anomalies that could indicate a threat to your business and autonomously respond, containing the threat in seconds.  

In May 2023, Darktrace Self-Learning AI detected and prevented an upload of over 1GB of data to a generative AI tool at one of its customers. [2]

With these guardrails in place, Darktrace customers can take advantage of the opportunity using generative AI and LLMs provide, while remaining protected against the potential security, IP, and privacy risks.

Using AI Safely and Responsibly

At Darktrace, we believe that recent advances in generative AI and LLMs are an important addition to the growing arsenal of AI techniques that will transform cyber security. After all, we have been utilizing AI, including LLMs and generative AI, across all of our products for years – including in Cyber AI Analyst for real time analysis of incidents, helping Darktrace customers use the power of AI to stay protected from cyber threats.

But we also believe in the responsible development and deployment of different AI techniques, which is why we are providing the tools customers need to use AI safely and responsibly. 

Our Self-Learning AI is already helping more than 8,400 businesses fight back and protect themselves against cyber threats and disruptions for the past ten years – with these new tools, CISOs can ensure that productivity is boosted by generative AI, without needing to worry about the potential security risks. Our AI learns the business in real time, all the time. It’s a Self-Learning AI. And the impact we’ve seen on improved security outcomes has been enormous.

Self-Learning AI informs Darktrace’s Cyber AI Loop, an interconnected, comprehensive set of dynamically related capabilities working together autonomously to create a continuous feedback loop to prevent, detect, respond, and heal from cyber-attacks. Ensuring that data, people, and businesses stay protected from cyber threats.

Figure 1: Darktrace Cyber AI Loop

References

[1] Based on data obtained on June 2nd, 2023, from active customer deployments with Call Home enabled, where Darktrace detected generative AI activity at some point.

[2]  Based on data obtained on June 2nd, 2023, from active customer deployments with Call Home enabled, where Darktrace detected generative AI activity at some point.

INSIDE THE SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
AUTHOR
ABOUT ThE AUTHOR
Jack Stockdale OBE
Chief Technology Officer

Jack Stockdale is the founding CTO at Darktrace. With over 20 years’ experience of software engineering, Jack is responsible for overseeing the development of Bayesian mathematical models and artificial intelligence algorithms that underpin Darktrace’s award-winning technology. Jack and his development team in Cambridge were recognized for their outstanding contribution to engineering by the Royal Academy of Engineering MacRobert Innovation Award Committee in 2017 and again in 2019. Jack has a degree in Computer Science from Lancaster University.

Book a 1-1 meeting with one of our experts
share this article
USE CASES
No items found.
COre coverage
No items found.

More in this series

No items found.

Safeguarding Distribution Centers in the Digital Age

Default blog imageDefault blog image
12
Jun 2024

Challenges securing distribution centers

For large retail providers, e-commerce organizations, logistics & supply chain organizations, and other companies who rely on the distribution of goods to consumers cybersecurity efforts are often focused on an immense IT infrastructure. However, there's a critical, often overlooked segment of infrastructure that demands vigilant monitoring and robust protection: distribution centers.

Distribution centers play a critical role in the business operations of supply chains, logistics, and the retail industry. They serve as comprehensive logistics hubs, with many organizations operating multiple centers worldwide to meet consumer needs. Depending on their size and hours of operation, even just one hour of downtime at these centers can result in significant financial losses, ranging from tens to hundreds of thousands of dollars per hour.

Due to the time-sensitive nature and business criticality of distribution centers, there has been a rise in applying modern technologies now including AI applications to enhance efficiency within these facilities. Today’s distribution centers are increasingly connected to Enterprise IT networks, the cloud and the internet to manage every stage of the supply chain. Additionally, it is common for organizations to allow 3rd party access to the distribution center networks and data for reasons including allowing them to scale their operations effectively.

However, this influx of new technologies and interconnected systems across IT, OT and cloud introduces new risks on the cybersecurity front. Distribution center networks include industrial operational technologies ICS/OT, IoT technologies, enterprise network technology, and cloud systems working in coordination. The convergence of these technologies creates a greater chance that blind spots exist for security practitioners and this increasing presence of networked technology increases the attack surface and potential for vulnerability. Thus, having cybersecurity measures that cover IT, OT or Cloud alone is not enough to secure a complex and dynamic distribution center network infrastructure.  

The OT network encompasses various systems, devices, hardware, and software, such as:

  • Enterprise Resource Planning (ERP)
  • Warehouse Execution System (WES)
  • Warehouse Control System (WCS)
  • Warehouse Management System (WMS)
  • Energy Management Systems (EMS)
  • Building Management Systems (BMS)
  • Distribution Control Systems (DCS)
  • Enterprise IT devices
  • OT and IoT: Engineering workstations, ICS application and management servers, PLCs, HMI, access control, cameras, and printers
  • Cloud applications

Distribution centers: An expanding attack surface

As these distribution centers have become increasingly automated, connected, and technologically advanced, their attack surfaces have inherently increased. Distribution centers now have a vastly different potential for cyber risk which includes:  

  • More networked devices present
  • Increased routable connectivity within industrial systems
  • Externally exposed industrial control systems
  • Increased remote access
  • IT/OT enterprise to industrial convergence
  • Cloud connectivity
  • Contractors, vendors, and consultants on site or remoting in  

Given the variety of connected systems, distribution centers are more exposed to external threats than ever before. Simultaneously, distribution center’s business criticality has positioned them as interesting targets to cyber adversaries seeking to cause disruption with significant financial impact.

Increased connectivity requires a unified security approach

When assessing the unique distribution center attack surface, the variety of interconnected systems and devices requires a cybersecurity approach that can cover the diverse technology environment.  

From a monitoring and visibility perspective, siloed IT, OT or cloud security solutions cannot provide the comprehensive asset management, threat detection, risk management, and response and remediation capabilities across interconnected digital infrastructure that a solution natively covering IT, cloud, OT, and IoT can provide.  

The problem with using siloed cybersecurity solutions to cover a distribution center is the visibility gaps that are inherently created when using multiple solutions to try and cover the totality of the diverse infrastructure. What this means is that for cross domain and multi-stage attacks, depending on the initial access point and where the adversary plans on actioning their objectives, multiple stages of the attack may not be detected or correlated if they security solutions lack visibility into OT, IT, IoT and cloud.

Comprehensive security under one solution

Darktrace leverages Self-Learning AI, which takes a new approach to cybersecurity. Instead of relying on rules and signatures, this AI trains on the specific business to learn a ‘pattern of life’ that models normal activity for every device, user, and connection. It can be applied anywhere an organization has data, and so can natively cover IT, OT, IoT, and cloud.  

With these models, Darktrace /OT provides improved visibility, threat detection and response, and risk management for proactive hardening recommendations.  

Visibility: Darktrace is the only OT security solution that natively covers IT, IoT and OT in unison. AI augmented workflows ensure OT cybersecurity analysts and operation engineers can manage IT and OT environments, leveraging a live asset inventory and tailored dashboards to optimize security workflows and minimize operator workload.

Threat detection, investigation, and response: The AI facilitates anomaly detection capable of detecting known, unknown, and insider threats and precise response for OT environments that contains threats at their earliest stages before they can jeopardize control systems. Darktrace immediately understands, identifies, and investigates all anomalous activity in OT networks, whether human or machine driven and uses Explainable AI to generate investigation reports via Darktrace’s Cyber AI Analyst.

Proactive risk identification: Risk management capabilities like attack path modeling can prioritize remediation and mitigation that will most effectively reduce derived risk scores. Rather than relying on knowledge of past attacks and CVE lists and scores, Darktrace AI learns what is ‘normal’ for its environment, discovering previously unknown threats and risks by detecting subtle shifts in behavior and connectivity. Through the application of Darktrace AI for OT environments, security teams can investigate novel attacks, discover blind spots, get live-time visibility across all their physical and digital assets, and reduce the time to detect, respond to, and triage security events.

Continue reading
About the author
Daniel Simonds
Director of Operational Technology

Blog

Inside the SOC

Medusa Ransomware: Looking Cyber Threats in the Eye with Darktrace

Default blog imageDefault blog image
10
Jun 2024

What is Living off the Land attack?

In the face of increasingly vigilant security teams and adept defense tools, attackers are continually looking for new ways to circumvent network security and gain access to their target environments. One common tactic is the leveraging of readily available utilities and services within a target organization’s environment in order to move through the kill chain; a popular method known as living off the land (LotL). Rather than having to leverage known malicious tools or write their own malware, attackers are able to easily exploit the existing infrastructure of their targets.

The Medusa ransomware group in particular are known to extensively employ LotL tactics, techniques and procedures (TTPs) in their attacks, as one Darktrace customer in the US discovered in early 2024.

What is Medusa Ransomware?

Medusa ransomware (not to be confused with MedusaLocker) was first observed in the wild towards the end of 2022 and has been a popular ransomware strain amongst threat actors since 2023 [1]. Medusa functions as a Ransomware-as-a-Service (RaaS) platform, providing would-be attackers, also know as affiliates, with malicious software and infrastructure required to carry out disruptive ransomware attacks. The ransomware is known to target organizations across many different industries and countries around the world, including healthcare, education, manufacturing and retail, with a particular focus on the US [2].

How does medusa ransomware work?

Medusa affiliates are known to employ a number of TTPs to propagate their malware, most prodominantly gaining initial access by exploiting vulnerable internet-facing assets and targeting valid local and domain accounts that are used for system administration.

The ransomware is typically delivered via phishing and spear phishing campaigns containing malicious attachments [3] [4], but it has also been observed using initial access brokers to access target networks [5]. In terms of the LotL strategies employed in Medusa compromises, affiliates are often observed leveraging legitimate services like the ConnectWise remote monitoring and management (RMM) software and PDQ Deploy, in order to evade the detection of security teams who may be unable to distinguish the activity from normal or expected network traffic [2].

According to researchers, Medusa has a public Telegram channel that is used by threat actors to post any data that may have been stolen, likely in an attempt to extort organizations and demand payment [2].  

Darktrace’s Coverage of Medusa Ransomware

Established Foothold and C2 activity

In March 2024, Darktrace /NETWORK identified over 80 devices, including an internet facing domain controller, on a customer network performing an unusual number of activities that were indicative of an emerging ransomware attack. The suspicious behavior started when devices were observed making HTTP connections to the two unusual endpoints, “wizarr.manate[.]ch” and “go-sw6-02.adventos[.]de”, with the PowerShell and JWrapperDownloader user agents.

Darktrace’s Cyber AI Analyst™ launched an autonomous investigation into the connections and was able to connect the seemingly separate events into one wider incident spanning multiple different devices. This allowed the customer to visualize the activity in chronological order and gain a better understanding of the scope of the attack.

At this point, given the nature and rarity of the observed activity, Darktrace /NETWORK's autonomous response would have been expected to take autonomous action against affected devices, blocking them from making external connections to suspicious locations. However, autonomous response was not configured to take autonomous action at the time of the attack, meaning any mitigative actions had to be manually approved by the customer’s security team.

Internal Reconnaissance

Following these extensive HTTP connections, between March 1 and 7, Darktrace detected two devices making internal connection attempts to other devices, suggesting network scanning activity. Furthermore, Darktrace identified one of the devices making a connection with the URI “/nice ports, /Trinity.txt.bak”, indicating the use of the Nmap vulnerability scanning tool. While Nmap is primarily used legitimately by security teams to perform security audits and discover vulnerabilities that require addressing, it can also be leveraged by attackers who seek to exploit this information.

Darktrace / NETWORK model alert showing the URI “/nice ports, /Trinity.txt.bak”, indicating the use of Nmap.
Figure 1: Darktrace /NETWORK model alert showing the URI “/nice ports, /Trinity.txt.bak”, indicating the use of Nmap.

Darktrace observed actors using multiple credentials, including “svc-ndscans”, which was also seen alongside DCE-RPC activity that took place on March 1. Affected devices were also observed making ExecQuery and ExecMethod requests for IWbemServices. ExecQuery is commonly utilized to execute WMI Query Language (WQL) queries that allow the retrieval of information from WI, including system information or hardware details, while ExecMethod can be used by attackers to gather detailed information about a targeted system and its running processes, as well as a tool for lateral movement.

Lateral Movement

A few hours after the first observed scanning activity on March 1, Darktrace identified a chain of administrative connections between multiple devices, including the aforementioned internet-facing server.

Cyber AI Analyst was able to connect these administrative connections and separate them into three distinct ‘hops’, i.e. the number of administrative connections made from device A to device B, including any devices leveraged in between. The AI Analyst investigation was also able to link the previously detailed scanning activity to these administrative connections, identifying that the same device was involved in both cases.

Cyber AI Analyst investigation into the chain of lateral movement activity.
Figure 2: Cyber AI Analyst investigation into the chain of lateral movement activity.

On March 7, the internet exposed server was observed transferring suspicious files over SMB to multiple internal devices. This activity was identified as unusual by Darktrace compared to the device's normal SMB activity, with an unusual number of executable (.exe) and srvsvc files transferred targeting the ADMIN$ and IPC$ shares.

Cyber AI Analyst investigation into the suspicious SMB write activity.
Figure 3: Cyber AI Analyst investigation into the suspicious SMB write activity.
Graph highlighting the number of successful SMB writes and the associated model alerts.
Figure 4: Graph highlighting the number of successful SMB writes and the associated model alerts.

The threat actor was also seen writing SQLite3*.dll files over SMB using a another credential this time. These files likely contained the malicious payload that resulted in the customer’s files being encrypted with the extension “.s3db”.

Darktrace’s visibility over an affected device performing successful SMB writes.
Figure 5: Darktrace’s visibility over an affected device performing successful SMB writes.

Encryption of Files

Finally, Darktrace observed the malicious actor beginning to encrypt and delete files on the customer’s environment. More specifically, the actor was observed using credentials previously seen on the network to encrypt files with the aforementioned “.s3db” extension.

Darktrace’s visibility over the encrypted files.
Figure 6: Darktrace’s visibility over the encrypted files.


After that, Darktrace observed the attacker encrypting  files and appending them with the extension “.MEDUSA” while also dropping a ransom note with the file name “!!!Read_me_Medusa!!!.txt”

Darktrace’s detection of threat actors deleting files with the extension “.MEDUSA”.
Figure 7: Darktrace’s detection of threat actors deleting files with the extension “.MEDUSA”.
Darktrace’s detection of the Medusa ransom note.
Figure 8: Darktrace’s detection of the Medusa ransom note.

At the same time as these events, Darktrace observed the attacker utilizing a number of LotL techniques including SSL connections to “services.pdq[.]tools”, “teamviewer[.]com” and “anydesk[.]com”. While the use of these legitimate services may have bypassed traditional security tools, Darktrace’s anomaly-based approach enabled it to detect the activity and distinguish it from ‘normal’’ network activity. It is highly likely that these SSL connections represented the attacker attempting to exfiltrate sensitive data from the customer’s network, with a view to using it to extort the customer.

Cyber AI Analyst’s detection of “services.pdq[.]tools” usage.
Figure 9: Cyber AI Analyst’s detection of “services.pdq[.]tools” usage.

If this customer had been subscribed to Darktrace's Proactive Threat Notification (PTN) service at the time of the attack, they would have been promptly notified of these suspicious activities by the Darktrace Security Operation Center (SOC). In this way they could have been aware of the suspicious activities taking place in their infrastructure before the escalation of the compromise. Despite this, they were able to receive assistance through the Ask the Expert service (ATE) whereby Darktrace’s expert analyst team was on hand to assist the customer by triaging and investigating the incident further, ensuring the customer was well equipped to remediate.  

As Darktrace /NETWORK's autonomous response was not enabled in autonomous response mode, this ransomware attack was able to progress to the point of encryption and data exfiltration. Had autonomous response been properly configured to take autonomous action, Darktrace would have blocked all connections by affected devices to both internal and external endpoints, as well as enforcing a previously established “pattern of life” on the device to stop it from deviating from its expected behavior.

Conclusion

The threat actors in this Medusa ransomware attack attempted to utilize LotL techniques in order to bypass human security teams and traditional security tools. By exploiting trusted systems and tools, like Nmap and PDQ Deploy, attackers are able to carry out malicious activity under the guise of legitimate network traffic.

Darktrace’s Self-Learning AI, however, allows it to recognize the subtle deviations in a device’s behavior that tend to be indicative of compromise, regardless of whether it appears legitimate or benign on the surface.

Further to the detection of the individual events that made up this ransomware attack, Darktrace’s Cyber AI Analyst was able to correlate the activity and collate it under one wider incident. This allowed the customer to track the compromise and its attack phases from start to finish, ensuring they could obtain a holistic view of their digital environment and remediate effectively.

Credit to Maria Geronikolou, Cyber Analyst, Ryan Traill, Threat Content Lead

Appendices

Darktrace DETECT Model Detections

Anomalous Connection / SMB Enumeration

Device / Anomalous SMB Followed By Multiple Model Alerts

Device / Suspicious SMB Scanning Activity

Device / Attack and Recon Tools

Device / Suspicious File Writes to Multiple Hidden SMB Share

Compromise / Ransomware / Ransom or Offensive Words Written to SMB

Device / Internet Facing Device with High Priority Alert

Device / Network Scan

Anomalous Connection / Powershell to Rare External

Device / New PowerShell User Agent

Possible HTTP Command and Control

Extensive Suspicious DCE-RPC Activity

Possible SSL Command and Control to Multiple Endpoints

Suspicious Remote WMI Activity

Scanning of Multiple Devices

Possible Ransom Note Accessed over SMB

List of Indicators of Compromise (IoCs)

IoC – Type – Description + Confidence

207.188.6[.]17      -     IP address   -      C2 Endpoint

172.64.154[.]227 - IP address -        C2 Endpoint

wizarr.manate[.]ch  - Hostname -       C2 Endpoint

go-sw6-02.adventos[.]de.  Hostname  - C2 Endpoint

.MEDUSA             -        File extension     - Extension to encrypted files

.s3db               -             File extension    -  Created file extension

SQLite3-64.dll    -        File           -               Used tool

!!!Read_me_Medusa!!!.txt - File -   Ransom note

Svc-ndscans         -         Credential     -     Possible compromised credential

Svc-NinjaRMM      -       Credential      -     Possible compromised credential

MITRE ATT&CK Mapping

Discovery  - File and Directory Discovery - T1083

Reconnaissance    -  Scanning IP            -          T1595.001

Reconnaissance -  Vulnerability Scanning -  T1595.002

Lateral Movement -Exploitation of Remote Service -  T1210

Lateral Movement - Exploitation of Remote Service -   T1210

Lateral Movement  -  SMB/Windows Admin Shares     -    T1021.002

Lateral Movement   -  Taint Shared Content          -            T1080

Execution   - PowerShell     - T1059.001

Execution  -   Service Execution   -    T1059.002

Impact   -    Data Encrypted for Impact  -  T1486

References

[1] https://unit42.paloaltonetworks.com/medusa-ransomware-escalation-new-leak-site/

[2] https://thehackernews.com/2024/01/medusa-ransomware-on-rise-from-data.html

[3] https://www.trustwave.com/en-us/resources/blogs/trustwave-blog/unveiling-the-latest-ransomware-threats-targeting-the-casino-and-entertainment-industry/

[4] https://www.sangfor.com/farsight-labs-threat-intelligence/cybersecurity/security-advisory-for-medusa-ransomware

[5] https://thehackernews.com/2024/01/medusa-ransomware-on-rise-from-data.html

[6]https://any.run/report/8be3304fec9d41d44012213ddbb28980d2570edeef3523b909af2f97768a8d85/e4c54c9d-12fd-477f-8cbb-a20f8fb98912

Continue reading
About the author
Maria Geronikolou
Cyber Analyst
Our ai. Your data.

Elevate your cyber defenses with Darktrace AI

Start your free trial
Darktrace AI protecting a business from cyber threats.