June 19, 2023

Darktrace Detection of 3CX Supply Chain Attack

Explore how the 3CX supply chain compromise was uncovered, revealing key insights into the detection of sophisticated cyber threats.

Written by

Nahisha Nobregas

SOC Analyst

Inside the SOC

Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.

Written by

Nahisha Nobregas

SOC Analyst

Jun 2023

Ever since the discovery of the SolarWinds hack that affected tens of thousands of organizations around the world in 2020, supply chain compromises have remained at the forefront of the minds of security teams and continue to pose a significant threat to their business operations.

Supply chain compromises can have far-reaching implications, from disrupting an organization’s daily operations, incurring huge financial and reputational damage, to affecting the critical infrastructure of entire countries. As such, it is essential for organizations to have effective security measures in place able to identify and halt these attacks at the earliest possible stage.

In March 2023 the 3CX Desktop application became the latest victim of a supply chain compromise dubbed as the “SmoothOperator” by SentinelOne. This application is used by over 600,000 companies worldwide and the customer list contains high-profile customers across a variety of industries [2]. The 3CX Desktop application is a Voice over Internet Protocol (VoIP) communication software for enterprises that allows for chats, video calls, and voice calls. [3] The 3CX installers for both Windows and macOS systems were affected by information stealing malware. Researchers were able to discern that threat actors also known as UNC 4736 related to financially motivated North Korean operators also known as AppleJeus were responsible for the supply chain compromise. Researchers have also linked it to another supply chain compromise that occurred prior on the Trading Technologies X_TRADER platform, making this the first known cascading software supply chain compromise used to distribute malware on a wide scale and still be able to align operator interests. [3] Customer reports following the compromise began to surface about the 3CX software being picked up as malicious by several cybersecurity vendors such as CrowdStrike, SentinelOne, and Palo Alto Networks. [6]

By leveraging integrations with other security vendors like CrowdStrike and SentinelOne, Darktrace DETECT™ was able to identify activity from the “SmoothOperator” across the customer base at multiple stages of the kill chain in March 2023. Darktrace RESPOND™ was then able to autonomously intervene against these emerging threats, preventing significant disruption to customer networks.

Background on the first known cascading supply chain attack

Initial Access

In April 2023, security researchers identified the initial target in this story was not the 3CX desktop application, rather, it was another software application called X_TRADER by Trading Technologies. [3] Trading Technologies is a provider that offers high-performance financial trading packages, allowing financial professionals to analyze and trade assets within the stock market more efficiently. Unfortunately, a compromise already existed in the supply chain for this organization. The X_TRADER installer, which had been retired in 2020, still had its code signing certificate set to expire in October 2022. This code signing certificate was exploited by attackers to digitally sign the malicious software. [3] It also inopportunely led to 3CX when an employee unknowingly downloaded a trojanized installer for the X_TRADER software from Trading Technologies prior to the certificate’s expiration. [4]. This compromise of 3CX via X_TRADER was the first case of a cascading supply chain attack reported on within the wider threat landscape.

Persistence and Privilege Escalation

Following these findings, researchers were able to identify the likely kill chain that occurred on Windows systems, beginning with the download of the 3CX DesktopApp installer that executed an executable (.exe) file before dropping two trojanized Data Link Libraries (DLLs) alongside a benign executable that was used to sideload malicious DLLs. These DLLs contained and used SIGFLIP and DAVESHELL; both publicly available projects. [3] In this case, the DLLs were used to decrypt using an RC4 key and load a payload into the memory of a compromised system. [3] SIGFLIP and DAVESHELL also extract and decrypt the modular backdoor named VEILEDSIGNAL, which also contains a command and control (C2) configuration. This malware allowed the North Korean threat operators to gain administrative control to the 3CX employee’s device. [3] This was followed by access to the employee’s corporate credentials, ultimately leading to access to 3CX systems. [4]

Lateral Movement and C2 activity

Security researchers were also able to identify other malware families that were mainly utilized in the supply chain attack to move laterally within the 3CX environment, and allow for C2 communication [3], these malware families are detailed below:

TaxHaul: when executed it decrypts shellcode payload, observed by Mandiant to persist via DLL search-order hijacking.
Coldcat: complex downloader, which also beacons to a C2 infrastructure.
PoolRat: collects system information and executes commands. This is the malware that was found to affect macOS systems.
IconicStealer: served as a third stage payload on 3CX systems to steal data or information.

Furthermore, it was also reported early on by Kaspersky that a backdoor named Gopuram, routinely used by the North Korean threat actors Lazarus and typically used against cryptocurrency companies, was also used as a second stage payload on a limited number of 3CX’s customers compromised systems. [5]

3CX detections observed by Darktrace

CrowdStrike and SentinelOne, two of the major detection platforms with which Darktrace partners through security integrations, initially revealed that their platforms had identified the campaign appeared to be targeting 3CXDesktopApp customers in March 2023.

At this time, Darktrace was also observing this activity and alerting customers to unusual behavior on their networks. [1][7] Darktrace DETECT identified activity related to the supply chain compromise primarily through host-level alerts associated with CrowdStrike and SentinelOne integrations, as well as model breaches related to lateral movement and C2 activity.

Some of the activity related to the 3CX supply chain compromise that Darktrace detected was observed solely via integration models picking up executable and Microsoft Software Installer (msi) file downloads for the 3CXDesktopApp, suggesting the compromise likely was stopped at the endpoint device.

Figure 1: CrowdStrike integration model breach identifying 3CXDesktopApp[.]exe as possible malware on March 30, 2023.

Figure 2: The above figure, showcases the Model Breach Event Log for the CrowdStrike integration model breach shown in Figure 1.

In another case highlighted in Figure 3 and 4, security platforms were associating 3CX as malicious. The device in these figures was observed downloading a 3CXDesktopApp executable followed by an msi file about an hour later. This pattern of activity correlates with the compromise process that had been on reported, where the “SmoothOperator” malware that affected 3CX systems was able to persist through DLL side-loading of malicious DLL files delivered with benign executable files, making it difficult for traditional security tools to detect. [2][3][7]

The activity in this case was detected by the DETECT integration model, ‘High Severity Integration Malware Detection’ and was later blocked by the Darktrace RESPOND/Network model, ‘Antigena Significant Anomaly from Client Block’ which applied the “Enforce Pattern of Life” action to intercept the malicious download that was taking place. Darktrace RESPOND uses AI to learn every devices normal pattern of life and act autonomously to enforce its normal activity. In this event, RESPOND would not only intercept the malicious download that was taking place on the device, but also not allow the device to significantly deviate from its normal pattern of activity.

Figure 3: The Model Breach Event log for the device displays the moment in which the SentinelOne integration model breached for the 3CXDesktopApp.exe file followed subsequently by the RESPOND model, ‘Antigena Significant Anomaly from Client Block’, on March 29, 2023.

Figure 4: Another ‘High Severity Integration Malware Detection’ breached for the same device in Figure 3 approximately one hour later because of the msi file, 3CXDesktopApp-18.12.416.msi, which also led to the Darktrace RESPOND model, ‘Antigena Significant Anomaly from Client Block’, on March 29, 2023.

In a separate case, Darktrace also detected a device performing unusual SMB drive writes for the file ‘3CXDesktopApp-18.10.461.msi’. This breached the DETECT model ‘SMB Drive Write’. This model detects when a device starts writing files to another internal device it does not usually communicate with via the SMB protocol using the admin$ or drive shares.

Figure 5: This Model Breach Event log highlights the moment Darktrace captured the msi application file for the 3CXDesktopApp being transferred internally on this customer’s network, this was picked up as new activity for the device on March 28, 2023.

In a couple of other cases observed by Darktrace, connections detected were made from affected devices to 3CX compromise related endpoints. In Figure 6, the device in question was detected connecting to the endpoint, journalide[.]org. This breached the model, ‘Suspicious Self-Signed SSL’, which looks for connections being made to an endpoint with a self-signed SSL certificate which is designed to look legitimate, as self-signed certificates are often used in malware communication.

Figure 6: Model Breach Event log for connections to the 3CX C2 related endpoint, journalide[.]org, these connections breached the model Suspicious Self-Signed SSL on April 24, 2023.

On another Darktrace customer environment, a 3CX C2 endpoint, pbxphonenetwork[.]com, had already been added to the Watched Domains list around the time reports of the 3CX application software being malicious had been reported. The Watched Domains list allows Darktrace to detect if any device on the network makes connections to these domains with more scrutiny and breach a model for further visibility of threats on the network. Activity in this case was detected and subsequently blocked by a Darktrace RESPOND action, “Block connections to 89.45.67[.]160 port 443 and pbxphonenetwork[.]com on port 443”, blocking the device from connecting to this 3CX C2 endpoints on the spot (see Figure 7). This activity subsequently breached the RESPOND model, ‘Antigena Watched Domain Block’.

Figure 7: History log of the Darktrace RESPOND action applied to the device breaching the Darktrace RESPOND model, Antigena Watched Domain Block and applying the action, “Block connections to 89.45.67[.]160 port 443 and pbxphonenetwork[.]com on port 443” on March 31, 2023.

Darktrace Coverage

Utilizing integrations with Darktrace such as those with CrowdStrike and SentinelOne, Darktrace was able to detect and respond to activity identified as malicious 3CX activity by CrowdStrike and SentinelOne as seen in Figures 1, 2, 3, and 4. This activity breached the following Darktrace DETECT models:

Integration / CrowdStrike Alert
Security Integration / High Severity Integration Malware Detection

Darktrace was also able to identify lateral movement activity such as in the case illustrated in Figure 5.

Compliance / SMB Drive Write

Lastly, C2 beaconing activity from malicious endpoints associated with the 3CX compromise was also detected as seen in Figure 6, this activity breached the following Darktrace DETECT model:

Anomalous Connection / Suspicious Self-Signed SSL

For customers with Darktrace RESPOND configured in autonomous response mode, Darktrace RESPOND models also breached to activity related to the 3CX supply chain compromise as seen in Figures 3, 4, and 7. Below are the models that breached and the following autonomous actions that were applied:

Antigena / Network / Significant Anomaly / Antigena Significant Anomaly from Client Block, “Enforce pattern of life”
Antigena / Network / External Threat / Antigena Watched Domain Block, “Block connections to 89.45.67[.]160 port 443 and pbxphonenetwork[.]com on port 443”

Conclusion

The first known cascading supply chain compromise occurred inopportunely for 3CX but conveniently for UNC 4736 North Korean threat actors. This “SmoothOperator” compromise was detected by endpoint security platforms such as CrowdStrike who was at the cusp of this discovery when it became one of the first platforms to report on malicious activity related to the 3CX DesktopApp supply chain compromise.

Although still novel at the time and largely without reported indicators of compromise, Darktrace was able to capture and identify activity related to the 3CX compromise across its customer base, as well as respond autonomously to contain it. Darktrace was able to amplify security integrations with CrowdStrike and SentinelOne, and via anomaly-based model breaches, contribute unique insights by highlighting activity in varied parts of the 3CX supply chain compromise kill chain. The “SmoothOperator” supply chain attack proves that the Darktrace suite of products, including DETECT and RESPOND, can not only act autonomously to identify and respond to novel threats, but also work with security integrations to further amplify intervention and prevent cyber disruption on customer networks.

Credit to Nahisha Nobregas, SOC Analyst and Trent Kessler, SOC Analyst.