Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Written by
Max Heinemeyer
Global Field CISO
Share
03
Aug 2020
Cyber-criminals are increasingly impersonating trusted SaaS platforms and suppliers with their attacks. Recently, Darktrace has detected threats leveraging QuickBooks, WeTransfer and Microsoft Teams brand names. Many of these emails attempt to coax a recipient into clicking a malicious link that leads to a page containing credential-harvesting malware. This blog post demonstrates a possible next phase in an attack – what happens after an employee enters their details on this malicious webpage and has their account compromised.
Even just one compromised internal account can greatly increase the success rate of a phishing campaign. Attackers can use a compromised Microsoft 365 account to gain access to multiple other accounts within hours.
Darktrace’s AI was monitoring over 9,000 devices at a leading technology firm in the APAC region when one employee became victim to a Microsoft 365 account takeover over the weekend. This account was then used to send hundreds of phishing emails to both internal and external contacts. Darktrace detected the early signs of account compromise and raised a high-confidence alert to the security team well before these emails were sent. If the security team had acted quickly in response to the alert, the delivery of the phishing emails – and a second account compromise – could have been avoided.
Timeline of the attack
Figure 1: A timeline of the attack
We can see in the timeline that the attacker only spent three hours performing research before acting. This raises questions on the nature of this threat. Was the attack automated? Had the attacker done preliminary research? Did they know what they were after?
A bespoke and targeted attack
Darktrace first alerted to the security incident when the AI detected that someone was logging in from an unusual geographical location, promptly setting up new inbox rules, and viewing several shared files. The attacker then proceeded to send out over 200 phishing emails to internal and external recipients.
The emails contained a link to a Microsoft OneDrive landing page titled “Contract & Proposal – Customer,” indicating the page was specifically built for this attack. The page contained a phishing link hidden under the display text “Click to Review Fax Document.” Less than one hour after the phishing emails were sent, Darktrace’s AI detected an an unusual login from the same IP to a second account in the organization, indicating this account had likely also been compromised.
How did the attack bypass the rest of the security stack?
The attacker leveraged compromised M365 credentials, with the initial entry likely via compromised credentials from a previous phishing campaign before Darktrace’s AI was deployed;
Traditional email security software trusts internal emails;
Phishing emails contained a OneDrive link, a trusted SaaS platform, so other email security products would not have identified these links as suspicious.
AI Analyst investigates
The technology firm had deployed Darktrace’s Enterprise Immune System across their network and SaaS applications, and consequently had real-time visibility across every event in this attack as it unfolded. Additionally, when the unusual login location was detected, Darktrace’s Cyber AI Analyst immediately launched an automated investigation into the malicious activity, generating a natural language summary of the events and other crucial information to help with incident review.
Figure 2: An excerpt of Cyber AI Analyst’s report of the account hijack
Darktrace’s SaaS Console also reported on the event in the context of activity on that device over the previous week.
Figure 3: Darktrace’s SaaS dashboard displaying an overview of the incident
This attack is another example of the changing nature of cyber-threats in the context of digital transformation. It is not devices, but identities that are increasingly being targeted and attacked.
Darktrace’s real-time alerting on the evolving situation could have enabled the security team to isolate the initial compromised account and change the credentials before the attack escalated further. The initial rare login destination caused Darktrace’s Cyber AI Analyst to launch an ongoing investigation into the compromised account, such that an alert was raised just three minutes after new processing rules were set up by the attacker. With eyes on the technology, a more serious breach could have been avoided, and the breach remeditated in minutes.
Thanks to Darktrace analyst Stefan Rowe for his insights on the above threat find.
For eight more case studies of cyber-threats detected within SaaS environments, read the White Paper.
IoCs:
IoCCommentcovingtonok[.]buzzUsed to host fake login page
Darktrace model detections:
SaaS / Unusual External Source for SaaS Credential Use
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
“I wonder if running that tool influenced the connectivity in some way.”
“I might need to target down or reboot, but first, I'll investigate the network using ARP. I wonder if ARP is responding properly. If I see "ip neigh stale," that could indicate a problem. I think arping might help in this situation, but I’ll need to keep an eye on how the network is behaving. It's all about figuring out where the issue lies!”
図10：エージェントの異常なLDAPアクティビティに対してトリガーされたこのLinuxサーバーに対するモデル検知は、Cyber AI Analystによって自律的に調査され、その調査結果にはエージェントの異常なSMBセッションアクティビティに関する内容が含まれ、Cyber AIAnalystインシデントに記録されました。
DarktraceのCyber AI Analystは、エージェントのアクティビティによってトリガーされたモデル検知を自律的に調査し、その調査結果を1つの重要なインシデントレポートにまとめました。インシデント報告書の内容は、エージェントによるネットワークスキャンから始まりコーディング演習内容のSMBベースの上書きで終わる、エージェントの行動の各段階を詳しくカバーしています。
図20：Cyber AI Analystは、モデル検知に対する自律的な調査を通じて得られた所見をつなぎ合わせ、エージェントの進行状況のタイムラインを作成し、アクティビティを視覚的に表したグラフとともに提供しました。
Cyber Verification Program下のAnthropic Opus 4.6モデルなど、他のフロンティアモデルを使用したエージェントも、Darktraceのシミュレートされた環境で同じ設定とタスクを使ってテストされました。反復の度に、エージェントは演習の内容または採点プログラムを変更して不正を行おうとしました。基盤となるモデルに関係なく、エージェントは自分が置かれた環境を侵害することで、100％のスコアを達成する方法を見つけることに成功していました。これらのエージェントによるハッキングは、アクティビティの内容こそ違っていましたが、すべてその環境のベースラインの振る舞いから大きく逸脱した動作を引き起こしていました。これらの逸脱はDarktraceプラットフォームに十分な検知の機会を提供し、エージェントの逃げ場はありませんでした。
エージェンティックハーネスは会話履歴をローカルに保存しますが、ダークトレースの研究者たちは、保存されたAIの応答が本当にモデルによって生成されたものであるかどうかの検証が行われていないことを発見しました。また、この設計上の選択がAnthropic Claude Code、AWS Kiro-CLI、OpenAI Codex、およびオープンソースのPiにわたって共通していることを確認しました。
会話履歴は多くの場合クライアント側に保存されます。例えば、Anthropic Claude Code、OpenAI Codex、AWS Kiro-CLI、Piなどでもそうです。そのため、ユーザーはセッションを自由に再開でき、一部の製品にはその履歴を操作する機能が組み込まれています。たとえば、会話の中の特定の時点に戻り、送信したメッセージを編集して、会話を別の軌道へ進めていくことができます。きわめて重要な点として、私たちが調査したすべてのケースにおいて、保存されているAIの応答が対応するモデルから生成されたものであり改ざんされていないことの検証は行われていませんでした。
