Blog
/
AI
/
March 7, 2024

Defending Against the New Normal in Cybercrime: AI

Default blog imageDefault blog imageDefault blog imageDefault blog imageDefault blog imageDefault blog image
07
Mar 2024
This blog outlines research & data points on the evolving threat landscape, the impact of malicious AI, and why proactive cyber readiness is essential.

AI in Cyber Security

Over the last 18 months, discussions about artificial intelligence (AI) – specifically generative AI – ranged from excitement and optimism about its transformative potential to fear and uncertainty about the new risks it introduces.  

New research1 commissioned by Darktrace shows that 89 percent of IT security teams polled globally believe AI-augmented cyber threats will have a significant impact on their organization within the next two years, yet 60 percent believe they are currently unprepared to defend against these attacks. Their concerns include increased volume and sophistication of malware that targets known vulnerabilities and increased exposure of sensitive or proprietary information from using generative AI tools.  

At Darktrace, we monitor trends across our global customer base to understand how the challenges facing security teams are evolving alongside industry advancements in AI. We’ve observed that AI, automation, and cybercrime-as-a-service have increased the speed, sophistication and efficacy of cyber security attacks.  

How AI Impacts Phishing Attempts

Darktrace has observed immediate impacts on phishing, which remains one of the most common forms of attack. In April 2023, Darktrace shared research that found a 135 percent increase in ‘novel social engineering attacks’ in the first two months of 2023, corresponding with the widespread adoption of ChatGPT2. These phishing attacks showed a strong linguistic deviation – semantically and syntactically – compared to other phishing emails, which suggested to us that generative AI is providing an avenue for threat actors to craft sophisticated and targeted attacks at speed and scale. A year later, we’ve seen this trend continue. Darktrace customers received approximately 2,867,000 phishing emails in December 2023 alone, a 14 percent increase on what was observed months prior in September3. Between September and December 2023, phishing attacks that used novel social engineering techniques grew by 35 percent on average across the Darktrace customer base4.  

These observations reinforce trends that others in the industry have shared. For example, Microsoft and OpenAI recently published research on tactics, techniques, and procedures (TTPs) augmented by large language models (LLMs) that they have observed nation-state threat actors using. That includes using LLMs to draft and generate social engineering attacks, inform reconnaissance, assist with vulnerability research and more.  

The Rise of Cybercrime-as-as-a-Service

The increasing cyber challenge facing defenders cannot be attributed to AI alone. The rise of cybercrime as-a-service is also changing the dynamic. Darktrace’s 2023 End of Year Threat Report found that cybercrime-as-a-service continue to dominate the threat landscape, with malware-as-a-Service (MaaS) and ransomware-as-a-Service (RaaS) tools making up most malicious tools in use by attackers. The as-a-Service ecosystem can provide attackers with everything from pre-made malware to templates for phishing emails, payment processing systems and even helplines to enable bad actors to mount attacks with limited technical knowledge.  

These trends make it clear that attackers now have a more widely accessible toolbox that reduces their barriers.

AI Enabling Accidental Insider Threats

However, the new risks facing businesses aren’t from external threat actors alone. Use of generative AI tools within the enterprise introduces a new category of accidental insider threats. Employees using generative AI tools now have easier access to more organizational data than ever before. Even the most well-intentioned employee could unintentionally leak or access restricted, sensitive data via these tools. In the second half of 2023, we observed that approximately half of Darktrace customers had employees accessing generative AI services. As this continues to increase, organizations need policies in place to guide the use cases for generative AI tools as well as strong data governance and the ability to enforce these policies to minimize risk.  

It is inevitable that AI will increase the risks and threats facing an organization, but this is not an unsolvable challenge from a defensive perspective. While advancements in generative AI may be worsening issues like novel social engineering and creating new types of accidental insider threats, AI itself offers a strong defense.  

The Shift to Proactive Cyber Readiness

According to the World Economic Forum’s Global Cybersecurity Outlook 2024, the number of organizations that “maintain minimum viable cyber resilience is down 30 percent compared to 2023”, and “while large organizations have demonstrated gains in cyber resilience, small and medium-sized companies showed significant decline.” The importance of cyber resilience cannot be understated in the face of today’s increasingly as-a-service, automated, and AI-augmented threat landscape.  

Historically, organizations wait for incidents to happen and rely on known attack data for threat detection and response, making it nearly impossible to identify never-before-seen threats. The traditional security stack has also relied heavily on point solutions focused on protecting different pieces of the digital environment, with individual tools for endpoint, email, network, on-premises data centers, SaaS applications, cloud, OT and beyond. These point solutions fail to correlate disparate incidents to form a complete picture of an orchestrated attack. Even with the addition of tools that can stitch together events from across the enterprise, they are in a reactive state that focuses heavily on threat detection and response.  

Organizations need to evolve from a reactive posture to a stance of proactive cyber readiness. To do so, they need an approach that proactively identifies internal and external vulnerabilities, identifies gaps in security policy and process before an attack occurs, breaks down silos to investigate all threats (known and unknown) during an attack, and uplifts the human analyst beyond menial tasks to incident validation and recovery after an attack.  

AI can help break down silos within the SOC and provide a more proactive approach to scale up and augment defenders. It provides richer context when it is fed information from multiple systems, data sets, and tools within the stack and can build an in-depth, real-time behavioural understanding of a business that humans alone cannot.

Lessons From AI in the SOC

At Darktrace, we’ve been applying AI to the challenge of cyber security for more than ten years, and we know that proactive cyber readiness requires the right mix of people, process, and technology.  

When the right AI is applied responsibly to the right cyber security challenge, the impact on both the human security team and the business is profound.

AI can bring machine speed and scale to some of the most time-intensive, error-prone, and psychologically draining components of cyber security, helping humans focus on the value-added work that only they can provide. Incident response and continuous monitoring are two areas where AI has already been proven to effectively augment defenders. For example, a civil engineering company used Darktrace’s AI to uplift its SOC team from the repetitive, manual tasks of analyzing and responding to email incidents. The analysts estimated they were each spending 10 hours per week on email incident analysis. With AI autonomously analyzing and responding to email incidents, the analysts could gain approximately 20 percent of their time back to focus on proactive cyber security measures

An effective human-AI partnership is key to proactive cyber readiness and can directly benefit the work-life of defenders. It can help to reduce burnout, support data-driven decision-making, and reduce the reliance on hard-to-find, specialized talent that has created a skills shortage in cyber security for many years. Most importantly, AI can free up team members to focus on more meaningful tasks, such as compliance initiatives, user education, and sophisticated threat hunting.  

Advancements in AI are happening at a rapid pace. As we’ve already observed, attackers will be watching these developments and looking for ways to use it to their advantage. Luckily, AI has already proved to be an asset for defenders, and embracing a proactive approach to cyber resilience can help organizations increase their readiness for this next phase. Prioritizing cyber security will be an enabler of innovation and progress as AI development continues.  

--

Join Darktrace on 9 April for a virtual event to explore the latest innovations needed to get ahead of the rapidly evolving threat landscape. Register today to hear more about our latest innovations coming to Darktrace’s offerings.

References

[1] The survey was undertaken by AimPoint Group & Dynata on behalf Darktrace between December 2023 & January 2024. The research polled 1773 security professionals in positions across the security team from junior roles to CISOs, across 14 countries – Australia, Brazil, France, Germany, Italy, Japan, Mexico, Netherlands, Singapore, Spain, Sweden, UAE, UK, and USA.

[2] Based on the average change in email attacks between January and February 2023 detected across Darktrace/Email deployments with control of outliers.

[3] Average calculated across Darktrace customers from 31st August to 21st December.

[4] Average calculated across Darktrace customers from 31st August to 21st December. Novel social engineering attacks use linguistic techniques that are different to techniques used in the past, as measured by a combination of semantics, phrasing, text volume, punctuation, and sentence length.

Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Author
Max Heinemeyer
Global Field CISO

Max is a cyber security expert with over a decade of experience in the field, specializing in a wide range of areas such as Penetration Testing, Red-Teaming, SIEM and SOC consulting and hunting Advanced Persistent Threat (APT) groups. At Darktrace, Max is closely involved with Darktrace’s strategic customers & prospects. He works with the R&D team at Darktrace, shaping research into new AI innovations and their various defensive and offensive applications. Max’s insights are regularly featured in international media outlets such as the BBC, Forbes and WIRED. Max holds an MSc from the University of Duisburg-Essen and a BSc from the Cooperative State University Stuttgart in International Business Information Systems.

Book a 1-1 meeting with one of our experts
Share this article

More in this series

No items found.

Blog

/

OT

/

March 28, 2025

Darktrace Recognized as the Only Visionary in the 2025 Gartner® Magic Quadrant™ for CPS Protection Platforms

Default blog imageDefault blog image

We are thrilled to announce that Darktrace has been named the only Visionary in the inaugural Gartner® Magic Quadrant™ for Cyber-Physical Systems (CPS) Protection Platforms. We feel This recognition highlights Darktrace’s AI-driven approach to securing industrial environments, where conventional security solutions struggle to keep pace with increasing cyber threats.

A milestone for CPS security

It's our opinion that the first-ever Gartner Magic Quadrant for CPS Protection Platforms reflects a growing industry shift toward purpose-built security solutions for critical infrastructure. As organizations integrate IT, OT, and cloud-connected systems, the cyber risk landscape continues to expand. Gartner evaluated 17 vendors based on their Ability to Execute and Completeness of Vision, establishing a benchmark for security leaders looking to enhance cyber resilience in industrial environments.

We believe the Gartner recognition of Darktrace as the only Visionary reaffirms the platform’s ability to proactively defend against cyber risks through AI-driven anomaly detection, autonomous response, and risk-based security strategies. With increasingly sophisticated attacks targeting industrial control systems, organizations need a solution that continuously evolves to defend against both known and unknown threats.

AI-driven security for CPS environments

Securing CPS environments requires an approach that adapts to the dynamic nature of industrial operations. Traditional security tools rely on static signatures and predefined rules, leaving gaps in protection against novel and sophisticated threats. Darktrace / OT takes a different approach, leveraging Self-Learning AI to detect and neutralize threats in real time, even in air-gapped or highly regulated environments.

Darktrace / OT continuously analyzes network behaviors to establish a deep understanding of what is “normal” for each industrial environment. This enables it to autonomously identify deviations that signal potential cyber threats, providing early warning and proactive defense before attacks can disrupt operations. Unlike rule-based security models that require constant manual updates, Darktrace / OT improves with the environment, ensuring long-term resilience against emerging cyber risks.

Bridging the IT-OT security gap

A major challenge for organizations protecting CPS environments is the disconnect between IT and OT security. While IT security has traditionally focused on data

protection and compliance, OT security is driven by operational uptime and safety, leading to siloed security programs that leave critical gaps in visibility and response.

Darktrace / OT eliminates these silos by providing unified visibility across IT, OT, and IoT assets, ensuring that security teams have a complete picture of their attack surface. Its AI-driven approach enables cross-domain threat detection, recognizing risks that move laterally between IT and OT environments. By seamlessly integrating with existing security architectures, Darktrace / OT helps organizations close security gaps without disrupting industrial processes.

Proactive OT risk management and resilience

Beyond detection and response, Darktrace / OT strengthens organizations’ ability to manage cyber risk proactively. By mapping vulnerabilities to real-world attack paths, it prioritizes remediation actions based on actual exploitability and business impact, rather than relying on isolated CVE scores. This risk-based approach enables security teams to focus resources where they matter most, reducing overall exposure to cyber threats.

With autonomous threat response capabilities, Darktrace / OT not only identifies risks but also contains them in real time, preventing attackers from escalating intrusions. Whether mitigating ransomware, insider threats, or sophisticated nation-state attacks, Darktrace / OT ensures that industrial environments remain secure, operational, and resilient, no matter how threats evolve.

AI-powered incident response and SOC automation

Security teams are facing an overwhelming volume of alerts, making it difficult to prioritize threats and respond effectively. Darktrace / OT’s Cyber AI Analyst acts as a force multiplier for security teams by automating threat investigation, alert triage, and response actions. By mimicking the workflow of a human SOC analyst, Cyber AI Analyst provides contextual insights that accelerate incident response and reduce the manual workload on security teams.

With 24/7 autonomous monitoring, Darktrace / OT ensures that threats are continuously detected and investigated in real time. Whether facing ransomware, insider threats, or sophisticated nation-state attacks, organizations can rely on AI-driven security to contain threats before they disrupt operations.

Trusted by customers: Darktrace / OT recognized in Gartner Peer Insights

Source: Gartner Peer Insights (Oct 28th)

Beyond our recognition in the Gartner Magic Quadrant, we feel Darktrace / OT is one of the highest-rated CPS security solutions on Gartner Peer Insights, reflecting strong customer trust and validation. With a 4.9/5 overall rating and the highest "Willingness to Recommend" score among CPS vendors, organizations across critical infrastructure and industrial sectors recognize the impact of our AI-driven security approach. Source: Gartner Peer Insights (Oct 28th)

This strong customer endorsement underscores why leading enterprises trust Darktrace / OT to secure their CPS environments today and in the future.

Redefining the future of CPS security

It's our view that Darktrace’s recognition as the only Visionary in the Gartner Magic Quadrant for CPS Protection Platforms validates its leadership in next-generation industrial security. As cyber threats targeting critical infrastructure continue to rise, organizations must adopt AI-driven security solutions that can adapt, respond, and mitigate risks in real time.

We believe this recognition reinforces our commitment to innovation and our mission to secure the world’s most essential systems. This recognition reinforces our commitment to innovation and our mission to secure the world’s most essential systems.

® Download the full Gartner Magic Quadrant for CPS Protection Platforms

® Request a demo to see Darktrace OT in action.

Gartner, Magic Quadrant for CPS Protection Platforms , Katell Thielemann, Wam Voster, Ruggero Contu 12 February 2025

Gartner does not endorse any vendor, product or service depicted in its research publications and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

GARTNER is a registered trademark and service mark of Gartner and Magic Quadrant and Peer Insights are a registered trademark, of Gartner, Inc. and/or its affiliates in the U.S. and internationally and are used herein with permission. All rights reserved. Gartner Peer Insights content consists of the opinions of individual end users based on their own experiences with the vendors listed on the platform, should not be construed as statements of fact, nor do they represent the views of Gartner or its affiliates. Gartner does not endorse any vendor, product or service depicted in this content nor makes any warranties, expressed or implied, with respect to this content, about its accuracy or completeness, including any warranties of merchantability or fitness for a particular purpose.

Continue reading
About the author
Pallavi Singh
Product Marketing Manager, OT Security & Compliance

Blog

/

AI

/

March 28, 2025

Survey Findings: AI Cybersecurity Priorities and Objectives in 2025

Default blog imageDefault blog image

AI is changing the cybersecurity field, both on the offensive and defensive sides. We surveyed over 1,500 cybersecurity professionals from around the world to uncover their attitudes, understanding, and priorities when it comes to AI cybersecurity in 2025. Our full report, unearthing some telling trends, is available now.  

Download the full report to explore these findings in depth

It is clear that security professionals know their field is changing fast, and that AI will continue to influence those changes. Our survey results show that they are aware that the rise of AI will require them to adopt new tools and learn to use them effectively. Still, they aren’t always certain about how to plan for the future, or what to invest in.

The top priorities of security stakeholders for improving their defenses against AI-powered threats include augmenting their existing tool stacks with AI-powered solutions and improving integration among their security tools.

Figure 1: Year-over-year changes to the priorities of securitystakeholders.

Increasing cybersecurity staff

As was also the case last year, security stakeholders are less interested in hiring additional staff than in adding new AI-powered tools onto their existing security stacks, with only with 11% (and only 8% of executives) planning to increase cybersecurity staff in 2025.

This suggests that leaders are looking for new methods to overcome talent resource shortages.

Adding AI-powered security tools to supplement existing solutions

Executives are particularly enthusiastic about adopting AI-driven tools. Within that goal, there is consensus about the qualities cyber professionals are looking for when purchasing new security capabilities or replacing existing products.

  • 87% of survey respondents prefer solutions that are part of a broader platform over individual point products

These results are similar to last year’s, where again, almost nine out of ten agreed that a platform-oriented security solution was more effective at stopping cyber threats than a collection of individual products.

  • 88% of survey respondents agree that the use of AI within the security stack is critical to freeing up time for security teams to become more proactive, compared to reactive

AI itself can contribute to this shift from reactive to proactive security, improving risk prioritization and automating preventative strategies like Attack Surface Management (ASM) and proactive exposure management.

  • 84% of survey respondents prefer defensive AI solutions that do not require the organization’s data to be shared externally

This preference may reflect increasing attention to the data privacy and security risks posed by generative AI (gen AI) adoption. It may also reflect growing awareness of data residency requirements and other restrictions that regulators are imposing.

Improving cybersecurity awareness training for end users

Based on the survey results, practitioners in SecOps are more interested in improving security awareness training.

This goal is not necessarily mutually exclusive from the addition of AI tools. For example, teams can leverage AI to build more effective security awareness training programs, and as gen AI tools are adopted, users will need to be taught about data privacy and associated security risks.

Looking towards the future

One conclusion we can draw from the attitudinal shifts from last year’s survey to this year’s: while hiring more security staff might be a nice-to-have, implementing AI-powered tools so that existing employees can work smarter is increasingly viewed as a must-have.

However, trending goals are not just about managing resources, whether headcount or AI investments, to keep up with workloads. Existing end users must also be trained to follow safe practices while using established and newly adopted tools.

Security professionals, including executives, SecOps, and every role in between, continue to shift their identified challenges and priorities as they gear up for the coming year in the Era of AI.

State of AI report

Download the full report to explore these findings in depth

The full report for Darktrace’s State of AI Cybersecurity is out now. Download the paper to dig deeper into these trends, and see how results differ by industry, region, organization size, and job title.  

Continue reading
About the author
The Darktrace Community
Your data. Our AI.
Elevate your network security with Darktrace AI