Many cyber-threats involve the transfer or modification of large volumes of data, which is usually stored in thousands of differently-named archives. These threats include the unauthorized gathering and preparation of sensitive data to be exfiltrated by malicious actors, and the internal or external transfers of sensitive information using obsolete, unencrypted protocols.
As it is difficult to manually investigate long lists of file names, we have developed an automated system that ranks the most interesting filenames and can quickly assess the importance of compromised files during an attack.
This process uses mathematical methods from signal processing, information retrieval, statistics, and natural-language processing to assign scores and sort a list of file names by their importance. The method also looks for recurring strings in the filenames using Term Frequency–Inverse Document Frequency (TF-IDF) statistics, and searches for rare or suspicious organization names. For instance, when given a list of several thousand numerical names of digital images, and a few large ZIP files, the method identifies the ZIP files as the most important, due to their file extension and anomalous size.
Preliminary results show that no supervised training is needed, and that the system can operate in different natural languages, e.g. English, Spanish, and Chinese with no additional changes or inputs. The system returned ordered lists of file names containing the most interesting specimens among the first 100 elements.
In existence since Darktrace’s inception in 2013, the Darktrace AI Research Centre is foundational to our continued innovation. Rather than a defined product roadmap, the Centre looks at how AI can be applied to real-world challenges, to find solutions that cannot be achieved by humans alone.