Gaussian constructions on surfaces for anomalous event identification

Anomalous events are often an early sign of a security compromise, and the nature of these anomalies can take multiple forms. For example, actions in AWS environments are associated with a broad range of metadata — in addition to the action itself, properties including the user, user agent, source IP address, ASN, and timestamp. Ideally, then, we should be able to use similar events in the past to identify potentially anomalous events, such as connections from an unusual location, or at an unusual time.

However, there may be only a small number of prior events associated with particular combinations of metadata. For instance, a given user and user agent might only be associated with a handful of AWS actions in the past month. Moreover, identifying typical behavior requires an understanding of cycles and other patterns in the data involved. It is useful to understand, for example, whether an action normally occurs at a certain hour of the day, or a certain day of the week.

These inherent relations can be investigated when they are represented on an appropriate surface. Trivially, geolocational data can be mapped to a sphere, but timing data may also be mapped to other surfaces such as a circle or a torus, depending on how many kinds of cyclicity it is necessary to represent.

A Gaussian (or its equivalent for the surface, e.g. a Kent distribution for a sphere, or a von Mises distribution for a circle) can then be constructed at each past observed event. Combining them produces an effective probability density function for events on the surface. New events can then be assessed for their anomalousness based on this function.

This technique allows for the generation of spatial and temporal heatmaps for events given any number of past observations and can rapidly identify anomalous events even with minimal data. Darktrace uses this technique to highlight activity in the map view of the Threat Visualizer’s SaaS console and is also used throughout AI Analyst.