Hypothesis in Threat Hunting

Jul 17, 2017 | by Demetrio Milea

Today's threat landscape requires organizations to operate more proactively to keep up with advanced and persistent threats. There is no doubt that the practice of threat hunting has emerged as a key capability to detect stealthy threat actors trying to gain access to the organizational IT infrastructure by evading traditional security measures.

Hunting aims to detect threat actors early in the cyber kill chain by investigating the IT environment for signs of an intrusion. However, unlike an alert-driven investigation, threat hunting is a proactive activity that begins with a hypothesis to verify (hypothesis-driven investigation).

This capability, including the exploration and analysis of network and systems data, leverages many toolsets and techniques, investigating network traffic and endpoints through various telemetry sources such as logs, packet capture, netflow, memory and so on.

This capability, necessarily includes some manual effort and the ability of the hunter to formulate a hypothesis, is mainly based on the understanding of the relationship between two primary dimensions:

The threat intelligence landscape can provide knowledge of current and emerging threats. In this capacity, the hunter examines strategic and operational intelligence from sources outside the organization. Examples include subscription and open source research and reports describing new TTPs, threat intelligence from working groups and peers, law enforcement communications and so on.

The second dimension is the context of the organizational operating environment that provides the threat hunter additional sources of data including policies, procedures and guidelines, previously managed incidents, penetration testing and vulnerability scan results, high value asset (HVA) data, network data flow and so on.

Both of these dimensions, combined together, provide the basis for the initial hypothesis. For example, to create a new hypothesis, the threat hunter can initially determine some key elements:

  • Target: Identification of systems, network segments or privileged accounts
  • Adversary Techniques: High level TTP clues
  • Analytics: Toolsets and methodologies needed to verify the hypothesis

The three elements above, if associated with a specific stage of the attack lifecycle, can establish the basis for the threat hunter to infer an attack scenario, with detection and analytics techniques. Overall, the more concise the hypothesis is, the easier it will be to verify (given the availability of the technology and the telemetry sources).

Consider the hypothetical scenario where the update mechanism of a third-party software vendor has been compromised and distributes a malicious payload (e.g. PowerShell code). To create and verify the hypothesis the threat hunter can proactively search for anomalies as follows:

  1. Identifying the network segments and the systems currently using the third-party software updater
  2. Focusing the analysis on a specific stage of the attack lifecycle such as internal recon and lateral movement
  3. And finally, given the toolsets available, begin hypothesis verification regarding the TTPs used in these phases including, for example, the usage of built-in commands followed by the usage of PowerShell and Windows Management Instrumentation (WMI) for lateral movements

Another aspect to consider is the natural bias introduced by the hunter. The generation of new hypotheses, by both beginner and experienced analysts, can be influenced by these biases eventually resulting in the hunter missing a critical fact, fail to search for specific evidence, or recognize key information. An analytical and open-minded approach can overcome these limitations and challenges. Such biases can be also mitigated by using structured analytic techniques.

Agility should also be maintained throughout the process. New data and threat indicators may result in a refined hypothesis enabling the hunter to evolve with the threat environment. There are cases where some tests can lead to in-depth systems and artifacts analysis. These investigations may discover new signs that were unknown at the beginning and should not be ignored, but rather used to refine and enhance the initial hypothesis.

Given the example above, if during the analysis, the hunter notices that the third-party software updater has not been compromised, but there are other indicators of compromise, such as an exploited IT service of a system in the same network segment; the hypothesis would be refined accordingly.

In addition, an investigation that did not initially produce the expected result - subsequently - might have different outcomes. As a best practice, the initial hypothesis, the investigation technique and the lessons learned should become part of an internal knowledge base to be used to drive future investigations.

Finally, it is worth remembering that there is no precedent to leverage for the threat hunter trying to discover an otherwise unknown intrusion. As a result, a robust threat hunting model and hypothesis with best practices should be developed, maintained and improved over time due to the ever evolving IT environment and threat landscape.

Author: Demetrio Milea

Category: Research and Innovation, RSA Fundamentals

Keywords: Analytics, Hunting, Threat Landscape, Threats