Threat Hunting Guide

Threat hunting is the process of proactively searching for dangerous or threatening activity within an IT environment. It moves beyond traditional detection technologies, as it proactively and iteratively scouts networks and other systems to detect and isolate advanced cyber threats that often evade standard security solutions.

Threat Hunting can be of many types: It can start from the weakness points identified as part of the organization’s risk assessment and unique vulnerabilities to its IT architecture. Structured Hunting, on the other hand, starts from a set of Indicators of Attack and specific tactics, techniques and procedures (TTPs) of specific Advanced Persistent Threat (APT) groups, before they can deal damage to the organization. Finally, hypothesis driven threat hunting starts with the hypothesis that the IT environment has already been compromised by an adversary. The objective is to determine how a threat actor might behave and deduce the attack TTPs that might be used. When the threat hunter has identified these TTPs, he can create detections that are automated to improve the detection capability of the organization. These can run in the background to detect real-life threats 24-7.

In this article we will explore why hypothesis driven threat hunting is crucial to the modern datacenter and cover a number of proven threat hunting strategies that you can use. We will also describe the threat hunting steps necessary for effective detection and cover some common pitfalls that you should avoid. For your convenience, please find an executive summary of this article below.

Executive Summary

  • Threat hunting, in the context of purple teaming, helps organizations identify any gaps that they may have in their threat detection capabilities. It also seeks to discover undetected threats and compromised systems.
  • Hypothesis threat hunting is an advanced technique that is based on a set of security capabilities and assets, described in terms of people, tools and technology. It takes advantage of centralized logs, behavioral analysis, and security analytics.
  • Threat hunting can be confused with, but must be distinguished from, Incident Response (IR). IR is the process of mitigating and remediating after an IT security incident, whereas Threat Hunting anticipates the incident from materializing by putting the effort in detecting any suspicious activity within the environment and identifying areas of weakness that could be exploited.

Threat Hunting Today

Threat hunting is a technique designed to augment automated cyber defense solutions, such as EDRs, NDRs, and other detection and prevention tools. It does so with the use of industry professionals who can counter threats that can slip past traditional defenses.

Contrary to what one might think, threat hunters are not uninterested in cyber defense automation tools. In fact, one of their roles is to identify how to improve and augment the automation tools, preventing hackers from exploiting gaps and weaknesses. Threat hunters do not replace automated threat detection processes. Instead, they identify and respond to threats that might already be active within the infrastructure.

Threat hunters must be vigilant, up to date, and technically proficient in order to stay ahead of hackers, who are always developing new tools and techniques. Threat actors  may spend months or years searching for vulnerabilities in an organization’s systems that they have identified as financially or politically lucrative.

According to a survey published in VMware’s Carbon Black Global Threat Report (2020), 88% of CISOs and IT Managers believe that threat hunting is rapidly becoming essential and are including it as part of their cybersecurity strategy. Another 36% stated that they had found “significant evidence of malicious activity” thanks to the threat hunting programs they had implemented.

Moreover, Statista estimates the average down-time after a major cyberattack (such as a ransomware attack) to be around 3 weeks. This causes immense disruption to business continuity and operations. These facts clearly show that threat hunting can be a valuable addition to an organization’s cyberdefense strategy.

Threat Hunting Strategies

The SANS Hunting Maturity Model (HMM)

In order to implement a threat hunting program, it’s important to first perform an assessment via a threat hunting Maturity Model, such as the one provided by SANS. A model like this helps an organization assess its current quality and amount of data collected that can impact threat hunting capabilities and identify areas for improvement.

Threat Hunting Maturity Model (SANS Institute – sans.org)

Here are the typical requirements to achieve each of the maturity levels, providing organizations with an idea of their current standing.

Maturity Level Requirements
Initial –          Putting in place automated security alerts (SIEM tools, IDS, etc.)
–          Developing a dedicated team of incident response professionals
Minimal –          Collecting and centralizing logs from different domains within the environment (network, database, applications, etc.)

–          Using threat intelligence at the forefront of detection

–          Leveraging Indicators of Compromise (IoCs) from published reports and authorities.

Procedural –          Following published procedures in hunting for novel security incidents

–          Implementing a regular hunting schedule (weekly, monthly, etc.)

–          Having specialized hunters within the SOC team

Innovative –          Using a wide variety of data analysis frameworks and applying them in identifying suspicious activity

–          Developing or publishing original hunting procedures based on their experience within your environment

–          Using a dedicated threat hunting platform to streamline the process and strengthen cooperation within the team

Leading –          Using Data Science, Machine Learning and Artificial Intelligence techniques within threat hunting procedures

–          Leveraging automated tools to get the output of threat hunting activities to improve alerts and detection efforts

–          Implementing a methodology to scale procedures so that they are applicable to a large scope within the organization.

Hypothesis Threat Hunting Steps

Hypothesis threat hunting breaks down into the following four steps:

Data Collection

A centralized platform to compile alerts and logs is critical to collect and process the required information. Threat hunting demands detailed data extracted from verbose logs, allowing for more concise and targeted analysis. The quality of data collection will allow the organization to formulate hypotheses and search historically for occurrences of impact.

Hypothesis

This is a fundamental assumption that needs to be either confirmed or denied. It is a possible threat, weakness, or attack vector that we can examine and test for. For example, the assumption that a threat actor is going to use a specific technique, or exploit a particular vulnerability to damage resources or exfiltrate sensitive data. The hypothesis can be formulated after an analysis of the data collected in the previous step.

Threat Intelligence and Investigation

This is one of the steps where the threat hunter shows their expertise: by analyzing the behavior of a suspicious file, or by checking for Tactics, Techniques, and Procedures (TTPs) using logs and other data. Determining whether a threat or malicious activity is in progress or evaded defenses.

Response

As threats are found by threat hunters and TTPs or IOCs are identified, they determine how to automate the detection. This automation can take the form of the development of a specific YARA rule within the existing Endpoint Detection and Response (EDR) system, or a new playbook within a SIEM.

These steps include not only escalation responses, but also the actions required to develop long term threat detection capability. Hard earned detection content can be used to fuel existing security controls and be utilized by traditional security analysts to watch for similar TTPs and behaviors in the future. In this way, threat hunting teams are not wasting time and resources by continuously conducting the same actions.

Threat Hunting: What to Avoid?

Many factors can result in a threat hunting program that is difficult to implement. One key factor is a lack of visibility often caused by lacking log centralization and threat detection tools. Good threat hunting requires full visibility and an advanced set of tools and techniques that must be built on a solid foundation of security measures, controls and tools. The absence of automated detection and response capabilities can also seriously hinder the efficiency and effectiveness of a threat hunting program. Finally, a lack of data collection capabilities can also reduce the efficiency of a Threat Hunting program, where a hypothesis is stated but no data can be collected to confirm or deny it.

Unfortunately, some organizations merely use threat hunting as a means to tick a box with regards to their security standards requirements. As a result they often hand over threat hunting responsibility to an existing, non-specialist team who usually have  other workloads. Instead of setting a long-term strategy and regularly assessing the effectiveness of the program, the whole process becomes simply a box-ticking exercise.

Although this can occasionally lead to results, it is far from ideal for organizations attempt to improve their maturity and is unlikely to be as effective as a dedicated threat hunting team running as part of a wider and well managed security program.

Conclusion: Threat Hunting in a Security Ecosystem

As we have seen, threat hunting is most successful when placed within a security ecosystem where people, processes, and technology are already in place.

Threat hunting is critical in improving the security posture and safety of any organization. It is therefore vital that it is given the resources it really needs. This includes a dedicated threat hunting team, modern and appropriate tools, such as SIEM, EDR/NDR and automation tools (amongst others), as well as the funding and support from the key stakeholders. Threat hunting, similar to many other security disciplines, is an important ongoing exercise that requires regular input, support and appropriate reassessment when required.

If you enjoyed this article, and wish to discover more, please feel free to look at some of our other security articles below:

Chapter 1 –  MITRE ATT&CK Matrix

Learn how information is organized in the MITRE ATT&CK Matrix and how this can be used to plan your security procedures.

Chapter 2 – Incident Response Life Cycle

Understand how an Incident Response Life Cycle helps your organization and how they are planned.

Chapter 3 – Vulnerability Management Lifecycle

Strengthen your organization’s cybersecurity posture by identifying, assessing, prioritizing, and addressing vulnerabilities with the vulnerability management lifecycle

Chapter 4 – Indicators of Compromise

Understand common indicators of compromise such as unusual outbound traffic, privileged account activity, high database reads, and more.

Chapter 5 – Threat Hunting Tools

Understand the key features of Threat Hunting Tools including Security Analytics, security monitoring, threat intelligence, malware analysis, vulnerability scanning, and more

Chapter 6 – Understanding Security Analytics

Dive into security analytics and understand what type of data will help you detect insider threats, analyze unauthorized data access, and research new security threats

Chapter 7 – SOAR vs SIEM: A Detailed Comparison

Understand the differences between SOAR and SIEM through use cases and some tools that can help proactively detect, remediate, and effectively respond to threats

Chapter 8 – SOC Automation

Learn how to use SOC automation to reduce false positives, triage threat analysis, analyze the threat landscape, isolate threatened endpoints, and more

Chapter 9 – The Guide to Threat Detection and Response

Understand the critical features to look for in threat detection and response tools including malware detection and analysis, threat intelligence, and more