Keys to Mature to a Level 4 Threat Hunting Program

Three Commonalities Among Level 4 Threat Hunting Programs

Threat hunting programs that have reached level 4 maturity have three commonalities:

• The have implemented automation wherever possible to scale their effectiveness
• They have developed threat hunting processes to operationalize how they handle data inputs and outputs.
• They have transparency with stakeholders on the key metrics to measure to prove the value of the threat hunting program.

In this article we’ll walk through how teams can tie all these concepts together to realize the value of threat hunting. For a more detailed discussion on these factors from the CISO of FirstBank, CSO of Red Canary, and Detection Engineers and Threat Researchers from Red Canary and Carbon Black, watch the webinar “Becoming a Leader - An Insider Look at a Level 4 Threat Hunting Program.”

How to Automate for Successful Data Analysis

Keith McCammon, CSO at Red Canary states that “The biggest differentiator between a level 1 and level 4 program is one that has figured out how to take automation of data collection, automation of data analysis and as much automation of the human selection and triage process as you can, codify it and build it into a working system.”

Fundamentally, some level of automation is necessary for a threat hunting program to even function. Tony Lambert, Detection Engineer at Red Canary comments “We have thousands upon thousands of endpoints and can’t hunt every threat result we get. We have to accelerate this with a process.”

So what is the approach for threat hunting automation? This can be as simple as looking at a high level data analysis process and thinking about the best places to automate. Which as Lambert comments is “any part that doesn’t require human cognition.”

What Can Be Automated in the Data Analysis Process

• Make observations - this is a great place to automate. The most valuable tools for threat hunters are the knowledge bases of attacker behavior and these can be utilized to automate this step.
• Formulate hypotheses - this is a little more difficult to automate because this stage often requires a human to review the observations and context in order to determine a place to start looking. However, you can automate documentation processes here and a more advanced threat hunting program will have this.
• Get data and test - this is the easiest and most important area to automate. Tools like ATT&CK, Carbon Black and Atomic Red Team are critical to automating data collection and automating testing.
• Take Action - this is mostly humans, but they can’t do anything without automation in the earlier steps. As Brenden Smith, CISO at First Bank states, “Getting data is the core of this. In order to automate you need the data. Context runs all of this. Being able to add in your knowledge and automate the context, not just the queries, is key to making all of this work.”

How to Implement Automation

The practice of implementing automation is simply figuring out which scripts and alerts don’t matter so you can suppress and see what is truly important. Tony Lambert, Detection Engineer at Red Canary says “You need to make sure your analysts are spending their time effectively - looking at things that actually matter.”

Lambert explains how this happens at Red Canary using tools and data from Carbon Black, “We automate some of the tuning and recommendations for detectors. We get reports back on which detectors produce the most events, which don’t, which are the most and least effective in terms of analyst hours. And it makes recommendations on how we can improve them by being able to exclude certain things.”

McCammon adds that, “Having this process to suppress and continuously improve is a hallmark of a mature threat hunting program. Fundamentally, it is very akin to application whitelisting. Just extended into behavior in the context of an organization.”

How to Improve Outcomes in a Threat Hunting Program

You have to have visibility. However, with visibility comes the burden of figuring out how to manage that data. Smith comments that “One of the biggest pain points I’ve seen threat hunting teams struggle with is tool overload.” He suggests that a formal process is a must to handle all this data and that is key to starting a threat hunting program. One of the first things he did at FirstBank was to operationalize hunting using a process similar to the one shown here.

Smith adds that “Once you formalize your threat hunting process, you can plug new tools and new layers visibility into your process. This has been a key for us to keep our program running effectively. Without this, you’ll get lost in the data when you bring in new tools.”

Brian Baskin, Sr. Threat Researcher at Carbon Black works with companies establishing threat hunting programs. He added that “One of the biggest mistakes I’ve seen organizations make is they suffer a major breach, they find the queries that were used, then they put them aside after IR is done. Meanwhile the attacker is just waiting and will come back with the same tactics, just slightly modified.” Baskin emphasizes that you need a threat hunting process to continue to improve. He advises that “You have to keep on those queries that you know will find bad things and keep fine tuning them.”

Threat Hunting Metrics to Prove Operational Impact

You’re investing a significant amount of time in threat hunting, so it is important to think about visibility of key metrics to stakeholders. Unfortunately, is all too common that stakeholders looking to evaluate your program ask questions like “How many times do we get attacked?” that really don’t get at the value of a threat hunting program.

Smith says that at FirstBank, “We are more interested in the threats we are hunting in our environment. And we aim to tell stakeholders the story of threat hunting and what we’re doing when we aren’t finding threats to make the program better.” That means that knowledge gained from a threat hunt and how it is used to improve defenses and durability is important to surface.

Another way to look at threat hunting is how hunters are spending their time reactively versus proactively. Smith says that at FirstBank “I have 5 people dedicated to threat hunting. I want 20-30% of their time being curious and exploring. We are very conscious about building this time into their schedules.”

Proactive threat hunting (or being curious and exploring) will result in another measurable outcome - IT tickets. Baskin comments that “Threat hunting curiosity will create more tickets - and that’s a good thing.” Baskin states that many organizations use this as a metric for a positive output of a threat hunting team.

The key is to first educate stakeholders that the role of a threat hunter both is reactively hunting attacks as well as exploring the environment to prevent future attacks. From there, you can agree on key metrics that will demonstrate that the program is healthy and continually improving the company’s defenses.

To see the live discussion on becoming a level 4 threat hunting program, check out the webinar referenced in this article from Red Canary, FIrstBank and Carbon Black threat hunting leaders.

The post Keys to Mature to a Level 4 Threat Hunting Program appeared first on Carbon Black.