![Default self created cybersecurity insiders image low res](https://www.cybersecurity-insiders.com/wp-content/uploads/Default-self-created-cybersecurity-insiders-image-low-res-696x397.jpg)
![Default self created cybersecurity insiders image low res](https://www.cybersecurity-insiders.com/wp-content/uploads/Default-self-created-cybersecurity-insiders-image-low-res-696x397.jpg)
This post was originally published here by Sqrrl Team.
Danny Akacki currently works as part of a hunt team at a Fortune 100 company in the financial sector. He started threat hunting at Mandiant as part ofĀ David Biancoās team, and has spent the past four years working inĀ threat huntingĀ and incident response.
Key Takeaways:
- Embrace āpurple teaming.ā The best SOCS have have red team and blue team analysts that closely coordinate with each other to share information.
- A good way to establish baselines for network behaviour is to use blogs to establish a timeline of events. This can serve as a useful jumping off point for pivoting through data.
- Hunting is useless without documentation. Thereās no use going down rabbit holes without having data to feed back into your program. You need to be able toĀ retrace your incident investigation steps.
How do you make the most out of the data youāve collected when threat hunting? How do you gather high-fidelity indicators to guide future hunts? Recently, we sat down withĀ Danny AkackiĀ (@DAkacki) to talk about concrete steps that every SOC can take to improve their hunting maturity level.
Question: How did you make the transition from being a tier-one analyst to doing proactive threat hunting? Were there a lot of techniques or skillsets that you were already familiar with when you started?
Danny Akacki (DA): Ā The whole methodology of being āproactive versus reactiveā was not new, but to actually be doing it getting a feel for it was like, āOh man, this is what this means, right?ā I had worked in a couple different SOCs before that and Mandiant was my first startup because that was really when the whole hunting wheels got rolling.
Iād worked in a couple of SOCs which was pretty much just putting out fires all day. When youāre a level one analyst or level two analyst, you are always reacting. You get these alerts and notifications, which is triage. Thatās following up something that cued you into something else. We decided to not wait for that anymore. That was no longer our charter to just wait around for these things to kind of slap you in the face. We really started to say, āOkay, letās look past this. Letās not wait for that. What is our threat landscape?ā
Since I worked at Mandiant and Ā then at GE and where Iām at now, itās been important to know what your clientās risk levels and how bad guys would come at them. Iāve said it for a long time: hunting is finding ways for evil to do evil things. So, in that spirit you build these different use cases of āif they were to go after this resource, or if they were to use this tactic, and hereās the current new hotness in malware or lan exploit kit, how would this look?ā Does it apply to us?. If it does, can we see it? No. You find out through your hunting, like, āHey, we have a perceived gap in A, B and C locations. Okay, cool. Letās attack that now.ā Trying to stay ahead of the curve pretty much boils down to the whole reactive versus proactive approaches.
Q: One of the common questions we get is āhow do I decide what to hunt for? Do you do it by data sets that you have available to you? Do you do it by kill chain analysis? Are you looking for gaps in your other detections systems? Are you trying to figure out where youāre most vulnerable or what your crown jewel assets might be? Was it kind of a mix of all of them?
DA:Ā Ā My answer to all of those is yes. Those are all factors in building out your hunting program in starting any kind of small hunting task. It really depends. Itās really hard to answer this as a blanket statement over multiple business verticals, over multiple companies because everybody values something differently. If youāre kind of at that low end of that hunting maturity model, youāre going to start looking at, āOkay, letās just start from the beginning. What data do we have?ā That comes from the very basic level of letās figure out what weāre already getting, figure out how we can put it in one place and then have somebody to look at it. If you know youāre only getting some proxy logs and firewall logs and maybe some BPM logs, your approach is going to be vastly different from another org that is more mature that has logs coming from their endpoints and DNS. DNS sounds easy but in my experience and a lot of our experiences, that one can be tricky. Not everybody is even doing thatā or doing it correctly.
It really comes from what level youāre at, if youāre just kind of starting out, like I said, you go and look for those things. As you mature and youāre finding those gaps in visibility and youāre getting more of those log sources and, okay, cool. Letās build up some use cases around these. If we were going to have CQ beaconing, X fill, that kind of thing, we know that we have a really good baseline of all of our log sources. Letās build up use cases around each one and how is this evil going to look coming across all of these different data services. Yeah, thereās just all of those things you mentioned at the very outset can be a starting point for depending on where your organization is at.
Q: Ā Ā Would you say that one of the skills of an analyst should be in deciding how youāre going to prioritize what youāre looking for based on that? That decisionās being made at a lot of different levels from the manager level.
DA: Right. It is. Your analysts, especially if youāre having analysts that are themselves mature enough to be doing the hunting, should have input into this. Itās also a really important factor here when youāre building up your program. We give all this great advice but doing this at scale gets harder and harder. Ā Just because your analysts can see something doesnāt mean they own it. Theyāre going to have to get different leaders out of different spots and organizations on board with this saying, āHey, hereās why we need this data. Can you help us get this?ā Then also speaking to those different owners saying, āWhatās important to you? What do you need to protecting? If this certain box got popped, if a bad guy got in, whatās going to do the most damage for you? Going all the way up to your C levels, what are the companyās crown jewels?ā
In the bigger orb that youāre in, those answers are really going to vary. A lot of times it could be a very daunting task. So, where do you start? Letās pinpoint this down and say, āThese are our most important things. Okay, letās start building our program around these things and guarding these things,ā and inevitably youāre going to find gaps in other areas. Youāre going to find other things that are as important, if not more important. Thatās just a natural byproduct of this. I always stress that this is not a sole exercise of any one security program with any five security guys. You need to have buy-in and support from a lot of different aspects within your businesses. I think thatās something that gets lost a lot of times when people will kind of spin their wheels wondering, āHey, why arenāt we getting anywhere? Why isnāt this being fruitful?ā Those channels have to be made and kept open so thatās also a very important aspect of how you can pinpoint where you want to focus your efforts in hunting.
Q: What are the things that you would need to do if youāre a junior level analyst that is trying to work your way up? What can you focus on thatās really going to be the most useful to you?
DA: This answer can be kind of two-fold. I think there are definitely skills that can be acquired and improved upon but the first one I really want to stress is that the most important skill that any hunter can have isnāt necessarily a skill, itās kind of innate, is a curiosity. A curiosity about the data, what do you have? What is it telling you? What looks weird? What should be normal? Your curiosity should start at letās have a baseline. What is our network actually telling us? What data do we have going across? Letās have a curiosity in the normal first, right?
To get a good baseline over everything and then expand that out. āOkay, what looks abnormal in here? Oh, that looks neat.ā Iād say 90% of what we do is just kind of not looking for the nation states and not looking for the specific hacker groups and all this sexy stuff that we always hear about but itās the pursuit of the, āHuh, thatās weird.ā Thatās really what it comes down to.
Say youāre a limited level one junior guy but youāre super curious and youāre always going to want to dig into things and find out how these things work. Youāre going to hit a well at a certain point as a blue teamer, as defense, as a hunter if you donāt embrace and really understand the methods of your attackers. If you donāt understand their techniques how could you ever really be as effective a hunter as you could be? Thatās really important.
There are people that put premiums on all kinds of skills like, āOh, you have to be a programmer,ā and, āOh, you have to do Python and all this.ā Itās not necessarily that but itās cool to be able to look at code and be able to understand what itās telling you. You donāt have to be able to light a program but thatās also helpful. A basic understanding of malware and how it gets downloaded, droppers and all this kind of thing, understanding those methodologies are very important to being an effective hunter.
Q: Ā Ā How can a hunter make the most out of adversary techniques that they discover?
DA: Probably the most important byproduct of discovering these techniques that these adversaries are using is being able to put that somewhere. Documentation in what we do is huge. Thereās no point in going through a hunt, asking a question, forming a use case and digging through all the data, going down rabbit holes, not finding anything and not doing it all over again if you are not feeding this back into your program.
The other really important thing is that thereās been a big push in the past year or so of this āpurple teamā mentality. The more you understand what your red team is doing, then the better off youāre going to be. Being able to take what youāve learned, work with your red team, have them explain certain topics to you that maybe you understand and if they just got done doing a campaign, having a red team, blue team debrief is so incredibly useful and itās not something that we see a whole lot. Thereās usually this kind of āgood guy, bad guyā situation where itās like āHey, weāve got this but are not going to tell you,ā and then, āHey, we found you but weāre not going to tell you that either.ā No. That communication has to happen and it will bear fruit, I promise you. Thatās really how you can make the most out of studying the techniques, knowing your enemy is being able to feed that information, back feed a program and working through the people that you have working for you to try to get in. Yeah, thatās probably the most important thing that youāre going to do.
Q: Ā Whatās something that you often look for ā maybe your go-to piece of information or piece of data or intel that youāre going to be using to do what you were just talking about?
DA: Iām a log monkey. Thatās what Iāve been doing forever because ā¦ thatās how we all started. Even in the very early days in this whole āhuntingā thing, it was really just combing through logs, looking at logs. Again, that goes back to knowing whatās normal so you can recognize the abnormal, and that was before we had fancy things like Sqrrl and Tap and all these kinds of things, being able to just do something as simple as go through logs and just grab all day long and being able to try to make a timeline. Say you know that there was an incident or a red team exercise and being able to pivot through different log sources so that you can form that picture and then create that timeline. Timeline is hugely important so that you can try to get a fuller picture of what happened.
My go-to has always been we have to start at the network level because a lot of times, especially when youāre in a consulting role and youāre doing these very limited engagements, you might not have access to the host or the host visibilityās going to be very low. Maybe youāre not getting or they donāt have an endpoint solution or theyāre AV solution is kind of wonky. I think that happens a lot. Mineās always been firewall, proxy, BPN, DNS, DHCP, that kind of thing.
Q: How do you determine the fidelity of different levels of indicators?
DA: Ā Ā That can be really subjective as to whatās worth it to look at and what means.
Sometimes just having IPs and domains and hashes is nice and those have their place. Ā But at a certain point the higher fidelity stuff is, being able to chain of events together based on behaviors and based on when A happened then B and then C, that kind of thing.
Chaining this these things together, knowing those behaviors is probably the most high fidelity āindicatorā that you might have that something wonky is going on like, āHey, hereās the service account that is logging onto this different machine and it shouldnāt ever be doing this,ā or, āHereās an advent account thatās logging on at these weird times a day,ā and that looks weird to you.
Ā The most high fidelity things you can have are based on behaviors and that goes back to knowing your environment. Itās that kind of a behavioral thing of what an attacker might do or how they might come at you. Itās not just one little thing anymore. Itās not one IP anymore. Itās a bunch of different things happening in a certain sequence but in relation to each other that can really cue you off to, āhey, something really wonky is going on right now.ā
Threat Hunting AcademyāsĀ ongoing Hunter Spotlight series features conversations with top-level threat hunters to discuss a range of topics, from spotting adversary tactics, techniques, and procedures to leading hunt teams. Each interview is loosely based around their āThreat Hunter Profileā which can be found on the Sqrrl blog. The original interview is availableĀ here.
Photo:BetaNews