Incident Management Chronicles: Striking The Right Balance

By Jeff Chan, Vice President of Technology, MOXFIVE

If you haven’t experienced a ransomware attack, it’s likely only a matter of time. Adding insult to injury, you will receive no warning. One minute the team is working hard to end the day, the next, your SaaS apps stop working, network access disappears, and the phones of each member of the security team start ringing.

That’s when all evening plans are canceled, and the coffee is brewed because getting systems back in order will likely be an all-night affair. This response is only natural since every second the systems are down, they are crippling to the business.

It is precisely when teams begin to scramble that mistakes are made. From my own experience, there are two critical missteps that I see time and again. First, they lose sight of three key protocols that are critical to follow when responding to an incident—containment, forensics, and recovery.

Second, they take a siloed containment approach as if containment, forensics, and recovery are all independent entities. For example, when an attack occurs, one group focuses solely on recovery, where the mantra is “recover at all costs.” In parallel, the remaining teams dive into forensics and containment, where their focus is keeping the data intact for the investigation. Operating on their islands, each group conducts a damage assessment, determines the underlying causes, kicks off damage containment, and inevitably cuts off all outside communication.

This approach isn’t wrong. All these response activities are valid and essential. What’s missing is balance across these three primary functions. While it might seem counter-intuitive, combining the three will ultimately accelerate the process and help ensure a smoother resolution. The following aims to show why giving equal focus to each area is so vital, starting with containment.

Containment: For anyone who has never conducted a forensic investigation, the aim is to find Indicators of Compromises (IOCs) which are essentially evidence that malicious activity exists. This could come in the form of unrecognized files in the system or unusual traffic, and they help guide containment measures designed to prevent further damage. One potential action could be for the forensic team to deploy an Endpoint Detection and Response (EDR) solution that can determine what’s been affected. That team then shares its findings with the containment group, which then gets to work. This process helps connect teams that may have previously been disjointed and deliver a more comprehensive response.

Recovery: To recover impacted systems, you need input from the containment team. More specifically, insights into their efforts, such as installing EDR on a restored system before putting it back into production. ​So, as these IOCs get identified by the forensics team, they are then fed into the EDR solution along with any other applications by the containment team. From there, the recovery team can go about restoring systems without being concerned about potential reinfection. They can then use the EDR to see if any of these indicators trigger on that system before putting it back into production. Without any indicators, recovering is a lot riskier. On the other hand, if a business decision is made to collect all IOCs before systems go back online, it will take longer to get the IT infrastructure up and running, which will cause increasing revenue loss. 

Forensics: During recovery, the collection of all forensic data is done by the recovery team, and it must be completed before any system restoration efforts are commenced. This helps the forensics team identify any other IOCs that may be present and then connect with the containment team and help determine what occurred and how it started so teams can take the necessary steps to tighten the perimeter.

The theme through this process is that each of these teams is connected, collaborating in an ongoing process where each area is equally balanced, and the process doesn’t stop until the incident is fully resolved. When one group takes precedence over the others, this process begins to break down, which can have a deleterious effect on the business.


No posts to display