The Key to Successful Business Recovery
Updated: Dec 10, 2020
MOXFIVE’s “Break Glass” Strategy to Recovering from a Ransomware Attack
Ransomware events can be the worst experience for any person or business. It is an unfortunate reality that we live in a situation where businesses must deal with them. Thankfully, businesses are not in this alone and are able to work through ransomware events with the help of cyber insurance carriers, breach coaches, forensic providers, ransom negotiators, and MOXFIVE Technical Advisors / IT Specialists. While business recovery is just one of the many aspects that MOXFIVE supports as Technical Advisors, it is a critical need that many businesses have in today’s threat landscape.
MOXFIVE’s mission is to minimize the impact of cyber-attacks. The goal for this blog post is to help guide businesses through their recovery efforts, whether alone or with MOXFIVE’s support. Our hope is that businesses can win the fight by themselves, but every business should know that they never have to lose the fight alone.
An Effective Business Recovery Strategy
For MOXFIVE, Business Recovery is a mixture of exceptional Incident Management, which bridges the gap between technical teams and business leaders, with Data Recovery efforts to help IT teams restore operations as quickly as possible. The result of these combined efforts is a faster recovery time with reduced business interruption costs.
What follows is MOXFIVE’s approach to Business Recovery, designed to get businesses back up and running as quickly and safely as possible, while facilitating a successful investigation into the root cause of the incident.
The planning stage of business recovery efforts is one of the most important steps in the process. It is easy for businesses to want to dive headfirst into recovery tasks without first checking the depth of the water. Businesses must take the time to analyze the situation, understanding that they will not have complete information, and develop a response strategy that balances recovery speed with the situation on the ground, looking specifically into the following:
What is the current state of the backups?
Do backups exist for critical systems and data?
Were backups negatively impacted (e.g. encrypted, deleted, etc.) during the incident?
What malware is in the environment? If systems are restored, are they at risk of getting reinfected?
What technical resources do we have available to assist with the recovery efforts (Internal IT, existing support agreements, etc.)?
Have we engaged the appropriate outside expertise (e.g. Privacy counsel (breach coaches), forensics, technical advisors / IT specialists (MOXFIVE), ransom negotiators, etc.)?
Once these questions have been addressed, businesses can then define what success means to them beyond “getting everything working again”. Specifically, MOXFIVE likes to define the following:
Define short-term success: Restore business critical applications to restore operations to the majority of the environment
Define long-term success: Have a stable environment (not necessarily the same exact environment) and strengthen the security and resiliency of the environment to prevent or mitigate future incidents.
At this stage, businesses should begin exploring the viability of backups for both systems and data. The objective is to determine whether backups are available for all critical systems / data. Ideally, businesses can avoid having to pay a ransom if they have viable backups and are able to restore their data. Important considerations for assessing the viability of backups include:
Were backups created prior to the incident?
What format are the backups in?
How quickly can backups be restored?
How often were the backups being taken?
What is the date of the latest viable backup?
Were backups impacted during the incident (deleted, encrypted, etc.)?
What systems / data have available backups? Do the available backups cover all business-critical systems?
What system / data do not have available backups? Are these only for non-essential systems?
As this is happening, the organization should begin putting a prioritized server recovery list together so that restoration can start on the most critical systems first and waterfall down to servers that are more of a “nice to have” than a necessity. The prioritization should focus on the following:
Systems that are necessary for the proper functioning of other systems or critical to the restoration of other services (e.g. Active Directory, backup servers, etc.)
Business Critical Services / Applications
Systems, applications, and data that cannot be rebuilt or replaced (File servers [data], email servers [mailboxes], internal applications, etc.)
Essential Services / Applications
Required systems but could be rebuilt if required (e.g. nice to have historical data but not essential)
Non-Essential Services / Applications
Systems that can easily be rebuilt or are no longer needed (e.g. legacy systems, development systems, etc.)
Not every recovery effort will require a full-blown containment strategy based on the details of the case. At a minimum, every containment plan should include the use of advanced endpoint technology that is proven to deal with the latest ransomware variants and associated malware that occur in some ransomware incidents (e.g. Emotet, Trickbot, Dridex, etc.). This should be deployed to as many systems as possible, understanding that it will be difficult, if not impossible, to get 100% coverage. Having a reliable asset management solution in place is extremely helpful during the contain phase and ensures that all systems that the business is aware of are covered in the roll-out of any containment methodology.
In more severe ransomware incidents that impact most of the environment, it is often prudent to create a quarantined network to restore servers to. This does not mean creating a parallel duplicate environment, rather it is creating a network segment in which the servers can be restored and not risk reinfection. While this is an extra step in the process, this simple mechanism can greatly mitigate the chance of reinfection and wasted cycles, which could increase response costs and business interruption.
With the prior items in place, you are set for the greatest likelihood of success and the technical teams can begin marching in the same direction with a shared vision on what the finish line looks like. This is “go” time, where the rubber meets the road. Teams should execute the plan, communicate issues / roadblocks, and adapt quickly.
This is where Incident Management is critical to success. MOXFIVE finds that many issues begin to emerge, both technical and emotional, when communication breaks down between the technical teams and business leaders — a common occurrence when teams are sleep deprived and restoration priorities (which always change) and status are not properly translated and communicated. When teams are executing, it is imperative that the following is done:
Closely monitor progress
Issues will pop up — identify and remove roadblocks (technical, logistical, organizational, etc.)
Establish a clear communication cadence
Identify the various stakeholders and conduct separate status updates (e.g. executive updates vs deep dive technical discussions) to avoid any rabbit holes
For more complex efforts, establish a conference bridge for the technical teams
Communicate successes and failures early and often
Provide ongoing technical guidance and recommendations
New issues will arise, continued technical support is critical to success
Adjust the approach based on forensic findings
Engage additional experts where needed
The start of a business recovery process can feel like a sprint, it’s important to pace the teams and ensure the health of everyone is taken into consideration. There is no need for individual heroes in these efforts, they only become bottlenecks. There is no shame in asking for and using help where you can! As the recovery process continues, the pressures will subside, and updates / recovery efforts will get into more of a paced rhythm.
It’s also important to continue to track and update what success was initially defined as. For businesses recovering from ransomware incidents, it can be difficult to have your environment be exactly what it was before. There will be a “new normal” that is created, and adjustments will need to be made. You often come out of a ransomware incident with scars. However, establishing a clear and agreed upon ending allows internal teams to go back to their day jobs and manage any additional fallout from the incident as part of the day to day operations of the company rather than staying in “fight” mode for eternity.
As recovery efforts begin to slow, organizations should start planning on how they can reinforce their environment to mitigate future incidents. It can be easy to think it’s best to create a new and super secure environment, your own personal Fort Knox, during recovery efforts. MOXFIVE finds that it can often distract and take away from recovery efforts, especially when additional recommendations can be provided as the teams spend more time in the environment and forensics completes their investigation of the incident. A balance must be struck between containing the incident and making significant improvements to your environment at the same time.
MOXFIVE focuses recommendations on areas that are most realistic for businesses to implement based on their specific environment. It is about implementing the recommendations that are going to provide the greatest impact for their associated costs. MOXFIVE recommendations focus on the following areas:
Reducing the Attack Surface: Minimize the number of future security incidents
Increasing Security Posture: Mitigate the impact of security incidents
Building a Resilient Infrastructure: Mitigate future environment downtime
When organizations can implement these types of recommendations, it can help prevent the number of security incidents and mitigate the impact of any threats that slip through the cracks.
MOXFIVE Business Recovery
While the world continues to watch the rise of ransomware, in both frequency and severity, there is an element hidden behind the headlines that the public seldom sees. Many know that ransomware is meant to encrypt systems, locking users out of their files, but what is often not seen (or appreciated) is the interruption that this type of attack causes businesses and the tremendous efforts that businesses go through to restore IT operations after a ransomware incident. This holds true whether a ransom is paid to obtain a decryption utility or if the victim decides to move forward without paying a ransom. MOXFIVE has witnessed, and performed, monumental efforts alongside ransomware victims to get their businesses back to operational with a restored sense of normalcy, something we can all appreciate in the year 2020.
MOXFIVE is here for businesses who want to prepare their environments for the worst through a robust backup strategy, helping to mitigate the impact of ransomware incidents. MOXFIVE is also here in those “break glass” situations, where some extra muscle is needed to restore operations following a ransomware incident. Our hope is that the MOXFIVE playbook will help guide teams in a successful recovery effort and reduce the threat that ransomware faces to businesses, because no business must face this fight alone.