Have a workable plan, or else…

by Martin Fisher

As we continue to discuss the Basic Truths of Incident Response Leadership, we’ve briefly gone over the three Basic Truths as well as done a deeper analysis of  “Succeeding By Planning to Fail”. This brings us to:

Basic Truth #2: Have A Workable Plan, or Else

As an Incident Response Leader, one of the most valuable parts of your role is to create, test, exercise, and (when called upon) execute Incident Response Plans (IRPs).  IRPs run the gamut from a Post-It note on the wall listing contact phone numbers, to plans that take up several 3-ring binders on a shelf somewhere.  Plans can be long or short, detailed or vague, paper or electronic, automated or manual…you get the picture.  What makes a good plan different from a not-so-good plan can be summed up in a few ways.

First, can you execute the plan using only the resources that you legitimately would have access to during the incident?  We’ve all seen plans that call for using network analyzers that aren’t accessible to the organization or that call for numbers of personnel that just don’t exist.  You may have written plans that assume that the responding team has skills and experience that your current team just doesn’t have (I have).  The key is to map out the current skills and capabilities of your team and employ them as best you can to meet the anticipated incident.

As you identify resources available to you, it pays to be creative.  Can other teams identify folks who could temporarily be available during an incident (think of it as an in-house “volunteer fire department”)?  Do you have relationships with designated outside incident response consultants? Do you have relationships with local, state, or federal law enforcement?  In today’s business environment, Incident Response Leaders need to be creative in identifying resources that can assist during a response cycle.

Second, you have to test the plan.  This sounds so intuitive, but many plans never get past the written-down stage before they are needed in an incident, because no leader stepped in to ensure that the plan would work as designed.  One of the most effective testing plans for an IRP is also the least expensive – the simple “Talk Through”, where all of the designated players sit at a conference table (pizza is optional, but highly recommended) and talk through the plan, noting any foreseen problems or issues.  The team needs to be encouraged to not only point out potential problems, but brainstorm solutions they can implement as-is since (as we talked about in Basic Truth #1) you can only plan on the resources you have, not the resources you want to have.

Plan testing needs to be redone each and every time the plan is modified, or at some regular interval (at least annually).  Testing can be announced or (my personal favorite) unannounced.  The time spent testing can help the Incident Response Leader assess not only the plan, but the team assigned to execute it.  The feedback loop should encompass applications, hardware, processes and procedures, as well as people.  Everything is fair game.

Lastly, you need to continually exercise your plan.  This, while not as intuitive as testing, is something that many organizations fail to do, claiming “it’s too hard” or “it’s too disruptive” or “it’s already been tested, why should I do an exercise?”  Having performed incident response on plans that have been exercised and plans that have not, I can tell you with complete assurance that plans that have been exercised are executed more smoothly, with fewer problems and a better resolution.

Exercises can range from a talk-through (similar to testing but without the constant feedback loop) to a full-on exercise using live equipment.  Talk-through exercises can help in quickly familiarizing a team with a new (or newly updated) plan.  Talk-through work will also quickly point out assumptions that, while seemingly accurate in testing, don’t fit the way the incident response team works.  All other things being equal, I believe that talk-through exercises offer the highest return for time spent in any aspect of prepping for a incident.

Full-on exercises, as powerful and complete as they are, can be very hard to accomplish.  Most organizations cannot fully replicate their production systems (even using virtual machines).  These exercises, when they can be done at all, are usually done in development or test environments and generate most of their value by allowing teams to actually assess and interpret adversary actions and data.  These exercises are an Incident Response Leader’s best chance to simulate the stress and activity of a real incident.

Taking all of this into account, it’s clear that the Incident Response Leader must be able to create, test, and exercise an IRP to be able to effectively respond during the inevitable incident.  By creating plans designed around available resources, qualifying the plans with testing, and regularly exercising the plan, you can ensure that you and your organization will be ready when the inevitable incident occurs.

But it’s not over yet. Once you’ve gotten this far you still have one vital task to accomplish.  We’ll cover that in the last article on the Basic Truths of Incident Response Leadership.