Westerra Credit Union ($1.2B, Denver, CO) traces its roots to various credit unions that have provided financial services in and around the Denver area for more than 75 years. The credit union now serves nearly 80,000 members who live, work, and worship in any of five Colorado counties as well as employees, students, graduates, retires, and families of several employment groups. Westerra’s geographic market puts it in close proximity to the series of extraordinary forest fires that devastated portions of Colorado last summer. Fires started as early as March 2012 and raged through July, burning, in total, tens of thousands of acres, destroying homes, and causing residents to evacuate. Bobby Matthis, Westerra’s vice president of IT, has been working on the credit union’s business continuity and disaster recovery plans.
Alicia, if you really want to start from scratch, I recommend hiring an outside firm that specializes in business continuity and disaster recovery. You’ll offset the cost by saving the time your own staff members would have to devote to the project should you have them tackle it in-house.
Note that a plan has to include more than what might be on your mind from reading recent headlines. A “disaster” can include an event that keeps key people away from their jobs — a flu pandemic, for example.
Also note that a plan is never something you entirely catch up to; it moves, and you have to be able to adjust. It is never just a book that sits on a shelf. Testing is important, and a regulator will want to see some sort of testing at least once a year. Full testing — that is, failing over your systems and bringing them back up again — can be expensive and is not easy to do, so I do not recommend doing it more frequently than annually. More immediate testing can be valuable without causing much disruption. “Table top” scenarios for management teams posit emergency situations and force participants to think about how they would address problems. Table tops can reveal shortcomings in the plan that the credit union should plug. As strong as I think outsiders helped make our plan here at Westerra, what we do internally with table tops makes a strong plan even stronger.
I recommend something like what we call our DRIT, which stands for Disaster Recovery Incident Team. It is composed of designated persons, primary and secondary, from vital areas such as lending, back-office, operations, and so on. These are the people involved in the table tops and who continually upgrade and improve our plan.
Be sensitive to the notion that a completely new disaster recovery plan might require you to alter or rebuild some of your infrastructure.
You need to be flexible, even to think anew. If you build on an existing plan, you are committing to infrastructure that underlies that plan or that was added with the plan in mind. Parts of that infrastructure might not be best for the kinds of disaster you might encounter in your area. If your headquarters is the lowest branch within a floodplain, for example, it is likely not the best building for your core system or backup. So be sensitive to the notion that a completely new disaster recovery plan might require you to alter or rebuild some of your infrastructure.
A data recovery site might have to be a considerable distance from your headquarters. Much depends on what kinds of disasters you face in your area. If I were in New Jersey, I’d consider a data recovery site out of state because, as Hurricane Sandy demonstrated, the whole state can be affected by a massive storm. The data recovery site does not need to be your own brick-and-mortar, especially for smaller or mid-sized credit unions.
Designate someone to be in charge of creating the plan. Generally that is an IT person but it could also be a risk and compliance officer. No matter who is in charge, creating a plan is going to affect every department. Creating a plan from scratch can take up to one year and consume half the time of the person in charge and 10% of the time of other managers.
In order to find a contractor, talk to your peer credit unions; this will be your best source of information. You could also reach out to some of the major trade organizations such as CUNA for advice. Your peers can be helpful in other ways, too. I’ve seen credit unions reach across state lines to find others with available data center space for co-locations, backup, and things like that.
How do you measure how effective a plan is? Conduct table tops tests using scenarios not already covered by your live tests. This can potentially show you how good the plan is going to be in a real disaster. Repeat them until you feel your plan is covering just about every contingency. I’d do table tops twice a year.
Remember, you’ll likely use your plan in more than a complete disaster, and your plan should be able to deal with incidents that fall short of disaster — power outages, for example. Test and refine, test and refine. The plan should be about business continuity as much or more as about overcoming a disaster.
Want to learn more? Click on the articles in the Subscriber Package below for a deeper dive into Executives Insights For Business Continuity .