This article is posted with permission from Sophicity’s CitySmart blog and shares non-technical, municipal-relevant insights about critical technology issues, focusing on how technology reduces costs, helps better serve citizens, and lessens cybersecurity risks. Sophicity is solely responsible for the article’s content.
If this question alarms you or you can’t answer it, you’ve got a problem. Testing is quite possibly the most important part of a data backup and disaster recovery solution—short of actually backing up the data. Even cities with expensive data backup solutions and systems sometimes find they fail when an actual disaster occurs. The tools and technology were fine. The city just didn’t test that it worked.
Testing is a critical data backup and disaster recovery practice for multiple reasons, and it’s something you need to do on a periodic schedule. Here’s why.
1. Make sure your data backup is actually working
Let’s start with the most obvious reason. No matter what kind of data backup solution at your city, only testing ensures that it will actually work when needed. So many times over the years, we’ve heard stories about:
- Tape backups not restoring fully.
- External hard drives failing because of data corruption.
- Data backup servers not backing up all important data.
- Malfunctions in the data backup system that affect backup and restoration.
By doing regular testing, you will ensure that your data backup works properly and will not fail you when an incident or disaster occurs.
2. Correct errors.
Data backup testing is also a great way to correct errors that may affect the restoration of your data. Errors may include:
- Misconfiguration of the data backup system.
- Critical data not selected for backup.
- Data backups not occurring on a regular schedule.
- Power outages or servers accidentally turned off.
By addressing errors during a test, these errors won’t affect you during an actual incident or disaster.
3. Ensure that your critical data is backed up.
If you use a consumer-grade data backup solution, dated tape backup, or manual backup process, a high chance exists that a default setting or a non-technical employee overseeing the backup will lead to your most critical data not getting backed up. Many cheap or consumer-grade data backup solutions often have trouble backing up databases, specialized software, or specific files. Plus, storage caps may sometimes occur without you necessarily knowing.
Having experienced IT engineers overseeing your city’s data backup will ensure that critical data such as databases, applications, and important records are all backed up—especially those critical to city operations or that need to follow a retention schedule by law.
4. Enact your plan effortlessly when disaster strikes.
Police train all the time so that they know what to do in a sudden, chaotic situation. Firefighters train so that they know how to handle a fire without conscious thought. Soldiers train and train so that they instinctively perform in a battle situation.
Similarly, your data backup plan will get enacted in a crisis that strikes out of nowhere. Ransomware hits. A tornado wipes out city hall. A fire rages through your building. You will likely not be warned.
What do you do in such a situation? Many cities panic, flail, and call for help from expensive experts when they did not plan for the worst. But data backup testing allows you to go through a worst-case scenario periodically so that you know the drill by heart.
A plan will help you understand:
- Who does what? What are the roles and responsibilities of city staff and IT vendors?
- How fast will the data be restored? This gives you a sense when specific city operations will come back online during data recovery efforts.
- What order will things happen? You will have a process for restoring data that follows a specific sequence of actions.
- Where will you access the recovered data? What if city hall gets destroyed? You will have a better understanding of where you might relocate. Once you have access to the internet, you will be able to access your data and set up a temporary remote site.
Testing your data backup gives you a huge sigh of relief because you just enact your known plan confidently when a crisis hits.
5. Document your plan so that others can enact it.
Testing your data backup forces you to also document your plan. Documentation helps you clarify your plan and captures it in writing so that others may follow it. In many cases, the idea of a city’s data backup plan resides in one person’s head. If that person leaves the city or is not available when a disaster strikes, city staff can be left not knowing (and not even having access to) what to do. Documentation makes your data backup plan transparent, shareable with other city staff, and easily followed.
Data backup testing goes far beyond just ensuring it works. It helps you think through your data backup and disaster recovery strategy, planning, and tactics. You will learn a lot about the severity of a disaster, how fast you will be able to access your data, and what impact will occur to city operations.
As the saying goes, failing to plan is planning to fail. Not testing is part of planning to fail.