You Can't Do Everything Over the Internet

Or why your backup strategy might suck. Don’t Wanna Pay Ransom Gangs? Test Your Backups.

Even if you don't pay to keep the ransomware gang from disclosing your data, and your customers' data, you may still have to pay for a decryption key if you were backing everything up over the intertubes, and you expect to restore from those backups in the same way.

Even with the data transfer rates we enjoy today, if you have enough data, it can be that sneaker-net will outperform the internet. Seriously out perform.

“In a lot of cases, companies do have backups, but they never actually tried to restore their network from backups before, so they have no idea how long it’s going to take,” said Fabian Wosar, chief technology officer at Emsisoft. “Suddenly the victim notices they have a couple of petabytes of data to restore over the Internet, and they realize that even with their fast connections it’s going to take three months to download all these backup files. A lot of IT teams never actually make even a back-of-the-napkin calculation of how long it would take them to restore from a data rate perspective.”

Then, if your backups are encrypted (What do you mean your backups are not encrypted?), did the decryption key just get encrypted by the bad guys? That is poor planning.

In rare cases, they can corrupt your backups.

But the assumption that "we can do everything over the internet" is what kills most people.

Back in ancient times, when I was still working in Information Technology, part of my job was disaster recovery planning, and part of that was off-site backup and recovery. About once a year, most organizations would test their backup/restore procedures at an off-site vendor-location. For a week, or more, before the recovery test, companies would scream "Do you know there is a backup test?" And even then, stuff would be missed.

I only ever worked for one organization where corporate would spring the tests on IT without warning. That was the one organization where I was fairly certain that my paycheck would be cut, even if there was a fire in the data center. Everywhere else? The best disaster-recovery plan was to keep your résumé up to date on your PC at home.

As XKCD pointed out, Bandwidth is a Limiting factor: FedEx Bandwidth.

If you want to transfer a few hundred gigabytes of data, it’s generally faster to FedEx a hard drive than to send the files over the internet. This isn’t a new idea—it’s often dubbed SneakerNet—and it’s how Google transfers large amounts of data internally.

Transmission rates have gone up since that was written, but so have the volumes of data involved.

So, what would you do, if you went out to lunch and came back to discover that a tanker-truck full of toxic chemicals had spilled on the highway in front of your building, and you couldn't get in? It wasn't a truck full of toxic chemicals; it was a bomb-scare, and we did get back in that afternoon, but it was an interesting thought experiment.