You might have read how Danger, the company that manages the Sidekick phone, managed to, uh, lose all its data. Like *all* of it. For all customers. Irrecoverably.
Granted, I’m probably following this story closer than most because I have a Sidekick in my pocket. (Actually, it’s not in my pocket this instant because it’s plugged in lest it run low on batteries and lose all its data forever.) I hosted the first developer dinner, when it was still called the Hiptop. I wrote its first calculator. (That’s right, it originally shipped without a calculator or an alarm clock.) I’ve upgraded through four different versions of the handset. I’ve been a loyal customer. Even when Sidekick retargeted its marketing to blinged-out rappers, I stuck by its side(kick).
So the fact that they completely blew this totally basic operation is simply infuriating. I mean, come on. This isn’t hard stuff. There are so many spectacular ways for problems to arise, but no backups? That’s just… demoralizing.
Anyway, why am I taking this depressing walk down memory lane? So I can say this: I commit to you, loyal Expensify users, that this will never, ever happen to you. Not on my watch. (And given that I’m the CEO, my watch is 24/7.)
Here’s why:
Expensify runs three geo-redundant datacenters, located in three different cities using different ISPs, replicated in realtime using distributed two-phase commit transactions. This means even if two of those cities suddenly fell off the face of the earth, we still wouldn’t lose a single transaction. Furthermore, we do 2 different nightly backups of the database to a storage volume that itself is backed up many times over — in entirely different datacenters. All this is encrypted in more ways you can count (including ways that even I don’t have the authority to decrypt), with procedures in place for how to recover from backups or even rebuild the entire site from scratch at a moment’s notice.
Anyway, I don’t know where I’m going with this. It’s just so absurd for a company like Microsoft to lose *everything* on their customers. I mean, we back up even our *log files* twice nightly. To not back up your customer data is downright offensive. Security and reliability aren’t just good ideas. They’re obvious ideas. Obvious ideas that we take incredibly seriously, even other trusted names don’t.
Expensify Blog 