As you probably noticed, NationBuilder along with many sites on the internet, including Reddit, Quora, Foursquare, and others, were down all day long yesterday.  Many sites are still down 26 hours later (we were down for about 12). This has been a trending topic on Twitter, and has been picked up by many news outlets, including the BBC.

The layman's explanation is that Armageddon happened in Amazon's cloud. Applications running in the cloud are designed to handle failure, but what was unique about yesterday is that all of the redundancy they have in place got wiped out simultaneously as well due to an Amazon data technology called EBS.

We use a service called Heroku, which manages the NationBuilder code and all your databases on top of Amazon's cloud.  They have been handling the recovery for us (and tens of thousands of their other customers). We pay them to deal with all the intricacies of running the app, redundancy, recovery, backups, etc. It's great because they are the smartest and most experienced people in the world for that, and in fact there was zero data loss as a result, and Heroku has a strong history in that regard.

If NationBuilder is down, we will post updates to our Twitter feed. You can also see the status from our providers directly. Heroku tends to be a bit faster to acknowledge problems on its status site, but it can still take a few minutes. Amazon frequently takes a bit longer to acknowledge problems on its status site.

We are exceedingly sorry about this, particularly to our customers who have elections in the next few weeks. We'll be extending everyone's plans (and free trials) by a week, it's the least we can do.

Needless to say, keeping our nation infrastructure healthy is the #1 most important thing.