I can now verify that the cause of this was a power failure. The datacenter was undergoing a routine UPS inspection when one of the UPS systems dropped load, causing power outages within the datacenter.
Here's the current status as far as I can tell:
- Email is down (the devel server is not fully up yet)
- Search is down (db02 is not fully up yet)
So notifications and search, as of this time, will not work. We're still working on this, however, getting the site accessible was the number one priority, first and foremost