Tue 11 August 4:40PM Hetzner has informed us that the RAID alarm is currently sounding, and that KEATS needs to be switched off in order to diagnose and fix the problem.
5:28PM Keats is being shutdown now
6:02PM Response from Hetzner:
This mail serves to confirm that the maintenance on your server tex001_truservcomm_jhb1_009, was completed successfully.
SDA was swapped, and RAID is currently rebuilding.
6:15PM The RAID rebuild is at 24%
6:27PM RAID rebuild is at 30%
6:41PM 41%
6:53PM RAID rebuild is at 50%
7:19PM 65%
7:51PM 79%
8:07PM RAID rebuild is at 90%
10:36PM Hetzner says that the server is “fixed”. Unfortunately, it won’t boot. I am therefore going to reinstall the server, and then restore all hosting accounts from backup. Please accept my sincere apologies for this. I will work through the night and tomorrow to get all sites up as soon as possible.
2AM Wed 12 August OS has been rebuilt, cPanel and Cloudlinux have been installed. Restoration of hosting accounts is starting now.
4:30AM All accounts have been restored from backup.
Texo: Network Status 