Texo: Network Status

2:40AM Scheduled maintenance was completed on Brienne at 9:10PM on Friday 28 November, but the server failed to reboot. Hetzner’s datacenter technicians are still working on the problem, almost 6 hours later. If they fail to get the server up and running, I will ask them to remove the harddrives so that we can move all the data to another server.

3:45AM the Hetzner technicians have been unable to boot the server, even after swapping out multiple bits of hardware. We will attache the hard drives to another server, and start copying the data to another one of our servers. Unfortunately, this will take many hours, during which time the sites formerly hosted on Brienne will be offline.

4:55AM The technicians have attached the hard drives to another server, and we have started copying the data.

11:15 There’s a problem on Dantari, which we’re working on.

11:39 The problem has been fixed, and Dantari is rebooting. Because the server has been online for so long (327 days) the OS is doing a filesystem check, which cannot be avoided. This should be completed within 30 minutes, and then the server will complete booting up. Sincere apologies for the inconvenience this is causing.

11:49 filesystem check is at 64%

11:55 66%

12:03 71% completed

12:08 83%

12:13 the filesystem check is 90% complete

12:17 filesystem check has completed, and boot will continue

12:20 Dantari is back online, load will be a bit high (slow sites) while the disk I/O settles down after the reboot.

11:09 We’re working to fix an apache/PHP error on Arryn

11:30 the problem has been fixed – if you see any errors on your website, please submit a ticket to support@texo.co.za

14:01 Shared hosting server Cersei is offline. We’re working on the problem.

14:12 Cersei is back online. we’re investigating the cause of the outage to prevent it from happening again.

21:05 Optimumcache on Twain has failed, and taken the server offline. We are working on the problem.

21:28 Twain is back online – apologies for the downtime!

13:30 Shared hosting server Cersei is offline. We’re investigating.

13:41 Cersei is back online. Apologies for any inconvenience.