Official off-network notifications for Texo clients
20:00 We’ve had to reboot Chiss to fix a problem. The server is currently doing a filesystem check, and should be back online within 30 minutes.
20:28 The filecheck is at 82% so it won’t be much longer before the server is back online.
20:38 Chiss is back up
Starting at 5PM today, clients will not be able to access websites, email or servers hosted in the Cape Town datacentre during a 10-minute emergency maintenance.
17:20 All sites on Chiss are displaying a “500 server error” due to a PHP error. We’re working on it.
17:36 Apache/PHP recompiling now
18:32 Everything should be working now – if you see anything wrong your website, please let us know.
15:38 Emergency maintenance in Yeats is underway to fix a PHP error
15:59 CloudLinux support has logged into the server
16:10 CloudLinux support is stumped, and the problem has been submitted to cPanel as a priority support request
16:18 cPanel tech support have logged into the server
17:01 cPanel support: “For testing purposes, and to get you online as quickly as possible, because of the fastcgi and suexec involvement being demonstrated, I’d like to test changing the PHP handler and SuExec toggle for the current PHP configuration, followed by a restart of Apache. “
22:33 cPanel needs to do more tests. There will be intermittent, brief periods of downtime whenever they restart apache.
cPanel support: “I appreciate your patience again – I’ve replicated your environment as closely as possible, and unfortunately I’m still not able to trigger the same issues to occur. With that in mind, we may need to continue working with your server for the time being, for any troubleshooting purposes in this regard. The primary issue there is that, in order to replicate the problem within your server, we need to change the handler back to FastCGI, effectively bringing your sites back to a state of 500 error. Ideally, this could be done for very brief periods of time (change it, check logs, check processes, change it back), but sporadic downtime would be likely.”
00:20 cPanel support is still working on the problem: “Would it be possible for me to switch back to fcgi to see exactly what’s going on as I think I may have an idea of what is breaking the sites when switching from DSO to fcgi?”
01:06 cPanel support: “What I thought could have been the issue didn’t work, so I’m continuing to look into this for you. It seems like some sort of potential permissions issue when suexec is enabled, which I’m currently in the process of trying to track down. I switched back to DSO to get the sites back up and will update you again after I’ve been able to look more into the issue.”
02:48 cPanel support are blaming CloudLinux, who originally blamed cPanel, for the problem (“I’m sorry but this issue is not related with work of CloudLinux. I would recommend contacting Cpanel support for this task.”) so it’s up to me to fix any problems your sites might still be experiencing. Please send an email to [email protected] if you see anything amiss with your site.
03:44 cPanel has agreed to take another look after I voiced my displeasure. The problem has been escalated to a level 3 technician.
Please note that all sites have been up and running since 17:00, but the performance has not been what it should be.
04:24 cPanel have found and fixed the problem. I’m off to sleep. Please accept my sincere apologies for the rough ride all the sites on Yeats have had over the past week.
11:10AM The server Yeats is currently experiencing very high loads, which is resulting in websites timing out. We are investigating the cause.
11:41 The server’s load is back to normal.
Texo: Network Status 