Author: Texo [Steve]

Starting at 5PM today, clients will not be able to access websites, email or servers hosted in the Cape Town datacentre during a  10-minute emergency maintenance.

15:38 Emergency maintenance in Yeats is underway to fix a PHP error

15:59 CloudLinux support has logged into the server

16:10 CloudLinux support is stumped, and the problem has been submitted to cPanel as a priority support request

16:18 cPanel tech support have logged into the server

17:01 cPanel support: “For testing purposes, and to get you online as quickly as possible, because of the fastcgi and suexec involvement being demonstrated, I’d like to test changing the PHP handler and SuExec toggle for the current PHP configuration, followed by a restart of Apache. “

22:33 cPanel needs to do more tests. There will be intermittent, brief periods of downtime whenever they restart apache.

cPanel support: “I appreciate your patience again – I’ve replicated your environment as closely as possible, and unfortunately I’m still not able to trigger the same issues to occur. With that in mind, we may need to continue working with your server for the time being, for any troubleshooting purposes in this regard. The primary issue there is that, in order to replicate the problem within your server, we need to change the handler back to FastCGI, effectively bringing your sites back to a state of 500 error. Ideally, this could be done for very brief periods of time (change it, check logs, check processes, change it back), but sporadic downtime would be likely.”

00:20 cPanel support is still working on the problem: “Would it be possible for me to switch back to fcgi to see exactly what’s going on as I think I may have an idea of what is breaking the sites when switching from DSO to fcgi?”

01:06 cPanel support: “What I thought could have been the issue didn’t work, so I’m continuing to look into this for you.  It seems like some sort of potential permissions issue when suexec is enabled, which I’m currently in the process of trying to track down.  I switched back to DSO to get the sites back up and will update you again after I’ve been able to look more into the issue.”

02:48 cPanel support are blaming  CloudLinux, who originally blamed cPanel, for the problem (“I’m sorry but this issue is not related with work of CloudLinux. I would recommend contacting Cpanel support for this task.”) so it’s up to me to fix any problems your sites might still be experiencing. Please send an email to support@texo.co.za if you see anything amiss with your site.

03:44 cPanel has agreed to take another look after I voiced my displeasure. The problem has been escalated to a level 3 technician.

Please note that all sites have been up and running since 17:00, but the performance has not been what it should be.

04:24 cPanel have found and fixed the problem. I’m off to sleep. Please accept my sincere apologies for the rough ride all the sites on Yeats have had over the past week.