Connectivity problems

10:55 AM Xneelo appear to have fixed the network issue. Let’s hope it’s a permanent fix! Sincere apologies for the inconvenience this has caused affected clients.

08:20 AM Xneelo has confirmed that there is a problem in their Samrand datacentre, which they are working on:

https://xneelostatus.co.za/notice/8729/

08:04 AM The following servers are currently experiencing intermittent network connectivity problems:

Bopster
Dododox
Toogit

We are investigating, and hope to have the problem fixed shortly.

Rhaegal High Load / Failure

09:46 We are seeing high loads on the server Rhaegal which is causing sites to time out. We are investigating and should have a fix soon.

10:04 We’ve have found the cause of the high load, and everything should be returning to normal with the next minute or so.

10:10 Our fix has not worked, so we are implementing another fix which requires a reboot of the server. All sites will be offline for at least the next 15 minutes while the server is rebooting.

10:45 Email sent to all clients on this server:

Dear client

The server Rhaegal which hosts the website for {DOMAIN} is offline. We have as yet been unable to find the root cause, but we suspect that it is a hardware failure. While the technicians investigate, we will begin restoring the {DOMAIN} website to the server Hoppler (IP address: 156.38.137.210 ) in the event that the server Rhaegal cannot be restored to a working condition.

We will keep you updated while we work through this.

14:35 We are 30% through the restoration process. If your domain is not using our nameservers, please update its DNS to point to 156.38.137.210 now.

15:30 We are still attempting to recover data from the crashed server, although the chances of that happening are not looking good. If we are unable to recover data from the server, then your email accounts and emails (which are never backed up by Texo) will be lost. As soon as your website is live again, please login to your cpanel and recreate any email accounts you need.

16:15 The restoration of websites to the new server is at 60% and should be done within another few hours

18:25 We have around 50 website left to restore

19:50 All websites have been restored to the new server. If you see any problems with your website, please submit a support ticket. If your domain is not using our nameservers, and you have not yet updated its A and WWW records to point to 156.38.137.210 then please do so now.

Viserion: MySQL failures

4PM MySQL is failing on this server, and a fix is being worked on. Occasional “failed to connect to mysql server” or similar messages might appear from time to time when browsing your website.

8:15PM We are starting an emergency migration of all accounts on this server to the server skizzle.texo.co.za

Multiple Servers Offline

12:50 xneelo appears to have fixed their electrical problem, and our servers are now booting up. I expect everything to be back to normal within 10 to 15 minutes.

12:30 Update from Xneelo (Hetzner):

xneelostatus.co.za/notice/4815/

xneelo network unreachable – Samrand data centre

Notice ID:#4815
Start: 03 Dec 2019 12:15 SAST
Maintenance Window: 30min
Issue: Electrical Failure

12:20PM There appears to be a problem in Hetzner’s Samrand datacentre which has taken all servers, including their own website, offline.