Yesterday and last night's site issues

Started by NT2C, July 09, 2026, 07:30:26 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

NT2C

Yesterday and last night we had some server issues that pretty much crippled us.  I'll let the president of our hosting company explain:

QuoteBernie, things should be back to normal now. We had a double whammy today:

An update last night caused a filesystem or network lock that I could not track down. This caused many PHP scripts to timeout forever, which raise resource usage at the account level, causing site loading problems for many accounts. This was resolved with a system reboot just now, and I will be adding additional monitoring once I've been able to investigate a bit further.
A cPanel update today (related to a security notice, so it was force-pushed out by cPanel) caused the cPanel license file to be deleted. This in turn caused cPanel to automatically disable many services and features, and ultimately resolved itself when the system did an automatic license refresh later in the day.
Somehow, these two things are completely unrelated to one another. ALL of our systems that received the cPanel update saw the license file issue. No other system saw the file lock issue. I think the lock issue was related to a kernel patch application that was auto-applied last night. I suspect something at the filesystem or operating system level did not get properly unlocked during that update process.

Let me know if things feel better now (with the site, not with your leg!). I hope you can make it out again before the wind takes down all the standing buildings we have left!

Regards, Matthew Eli

GeekStorage.com "Web Hosting For Geeks, By Geeks"

So, things look to be back to normal today, and I'll be sure to stop by and have a meal with Matt the next time I pass through Oklahoma.
Nonsolis Radios Sediouis Fulmina Mitto. - USN Gunner's Mate motto

Current Weather in My AO
Current Tracking Info for My Jeep

MacWa77ace

I was getting an intermittent 503 Service Unavailable error yesterday. I was thinking it was one of those directed attacks you were talking about.

But it wasn't even annoying because I'd just try again and I could get on the forum. Sometimes it took a few tries.

Lifetime gamer watch at MacWa77ace YouTube Channel

Ask me about my 50 caliber Fully Semi-Automatic 30-Mag clip death gun that's as heavy as 10 boxes that you might be moving.


majorhavoc

No worries. It was clocking for a while on one of my new posts and didn't want me to like someone's topic, but I wasn't too concerned.
A post-apocalyptic tale of love, loss and redemption. And zombies!
<br />https://ufozs.com/smf/index.php?topic=105.0

DarkAxel

The forums loaded for me yesterday, but the pages were slow to fully load and the button to click to get to the newest post on a thread didn't work. Kinda figured it was server errors. Good to see things back up and running fully. 

12_Gauge_Chimp

I kept getting 503 errors and refreshing the page let me get to the main forum page, but I still couldn't log in.

I had a feeling it was some server issue and that Matt would be on top of fixing it.

SMF spam blocked by CleanTalk