Tuesday, September 5, 2023

[OPS] Extended Downtime…

We have…had a day.

We are up and somewhat alive again on the new SSD, the server
is responding much quicker now. Skylar is in this process of
restoring the network from images and it could be 21:00EST
before we are fully alive again.

A little information on what happened:
When Skylar and TyDwagon originaly setup the new server for
the project, the main system drive was a 2TB platter drive
and we chose that for the storage due to some of the sub projects
that we have going on. However, this proved to be a very bad
decision rather quickly as VM activity started to just swamp the
platter drive and was causing massive IO delay and the server to
become unresponsive at times. We tried to wing a few tweaks to
mask the issue for a bit, but nothing helped.

Now today, we are restoring MFN from images to a 480GB SSD and
that should help us out and we are already noticing that restores
are zooming by.