The MFN Project

I forgot about the expiration date on the SSL cert
and the website was down for a little bit today.
This has been resolved and I am working on some
automations to take care of this in the future.

We are finally on 0.18.1 and everything but our theme
seems to work again!

Boy, that was a pain in the ass. The first time I pulled
the new playbook down and ran it, it failed. Figuring it
was just an issue with the tech VM being slow, I ran the
playbook again and it reported all was good.

Lemmy was awful slow to load, the modlog wouldn’t come up,
there were some blank/empty post and metric fuck ton of
postgres errors in the logs. So, I restored the VM’s image
from before the upgrade, shut down the docker containers
manually, ran the playbook again and it reported all okay.
It took Lemmon Bar about 15 minutes to become accessible,
slowly at first and as of about 5 minutes ago it is
responding normally. Modlog is working, no mucked up post
and the incoming traffic seems to be slowing a bit.

I think all is well now. I will work on updating the themes
later on, they aren’t essential to the instance working.
Also, the instance may be slower than normal for the next
10-15 minutes as I make an image of the VM in a good state.

While mostly everything work okay for the upgrade from
0.18.0 to 0.18.1, the modlog no longer will load. This
is a problem that is giving no actual error message as
to why it is doing this.

I am restoring a pre upgrade backup image to see if the
modlog works there and then will try to upgrade again.
Something of note is that the upgrade playbook failed on
the first attempt to upgrade things.

After a failure on the CX router caused by our network
engineers playing with things without a backup in place,
the CX router has been reinstated running a newer firmware.
Along with the fixin’s and what not, the engineers have
also laid the groundwork for us to host games and other
small service VMs down the road. This includes NAT’d
IPv4 and native IPv6 for the networking!

Side note of some importance? Maybe, not sure yet…
We are looking into placing a Lemmy instance online for
members of MFN to make use of if they wish. There are
no concrete plans as to when the instance will come
online because the setup is a bit involved.