Saturday, November 4, 2023

OVPN Testing Underway…

One of our project members had requested a while back that we
get them connected to our network stack via OpenVPN, however,
at the time that wasn’t something we support because the firmware
we were running didn’t make this easy.

Poking around tonight and revisiting this request, we found out
that the new firmware we used makes OVPN setup pretty easy to deal
with and we are able to offer IPv6 support from our /48 to client
devices.

We don’t know if this is something we plan to offer to members of
our project, but it does seem to actually work and could help with
supporting older platforms that have OVPN but not Wireguard support.

Wednesday, October 18, 2023

Updates to core services…

For team members, you guys should see the new mail system in place
and be able to fully manage your own mail accounts. You should not
have to submit tickets or message Skylar to get account settings
changed or details updated anymore. Along with that, the mail server
was brought up to date last night and has new settings in place and
dropped support for some older clients. The team is also watching
the traffic and logs, a handful of /24s from Digital Ocean have been
blocked at the router level due to the amount of spam attempts. We
will continue to watch things and issue blocks as needed.

We are aware that some certs on the network are expired and will be
fixing this tonight when there is some down time to do so. Certs still
haven’t been automated due to a lack of time to handle such.

We are also aware that Cloudflare is being rather aggressive towards
our network allocations. There is nothing that we can really do about
this and you’ll just have to put up with the annoying captchas until
they figure their shit out. I have dropped into the network and looked
for anything that might be triggering this from them and our day to day
doesn’t look much different than it has for the past several months.
So, I’m not sure what has triggered them, aside from it being a data
center network range.

Tuesday, September 5, 2023

[OPS] Extended Downtime…

We have…had a day.

We are up and somewhat alive again on the new SSD, the server
is responding much quicker now. Skylar is in this process of
restoring the network from images and it could be 21:00EST
before we are fully alive again.

A little information on what happened:
When Skylar and TyDwagon originaly setup the new server for
the project, the main system drive was a 2TB platter drive
and we chose that for the storage due to some of the sub projects
that we have going on. However, this proved to be a very bad
decision rather quickly as VM activity started to just swamp the
platter drive and was causing massive IO delay and the server to
become unresponsive at times. We tried to wing a few tweaks to
mask the issue for a bit, but nothing helped.

Now today, we are restoring MFN from images to a 480GB SSD and
that should help us out and we are already noticing that restores
are zooming by.