R720

Over the summer I got a R1500 G3 ups for my server it is quite over kill however i hope to connect my POE switch and starlink internet to also keep wifi up during a power outage. To that end it would be rather nice if my server actually shut down as the power was running out rather than just continuing until it dies when the battery runs out. The last piece of this puzzle was a usb a to usb b cable which I recently got. This connection goes from my ups to my r720 on which i decided to install nut. The server is running proxmox as its primary OS so I was able to install nut with apt however the first few guides I followed ended up not working for what ever reason. The one that finally worked was this one. Once I had that sorted I setup a web nut docker container in terraform which all went smoothly I hope to eventually add the stats to graphana for data visulisation but for now i am happy.

I had for a while been noticing large spikes in the disk backlog according to netdata as wells as swap useage errors. The swap useage errors especially were wierd because the VM can ballon up to 200GB of RAM and i know i am not using that much. These errors combined were leading to system freezes where i had to go into proxmox and stop the VM and then start it. So to start investigating i ran ...

So in my process of preparing to switch to starlink I setup headscale to replace my wireguard vpn. I thought that I would have to use tailscale as I presumed the headscale server would require a port. It does but it just so happens to use https which I can tunnel with cloudflare. So with that in mind I setup headscale in a docker container and connected my phone with no issue. The server side was slightly harder as a i used the tailscale docker container and accidentally removed the envs so i spent a while trying to figure out why the state file wasn’t saving properly. That is fixed now and it works fine. However on further investigation I realised that I could not use it for my main purpose as cloudflare tunnels do not support web socket PUSH.

I recently decided that I wanted to explore the entire hashicorp stack. I have been using vault and terraform since I moved to gitea for container management. However I decided that learning the entire stack would be interesting and teach me more about IaC and cloud services. I chose to do a install of all the software in a VM running arch that has 200GB of ballooning ram and 20 CPU cores at 3 GHz. It will run the containers in docker on the same server. I managed to install nomad and consul quickly however vault took a minute as the config file is just in /etc not /etc/vault.d. I then also had to add tls_disable = true to the config files which I am currently storing in gitea and deploying with gitea actions. I initialized vault with 5 keys and 3 minimum. I decided not to use my existing vault as I want to start clean and fresh. I installed vagrant and boundary-bin from the AUR and had to curl the binary for waypoint which is currently had coded to version 11.4 but i hope to make dynamic at some point. I then setup waypoint with waypoint install -accept-tos -platform nomad -nomad-host-volume=waypoint-data -nomad-runner-host-volume=waypoint-data-client. However I needed to set export NOMAD_ADDR=http://192.168.1.13:4646 first.

I have been using nginx reverse proxy for quite a while and it works great. However with my recent switch to automation and git managment i have decieded to switch to caddy i know many people love traefik however i have tried it in the past and had some errors so decided to investigate caddy. My current setup works as such in each projects git repo I store the caddy file their is then an action which will copy the caddy file to the correct folder before running caddys hot reload command caddy reload --config /etc/caddy/Caddyfile --adapter caddyfile.

Since I started storing all my docker compose files in git i have wanted to create a public mirror as a disaster recovery option. To that end i needed a way to remove sensitive envs from the files. To achive this I set the docker env to a shell env like so TEMP: ${TEMP} This shell env was set by a script which contained export TEMP=$(vault kv get -field=temp kv/data/docker). This vault currently runs as a single node in docker and has to be manually unsealed as needed.

It started when i tired to set up my emails on Thunderbird mail on arch I noticed it wouldn’t connect. I checked the logs and found “2025-01-16 21:04:11,919 WARN exited: amavis (exit status 1; not expected)” I fixed this relatively quickly by removing “amavisd.pid”. After this it booted but I still couldn’t connect. I tracked this down by using dig and whats my ip to see that the dns record was wrong. I fixed this by switching to a cname record set to @ so i only need 1 ddns setup. This finally fixed it.

I decided that I wanted to use Proxmox as my main operating system as it has a large community and a large available library of tutorials. So i followed this guide (https://www.youtube.com/watch?v=sZcOlW-DwrU&t=487s) which made installing relitivly easy. The only small issue i found post install was that the 4 drives i had setup for proxmox were set up as 2 storage sections called local and local-zfs. I still don’t fully understand what happened here but i just put important files on local-zfs and it seem to work fine.

Ever since I setup mailcow I have been annoyed that it runs with docker volumes and updates with a script. This makes it imposible to run how I want. I feel the main benefit of docker should be all the files needed are backed up and I can use the docker compose file to spin the container up anywhere. I have tried may times to fix these problems in mailcow and have always failed. So I finally gave up with mailcow and started looking at other options. I quite quickly found docker mailserver as my main option. I don’t know if I didn’t see it the first time round or if I was scared of by the lack of a GUI. What ever the case now I am happy to use the command line for everything. ...

It started when I went back to my pterodactyl panel after a long time. I saw a red heart and a invalid ssl cert warning in console. I renewed the cert and the heart went green. However my servers still wern’t loading. So I changed the forwarding from cloudflare to nginx proxy. This still didn’t work. Then I renewed the cert as rsa type and noticed it put it a folder that ended in 00001. So I deleted the one with out the numbers and then renamed the one left to remove the numbers. One restart later and it was all working.