tech hobby

Home servers and Proxmox

2020-04-06

So, didn't write anything in a while, time to fix that, eh?

There are some interesting things going on that I could write about, like me joining Panzerdog to work on Tacticool , BroRig getting to a point where I am going to start using it for production, and after testing and improvements it may get released for purchase alongside BroDynamics... But that's for another post.

Today I wanna talk a bit about one of my hobbies - home server, virtualization and Proxmox. Partially because I found myself describing it quite often to other people on Reddit and other platforms, figured I might just write it once here and just link it next time :)

Why do I need a home server?

For various reasons, of course. The simplest example is a NAS, network attached storage. You can have your PC small and light with only a few SSD disks, and keep everything on a NAS with lots of HDDs in it, tucked away somewhere where it's noise wont bother you. Also it's central storage that's available from any device in the network.

Then there are a lot of fun self-hosted services you could run. Here's a few that I like:

  • Plex - media server. Think Netflix and Spotify but your own, that works over your gigabit LAN, does not consume traffic, uses your own media files (obtained through legal means, of course).
  • Nextcloud - something like Google Drive but your own. File sync and sharing, and lots of apps like document editors, contacts sync, notes, mind maps and more.
  • HomeAssistant - home automation platform, very powerful, allows you to control and automate practically any smart device there is from any manufacturer and using any protocol as long as you have all the required radios (ZigBee, ZWave, Bluetooth) installed.
  • Bitwarden - keeps your passwords, logins, credit card information and secure notes encrypted and secured. It's available as a service running on their servers, for extra privacy you can run it on your own.
  • Bookstack and WikiJS - self hosted Wikis
  • Transmission - torrent downloader. Since your server runs 24\7 you can just leave it to download your files overnight.
  • Couchpotato, Radarr, Sonarr, Headphones, pyMedusa, Jackett, Ombi - these work together to automate downloading shows and movies and music, just make a request in Ombi and wait for a while (you should own legal copies of whatever you're downloading of course ^^)
  • PiHole - network wide Ad and Malware blocker (DNS Filter)
  • Gitea - lightweight git
  • Piwigo - photo library managment
  • Traggo - time tracking
  • BabyBuddy - baby tracking journal, feedings, weight, notes and more

And there's a lot more, here's a list that has a lot of stuff that you can host on your own server: https://github.com/awesome-selfhosted/awesome-selfhosted

Why would you like to host it on your server and not just use some "Cloud" service? It's free and private. Some things are just not available from other providers. Some, like plex, offer benefits like higher quality (because it's local network) and lower latency.

What is Proxmox?

Proxmox is an operating system based on Debian Linux that is designed to be run as bare-metal hypervisor to run Virtual Machines (VMs) and LXC containers (CTs).

Why do I want to run my HomeServer on Proxmox or virtualize anything?

There are, of course, other options like ESXi or even just Windows with Hyper-V or VirtualBox. Proxmox is just a fully free and open source OS (you can pay for enterprise support, but it's not required) that uses QEMU KVM for VMs which is also FOSS and fast, especially compared to VirtualBox.

As for why you'd like to virtualize everything... TLDR - Because separating services from each other protects you from one service or one mistake bringing down everything, because you can schedule full VM or CT backups and because you can manage host resourse usage like CPU and RAM to make sure that, for example, one single VM won't use 100% CPU or RAM and thus stall other services.

I always compare running your server on general purpose OS like Windows or Ubuntu to a house of cards. I did it. I once ran my server on Ubuntu. Adding more and more services to it, some in docker, some just installed on OS. The more I added the more I realized how much time it would take me to re-create everything in case it went down. And there's no way I would remember all those install and setup guides even if I documented every step I made. And what if hardware malfunctions? Boot drive fails? The more I thought about it the more anxious I felt.

And then one day it happened. Something went wrong during routine update, either I made a misprint in command or there was just a bug, I don't know. apt-get started to remove packages from the system, and I thought they were just some obsolete and unused ones. But it just went on and on, I soon realized that something is wrong, it was basically removing the whole system. I stopped it, but damage was already done.

I could've tried to fix it. But I decided that his was it. My house of cards had fallen, and it's time to move to Proxmox, which I was eyeing for a while.

So what now? Now I don't care even if the server burns. I don't mess much with the host OS, only HDD managment is done there, and occasional update. Everything else runs in separate VMs or CTs, there's one VM for Docker which runs a lot of stuff, another VM for HomeAssistant, a VM with desktop which I can remote into if I need to access my network from remote location, containers running a lot of other services, which I will list below.

Everything is backed up daily, and Proxmox can do snapshots, backups of running VMs and containers. Which are then uploaded to another server and to the cloud. So if my server literally burns down to a crisp all I have to do is build a new one, install Proxmox, add HDDs, download backup files, select them all and hit "Restore". That's it. A few minutes to unpack and restore and it's all up and running again.

And yes it works. There was a moment when I decided to upgrade major Proxmox version. There's a way to do it, but instead I just wiped it clean and installed a fresh new version.

Backups also saved my ass a few times when I broke something, or when, for example, HomeAssistant update had a bug in it and would not start.

And you can also use it to spin up multiple instances of services, for example I used it when adding extra maps to my Ark server cluster. I just made a copy of the first CT, changed map and ports and voila.

Why not just use docker?

Docker itself is a solid option, as I mentioned I do run Docker in a VM on top of Debian. Again it's all about disaster recovery. If you run docker on bare metal OS, like on Ubuntu or Debian, not virtualized, then even if you just mess the system up you will need to reinstall it, install docker, pull all images, and set them up to work with same mountpoints and config parameters. Which you may not even remember all. Some use Ansible for these tasks, and it's a solid option.

To me I still find it the easiest and fastest to just restore the whole thing from a whole system image backup.

So, I hope someone will find this useful. Not my usual kind of post, as I usually focus on animation and rigging, but hey, there's a Hobby section on my website, so why not use it more? :D