# My personal infrastructure
-I'm migrating to a new Hetzner auction server with 128Gb RAM, 2x1Tb SSD. [New documentation here](personal_infra/README.md).
+[Further documentation](personal_infra/README.md).
-Many stuff below is slowly being deprecated, and I'm updating the doc above.
-
-* Hetzner auction server: 48Gb RAM, 2x2Tb HDD. Runs Proxmox, tinc/ocserv, Apache as reverse proxy
- * LXC container running Dokku, hosting a few personal apps
+* Hetzner auction server: 128Gb RAM, 2x1Tb SSD. Runs Proxmox, tinc/ocserv, Apache as reverse proxy
* LXC container running Nagios
* LXC container running Grafana
* LXC container running Ipsilon
* LXC container running PostgreSQL
* LXC container running a workstation
* LXC container running Gitolite
+ * LXC container running a FreeIPA replica
+ * LXC container running Miniflux
+ * LXC container running Nextcloud
+ * LXC container running FreeSWITCH
+ * LXC container running Bitwarden
+ * Two VMs running Talos, providing two Kubernetes clusters (production/test)
+ * My blog
+ * A CRUD system I ran two track my weight
* Flat 1
* HP Proliant Microserver: 4Gb RAM, 2x4Tb HDD
* DHCP/DNS
Many systems and services are integrated in FreeIPA.
My laptop is joined to the domain so I can even log in to some web applications without typing a password.
-Ipsilon adds SAML for some applications which do not support Kerberos.
+Ipsilon adds OpenID for web application authentication.
Ipsilon is backed by Red Hat, although they seem to have shifted their focus to KeyCloak. KeyCloak is much more featureful, but I prefer Ipsilon because:
* Integration with FreeIPA is a one-liner
* It's still used by the Fedora Project infrastructure
-FreeIPA and Ipsilon are running in CentOS 7- I will probably reconsider this stack around 2024 when CentOS gets close to EOL.
+FreeIPA and Ipsilon are running on Rocky Linux 9.
## Mail
-All systems are running Postfix configured to send emails through an account on my free G Suite account.
+All systems are running Postfix configured to send emails.
This way I get notifications on failed cronjobs or automated updates.
## TLS
-I set up certificates using certbot-route53 on Ansible playbooks.
-DNS verification allows me to run TLS on non-reachable hosts.
-
-I run the playbooks from my workstation periodically to renew certificates.
+I set up certificates using [mod_md](https://httpd.apache.org/docs/2.4/mod/mod_md.html).
## Monitoring
I get alerts for hosts and services being down.
I use https://github.com/alexpdp7/ragent as the monitor, which also means I get notifications when a host is updated and requires a reboot.
-To monitor certain things, such as FreeIPA, I set up cronjobs which run health checks and drop the output somewhere in `/var/www/html/*`, which then I check using check_http.
-
-I also run Netdata on many hosts, which I can access via a reverse proxy at https://netdata.mydomain/$HOSTNAME with single sign on.
-
## Configuration management
-I use Ansible playbooks to provision VMs and LXC containers on Proxmox.
-The playbooks add the new hosts automatically to FreeIPA, set up SSH, etc. See:
-
-https://github.com/alexpdp7/ansible-create-proxmox-host
-https://github.com/alexpdp7/ansible-create-proxmox-centos7-ipa
-
-I also use Ansible for some orchestration tasks (such as deploying FreeIPA replicas, handling Letsencrypt certificates, etc.).
-
-I use an Ansible playbook using https://github.com/alexpdp7/ansible-puppet/ to run Puppet to configure individual systems.
+[Further documentation](personal_infra/README.md).
### Operating systems
I use:
* Proxmox, as it provides LXC containers (and VMs if needed) and ZFS storage. I like ZFS for its protection about bitrot, and because send/recv and snapshots are great for backups
-* EL7/EL8, due to the long life cycle and stability. Due to the CentOS 8 life cycle changes, I'm switching CentOS 8 hosts to Rocky Linux, while CentOS 7 remains.
+* EL8/EL9, using Rocky Linux. FreeSWITCH is on EL8 (Rocky Linux too), because the OKay repo RPMs for EL9 don't seem to work
* Rocky Linux for my server Raspberry.
* LibreElec for my mediacenter Raspberry. Common distros are not an option, as they don't support hardware video acceleration. LibreElec sets up everything I need with minimal fuss, so while it's the system that doesn't use configuration management, it works fine.
### Software updates
-I use `yum-cron` on EL7, `dnf-automatic` on EL8 and `unattended-upgrades` on Debian/Ubuntu so updates are automatically installed.
+I use `dnf-automatic` on EL8 and EL9, and `unattended-upgrades` on Debian/Ubuntu so updates are automatically installed.
`ragent` monitors when systems need a reboot and warns me through Nagios.
I send/receive datasets from the Proliant to USB drives using ZFS.
-## Dokku
-
-I use Dokku to host a few personal applications, so I can update them with `git push`. I also have Ansible playbooks to set up the applications and handle some of them which have more complex deployments.
-
-## Why not Docker/Kubernetes?
-
-Delivering applications as Docker images is massively popular right now, so it's worth explaining why I'm running VMs and LXC containers and I'm not more container-driven.
-
-#### Some things are not containerized
-
-FreeIPA has some support for running in containers, but it doesn't seem to be the most recommended way of running it.
-While it's not a "core" service, I don't think I have an alternative way to get some of its benefits (single users/passwords database across other services and systems, single-sign on, sudo management, etc.).
-It looks like integrating Ipsilon with FreeIPA if Ipsilon is running in a container would not be easy/supported either.
+## Kubernetes
-WordPress has Docker images, but like the EPEL packages I use, they don't seem to be officially supported by Wordpress.
-However, both seem to be well maintained, but EPEL packages are automatically updated using the same process than the rest of my systems (`yum-cron`).
-
-I need to have non-container infrastructure in place, so I have the option of running additional things there or add the overhead of setting up container infrastructure on top.
-Containerization has its advantages, so it's just an equation about how much you benefit from containers compared to the overhead of having container infrastructure.
-
-#### Some containerized things are special
-
-Dokku is its own special system. It could be replaced completely with Kubernetes, but with additional complexity.
-
-#### Containerization infrastructure has its cost
-
-There are lightweight ways to run containers; docker-compose, plain Docker, etc.
-These require significant additional work to automate them to the level of my existing non-containerized infrastructure (automation, backups, etc.), but they consume little additional resources.
-The cost analysis for those is that my existing automation works and the cost of re-implementing them makes them not worthwhile at this point.
-
-Heavyweight solutions like Kubernetes tend to consume more resources, but have better automation features built-in.
-The cost analysis for those is that with the money I'm spending now (single 48gb RAM Hetzner dedicated server) I wouldn't be able to run the same amount of stuff.
-If I was running a significantly greater amount of stuff or I had high-availability requirements, then this would change.
-
-#### Conclusion
-
-If I was starting from scratch, perhaps a light-weight container solution would have been worthwhile, as some services might be easier to provide using a container approach. Also perhaps setting up the automation/etc. would be easier and would give me some advantages.
-
-If I was running more services or had greater availability requirements, a cluster-ready solution like Kubernetes would probably be required.
-
-In my current situation, with the work already performed, I don't think investing more in containers is the most effective use of my limited resources.
+I use Talos Linux to run Kubernetes.
## My blog
-I was never a fan of Wordpress (I prefer other platforms to PHP and MySQL), Remi maintains very up to date EPEL 7 packages. Remi also maintains some packages for EL 8 (derived from Remi's own packages in Fedora, hosted on Remi's personal repositories).
-
-However, after reading about Geminispace, I decided to port my blog to Geminispace and skip migrating from EL 7 to EL 8. Right now I run some custom scripts that generate a static blog and serves them using Agate in my workstation container. I run a Kineto proxy on Dokku that makes the content available through conventional HTML/HTTP. See details at:
-
-https://github.com/alexpdp7/gemini_blog
+See [blog](blog).
## Phones
## Possible improvements
-* Better sync'ing of user files. NextCloud out of the box only works on systems with a graphical interface. There are solutions to mount NextCloud using WebDav, but I prefer to do a sync (so if the server is down I still can access my files) and to run the client headless, but I prefer to stay within supported solutions. Probably syncthing would be a good solution for headless systems to sync dotfiles, etc.
* Add a lab so I can experiment with things in isolated environments.
* Set up SSO on my smartphone, perhaps do some MDM