Changing Wireguard configurations through Ansible is always a double edged sword
You either manage to do what a whole department whole do over a week in just 5 minutes, or you take your whole infrastructure down
Yeah I did take my whole infrastructure down
Yeah, I need a testing cluster
One day, I believe...
#ansible #wireguard #configuration #infrastructure #cluster #k3s
Today marks the first day my infrastructure shows all green since I started the original migration to k3s earlier this year
It has been a journey, and some other things are not green, but as far as things are going, data is safe, and uptime is reliable 👌
#k3s #homelab #selfhosted #longhorn #kener #opensource #kubernetes #uptime #migration
Allez hop, nouvel article sur le #blog concernant la récupération de mes bases de données #mariadb suite à la mort de mon cluster #k3s https://blog.seboss666.info/2025/05/recuperer-les-bases-de-donnees-de-feu-mon-cluster-k3s-sans-backup/
I'm starting to regret going for a full Rasp 5 + USB -> Sata SSD for my k3s control plane...
They cannot handle big load, and go into full lock if too much resource is requested (around 80% of cpu & mem)
On the other hand, all of my amd64 nodes have strictly no issue in handling high load, and literally dances with what would crash the Pis...
You live and you learn, I guess
I am trying to configure Owncast to share some Homelabbing during the day
The problem is, I cannot get OBS to connect to it, via its RTMP instance
Has anyone tried deploying Owncast on a Kubernetes cluster, and was successful ?
I checked online and very little resource...
Any help very much welcome
Migrating all of my CloudNativePG cluster's s3 backup endpoints was... Weirdly easy
We'll see during the night if everything goes well and only green is to be reported tomorrow morning
#Homelab #selfhosted #s3 #psql #cloudnativepg #kubernetes #k3s
Planning to document my current #HomeLab setup on my website — a lot has changed over the years.
It all started with a 3-node #k3s cluster running on #RaspberryPi 4s.
One of them is still in use today as the master node!
Hmmm, it seems like my websites appearing as down in Kener (or is it kener.ing ?) is because my coredns instance cannot solve the DNS request ?
But I'm thinking, Longhorn has also shown, for a long time, dropped requests when rebuilding big volumes...
Could it be, my wireguard under the hood dropping packages ? Hmmmmmm...
#homelab #selfhosted #longhorn #wireguard #kener #dns #coredns #k3s #kubernetes #k8s
Oh, one last thing!
If you want some of this #selfHosted fun for yourselves, please feel free to check out my #homelab setup:
So my #homelab #k3s cluster suffered an outage I've been trying to recover from since yesterday.
The outage itself started 4 days ago, and I discovered it when I couldn't get back into the house:
#HomeAssistant was unresponsive and I couldn't open my garage door.
To make for "one of those mornings" I then also couldn't unlock my front door **with my keys**. No electronic locks involved, no keys behind the door...
The lock itself would unlock but the door latch wouldn't open.
Yeah. Fun.
Some time ago, I had random node crashes because the CPU was pinned to 100% with iowait, I disabled promtail then never got those crashes again
Recently, I decided to re-enable it, and I just got a similar 100% pin on my ingress node, meaning loads of downtime
Now I saw promtail has been deprecated in favor of Alloy ? Has anyone had positive experience when doing the switch ?
#kubernetes #homelab #selfhosted #promtail #grafana #k3s #alloy
Has anyone tried running #universalblue / #ublue / #bluefin headless? I want to use it to run #k3s and #ollama on an AMD ryzen mini PC as a lab machine to play around with K8s and ollama at home to try things out that I don't often get to do at work. I know I can use the base ublue image but that looks like a PITA to install compared to just running bluefin headless
For science, fun and profit, I set up a #WireGuard mesh network for my #FreeBSD, #OpenBSD, #RockyLinux and #Kubernetes #homelab:
https://foo.zone/gemfeed/2025-05-11-f3s-kubernetes-with-freebsd-part-5.html
There's also a mesh generator, which I wrote in #Ruby.
Aaaand turns out the only PVC I had forgotten to fix with the aforementioned longhorn config was Mastodon... So it went down and getting it back up was PAINFUL
I am getting SO tired of Longhorn, nothing ever works, volumes do not attach, and when they do, the container doesn't detect it, and it required manual intervention
If only Ceph had S3 sync compatibility, it would be a no brainer, but at this point, I might as well try it out...
6k Leute im #k3s slack Channel und keiner Antwortet mir. *seufz*
vermutlich ists einfach und ich bin einfach zu dumm...
Wow, that was a long outage this time (yeah, again)
One node handling a lot of traffic crashed, bringing down a bunch of others in cascade
But impossible to bring the whole cluster back up
The reason ? I had changed the flannel-backend some time ago, and reverted the config, but that never ACTUALLY reverted to default, so it had to be manually be defined to default
Wow, what a roller coaster
And again, Mastodon coming down for no apparent reason, aside from the fact I was trying to export some rss feed a bit hastily 🤔
I really have to fix this promtail deployment to do some delayed debugging
Well, bugger. I seem to have accidentally entirely deleted #VictoriaMetrics from my #K3s cluster while converting it from hand-installed to Helm-managed.
It's a good job that there was nothing particularly important in it.
The problem I have now is that Longhorn is now stuck while provisioning a PV while bringing VM back up again.
Homelab update, since I added those new nodes to my k3s cluster, I cannot connect through Netbird from my iOS device, and I have no idea why...
I'm really close to giving a serious try to Tailscale with a private Headscale instance, just to see how it differs in a similar environment
#homelab #k3s #cluster #netbird #tailscale #headscale #networking #wireguard
