#PyInfra

2025-06-16

#CommunityHosting #Ops

A small group of us are working on community level hosting of 'self-hosted' FOSS tools (think #NextCloud and more) setup as a local service offering for local grassroots organisations. We're seeking advice/tips/guidance.

We're keen to do some orchastration but want to avoid the complexity of say Kubernetes.

As a start we were looking at Ansible with Docker Swarm but we're now exploring other alternatives.

Anyone have experience at this sort of hobbist just a bit bigger than #HomeLab Ops scale?

#Pyinfra is being considered as an Ansible, in the projects words "Think ansible but Python instead of YAML, and a lot faster." (pyinfra.com)

Anyone have experience at this homelab/small hosting level? Would love any tips/suggestions for tools/approaches.

One source of inspiration is the 12Factor app methodology: 12factor.net

Personally, as a rubyist I'm always keen to know what the ruby community is doing in this space also.

Haven't seen many others doing work at this scale, lets use the tag #CommunityHosting to keep connected :)

@digital_justice_society

cc: @jadehopepunk @ryan @gilbert @bounding_star @steph @moxvallix @organvoid @teq

#CommunityHosting #Python #DevOps #ruby #RubyOnRails #SelfHosting #Orchastration #FOSS #OpenTofu #SelfHosting #SelfHosted

2025-06-02

The idea of #pyinfra is great. The execution however …

Let's just say, I would've done a few things in a different way.

2025-05-04

Is it crazy to put pyinfra code into a #Nix derivation to handle configuration of things like PostgreSQL (creating users etc.)? #Nix itself kinda sucks at managing state of services and #pyinfra looks quite nice for that.

2025-05-01

In the #Hetzner Cloud API, you can refer to resources by their ID (an integer) or their name. Both are unique. The name is user-defined and designed to be human-readable, but needs to be identifier-like.

For example, you'd name a server "db01" or a volume "db-main" or an IP address "db01-v6".

However, in #pyinfra, "name" is a reserved word and I can't use it, but "id" is something else, too.

For now I've used "handle" instead of "name", but I wonder whether you have better suggestions.

Screenshot of the following Python code:

from hcloud.firewalls import (
    FirewallResource,
    FirewallResourceLabelSelector,
    FirewallRule,
)
from pyinfra import host
from pyinfra_hcloud import operations as hco


WHOLE_INTERNET = ["0.0.0.0/0", "::/0"]


hco.firewall(
    name="default firewall",
    handle="default",
    rules=[
        FirewallRule(
            description="allow all ICMP",
            direction="in",
            protocol="icmp",
            source_ips=WHOLE_INTERNET,
        ),
        FirewallRule(
            description="allow SSH from anywhere",
            direction="in",
            protocol="tcp",
            port="22",
            source_ips=WHOLE_INTERNET,
        ),
    ],Screenshot of the following Python code:

for handle, pubkey in host.data.get("ssh_keys", {}).items():
    hco.ssh_key(
        name=f"SSH key: {handle}",
        handle=handle,
        public_key=pubkey,
    )


for i in range(1, host.data.map_servers + 1):
    inum = f"{i:02}"

    hco.volume(
        name=f"volume: mapdata{inum}",
        handle=f"mapdata{inum}",
        location=host.data.location,
        size=host.data.mapdata_volume_size_gb,
        format="ext4",
        labels={"contents": "mapdata", "filesystem": "ext4"},
    )

    for ipv in ["v4", "v6"]:
        hco.primary_ip(
            name=f"primary IP{ipv}: map{inum}",
            handle=f"map{inum}_{ipv}",
            type=f"ip{ipv}",
            datacenter=host.data.datacenter,
            dns_ptr=f"map{inum}.{host.data.domain}",
        )

    hco.server(
        name=f"server: map{inum}",
        handle=f"map{inum}.{host.data.domain}",
        server_type=host.data.map_server_type,
        image=host.data.map_server_image,
        datacenter=host.data.datacenter,
        ipv4=f"map{inum}_v4",
        ipv6=f"map{inum}_v6",
        ssh_keys=host.data.ssh_keys,
        volumes=[f"mapdata{inum}"],
        labels={"application": "map"},
    )
2025-04-11

Wait, neither #pyinfra nor #BundleWrap run actual Python code on the remote hosts, right? They're apparently both basically just fancy wrappers for running shell commands, mainly via SSH.

In contrast, #Ansible _does_ run Python on the remote side. Which means that Python is required to be installed, but also means that you can do more sophisticated stuff than parsing CLI tool output. 🤔

2025-04-06

Have been playing around with PyInfra for config of physical servers up till the point of having container orchestration going. So far it feels like a great SaltStack alternative but obvs will take me several days to weeks to rewrite everything. Perhaps I'll iterate some more with just one module to find the right abstraction level before committing to a full rewrite. Some of the primitives are a bit more primitive, for example the apt.repo operation has no concept of signing keys and need to write the logic myself.

#homelab #PyInfra #SaltStack

2025-03-11

#pyinfra is good, almost great! Give it a try.

Nothing against #ansible but its nice to write #python over markup/text.

```bash
uvx pyinfra -y -vvv \
@ssh/gitlab \
--data ssh_key=~/.ssh/dilawar-ap-south-1.pem \
--data ssh_user=ec2-user \
operations/update_gitlab.py
```

Eine Frage an die PyInfra-Admins hier im fediverse. Ich hab mich in den letzten Tagen mit diesem Tool beschäftigt und so einige deploy-scripte geschrieben. Nun komme ich an Punkte, wo ich nicht sicher bin, ob das, was ich gerade mache, so im Sinne des Erfinders ist.

Ich möchte ein paar Server (6 - 10) per pyinfra einrichten. Dabei werden einige Server komplette Webserver mit PHP, PHP-FPM, diversen WebApps (Nextcloud, CMS, WIKI, etc.) und einige bekommen nur einen VPN-Server. Das ist so ganz grob mein Ziel.

Ich bin so weit, dass PyInra mir den kompletten Server aufsetzt. Von der statischen IP, ein paar Anpassungen an der .bashrc, den SSH-Port ändert, Apache, LetsEncrypt, fail2ban, php, php-fpm (beides für verschiedene php-versionen) usw. alles installiert. Toll!

Jetzt bin ich garde dabei, ein Deploy-Script zu schreiben, das mir die Umgebung für eine WebApp einrichtet: vhost.conf für die Domain erstellen und konfigurieren, User, Gruppe, HomeDirs, LogDirs usw. erstellen, LetsEncrypt-Zertifikate holen, usw.

Danach sollen dann all die WebApps (also zum Beispiel nextcloud) per deploy-script installiert werden.

Das sieht alles schon ziemlich gut aus und ich kann mir vorstellen, wie das alles zusammen spielen wird. Aber es ist noch ein weiter Weg.

Mein Problem ist gerade, dass mein Inventory-Script allein nur für einen Host in Kürze bei etwa 200 Zeilen an Host-Data Umfang hat. Ich habe die Befürchtung, dass ich bei dem Umfang an Parametern den Überblick verliere und Fehler mache. Wenn man nur mit dem simplen Dictonary von Python arbeitet, kann man leicht einen Fehler machen, wenn man einen Key angibt und sich dabei vertippt. Es gibt ja keine Fehlerkorrektur oder ein Check, dass der Name des Keys richtig geschrieben wurde. Oder man vergisst einen Eintrag im Dictonary - also in der Beschreibung des Hostes - vorzunehmen. Und dann fliegt hinterher alles auseinander. Ich bin nun dabei, eigene Klasse für die Hosts, Apps und so weiter zu erstellen und dafür zu sorgen, dass dort alles korrekt ist. Aber das ist eine riesige Menge an Arbeit, die mal nicht in 2 Tagen erledigt scheint.

Ist das sinnvoll? Ist das der richtige Weg? Wie handhabt ihr das?


@robertmx @linux #pyinfra #linux #administration #it

Wer hat Ahnung von PyInfra? Ich arbeite mich da gerade rein und bin eigentlich ziemlich angetan. Abgesehen davon, dass ich ständig bei meinen Suchanfragen im Netz auf Tipps zu alten Versionen stoße, die dann meist nicht funktioniert 🙁

Ich würde zum Beispiel aktuell gern wissen, wie ich die Existenz einer Datei auf dem Server prüfen kann, wenn die Datei nur mit root-Rechten zu sehen ist. Ein "_sudo=True" kann man nicht überall einbauen. Auch das globale "_sudo=True" scheint nicht zu helfen.

Also irgendwas der Art:

myfile = host.get_fact(
File,
"/root/.bashrc",
)


Hat da einer eine Idee?

#pyinfra #Python @linux

nogajun🍉nogajun
2025-02-25

Pyinfraの日本語ドキュメントがないので簡単な使い方を書いていた…つもりだったけど本が書けそうな勢いになってる。どうしようか
pyinfra.com/

David Zaslavskydiazona@techhub.social
2025-02-17

@malte 😂

I have also been looking into pyinfra lately. If nothing else, it's that much less code I have to write in YAML and that's a win.

#DevOps #Ansible #pyinfra

2025-02-17

It's this time of year again where I think about replacing ansible with something else.

This year it's pyinfra.

Funny side note: macos wants to replace ansible with unusable. :D

#ansible #pyinfra

Screenshot of text on macOS saying ansible. Underneath is a correction box of macOS suggesting to replace it with unusable.
2025-02-12

* The selfhosting docs on deltachat are somewhat rudimentary. Theres a github repo and some #pyinfra scripts, but compared to #synapse and #xmpp, the docs are less sophisticated

In conclusion I would recommend #xmpp for a closed organizational messaging due to LDAP integration and "enterprise" features and delta chat for nontechnical communication (if you can live without video / audio).

3/3

David Zaslavskydiazona@techhub.social
2025-01-24

@sumanthvepa For everything you want to do, I'm pretty sure you'll have to do a lot of custom coding regardless of how you do it, but you could check out #pyinfra which is like a more lightweight version of Ansible. Or I should say, it's a lighter-weight tool that tries to do the same thing Ansible does (basically, scripting but optimized for configuring systems). I don't know if it will actually make your job easier, I'm just suggesting it's worth a quick look to see.

David Zaslavskydiazona@techhub.social
2025-01-21

@nebucatnetzer I'm working on it, for my personal infrastructure. It is wayyyy more pleasant (and efficient) to be writing code in an actual programming language instead of in YAML, but the tradeoff is losing access to Ansible's huge library of tasks and roles. So you wind up having to implement more things yourself in pyinfra. Personally, I think it's well worth it so I will continue to plug away with pyinfra, but I can see how someone who's working on a large body of infrastructure and has limited time might consider it not viable.

#DevOps #Ansible #pyinfra

Alejandro Baezzeab@fosstodon.org
2025-01-21

@Zer0Rank @nebucatnetzer as for if I would do it again? Definitely! I wrote so many ansible modules in my day. But a lot qas due to the only tool that made the least trade offs for me. Ansible now over a decade old and for all its capabilities, a lot of the cuts are all still there with it.

#pyinfra feels fresh. You can also get very far depending on how well you know the language. Ansible is more of learn how to code in yaml and jinja REALLY well. 🫠

2025-01-21

Did someone migrate from Ansible to #pyinfra?

How much work was it?
Would you do it again?
Was it an improvement in terms of maintenance?

Alejandro Baezzeab@fosstodon.org
2024-12-31

And that's not to say I can't with #fasthtml!

I been building out a small dashboard to auto manage some stuff with #pyinfra... Yes. I'm aware I'm making a #k8s again. shut up! 🤭

But right now, my proficiency in #golang is too high. Likely due to so many architectural changes at dayjob. 😅

Serge from Babkaserge@babka.social
2024-12-29

Setting up Podman with PyInfra, I keep thinking "I should make this a PyInfra module", but if I do that now, I'll never get to my goal.

#Programming #Devops #Podman #PyInfra #ADHD

Alejandro Baezzeab@fosstodon.org
2024-12-14

@SpeechToTextCloud that is certainly true. Ansible has already a decade+ of development on it. At this point, it covers everything under the sun. 😅

I don't think the point with #pyinfra is to compete with that. Instead maybe to do an alternative route for handling state on machines.

Client Info

Server: https://mastodon.social
Version: 2025.04
Repository: https://github.com/cyevgeniy/lmst