If I had to explain my job... #IOER_FDz .. I'd just show this workflow from the last two days. It's a great example of building reproducible, automated documentation. 🚀
1. Start: A project schema, drafted by colleagues in Excel.
2. Problem: Excel isn't reproducible and is hard to use as a single source of truth for a database.
Solution: I migrated the schema to the universal Protobuf format, making it version-controlled and language-neutral. [1]
3. Problem: Protobuf definitions aren't easy for everyone to read.
Solution: Using an existing #CI/#CD workflow from [3] (cheers @mcnesium), I put the schema on a documentation website. Now it's accessible and legible. [4]
4. Problem: Colleagues still needed a familiar template for data collection.
Solution: Two Python scripts using #pandas now auto-generates both `.xlsx` and open-source `.ods` schema [5] and templates [6] directly from the Protobuf single-point-of-truth, and make all of these available for download, too.
5. Problem: The text-based schema wasn't visual enough.
Solution: Added a script [7] to the CI that transforms the Protobuf files into a #MermaidJS class diagram, rendered directly on the docs site. [8]
6. Problem: The diagram was too narrow in the site's layout.
Solution: The script now generates a second, wide-format version of the diagram for a dedicated fullscreen view. [9]
7. Problem: The static diagram was hard to edit or restyle.
Solution: The CI now generates a shareable link to the Mermaid Live Editor [10]. It reads our latest diagram file, compresses it (using pako/zlib), and bakes it into the URL. Thanks to a code snippet from a friendly stranger on GitHub [11], anyone can now open the *latest* version of the schema in the editor with a single click.
From a static Excel file to version-controlled, multi-format, visual, and interactive documentation, all fully automated. That's the job.
#Automation #GitLab #Python #Protobuf #OpenData #Documentation
@ioer
[1]: https://gitlab.vgiscience.de/casereports/rwlschema
[2]: https://docs.casereports.fdz.ioer.info/
[3]: https://lbsn.vgiscience.org
[4]: https://docs.casereports.fdz.ioer.info/protobuf/
[5]: https://gitlab.vgiscience.de/casereports/rwlschema/-/blob/main/resources/generate_schema.py?ref_type=heads
[6]: https://gitlab.vgiscience.de/casereports/rwlschema/-/blob/main/resources/generate_template.py?ref_type=heads
[7]: https://gitlab.vgiscience.de/casereports/rwlschema/-/blob/main/resources/generate_proto_uml.sh?ref_type=heads
[8]: https://docs.casereports.fdz.ioer.info/konzept/
[9]: https://docs.casereports.fdz.ioer.info/uml/fullscreen.html
[10]: https://mermaidlive.com/play
[11]: https://github.com/mermaid-js/mermaid-live-editor/discussions/1291#discussioncomment-6837936
