Open source AI enthusiast.
Maintainer of several generative AI open source projects on Github.
I post about AI and open source.
I'm pleased to announce localwriter, a libreoffice writer extension that allows local-inference language model editing using text-generation-webui as a backend.
Repository:
https://github.com/balisujohn/localwriter
Demo:
https://www.youtube.com/watch?v=rassvcjpTA0
#opensource #libreoffice #LibreOfficeWriter #localllama #generativeAI
If anyone is interested in converting PyTorch models to GGML, I have a new blog post on the subject, complete with example code.
If you want to get GGML and C++ experience and work on a (hopefully) impactful FOSS project, feel free to join the tortoise.cpp discord. I have tasks to work on for all skill levels. If you see this after the link is expired you can also DM for an invite.
@lana Very basic math textbooks seem like they would be incredibly valuable for this purpose. I'm confident you could could fully decipher an alien language if there were some math textbooks in the corpus.
tortoise.cpp (tortoise-tts ggml rewrite) is looking for open source contributors! Feel free to DM if you are interested. All C++ skill levels welcome.
If anyone is looking to work on an #opensource #AI (text to speech) project and have some C++ experience, here is a great first task to work on!
Does anyone know of a Linux app that will let me create forms in non-form PDFs? Because occasionally I get PDFs that definitely should be form-fillable but aren't
I am specifically talking about forms, not freeform editing/annotations.
@brainblasted I don't think Firefox supports this now, but this would be a great feature to add to it.
I keep being pleasantly surprised by how frequently just reporting an open source software bug or packaging issue with specific examples -- often not much more than I needed to do to figure out that it wasn't *me* causing the problem -- results in the thing being fixed.
Honestly, given how effective it is (and how good it feels!) I'm kind of shocked that more people don't report issues. Like how am I the first one to report this when the package has 1,000,000+ downloads?
Turns out you on a #Librem5 you can install android in a container and it runs at native speed. The android window shows as an app, so I can just use an android app if I find a native version lacking, neat!
@vincent does this work w duo
If anyone is looking for low-stress opportunities to contribute to open source, getting minecraft-style automatic chicken farms to work in #minetest #mineclone2 seems to be an unsolved problem that will need changes to mineclone2's mechanics to solve. Definitely a cool opportunity to get your feet wet with Lua programming and improve a free and open source video game.
@marcorobotics @Riedl I think it's a sound idea, elevators have this feature already.
@johnubalis 12 gigs, although it's only using ~8gigs of that for inference.
The backend is based on https://github.com/ggerganov/llama.cpp which can combine both cpu and graphics card inference
Right now, because I have other things running that need the GPU I only have 30/43 layers offloaded to the GPU, the rest are CPU.
I found 13b was the right balance speed/usefulness. But with this setup I can also run quantized 34b model with some gpu offloading at a few tokens per second.
@sarahjamielewis how much vram does it have? If you are managing it with 8 gigs of vram, I am very interested to hear how.
Integrated some fancy local llm magic into my custom IDE project and now I can request arbitrary code reviews.
Sometimes the reviews are not great, but they also don't tend to be terribly off base (and occasionally point towards an actual issue).
And this is without many of the techniques to squeeze even greater performance / context awareness (I am really interested in playing around with some ast-aware / grammar-driven sampling)
@sarahjamielewis is this local inference? And if so, what model are you using?
As an experiment, I asked #ChatGPT to write #Python code to compute, for each 𝑛, the length 𝑀(𝑛) of the longest subsequence of \(1,\dots,n\) on which the Euler totient function ϕ is non-decreasing. For instance, 𝑀(6)=5, because ϕ is non-decreasing on 1,2,3,4,5 (or 1,2,3,4,6) but not 1,2,3,4,5,6. Interestingly, it was able to produce an extremely clever routine to compute the totient function (that I had to stare at for a few minutes to see why it actually worked), but the code to compute \(M(n)\) was slightly off: it only considered subsequences of consecutive integers, rather than arbitrary subsequences. Nevertheless it was close enough that I was able to manually produce the code I wanted using the initial GPT-produced code as a starting point, probably saving me about half an hour of work. (and I now have the first 10,000 values of \(M\)). The results were good enough that I would likely turn to GPT again to provide initial code for similar calculations in the future. https://chat.openai.com/share/a022e1d6-dddc-4817-8bbd-944a3e742d9f
@nomdeb I was pleasantly surprised to see their licensing page: https://meta.farm.bot/v2023/farmbot/intro/licensing
They are telling the truth about being open source.
Another feature of localpilot is prompt-based selection editing, which can be quite powerful with a powerful underlying model. Here is what's already possible on CPU with a 13b parameter model:
