@redmp i often think about how the fundamental problems with LLMs are completely unsolved and i have not seen a single serious proposal to solve them (or even acknowledgement from anyone working on them that these problems exist), and this is a perfect example for that.
there is a actual true answer to "what is the set of options that exist in the current version of cargo", but trying to answer that with a statistical model trained on a bunch of random text is just fundamentally foolhardy.
i really don't understand why the industry is just bashing their heads into the wall with bigger and bigger clusters of "GPUs" (really ML ASICs but people call them GPUs for historical reasons i guess) instead of like. trying to apply the parts of transformer models that work so well with language to actual human-validated knowledge graph types of things.