@JessieHealdUK @BenjaminHCCarr
I want to preface this long message by saying that I am not a developer. By training I am a security analyst, however I do like to develop stuff, and am currently studying to become one.
I do use LLMs, especially when it comes to libraries that I am not familiar with (have been using NotebookLM pretty extensively to learn the basics of the #EclipsePaho lib to some success), but I will never trade real documentation for a LLM response.
And I wil never, ever just paste #AI code into my repos. At least not without being 100% sure that I understand the code fully.
All that to say that I dont think that AI is the problem. It is a tool. It definitely is not what Sam Altman & Co have been trying to sell to us, but it is a tool nontheless.
And people have been misusing it. People have been foregoeing actually developing the tech in favor of just shipping stuff as fast as non-humanly possible.
That results in unsecure, unstable, crappy code. Which is fine. All of us have written bad code, especially when we were starting out. What is harmful is people trying to sell it as good, putting it directly in prod, or worse, not actually learning the tech along the way, getting stuck with it ad eternum. But learning requires work, time and humility. It is hard, and when presented with an easy way and a hard way it's human nature to choose the easy, instant gratification way, especially when we are misinformed. That is very harmful, but it is basically the product working as intended.
I am sure there are thousands of examples in desing, but one example that is very emblematic of this effect is the creator of #cURL Daniel Stenberg closing the bug bounty program because it was being flooded with crappy AI generated garbage.
My conscience knows that the people responsible for that were mainly people trying to get a quick buck, but my heart knows there were people that were genuinely interested in security, but either got in the bandwagon or were fed a lie that AI bug/vulnerability reports are the future. They didn't learn, but they "shipped" what they thought was a good bug report. And that makes me sad.