In this (relatively) recent piece, Venkat Rao compares generative AI systems to the Webb telescope, and argues that AI are not machines that produce something, but rather discover things. And the thing that they discover is information / intelligence that is inherent to data.
The argument - as often with Venkat's writing - often gets quite complicated. But the core argument is worth noting also for much less philosophical discussions about generative AI: that ulimately it's the data, and not the model that are crucial.
In the last months, I've been spending much time thinking about dataset governance and developing a commons-based framework for such governance. So Venkat's piece offers a useful theoretical underpinning, a story explaining why this is important.
There's been a lot of progress in 2023 on AI models, with dev teams playing the game of "who can count more billions of parameters?". It was also a year where there few positive developments in terms of dataset development and governance.
Hopefully, in 2024 this trend will reverse.
Venkat's piece:
https://studio.ribbonfarm.com/p/a-camera-not-an-engine