I love
@amycastor@econtwitter.net 's analysis of the
#google /
#GenAI DOOM stunt:
This is like using ChatGPT to simulate a calculator that gets wrong answers — they used stupendous computational resources to imitate a game that ran on a 386 in 1993, for three seconds.
From
https://pivot-to-ai.com/2024/09/07/googles-gamengen-ai-doom-video-game-generator-dissecting-a-rigged-demo/Rigged demo indeed. I hadn't bothered to read the paper or watch the videos--I was over these exercises in cherrypicking years ago--but holy hell.
In case it needs to be said, showing a few seconds of carefully-chosen video in an attempt to claim that
#GenerativeAI can now be a game engine is malpractice.
#Google should not be doing this. Diffusion models are
not game engines.
Here's a snobby confession. Back when I used to review papers for conferences more regularly, colleagues and I used to eyeroll and snicker about what we called "what I did on my summer vacation" submissions. Basically, these are papers along the lines of "We applied algorithm X to problem Y and observed Z". The overwhelming majority of papers of this shape were of low quality and most were rejected, in my view correctly, because they failed to meet some very basic criteria of scientific inquiry. For instance,
- Testing a hypothesis. "Algorithm X applied to problem Y can or will do Z" is not usually a scientific hypothesis; even then it's really only an interesting one if it's part of a larger inquiry
- Addressing a research question. "Can algorithm X applied to problem Y do Z?" is not a research question
- Being falsifiable. Throwing a proprietary model at a problem and publicizing what splurts out is not even replicable, let alone falsifiable
This thing from Google -- hell, much of the pseudo-scientific generative AI hype polluting
#arXiv -- smells like a "what I did on my summer vacation" paper, except with a lot of money and resources behind it.
It also smells like crypto, web3, and NFTs in terms of the level of future-tense-fantasizing necessary to believe any of it. GameNGen looks more like a new cryptocurrency than anything else.
#ComputerScience #science #SciencePublishing #AI