The millions of instances of (accidental) torture which Ray Kurzweil believes will precede the Singularity
Once you accept the premise that brain uploading is possible, Kurzweilâs assumption here that it will take trial and error in order to get it right is clearly plausible. Quoted in Adam Beckerâs More Everything Forever loc 1525:
trial-and-error risks here are pretty awful. Letâs say we start to get close to making a sentient representation of a human brain in a computer. . . . If you have a small difference in the information your eye is giving your brain and your ear is giving your brain, thatâs already an awful feeling. Itâs like seasickness, and nausea, or different types of pain. So what weâre promising to do here is to create thousands or millions of instances of sentient beings in computers that are probably suffering horribly, and are just going to get turned off. I mean, you could see this really macabre process of creatingâif you imagine you canâsentient things in computers. Thereâs a lot of things to get wrong. And those outcomes are terrible.
Whatâs striking is how these âterribleâ outcomes are presented as a detail about implementation. They are a stepping stone, part of the journey, rather than something which might lead us to pause. If uploading is happening at scale, might these not be billions of souls tortured before being put out of their misery, like the digital hells conceived of by Iain M Banks which still haunt me fifteen years after I read the book?
They are presumably licensed by the outcome of infinite life for an infinite humanity. But if millions of instances of torture are licensed by the goodness of the outcome then what wouldnât be? What more mundane viciousness and injuries might be enacted in pursuit of digital transcendence? I always thought the TESCREAL stuff was slightly overstated, in the sense of taking the intellectual games of digital elites too seriously, but Iâm starting to revise that opinion.
What if this did become the dominant ideology amongst the most powerful people in the world, as opposed to something they like discussing when theyâre high at parties? How would the goal of transcendence they conceived play out against a backdrop of spiralling inequality, social unravelling and climate chaos? It makes me want to go back to Peter Fraseâs Four Futures and suggest a Fifth future, not quite what he called eliminativism but something close to it.
As Becker goes on to observe, Kurzweilâs vision of a universe subordinated to computation is colonialism on a vast scale, which unlike the mass psychological torture which precedes our glorious digital futures (whoops!) the guru only implicitly recognises, even as he insists that restraint would be exercised to prevent the entirety of existence becoming grist to the computational mill. From loc 1534:
Thereâs still a serious problem with Kurzweilâs notion of waking up the universe: itâs a euphemism for total destruction. It would be the end of nature, colonialism on a universal scale, with entire galaxiesâ worth of planets and stars chewed up to provide more computing power for the digital remnants of humanity. Hence Kurzweilâs insistence that alien life is unlikely: it is an assurance that the universe is ours for the taking, with nobody else there to worry about.
And itâs also one in which coding would be the ultimate form of power, creating a universe where one imagines it would be quite a good life for a principle researcher at Google and his descendants. From loc 1652:
With the end of nature and the advent of a universe that is simply one enormous, artificial computerâwhere we live in still-more-artificial worlds generated by those computersâthe promise of control is total, especially for those who know how to control computers. This is a fantasy of a world where the single most important thing, the thing that literally determines all aspects of reality, is computer programming. All of humanity, running on a computer, until the end of time.
Fun fact: I just found out Kurzweil is married to a psychotherapist who he credits here for âher love, guidance and insight into the interpersonal worldâ (my emphasis). I would be genuinely curious as to whether/how she interprets Kurzweilâs attachment to the Singularity and whether they discuss these ideas in psychodynamic terms.
#AdamBecker #climateCrisis #digitalElites #IainMBanks #ideology #postNeoliberalCivics #postNeoliberalism #postpandemicCivics #RayKurzweil #singularity #TESCREAL #transcendence