For AI leaders Doom is a form of hype
Ranked #1 on Hacker News with 80 points and 26 comments.
I am so fed up with doomy statements from the AI leaders. I may have lost my patience with it. The latest instance is the BBC’s report that Evan Hubinger, who leads alignment science at Anthropic, believes there is a “greater than 10% chance” that AI could “kill all humans” within the next decade. The occasion was the resignation of Jacob Coxon, a 27-year-old pretraining researcher who had worked at both OpenAI and Anthropic, and who wrote: “Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.”[^3]
In what follows, you will find an in-depth report curated by Perplexity.
I do not doubt that these people are sincere in some private, psychological sense. Coxon’s thread is genuinely anguished; he reports that executives who “couch their phrasing in the press to sound sensible” express real fear in private. But sincerity of feeling is not the same thing as honesty of speech act. As a media scholar, what interests me is not whether Hubinger believes his own number. What interests me is what this genre of utterance does in the world — who gets to speak it, from what institutional position, with what effects on markets, regulators, publics, and on the far less glamorous harms already underway. My complaint is not that these statements are alarming. It is that they are apocalyptic in form, hypocritical in structure, and strategically productive in effect — and that they crowd out almost everything else we could be discussing.
Let me lay out the pattern, then the critiques, then what I think we in media and communication studies should actually do with it.
It helps to read these statements as a series rather than as isolated confessions. The genre has a history and a remarkably stable grammar.
May 2023 — the founding text. The Center for AI Safety publishes a single sentence: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.” It is signed by Sam Altman (OpenAI), Demis Hassabis (Google DeepMind), and Dario Amodei (Anthropic) — that is, by the three men most responsible for building the thing — alongside Geoffrey Hinton, Yoshua Bengio, Ilya Sutskever, Daniela Amodei, and some 350 others.[^7] Leaders from Meta did not sign. Yann LeCun’s response was that “the most common reaction by AI researchers to these prophecies of doom is face palming.” Two months later, more than 1,300 signatories of a counter-letter organised by BCS, the UK’s professional body for IT, declared AI “not an existential threat to humanity.”