Does Anyone Have a Plan?
Crises have a way of revealing things.
The slow-burning emergency over the risks of artificial intelligence reached a fever-pitch this last week, with Dario’s “pacing of the frontier” essay. Everyone from Senator Sanders to President Trump to the C.C.P to the most important global CEOs felt compelled to weigh in on the implications of Amodei’s missive.
What I have noticed, amid the panic, is that none of the key stakeholders are really saying anything: No one commenting from positions of corporate or sovereign authority has a plan. The crisis has forced people to say something, and it turns out that they have nothing to say.
There are four types of people all saying nothing: the Boomers, the Pacers, the Politicians, and the Sinologists.
The Boomers. The principal figures here include Nvidia’s Jensen Huang, Meta’s Mark Zuckerberg, Palantir’s Alex Karp, and the generally vapid and vacuous hosts of the All-In Podcast. In their view, the notion of pacing the frontier, of slowing the speed of new model development to allow for the safety architecture to catch up, is premised on a baseless overestimation of the model risks and a hawkish approach towards China. Putting aside that those are two contradictory reasons to oppose Amodei’s view—either the risks are so grave as to justify concern about China, or the risks are overblown, in which China is not a threat either—one of the defining qualities of their position is its emptiness. It is, fundamentally, an anti-plan plan. They are lobbying for nothing more than an unrestricted and unrestrained build-out of artificial intelligence at the frontier, with no discernable appreciation for the downside risks. Perhaps more importantly, it is a strategy that lacks any acknowledgement of the popular reaction if they are wrong.
This has been a summer in which a number of frightening stories about swarms of agents escaping containment and hacking sites on the open internet have shaken people’s confidence in this technology and the safeguards supposedly around it. A combination of luck, careful monitoring, and good-enough cybersecurity prevented a problem from turning into a catastrophe. But, what happens if those safeguards fail next time? What do they think will happen if, instead of hacking Hugging Face, a platform foreign to most Americans, the next swarm hacks Chase, or Con Edison, draining bank accounts or shutting off gas in the winter? Do the Karps and Huangs of the world really think that the country is going to stand by as this happens? Or is the more likely response that political pressure—already building in opposition to AI—forces a complete pause, if not a semi-permanent end, to the AI industry in America. Wise or not, the notion that the US government will not reach for a blunt tool to stem an emergency and win political points is fanciful. Failure to engage with that possibility is both myopic and irresponsible, particularly if you do think that there is a race to the frontier that we must win. It seems that the position of the Boomers, the run-as-fast-as-you-can team, lacks a coherent understanding of precisely how this country will react in the face of a genuine crisis. This lack of planning strikes me as remarkable.
The Pacers. Amodei’s essay triggering this national conversation was swiftly co-signed by two unlikely allies: Sam Altman and Elon Musk, who agreed in principle that the speed of AI development must decrease so as to allow the safety architecture to catch up. Given their mutual distrust and deep animosity this was a remarkable development (and perhaps a source of hope for US-China dialogue, wherein the parties exhibit similar pathologies). Amodei’s call to pace the frontier is impressive as a headline, but little lurks beneath the surface. The closest he comes to a concrete plan is to embed external evaluators inside the training process for future models. Putting aside for now Sebastian Mallaby’s thoughtful critique of this suggestion, in which he points to a recent and failed attempt to do the same at DeepMind, and putting aside the fact that Altman and Musk are already splitting from Amodei on exactly how they will implement these new rules, the rest of the essay merely gestures towards general suggestions for U.S. government policy and international multilateral cooperation.
The essay has three sets of ‘suggestions.’ The first—situated in the realm of the technical where Amodei is most at home—seems almost comical in the face of the challenge the frontier labs are supposedly facing. Are we truly supposed to believe that a small team of British AI researchers supposedly given access to the internal training process for the next frontier models and all the relevant data (just wait until the lawyers from Latham hear about that part) will put a finger in the dyke? Forgive my incredulity. Reading it, I could hardly believe it: the mismatch in the scale of the crisis and proposed solution feels almost insulting. How they hope for us to believe that they take AI alignment seriously with this proposition requires mental gymnastics that I lack the flexibility for. And while Amodei may find himself at home in the first section, he is wandering in the last two. The hopes for constructive global cooperation on AI safety amount to little more than that. But, famously, hope is not a plan.
What the essay lacks in substance, it fails to make up for in novelty, as Amodei will almost proudly confess. The hagiography of Amodei and his Anthropic co-founders orbits around one central point: that they alone are properly concerned with AI risk and are working much harder to mitigate against it. They must exist, according to their founding myth, because, if they do not, then other AI technologists, lacking in moral scruples and an interest in humanity’s survival, will hold a monopoly over this technology.
With that myth putting wind in his sails, has Amodei not, in one form or another, been calling for a kind of pacing since he founded Anthropic? Isn’t the point of Anthropic to build frontier AI at a speed at which safety and alignment can keep up? The two possible conclusions from his essay are either that Anthropic has not, since its inception, been running alignment alongside innovation (as it promised us all it was doing the whole time), or it is, in which case the new essay includes nothing new at all. In an industry long famous for open letters and Twitter essays which make news but not impact, one cannot help but feel that this essay is simply part of a storied tradition.
Finding neither novelty nor a plan in this essay, it’s remarkable that it managed to kick off such a storm.
The Politicians. The lack of substantive policy solutions coming from the government is so overwhelmingly obvious as to hardly be worth stating, save for the fact that it ought to be genuinely concerning that neither party appears to be capable of meeting this moment when government is desperately needed. In this time of hyperpolarization, it is perhaps not surprising that we are only receiving two extreme positions: pause all AI training from the Sanders-Bannon populist left-right alliance, and run-as-fast-as-you-can-without-restrictions from a White House wholly captured by the Silicon Valley billionaires whose profits rest on running as fast as they can. Neither of these are plans. They are both fundamentally nihilistic and fail to clear a fairly basic standard of human intelligence, namely the ability to hold two ideas in your head at once without falling down: that AI is both the future and a risk to the future. Neither Sanders nor the White House have produced an actual plan. At best, they are, to borrow a phrase, concepts of a plan.
The Sinologists. Those with the most plausible veneer of substance and seriousness are the China experts. Writing in the New York Times and on Substack, they have raised a number of legitimate concerns and limitations with respect to engaging with China on this issue. On Ezra Klein’s show, one expert will tell us how the U.S. and China have fundamentally different goals when it comes to AI and negotiations must take this into account. In the opinion pages of the New York Times, another expert will describe his experience negotiating with China over AI, recalling the obfuscation and mistrust that emanated from the Chinese side. On Substack, experts will outline how China’s perception of the U.S. remains deeply tied to a view of containment, within which the U.S. is deploying its power to hold back the Chinese. These are all interesting and important contributions to the debate. They add necessary context and color and those leading the negotiations should be armed with that and much more. But it’s not a plan. It does not actually lay out how we should go about it, what we should ask for, why we should ask for it, what we should be willing to give up, why we should give that up, and the implications of succeeding or failing in pursuit of those goals. It is from precisely these experts who we need actual answers from: this dancing around the issue is only so helpful.
Putting out a plan is risky. You are tying yourself to a position and people can judge it and you. By having no plan, you have no stakes nor vulnerability.