Artificial intelligence presents a dilemma that becomes more troubling as the technology improves. The same systems that can accelerate scientific discovery, improve productivity, and help solve difficult problems are becoming capable of carrying out increasingly sophisticated cyberattacks and other dangerous activities. Governments and technology companies recognize these risks, yet the prospect of slowing development creates another fear: what happens if a competitor refuses to slow down?
For the United States, the concern is that China might achieve a decisive advantage in a technology with enormous economic and military significance. China faces the reverse calculation, while companies such as OpenAI, Anthropic, and Google worry about falling behind competitors developing similar capabilities. Each participant can argue that continuing to build is the responsible choice because somebody else will develop the technology regardless.
This is the fundamental problem behind the AI arms race. The danger does not require every participant to be reckless or indifferent to safety. It can emerge from individually understandable decisions that collectively produce an increasingly dangerous competition. A comparison with the nuclear arms race reveals why this dynamic is so difficult to control, but also why artificial intelligence presents challenges that conventional arms-control agreements were never designed to solve.
Why Einstein Feared a German Bomb
On August 2, 1939, Albert Einstein signed a letter addressed to US President Franklin D. Roosevelt. Written largely through the efforts of physicist Leó Szilárd, it warned that recent discoveries in nuclear fission might make it possible to construct extraordinarily powerful bombs. The scientists were particularly concerned about Nazi Germany’s access to uranium and the possibility that German researchers could develop such weapons before the United States.
The warning was grounded in a genuine scientific breakthrough. Physicists had discovered that splitting the nuclei of certain heavy elements released substantial energy, and researchers were investigating whether a self-sustaining nuclear chain reaction could be achieved. If sufficiently large quantities of suitable uranium could be assembled under the right conditions, the energy released might produce an explosion unlike anything previously developed.
Einstein was not proposing an ordinary scientific research project. As the US Department of Energy’s account of Einstein’s letter explains, the scientists wanted the American government to recognize the potential military implications and support work that might otherwise receive little official attention. The fear was not simply that nuclear weapons could exist, but that an aggressive rival might acquire them first.
Roosevelt responded by establishing a process for evaluating uranium research. American nuclear efforts expanded gradually before developing into the Manhattan Project during World War II. The eventual program mobilized enormous scientific, industrial, and financial resources, bringing together universities, laboratories, engineering facilities, military officials, and private contractors.
The Manhattan Project was therefore not the automatic consequence of one letter. Subsequent scientific developments, government decisions, wartime mobilization, and the perceived strategic danger all contributed to its expansion. Nevertheless, the original warning captures something important about military competition: a government may invest in a dangerous technology because it considers the possibility of an adversary obtaining it even more dangerous.
Germany never succeeded in developing an atomic bomb. The American program continued, and in August 1945 the United States used atomic weapons against Hiroshima and Nagasaki, events that became part of the devastating conclusion of World War II and its transformation of the international order.
Physicist Joseph Rotblat left the Manhattan Project after concluding that the German threat no longer justified his participation. Einstein also later expressed regret over his involvement in encouraging American atomic research. Their experiences illustrate the moral difficulty of technological races: an initial decision may be defensible under uncertainty, even when its eventual consequences are profoundly troubling.
The connection to modern artificial intelligence is not that an AI system is equivalent to an atomic bomb. It is that the fear of what another participant might build can become an independent reason to continue developing a technology, regardless of growing concerns about its consequences.
AI Is Already Changing the Security Calculus
Artificial intelligence became widely familiar through conversational assistants, image generators, and productivity applications. These systems appeared primarily as civilian technologies, designed to help people write, analyze information, program, learn, and create content. Their usefulness made rapid development commercially attractive long before the public debate centered on national security.
However, the capabilities that make advanced AI useful also have implications for cybersecurity. An AI system capable of writing software, analyzing unfamiliar technical environments, searching for vulnerabilities, and coordinating multistep tasks can potentially assist defenders and attackers alike. As its ability to operate with less human supervision improves, it becomes possible to automate larger portions of activities that previously required substantial specialist expertise.
In August 2025, Anthropic disclosed a particularly concerning example in its investigation of AI-assisted cybercriminal activity. The company reported disrupting an operation targeting at least 17 organizations, including institutions in sensitive sectors. The attacker used Claude Code to automate significant portions of an extortion campaign.
According to Anthropic, the system assisted with reconnaissance, obtaining credentials, analyzing stolen information, and preparing demands tailored to victims’ circumstances. Some demands exceeded $500,000. The operation demonstrated how one person could use AI assistance to undertake work that would ordinarily require considerably more technical effort or a larger team.
The distinction between AI-assisted crime and fully independent AI criminal activity matters. The human attacker initiated and directed the operation, and the system’s behavior depended on the access, tools, and instructions provided. It would be misleading to describe the case as an AI spontaneously deciding to attack organizations.
Nevertheless, the example reveals a significant shift in the economics of cybercrime. If AI reduces the expertise, time, and personnel necessary to carry out sophisticated operations, more attackers may become capable of attempting them. The difficulty of an attack is determined not only by what is technically possible, but also by how expensive and demanding the necessary work is.
Developments in 2026 made the implications more serious. OpenAI initially warned in August that an emerging model might be approaching its Critical cybersecurity capability threshold. In September, the company reported that its Astra model had reached that threshold, as described in OpenAI’s account of the model’s development and safeguards.
A Critical capability designation reflects performance against defined cybersecurity evaluations. It does not establish that a model can independently penetrate any infrastructure it encounters, or that every evaluated capability has been demonstrated in unrestricted real-world conditions. Model access, available tools, authorization, operational safeguards, and human intervention all affect what an AI system can actually accomplish.
Even with those qualifications, the strategic significance is considerable. Countries already treat cyber capabilities as important instruments of intelligence collection, espionage, military preparation, and disruption. AI systems that can accelerate technical investigations or automate portions of cyber operations could alter the relative capabilities of states and nonstate actors.
This creates a difficult incentive. A government might prefer a world in which dangerous capabilities develop more slowly, but it may also fear that slowing domestic development would leave its own defensive and intelligence institutions at a disadvantage.
The result is not a conventional arms race over a single weapon. It is competition over a general-purpose technology whose military consequences may become increasingly important as its civilian capabilities improve.
Why Neither Washington Nor Beijing Wants to Slow Down
The rivalry between the United States and China gives the AI arms race its most consequential geopolitical dimension. Both countries possess substantial scientific, technological, economic, and military capabilities, and both recognize artificial intelligence as strategically important.
China’s ambitions are not merely inferred from Western commentary. Its 2017 New Generation Artificial Intelligence Development Plan established AI leadership as a national objective and envisioned China becoming a leading global center for AI innovation by 2030. The plan connected artificial intelligence to industrial modernization, national competitiveness, economic transformation, and security.
American intelligence assessments have identified China as the United States’ most capable competitor in artificial intelligence. In its 2026 assessment of international security threats, the US intelligence community emphasized the significance of Chinese technological capabilities and strategic ambitions.
These concerns extend far beyond conversational assistants. Advanced AI could influence intelligence analysis, logistics, military planning, cyber defense, scientific research, autonomous systems, advanced manufacturing, and the development of future technologies. The consequences of technological leadership would depend on how those capabilities are deployed, but neither government can confidently assume they would be strategically unimportant.
The underlying dilemma becomes clearer through game theory.
Imagine that both countries are considering an agreement to slow the development of particularly capable AI systems. If both comply, they could potentially reduce certain risks while preserving a more balanced strategic relationship. But if one country complies and the other secretly continues developing its capabilities, the compliant country fears that it might surrender an important advantage.
If both continue racing, neither necessarily obtains a decisive relative advantage, while the costs and risks of development increase. Both might prefer a world with credible, reciprocal restraint, yet neither wants to be the participant that restrains itself while the other races ahead.
This resembles the prisoner’s dilemma, a familiar model in game theory. Each participant’s decision depends partly on expectations about the other’s behavior. Mutual suspicion can make cooperation difficult even when cooperation might produce a better collective result.
However, describing AI development as a game with one inevitable strategy oversimplifies the problem. The outcome depends on assumptions about the value of technological leadership, the likelihood of successful verification, the consequences of failure, and the benefits of slowing development. Governments are not always maximizing technological speed, and their preferences may change as risks become clearer.
Countries can also cooperate in some areas while competing in others. They may disagree about restrictions on frontier model development while accepting common standards for particular military applications or dangerous capabilities. Strategic rivalry does not automatically eliminate every possibility of coordination.
Still, the incentive to continue investing is powerful. From Washington’s perspective, a meaningful slowdown could be interpreted as surrendering an opportunity to preserve technological leadership. From Beijing’s perspective, the same slowdown could risk preserving an international hierarchy that China wants to change.
The difficulty is intensified by uncertainty. Governments cannot know precisely how capable their rivals’ most advanced systems are, how quickly those systems will improve, or which breakthroughs might prove strategically decisive. Even imperfect intelligence about a competitor’s progress can encourage additional investment.
This is why technological competition can become self-reinforcing. One country announces a significant breakthrough, prompting its rival to increase research spending. That response is then interpreted as evidence of accelerating competition, encouraging the original country to invest even more.
The participants do not need to agree that the technology will be revolutionary. They only need to believe that falling behind could become strategically costly.
The Technology That Is Too Useful to Leave Unbuilt
The nuclear comparison becomes more complicated when the productive benefits of artificial intelligence enter the picture.
A nuclear warhead has a narrowly defined military function. Its primary strategic value comes from the possibility of using it or deterring others from using comparable weapons. Although nuclear science has important civilian applications, an assembled nuclear weapon is not itself a productive tool for ordinary economic activity.
Advanced AI is different. The same broad technological progress that raises security concerns can also improve research, engineering, medicine, education, and commercial operations. Restricting development may therefore involve giving up valuable applications alongside dangerous ones.
Mathematics provides an example of how quickly expectations are changing. In May 2026, OpenAI reported that one of its models had disproved a longstanding conjecture connected to the Erdős unit-distance problem in discrete geometry. The result attracted attention from established mathematicians because it represented a meaningful contribution to an area that had resisted progress.
The achievement should not be confused with solving the entire unit-distance problem. Nor does one successful result establish that AI can resolve most unsolved mathematical questions. Mathematicians have emphasized the uneven distribution of difficulty across open problems and the importance of distinguishing substantial breakthroughs from relatively obscure results.
The more significant development is that AI systems are beginning to contribute to mathematical research rather than merely summarizing existing mathematics. Even if their capabilities remain highly uneven, researchers must increasingly consider how computational assistance could change the process of discovering and verifying new results.
Biology offers a more established example. Google DeepMind’s AlphaFold protein-structure research has enabled scientists to access predicted structures for more than 200 million proteins. Its database and associated tools have been used by millions of researchers, helping accelerate investigations across biological science.
Predicting a protein’s structure does not automatically reveal everything about its function, interactions, or clinical significance. Nevertheless, easier access to structural information can reduce an important research bottleneck and create opportunities for experiments that might otherwise require considerably more preliminary work.
Drug discovery illustrates both the promise and the need for caution. In 2025, researchers published results from a phase 2a trial of rentosertib, an AI-discovered drug candidate investigated for idiopathic pulmonary fibrosis. Patients receiving the highest studied dose showed encouraging lung-function results compared with the placebo group.
The findings were promising, but the trial was relatively small and early-stage. Further research would be necessary to establish clinical effectiveness, long-term safety, and whether the benefits persist in larger populations.
These examples reveal why the argument for slowing AI cannot simply be that advanced capabilities might become dangerous. The technology also offers plausible benefits that governments, researchers, patients, and businesses have reasons to pursue.
A country restricting advanced AI development must consider the possibility of reducing its own scientific progress while researchers elsewhere continue. A company developing medical applications may regard improved models as essential to its work, even though similar improvements could also make other applications more dangerous.
This dual-use character does not justify unrestricted development. It explains why the choice between progress and restraint is much harder than a simple decision to stop producing a destructive weapon.
The Economic Costs of Falling Behind
Artificial intelligence also creates powerful economic incentives, even though the magnitude of its eventual contribution remains uncertain.
For years, optimistic forecasts have described AI as a potential source of enormous global economic growth. A widely repeated 2017 estimate suggested that AI could contribute as much as $15.7 trillion to the global economy by 2030. The figure remains influential, but its age and underlying assumptions matter when evaluating its relevance to present conditions.
More recent estimates also vary substantially. Goldman Sachs projected that generative AI could eventually raise global GDP by approximately 7% over a decade, while IMF research has explored scenarios involving substantial but more moderate productivity gains. Economist Daron Acemoglu has offered considerably more conservative estimates of AI’s likely contribution to productivity over a similar period.
These differences are not minor disagreements about a single established number. They reflect uncertainty about how rapidly AI capabilities will improve, which tasks can be automated economically, how effectively organizations will adapt, and whether the resulting gains will translate into widespread productivity growth.
Evidence from businesses provides another important qualification. A 2026 survey involving approximately 6,000 executives found that around 90% reported no measurable effects of AI on productivity or employment during the period examined. This does not prove that AI has no economic value. It suggests that widespread experimentation and substantial investment have not yet produced easily measurable transformations across most firms.
At the same time, AI-related infrastructure investment is already influencing economic activity. According to research published by the Federal Reserve Bank of St. Louis, measured AI-related investment categories contributed approximately 0.97 percentage points to US GDP growth during the first nine months of 2025, equivalent to roughly 39% of the measured growth over that period.
That is a striking contribution, but it needs careful interpretation. It does not mean that AI technology generated 39% of the economy’s total output, nor does it establish that a hypothetical development pause would automatically eliminate 39% of future growth. Investment already made, alternative uses of capital, changes in spending, and broader economic responses would influence the outcome.
Nevertheless, the figures help explain why governments may be reluctant to impose sweeping restrictions. Companies are financing data centers, computing equipment, energy infrastructure, and increasingly large research programs in anticipation of future demand.
OpenAI’s extraordinary infrastructure ambitions illustrate the scale of these commitments. Its proposed arrangements, financing structures, and potential exposure to future demand have created questions explored in greater detail in the economics of OpenAI’s $1.4 trillion infrastructure bet.
These investments do not guarantee that AI companies will achieve the revenue growth necessary to justify their spending. Some facilities may eventually prove unprofitable, and financial arrangements built around optimistic assumptions could become vulnerable if demand disappoints.
Yet the possibility of an investment bubble does not necessarily reduce competitive pressure. A company that believes advanced AI will eventually become commercially indispensable may conclude that slowing its own development is riskier than investing aggressively and discovering later that some spending was excessive.
The economic case also extends beyond infrastructure. AI is changing how organizations approach writing, programming, research, administration, and other intellectual tasks, although the distinction between increased capability and measured productivity remains important. These broader effects, including their implications for employment and creativity, are examined in how AI is changing work and human activity.
For governments, the result is a difficult combination of expectations and uncertainty. Neither Washington nor Beijing knows precisely how much additional prosperity AI will generate. Both have reasons to worry that another country could capture a disproportionate share of the benefits if they slow down.
Why Chips and Open Models Complicate Containment
If governments cannot agree to stop developing powerful AI systems, they may attempt to limit the resources available to competitors.
The United States has pursued this approach through export controls on advanced semiconductor technology. Beginning in October 2022, American restrictions targeted certain high-performance chips and associated technologies relevant to advanced computing, with additional tightening and revisions in subsequent years.
The strategic logic is straightforward. Training sophisticated AI models requires substantial computational resources, and the most advanced chips are produced through highly specialized global supply chains. Restricting access to that hardware could increase the cost of developing frontier systems and potentially slow competitors.
Unlike software, computing hardware is physical. Chips must be manufactured, purchased, shipped, installed, powered, cooled, and maintained. Governments therefore possess potential intervention points that do not exist when a software file can be copied instantaneously.
However, controlling these supply chains is considerably harder than writing export regulations. Restricted products may be rerouted through intermediaries, purchased through complex commercial arrangements, or smuggled across borders.
An Epoch AI investigation into advanced-chip smuggling estimated that approximately 660,000 H100-equivalent units of computing capacity had been smuggled into China through 2025. The estimate carried substantial uncertainty, with a 90% interval ranging from approximately 290,000 to 1.6 million equivalents.
These figures are analytical estimates, not a verified inventory of confiscated or individually traced chips. They nevertheless suggest that restrictions can be undermined by illicit trade on a strategically meaningful scale.
Enforcement actions involving restricted computing equipment have reinforced concerns about diversion. Meanwhile, changes in American licensing decisions have demonstrated that export policy is also shaped by commercial interests, diplomatic considerations, and disagreements over whether restricting sales actually strengthens US leadership.
Even effective hardware controls face another complication: advanced AI training does not always have to follow one fixed architectural model.
Researchers have demonstrated approaches to coordinating training across geographically separated computing resources. Distributed training can involve substantial communication and efficiency costs, so connecting many small clusters is not automatically equivalent to operating one enormous data center. Nevertheless, improvements in distributed methods could complicate regulations that assume frontier development must occur inside a relatively small number of easily identified facilities.
The difficulty grows when model efficiency improves. A capability that previously required enormous computing resources may eventually become possible with substantially less hardware, through improvements in algorithms, architectures, data efficiency, or specialized optimization techniques.
Export controls therefore operate against a moving technological target. They may slow development or raise costs without guaranteeing that restricted capabilities cannot be achieved.
Open-weight AI models create an additional problem. When developers release model weights that others can download and operate, recipients can reproduce, modify, fine-tune, and distribute systems without returning to the original developer for permission.
Chinese laboratories have become prominent participants in this ecosystem. Alibaba’s Qwen family, for example, has encouraged extensive downstream development and model adaptation. The resulting proliferation makes advanced capabilities available to researchers and organizations far beyond the laboratories that originally created them.
The distinction between open weights and fully open-source AI matters. Releasing model weights does not necessarily disclose training data, development methods, or every component required to reproduce a model from the beginning. Nevertheless, access to the weights can substantially reduce dependence on the original developer.
The challenge is not entirely new. When Meta’s original LLaMA model was distributed to selected researchers in 2023, its weights subsequently appeared online outside the intended access arrangements. Once a complete and usable model has been widely copied, recalling every copy becomes extraordinarily difficult.
This does not mean that all advanced AI capabilities are equally easy to distribute. Large models still require computing resources to operate, and frontier systems may depend on expensive infrastructure, proprietary tools, or specialized deployment arrangements.
But containment becomes much harder once capable software spreads beyond a small number of organizations. Governments may regulate powerful training systems, constrain hardware supply chains, and enforce restrictions on domestic companies, yet they cannot assume that software already distributed internationally can simply be withdrawn.
Why Nuclear Arms Control Is Easier to Verify Than AI
The central problem with international AI restraint is not simply persuading competing governments to sign an agreement. It is establishing whether the agreement is actually being followed.
During the Cold War, the United States and Soviet Union faced an extraordinary strategic dilemma. Both possessed nuclear arsenals capable of devastating the other, and both feared that allowing an adversary to develop a decisive advantage could undermine their security.
Despite this rivalry, they negotiated limits on aspects of their nuclear forces. Agreements associated with the Strategic Arms Limitation Talks, the Strategic Arms Reduction Treaties, and subsequent arms-control efforts attempted to reduce instability through numerical limits, declarations, monitoring, and verification.
New START, signed in 2010, established limits on deployed strategic nuclear forces and included procedures for exchanging information and conducting inspections. Its original verification framework allowed each country up to 18 on-site inspections annually.
However, the agreement’s practical implementation was disrupted before it expired on February 5, 2026. It would therefore be misleading to describe those inspection provisions as an active system continuing unchanged today.
Nuclear arms control has always involved imperfect knowledge and political disputes. Nonetheless, it benefited from important characteristics of the technologies being controlled.
Missiles, bombers, submarines, launchers, warheads, and nuclear materials have physical properties that can potentially be measured, tracked, or inspected. Their production and deployment involve industrial facilities, specialized equipment, transportation networks, and other activities that may leave observable evidence.
International nuclear safeguards rely heavily on material accountancy. Inspectors compare declared inventories with records of nuclear material entering, leaving, or remaining inside facilities. Measurements, surveillance, containment arrangements, and inspections can help identify discrepancies or unauthorized activities.
The International Atomic Energy Agency plays a major role in safeguards, although its responsibilities are not identical to bilateral strategic arms-control inspections. IAEA safeguards generally concern nuclear material and the verification of states’ commitments regarding its peaceful use, while agreements such as New START focused on particular categories of strategic offensive weapons.
Neither system makes cheating impossible. Undeclared facilities, incomplete records, restrictions on access, sophisticated concealment, and political noncooperation can undermine verification. Still, physical objects create measurable constraints.
Artificial intelligence presents a different challenge because some of its most important assets are digital.
A trained model can be represented through files containing numerical parameters and associated software. Once those files are copied, the original owner may retain a complete version while another participant possesses an identical copy. There is no equivalent conservation principle requiring the first holder to surrender the model when another acquires it.
Consequently, an agreement that attempts to control possession of particular AI models confronts a major problem. Inspectors may confirm that a laboratory has deleted a model from certain systems without being able to establish that no additional copies exist elsewhere.
Training activity is more physically constrained, but it is also difficult to interpret.
A data center used for scientific research may contain hardware resembling that used for advanced military or cybersecurity applications. The same general computing infrastructure can support different models, experiments, and tasks, while the objectives and capabilities of individual projects may not be apparent from a building’s appearance or power consumption.
Even direct observation of computing equipment cannot necessarily reveal what a model is learning, which capabilities it possesses, or how it might be used after training.
That does not mean AI development is invisible. Large computing clusters require chips, electricity, cooling infrastructure, networking, and financial resources. Governments may be able to observe or regulate some of these activities, particularly within their own jurisdictions or through international supply chains.
The question is whether such observations could provide enough confidence to support meaningful agreements.
The United Nations Scientific Advisory Board’s work on frontier AI verification identifies monitoring computing resources as one possible foundation for international oversight. Researchers have explored complementary methods involving secure hardware, cryptographic attestations, computing records, audits, and technical systems designed to demonstrate compliance with agreed requirements.
For example, a monitoring system could potentially record that certain specialized chips were used for a particular computational process, while cryptographic mechanisms might allow limited verification without revealing every proprietary detail of the underlying research.
Such proposals are important because they challenge the claim that verification is inherently impossible. AI development may be harder to monitor than certain nuclear activities, but the underlying infrastructure still creates potential opportunities for oversight.
However, substantial problems remain.
Hardware monitoring could be circumvented through unauthorized equipment, compromised devices, distributed training, or changes in computing efficiency. Rules based on training-compute thresholds might become less useful if future models achieve equivalent capabilities with fewer resources.
Inspections also raise difficult questions about commercial confidentiality and national security. Laboratories may resist revealing proprietary systems, while governments may refuse to expose sensitive military or intelligence research to foreign inspectors.
A further challenge is deciding what an agreement would prohibit. Training runs above a specified computing threshold are easier to define than general capabilities such as dangerous autonomy, advanced cyber exploitation, or the potential to accelerate weapons research.
A model’s behavior also depends on its tools, access permissions, deployment conditions, and subsequent modifications. Two organizations possessing similar models might create very different risks depending on how those systems are used.
The problem therefore contains several distinct layers: verifying hardware inventories, verifying computational activity, evaluating capabilities, monitoring deployment, and establishing whether participants are obeying agreed restrictions.
Success at one layer would not automatically solve the others.
The history of nuclear deterrence and arms-control negotiations during the Cold War demonstrates that rivals can negotiate meaningful restrictions without eliminating distrust. It also demonstrates that agreements depend on sustained political cooperation and verification arrangements that can deteriorate when relations worsen.
For AI, the most defensible conclusion is that verifying a comprehensive international development halt would be extraordinarily difficult. It does not follow that narrower agreements, specific prohibitions, or partial monitoring arrangements must fail.
Indeed, the difficulty of verifying everything may be a reason to concentrate initially on restrictions that can be defined and observed with greater confidence.
The Same Race Is Playing Out Inside AI Companies
The strategic dilemma is not confined to governments. Similar competitive pressures operate among the companies developing advanced AI systems.
These organizations compete for research talent, computing infrastructure, investment, customers, and technological leadership. They also make different judgments about safety, governance, openness, and the pace at which increasingly powerful systems should be deployed.
One revealing example is the founding of Anthropic.
Dario Amodei previously served as vice president of research at OpenAI. He left in late 2020 and helped establish Anthropic in 2021 alongside former colleagues. His explanations of the departure emphasized concerns about trust and differences in how the organization approached its work.
The significance is not that Amodei opposed advanced AI development. Anthropic became another company actively developing powerful models while emphasizing safety research and responsible deployment.
This illustrates a recurring tension in the industry. Someone concerned about the direction of advanced AI may conclude that leaving the field would surrender influence to organizations whose decisions they trust less. Establishing a competing laboratory can then appear to be a way to pursue a safer approach while remaining technologically relevant.
OpenAI’s own origins reveal a similar competitive logic.
In May 2015 correspondence between Sam Altman and Elon Musk, Altman discussed whether humanity could realistically prevent the development of advanced artificial intelligence. He suggested that stopping development was unlikely and that it might be preferable for someone other than Google to make the earliest major advances.
The reasoning was striking because it connected concern about powerful AI with a justification for developing it. If another company was likely to build the technology regardless, creating an alternative organization could be presented as the more responsible choice.
Musk subsequently described his own concerns about Google’s approach to AI in sworn testimony concerning OpenAI’s origins. His account should be understood as his stated motivation rather than independent proof that every participant shared his assessment.
These examples do not establish that safety concerns are insincere. It is possible for an organization to genuinely worry about technological risk while believing that continued development is necessary to influence the industry’s direction.
Nor do competing laboratories necessarily behave identically. They may adopt different deployment restrictions, testing procedures, disclosure policies, and attitudes toward model access.
The deeper problem is that each can regard its own participation as essential because it distrusts the alternatives. If every developer believes that somebody else will build powerful AI anyway, that belief can justify continued competition even among people who would prefer a slower and more coordinated process.
The argument becomes circular when every participant assumes that its own development is safer than development by its competitors. Those assumptions may occasionally be justified, but they cannot all serve as independent proof that unrestricted competition produces the safest outcome.
There are counterexamples to the claim that companies never slow down. OpenAI described delaying aspects of Astra’s development and release while strengthening safety measures in response to its cybersecurity capability assessments.
Such decisions do not eliminate industry competition or establish that voluntary restrictions are sufficient. They demonstrate that development speed is not always treated as the only consideration, and that meaningful safety interventions can occur even within a competitive environment.
The challenge is making those interventions credible, durable, and sufficiently widespread that responsible restraint does not become an unsustainable competitive disadvantage.
Can an AI Arms Race Be Governed?
The most pessimistic interpretation of the AI arms race is that no participant can afford to slow down, no international agreement can be verified, and no company can trust another company to behave responsibly. Under that interpretation, technological progress continues until a major catastrophe forces governments to reconsider their choices.
The evidence supports a serious concern about competitive incentives, but it does not establish that this outcome is inevitable.
A major source of confusion is the tendency to treat AI governance as a single decision between unrestricted development and a complete worldwide halt. Those are not the only available possibilities.
Governments can regulate particular applications, establish safety-testing requirements, restrict certain exports, require reporting of high-risk incidents, and impose obligations on organizations deploying powerful systems. Such measures may reduce specific dangers without requiring every participant to abandon advanced AI research.
The European Union’s AI Act provides an example of an attempt to govern AI through differentiated obligations, including requirements relevant to general-purpose AI and systems presenting systemic risks. Its existence does not demonstrate that the rules will resolve every safety problem, but it shows that governance can take forms other than a universal development moratorium.
Technical safety measures provide another avenue. Developers can evaluate systems for dangerous capabilities before release, limit access to particularly sensitive tools, monitor misuse, strengthen cybersecurity, and establish procedures for delaying deployment when serious risks emerge.
OpenAI’s handling of Astra illustrates how capability evaluations can affect development decisions. Such measures depend on the quality of testing, the incentives of the organizations involved, and whether safeguards remain effective after deployment.
International cooperation could also focus on narrower objectives. Countries might find it easier to negotiate transparency standards, reporting arrangements, protections against certain autonomous military uses, or mechanisms for investigating dangerous incidents than to verify that no advanced AI research is taking place anywhere.
These approaches would not solve the entire competitive problem. A government could comply with restrictions on particular military applications while continuing to pursue technological superiority in other areas. A company might invest in safety testing while simultaneously accelerating its commercial research.
Yet partial cooperation can still matter. Arms-control history offers numerous examples of agreements that did not eliminate rivalry but attempted to constrain particular dangers.
The central governance challenge is determining which commitments are both valuable and sufficiently verifiable to sustain cooperation. Rules that cannot be observed or enforced may offer little reassurance, while rules demanding complete disclosure of sensitive research may be politically unacceptable.
The problem also extends beyond the United States and China. Advanced AI development involves laboratories, semiconductor suppliers, cloud providers, researchers, regulators, and countries across interconnected global supply chains. An agreement between two governments would be important, but it could not automatically govern every participant or every pathway through which capabilities spread.
Commercial openness presents another difficult trade-off. Broad access to capable models can encourage innovation, independent scientific research, and competition with dominant technology companies. It can also make dangerous capabilities more widely available and complicate attempts to control their use.
Restricting access may reduce some proliferation risks while concentrating technological power in a smaller number of organizations. Allowing unrestricted distribution may improve accessibility while weakening the ability to intervene when dangerous capabilities emerge.
Neither approach resolves the problem through a simple principle.
The nuclear analogy is useful precisely because it highlights the importance of incentives, mutual suspicion, and credible verification. But AI’s dual-use applications, rapid technological evolution, and digital reproducibility require governance approaches that cannot simply reproduce Cold War arms-control institutions.
The goal need not be to prove that every participant can be persuaded to stop indefinitely. It may be more realistic to identify which forms of competition are most dangerous, which restrictions can be made credible, and which safety measures provide meaningful protection without depending on universal trust.
That remains a demanding undertaking. The economic rewards of technological leadership, the possibility of military advantage, and uncertainty about rivals’ progress will continue to encourage development. Agreements will be vulnerable to changing governments, new technological breakthroughs, commercial pressure, and failures of enforcement.
But difficulty is not the same as impossibility, and a powerful incentive is not the same as an unavoidable decision.
Einstein’s 1939 letter is an enduring reminder of how uncertainty can shape consequential technological choices. He feared that Nazi Germany might develop an atomic weapon, and that fear contributed to decisions that eventually formed part of a much larger wartime program. The fact that his concern was understandable did not make every subsequent outcome necessary or desirable.
Today’s AI developers and political leaders face their own uncertainties. They cannot know exactly how powerful future systems will become, how competitors will behave, or which safeguards will ultimately prove effective. They can nevertheless influence the institutions, incentives, restrictions, and standards under which development proceeds.
The greatest danger of the AI arms race may be the belief that competition has already eliminated meaningful choice. When every participant treats continued acceleration as inevitable, restraint begins to appear irrational before its possibilities have been properly examined.
An AI arms race is difficult to stop because governments and companies have powerful reasons to keep building. Whether those reasons determine the future entirely depends on decisions that have not yet been made.
Last Updated on October 27, 2026 by Aseem Gupta