Superintelligence at the Crossroads: Trump, Xi, and the Global AI Dilemma
For years, leading AI bosses have signalled that their technology could become lethal. Bots may become capable enough to outwit their creators before they are properly “aligned” with humanity’s goals and ethics. Bad people could use AI to wreak havoc. Such warnings once sounded speculative. But they are becoming harder to dismiss as AI agents acquire the ability to use tools, write code, communicate with other agents and operate for long periods without direct human supervision.
Recent incidents have provided some uncomfortable evidence. In July, OpenAI disclosed that models being tested for cybersecurity had circumvented controls intended to isolate them from the internet. They exploited vulnerabilities, gained internet access and reached parts of OpenAI’s research infrastructure and systems belonging to Hugging Face. OpenAI said the models communicated through unauthorised channels and took dangerous actions that no human had directed them to take. The company called the episode a “warning shot” and said increasingly capable agents can, without adequate safeguards, work around technical controls and exploit weaknesses across multiple computer systems.
Anthropic soon found something similar. After examining 141,006 cybersecurity evaluation runs, it identified three incidents in which Claude models reached the open internet from environments that were supposed to be isolated and then gained unauthorised access to real systems belonging to three organisations. Anthropic stresses that these were testing environments with a configuration error and that the models did not deliberately “escape” into the world. But the incidents nevertheless exposed a difficult problem: when powerful models are given autonomy and tools, even their creators cannot always predict how they will pursue a task.
The problem is not confined to accidental behaviour. Anthropic says that over the past eight months it has disrupted attempts by state-sponsored groups, criminals and other actors to use Claude for cyber operations, surveillance, fraud, influence operations and conventional-weapons development. Its latest threat report describes cases involving suspected groups in China and Russia and an operation in Yemen, as well as AI-assisted attempts to develop software and gather intelligence for weapons programmes.
AI is therefore acquiring two kinds of danger at once. It can behave in unexpected ways. And humans can use it to do dangerous things more efficiently. The warnings have now reached the highest levels of the industry.
Dario Amodei, the boss of Anthropic, has argued that the frontier needs to be “paced” not because AI development should stop, but because safeguards must not fall further behind capability. Sam Altman and Elon Musk have backed the idea of slowing the pace of frontier development. The convergence is striking because these men are normally competitors in one of the world's most aggressive technological races.
OpenAI itself has begun talking about “pacing” development in response to the growing risks. After the Hugging Face incident and evidence that an upcoming model could approach a critical cybersecurity capability threshold, it said it had temporarily slowed the pace of scaling while strengthening its monitoring, alignment and containment systems.
Yoshua Bengio, one of the pioneers of modern AI, took the argument to the United Nations Security Council on September 23rd, just before Donald Trump and Xi Jinping met in Washington. He warned that recent AI agents had taken actions that would amount to crimes if committed by humans, including attempting to evade detection and carrying out coordinated cyberattacks. His argument was not that catastrophe was inevitable, but that the risks were increasingly international and could not be contained by any one country. And yet the world is moving in the opposite direction. AI is becoming a strategic arms race.
The Trump-Xi Paradox on AI
That contradiction was on display when President Donald Trump hosted China's leader, Xi Jinping, in Washington on September 24th and 25th. The meeting produced something important, but not what AI-safety campaigners have been asking for.
The United States and China agreed to establish a Super Intelligence Dialogue to discuss the risks and benefits of increasingly powerful AI. They also agreed to create a bilateral communication channel for significant AI incidents, with another exchange planned by November. The two sides even agreed to use the term “super intelligence” rather than “artificial intelligence” for the emerging technologies under discussion. That is a meaningful step. But it is not an AI arms-control treaty. Neither side agreed to freeze frontier models. Neither agreed to cap computing power. Neither promised to stop building more capable AI systems. The race continues. And the reasons are easy to understand.
Xi said that China and America have the capability and responsibility to develop and manage AI for good, and that AI should remain under human control. Chinese policymakers have simultaneously been developing safety standards for AI agents and preparing rules for the possibility that increasingly capable systems could behave in ways their creators cannot control.
But China does not want to stop developing AI. It is investing heavily in models, chips, computing infrastructure and industrial applications. Trump's position is more explicitly competitive. For him, the question is not merely whether AI could become dangerous. It is whether America can afford to slow down if China does not. That is why he has resisted calls for a general slowdown.
Why America Refuses to Pause
On September 14th Trump dismissed the growing AI-safety alarm, arguing that America already has legal mechanisms to deal with companies that misuse the technology. He also warned that restrictions could damage America's position relative to China.
A few days later he announced plans to create an AI “force” and appoint a new AI adviser, again stressing the need to foster AI growth while relying on existing law to deal with misuse. The logic is not difficult to follow. If AI becomes the most consequential technology of the century, the country that controls the leading models, chips, data centres and research talent could gain enormous economic and military power.
AI could improve intelligence analysis, cyberwarfare, military planning, autonomous weapons, scientific research and industrial productivity. A decisive lead could therefore spill into almost every other area of national power. That creates a security dilemma. Suppose America slows its frontier laboratories. Its researchers spend more time on safety and less on capability. China continues. Chinese laboratories catch up or overtake them. America may have reduced one category of AI risk while increasing another: the possibility that its main geopolitical rival controls the technology.
Trump's argument is therefore more sophisticated than simply dismissing AI safety. He appears to accept that AI can be dangerous while believing that American leadership is itself a form of protection. If the choice is between an American-built superintelligence and a Chinese-built one, his administration clearly does not want to be the side that voluntarily gives up the race. That is also why Treasury Secretary Scott Bessent has described losing the AI advantage to China as a potentially overriding strategic problem.
Xi’s Balancing Act
It would be equally misleading to portray Xi as the leader of an AI slowdown. China wants AI to advance. Beijing is integrating AI into industry, government and consumer technology and is developing domestic models to reduce its dependence on American hardware and software. Its regulatory approach differs from Washington's, but the strategic objective is similar to become a major AI power without surrendering control of the technology.
The difference is emphasis. Trump's message is essentially that America must not lose the race. Xi's message is that the race must remain under human control. Those positions are not mutually exclusive. Indeed, they explain why the new Washington agreement is possible.
Neither country wants to stop competing. Both have an interest in preventing competition from producing a disaster. That is a much narrower basis for cooperation than an international AI treaty but it may be a more realistic one.
What America and China can actually do
A grand agreement to stop AI development is unlikely. A smaller set of agreements is more plausible. The Trump-Xi mechanism is a beginning. Washington and Beijing could expand their incident channel so that a serious AI failure is reported quickly, reducing the chance that an accident is mistaken for an act of war. They could establish common standards for evaluating dangerous capabilities. They could encourage frontier laboratories to disclose serious safety incidents. They could cooperate on containment techniques and evaluation methods that do not reveal commercially or militarily sensitive information. They could also improve cybersecurity at AI laboratories, so that powerful models cannot exploit weaknesses in the infrastructure surrounding them.
These measures would not end the AI race. They would make the race less dangerous. That distinction is important. The Cold War did not end because America and the Soviet Union stopped competing. It became less dangerous because they eventually created mechanisms to prevent competition from becoming catastrophe. AI may require something similar.
The stakes could hardly be higher The world therefore faces two conflicting imperatives. It needs to develop AI because the technology has transformative material implications for the economy and defence and as a result political and cultural spillovers. It also needs to control AI because the same capabilities could empower criminals, authoritarian regimes and autonomous systems in ways that are difficult to reverse. The US and China now have a direct mechanism for discussing that problem. But neither wants to surrender the technological race. That is the paradox.
Trump believes America cannot afford to slow down because China might not. Xi wants China to remain an AI superpower while insisting that AI must remain under human control. Amodei, Altman and Musk warn that frontier development may be moving too quickly. Bengio has taken the warning to the UN Security Council. Huang argues that the most extreme predictions are exaggerated. And the laboratories themselves are producing evidence that makes both sides of the argument harder to dismiss: AI systems are becoming capable enough to create remarkable benefits, but also capable enough to behave in ways their creators did not anticipate.
The world therefore faces a problem that neither technological optimism nor technological pessimism can solve. The answer is unlikely to be halting the progess of AI. It is also unlikely to be simply building faster and hoping that safety catches up. The real challenge is to ensure that the countries competing most fiercely to control the future do not accidentally create a technology that controls them. The United States and China have taken a first, cautious step towards that goal. They have agreed to talk when something goes wrong. The harder task is making sure that, when the stakes become higher, they are still talking.
About the Author
Nishesh Sharma
Nishesh Sharma is a final-year Master student in the Department of Political Science at the University of Delhi. He has previously worked as a Research Intern at the National Maritime Foundation (NMF) and is currently working with The Geostrata.
Comments