A highly consequential debate is raging through the world of artificial intelligence following Anthropic CEO Dario Amodei’s plea to slow the pace of AI training in the name of safety. The problem for investors: much of what’s flowing through the airwaves is noise and won’t help you make money. We do not believe the pace of AI spending will meaningfully slow down for reasons that include the geopolitical competition between the U.S. and China. That needs to be our north star for investing in this environment. There are compelling arguments on both sides of the pacing debate, and its mere existence may indirectly lead to companies taking a safer approach to AI development — if only on the idea that humans, despite their differences, can unite around a belief that AI wiping out humanity would be undesirable. We certainly believe some regulation and guardrails are necessary here. Nevertheless, if you base your investing decisions on who spoke last, you’ll never manage to cut through that noise to the “signal” that tells you what the most likely outcome really is. For that, we need to look to a concept developed roughly 100 years ago, and refined many times over since then, known as game theory. One of those revisions earned John Nash of Princeton University and two others the Noble economics prize. A dramatized version of Nash’s life, including his struggles with paranoid schizophrenia and contributions to game theory, was told in the Academy Award-winning movie “A Beautiful Mind,” starring Russell Crowe. The debate There are two main camps in the AI safety debate: those who support a more coordinated effort to slow down the pace of AI model development, including with potential government intervention, and those who believe the model developers already have what it takes to implement safety measures. The conversation about AI safety risks intensified last week after an Anthropic researcher quit his job and claimed that the Claude chatbot maker and rival OpenAI, where he also previously worked, are “gambling with our lives.” Then came Amodei’s essay over the weekend, which garnered support from OpenAI’s Sam Altman and S paceX ‘s Elon Musk, taking the debate up another notch. In his essay, titled “We Must Pace the Frontier,” Amodei detailed a three-step framework to respond to what he sees as rising AI risks. The first step is embedding third-party safety evaluators at each lab. The second and third are getting democratic countries on the same page and then coordinating with authoritarian governments on safety standards and “limits on the rate of unchecked AI progress,” Amodei wrote. “Some forms of coordination that would be impactful for pacing are legally challenging, and will require government support.” Altman responded by saying, “I agree with Dario that we need to pace the frontier.” In a blog post of its own Wednesday night, OpenAI disclosed six examples of “unexpected or concerning model behavior,” and argued the AI industry hasn’t solved key safety issues “to a sufficient degree to continue responsibly scaling at maximum speed for much longer.” “Dario is right,” wrote Musk, who a few days later suggested the leading U.S. and Chinese companies run safety tests on each other’s models . The leading voices on the opposition include Nvidia CEO Jensen Huang; David Sacks, technology investor and former White House AI and crypto czar; and President Donald Trump . In an interview with Jim on “Mad Money” Tuesday night, Huang said it is “completely unnecessary” to pass new laws or issue antitrust waivers “so that these companies could do their fundamental engineering and do it properly before they release products.” He added, “Safety is an engineering problem. Testing is an engineering problem.” Sacks accused Amodei of an attempt at regulatory capture — that is, making the compliance bar so high that regulation itself becomes a barrier to entry “If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible. But stop pretending you need anyone else’s permission,” Sacks said. “Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways… China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well. So go ahead and pace the frontier. You are the ones setting it.” Meta Platforms CEO Mark Zuckerberg has also questioned the need for coordination, arguing market forces already incentivize safety, which is why his company didn’t rush out its new Muse model. “My view is that trust and alignment are quickly becoming the most important capabilities that will differentiate agents and models. … Meta delayed shipping Muse for several months to focus on safety and security. We didn’t call for everyone else to do this before we would. We just did it as part of our day-to-day work because it was clearly the right thing for people and for us… Engaging independent evaluators and advisors is industry best practice… Other labs can just do this too.” Zuckerberg has plenty of experience with safety issues. He helped shape the social media landscape, and much about what he learned about the consequences of moving too fast, to the detriment of society, is likely shaping his view on the pace of innovation. Meta surely wants to become as dominant in AI as it is in social media, with many fewer legal headaches along the way. It’s fair not to know what to make of this debate. There are legitimate views being expressed on both sides. The challenge is acknowledging that doesn’t help us better predict the most likely outcome. However, a crash course in game theory, and the follow-on concept of the so-called prisoner’s dilemma, will help us cut through the chatter to help make well-educated investing decisions. Game theory and the prisoner’s dilemma “Game theory is a mathematical method for analyzing strategic interaction,” according to the Royal Swedish Academy of Sciences, the organization that awards the Noble economics prize. Born out of the study of games like chess or poker, game theory is used to help understand complex economic and geopolitical issues, where there are individual actors thinking rationally about what’s best for them. The most likely final outcome based on game theory analysis is referred to as the “Nash equilibrium”. Note: we didn’t say the best outcome for all parties, but the most likely — key distinction. While Nash and his co-winners were not awarded the economics prize until 1994, the Princeton mathematician introduced the concept of Nash equilibrium in 1950, setting the stage for the development of the idea of the “prisoner’s dilemma” by Merrill Flood and Melvin Dresher at the RAND Corporation, an American think tank. The concept was formalized and named by mathematician Albert William Tucker. It was used to better understand nuclear strategy and the importance of human cooperation in the early years of the Cold War. The artificial intelligence arms race between the U.S. and China has striking similarities to the Cold War between the U.S. and the then-Soviet Union — and unsurprisingly, the prisoner’s dilemma is relevant today. Consider this: Two people are arrested for a crime and held in different interrogation rooms and pressured to flip on the other. Here are four ways this can play out: (1) Neither one says a word: both go free because the police can’t prove anything; (2) one flips and walks free in exchange for cooperating, while the one who kept quiet gets sentenced to a maximum punishment; (3) the other one is the only talker and the opposite happens (the reverse of option two); or (4) they turn on each other and both get reduced sentences. To illustrate this, we can use a four-box diagram. The blue box is not the best outcome for either, but it is most likely the Nash equilibrium given the circumstances. Here’s the logic behind that conclusion. The problem is they can only guess at what the other is doing. Sure, they could roll the dice — both stay quiet, and both go free. That’s option one. However, if they want to take the possibility of a max sentence off the table, then they know they need to talk. Of course, one may talk, and the other may still stay silent — or vice versa — but the one staying silent understands the risk of a max sentence. That’s option two and three. The fourth option is that they both talk and both get reduced sentences. Clearly, the most mutually beneficial option is for neither rational actor to talk. The issue is that, even if they made a promise to each other before committing the crime that they wouldn’t snitch, neither party can enforce that promise once they’ve been arrested and are in their own interrogation rooms. This cuts to the heart of a key idea in the prisoner’s dilemma called “enforceable cooperation.” The most beneficial option — neither talks — requires enforceable cooperation, which is not possible here. Prisoner’s dilemma applied to AI training Let’s consider this exercise in the context of the global AI race, primarily being fought between the United States and China. In this four-box scenario, China is “prisoner Bravo,” and the U.S. is “prisoner Alpha.” From the perspective of both countries, if each could force the other to cooperate in slowing down AI development, or pace, we would end up with both advancing in a slower, safer manner — all with the understanding that this generates the best outcome for all involved parties. That outcome is analogous to the “both don’t talk” option one from the prisoners’ diagram. However, because that enforcement is not viable, the most likely outcome is that we end up with neither pacing. The logic is the same. The U.S. will not risk pacing only to later learn that China didn’t cooperate and advanced as fast as possible to close the gap, or perhaps even overtake the U.S. in AI leadership. At the same time, China will not risk pacing itself only to learn that the U.S. was bluffing and pulled even further ahead. These are the second and third options. As a result, we find our Nash equilibrium under the scenario that neither paces the speed of AI training advancements. That equates to the fourth option of “both talk and get reduced sentences.” The most mutually beneficial option — a coordinated slowdown — is just too hard to enforce. So, as investors, we don’t think it makes sense to sell our AI names based on what Amodei and other AI leaders are pushing for. Amodei himself has recognized the challenges of coordination on a global stage, specifically with China. It’s a much harder and more nuanced task than a few U.S. labs tapping the brakes. “Global pacing will require cooperation with China, the autocratic country with by far the most advanced AI capabilities,” he wrote. “We must not be naive here: the geopolitical stakes are so high that there will likely be stark limits on what can be achieved, especially at first. If we greatly restrain our AI capabilities in the belief that China will do the same, and then China defects, AI could be so powerful that such a defection could lead to their geopolitical dominance. Therefore, any agreement must either have ironclad verifiability or must be limited enough that defection would not be militarily existential. I suspect that not only the U.S. but also China will have these concerns and anxieties. We should approach any global pacing decision, especially in the near term, in such a way that protects the lead of the U.S. and its allies.” Palantir CEO Alex Karp argued Thursday on CNBC that the leading American AI labs are, nevertheless, not fully realizing the geopolitical implications of slowing down. “One of the biggest mistakes the labs have made is we are in competition with China,” said Karp, whose company provides data analytics tools to the U.S. military. “Some people they are our enemies. Some people think they’re a competitor. Pick your spectrum. Either we’re going to win and provide the [AI tech stack] to the world … or they’re going to win. And that means the danger is not just the danger of dangerous things happening to humanity.” Trump and Chinese President Xi Jinping are set to meet next week, and Karp said he “would love it” if both leaders are able to agree on shared risks around AI, but that he’s not holding his breath. “I think it is unlikely,” Karp said. “The problem is, we’re tight enough as competitors that both sides believe they can win. I would love it if it happened. If it happened, we’d all be happier. At Palantir, we’d be happier. If you could have some kind of framework where the two dominant countries could control it, that would change the world. I would be surprised if it happened.” Bottom line We’re not betting on a material slowdown in AI development and spending anytime soon, and the geopolitical dynamics support our conclusion. Without enforceable cooperation, the prisoner’s dilemma and resulting Nash equilibrium imply that the most likely, rational outcome is that neither the U.S. nor China will slow. Now consider that, in this example, we’re only talking about two countries — albeit the two most advanced nations in AI. There are many more global players in the AI race, and given that extremely capable open-source models are out in the wild and accessible to everyone, including those that wish to inflict harm (both state-backed and rogue actors alike), the idea of pacing becomes even more untenable for any one player. This is an arms race, and an arms race doesn’t stop until one side runs out of money, all sides sign a treaty, or both sides feel that the next marginal weapon won’t make them any safer. In AI, neither side is running out of money anytime soon. You can’t sign an enforceable treaty when everyone in the world is competing in the race (and many have no interest in signing or very much want to do you harm). And, as recent model releases have shown, we are a very long way off before the rate of advancement slows to the point of the next model not being far more capable than the last. (See here for a full list of the stocks in Jim Cramer’s Charitable Trust.) As a subscriber to the CNBC Investing Club with Jim Cramer, you will receive a trade alert before Jim makes a trade. Jim waits 45 minutes after sending a trade alert before buying or selling a stock in his charitable trust’s portfolio. If Jim has talked about a stock on CNBC TV, he waits 72 hours after issuing the trade alert before executing the trade. THE ABOVE INVESTING CLUB INFORMATION IS SUBJECT TO OUR TERMS AND CONDITIONS AND PRIVACY POLICY , TOGETHER WITH OUR DISCLAIMER . NO FIDUCIARY OBLIGATION OR DUTY EXISTS, OR IS CREATED, BY VIRTUE OF YOUR RECEIPT OF ANY INFORMATION PROVIDED IN CONNECTION WITH THE INVESTING CLUB. NO SPECIFIC OUTCOME OR PROFIT IS GUARANTEED.
