We’ve seen a big debate surrounding the safety and potential slowdown of AI, as Anthropic CEO Dario Amodei recently announced plans to “pace the frontier,” while Nvidia CEO Jensen Huang has publicly echoed President Donald Trump’s claims that the AI backlash is a hoax and that regulation is unnecessary.
On the latest episode of TechCrunch’s Equity podcast, Kirsten Korosec, Sean O’Kane, and I discussed whether executives like Amodei and OpenAI CEO Sam Altman seriously want to slow down.
I was surprised at how many industry leaders supported Amodei’s plan, but Sean pointed out that the plan seemed to lack detail.
And when Kristen asked whether the combination of existing regulations and free market mechanisms could provide enough safeguards on their own, Sean replied, “We have the federal government at this point, but they certainly don’t seem keen on enforcing broad regulations, let alone being specific here.”
He added: “There doesn’t seem to be a lot of choice for consumers to drive this market, in the sense that if one of these companies did something really bad, it would impact things like the number of cancellations.”
Continue reading for a preview of the conversation, edited for length and clarity.
Anthony Ha: What do you think, Sean? Are we really going to “push the frontier”?
Sean O’Kane: That somehow sounds like something out of “Point Break.” It’s a very Californian, “We’re going to walk the frontier, okay?” kind of thing.
I don’t know. I said on this show last week that I don’t think we’re headed for any kind of slowdown. The main reason is because I don’t think these companies are structured the way it works.
So this is a pretty immediate reversal of what I said a week ago, and I heard this from Dario from Anthropic and Sam from OpenAI, and to some extent from Elon Musk. And we can debate whether we believe it or not. Let me tell you, I don’t think I was completely wrong. Because despite these high-level plans that Mr. Dalio has announced in blog posts and that other leaders have talked about, it feels like a lot of details are still missing, including some of the arguments about why these people think they’re dangerous or at risk, as well as details about what it means for them to slow down.
Anthony, I know you read this article pretty thoroughly over the weekend, but all the content just jumped out at me, so I’m curious if you’re more forgiving than I am as far as understanding what they’re trying to say here.
Anthony: So I have a question as well. What particularly struck me was the fact that so many people in the industry seemed to come together on this fairly quickly. Even before Amodei published his blog post, I had seen other people posting versions of this (plan) a few days earlier.
My understanding is that these suggestions came from the AI safety community. These are things that have been percolating for a while. It wasn’t something that (Amodei) came up with on his own. But it seemed like a lot of people were ready for this, and in some ways it’s encouraging that there’s a consensus. But I’m also a bit skeptical that this will be a truly major step towards slowing down.
Kirsten Kolosek: First of all, consensus is a good thing. It may also lead to the formation of a cartel. So let’s be clear about who we’re talking about here. We’re talking about Frontier AI Labs. It’s something specific and important.
You mentioned earlier that the details aren’t very clear, but we know that the general idea here is that there will be independent third-party evaluators coming in that work internally to[companies]like Anthropic and OpenAI to monitor AI safety practices and incidents. However, there are many ambiguous areas around this.
Another proposal is to force major AI companies in democracies to coordinate safety standards and restrictions. And this international coordination. These are the general thoughts on this.
I would also like to add that there is quite a bit of backlash from very specific people. And to my surprise, there are some people I didn’t expect to rebel against this. The obvious one is Nvidia CEO Jensen Huang. He chose his words very well. I don’t know what either of you think about his on-stage comments and some of the phone calls from President Trump.
Sean: I think Jensen has some understanding that he’s seen as an adult in this room. This is interesting. Because that’s the dynamic that I think has changed. If you asked me a year ago, almost two years ago now, who was in charge when these AI companies really started to explode, I think a lot of people would have mentioned Microsoft and Satya Nadella. I feel like Microsoft has lost its grip as an adult in the room for a variety of reasons that I don’t need to go into right now. True, that’s partly because Mr. Nadella said he was going to make Google dance, and that didn’t happen. And in fact, in many ways the opposite happened.
Jensen not only understands that he needs to please the most people and therefore want to say the most thoughtful things, but he is also one of the most effective liaisons between this field and this administration, even receiving a phone call from the president while on the podium at this very well-attended, very high-profile meeting where both sides were able to respond to all of this turmoil.
Kirsten: To your point about him being the adult in the room and the liaison, he’s in a position to do that. But in my view,[his appearance]felt a little staged, but I think it went a step too far and really highlighted the fact that Nvidia is benefiting greatly from the slow paced advances in AI.
I don’t want to be cynical, but that was kind of my first reaction.
Anthony: Trump and his fans have aligned interests, and (fans) are telling (Trump) what they want to hear.
I thought it was worth unpacking a little bit about some of the things he specifically focused on. First he talked about this idea that “we’re not going to slow down.” What’s interesting is that the phrase “slowdown” comes up a lot, and at the same time, both Altman and Amade use the word slowdown (planned), but the word they would like to use more is “pace.” right?
And these three proposals — Kirsten, the ones you just described — could theoretically lead to slowing things down, but they’re not explicitly saying that. It’s saying, “Hey, we’re taking basic safety precautions and we’re having some people watching over us.” But these things don’t inherently mean that things have to go slower than they are now, at least as I understand it.
Kirsten: So I have a question for Sean. In a free market society, with existing regulations and a competitive environment without protectionism, if a company releases something dangerous, can’t you argue that the company can no longer survive? And are there now regulators that can go after these companies? Or do you reject the idea?
Sean: In a vacuum, or in a market that isn’t distorted by various external pressures, yeah, I think there’s an element of that. But at this point, the federal government doesn’t seem to be keen on enforcing regulations, let alone specifically here or broadly.
And we find ourselves in this strange situation again. So, I don’t know the right way to say it, but consumer choice doesn’t seem to be driving this market. That means if one of these companies did something really bad, it would impact things like the number of cancellations.
Especially now that they’re making more money by going into corporations and selling services to corporations, it’s naturally harder to put money there and vote. Companies aren’t going to move from (OpenAI’s) Codex to (Anthropic’s) Claude Code just because they don’t agree in principle with what OpenAI has done.
And since they are simply kept afloat by large amounts of invested capital, it is easier for them to absorb losses if they actually face them. Sounds like a good idea in theory, but I feel like nothing is happening in practice.
If you make a purchase through links in our articles, we may earn a small commission. This does not affect editorial independence.
