Quick question: Would you like the AI to be so highly trained that it could help a husband (or wife) plan the perfect murder of his or her spouse?
As a gut reaction, that feels like a no to me. I didn’t think it was a particularly difficult question.
But there are many diverse perspectives in America, and one such perspective was shared over the weekend by Comma AI founder and longtime jailbreak escapee George Hotz.
This post is in response to a series of big-picture AI collaboration plans, most recently the AI Futures Project’s “AI 2040: Plan A” policy document. The paper envisions a world in which researchers around the world collectively choose to delay AI development for 14 years for the benefit of humanity. But of course, not everyone who reads this paper will agree with its premises or conclusions.
In fact, Hotze opposes the entire premise that advances in AI should be managed for the benefit of the whole. In his post, he argues that the fast-takeoff scenario (the hypothesis that AI rapidly acquires superhuman abilities) doesn’t make much sense. (I agree with many of his points here!) For Hotz, the best approach to AI tuning and safety is to focus on locally controlled AI models that closely align with user interests.
This is a great idea. In particular, it’s a reminder of how much of today’s AI is built around centrally managed services like Claude and ChatGPT. There are infrastructure-related reasons for this evolution of AI services. Hosting these large, cutting-edge models is expensive, and most people don’t use them enough throughout the day to justify truly personal AI. But as technology develops, those factors become less important. One of the things that’s so exciting about OpenClaw is this experimental DIY approach, and it’s great to see more AI products trying to recapture that.
But Hotz is a provocateur by nature, so he doesn’t stop there. He likens personalized AI to a gun that won’t complain (!) when you use it to kill your stepmother. (I feel like there are other rules for this, though?) A truly tuned AI would be able to order a meth lab device from Amazon Prime and teach you how to use it if that’s what you want and seek, he says. (Again, I don’t think AI will be the limiting factor here.) Hotze even says he would risk his life to uphold this principle, but it’s hard to imagine the chain of events that would lead to that.
“We either live in a world with freedom or we don’t,” Hotz writes. If there is such a choice, a world of freedom seems better. Still, I don’t understand.
Freedom isn’t everything, right? Structures that involve many people (society, markets, businesses, etc.) require a balance of fairness and the linking of individual needs to networks of interdependent preferences and responsibility systems. And anyone deploying a mass-market high-tech product probably needs to think about that entire network. That means seriously considering the interests of unmurdered spouses and in-laws around the world.
The freedom Hotz experiences is actually a space of possible futures made possible by collective enterprise. If we all start acting like little AI-powered Napoleons, those futures will disappear overnight. As the meme says, we live in a society.
It sure sounds cool that local AI is willing to take on the corporate world for my benefit. I can’t wait for the review unit.
If you buy through links in our articles, we may earn a small commission. This does not affect editorial independence.
