On Monday, Mark Zuckerberg released a 6,500-word manifesto on personal AI, mostly about the potential for “personal superintelligence” systems that meta-AI is building. The ideas in this post are not entirely new. A version of this essay was published in the Wall Street Journal two weeks ago, and he previously spoke about them on Meta’s earnings call. But this is probably the most detailed version he has shared.
I’ve seen this described as an anti-Dwemer essay, but that’s not entirely correct. This rather explains why Mark Zuckerberg is personally excited about how AI will change society. But even as Zuckerberg tries to envision a glorious future with abundant superintelligence, he keeps reminding us that it can go wrong.
I also find AI exciting, which is why I continue to write about it. However, many citizens view AI as creepy and unpleasant. So it’s worth finding out exactly what’s going on here and why Zuckerberg doesn’t benefit the industry with essays like this.
Social media remains a sensitive topic in the tech world, and there’s no room here for all the complaints people have about Facebook. Suffice it to say that both Facebook as a product and Zuckerberg as a person are unpopular with the American public. A recent survey found that 64% of Americans believe social media is harmful to democracy, and a similar proportion think social media should be more tightly regulated, a number that is evenly distributed across party lines. Just this weekend, a court fined the company $567 million for being harmful to children. The atmosphere is bad.
I bring this up not to mean that Mr. Zuckerberg should retreat into the wilderness in shame. However, the influence from social media is one of the main reasons why there is currently so much anxiety about the social impact of AI. Whether it’s fair or not, the public doesn’t trust technology company executives to believe that new technologies like this will have a positive impact on society. Rather than acknowledging that and trying to rebuild trust, this essay shows over and over again how trust was lost in the first place.
Much of the essay is devoted to vague generalizations about intelligence, similar to the vague generalizations that Zuckerberg and Dorsey were making about free speech. For example:
As everyone gains more powerful tools, each person’s ability to shape the future will increase, not diminish. … People and organizations with competing interests naturally check and balance each other, leading to positive outcomes. The best and most realistic path to building a positive future for AI is to make superintelligence available to everyone.
i guess? I can think of several examples where conflicts of interest can lead to negative outcomes, but let’s leave them aside. What’s even weirder is that this is presented as a serious philosophical conclusion, rather than a specific product called “personal intelligence” that Meta is bringing to market. If you have already stopped trusting this person, you may worry that they are writing something like this to convince themselves that nothing bad will happen.
That feeling was reinforced for me when Mr. Zuckerberg gave specific examples, many of which are shockingly out of touch with the reality of how AI tools are used. For example, here are Zuckerberg’s views on AI in education:
Everyone has a personal tutor and coach with a PhD in every subject, giving you endless patience to learn anything you want. Students can receive additional support in areas they need, but it is currently only available to students whose parents can pay for it. Adults get a super-intelligent learning assistant who knows exactly how to teach you whatever interests you, whether it’s a new job skill, a new language, or a new hobby.
Of course, the product Zuckerberg is describing already exists. This is a consumer chatbot like ChatGPT, Claude, and Gemini. These tools are very useful if you want to learn about a particular topic, but their primary use in education is to avoid learning. Because your personal tutor can do your homework and write your assigned essays. Additionally, there is no robust watermarking system in place, so teachers have no way to see which essays were generated by AI.
This is a fairly low-risk example of harm from AI, but it’s happening in real life. Designing technology to do good can have unintended consequences that make things worse. And the fact that one of the most powerful people in the world is unwilling to admit this can understandably make people nervous.
Zuckerberg also gives examples of how AI could change the legal system for the better.
As a thought experiment, imagine that you had only one highly intelligent lawyer. Even if the case is wrong on the merits, they will have an unfair advantage in court. It will make society even worse. But now imagine that everyone has a very intelligent lawyer. In this case, justice would be served far more fairly and efficiently than currently, where there is often an imbalance in litigation skills and resources.
Again, this is a loaded example. Indeed, access to legal AI may lead to a more just society, but it may also complicate the already existing bureaucratic hellscape. Or it could spark a new wave of relentless litigation that only clogs up the work with what legally amounts to spam. It’s hard to stay calm when it comes to things like this, and the fact that Zuckerberg isn’t worried makes me even more worried.
Elsewhere, even Meta’s descriptions of existing business practices begin to take on a strangely abstract character. For example, Zuckerberg describes Meta’s approach to the freemium model:
Everyone will have free or affordable access to these tools. …We offer a free version that is accessible to billions of people. For those who want to pay to use more computing, there will be a dynamic auction mechanism that guarantees the lowest possible price for the intelligence and computing that everyone uses, while also ensuring that capacity is used for what people collectively think is most valuable. This ensures that the benefits of superintelligence are widespread.
Broadly speaking, I agree with everything he’s saying here. Zuckerberg is right that access is an issue, and that a freemium model gives everyone more access than paying for computing upfront. A dynamic market for spot computing already exists, so he’s not imagining something completely outside the norm. But there’s a reason why all consumer AI products protect end users from the spot price of the compute they’re using. Surprising pricing is a terrible user experience, especially when implemented in tools that your business actually relies on.
Again, I have to think that Mr. Zuckerberg knows this and does not actually intend to set the price of his tokens through a dynamic auction. But the truth is, I’m less convinced of that now than I was before I read this piece.
There are better ways to talk about this. Despite their shortcomings, Sam Altman and Dario Amodei do a very good job of communicating directly with the public. They’re doing exactly what Zuckerberg is resisting here: recognizing the dangers of AI, highlighting their precautions, and trying to convince listeners that they can be trusted.
There are limits to what you can do with that communication. Sometimes (especially recently) safety measures fail, exposing the inherent dangers throughout the enterprise. But without some trust-building, the industry won’t necessarily be able to overcome these failures.
If you buy through links in our articles, we may earn a small commission. This does not affect editorial independence.
