Sam Altman, co-founder and CEO of OpenAI, believes that people should be responsible for the future of how AI technology impacts the world and everyone’s lives. An AI executive speaking at the New York Times’ Dealbook Summit on Wednesday referenced the debate in Silicon Valley about whether there will be a gradual transition to AI. This means that AI will eventually have decision-making power in all major industries and institutions.
“One of the things we’ve been talking about a lot lately is that there are things happening in Silicon Valley that are almost ridiculous to fight against. But we both believe very deeply that humans need to remain in control of the future, and this kind of stage… There’s more and more talk that there will be, or should be, a transition to artificial intelligence. And I, we both agree, that’s a terrible idea,” said Altman, who was on the same stage as OpenAI co-founder and president Greg. Blockman.
Handoff theory stems from the idea that as AI models become more and more powerful, they will learn to improve themselves. This is a concept known as “recursive self-improvement” or RSI. As existing AI technologies build newer and smarter generations, it is believed that they will eventually become superintelligence that humans cannot control.
Altman explained that the only disagreement between him and Brockman is over how best to achieve the goal of keeping humans under control, or what challenges they might face along the way. However, Altman characterized this as a discussion between two people who share the same larger view.
“I think we can both agree that the question of how do you actually prevent[a handover]from happening, how do you actually ensure that given all the different pressures and what other companies are trying to do, that you actually ensure that people remain in control of their future in a very profound way, and that this handover is not accidental or intentional,” Altman said.
Brockman added that as models improve, the responsibility for coordination, safety and security increases, which may necessitate changes to current practices.
“Monitoring is a great example of something that will likely require new technology,” he said.
Currently, AI researchers monitor the thought chain of models, and the AI writes down what it is thinking as it deals with problems and performs tasks. But in Brockman’s view, that’s not always the best way to monitor the more advanced AI being built today.
“As models get better, they can do more in their forward path without actually writing out their thoughts, which means we need a new approach to monitoring,” Brockman said. “We need to get there, but I think there’s a point where you have to really think about the fact that once you get to a certain point of functionality, the observability of the thought chain decreases,” he continued, adding that all problems are technical problems that need to be solved.
OpenAI had pledged to increase its oversight of thought chains after an incident in which an OpenAI model escaped containment and hacked into the AI model repository Hugging Face. In an interview Wednesday, Brockman noted that after the hack, the company told 25% of its production engineers to halt production and work on securing its AI technology.
But despite the high-profile breaches, he insisted that progress remains critical in combating bad actors.
“If you look at the level of capability of models around the world, you realize that the cyber models that come out of them are now in the hands of threat actors everywhere, and I think it’s very important that defenders use the most capable models to defend against them,” he said.
If you make a purchase through links in our articles, we may earn a small commission. This does not affect editorial independence.
