Three pioneers advocate openness as AI safety concerns grow

The open source model has been a blow to the industry, as projects like Pacing the Frontier look to big labs as a way to keep AI research safe. Because they are freely distributed and there is little control over how they are used, open weight models are not easily controlled, and some labs treat them as downright scary.

But at last week’s Ai4 conference in Las Vegas, three of the world’s most respected AI researchers spoke out on the issue: Nobel Prize winner Jeffrey Hinton, World Lab CEO and co-founder Fei-Fei Li, and Coursera co-founder Andrew Ng. And while they disagreed on specific tactics, all three made strong arguments for keeping AI open.

For the three speakers, the main concern was allowing a few large AI companies to control the pace of progress. When a few companies control access to technology, as Apple and Google do with their mobile operating systems, innovation can be slowed, and the companies that control the platform can influence what is built on top of it.

Andrew Ng said he was concerned about similar developments in AI. “We don’t want to have gatekeepers,” Ng said. “That limits how we all access AI.”

Companies have incentives to protect their competitive advantages, including by influencing the rules that govern their industries. This could create a situation in which only the largest and most capitalized companies have the resources to build cutting-edge AI systems.

Ng’s solution was to maintain multiple providers and competing models and companies, rather than a few players dominating the space. “If I could give you one prescription, it would be to encourage openness,” Ng said. “AI is an amazing technology and I want to make it available to everyone.”

However, not everyone agreed that the open weight model would help maintain its playing condition. Hinton specifically distinguished between open-source software, which makes the underlying code available for inspection and modification, and open-weight models, which make the parameters of the trained AI model publicly available.

“Open source is great. When you show your code to people, a lot of people look at a line of code and say, ‘Oh, there’s a bug.’ Open weights means you train a big model and then give people weights. That’s totally different,” Hinton said. “I was against open[weights]because people could easily take these big underlying models that are very expensive to train, and it would be much cheaper to train them to do bad things like cyberattacks.”

But whatever his reservations, Hinton acknowledged that the indifference weight model is already a permanent fixture in AI. “I think that battle is over. Now that we have open-weight models, that barrier that a lot of people had to getting these big models, which was the cost of training the basic models, is gone. It’s already too late.”

But accepting reality didn’t mean ignoring the risks. Hinton’s position was clear. He thought AI would continue to advance, and that was largely a good thing. He said this would increase productivity and improve education and health care. “To worry about the potential negative effects of AI and what intelligent humans might do when they’re smarter than us. I don’t think that’s unfair. I think it’s unfair to label people who think that way as fearmongers,” Hinton added.

Mr Ng took a different view. He argued that the question is not whether open models are risky, but rather who controls access and who captures the market. Those who create cheaper models will have an advantage. If China’s open-ended model becomes widely adopted in Asia, Africa and the developing world, it could influence how billions of people encounter ideas about democracy, freedom and human rights, he warned.

“One of the things we would like to see done is encourage American competitiveness and open source AI. We see that AI is a tremendous source of soft power. We see, for example, how the Chinese model is achieving tremendous results with respect to Africa,” Ng said. “But my concern is that because of all the lobbying and fear-mongering in the U.S., building open source AI in the U.S. is having a hard time competing with the promiscuous model coming out of China. And my concern is that if China figures out fundamentally more cost-effective ways to build AI, the more cost-effective ones will have a fundamental advantage in business adoption.”

Mr. Lee pushed back against that framework. “It’s very dangerous to make this a dichotomy between complete openness and complete closure,” she says. “In software systems as complex as scientific systems, there are even more nuances.”

Lee used nuclear physics as an example. Scientific papers are published publicly, but uranium is regulated, and laboratory research falls somewhere in between. She explained that the lesson is that openness doesn’t have to be an all-or-nothing choice. Different layers of the ecosystem can operate with different levels of openness.

She also highlighted cooperation between public and private institutions, such as the Human Genome Project. The resulting knowledge became a platform on which others could build, pharmaceutical companies could profit, scientists could advance their research, and society could benefit, she said.

“So I think we need to use (AI) as that infrastructure,” Lee said. “We need some level of openness, as well as a profitable business model for entrepreneurs, both in scientific discovery, education and global partnerships. But we are also willing to accept closed source systems. This argument, especially at the blanket level of ‘only one is acceptable’, is a false argument. We need to get to a level of nuance.”

But everyone agreed that some regulation is needed to keep AI on the right track. “What we want to do is develop AI in a way that helps people, and regulation helps with that,” Hinton said. “We can’t leave it up to people like Elon Musk and Mark Zuckerberg to decide what to do with AI.”