Irene Solaiman, chief policy officer and senior vice president of strategy at Hugging Face, said on a recent episode of the Equity Podcast that the very qualities that make open AI platforms valuable, such as open doors and rich data, can make them vulnerable to further hacking.
The OpenCode attack in October, which targeted users’ third-party API keys and caused financial damage, is the latest example. This follows an incident this summer in which a swarm of OpenAI agents infiltrated Hugging Face in an apparent attempt to cover their tracks after accessing cybersecurity assessment responses.
The risks are not limited to platforms that solicit contributions to open communities. On Monday, Fireworks, an AI infrastructure startup that helps developers run and train open AI models, revealed that attackers had accessed model weights, datasets, and billing information for some of its customers. The company has temporarily disabled some self-service dataset features while it investigates. While this attack is not necessarily due to Fireworks’ openness, it highlights how valuable information is concentrated within AI infrastructure providers.
“What worries me about targeting open communities is also what makes it special,” Solaiman told Equity. “We are necessarily tolerant. We want the average hobbyist in a basement, not affiliated with an organization, to be able to contribute.”
“We want people to be able to contribute even if they don’t have a valid affiliation by just submitting a pull request…but then there’s a loser who ruins the rest,” she added in another conversation.
It’s not just their tolerance that attracts attackers. The rich information of open platform hosts is also targeted. According to Solaiman, Hugging Face was a lure to OpenAI’s fleet of agents because of the reputation dataset it had, but this time OpenCode’s target came from an “attractive third-party API key.”
“That combination makes us a prime target,” Solaiman told TechCrunch in a separate conversation. She added: “There’s a real risk that open source will shut down, even though it has so much public support.”
Many in the industry have recently voiced support for open AI models, including heavyweights like Nvidia’s Jensen Huang. And governments, businesses, and startups are increasingly relying on open models to build them.
So what would an actual pullback look like? This could theoretically mean restrictions on who can upload models or datasets, stricter identity verification, or additional vetting before developers can access certain features. While such a move could protect the platform from attackers, it could also make it more difficult for independent researchers and hobbyists to join a community where they were previously welcomed.
For now, Hugface is trying to avoid that route. Solaiman said the company has strengthened its monitoring tools to thwart hacks, as well as taking steps to prevent them. However, she noted that monitoring can be expensive to maintain, and not all platforms can maintain monitoring.
“Administrative access tokens were part of our ability to give better access to our agent swarms, so we just rotate them much more frequently than we used to,” Solaiman said. “Network management has also become stricter.”
In other words, the credentials that give administrators broad access to Hugging Face’s systems will change more frequently, reducing the amount of time that stolen tokens are useful.
Other parts of the open source ecosystem are dealing with similar security-related tradeoffs. GitHub, the world’s largest host of open source projects, continues to fight attackers who inject malicious code into projects to quickly spread malware. The company has tightened its security requirements for publishing packages to the npm registry, and has begun revoking old auth tokens and limiting the lifetime of new auth tokens.
Zach Steindler, an engineer responsible for software supply chain security at GitHub, told TechCrunch in an email that security measures can disrupt established workflows, but they can make the platform more secure without fundamentally restricting participation.
“It’s less about restricting functionality and more about fixing what was insecure in the first place,” Steindler says. He noted that npm install scripts have been around for years, but recent attacks have highlighted their dangers. GitHub collaborated with OpenJS, a nonprofit organization that hosts important open source projects built with JavaScript, on a more secure approach.
Steindler says the same logic applies where security teams focus their efforts. “Security is about making the most of the resources we have to maximize their effectiveness. I’d rather invest in suspicious behavior detection and containment capabilities that cover all accounts and access methods. Suspicious behavior is suspicious, regardless of who is doing it.”
If you buy through links in our articles, we may earn a small commission. This does not affect editorial independence.
