Releasing model weights raises the risk of catastrophic misuse, because guardrails cannot be applied and usage cannot be monitored once the weights are out.

2 source(s) assert this claim, 4 deny it.

Who asserts it

In fact, the most dangerous model may be one that is trained in secret and handed only to the People's Liberation Army for use in drones and the Ministry of State Security for surveillance and repression. 2. My secondary concern is the risk that powerful AI models may be misused to carry out cyberattacks or biological attacks, and may have [serious alignment problems](https://time.com/article/2026/07/24/openai-hugging-face-attack/).

Our position on open-weights models — 2026-07-27

I agree with all of that. I will add a few things. One, one concern I have is that even if companies use watermarking and especially because there's now several open source versions to train LLMs or use them, including model weights that have been made available to the global community. We also need to understand how things can go wrong on that front. In other words, people are not all going to obey that law. And one important thing I'm concerned about is one can take a pre-trained model, say by, by a company that made it public, and then without huge computing resources.

Oversight of A.I.: Principles for Regulation — Full Hearing Transcript — 2023-07-25

I think the path that things are going in terms of the scaling of, of open source models, I think it's going down a very dangerous path. And if the, if again, if the path continues, I think we could get to a very dangerous place. I think it's worth saying some things on open source models that are clear to all the experts, but I want to make sure is, is understood by, by this committee, which is when, when you control a model and you're deploying it, you have the ability to moderate usage. It might be misused at one point, but then you can alter the model.

Oversight of A.I.: Principles for Regulation — Full Hearing Transcript — 2023-07-25

Who denies it

There's a lot of things that are said about this that are basically complete fantasy. There's actually a report that was just published by the RAND Corporation where they studied, with current systems, how much easier does it make [it] for badly-intentioned people to come up with recipes for bioweapons? And the answer is: it doesn't. The reason is because current systems are really not that smart. They're trained on public data. So basically, they can't invent new things.

Yann LeCun on AGI, Open-Source, and AI Risk — TIME Interview Full Text — 2024-02-13

Now, future systems are a different story. So maybe once we get a powerful system that is super-smart, they're going to help science, they're going to help medicine, they're going to help business, they're going to erase cultural barriers by allowing simultaneous translation. So there's a lot of benefits. So there's a risk-benefit analysis, which is: is it productive to try to keep the technology under wraps, in the hope that the bad guys won't get their hands on it?

Yann LeCun on AGI, Open-Source, and AI Risk — TIME Interview Full Text — 2024-02-13

There is an ongoing debate about the safety of open source AI models, and my view is that open source AI will be safer than the alternatives. I think governments will conclude it's in their interest to support open source because it will make the world more prosperous and safer.

Mark Zuckerberg: Open Source AI Is the Path Forward — Full Manifesto — 2024-07-23

Every quotation on this page was fetched from the archive when the page was built. A quotation the archive cannot supply fails the build, so nothing here is paraphrased and no source is quoted beyond what it actually wrote.