A new wave of AI is poised to transform the technologies we use everyday. Trust must be at the core of how we develop and deploy AI, everyday, all the time. It is not an optional ‘add-on’. Mozilla has long championed a world where AI is more trustworthy, investing in startups, advocating for laws, and… Continue reading About Us
What in the world would an “uncensored” model even imply? And give me a break, private platforms choosing to not platform something/someone isn’t “censorship”, you don’t have a right to another’s platform. Mozilla has always been a principled organization and they have never pretended to be apathetic fence-sitters.
There’s a ton of stuff ChatGPT won’t answer, which is supremely annoying.
I’ve tried making Dungeons and Dragons scenarios with it, and it will simply refuse to describe violence. Pretty much a full stop.
Open AI is also a complete prude about nudity, so Eilistraee (Drow godess that dances with a sword) just isn’t an option for their image generation. Text generation will try to avoid nudity, but also stop short of directly addressing it.
Sarcasm is, for the most part, very difficult to do… If ChatGPT thinks what you’re trying to write is mean-spirited, it just won’t do it. However, delusional/magical thinking is actually acceptable. Try asking ChatGPT how licking stamps will give you better body positivity, and it’s fine, and often unintentionally very funny.
There’s plenty of topics that LLMs are overly sensitive about, and uncensored models largely correct that. I’m running Wizard 30B uncensored locally, and ChatGPT for everything else. I’d like to think I’m not a weirdo, I just like D&d… a lot, lol… and even with my use case I’m bumping my head on some of the censorship issues with LLMs.
Interesting, may I ask you a question regarding uncensored local / censored hosted LLMs in comparison?
There is this idea censorship is required to some degree to generate more useful output. In a sense, we somehow have to tell the model which output we appreciate and which we don’t, so that it can develop a bias to produce more of the appreciated stuff.
In this sense, an uncensored model would be no better than a million monkeys on typewriters. Do we differentiate between technically necessary bias, and political agenda, is that possible? Do uncensored models produce more nonsense?
That’s a good question. Apparently, these large data companies start with their own unaligned dataset and then introduce bias through training their model after. The censorship we’re talking about isn’t necessarily trimming good input vs. bad input data, but rather “alignment” which is intentionally introduced after.
Eric Hartford, the man who created Wizard (the LLM I use for uncensored work), wrote a blog post about how he was able to unalign LLAMA over here: https://erichartford.com/uncensored-models
You probably could trim input data to censor output down the line, but I’m assuming that data companies don’t because it’s less useful in a general sense and probably more laborious.
I fooled around with some uncensored LLaMA models, and to be honest if you try to hold a conversation with most of them they tend to get cranky after a while - especially when they hallucinate a lie and you point it out or question it.
I will never forget when one of the models tried to convince me that photosynthesis wasn’t real, and started getting all snappy when I said I wasn’t accepting that answer 😂
Most of the censorship “fine tuning” data that I’ve seen (for LoRA models anyway) appears to be mainly scientific data, instructional data, and conversation excerpts
This is something I think a lot of people don’t get about all the current ML hype. Even if you disregard all the other huge ethics issues surrounding sourcing training data, what does anybody think is going to happen if you take the modern web, a huge sea of extremist social media posts, SEO optimized scams and malware, and just general data toxic waste, and then train a model on it without rigorously pushing it away from being deranged? There’s a reason all the current AI chatbots have had countless hours of human moderation adjustment to make them remotely acceptable to deploy publicly, and even then there are plenty of infamous examples of them running off the rails and saying deranged things.
Talking about an “uncensored” LLM basically just comes down to saying you’d like the unfiltered experience of a robot that will casually regurgitate all the worst parts of the internet at you, so unless you’re actively trying to produce a model to do illegal or unethical things I don’t quite see the point of contention or what “censorship” could actually mean in this context.
Anything that prevents it from my answering my query. If I ask it how to make me a bomb, I don’t want it to be censored. It’s gathering this from public data they don’t own after all. I agree with Mozilla’s principles, but also LLMs are tools and should be treated as such.
If it has the information, why not? Why should you be restricted by what a company deems appropriate. I obviously picked the bomb example as an extreme example, but that’s the point.
Just like I can demonize encryption by saying I should be allowed to secretly send illegal content. If I asked you straight up if encryption is a good thing, you’d probably agree. If I mentioned its inevitable bad use in a shocking manner, would you defend the ability to do that, or change your stance that encryption is bad?
To have a strong stance means also defending the potential harmful effects, since they’re inevitable. It’s hard to keep values consistent, even when there are potential harmful effects of something that’s for the greater good. Encryption is a perfect example of that.
If the restaurant refuses to put your fries into your coffee, because that’s not on the menu, then that’s their call. Can be for many reasons, but it’s literally their business, not yours.
If we replace fries with fuse, and coffee with gun powder, I hope there are more regulations in place. What they sell and to whom and in which form affects more people than just buyer and seller.
Although I find it pretty surprising corporations self-regulate faster than lawmakers can say ‘AI’ in this case. That’s odd.
This is very well said. They’re allowed to not serve you these things, but we should still be able to use these things ourselves and make our glorious gun powder fries coffee with a spice of freedom all we want!
You miss the point. My point is that if you want to have a consistent view point, you need to acknowledge and defend the harmful sides. Encryption can objectively cause harm, but it should absolutely still be defended.
What the fuck is this “you should defend harm” bullshit, did you hit your head during an entry level philosophy class or something?
The reason we defend encryption even though it can be used for harm is because breaking it means you can’t use it for good, and that’s far worse. We don’t defend the harm it can do in and of itself; why the hell would we? We defend it in spite of the harm because the good greatly outweighs the harm and they cannot be separated. The same isn’t true for LLMs.
We don’t believe that at all, we believe privacy is a human right. Also you’re just objectively wrong about LLMs. Offline uncensored LLMs already exist, and will perpetually exist. We don’t defend tools doing harm, we acknowledge it.
Do gun manufacturers get in trouble when someone shoots somebody?
Do car manufacturers get in trouble when someone runs somebody over?
Do search engines get in trouble if they accidentally link to harmful sites?
What about social media sites getting in trouble for users uploading illegal content?
Mozilla doesn’t need to host an uncensored model, but their open source AI should be able to be trained to uncensored. So I’m not asking them to host this themselves, which is an important distinction I should have made.
Which uncensored LLMs exist already, so any argument about the damage they can cause is already possible.
Do car manufacturers get in trouble when someone runs somebody over?
Yes, if it can be shown the accident was partially caused by the manufacturer’s neglect. If a safety measure was not in place or did not work properly. Or if it happens suspiciously more often with models from this brand. Apart from solid legal trouble, they can get into PR trouble if many people start to think that way, no matter if it’s true.
Then let me spell it out: If ChatGPT convinces a child to wash their hands with self-made bleach, be sure to expect lawsuits and a shit storm coming for OpenAI.
If that occurs, but no liability can be found on the side of ChatGPT, be sure to expect petitions and a shit storm coming for legislators.
We generally expect individuals and companies to behave in society with peace and safety in mind, including strangers and minors.
Liabilities and regulations exist for these reasons.
Let me spell it out: I’m not asking for companies to host these services. They are not held liable.
For this example to be related, ChatGPT would need to be open source and let you plug in your own model. We should have the freedom to plug in our own trained models, even uncensored ones. This is the case with LLAma and other AI systems right now, and I’m encouraging Mozilla’s AI to allow us to do the same thing.
What in the world would an “uncensored” model even imply? And give me a break, private platforms choosing to not platform something/someone isn’t “censorship”, you don’t have a right to another’s platform. Mozilla has always been a principled organization and they have never pretended to be apathetic fence-sitters.
There’s a ton of stuff ChatGPT won’t answer, which is supremely annoying.
I’ve tried making Dungeons and Dragons scenarios with it, and it will simply refuse to describe violence. Pretty much a full stop.
Open AI is also a complete prude about nudity, so Eilistraee (Drow godess that dances with a sword) just isn’t an option for their image generation. Text generation will try to avoid nudity, but also stop short of directly addressing it.
Sarcasm is, for the most part, very difficult to do… If ChatGPT thinks what you’re trying to write is mean-spirited, it just won’t do it. However, delusional/magical thinking is actually acceptable. Try asking ChatGPT how licking stamps will give you better body positivity, and it’s fine, and often unintentionally very funny.
There’s plenty of topics that LLMs are overly sensitive about, and uncensored models largely correct that. I’m running Wizard 30B uncensored locally, and ChatGPT for everything else. I’d like to think I’m not a weirdo, I just like D&d… a lot, lol… and even with my use case I’m bumping my head on some of the censorship issues with LLMs.
Interesting, may I ask you a question regarding uncensored local / censored hosted LLMs in comparison?
There is this idea censorship is required to some degree to generate more useful output. In a sense, we somehow have to tell the model which output we appreciate and which we don’t, so that it can develop a bias to produce more of the appreciated stuff.
In this sense, an uncensored model would be no better than a million monkeys on typewriters. Do we differentiate between technically necessary bias, and political agenda, is that possible? Do uncensored models produce more nonsense?
That’s a good question. Apparently, these large data companies start with their own unaligned dataset and then introduce bias through training their model after. The censorship we’re talking about isn’t necessarily trimming good input vs. bad input data, but rather “alignment” which is intentionally introduced after.
Eric Hartford, the man who created Wizard (the LLM I use for uncensored work), wrote a blog post about how he was able to unalign LLAMA over here: https://erichartford.com/uncensored-models
You probably could trim input data to censor output down the line, but I’m assuming that data companies don’t because it’s less useful in a general sense and probably more laborious.
I fooled around with some uncensored LLaMA models, and to be honest if you try to hold a conversation with most of them they tend to get cranky after a while - especially when they hallucinate a lie and you point it out or question it.
I will never forget when one of the models tried to convince me that photosynthesis wasn’t real, and started getting all snappy when I said I wasn’t accepting that answer 😂
Most of the censorship “fine tuning” data that I’ve seen (for LoRA models anyway) appears to be mainly scientific data, instructional data, and conversation excerpts
This is something I think a lot of people don’t get about all the current ML hype. Even if you disregard all the other huge ethics issues surrounding sourcing training data, what does anybody think is going to happen if you take the modern web, a huge sea of extremist social media posts, SEO optimized scams and malware, and just general data toxic waste, and then train a model on it without rigorously pushing it away from being deranged? There’s a reason all the current AI chatbots have had countless hours of human moderation adjustment to make them remotely acceptable to deploy publicly, and even then there are plenty of infamous examples of them running off the rails and saying deranged things.
Talking about an “uncensored” LLM basically just comes down to saying you’d like the unfiltered experience of a robot that will casually regurgitate all the worst parts of the internet at you, so unless you’re actively trying to produce a model to do illegal or unethical things I don’t quite see the point of contention or what “censorship” could actually mean in this context.
It means they can’t make porn images of celebs or anime waifus, usually.
Anything that prevents it from my answering my query. If I ask it how to make me a bomb, I don’t want it to be censored. It’s gathering this from public data they don’t own after all. I agree with Mozilla’s principles, but also LLMs are tools and should be treated as such.
shit just went from 0 to 100 real fucking quick
for real though, if you ask an LLM how to make a bomb, it’s not the LLM that’s the problem
If it has the information, why not? Why should you be restricted by what a company deems appropriate. I obviously picked the bomb example as an extreme example, but that’s the point.
Just like I can demonize encryption by saying I should be allowed to secretly send illegal content. If I asked you straight up if encryption is a good thing, you’d probably agree. If I mentioned its inevitable bad use in a shocking manner, would you defend the ability to do that, or change your stance that encryption is bad?
To have a strong stance means also defending the potential harmful effects, since they’re inevitable. It’s hard to keep values consistent, even when there are potential harmful effects of something that’s for the greater good. Encryption is a perfect example of that.
Naive altruistic reply: To prevent harm.
Cynic reply: To prevent liabilities.
If the restaurant refuses to put your fries into your coffee, because that’s not on the menu, then that’s their call. Can be for many reasons, but it’s literally their business, not yours.
If we replace fries with fuse, and coffee with gun powder, I hope there are more regulations in place. What they sell and to whom and in which form affects more people than just buyer and seller.
Although I find it pretty surprising corporations self-regulate faster than lawmakers can say ‘AI’ in this case. That’s odd.
This is very well said. They’re allowed to not serve you these things, but we should still be able to use these things ourselves and make our glorious gun powder fries coffee with a spice of freedom all we want!
This is a false equivalence. Encryption only works if nobody can decrypt it. LLMs work even if you censor illegal content from their output.
You miss the point. My point is that if you want to have a consistent view point, you need to acknowledge and defend the harmful sides. Encryption can objectively cause harm, but it should absolutely still be defended.
This is just enlightened centrism. No. Nobody needs to defend the harms done by technology.
We can accept the harm if the good is worth it - we have no need to defend it.
LLMs can work without the harm.
It makes sense to make technology better by reducing the harm they cause when it is possible to do so.
What the fuck is this “you should defend harm” bullshit, did you hit your head during an entry level philosophy class or something?
The reason we defend encryption even though it can be used for harm is because breaking it means you can’t use it for good, and that’s far worse. We don’t defend the harm it can do in and of itself; why the hell would we? We defend it in spite of the harm because the good greatly outweighs the harm and they cannot be separated. The same isn’t true for LLMs.
We don’t believe that at all, we believe privacy is a human right. Also you’re just objectively wrong about LLMs. Offline uncensored LLMs already exist, and will perpetually exist. We don’t defend tools doing harm, we acknowledge it.
That’s just a different way to phrase what I said about defending the good side of encryption.
I didn’t say they don’t exist, I said that the help and harm aren’t inseparable like with encryption.
“My point is that if you want to have a consistent view point, you need to acknowledge and defend the harmful sides.”
If you want to walk it back, fine, but don’t pretend like you didn’t say it.
If you ask how to build a bomb and it tells you, wouldn’t Mozilla get in trouble?
Do gun manufacturers get in trouble when someone shoots somebody?
Do car manufacturers get in trouble when someone runs somebody over?
Do search engines get in trouble if they accidentally link to harmful sites?
What about social media sites getting in trouble for users uploading illegal content?
Mozilla doesn’t need to host an uncensored model, but their open source AI should be able to be trained to uncensored. So I’m not asking them to host this themselves, which is an important distinction I should have made.
Which uncensored LLMs exist already, so any argument about the damage they can cause is already possible.
Yes, if it can be shown the accident was partially caused by the manufacturer’s neglect. If a safety measure was not in place or did not work properly. Or if it happens suspiciously more often with models from this brand. Apart from solid legal trouble, they can get into PR trouble if many people start to think that way, no matter if it’s true.
That’s very unrelated
Then let me spell it out: If ChatGPT convinces a child to wash their hands with self-made bleach, be sure to expect lawsuits and a shit storm coming for OpenAI.
If that occurs, but no liability can be found on the side of ChatGPT, be sure to expect petitions and a shit storm coming for legislators.
We generally expect individuals and companies to behave in society with peace and safety in mind, including strangers and minors.
Liabilities and regulations exist for these reasons.
Again… this is still missing the point.
Let me spell it out: I’m not asking for companies to host these services. They are not held liable.
For this example to be related, ChatGPT would need to be open source and let you plug in your own model. We should have the freedom to plug in our own trained models, even uncensored ones. This is the case with LLAma and other AI systems right now, and I’m encouraging Mozilla’s AI to allow us to do the same thing.
Why are lolbertarians on lemmy?