- cross-posted to:
- news@kbin.social
- cross-posted to:
- news@kbin.social
WhatsApp’s AI shows gun-wielding children when prompted with ‘Palestine’::By contrast, prompts for ‘Israeli’ do not generate images of people wielding guns, even in response to a prompt for ‘Israel army’
The thing is, it’s almost impossible to perfectly prevent something like this before it happens. The data comes from humans, it will include all the biases and racism humans have. You can try to clean it up if you know what you want to avoid, but you can’t make it sterile for every single thing that exists. Once the AI is trained, you can pre-censor it so that it doesn’t generate certain types of images you know are “true” from the data but not acceptable to depict - e.g “jews have huge noses in drawings” is a thing it would learn because that’s a caricature we have used for ages - but again, only if you know what you are looking for and you won’t make it perfect.
If the word “palestine” makes it generate children with guns, it’s simply because the data it trained on made it think those two things are correlated somehow, and that wasn’t known until now. It will get added to the list of things to censor next time.