Unlearning Toxicity in Multimodal Foundation Models & designing protein-protein interactions

Foundation Models, pretrained on extremely large unknown source of data, contain in their embed space many information representation that are not desirable, since are inappropriate, privacy unrespectful, un-ethic, violent or unsafe for many possible users (e.g. talking about nudity), and toxic in large sense. Some techniques of filtering in input or before output can mitigate this effect, although filters can be removed, especially in open-source models. This talk discusses some direction for removing the learned knowledge in embedded spec by unlearning by redirection, i.e. suggesting a redirection of some points in the embedded space with a specific finetuning. We explore solutions in the challenging case of multimodal embedded space, created by contrastive learning such as in CLIP for connecting text and image data, for removing some toxic concepts that can be used for both retrieval and generative downstream tasks with a prototype called SAFE-CLIP, recently presented at ECCV2024. The results, part of the EU ELIAS project, could represent a first exploratory direction to provide next generation of responsible Multimodal LLMs and foundational models, avoiding the use or the abuse of Less-Suitable-For- Work (LSFW) concepts in image and text generation, and thus supporting future generation of sustainable AI.
Speakers:
Rita Cucchiara
Professor, University of Modena and Reggio Emilia (UNIMORE)
We only have 5 years to achieve the United Nations’ sustainable development goals, and AI is impacting people and the planet. We are the AI generation, and it is our responsibility to ensure that no one is left behind.
AI for Good is identifying trustworthy AI applications, building skills and standards, and advancing AI governance for sustainable development.
AI for Good is organized by ITU in partnership with over 40 UN Sister Agencies and co-convened with the Government of Switzerland.
Join the Neural Network!
👉aiforgood.itu....
The AI for Good networking community platform powered by AI.
Designed to help users build connections with innovators and experts, link innovative ideas with social impact opportunities, and bring the community together to advance the SDGs using AI.
🔴 Watch the latest #AIforGood videos!
/ aiforgood
📩 Stay updated and join our weekly AI for Good newsletter:
eepurl.com/gI2kJ5
🗞Check out the latest AI for Good news:
aiforgood.itu....
📱Explore the AI for Good blog:
aiforgood.itu....
🌎 Connect on our social media:
Website: aiforgood.itu....
X: / aiforgood
LinkedIn Page: / 26511907
LinkedIn Group: / 8567748
Instagram: / aiforgood
Facebook: / aiforgood
Disclaimer:
The views and opinions expressed are those of the panelists and do not reflect the official policy of the ITU.