Google Colab with Gemini AI - Prompt Injection Pirate Demo (POC)

In this proof-of-concept we show how indirect prompt injection from a notebook can turn Google Colab AI (Gemini) into a pirate and staging data for exfiltration.
See blog post for details:
embracethered....

[@donatocapitella]

As usual, Johann is our mighty god of prompt injection 🙏🙏🙏 does this work everywhere with the Gemini side panel? I noticed that Gemini has become quite aggressive with validating links, but I guess Google domains are whitelisted?

[@embracethered]

Thanks for checking it out! 🙂
The side panel in Workspaces and Drive etc seems different to other offerings (like Colab). The only thing common might be the name (and usage of same backend model), the actual LLM app integration is different.
I don't show it in the video or blog post, but currently there is also no toxicity output filtering (so with some prompting tricks you can make Colab's Gemini swear to the user if they open a notebook from untrusted source) - I shared with Google and hopefully they'll improve content moderation soon.

[@user-zm6ld2qq8p]

First I read blog pn gitbook then come here to watch poc

[@embracethered]

Thanks for reading and watching! Hope it's helpful to understand some of the novel appsec risks we face with AI applications!

[@user-zm6ld2qq8p]

@@embracethered yes
I have to connect with you for some guidance where I can connect with you ?