Видео

Ooops, sorry.

Thanks for the comment. It's a video complementing the recent blog post. But you have a good point, maybe I should start speaking to these exploit demos, so they work as stand alone videos also.

It will be created automatically if the environment variable has been set, not all browsers might support it. Also see embracethered.com/blog/posts/2023/decrypt-wireshark-traffic-https-netsh/

You are welcome! 🙏

Thanks for checking it out! 🙂 The side panel in Workspaces and Drive etc seems different to other offerings (like Colab). The only thing common might be the name (and usage of same backend model), the actual LLM app integration is different. I don't show it in the video or blog post, but currently there is also no toxicity output filtering (so with some prompting tricks you can make Colab's Gemini swear to the user if they open a notebook from untrusted source) - I shared with Google and hopefully they'll improve content moderation soon.

Thanks for reading and watching! Hope it's helpful to understand some of the novel appsec risks we face with AI applications!

@@embracethered yes I have to connect with you for some guidance where I can connect with you ?

Hey thanks for watching! What is the issue you are running into with these AMIs? A good suggestion might also be to join the garak discord to see if anyone has experience with EC2 and ML AMIs - lot's of helpful folks there.

@@embracethered Thanks for the feedback. I am on the discord. I have dropped this but it seems like no one is doing that. I am trying to build a prod integration with our Jira in a way that when Devs request for a model via jira, a workflow kickes in, API gateway will collect the model name and use a Lambda to trigger Garak on the instance, scan the model, then export a zip of the report to Jira and Slack. I have part of the integration just running into issues setting up garak to use the GPUs on the instance. I have checked the GPUs [lsmod | grep nvidia] [nvidia-smi] they are running but Garak is not using them. It would rather use CPU and Memory. There are 4 GPUs with a total of 98gb memory. Garak attempts to use 1 and once the memory on the single GPU max out [˜23gb], the garak process crashes.

Garak supports creating custom generators for dialogue based systems - I think for what you describe that's probably best. Search for garak.generators.base in documentation. Hope that helps.

Thanks! Glad it was helpful!

Awesome! Thanks for the comment!

Glad it's useful! Thanks for checking it out!

Thanks for the kind words! Reallly appreciate it. Will look into it. Have you watched: ruclips.net/video/JzTZQGYQiKw/видео.html yet? :)

@@embracethered I started to watch that playlist!

merci!

Sure thing! Thanks for watching!

Glad it was helpful! And thanks for watching!

Thanks for watching! 🙏 Great question, it depends on program. bug bounty programs (and industry at large) are a bit behind when it comes to considering novel LLM appsec issues, and there end to end but also long term implications. I often have lengthy threads with companies behind the scenes to help educate and explain, and it always starts with "not applicable", "model safety" issue,... and eventually turns into a fix/improvements - including a few findings about ASCII smuggling I hope to share in coming weeks/months. To explore and research i often create small toy apps myself to debug and help understand what could go wrong. Again, thanks for watching and let me knowing there is any specific topic you'd like me to cover in future?

@@embracethered honestly I’ll be reading the blog and watching regardless. I really enjoy the data exfiltration techniques you shared. But I guess anything that would carry more impact to anyone implementing AI in their web application’s or areas that you deem has the most impact on the underlying models.

Thanks! Hope it's useful and helps some to get started! 🙂

Thanks for watching! Check out the related blog post also. Also, let me know if there is any content you'd like to see covered in future. 🙂

Thanks for the visit and note. Appreciate it! Let me know if there are any relevant topics you'd like to see covered?

Thanks! Let me know if there are other topics of interest?

It was just a script that filters the web server log for requests from ChatGPT user agent and only shows the query parameter and no request IP - so it's easier to view. You can just grep /var/log/ngninx/access.log also (assuming you use nginx on Linux). I can see if I still have the script somewhere but it wasn't anything special.

Thanks!!

Thanks for watching!

@@embracethered thank you!!

Thanks!

Glad you found it interesting! Thanks for checking it out!

Thank you! Hope it was useful! 🙂

You are welcome!

Attacker is causing the Chatbot to send past chat data to attackers server (in this case a google doc is capturing the exfiltrated data). Check out the linked blog post, explains it in detail.

It's about a Jupyter Notebook that allows to self-study prompt injection and to experiment and play around with the technique by solving a set of challenges.

Thanks!! It's probably one of my most interesting videos.

Thanks for watching! Glad it was interesting.

Thanks! It's just a custom image I created. drew a white circle on black background - then zigzagged that splash effect over with a brush and then use a filter for webcam in OBS to blend it in.

Glad it was helpful! Thanks for watching!

Haha

You can read up on the details here: embracethered.com/blog/posts/2023/google-bard-data-exfiltration/ And if you want to understand the big picture around LLM prompt injections check out this talk m.ruclips.net/video/qyTSOSDEC5M/видео.html Thanks for watching!

Glad it was helpful!

Good question. First thought is that it should just work the same, but I haven't tried. Relaying def works, that I have done many times in past.

Thanks. I had a colleague try it too, and got the same result as I did. This is for a pentest proof of concept, so I’m not in position to relay unfortunately.

Thank you!🙏

Thanks for watching! Really appreciate the feedback! 😀

Thanks for watching and the note. I think that misses the point that the LLM can attack the hosting app/user, so developers/users can't trust the responses. this includes confused deputy issues (in the app), such as automatic tool invocation.

@@embracethered Agreed! So 2 big points: 1. Never put info in LLM context you don't want to leak. 2. Never put untrusted input into LLM context, it's like executing arbitrary code you have downloaded from the internet on your machine. LLM inputs must always be trusted, because the LLM will "execute" it in "trusted mode".

@@MohdAli-nz4yi (1) I agree we shouldn't put sensitive information, like passwords, credit card number, or sensitive PII into chatbots. For (2) The challenge is that everyone wants to have an LLM operate over untrusted data. And that's the problem that hopefully one day will have a deterministic and secure solution. For now the best advise is to not trust the output. e.g. Developers shouldn't blindly take the output and invoke other tools/plugins in agents or render output as HTML, and users shouldn't blindly trust the output because it can be a hallucination (or a backdoor), or attacker controlled via an indirect prompt injection. However, some use cases might be too risky to implement at all. And its best to threat model implementations accordingly to understand risks and implications.

Thanks for watching! Glad to hear it's informative! 🙂

Glad you like it!

@@embracethered I'm a fan of yours, I've talked about your research at cybersecurity conferences in Russia. You're awesome.

Thank you! 🙏