Fine-tune PaliGemma for image to JSON use cases
HTML-код
- Опубликовано: 22 авг 2024
- In this tutorial, I'll showcase how to fine-tune PaliGemma, a new open vision-language model by Google on a receipt image to JSON use case. The goal for the model is to learn to output a JSON containing all key fields from a receipt, such as the product items, their prices and quantities.
Do note that PaliGemma is just one of many vision-language models released recently.
The notebook can be found here: github.com/Nie...
This is awesome, thanks for sharing, I'm only 5 minutes in right now but will definitely follow it all later. These types of tutorials are gold-dust, calm, concise and not just for entertainment.
Thanks for making this super in-depth tutorial Niels! And very nice dataset/task you chose, I like it.
Best video i've seen so far about finetuning with huggingface. I appreciate that you also take time to explain some concepts behind. Other youtubers are just reading out loud their notebooks...
Great Explanation!!
Amazingg. Such a comprehensive and detailed video. Loved it
Thanks for your video.
Thank you so much for the amazing tutorial!
Thank you for this excellent presentation. In future can you do a video on how to take two take two different HF models and merge them , like this one .
amazing video, great explanation
@Niels Rogge it is amazing content and i have been following you since the time of donut tutorial keep going buddy.
I have one question is it possible to get the snippets extractions also from the invoices?
please let me know and reply
Wow, Look who's back 🔥
banger video as always 🔥
Is this model robust against incomplete labels/ground truth ? Let say you have for some images all details and for others not so many
Superb content however if I want to add specific key elements to be extracted from custom data how will i do it pls reply
Thanks for the awesome tutorial, can this be run in colab T4 GPUs (16GB)?
no more runpod?