ComfyUI With Florence 2 Vision LLM - This Is Not Just A Segmentation Model

[@yklandares]

Error occurred when executing DownloadAndLoadFlorence2Model:
Failed to import transformers.models.cohere.configuration_cohere because of the following error (look up to see its traceback):
No module named 'transformers.models.cohere.configuration_cohere

[@reaperhammer]

It will be interesting to see how you integrate this into other workflows as you suggested

[@TheFutureThinker]

Yes I will make another video about that. It should be interesting.

[@Ai-dl2ut]

@@TheFutureThinker Can't Wait :)

[@TheCinefotografiando]

I have found myself watching your videos more and more

[@x3gxu]

Hi. How did you make the talking head in the corner? Looks pretty good.
It's not wav2lip. V-express? Hallo? I wasn't able to achieve results like this, so I'm very interested. Can you point me in the right direction?

[@pabloapiolazza4353]

Also interested!

[@TheFutureThinker]

This is a short videos done for you guys : ruclips.net/video/EcAMY-sZ9mY/видео.html
:) hope this help

[@SageGoatKing]

I don't miss the right click menu bar at all since getting the sidebar where I can pin my favorite nodes, etc.

[@TheFutureThinker]

Normally, I use search. I don't have favourite nodes. Cause I use too many

[@kalakala4803]

nice! I just check the Florence-2 LLM video you did. this AI model looks promising. Can you integrate this with AnimateDiff V2V?

[@TheFutureThinker]

😉👍you got it

[@A.eye_101]

Florence-2 LLM is not something diff would use to make it better, however you can use it to help with prompting

[@nkofr]

what's the use of the 'finetuned' versions?

[@TheFutureThinker]

Finetuned model on a collection of downstream tasks

[@nkofr]

@@TheFutureThinker excuse me but what does that mean?

[@riggitywrckd4325]

It means that if you have a dataset of stuff you want it to learn you can train those ideas and it will learn to spot them like it does in this one.

[@patagonia4kvideodrone91]

There are other nodes, I don't remember the name now, but what do you say detect me such a thing, and it generates the automatic mask, (but not square) but with its real contour,

[@TheFutureThinker]

Segment Anything

[@promptaganda]

@@TheFutureThinker every time ive ever tried to add a prompt to a segment anything mode it makes zero mask ....... any suggestions?

[@TheFutureThinker]

@@promptaganda what is your setting?

[@promptaganda]

using spacepxl node, i am getting strange polygons for all images i try to run region to segmentation on. The captioning is working correct. any ideas?

[@TheFutureThinker]

This might possibly happen, when the node have no condition logic to specific conditions you are setting.
Hopefully the developer will do a new update and add something to handle this.

[@davidsmith-lv4kq]

@@TheFutureThinker do u mean the prompt must match a word in the caption or region suggestions? i seemed to try alot of different images and regions and could never get it to do anything other than make the polygons

[@RamonGuthrie]

Hey can you prompt the Florence2 model on what parts of an image you want to describe?
Example Describe the background only or Describe the person in detail only!
or are there better vision models for this?

[@TheFutureThinker]

Yes this vision model can be segment the background and then do captioning

[@Rico-nj3vl]

Very nice !

[@TheFutureThinker]

Thank you! Cheers!

[@swannschilling474]

Great tutorial! Thanks!!😊

[@TheFutureThinker]

Glad it was helpful!

[@yklandares]

Спасибо за урок, ты мне очень помог. Особенно у меня узлы не работали но когда увидел что можно так что он сам скачал, А там оказывается целая папка а не просто модель. В этом и была ошибка. Я помещал модель в папку LLM и он писал ошибку что нет файла JCON а оказывается там скачивается целая папка как узел

[@triojakeson116]

Hey bro i wanted to ask about the kling ai video, can u tell me how much time it will take for u to get accepted after getting into waitlist, cz am already in waitlist for one day, just wanna make sure, if u know pls reply thanks?

[@TheFutureThinker]

Depends, some got it few days in waiting.

[@triojakeson116]

​@@TheFutureThinkeralso bro am from india, and here all chinese apps are banned, so i had to use an American vpn and a fake chinese number, do u think they will accept my waitlist request if they see all this, will they check all this info 😢

[@TheFutureThinker]

@@triojakeson116 sorry to hear that. This is more about the company policy, I have no comment.

[@triojakeson116]

@@TheFutureThinker ok bro i will update if something happens 😭

[@TheFutureThinker]

@@triojakeson116 but wish you good luck , i see other people are getting access now. So hopefully it will be okay for ya

[@crazyleafdesignweb]

that is great stuff! We are getting out of Stable Diffusion with more alternatives.

[@TheFutureThinker]

There's more 😉 well, I think many users are going to keep SD1.5 and SDXL, others from Stupidity AI throw to bin. 🤭

[@A.eye_101]

Stable Diffusion is a process of generative image creation, this is an LLM for labeling and identification. Apples and sand on the beach, 2 different things

[@jairuskersey8311]

Nice vid. Can you also make a tutorial on how you made the talking avatar in this video? Thanks~

[@TheFutureThinker]

just use Hedra, very easy website no need tutorial :) I believe you can do it

[@bilalalam1]

Hi , I would like to use Florence 2 with LM studio

[@TheFutureThinker]

No sure, but I use OpenWebUi x Ollama

[@A.eye_101]

SpaceXL doesn't do video and is more limited, no mask out put and no way to connect it to SAM2 for better SEG, KJ is far better and compatible with SAM2

[@vitalis]

So cool

[@TheFutureThinker]

Yup

[@context_eidolon_music]

Holy crap!

[@yklandares]

help please

[@Beauty.and.FashionPhotographer]

All Photos made with Flux look really fake.....

[@thegtlab]

Not true at all

[@robd7724]

I'm sorry but the AI voice you are using sounds awful and the video is very creepy.