Fine tuning Embeddings Model

[@thevadimb]

First, thank you for your video - I really appreciate your work!
A question - I see the validation loss is actually growing... Am I missing some point here?

[@moslehmahamud]

You are right, i didn’t properly train the model with sufficient data or necessary steps/epochs. Please don’t be like me hahaha
Hope that answers your question

[@nguyenminhhoang5541]

May I ask how many Gb of Vram needed to finetune?

[@חננאלרועיארבל]

you can lower the batch size if you run out of vram. it doesn't affect the quality of training, but it does affect how much time the training takes

[@rahul01483]

do you have any video on how I can train my own dataset from scratch and create embedding vector store

[@moslehmahamud]

yes, a new video will be uploaded tomorrow (as of writing), using hf model to get embeddings. You can use a chroma db to store the embeddings
Hope that helps

[@rahul01483]

@@moslehmahamud sure it helps, as have been using chromadb for some time now... would love to see ur impl

[@tunguyet3615]

@@moslehmahamud where video?

[@ashleeclaral3271]

how should my own custom dataset look like?

[@moslehmahamud]

you can try using pair-wise, labeled dataset to train the embeddings model

[@wilfredomartel7781]

Great video! is it only for english?

[@moslehmahamud]

Thanks, you can train on other languages too, make sure to pick a multi-lingual model.