Not bad, but if i may say any tip, i would say to use a sentence-transformer from hugging face for embeddings and then use a llama like llm only to customize the answer, because models created just for embeddings seems to be more accurate for this task. Also, langchain module is easier and useful than using numpy and requests, with lot more features. I use this setup most for RAG and seems to work pretty well.
Very interesting hint. Why would you suggest to use a sentence-transformer from hugging face for creating the embeddings instead? Which method does a sentence-transformer from hugging face use to create the embeddings and which one does ollama use?
I tried llama2 and mistral embeddings through ollama embeddings api. The similarity search results were sht compared to Xenova/all-MiniLM-L6-v2 or gte-small
@@hackdonalds yes, llama for embeddings is not good, with sentence-transformers i mean that one you mentioned, then use llama to elaborate the sentence. There's another model better for embeddings but it will require more resources and i don't remember the name now.
Hey thanks for this (you save me some time) I want to use a hugging face model but I am using dotnet the model i want to ise is sentence-transformers/distiluse-base-multilingual-cased-v2, this model has many 539mb files in different extensions, I dont whant to use python i want to keep my project in c# so, how can i do this? Thank you for your time!
that's very interesting! I imagine you can build your local knowledge base in this way... I need to make one for code-snippets that store knowledge bits find around.... Is `d` dimension fixed by the model? does it mean I have 4096 bytes to store my embedding?
I tried this with LLAMA3 8b locally. It can work if the sentences are short enough. But when I started plugins in long paragraphs (youtube transcripts) it becomes basically useless. Transformers that are creating for embedding (BERT for example) seems to work better. They also have lower computational cost compared to LLAMA. I tried again with "nomic-embed-text-v1.f16.gguf" and it works much better.
i just love how diverse your videos are ! even when i don't particulary look for something your videos drives me to learn more. thanks ❤
Not bad, but if i may say any tip, i would say to use a sentence-transformer from hugging face for embeddings and then use a llama like llm only to customize the answer, because models created just for embeddings seems to be more accurate for this task. Also, langchain module is easier and useful than using numpy and requests, with lot more features. I use this setup most for RAG and seems to work pretty well.
Very interesting hint. Why would you suggest to use a sentence-transformer from hugging face for creating the embeddings instead?
Which method does a sentence-transformer from hugging face use to create the embeddings and which one does ollama use?
I tried llama2 and mistral embeddings through ollama embeddings api. The similarity search results were sht compared to Xenova/all-MiniLM-L6-v2 or gte-small
@@hackdonalds yes, llama for embeddings is not good, with sentence-transformers i mean that one you mentioned, then use llama to elaborate the sentence. There's another model better for embeddings but it will require more resources and i don't remember the name now.
@@rons96instructorembedding
Hey thanks for this (you save me some time) I want to use a hugging face model but I am using dotnet the model i want to ise is sentence-transformers/distiluse-base-multilingual-cased-v2, this model has many 539mb files in different extensions, I dont whant to use python i want to keep my project in c# so, how can i do this? Thank you for your time!
Spot on. Simplified presentation
thanks, bro... really helpful ❤
that's very interesting! I imagine you can build your local knowledge base in this way...
I need to make one for code-snippets that store knowledge bits find around....
Is `d` dimension fixed by the model? does it mean I have 4096 bytes to store my embedding?
Great video! But where comes faiss into play?
if you do it in Colab what url do you use?
I tried this with LLAMA3 8b locally. It can work if the sentences are short enough. But when I started plugins in long paragraphs (youtube transcripts) it becomes basically useless. Transformers that are creating for embedding (BERT for example) seems to work better. They also have lower computational cost compared to LLAMA. I tried again with "nomic-embed-text-v1.f16.gguf" and it works much better.
Powerful video! Guys, anyone knows how I can generate these embeddings if I were to deploy my app remotely?
Cool guide. Very well explained. +1
Great video! 😃
where is the source code
First comment ❤ and ur videos are fantastic
Brilliant!
🔥
comments[-1] #FirstComment😅