Not bad, but if i may say any tip, i would say to use a sentence-transformer from hugging face for embeddings and then use a llama like llm only to customize the answer, because models created just for embeddings seems to be more accurate for this task. Also, langchain module is easier and useful than using numpy and requests, with lot more features. I use this setup most for RAG and seems to work pretty well.
Very interesting hint. Why would you suggest to use a sentence-transformer from hugging face for creating the embeddings instead? Which method does a sentence-transformer from hugging face use to create the embeddings and which one does ollama use?
I tried llama2 and mistral embeddings through ollama embeddings api. The similarity search results were sht compared to Xenova/all-MiniLM-L6-v2 or gte-small
@@hackdonalds yes, llama for embeddings is not good, with sentence-transformers i mean that one you mentioned, then use llama to elaborate the sentence. There's another model better for embeddings but it will require more resources and i don't remember the name now.
I tried this with LLAMA3 8b locally. It can work if the sentences are short enough. But when I started plugins in long paragraphs (youtube transcripts) it becomes basically useless. Transformers that are creating for embedding (BERT for example) seems to work better. They also have lower computational cost compared to LLAMA. I tried again with "nomic-embed-text-v1.f16.gguf" and it works much better.
that's very interesting! I imagine you can build your local knowledge base in this way... I need to make one for code-snippets that store knowledge bits find around.... Is `d` dimension fixed by the model? does it mean I have 4096 bytes to store my embedding?
i just love how diverse your videos are ! even when i don't particulary look for something your videos drives me to learn more. thanks ❤
Not bad, but if i may say any tip, i would say to use a sentence-transformer from hugging face for embeddings and then use a llama like llm only to customize the answer, because models created just for embeddings seems to be more accurate for this task. Also, langchain module is easier and useful than using numpy and requests, with lot more features. I use this setup most for RAG and seems to work pretty well.
Very interesting hint. Why would you suggest to use a sentence-transformer from hugging face for creating the embeddings instead?
Which method does a sentence-transformer from hugging face use to create the embeddings and which one does ollama use?
I tried llama2 and mistral embeddings through ollama embeddings api. The similarity search results were sht compared to Xenova/all-MiniLM-L6-v2 or gte-small
@@hackdonalds yes, llama for embeddings is not good, with sentence-transformers i mean that one you mentioned, then use llama to elaborate the sentence. There's another model better for embeddings but it will require more resources and i don't remember the name now.
@@rons96instructorembedding
Spot on. Simplified presentation
Powerful video! Guys, anyone knows how I can generate these embeddings if I were to deploy my app remotely?
Cool guide. Very well explained. +1
Great video! But where comes faiss into play?
thanks, bro... really helpful ❤
I tried this with LLAMA3 8b locally. It can work if the sentences are short enough. But when I started plugins in long paragraphs (youtube transcripts) it becomes basically useless. Transformers that are creating for embedding (BERT for example) seems to work better. They also have lower computational cost compared to LLAMA. I tried again with "nomic-embed-text-v1.f16.gguf" and it works much better.
that's very interesting! I imagine you can build your local knowledge base in this way...
I need to make one for code-snippets that store knowledge bits find around....
Is `d` dimension fixed by the model? does it mean I have 4096 bytes to store my embedding?
Brilliant!
Great video! 😃
🔥
First comment ❤ and ur videos are fantastic
where is the source code
if you do it in Colab what url do you use?
comments[-1] #FirstComment😅