👉More on LLMs: ruclips.net/p/PLz-ep5RbHosU2hnz5ejezwaYpdMutMVB0 -- References [1] ruclips.net/video/A8HEPBdKVMA/видео.htmlsi=PA4kCnfgd3nx24LR [2] R. Patil, S. Boit, V. Gudivada and J. Nandigam, “A Survey of Text Representation and Embedding Techniques in NLP,” in IEEE Access, vol. 11, pp. 36120-36146, 2023, doi: 10.1109/ACCESS.2023.3266377. [3] owasp.org/www-project-top-10-for-large-language-model-applications/
Love that you’re bringing real knowledge, insights and code here! So many AI RUclipsrs are just clickbaiting their way through the hype cycle by reading the same SHOCKING news as everyone else.
I mean, the guy clickbaited the thumbnail. Also, this is insanely old news at this point(if considered news at all). Video content was on point, but we shouldn't be promoting clickbait methods.
I clicked this video for technical explanations and code, not news. There are plenty of dudes reading off the same SHOCKING news across AI RUclips. I got exactly what I wanted from this video and feel like the title was clear.
I have learnt so much by watching the entire series. Thank you so much Shaw! I think this is one of the best playlists out there for anyone looking to get into the field of LLMs and GenAI.
I love how you give a low level lesson. It helps me understand more in the topic and also see more potential on applying in another area. Long story short, you got a new subscriber. I will consume all your knowledge and make the best out of it.
Great video. The practical use cases for embeddings themselves are undervalued IMHO and this video is fantastic for showing ways to use embeddings. Even if you use OpenAI embeddings, they are dirt cheap, and can provide fantastic vectors for further analysis, manipulation, and comparison.
Awesome teaching quality. Simple start into text embeddings for begineers. But it would be better to use a Open Source LLM to create embeddings. OpenAI api is for paid.
Hi Shaw!!! Really great stuff... I am loving this series!!! I echo with everyone else agreeing that your videos are super informative and hands on!!! Very Very Useful!!! Many thanks man!
Thanks for sharing your insight! Indeed agents and embeddings solve different problems. However, some agent use cases could be reconfigured to be solved with text embeddings + human in the loop.
I have watched most of the videos in this series and found them really helpful. Something I am looking for that I haven't seen you cover yet. Is some more guidance on preparing data for either RAG or fine tuning. I am sure you have practical tips you can give. I have a large old codebase, we have loads of documentation and tutorials etc, but it is a lot of someone to pickup. This new world of GPTs seams perfect for building an assistant. I will be able to work through it ok, but I suspect there will be a load of learnt best practices or pitfalls to avoid that are a bit more subtle. For example I am looking through our support emails / tickets, lots of them all start with please send logs :) and after a load of back and forth we have info. This is much like a conversation with ChatGPT. For fine tuning is it best to fine tune on a whole thread? Or each chunk of the conversation?
Great suggestion! I plan to do a series on data engineering and this would be a great thing to incorporate into it. For you use case, the best choice would depend on what you want the assistant to do. For instance, if you want the assistant to mimic the support rep, then you'd likely want to use each message in the thread with its appropriate context (i.e. preceding messages).
@@ShawhinTalebi thanks for the tip. That’s what I ended up doing it. Not yet tried actually fine tuning yet. Just finished my data prep. Looking forward to you next series 😊
Hey Shaw thanks for this wonderful series. I have completed it and learned so many new things but one thing I felt is that the code is very high level and it feels like to me that I have to remember most of the things during coding while practicing with those hugging face models. Do you have any suggestions for that?
Thank you for your useful video. I have a question regarding openai embedding model. Does openai fine-tune its model if we use these (embedding) models? ... My data is private and I don't want to expose it. Thanks
Hi Shawhin, Thanks. I ran into a problem. I tried to use Sentence_transformers model by installing it. It always givens an error no file found config_sentence_transformers.json' in the .cache/huggingface/... folder. Your help is appreciated
Many thanks for the video Shaw, great content! One simple question: when using OpenAI's embedding model, each resume is represented by an embedding vector. Is this embedding computed as the average of all word vectors?
Great question! Embedding models do not operate on specific words, but rather on the text as a whole. This is valuable because the meaning of specific words is driven by the context it appears in.
it's possible to extract software names from the query with a text classifier and apply only e. g. apache airflow to kw search? also what db do you suggest? is postgres with vector db good?
Good question. While I haven't seen a text classifier used for KW search, that could be a clever way to implement it. There are several DBs to choose from these days. I'd say go with what makes sense with the existing data infrastructure. If starting from scratch, Elastic search or Pinecone might be good jumping off points.
LDA - Latent Dirichlet Allocation is kinda trivial these days… Matlab text analytics toolbox works great on pdf’s with bi-grams… a la bag-of-N-Grams. Cool… thanks…
It's not very good practice to compare ai assistants and text embeddings since ai assistants are also working with tokens ( which are the embeddings for each text chunk) .
This is not always a good solution if we have multilingual text. I see that LLM context 1M token/character They offer other solutions with functions and external API calls.
I'm curious about this. I've seen embedding models that can handle multiple languages, so I'd expect them to work pretty well. Can you shed any more light on this?
Good question! These are different things. Tokenization is the process of taking a some text and deriving a vocabulary from which the original text can be generated, where each element in the vocabulary is assigned a unique integer value. Text embeddings on the other hand, take tokens and translate them into meaningful (numerical) representations. I talk a little more about tokenization here: ruclips.net/video/czvVibB2lRA/видео.htmlsi=FwqmkB9Ltyq45n0w&t=348
👉More on LLMs: ruclips.net/p/PLz-ep5RbHosU2hnz5ejezwaYpdMutMVB0
--
References
[1] ruclips.net/video/A8HEPBdKVMA/видео.htmlsi=PA4kCnfgd3nx24LR
[2] R. Patil, S. Boit, V. Gudivada and J. Nandigam, “A Survey of Text Representation and Embedding Techniques in NLP,” in IEEE Access, vol. 11, pp. 36120-36146, 2023, doi: 10.1109/ACCESS.2023.3266377.
[3] owasp.org/www-project-top-10-for-large-language-model-applications/
Love that you’re bringing real knowledge, insights and code here! So many AI RUclipsrs are just clickbaiting their way through the hype cycle by reading the same SHOCKING news as everyone else.
I mean, the guy clickbaited the thumbnail. Also, this is insanely old news at this point(if considered news at all).
Video content was on point, but we shouldn't be promoting clickbait methods.
I clicked this video for technical explanations and code, not news. There are plenty of dudes reading off the same SHOCKING news across AI RUclips. I got exactly what I wanted from this video and feel like the title was clear.
I have learnt so much by watching the entire series. Thank you so much Shaw! I think this is one of the best playlists out there for anyone looking to get into the field of LLMs and GenAI.
Great to hear! Feel free to share any suggestions for future content :)
I love how you give a low level lesson. It helps me understand more in the topic and also see more potential on applying in another area. Long story short, you got a new subscriber. I will consume all your knowledge and make the best out of it.
Thanks for subscribing :) Glad it was helpful!
Great video. The practical use cases for embeddings themselves are undervalued IMHO and this video is fantastic for showing ways to use embeddings. Even if you use OpenAI embeddings, they are dirt cheap, and can provide fantastic vectors for further analysis, manipulation, and comparison.
Thanks Brandon! I completely agree. Agents are great, but they seem to overshadow all the relatively simple text embedding-based applications.
Exceptionally clear as always!
Thanks Aldo :)
You are the real guy to subscribe and learn
SEO here, enjoyed your examples of semantic search and explanation of hybrid search. Great vid and easy to follow. Will explore your channel. Cheers!
Clear and understandable explanation of these concepts. Thanks and really enjoyed!
Congrats man! Keep going with more real examples with code sharing
Awesome teaching quality. Simple start into text embeddings for begineers. But it would be better to use a Open Source LLM to create embeddings. OpenAI api is for paid.
Thanks :)
I use open source embeddings in my latest video: ruclips.net/video/3JsgtpX_rpU/видео.htmlsi=ricuwaoSJSYnSAQM&t=843
Hi Shaw!!! Really great stuff... I am loving this series!!! I echo with everyone else agreeing that your videos are super informative and hands on!!! Very Very Useful!!! Many thanks man!
Glad you like them!
great video! super clear and on point. Thanks Shaw!
Excellent work sir! ❤
Thanks, useful material 👍
Iv been using embeddings for awhile but i find that agents can call specialized tools that can be very useful depending on the applications.
Thanks for sharing your insight! Indeed agents and embeddings solve different problems. However, some agent use cases could be reconfigured to be solved with text embeddings + human in the loop.
Wow! Thank you for breaking this down, been trying to figure it out!
Glad to help!
Thank you very good information, will try to make a database for audio sound effects using vector databases text to audio
I have watched most of the videos in this series and found them really helpful. Something I am looking for that I haven't seen you cover yet. Is some more guidance on preparing data for either RAG or fine tuning. I am sure you have practical tips you can give. I have a large old codebase, we have loads of documentation and tutorials etc, but it is a lot of someone to pickup. This new world of GPTs seams perfect for building an assistant. I will be able to work through it ok, but I suspect there will be a load of learnt best practices or pitfalls to avoid that are a bit more subtle. For example I am looking through our support emails / tickets, lots of them all start with please send logs :) and after a load of back and forth we have info. This is much like a conversation with ChatGPT. For fine tuning is it best to fine tune on a whole thread? Or each chunk of the conversation?
Great suggestion! I plan to do a series on data engineering and this would be a great thing to incorporate into it.
For you use case, the best choice would depend on what you want the assistant to do. For instance, if you want the assistant to mimic the support rep, then you'd likely want to use each message in the thread with its appropriate context (i.e. preceding messages).
@@ShawhinTalebi thanks for the tip. That’s what I ended up doing it. Not yet tried actually fine tuning yet. Just finished my data prep. Looking forward to you next series 😊
Hey Shaw thanks for this wonderful series. I have completed it and learned so many new things but one thing I felt is that the code is very high level and it feels like to me that I have to remember most of the things during coding while practicing with those hugging face models. Do you have any suggestions for that?
I think the best way to solidify your understanding is to apply it to real-world use cases.
Great info
Thank you for your useful video. I have a question regarding openai embedding model. Does openai fine-tune its model if we use these (embedding) models? ... My data is private and I don't want to expose it. Thanks
OpenAI's privacy policy says they do not train on API data: openai.com/enterprise-privacy/
Love you shaw!
❤️
Great format subd
Hi Shawhin, Thanks. I ran into a problem. I tried to use Sentence_transformers model by installing it. It always givens an error no file found config_sentence_transformers.json' in the .cache/huggingface/... folder. Your help is appreciated
Not sure what the issue could be. Did you install all the requirements on the GitHub?
github.com/ShawhinT/RUclips-Blog/tree/main/LLMs/text-embeddings
Excellent!
Many thanks for the video Shaw, great content!
One simple question: when using OpenAI's embedding model, each resume is represented by an embedding vector. Is this embedding computed as the average of all word vectors?
Great question! Embedding models do not operate on specific words, but rather on the text as a whole. This is valuable because the meaning of specific words is driven by the context it appears in.
finally someone who speaks with their hands more than I do, lol...
😂😂.. 👋 👍
I call him the hand whisperer.... but really loud.
Can only two kinds of classification be made? If I have lots of types, for example, product classification, can it be applied?
You can have several target classes. Here's a nice write-up about doing that with sklearn: scikit-learn.org/stable/modules/multiclass.html
it's possible to extract software names from the query with a text classifier and apply only e. g. apache airflow to kw search? also what db do you suggest? is postgres with vector db good?
Good question. While I haven't seen a text classifier used for KW search, that could be a clever way to implement it.
There are several DBs to choose from these days. I'd say go with what makes sense with the existing data infrastructure. If starting from scratch, Elastic search or Pinecone might be good jumping off points.
lanceDB is also quite good.
LDA - Latent Dirichlet Allocation is kinda trivial these days… Matlab text analytics toolbox works great on pdf’s with bi-grams… a la bag-of-N-Grams. Cool… thanks…
Can i use these embeddings with bert based models instead?
Yes! In fact, sentence transformers has a few bert-based embedding models: sbert.net/docs/pretrained_models.html
It's not very good practice to compare ai assistants and text embeddings since ai assistants are also working with tokens ( which are the embeddings for each text chunk) .
This is not always a good solution if we have multilingual text. I see that LLM context 1M token/character They offer other solutions with functions and external API calls.
I'm curious about this. I've seen embedding models that can handle multiple languages, so I'd expect them to work pretty well. Can you shed any more light on this?
✌✌🧑💻🧑💻
Soo, unlike your thumbnail, this has nothing to do with agents...
Why mention them?
Thumbnail is "Forget AI agents... use this instead". I explain this a bit @3:15.
By end of next year all the drawbacks of LLMs will be erased.
I hope so!
are you persian
Yes :)
cool...
is text embedding same as text tokenization done in training ?
Good question! These are different things.
Tokenization is the process of taking a some text and deriving a vocabulary from which the original text can be generated, where each element in the vocabulary is assigned a unique integer value.
Text embeddings on the other hand, take tokens and translate them into meaningful (numerical) representations.
I talk a little more about tokenization here: ruclips.net/video/czvVibB2lRA/видео.htmlsi=FwqmkB9Ltyq45n0w&t=348