These videos are amazing! You are an absolutely brilliant teacher. I have just made the viewing of this video mandatory for my entire team. The ability to explain complex topics in an understandable way is a sign of genius - thank you.
Edit: My father-in-law had a heart attack this morning. Fortunately he is going to be fine, but the universe is opposed to me being on schedule this week. My apologies all. Hey all, the LoRA training video is almost done! I decided to add some extra material on how we to prepare and preprocess datasets, since the video didn’t feel super cohesive without it. I figured you all would prefer quality over rushing, so I’ll have it out by the morning.
Japanese is read left to right, unless you are talking about vertical text (in that case is top to bottom, right to left), but most Japanese text online, which is what an llm would be trained on, would be regular left to right.
Great video! Are sentence embeddings simply constructed from aggregating word embeddings and applying some operation, such as mean pooling or max pooling?
I was thinking to Learn about LoRA for the past several days but you just rocked it. it is so simple. Thanks alot and i really appreciate your videos. One more thing i am currently doing my PhD and the topic of my PhD is Vision-language Pre-trained models. However my main problem until now is that i have small resources at most i can get two 3090 GPUs. I would appreciate any useful suggestion and help regarding Pre-training The large Vision-language Pre-trained models with such resources. I would also like to have any colloborations in this regard. Thank you so much again
ViT’s use a ton of resources for training, though you do have some options. 1. If you have some funds, you could use Lambda Labs which lets you rent an A100 for as little as $1.10/hr. I use them for work and they have been great. 2. You could always try 8/4-bit quantizing them, GPTQ for LLaMA has a great example of how to implement the algorithm. 1. There is the Hyena paper which shows some ways to diagonalize the attention layer, but there is no algorithm for it yet. My discord is Aemon Algiz#0033 if you’d like to chat.
So, it looks like it is English, though this one is multi-lingual: huggingface.co/sentence-transformers/stsb-xlm-r-multilingual I would personally test instructor-xl and see if works for multi-lingual
I’m glad I discovered this valuable resource. Your simple, straightforward explanations are very helpful. One suggestion: The thin white scribe pen you use is difficult to follow at times, so if you could supplement it with a surrounding halo or some other highlight those of us with slight visual impairment would appreciate it. 🫡
Man I don't know anyone who explains things in a clear, technical, and straightforward way. This is great stuff.
Thank you! I try my best to make the content digestible
These videos are amazing! You are an absolutely brilliant teacher. I have just made the viewing of this video mandatory for my entire team. The ability to explain complex topics in an understandable way is a sign of genius - thank you.
Thank you! Hopefully the training corpus one is as helpful
Edit: My father-in-law had a heart attack this morning. Fortunately he is going to be fine, but the universe is opposed to me being on schedule this week. My apologies all.
Hey all, the LoRA training video is almost done! I decided to add some extra material on how we to prepare and preprocess datasets, since the video didn’t feel super cohesive without it. I figured you all would prefer quality over rushing, so I’ll have it out by the morning.
This one was easy to follow and understand. Thank you for the great tutorial!
Thank you!
Wow! Love your detailed explanations and code examples. Subscribed ❤
Thank you, I’m glad it was helpful!
Yet another easy to understand video. Thank you for the time and effort you put into these!
Thanks, it means a lot!
Japanese is read left to right, unless you are talking about vertical text (in that case is top to bottom, right to left), but most Japanese text online, which is what an llm would be trained on, would be regular left to right.
I was thinking about tategaki, I should have been more specific. Thanks for the correction!
Great video!
Are sentence embeddings simply constructed from aggregating word embeddings and applying some operation, such as mean pooling or max pooling?
Indeed, typically you can capture the output from attention and use that as an embedding
I was thinking to Learn about LoRA for the past several days but you just rocked it. it is so simple. Thanks alot and i really appreciate your videos.
One more thing i am currently doing my PhD and the topic of my PhD is Vision-language Pre-trained models. However my main problem until now is that i have small resources at most i can get two 3090 GPUs. I would appreciate any useful suggestion and help regarding Pre-training The large Vision-language Pre-trained models with such resources. I would also like to have any colloborations in this regard. Thank you so much again
ViT’s use a ton of resources for training, though you do have some options.
1. If you have some funds, you could use Lambda Labs which lets you rent an A100 for as little as $1.10/hr. I use them for work and they have been great.
2. You could always try 8/4-bit quantizing them, GPTQ for LLaMA has a great example of how to implement the algorithm.
1. There is the Hyena paper which shows some ways to diagonalize the attention layer, but there is no algorithm for it yet.
My discord is Aemon Algiz#0033 if you’d like to chat.
@@AemonAlgiz Thank you i will explore each of these options. especially 4 bit quantization GPTQ for LLaMA.
Such a good explanation! by the way, is instructor-xl multilingual?
You know, I’m honestly not sure, though I suspect it’s primarily English. There are multilingual embedding models, though. I’ll check when I get home!
@@AemonAlgiz thank you
So, it looks like it is English, though this one is multi-lingual:
huggingface.co/sentence-transformers/stsb-xlm-r-multilingual
I would personally test instructor-xl and see if works for multi-lingual
this is exactly where my research is lacking.
I’m glad it was helpful!
Hey thanks for the great video but the IDE font is a little bit small.
the code part is fine but the project file names and debugger infomation are hard to read
hey mate your audio and video are not in sync, mabe be out by a second ,
Huh, I wonder why that’s happened. It seemed fine before I uploaded it.
I’m glad I discovered this valuable resource. Your simple, straightforward explanations are very helpful. One suggestion: The thin white scribe pen you use is difficult to follow at times, so if you could supplement it with a surrounding halo or some other highlight those of us with slight visual impairment would appreciate it. 🫡
I can do that! Do you have an example of what would be best for you?