Видео

It’s not being lazy, it’s called being smarter. A regular Uber driver makes $10 after all the deductions but a Waymo software engineer who drives the cars from afar makes $162,000 just at the base level.

so safety is lazy now

Thanks!

Ah yes, that’s a good idea: to have an LLM provide the response, but give it access to fine-tuned models to leverage each of their strengths Yes humans also have biases. Then when the models are trained using techniques such as RLHF, then those biases can be imprinted into the model. The filters that are placed on the models may not be the right filters. In the open-source space, luckily there are a lot of models, so a different model can be used if the filter isn’t helpful

That sounds like a natural language understanding task, so a fine-tuned model would likely perform the best. It’s a task where the input is a sequence and the output is a prediction, which means that an encoder-type model would be preferred. So then it would be a matter of curating a dataset, deciding on a model (ideally a model pre-trained on similar data), and then fine-tuning the model

For sure! Yea it’s an important topic

Hmm 🤔 I haven’t done any work with the TPUs, so I’m not sure on this one

No I do t have any videos on that. Pre-training can be a lot more resource intensive and require more gpu vram. With the parameter efficient techniques, fine-tuning can now be done on a single gpu. But when it comes to pre-training from scratch, that would require substantial compute. For example, MPT-7B required 440 GPUs trained over the course of 9.5 days

May be a discussion on what issues arise with scaling. Like parallelism.

@@user-wr4yl7tx3w The parallelism is an interesting point. I remember reading the research papers as these transformers were growing. Initially, the entire model could fit on a single gpu. But then the models grew too large to fit on a single gpu, so they had to find a way to split a model onto multiple gpus by putting a layer onto each gpu (or some other splitting method) Then the models kept growing and got to the size that a single layer couldn't fit on a single gpu. So that required further advancements such that a part of a layer could fit on each gpu. This all ties back to the hardware problem of ai - the fact that model size has increased by 1000x+ while gpu vram size has only increased by ~3x. I was thinking about creating a video on that. It might be an interesting video

You’re welcome!

For sure!

I don’t think any models compare with chatgpt and gpt4 when it comes to coding. Not even Google Bard / PaLM 2

No problem!

That's going to also depend on factors such as model size and context length as well. For a smaller model, the fine-tuning will be much faster. Last night I was fine-tuning a 5 Billion parameter model on a sequence-to-sequence dataset of ~50,000 rows, and it was looking like it would take 36 hours on my RTX 4090. But this is going to depend a lot. Ideally, it would be good to find ways to fine-tune the models so that they can be done in less time. The QLoRA paper says: "Using QLORA, we train the Guanaco family of models, with the second best model reaching 97.8% of the performance level of ChatGPT on the Vicuna [10] benchmark, while being trainable in less than 12 hours on a single consumer GPU; using a single professional GPU over 24 hours we achieve 99.3% with our largest model, essentially closing the gap to ChatGPT on the Vicuna benchmark." So I certainly think it is possible to fine-tune a model and get great performance in 12-24 hours.

Yea for sure! I'm excited to do some fine-tuning with Falcon 40B

Yea it’s definitely slow. It’s running on the cpu, which is going to cause a massive performance hit. If there was a way to connect it up to a gpu, the speed would be much better. Glad you got it working!

Fine-tuning and using a vector store are two distinct methods for incorporating domain-specific knowledge into an AI model. There are advantages for each. Fine-tuning involves adjusting the parameters of an already trained model using a smaller, specific dataset. This allows the model to specialize in a particular domain or task. Fine-tuning can provide excellent results but it may require a sizable dataset of the specific knowledge domain. Fine-tuning can also be used to train the model to respond in a particular way, which would be important in a business context because the business may want the LLM to respond in a particular way that aligns with the brand. Fine-tuning can also be used to create filters so that the LLM doesn't say anything that it isn't supposed to do. There are other ways of creating a filter though, such as RLHF. A vector store can allow the model to retrieve information that it might not have had access to during training. It's a more flexible way to add new information after training, but the AI must still be capable of utilizing that information effectively. The retrieval of information from the vector store may not be as smooth or contextual as with a fine-tuned model. In terms of achieving better results, it depends on the specifics of the task or application. For many real-world AI applications, a combination of fine-tuning and a vector store might be used. Fine-tuning can help the model understand and generate language more effectively in a specific context, while a vector store can provide the model with access to a large, up-to-date knowledge base.

well there's multiple different considerations that would go with that. For example, what kind of task are you looking to do in the other language? Is it just next-token prediction? Or are you looking to do something like sequence-to-sequence translation, where you give it input in one language and then it translates into another language? In a case like that, you would set up your training pipeline slightly differently from the way that it is done in the video. This would be done by designing a dataset that has an input and output, and then maybe creating a pipeline using AutoModelForSeq2SeqLM. The amount of data required would also depend on the task and also on the size of the model. Some of the most powerful LLMs have an emergent capability where it has already learned additional languages that it was not explicitly meant to learn. Additionally, there are many LLMs available on the HuggingFace hub that already are trained to work with multiple languages. So, depending on your use case, you can pick the correct model accordingly. The larger models are much more powerful. Usually I try to use the largest LLM possible since that will get the best response. I notice a huge jump in performance as I move from using a 3B model to a 7B and up to 30B+. To answer your question about how much text would be required, this can vary, but on the OpenAi website, they say "The more training examples you have, the better. We recommend having at least a couple hundred examples. In general, we've found that each doubling of the dataset size leads to a linear increase in model quality." But I think if you were teaching an LLM that had no prior exposure to a new language, it would likely take much more than a few hundred examples

@@lukemonington1829 thanks for the elaborate response. Actually the language I want to train it on (it's Persian btw) is already generated by some of the free language models however they don't make a single error free sentence at the moment and even GPT-4 makes many mistakes writing it. As my country is lacking teachers I was thinking about training it to explain educational material in a back and forth fashion where the students can ask anything. So it will be similar to training an assistant I think. The fact that even any kind of training can be done just by a few hundred examples is a great news to me. I was thinking about much higher numbers as I am very new to all this. Thanks

@@sinayagubi8805 I'm going to be posting my next video (hopefully tomorrow) where I'll touch on this point a little bit. The video will be a practical guide to using LLMs. This falls into the machine translation task of text generation. It turns out that for general machine translation tasks, using a regular LLM (without fine-tuning) is better. But when it comes to low-resource machine translation, where there is a limited amount of parallel corpora available, fine-tuning the model can lead to a large increase in performance

What about using a vector db like chrome db for dataset is that pos in this case?

@@lukemonington1829 cool, eagerly waiting for it.

I thought the same thing when I read the research paper and they were implying that 48gb is consumer grade. When I wrote this article, I kept that idea in. I can't speak for them on why they consider 48gb to be consumer grade. But if I were to hypothesize, I think the reason why they consider 48gb to be consumer grade is that a professional gpu with 48gb of vram can be rented online for less than $1 per hour. In the paper, they didn't do any fine-tuning runs for longer than 24 hours, which means that it would cost at most $24 per fine-tuning run. This is cheap enough that a normal person can do their own fine-tuning runs if they wanted to

@@lukemonington1829 Its amazing a single gpu with 48gb can train any LLM but consumer grade is pushing things to far :)

@@rolyantrauts2304 yea, I agree. At the 48gb size, that means that it couldn't be done locally anymore, so that feels like it should be another tier higher

I guess if you have unfied memory as in a Mac then maybe possible?

@@rolyantrauts2304 yes it definitely could be done by splitting the model between the gpu and the cpu. This can also be done in Windows / Linux. It's even possible to split between the gpu, cpu, and harddrive for extremely large models. Through a process like that, inferencing can can be done with even the largest of models on a consumer level computer. But splitting between devices dramatically reduces the speed that training and inferencing can be done. Personally, if I can't load the full model onto the gpu, then it's not practical for me to use since it is too slow. I think training would be impractical too, it would take a lot longer. But it would be possible if someone was determined to do that lol

It does appear that the big companies will be able to use regulations in order to build a moat so that startups and small companies won’t be able to compete without huge amounts of funding

I'm not quite sure what you mean

I appreciate the suggestions! Yea I think that would be good. I'm looking into preparing and cleaning my own datasets too. I might make a video on that! And yes, it looks like it doesn't work with all models. So far, I've tested it with pygmalian, wizardLM. I'll be testing with others in the future. I tried llama as well and got errors as well. Also, I was having trouble initially getting this running locally, so I ended up creating a Dockerfile and docker-compose, and that solved my problem. Let me know if it would be helpful for me to post the code on github for that!

thinking about it a little more, it should definitely work with the llama model, since the Guanaco model family was fine tuned based off of the llama model. There must be a way to do it

@@lukemonington1829 Thank you