Whenever you release a vedio, honestly it narrows down the understanding of base case use of these LLM. Things I've been searching for the last years summarised in just 15min. Wow
If you read the NYUtron paper, they are using LLMs in healthcare to replace survival analysis, and other traditional ML models. Their LLM beats their boosted trees and other SOTA models. The thesis is that a lot of time-stamped healthcare data is interpreted and summarized by an expert into human language, which can be picked up and analyzed by a language model.
Hey Sam. Thank you so much for this video and I actually didnt know about the 4th Point which is the NLP Tasks. I was wondering could you do videos on how can we do that? For example, do you use langchain, llama index or how exactly? Thanks again!
Great video. You asked for engaging comments, so let me try. 1. To me, categories 1 and 3 are the same. It's à chatbot. With or without RAG... 2. I think that there is room for RAG apps, especially SQL based. It's not trivial, and customers won't upload their databases to chatgpt. So, for example, I've created a langchain app for my customer. On top of their DB. It does have value, and it's not trivial. 3. Automation and self learning systems, yeah, but again, it's only for big firms. And they won't take a freelancer for that in general.
Wow, excellent video. I am starting to piece things together. It would be great if you can make a video in which you can summarize the current approach for each of the five types of large language models that you mentioned, and what open source models are available for each type. Thank you so much!
I use LLM's for embedding things like planning in autonomous systems outside of the language domain. You can see some papers on using language as an augmentation to things like robotic helpers (lots of the intelligent agents stuff at deepmind). My own work uses language as grounding for instruction following and extraction of behaviours for AI characters. So maybe there's actually another app type that isn't directly an LLM app but uses it as a foundational model? I do enjoy coding the other types, but they're a bit cheesy :)
This ia a great summary. thanks Sam. Can you share more on the autonomous agents part? I'm assuming you don't just mean software tools that are already making ML decisions?
@sam , Very information video. Are there any github projects or tutorials you recommend to learn a moderately complex autonomous agent. I'm looking at use cases where the LLM App listens to chat conversations between multiple users of an ERP application and interjects autonomously if the LLM can carry out tasks that it can perform with-in the context of that conversation. For example , Rerunning an account reconciliation task, if the conversation between the two accountants has messages such as "I've updated the debit entries and ready for reconciliation" Specifically, how can i enable the LLM to continuously listen to the conversation and only jump in and perform the task if the task is relevant to the content of conversation and ignore all other conversations.
Don’t forget that a traditional ML company expanding into these GenAI apps are going to have an advantage over the start up. They already have customers, and they have a history long enough to suggest they will still exist a year from now.
Hi! thanks for your nice videos. I wanted to know that how we can evaluate the performance of LLM in terms of text generation like how factual they are how easily understandable they are and so on. If we do not have the reference text to calculate the BLEU score or ROUGE score, what could be the possible parameters or method to automatically evaluate or compare the performance of the LLM? This question is for other viewers as well. Please share some ideas.
wonderful and intriguing times, indeed. Before, in the field of AI, demand for hard skills were predominant, but now, I think, imagination and creativity are getting more and more important. Kinda reminds me the early 00’s when we thought that the ability to rotate and having millions of basically “unnecessary features” - were the purpose of cell phones. but then iPhone came up and literally changed everything. I believe the same thing is happening right now with the AI
I don't think the iPhone changed much, it was a continuation of progress from the PDA. What changed a lot was the advent of app stores and making phones general purpose devices.
I would suggest to find a new name for those very large models, since many inputs for autonomous agents might be vision or audio frequency and output can be anything like policy, action rather than text response.
yeah this was something someone asked me at a live talk last weekend. Do we still call these things LLMs as they become fully multi modal (including robotics etc). My guess is the name may change over time, technically (in a deep learning sense) they are all all Autoregressive Modals, but not sure many people would go for that or (lol) click on a video with that.
We often forget the real power of LLMs Which is text generation. It's a shame why there isn't much going on for automation in docs management, compliance or other mundane tasks..that LLMS can really take over by doing the "heavy lifting"
Great point. I do think this is where startups can do well by going super deep in a vertical and applying LLMs and Agents to a very focused niche. Too many startups are trying to do wrappers ala "Chat with PDF" rather than go deep on a industry and customize everything for that.
Thanks for your input, but I do not fully agree with you. Chat UI is just one type of app. It doesn't matter if it has RAG, plugins, web browsing, code interpreters, or agents. If it can control your app inside this app it's a second type. If the app is built on LLM features and was impossible before, that is the third type. I mean research agents, news apps, SEO apps, AI SDR apps, etc.
Наука
![Rich Dunk (Ft. DaBaby) - BIG DAWG [Official Video]](http://i.ytimg.com/vi/Pz2ewsSKAEM/mqdefault.jpg)
Whenever you release a vedio, honestly it narrows down the understanding of base case use of these LLM. Things I've been searching for the last years summarised in just 15min. Wow
If you read the NYUtron paper, they are using LLMs in healthcare to replace survival analysis, and other traditional ML models. Their LLM beats their boosted trees and other SOTA models.
The thesis is that a lot of time-stamped healthcare data is interpreted and summarized by an expert into human language, which can be picked up and analyzed by a language model.
very interesting, do you have a link to the paper?
Very good. Totally my impression as well. Building with an agent that assists customers and checks the logfiles to troubleshoot their issues.
This video made me turn on the notification bell. Excellent value of content and I would love to hear more on this kind of practical AI topic. 🙏🏻
Sam is a genius. Sam fine tune a Sam LLM so we can ask it LLM questions
Hey Sam. Thank you so much for this video and I actually didnt know about the 4th Point which is the NLP Tasks. I was wondering could you do videos on how can we do that? For example, do you use langchain, llama index or how exactly? Thanks again!
Great video. You asked for engaging comments, so let me try.
1. To me, categories 1 and 3 are the same. It's à chatbot. With or without RAG...
2. I think that there is room for RAG apps, especially SQL based. It's not trivial, and customers won't upload their databases to chatgpt. So, for example, I've created a langchain app for my customer. On top of their DB. It does have value, and it's not trivial.
3. Automation and self learning systems, yeah, but again, it's only for big firms. And they won't take a freelancer for that in general.
Thanks Sam for another great video! More autonomous agents please :)
Wow, excellent video. I am starting to piece things together. It would be great if you can make a video in which you can summarize the current approach for each of the five types of large language models that you mentioned, and what open source models are available for each type.
Thank you so much!
Great video, thanks @Sam.
This is categorization quite useful to see, how other applications gets clustered
I use LLM's for embedding things like planning in autonomous systems outside of the language domain. You can see some papers on using language as an augmentation to things like robotic helpers (lots of the intelligent agents stuff at deepmind). My own work uses language as grounding for instruction following and extraction of behaviours for AI characters. So maybe there's actually another app type that isn't directly an LLM app but uses it as a foundational model? I do enjoy coding the other types, but they're a bit cheesy :)
This ia a great summary. thanks Sam. Can you share more on the autonomous agents part? I'm assuming you don't just mean software tools that are already making ML decisions?
What a great video!!
Great video - thank you:)
Cool breakdown
@sam , Very information video. Are there any github projects or tutorials you recommend to learn a moderately complex autonomous agent.
I'm looking at use cases where the LLM App listens to chat conversations between multiple users of an ERP application and interjects autonomously if the LLM can carry out tasks that it can perform with-in the context of that conversation. For example , Rerunning an account reconciliation task, if the conversation between the two accountants has messages such as "I've updated the debit entries and ready for reconciliation"
Specifically, how can i enable the LLM to continuously listen to the conversation and only jump in and perform the task if the task is relevant to the content of conversation and ignore all other conversations.
Don’t forget that a traditional ML company expanding into these GenAI apps are going to have an advantage over the start up. They already have customers, and they have a history long enough to suggest they will still exist a year from now.
Do create a RUclips playlist dedicated to Autonomous Agents. I prefer to focus on that area. Thank you, Sam
Same!
Hi! thanks for your nice videos. I wanted to know that how we can evaluate the performance of LLM in terms of text generation like how factual they are how easily understandable they are and so on. If we do not have the reference text to calculate the BLEU score or ROUGE score, what could be the possible parameters or method to automatically evaluate or compare the performance of the LLM? This question is for other viewers as well. Please share some ideas.
What are the prerequisites to learning how to build autonomous agents
Would appreciate videos on autonomous Open Source agents
Hi @Sam .i want help that Is langchain or pretrained llama2 will help to make conversational chatbot on large dataset?
which python concept including data science is important for llms langchain
Sam are you going to do an seminars? I am based in Cambodia and do not have any one to bounce ideas off of.
I already do some courses in Singapore for Deep Learning and NLP etc. I am thinking of doing a LLM Apps course or something online here as well.
wonderful and intriguing times, indeed. Before, in the field of AI, demand for hard skills were predominant, but now, I think, imagination and creativity are getting more and more important.
Kinda reminds me the early 00’s when we thought that the ability to rotate and having millions of basically “unnecessary features” - were the purpose of cell phones.
but then iPhone came up and literally changed everything. I believe the same thing is happening right now with the AI
I don't think the iPhone changed much, it was a continuation of progress from the PDA. What changed a lot was the advent of app stores and making phones general purpose devices.
Great video, would be very helpful to add timestamps next time
I would suggest to find a new name for those very large models, since many inputs for autonomous agents might be vision or audio frequency and output can be anything like policy, action rather than text response.
yeah this was something someone asked me at a live talk last weekend. Do we still call these things LLMs as they become fully multi modal (including robotics etc). My guess is the name may change over time, technically (in a deep learning sense) they are all all Autoregressive Modals, but not sure many people would go for that or (lol) click on a video with that.
I'm trying to build a simple chatbot that have memory, but it's a bit difficult.
We often forget the real power of LLMs
Which is text generation.
It's a shame why there isn't much going on for automation in docs management, compliance or other mundane tasks..that LLMS can really take over by doing the "heavy lifting"
Great point. I do think this is where startups can do well by going super deep in a vertical and applying LLMs and Agents to a very focused niche. Too many startups are trying to do wrappers ala "Chat with PDF" rather than go deep on a industry and customize everything for that.
@@samwitteveenai yes, and remember NFTs? They had great potential, in tiketing or IP or credentialing.. but ended up being .. well... 😮💨
👍
Ohhh... I want "chat with SQL" real bad.
Thanks for your input, but I do not fully agree with you. Chat UI is just one type of app. It doesn't matter if it has RAG, plugins, web browsing, code interpreters, or agents. If it can control your app inside this app it's a second type. If the app is built on LLM features and was impossible before, that is the third type. I mean research agents, news apps, SEO apps, AI SDR apps, etc.