⭐ Just some clarifications in case I wasn't clear enough in the video :) Chat With RTX does *not* run ChatGPT locally, it uses *the same type of* AI model - a GPT LLM. The training dataset is different and so are the responses.
I used it with RTX3080 + 64GB RAM, to be honest it seems ok when you use it with PDF text documents and ask it questions on the text content. It doesn't seem quite as good for the you tube video transcripts I've tried it with, but I suppose that's a bit hit and miss. I don't know how good it would be with numerical data sets
I tried it with a pdf of frankenstein, 20000leagues under the sea, and snowcrash, a few hundred pages. I tried them all together and then separately, obviously it works better sepearatly. These took less than 5 minutes to process and you could ask questions about the text, but it completely loses context after one or two questions if you try to follow up on the last conversation. I tried it with some french homework, and it could do some translation and inference but struggles with how text is formatted on a page.The youtube thing probably depends on how good the transcript is@@mariogross
I don't have the option for youtube URL and no matter how much I search about it I cannot find anyone talking about it, is it only me who has this problem?
After watching your video I tried this "Chat with RTX" demo, not too impressed with it. It runs perfect on a 14900K with 48Gb of ram and a RTX 4090. Unfortunately it did not impress me much with its responses. It seems Llama 2 and Mistral 7B were educated by car mechanics. Every single question about cars and their engine specifications gave me correct answers but asking it the specifications of an Intel I9 14900K were all wrong. It said it had a total of 20 threads and 10 cores, it also said its turbo boost was 5Ghz and a TDP of 95w. With UHD 630 graphics, why it would know more about cars than CPU's makes me wonder. Will continue to play with it but, I can't trust its answers 100%.
Oh I've found plenty of inaccuracies with it - as I have with all other chatbots, including ChatGPT ;) It once told me I should heat my oven to 60 degrees to bake my pizza... ☃️ All (current) AI LLMs suffer from false positives and *any* information you get out of them should be double checked if you're planning to rely on it in any sort of meaningful way. They're great for ideas, rephrasing or summarising text, writing content outlines and much much more, but asking specific technical questions is likely going to give you some inaccuracies... ChatGPT does a better job of this mostly because of the much larger dataset it has consumed for information :)
Does it retain the information after you've fed it the data once? I'm thinking about making a model for a specific software and just feeding it a ton if info. Thx
I don't think there's much 'teaching' involved. You point it at a folder / YT video or playlist and it just consumes the info into the model. But it's working pretty well for a tech demo
Everything you feed it from your hard drive folders is kept locally on your computer? I know you already answered that..am asking just in case there was any concerning updates since your upload.
Nothing I've heard changes my understanding of how this works. Everything runs locally on your graphics card. No data is shared or uploaded anywhere :)
It already runs on a local server and you can easily run this in the cloud and expose the API if you wanted to - assuming your compute resources have the required GPU hardware to run it
You can try to locate where it installed and launch it directly. It usually adds a shortcut onto your Desktop. Mine points to C:\Users\surfa\AppData\Local\NVIDIA\ChatWithRTX\RAG\trt-llm-rag-windows-main\app_launch.bat
@@SurfacedStudio unless there is a repeating pattern if it is rigged... which happened in the US before on a Certain date there were certain numbers but LLMs are pretty bad at math i believe so anyway
I think it's the only model available that works with Chat With RTX at this stage. In a few clips I had 2 models but I was using an early access version. In the official release I only have the one :)
@@SurfacedStudio nooope! I found the solution. To install Llama 13b you need to have GPU with 16gb vram or... just rewrite some file in the installer folder. Worked for me
The specs clearly state 30 or 40 series RTX so I doubt it as your hardware likely doesn’t have the required tensor cores, but you can always give it a shot
"GPT" is just a term describing a type of AI model and how it works. It's a more specific term than "LLM". Yes you can run a GPT model on your local machine. You can't run (the) ChatGPT (TM) on your local machine if that's what you mean. ChatGPT is just a type of GPT that happens to be only offered online ;)
@@SurfacedStudio "GPT, or Generative Pre-trained Transformer, is a class of natural language processing (NLP) models developed by OpenAI." in the post you sent me, I see this statement.
Yup. It's exactly what I said in the video and in my comments :) That statement doesn't in any way contradict the ability to run a GPT model on your local computer. I think we're talking two different languages here... ;)
⭐ Just some clarifications in case I wasn't clear enough in the video :)
Chat With RTX does *not* run ChatGPT locally, it uses *the same type of* AI model - a GPT LLM. The training dataset is different and so are the responses.
I used it with RTX3080 + 64GB RAM, to be honest it seems ok when you use it with PDF text documents and ask it questions on the text content. It doesn't seem quite as good for the you tube video transcripts I've tried it with, but I suppose that's a bit hit and miss. I don't know how good it would be with numerical data sets
how long takes to process a PDF with many pages? any benchmarks? what about processing a youtube video?
I tried it with a pdf of frankenstein, 20000leagues under the sea, and snowcrash, a few hundred pages. I tried them all together and then separately, obviously it works better sepearatly. These took less than 5 minutes to process and you could ask questions about the text, but it completely loses context after one or two questions if you try to follow up on the last conversation. I tried it with some french homework, and it could do some translation and inference but struggles with how text is formatted on a page.The youtube thing probably depends on how good the transcript is@@mariogross
Nice and Clean Video. Keep going 🙌
Thank you 🙌
Hahaha finally some use to my RTX
Haha! And this is a pretty good one IMO. Though I have to say a lot of RTX powered games look fantastic!
@@SurfacedStudio yeah, it's good to see things other than games, it is a pretty powerful tool
I don't have the option for youtube URL and no matter how much I search about it I cannot find anyone talking about it, is it only me who has this problem?
same
After watching your video I tried this "Chat with RTX" demo, not too impressed with it. It runs perfect on a 14900K with 48Gb of ram and a RTX 4090. Unfortunately it did not impress me much with its responses. It seems Llama 2 and Mistral 7B were educated by car mechanics. Every single question about cars and their engine specifications gave me correct answers but asking it the specifications of an Intel I9 14900K were all wrong. It said it had a total of 20 threads and 10 cores, it also said its turbo boost was 5Ghz and a TDP of 95w. With UHD 630 graphics, why it would know more about cars than CPU's makes me wonder. Will continue to play with it but, I can't trust its answers 100%.
Oh I've found plenty of inaccuracies with it - as I have with all other chatbots, including ChatGPT ;)
It once told me I should heat my oven to 60 degrees to bake my pizza... ☃️
All (current) AI LLMs suffer from false positives and *any* information you get out of them should be double checked if you're planning to rely on it in any sort of meaningful way. They're great for ideas, rephrasing or summarising text, writing content outlines and much much more, but asking specific technical questions is likely going to give you some inaccuracies...
ChatGPT does a better job of this mostly because of the much larger dataset it has consumed for information :)
Nice. But at the moment not useful for German documents. Hopefully they improve ChatRTX.
Does it retain the information after you've fed it the data once? I'm thinking about making a model for a specific software and just feeding it a ton if info. Thx
no i don't think so
So, basically it's a platform to teach people how to train AI? If so, it's a great initiative from Nvidia.
I don't think there's much 'teaching' involved. You point it at a folder / YT video or playlist and it just consumes the info into the model. But it's working pretty well for a tech demo
Everything you feed it from your hard drive folders is kept locally on your computer? I know you already answered that..am asking just in case there was any concerning updates since your upload.
Nothing I've heard changes my understanding of how this works. Everything runs locally on your graphics card. No data is shared or uploaded anywhere :)
@@SurfacedStudio Thank you for your time and response :) I just got my new RTX4060 and i will try this as well.
Wondering if this could replace the bots on Zendesk, and whether an API can be created.
It already runs on a local server and you can easily run this in the cloud and expose the API if you wanted to - assuming your compute resources have the required GPU hardware to run it
@@SurfacedStudio I figured dual 3090 in SLI should be enough. Thanks!!
I can't find it on my start menu which is very Annoying you got any fix about it
You can try to locate where it installed and launch it directly. It usually adds a shortcut onto your Desktop. Mine points to C:\Users\surfa\AppData\Local\NVIDIA\ChatWithRTX\RAG\trt-llm-rag-windows-main\app_launch.bat
But can u load it with every lotto number what's been drawn over a Yr and get it to make a prodict on which are the most likely number to come out
I don’t think that’s how probability works ;)
@@SurfacedStudio unless there is a repeating pattern if it is rigged... which happened in the US before on a Certain date there were certain numbers
but LLMs are pretty bad at math i believe so anyway
Which GPU are you using? Will it work in 3050?
It works on 30 and 40 series graphics card. I've had it run on a 3090 and a 4080
any big different in their performance? @@SurfacedStudio
Nice video! Do you know if it's possible to change the ai model it uses? (Mistral 7B int4). It's the only model available to me.
I would like to know this as well
I think it's the only model available that works with Chat With RTX at this stage. In a few clips I had 2 models but I was using an early access version. In the official release I only have the one :)
@@SurfacedStudio nooope! I found the solution. To install Llama 13b you need to have GPU with 16gb vram or... just rewrite some file in the installer folder. Worked for me
the official site has removed the download button
Can you share a screenshot? I can still see a big 'Download Now' button
Is there a way to run using an rtx 2070 ?
The specs clearly state 30 or 40 series RTX so I doubt it as your hardware likely doesn’t have the required tensor cores, but you can always give it a shot
is it availible for mac download ?
Which part of "Only for 30 and 40 series rtx gpu's" did you not understand?
Hi there bro NVIDIA Chat With RTX looks nice I m interest to see of it in the near future by the way how are you today? man...
I'm good, thank you! How have you been? :)
Cool stuff, Thanks.
Glad you liked it!
👍azerbaijan
? 👍
6:28
how to say goodbye in hindi....
idk which language is this but i think rtx maked new language hindi 2.0😂😂
Seems to be hindi but instead of goodbye its something like "The villagers have their own relationship with Durga. "😂
might wanna delete the video.. there is no download link on the website
Can you share a screenshot? I can still see a big 'Download Now' button on their website
@@SurfacedStudio where would you like me to give you the screenshot. no im not joining your discord
Share on imgur? Also happy for an email :)
Outstanding!!
:D
In hindi 😂 not even close to goodbye
😎😎🥰🥰😀😀
😊🙂😺😸😸
50 Gigs ????!!!!
Yup, about the size of your average game these days 😅
yeah downloading the package right now, the compressed file is 35GB approx
You cannot run GPT locally on your own machine. Your statement is completely wrong. Those models are alternatives to GPT, but they are not GPT.
"GPT" is just a term describing a type of AI model and how it works. It's a more specific term than "LLM". Yes you can run a GPT model on your local machine. You can't run (the) ChatGPT (TM) on your local machine if that's what you mean. ChatGPT is just a type of GPT that happens to be only offered online ;)
@@SurfacedStudioYou surely don't know what GPT is. Do your homework first.
Right back at ya: blog.stackademic.com/understanding-the-difference-between-gpt-and-llm-a-comprehensive-comparison-1f624c713507
@@SurfacedStudio "GPT, or Generative Pre-trained Transformer, is a class of natural language processing (NLP) models developed by OpenAI."
in the post you sent me, I see this statement.
Yup. It's exactly what I said in the video and in my comments :) That statement doesn't in any way contradict the ability to run a GPT model on your local computer.
I think we're talking two different languages here... ;)