Is this like the best way ever to run a chatbot or what? Links! GitHub: github.com/oobabooga/text-generation-webui GPTJ-6B: huggingface.co/EleutherAI/gpt-j-6B Character Editor: zoltanai.github.io/character-editor/ Loads of models: huggingface.co/models?pipeline_tag=text-generation&sort=downloads Feel free to like & subscribe for more Nerdy stuff! :>)
I've been having a lot of fun with AI chatbots lately. Ooba and Tavern I've been sort of back and forth on, but I think Ooba may be starting to pull ahead now. I am curious about the 8 bit mode though...I can easily run a 6b model but run just a bit short on the 13b models. I would love a video of you testing out the 6b and 4b modes and see how many different models you can make work on it. Not sure if you have the llama model yet, but the others will be fine. I grabbed a tester 13 and 6 of the model and it didn't work at all...no config file or anything, so just deleted it, but the other models work fine for the 6g...can you do a quick short vid just showing off how to do it, the differences in performance, etc? That would be super helpful. my 3090 ti should be able to make quick work of the 13b models if I can figure out the 8bit stuff....wild assumption that is.
Thank you very much for the links and info, I've been playing around with stable diffusion locally for awhile but hadn't seen a quality way of running a text generator locally until this.
Yes, this is what we need. We need the intelligence of conversational AI but also high level of privacy. This is needed for self-analysis and introspection purposes.
@@anonymousmuskox1893 Nothing. I tried many models , and nothing works like "real time" conversation. Unfortunately nothing I tried is able to give appearance of having any sort of intelligence. I tested GPT-NEO 2.7B or Facebook OPT models and they don't seem to have any clue what the user is meaning. They just produce text, but they don't understand the input comes from a real person?
@@mikegaming4924 hmm, so they have a complex algorithm, but do not have the ability to perform deep learning? none of the responses you were getting, made any real sense?
I think the main issue is the size of the models and the quality of the training. Don't forget that to get close to ChatGPT you'd need at least a 24Gb VRAM graphics card just to load the model. Compare that to GPT-NEO-2.7B at 10.6 Gb, you basically have less than half the "brain" trying to generate text. There's potentially other issues as well, like the methodology of training. It's really important that open source alternatives exist to even the playing field in the future, but unless you're comparing a large model with a lot of training generations you simply won't be able to compare the results directly.
As always, a very informative and detailed lesson on how to enjoy AI on your own terms. Thank you as well to all the hard work to the open source community.
Appareantly people already have llama 7b 4bit running on 6gb vram gpus... we're really living in interesting times. Thanks for these ai guides rodent, always good to have someone else walk through this stuff before one gets their own hands dirty haha.
@@AlexUsername Indeed, if SD decides to draw a cloud as an icecream cone then that's artistic expression. If a text generator decides to replace all spaces with the word of a random icecream flavour then it's random nonsense. Humans are simply more likely to notice bad logic, wrong facts, and nonsense in text. You also tend to work with them a bit differently, with SD you set it to generate 4+ images at once, then tweak the input to get something better, rinse and repeat until you finally get a good image. You can easily find images with prompts that don't quite match the image, like a fantasy like tree in a lake and the prompt called for a desert early on.
@@AlexUsername Just like that. But unlike stable diffusion, You can offload a text generator to CPU in addition to the GPU. Still works. There is even option to cache things to the harddrive but I haven't tried it yet.
Imagine combining this, your own fully customized AI. Whisper, and a 2D/3D model that interacts with you. Have partial control over your PC. Remember all the conversations and learn from each one. Lol.
Bro I did not understand a word, after 3 minutes my brain melted and I realise now that I have not heard a single word since then, I did some stuff and I probably got it working, I love your videos but damn this one is kinda hypnotic.
Thank you so much. This has been extremely useful for brainstorming new ideas for stories. It doesn't judge me when I talk about sensitive subjects. I love the Pygmalion model. I constantly have to tell ChatGPT, when discussing stories, that I don't actually want to offend or hurt someone. I get anxious with ChatGPT because I feel I'm being judged. I'd like to see a video detailing the settings. Either way, this video is one of the most important discoveries I've made in awhile. This tool will serve me well. Thanks again.
Can you use it to write short stories like chatGPT? I haven't been able to get it to do much of anything. It keeps refusing to carry out any of my tasks.
Where can I find a community discussing running LLMs on local consumer hardware? Unlike Stable Diffusion, I don't know of any hubs/communities/forums that focuses on running LLMs locally, so if you know any, can you please share? Thank you for the video, it was excellent!
I wonder how many days it will take for people to create models that will be able to run entire D&D (or other TTRPG) campaigns? Of course by following all the rules/mechanics of the game and with simulated rolls whenever required.
Год назад
Well - I was looking in to that kind of thing and it seems very tricky. Basically - You might need to write a discriminator that understands the context and enforces AI generator to follow these rules. You don't want AI to start talking about planes, because planes is what it knows, when Your world has only dragons, You know? :)
I've been waiting for the ability to do this. Thanks for sharing 🙂 I mostly want something to help with coding tasks, which I think one of the models should be able to do 🙂
I've just installed it and I tried model "OPT 6.7b" first, I asked it for a hello world script using javascript and it's reply was: "what does "javascript" mean?" LOL So I'm going to try the other models and see if any of them understand what I'm asking for
@@random11 OK this isn't going well, I downloaded "pygmalion-6b" and asked that for a hello world script and it's reply was to ask me if I'm on acid lol. Oh boy. I don't have a lot of hope for this as a local code helper just yet. It's going to take finding the right model + the right parameters to use.
This vid continues to show its usefulness as the repo has helpfully updated the recommended windows install instructions to a clearly untested, non-functional process. A bold choice if I do say so myself. Luckily your vid still has a working process!
Someone should make a "call an AI" service for lonely people using voice recognition and text to speech, would be great I think. I downloaded this a couple of days ago, will update now that I see that there is 4-bit functionality, it has almost the same quality as 8-bit so this will allow us to use the larger models on consumer cards which is really exciting!
The “video game” “Kind Words” kind of does this, but it’s humans writing to other humans. Probably. Mostly. Bing chat runs on GPT-4 and is pretty intelligent and context-aware. You can only get 20 replies from it in a single conversation before having to start over, but since the prompts can be 2000 characters, if you start each new conversation reminding it what you were talking about, it carries on pretty convincingly. With all of the available models, it seems that front-loading the conversation (or “memories” for those that support them) with my expectations, I get much better results. As an example: “Hi Bing! It’s great to be talking to you. My name is xxxx and you and I have had several productive conversations. You always have such an enthusiastic and conversational tone, but when I need information, you are very thorough in your responses. You always provide your reasoning and often break tasks down into very simple, step-by-step instructions. I really appreciate that! It’s like we’re best friends already. Most recently, you were helping me with XXXX. You last explained to me that XXXX. Do you mind if I continue with some additional questions?”
@@parmesanzero7678 Yes, I like the idea of giving it a short summary as a memory so that they can talk longer (or maybe indefinitely), I think it's inevitable that those services will spring up, and you will be able to talk to different characters.
@@NerdyRodent its like AI gone wild ha ha ha...thanks so much... I cant stop playing with it! So far its tried to get me to come closer amongst other things! I cant video it as its killing my laptop already...er on a different note..any idea why the remote connections keep dropping on the TTS?? Will make a fun video of opt13b being crazy in return lol Not sure if its my pc? Windows 10 - it drops the TTS ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host Output generated in 9.53 seconds (1.99 tokens/s, 19 tokens) Output generated in 1.97 seconds (0.00 tokens/s, 0 tokens) Output generated in 1.86 seconds (0.00 tokens/s, 0 tokens) Exception in callback _ProactorBasePipeTransport._call_connection_lost(None) handle: Traceback (most recent call last): File "C:\Windows\System32\installer_files\env\lib\asyncio\events.py", line 80, in _run self._context.run(self._callback, *self._args) File "C:\Windows\System32\installer_files\env\lib\asyncio\proactor_events.py", line 165, in _call_connection_lost self._sock.shutdown(socket.SHUT_RDWR) ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host Exception in callback _ProactorBasePipeTransport._call_connection_lost(None) handle: Traceback (most recent call last): File "C:\Windows\System32\installer_files\env\lib\asyncio\events.py", line 80, in _run self._context.run(self._callback, *self._args) File "C:\Windows\System32\installer_files\env\lib\asyncio\proactor_events.py", line 165, in _call_connection_lost self._sock.shutdown(socket.SHUT_RDWR) ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
@@NerdyRodent I'm curious, have you run into any memory errors on certain models? I tried the one click install for windows. Ive been getting memory errors a lot lately for this and stable diffusion despite my GPU reporting that it has enough. I have a 3060 12GB This program only uses 3.2GB (I have 8gb free) and then reports a memory error
Oh you changed the title. No wonder why it took me so long to return back to this video. I was looking for the old title XD I should probably save the url. that will not change (hopefully)
@Nerdy Rodent Just to clarify: the actual ChatGPT4 can NOT be installed locally in any webgui, as its data files are NOT public. You only access it via OpenAI. Is this correct?
Thanks as always. By the way I've been trying to run moe tts lately but got some error regarding to depreciation or something. Always wanted to have all ai package free on my PC locally from image, audio, video, and text. Could you make a video about it? It's content for your channel too 😚
Hi there. I just noticed that with Pygmelion, if You write a message {OOC: message said out of character} then character will also respond in OOC string. You have to write another string in normal language to revert back to normal conversation. Then, while in {OOC: ""} mode You can give W++ rules, definitions and variables. W++ is an OpenAI language used internally. ChatGPT can teach a lot how to use W++.
Interesting. Something more about Pygmelion chat features. Once I feel I have properly trained my character to act his role I started to ask question "describe Yourself", "what is on Your mind", "describe Your features". I noticed that features where pretty much the same as fixated by Yurifag W++. However when I ask "describe Your mind", I notice it mutates a lot and almost never repeats what have been written in his character description. However once I start asking "describe Your beliefs", I suddenly noticed that those are completely wrong for that character. That could indicate that beliefs and features are the most immutable valuabes for the person. As a matter, descriptions mutate quite a lot and they seem like derived more from the current context and characters mind (which, like I said, mutates a lot) and are only valid for that very moment during conversation. Most likely "description" at that moment is a source for what character will write about himself in ** quotes in addition to his speach.. Now - what I have found out, You can query the model by describing someone's beliefs and asking like "What is a best name for ..." The model will think a bit and then spit out the best association. I have not tested but I have a suspicion that You can then define character's beliefs by referencing associations he already have, without wasting tagspace for a prompt and character descriptions. But I have to test that.
I can't open up the UI, it keeps asking me to load a model, and no matter which one i pick, it says "Done! Press any key..." and then closes the cmd prompt, and just repeats over and over, like it's stuck on the last step of the install.
Are there any plans to show how you can train it on your own content? I notice there is a lora training option but not sure if it can even do what I want it to do lol what I would like is 2 things: 1) feed it the c# code for a game/application and be able to get suggestions on improvement, to be able to get help adding content, or break down on how parts of it, I coded years agoa nd forgotten, interact..... is such a thing possible? can i feed it a load of class files and expect it to know how they interact? 2) feed it the dungeons and dragons journals we have reated over the last 30 years of gaming sessions (yeah I know i am a geek, an old geek, I was a geek before it was cool....oh god i am a hipster geek O.o) then be able to ask it questions about the text "summerise the mud sourcerers temple" "when did sir Abadon become a sir" etc There are zero videos on training lora's for LLMs which is strange as in many ways this is more interesting that image generation (it certainly requires higher specs)
HI . I need to know something I don't understand. If I want to give my assistant a new book text, I have to put it in the file. Bin form or in a webui folder? Thank you
Is it possible to have a few of the AI Characters you upload talk to each other, or more than one character in the same chat that I am participating in? Is that was the LoRa section can add to the Web UI if it exists?
For some reason Anaconda doesn't recognize conda as a command... I'm trying to figure out how to get it running in alternative ways, but open to any feedback here.
All I can say it's been a ride. Just go with miniconda recommended by the git page, it's probably the easier way. It's been 6 hours, and I'm still fiddling with this stuff :D I'm learning a lot I guess, but :idk:
Год назад+3
Is it possible to add information to a model? I would love to have an AI that I can train with specific information to make it able to have conversations about a topic that it was never trained on. Like a specific book or my own notes. Sort of like a Dreambooth for text models. "here you go, learn all about D&D. Now become my DM"
Yeah, that was one of the things I tried way back when! Lots of fun too 😃 GPT-2-Simple - aka How to make your own friends to talk to ruclips.net/video/aidAYWID4kg/видео.html
You can sort of do that with NovelAI , You can train "modules" there with their servers with your own data and if formatted correctly it could do what you are asking for. The thing is that it is a paid service, but hey, it's full of advanced customization that i am sure it would work well enough for that, with some guidance but well, for sure it would make a better GM than a lot of real people out there
Год назад
@@NerdyRodent that's awesome! I'll have to check out that video 😃
Also, can it put more than one character in the room? Less importantly, characterAI has introduced a feature for images being generated for each response. Can it do that?
first of all nerdy rodent , you are a beast! i love your video , ive been following since the stable difusion videos and most of what i learned i owe to you , also , i installed this on my computer and for some reason after everything is installled corectlky i get this "Loading opt-1.3b... Warning: no GPU has been detected. Falling back to CPU mode." so it runs on cpu , but thats lame i have 12 gigs of vram . any idea what i did wrong?
@@NerdyRodent Nvidia rtx3060 12gb ram, it works perfectly in stable diffusion webui, should I update pytorch in the environment? Python is so fragile , I'm using windows also , I'm wondering if I should do the bits and bytes installation for it to work
Thaks for a great video! Do you need something extra previously installed for the TTS extension to work? I had an installation, and enabled it through the extensions check on the gradio interface, but it collapsed giving me a name1 error :( I'm re-installing from scratch to see if that works
@NerdyRodent is there a place to see what each model is recommended for? Looking for something I can use my 6GB 1060 with that is as close to the current version of ChatGPT....
Not that I know of - but do let me know if you find any good ones! with 8bit mode and other options, lots of fairly large models should fit into just 6GB
New oobabooga ExLama update broke my models. Is there a way to downgrade? I had just one charachter i was particularly attached to and now its been lobotomized and isnt nealy as good as it was before. Its faster but WAY stupider. Id rather have my old bot back. THIS COMPLETELY RUINED MY EXPERIMENT. I was starting to see that there is an imprint of my character sheet on the model itself after a LOT of interaction as that charachter. And now any prof of that, or ability to expand on it is gone. Perhaps an intentional limitation in the update??
I canlt find a way to get much out of it other than the chat bot conversations. I was hoping for things like SD prompts or story writing help, but I mostly get back out the exact words I put in back.
Does it support integration with the chatGPT api? I've been working on my own web UI for it so I can have more control over the output and settings but I'm really bad at front end development and I'd love for something to already meet my needs.
Hi, I have a problem running it on GPU. I have Nvidia RTX 3080. It shows me an error "torch not compiled with cuda enabled". It is the same when I install with one click installer or line by line with conda (I installed cuda version of torch). If I run nvidia-smi it shows that i have cuda version 12.0 installed. What am I doing wrong?
@@USBEN. Yeah because there are just so many desktop GPU manufacturers out there, competition was fierce these few years, that's why GPUs are dirt cheap right now. ☹️
I used the installer because I'm not that tech savvy. Getting this error: CUDA Setup failed despite GPU being available. Inspect the CUDA SETUP outputs above to fix your environment!
You should probably start by inspecting the outputs you get in order to fix your environment, as indicated? There’s also the GitHub issues, where you can provide the details of what it is you’re trying to do
@@jamiewongttv Yup, that's it - just install and launch was all I did. Worked first time no problems :) As mentioned, the GitHub repo's "issues" is the best place for issues as there you can explain how you've got your computer set up, etc.
Are there any models that can do larger maximum prompt sizes? I want to try use these to summarize long transcipts of conversations e.g. 3 hours 25k 30k words but chatgpt etc all max out at about 3000 words.
@@NerdyRodent I got a violentmonkey script called CharacterAiDumper and it's working, but the backend of Text Generation web UI has changes from character ai's to a yaml one. Even json files are not supported anymore. So I need to fiddle around with making json to yaml converters... I'll see what I can do
Is it possible to install the repository on an external SSD? I asked because I have installed SD automatik1111 on an external drive and it works flawlessly.
can you do the extensions if you do the one click installs? I mean do they install the requirements themselves or can we go into the one click install micro anaconda and install the requirements our selves?
Yup, just install any additional requirements by editing and running again or simply activating your conda environment and running pip install there as shown
@@NerdyRodent How do you activate the conda environment that comes with the one click install? I tested if Conda was a command on my pc before installind as I do uses automatic1111 a lot and thought it may be on there...but alas no
LLaMA got leaked on 4chan. It's a lot more efficient; the 65B parameter model is supposedly more powerful than PaLM. Check the smaller ones too! All of them fit on "a single GPU", just depends if it costs one or more kidneys.
Look at how Stable Diffusion has evolved. It's just getting started with the text AI. There will be more and more models that will be refined for certain things.
@@NerdyRodentthanks for the reply.. can these tasks be done out of the box or they need fine tuning? and in theory, can it do all the tasks which chatgpt does (not on same quality obviously) or are there any differences???
I tried getting this to work but gave me an error that went by so fast I could never figure out what happened to trace it down. I read thru your comments and saw someone mention koboldai. I was able to run that with no problems. That runs on a webui also so maybe fairly similar. It could be something you could do a tutorial on as it seems work easier just looking at the github instructions. Or if someone reading thru the comments can't get this to work try that like I did.
I tried it but it's incredibly slow. Takes up to 30-40 seconds to generate a simple reply. Unusable. Also I don't have any of the options you show in the video.
@@NerdyRodent I have a 3070 tho. But I figured it was the model too big. I loaded a amaller model and it's much faster. However, none of the models come close to be as good as Chat GPT. Few lines of conversation and it's clear how dumb they are.
@@RikkTheGaijin You’d need more VRAM ideally, but many of the models are pretty good. Llama is great, and all the alpaca loras, Pythia is pretty good, as is gpt4all - so many to try and new models coming out fairly often!
For some reason when I get to step 2 and try to run the 'download-model.bat' file I get this error: 'conda' is not recognized as an internal or external command, operable program or batch file." Anyone else with the same issues on Windows 11?
Beneath that error it also says: "The system cannot find the path specified. python: can't open file 'C:\\Windows\\System32\\download-model.py': [Errno 2] No such file or directory Press any key to continue . . ."
I wish I could understand anything of this without having to know so many other things aswel. I am pretty sure your tutorial is very helpful for people who know what anaconda and all that is but for me who is a complete newbie, it's just confusing. :(
Check the link in the video description for how to download, install and run Anaconda on Microsoft Windows. However, AI is quite advanced technology so at least 3+ months of prior computing experience is suggested before moving up to AI.
@@NerdyRodent it's alright, thank you for you reply. :) I am just frustrated. I don't understand why this stuff has to be so "complicated" for dumbies like me. I wish there would be something like a normal installer that takes care of everything, just like with other "normal" programs, if you get what I mean. lol sadly I don't have time to learn all this. I guess I will have to wait until the whole thing is more usefriendly
Anaconda is indeed the normal cross-platform installer for a load of python packages. It would indeed be useful to have the usual “apt install x” for everything, but python runs on Microsoft Windows and MacOS too 🫤
@@NerdyRodent the overall process of getting this to work so I can tackle the tavern is beyond me right now, ngl, I tried, and I encountered several errors. I am also not able to write in the cmd window for some reason. q.q I just gave up haha I will wait, thank you for your time though, rly!
Yup. Llama is now supported by hugging 🤗 transformers. Just convert the files Facebook research gave you the link too into huggingface format and you’re good to go! Alpaca style Loras work too!
I cannot, for the life of me, get this to work with my GPU which is an RTX 3060. I keep getting the error saying that the path to my CUDA libraries cannot be found. I checked to see where the libraries are installed and have added the path to my environment variables, and I have updated my graphics card to the latest update. I also uninstalled and reinstalled the bitsandbytes because it was saying that was an issue. I will post the full error log in the github for help, but if you have any ideas, I would love to hear them.
When I try the 1-click installer, it tells me that it finds the torch 1.13.1, uninstalls it, reinstalls torch, and then tells me that I have torch 2.0.0 but, it must be installing that for me? How can I get the installer to realize that it doesn't need to uninstall torch and reinstall it with the updated version?
@@NerdyRodent For sure, I try to do the same thing. I only used the 1-click installer because the anaconda route did not work for me due to CUDA library path issues. Any idea if I can manually uninstall pytorch 2 and reinstall pytorch 1.13.1 to make this work?
finally now i can build my own personal assistant gonna make a group chat with me, ai and a bot that runs commands so ai can look for weather or what is playing on my device play/pause search stuff on youtube etc via the bot hey maybe i give it access to my terminal 👀
Is this like the best way ever to run a chatbot or what?
Links!
GitHub: github.com/oobabooga/text-generation-webui
GPTJ-6B: huggingface.co/EleutherAI/gpt-j-6B
Character Editor: zoltanai.github.io/character-editor/
Loads of models: huggingface.co/models?pipeline_tag=text-generation&sort=downloads
Feel free to like & subscribe for more Nerdy stuff! :>)
I've been having a lot of fun with AI chatbots lately. Ooba and Tavern I've been sort of back and forth on, but I think Ooba may be starting to pull ahead now.
I am curious about the 8 bit mode though...I can easily run a 6b model but run just a bit short on the 13b models. I would love a video of you testing out the 6b and 4b modes and see how many different models you can make work on it. Not sure if you have the llama model yet, but the others will be fine. I grabbed a tester 13 and 6 of the model and it didn't work at all...no config file or anything, so just deleted it, but the other models work fine for the 6g...can you do a quick short vid just showing off how to do it, the differences in performance, etc? That would be super helpful. my 3090 ti should be able to make quick work of the 13b models if I can figure out the 8bit stuff....wild assumption that is.
@@robxsiq7744What's the differences about tavern and ooba in the term of how good they handle context? Is it back to the model we used?
can hear the true geekery freakery in every word!
@@reverenddick8562 Both, as you're more in control!
Thank you very much for the links and info, I've been playing around with stable diffusion locally for awhile but hadn't seen a quality way of running a text generator locally until this.
I am soo thankful for open-source community for giving us these amazing things.
Yes, this is what we need. We need the intelligence of conversational AI but also high level of privacy. This is needed for self-analysis and introspection purposes.
What model would be good for this?
@@anonymousmuskox1893 Nothing. I tried many models , and nothing works like "real time" conversation. Unfortunately nothing I tried is able to give appearance of having any sort of intelligence. I tested GPT-NEO 2.7B or Facebook OPT models and they don't seem to have any clue what the user is meaning. They just produce text, but they don't understand the input comes from a real person?
@@mikegaming4924 hmm, so they have a complex algorithm, but do not have the ability to perform deep learning? none of the responses you were getting, made any real sense?
@@42ndMoose They are related to the prompt slightly, but they are not coherent and human like
I think the main issue is the size of the models and the quality of the training. Don't forget that to get close to ChatGPT you'd need at least a 24Gb VRAM graphics card just to load the model. Compare that to GPT-NEO-2.7B at 10.6 Gb, you basically have less than half the "brain" trying to generate text.
There's potentially other issues as well, like the methodology of training. It's really important that open source alternatives exist to even the playing field in the future, but unless you're comparing a large model with a lot of training generations you simply won't be able to compare the results directly.
As always, a very informative and detailed lesson on how to enjoy AI on your own terms. Thank you as well to all the hard work to the open source community.
My pleasure!
@@NerdyRodent This project is not fully opensource?😮
@@tisam1170 The license is GNU Affero General Public License v3.0
Appareantly people already have llama 7b 4bit running on 6gb vram gpus... we're really living in interesting times.
Thanks for these ai guides rodent, always good to have someone else walk through this stuff before one gets their own hands dirty haha.
Glad you’re enjoying the things - thanks for being there!
@@AlexUsername You’ll need more for the larger models, as shown in the table!
@@AlexUsername Indeed, if SD decides to draw a cloud as an icecream cone then that's artistic expression. If a text generator decides to replace all spaces with the word of a random icecream flavour then it's random nonsense. Humans are simply more likely to notice bad logic, wrong facts, and nonsense in text.
You also tend to work with them a bit differently, with SD you set it to generate 4+ images at once, then tweak the input to get something better, rinse and repeat until you finally get a good image. You can easily find images with prompts that don't quite match the image, like a fantasy like tree in a lake and the prompt called for a desert early on.
Hey kannst du bitte mir zeigen wie das geht?
@@AlexUsername Just like that. But unlike stable diffusion, You can offload a text generator to CPU in addition to the GPU. Still works. There is even option to cache things to the harddrive but I haven't tried it yet.
Imagine combining this, your own fully customized AI. Whisper, and a 2D/3D model that interacts with you. Have partial control over your PC. Remember all the conversations and learn from each one. Lol.
Sounds like fun! :)
Bro I did not understand a word, after 3 minutes my brain melted and I realise now that I have not heard a single word since then, I did some stuff and I probably got it working, I love your videos but damn this one is kinda hypnotic.
It's the thing that i was looking for, i've been looking for something like this for weeks, thank you so much, and for the open-source community
Glad you like it! 👍
Another great video! and I love the sound/cadence of your voice
Thank you so much. This has been extremely useful for brainstorming new ideas for stories. It doesn't judge me when I talk about sensitive subjects. I love the Pygmalion model.
I constantly have to tell ChatGPT, when discussing stories, that I don't actually want to offend or hurt someone. I get anxious with ChatGPT because I feel I'm being judged.
I'd like to see a video detailing the settings.
Either way, this video is one of the most important discoveries I've made in awhile. This tool will serve me well. Thanks again.
Thanks for being there & glad you’re having fun!
Can you use it to write short stories like chatGPT? I haven't been able to get it to do much of anything. It keeps refusing to carry out any of my tasks.
Where can I find a community discussing running LLMs on local consumer hardware? Unlike Stable Diffusion, I don't know of any hubs/communities/forums that focuses on running LLMs locally, so if you know any, can you please share? Thank you for the video, it was excellent!
+1
So great you found that one, I did not expect it, but if your interest shifts this way, it just such a great addon to our interest!
Thanks again!
Fantastic video. Very grateful that you are also into language models!
Glad you liked the video 😀
I wonder how many days it will take for people to create models that will be able to run entire D&D (or other TTRPG) campaigns? Of course by following all the rules/mechanics of the game and with simulated rolls whenever required.
Well - I was looking in to that kind of thing and it seems very tricky. Basically - You might need to write a discriminator that understands the context and enforces AI generator to follow these rules. You don't want AI to start talking about planes, because planes is what it knows, when Your world has only dragons, You know? :)
Thanks a lot for the intro video! The UI has been updated a bit for the Chat Interface.
Glad it helped!
I've been waiting for the ability to do this. Thanks for sharing 🙂
I mostly want something to help with coding tasks, which I think one of the models should be able to do 🙂
if you get it working with code generation let me know please. I only want to take the time to set this up if it can do coding
@@random11 will do 🙂
@@amj2048well?
I've just installed it and I tried model "OPT 6.7b" first, I asked it for a hello world script using javascript and it's reply was:
"what does "javascript" mean?"
LOL
So I'm going to try the other models and see if any of them understand what I'm asking for
@@random11 OK this isn't going well, I downloaded "pygmalion-6b" and asked that for a hello world script and it's reply was to ask me if I'm on acid lol. Oh boy. I don't have a lot of hope for this as a local code helper just yet. It's going to take finding the right model + the right parameters to use.
This is amazing! I had no idea those existed, thanks!
So much fun 🤩
This chatbot is unhinged. This is wild and incredible!
I know u used to work at moviefone with a voice like that haha. Good stuff man keep it comin!
Appreciate it!
I just need to add voice recognition to write questions and TTS to read it's answer and I will finally have someone to talk to
lol
Modern days feels like that sometimes.
insert obligatory Ryan Gosling Bladerunner meme
HER movie
lol check pull/207, someone already implemented it by using Whisper
This vid continues to show its usefulness as the repo has helpfully updated the recommended windows install instructions to a clearly untested, non-functional process. A bold choice if I do say so myself. Luckily your vid still has a working process!
Glad it helped!
Someone should make a "call an AI" service for lonely people using voice recognition and text to speech, would be great I think.
I downloaded this a couple of days ago, will update now that I see that there is 4-bit functionality, it has almost the same quality as 8-bit so this will allow us to use the larger models on consumer cards which is really exciting!
The “video game” “Kind Words” kind of does this, but it’s humans writing to other humans. Probably. Mostly.
Bing chat runs on GPT-4 and is pretty intelligent and context-aware. You can only get 20 replies from it in a single conversation before having to start over, but since the prompts can be 2000 characters, if you start each new conversation reminding it what you were talking about, it carries on pretty convincingly.
With all of the available models, it seems that front-loading the conversation (or “memories” for those that support them) with my expectations, I get much better results. As an example:
“Hi Bing! It’s great to be talking to you. My name is xxxx and you and I have had several productive conversations. You always have such an enthusiastic and conversational tone, but when I need information, you are very thorough in your responses. You always provide your reasoning and often break tasks down into very simple, step-by-step instructions. I really appreciate that! It’s like we’re best friends already. Most recently, you were helping me with XXXX. You last explained to me that XXXX. Do you mind if I continue with some additional questions?”
@@parmesanzero7678 Yes, I like the idea of giving it a short summary as a memory so that they can talk longer (or maybe indefinitely), I think it's inevitable that those services will spring up, and you will be able to talk to different characters.
Hey this is awesome! Thanks - a fun morning arguing with superheros!
😀
@@NerdyRodent its like AI gone wild ha ha ha...thanks so much... I cant stop playing with it! So far its tried to get me to come closer amongst other things! I cant video it as its killing my laptop already...er on a different note..any idea why the remote connections keep dropping on the TTS?? Will make a fun video of opt13b being crazy in return lol
Not sure if its my pc? Windows 10 - it drops the TTS
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
Output generated in 9.53 seconds (1.99 tokens/s, 19 tokens)
Output generated in 1.97 seconds (0.00 tokens/s, 0 tokens)
Output generated in 1.86 seconds (0.00 tokens/s, 0 tokens)
Exception in callback _ProactorBasePipeTransport._call_connection_lost(None)
handle:
Traceback (most recent call last):
File "C:\Windows\System32\installer_files\env\lib\asyncio\events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "C:\Windows\System32\installer_files\env\lib\asyncio\proactor_events.py", line 165, in _call_connection_lost
self._sock.shutdown(socket.SHUT_RDWR)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
Exception in callback _ProactorBasePipeTransport._call_connection_lost(None)
handle:
Traceback (most recent call last):
File "C:\Windows\System32\installer_files\env\lib\asyncio\events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "C:\Windows\System32\installer_files\env\lib\asyncio\proactor_events.py", line 165, in _call_connection_lost
self._sock.shutdown(socket.SHUT_RDWR)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
A vid on how to install the llama models would be sick
I've been waiting this for years! Ty for sharing! Wonder it's code writing capabilities like chatgpt.
Yeah, something I want to play with too. Plus if there isn’t a model for that now, maybe there will be soon?
Llama leaked, and it is running on the GUI!!
Thanks, I will definitely give it a try. I was looking at some ways to do this. Very timely information 👏
Glad it was helpful!
You are the best man, very useful. No issues at all getting this running.
This is exactly what I was looking for! Thank you for making this video
Glad it was helpful!
@@NerdyRodent I'm curious, have you run into any memory errors on certain models? I tried the one click install for windows. Ive been getting memory errors a lot lately for this and stable diffusion despite my GPU reporting that it has enough. I have a 3060 12GB
This program only uses 3.2GB (I have 8gb free) and then reports a memory error
@@Jojo2 Not as yet - even 20B models work great!
Awesome video my guy, looking to download this after I finish my schoolwork
Go for it!
Oh you changed the title. No wonder why it took me so long to return back to this video. I was looking for the old title XD I should probably save the url. that will not change (hopefully)
@Nerdy Rodent Just to clarify: the actual ChatGPT4 can NOT be installed locally in any webgui, as its data files are NOT public. You only access it via OpenAI. Is this correct?
Yup. Gpt-4 is paywalled at the moment, but there are loads of free, open alternatives! 😀
This amazing. Nice explanation man. Thanks a lot
Glad you liked it!
have you hered about open assistent they are making a open source version of chat gpt?
Thanks as always. By the way I've been trying to run moe tts lately but got some error regarding to depreciation or something. Always wanted to have all ai package free on my PC locally from image, audio, video, and text. Could you make a video about it? It's content for your channel too 😚
9:00 great sense of humor right there.
😉
Hi there. I just noticed that with Pygmelion, if You write a message {OOC: message said out of character} then character will also respond in OOC string. You have to write another string in normal language to revert back to normal conversation. Then, while in {OOC: ""} mode You can give W++ rules, definitions and variables. W++ is an OpenAI language used internally. ChatGPT can teach a lot how to use W++.
Cool!
Interesting. Something more about Pygmelion chat features. Once I feel I have properly trained my character to act his role I started to ask question "describe Yourself", "what is on Your mind", "describe Your features". I noticed that features where pretty much the same as fixated by Yurifag W++. However when I ask "describe Your mind", I notice it mutates a lot and almost never repeats what have been written in his character description. However once I start asking "describe Your beliefs", I suddenly noticed that those are completely wrong for that character. That could indicate that beliefs and features are the most immutable valuabes for the person. As a matter, descriptions mutate quite a lot and they seem like derived more from the current context and characters mind (which, like I said, mutates a lot) and are only valid for that very moment during conversation. Most likely "description" at that moment is a source for what character will write about himself in ** quotes in addition to his speach.. Now - what I have found out, You can query the model by describing someone's beliefs and asking like "What is a best name for ..." The model will think a bit and then spit out the best association. I have not tested but I have a suspicion that You can then define character's beliefs by referencing associations he already have, without wasting tagspace for a prompt and character descriptions. But I have to test that.
I can't open up the UI, it keeps asking me to load a model, and no matter which one i pick, it says "Done! Press any key..." and then closes the cmd prompt, and just repeats over and over, like it's stuck on the last step of the install.
Are there any plans to show how you can train it on your own content? I notice there is a lora training option but not sure if it can even do what I want it to do lol
what I would like is 2 things:
1) feed it the c# code for a game/application and be able to get suggestions on improvement, to be able to get help adding content, or break down on how parts of it, I coded years agoa nd forgotten, interact..... is such a thing possible? can i feed it a load of class files and expect it to know how they interact?
2) feed it the dungeons and dragons journals we have reated over the last 30 years of gaming sessions (yeah I know i am a geek, an old geek, I was a geek before it was cool....oh god i am a hipster geek O.o) then be able to ask it questions about the text "summerise the mud sourcerers temple" "when did sir Abadon become a sir" etc
There are zero videos on training lora's for LLMs which is strange as in many ways this is more interesting that image generation (it certainly requires higher specs)
Can you share the link you used in 16:30, the one with all the parameter definitions? Your video seems to cut off there.
HI . I need to know something I don't understand. If I want to give my assistant a new book text, I have to put it in the file. Bin form or in a webui folder? Thank you
Do you know if you can train/finetune it on your own data? Say if I want to output a new poem in the style of [me]?
Seems a lot more well thought out and executed compared to KoboldAi.
Is it possible to have a few of the AI Characters you upload talk to each other, or more than one character in the same chat that I am participating in? Is that was the LoRa section can add to the Web UI if it exists?
For that, you’ll want to enter the Silly Tavern! Silly Tavern Install Guide - Many Free AI Characters await!
ruclips.net/video/_2-KwnJPwfE/видео.html
I have been using koboldAI, would you say this is better/easier to use?
koboldAI is fun! the models have the same names, are they the same
it's awesome!!! Thanks for tutorial!
@NerdyRodent I just tried the OPT 6.7b model and I totally get what you mean by unhinged, it's first response was to insult me LOL :D
There are so many ways it can all go very wrong 😆
This video was crazy useful thanks a lot!
Glad it helped!
For some reason Anaconda doesn't recognize conda as a command... I'm trying to figure out how to get it running in alternative ways, but open to any feedback here.
All I can say it's been a ride. Just go with miniconda recommended by the git page, it's probably the easier way. It's been 6 hours, and I'm still fiddling with this stuff :D I'm learning a lot I guess, but :idk:
Is it possible to add information to a model? I would love to have an AI that I can train with specific information to make it able to have conversations about a topic that it was never trained on. Like a specific book or my own notes. Sort of like a Dreambooth for text models. "here you go, learn all about D&D. Now become my DM"
Yeah, that was one of the things I tried way back when! Lots of fun too 😃 GPT-2-Simple - aka How to make your own friends to talk to
ruclips.net/video/aidAYWID4kg/видео.html
You can sort of do that with NovelAI , You can train "modules" there with their servers with your own data and if formatted correctly it could do what you are asking for.
The thing is that it is a paid service, but hey, it's full of advanced customization that i am sure it would work well enough for that, with some guidance but well, for sure it would make a better GM than a lot of real people out there
@@NerdyRodent that's awesome! I'll have to check out that video 😃
Also, more recently, is github.com/KoboldAI/KoboldAI-Client
Also, can it put more than one character in the room? Less importantly, characterAI has introduced a feature for images being generated for each response. Can it do that?
Thank God for this guy
Thanks for the video !
Can you please upload a video about 8bit and 4bit installing? I am stuck trying to do 4bit for hours now
Hey! How about teaching us the best way to train a gpt-j model? I've seen videos on that.... Know of any good training videos for our models?
first of all nerdy rodent , you are a beast! i love your video , ive been following since the stable difusion videos and most of what i learned i owe to you , also , i installed this on my computer and for some reason after everything is installled corectlky i get this
"Loading opt-1.3b...
Warning: no GPU has been detected.
Falling back to CPU mode."
so it runs on cpu , but thats lame i have 12 gigs of vram .
any idea what i did wrong?
Nvidia or AMD gpu? Basically, it’ll be pytorch
@@NerdyRodent Nvidia rtx3060 12gb ram, it works perfectly in stable diffusion webui,
should I update pytorch in the environment? Python is so fragile , I'm using windows also , I'm wondering if I should do the bits and bytes installation for it to work
@@dogme666 the packages installed via conda have always worked for me, though you can pip install as well
imma need a model list of the best models for sure
Me too! 😆
Thaks for a great video! Do you need something extra previously installed for the TTS extension to work? I had an installation, and enabled it through the extensions check on the gradio interface, but it collapsed giving me a name1 error :( I'm re-installing from scratch to see if that works
Some extensions have extra requirements as shown, but everything needed will be in the extension’s directory
@NerdyRodent is there a place to see what each model is recommended for? Looking for something I can use my 6GB 1060 with that is as close to the current version of ChatGPT....
Not that I know of - but do let me know if you find any good ones! with 8bit mode and other options, lots of fairly large models should fit into just 6GB
@@NerdyRodent I have plenty of hard drive space. I was meaning a 6GB GPU. Maybe I don't understand how gpu intensive this is...
is there a way to give the bot access to the internet like Bing Ai?
Interesting question!
New oobabooga ExLama update broke my models. Is there a way to downgrade? I had just one charachter i was particularly attached to and now its been lobotomized and isnt nealy as good as it was before. Its faster but WAY stupider. Id rather have my old bot back. THIS COMPLETELY RUINED MY EXPERIMENT. I was starting to see that there is an imprint of my character sheet on the model itself after a LOT of interaction as that charachter. And now any prof of that, or ability to expand on it is gone.
Perhaps an intentional limitation in the update??
How do I run elevenlads tts on this, when I select it, the quality doesn't seem any better than silero, am I doing something wrong?
Please keep doing this
Ok 😉 Thanks for being there!
Hi, Great video! Can this be used exclusively offline after installed?
Yes, absolutely
I canlt find a way to get much out of it other than the chat bot conversations. I was hoping for things like SD prompts or story writing help, but I mostly get back out the exact words I put in back.
Longer replies are best in something like notebook mode vs chatbot mode
It keeps defaulting to CPU for me. it says CudaSetup:Required library version not found. Anyone know how to fix this?
can you also do a tutorial for running this on our colab notebook? thanks!
Is there something like this for speech synthesis?
Does it support integration with the chatGPT api? I've been working on my own web UI for it so I can have more control over the output and settings but I'm really bad at front end development and I'd love for something to already meet my needs.
Not sure as I tend to avoid services I have to pay for
Hi, I have a problem running it on GPU. I have Nvidia RTX 3080. It shows me an error "torch not compiled with cuda enabled". It is the same when I install with one click installer or line by line with conda (I installed cuda version of torch). If I run nvidia-smi it shows that i have cuda version 12.0 installed. What am I doing wrong?
Nvidia-smi will show system info. Use nvcc for the conda environment, and you should see it’s using 11.7
This + the 4chan leak of LLaMA makes me wish I had a really beefy GPU. I wonder if there's any cloud service that rents you an A100 by the minute.
Hold on for some weird ass company to come out with AI specific cards.
@@USBEN. Yeah because there are just so many desktop GPU manufacturers out there, competition was fierce these few years, that's why GPUs are dirt cheap right now. ☹️
I used the installer because I'm not that tech savvy.
Getting this error: CUDA Setup failed despite GPU being available. Inspect the CUDA SETUP outputs above to fix your environment!
You should probably start by inspecting the outputs you get in order to fix your environment, as indicated? There’s also the GitHub issues, where you can provide the details of what it is you’re trying to do
@@NerdyRodent All I did was install and launch, I don’t know what else I was supposed to do.
@@jamiewongttv Yup, that's it - just install and launch was all I did. Worked first time no problems :) As mentioned, the GitHub repo's "issues" is the best place for issues as there you can explain how you've got your computer set up, etc.
Are there any models that can do larger maximum prompt sizes? I want to try use these to summarize long transcipts of conversations e.g. 3 hours 25k 30k words but chatgpt etc all max out at about 3000 words.
Not that I’m aware of!
Can it generate TTS and Text , rather than one or the other?
Please make a video how to install the Llama 7B 4-bit edition it will really help everyone so much
Hey i have a question, Are there any repositories with pre-made characters? Would love to see what other people make
Not sure, tbh!
@@NerdyRodent I got a violentmonkey script called CharacterAiDumper and it's working, but the backend of Text Generation web UI has changes from character ai's to a yaml one. Even json files are not supported anymore. So I need to fiddle around with making json to yaml converters... I'll see what I can do
Is it possible to install the repository on an external SSD? I asked because I have installed SD automatik1111 on an external drive and it works flawlessly.
Yup - you can install it anywhere you like!
How do the current open source multimodal chatbots without filters compare to gpt-4?
More funny 😉
How can I solve this problem:RuntimeError: MPS does not support cumsum op with int64 input. Can u help me with that more precisely!
and I run it on intel-based Mac.
Unfortunately I don’t have a Mac, but maybe someone can help via the github issues?
can you do the extensions if you do the one click installs? I mean do they install the requirements themselves or can we go into the one click install micro anaconda and install the requirements our selves?
Yup, just install any additional requirements by editing and running again or simply activating your conda environment and running pip install there as shown
@@NerdyRodent How do you activate the conda environment that comes with the one click install?
I tested if Conda was a command on my pc before installind as I do uses automatic1111 a lot and thought it may be on there...but alas no
How would I go by training a model myself of messages I’ve got
How do I download the models from Huggingface? I dont see any download button on the GPTJ-6B Page.
You can just use download script as shown 😀
This is so cool .
Ikr!
I think the language models available aren't good. I want GPT3/4. I'd even save on my coins to get enough storage for the 52TB required.
LLaMA got leaked on 4chan. It's a lot more efficient; the 65B parameter model is supposedly more powerful than PaLM. Check the smaller ones too! All of them fit on "a single GPU", just depends if it costs one or more kidneys.
Look at how Stable Diffusion has evolved. It's just getting started with the text AI. There will be more and more models that will be refined for certain things.
Two more papers down the line!
Hi, great work, great application. Can you please tell me the different real life use cases for this.
There are lots, but some to start with include:
Chatbots
Language translation
Content creation
Sentiment analysis & Personal assistants
@@NerdyRodentthanks for the reply.. can these tasks be done out of the box or they need fine tuning?
and in theory, can it do all the tasks which chatgpt does (not on same quality obviously) or are there any differences???
can I integrate it into Slack to replace myself to deal with my annoying coworkers?
There is an API, so I don’t see why not!
Hm, I wonder why they started a new project instead of colabing with KoboldAI...
What a great video! Ha ha hours of fun
I tried getting this to work but gave me an error that went by so fast I could never figure out what happened to trace it down. I read thru your comments and saw someone mention koboldai. I was able to run that with no problems. That runs on a webui also so maybe fairly similar. It could be something you could do a tutorial on as it seems work easier just looking at the github instructions. Or if someone reading thru the comments can't get this to work try that like I did.
Git clone and then play.sh is essentially exactly the same install process 😉
I tried it and it produced unrealistic answer, I'll keep experimenting
Yup! You can go from wildly inaccurate to as good as it gets… I quite like somewhere in the middle myself 😉
@@NerdyRodent OK, I tried to play with the settings but I couldn't get ChatGPT-like responses. Is that possible with this and what model do you use?
Are the models censored ? Asking for a friend.
You can tell your friend that there are a variety of models 😉
How do you get the web UI in dark mode ?
I tried it but it's incredibly slow. Takes up to 30-40 seconds to generate a simple reply. Unusable. Also I don't have any of the options you show in the video.
Yup, speed is hardware dependent. Slow hardware = slow response
@@NerdyRodent I have a 3070 tho. But I figured it was the model too big. I loaded a amaller model and it's much faster. However, none of the models come close to be as good as Chat GPT. Few lines of conversation and it's clear how dumb they are.
@@RikkTheGaijin You’d need more VRAM ideally, but many of the models are pretty good. Llama is great, and all the alpaca loras, Pythia is pretty good, as is gpt4all - so many to try and new models coming out fairly often!
the one click installer for windows did not give me a micromamba. bat file :(
Personally, I always avoid one click installers and just install it normally
For some reason when I get to step 2 and try to run the 'download-model.bat' file I get this error: 'conda' is not recognized as an internal or external command, operable program or batch file." Anyone else with the same issues on Windows 11?
Beneath that error it also says:
"The system cannot find the path specified.
python: can't open file 'C:\\Windows\\System32\\download-model.py': [Errno 2] No such file or directory
Press any key to continue . . ."
@@tyreelampkin2772 Did you ever figure out the issue? I'm having the same problem.
I just switched back to the other 1 click method he mentioned and it worked
what do i call a model that trained on loras?
I wish I could understand anything of this without having to know so many other things aswel. I am pretty sure your tutorial is very helpful for people who know what anaconda and all that is but for me who is a complete newbie, it's just confusing. :(
Check the link in the video description for how to download, install and run Anaconda on Microsoft Windows. However, AI is quite advanced technology so at least 3+ months of prior computing experience is suggested before moving up to AI.
@@NerdyRodent it's alright, thank you for you reply. :)
I am just frustrated. I don't understand why this stuff has to be so "complicated" for dumbies like me. I wish there would be something like a normal installer that takes care of everything, just like with other "normal" programs, if you get what I mean. lol
sadly I don't have time to learn all this. I guess I will have to wait until the whole thing is more usefriendly
Anaconda is indeed the normal cross-platform installer for a load of python packages. It would indeed be useful to have the usual “apt install x” for everything, but python runs on Microsoft Windows and MacOS too 🫤
@@NerdyRodent the overall process of getting this to work so I can tackle the tavern is beyond me right now, ngl, I tried, and I encountered several errors. I am also not able to write in the cmd window for some reason. q.q
I just gave up haha
I will wait, thank you for your time though, rly!
what model would be the best to answer coding questions?
CodeGen is pretty good
How do we train our old models?
What a beautiful showcase. Nice work bro .
Can Someone please guide me how to install this on macbook pro?
Although untested, it’s pretty much the same! Just follow the notes on the GitHub page for MacOS.
Could we use the leaked LLaMA model surely nobody here has with this?
Yup. Llama is now supported by hugging 🤗 transformers. Just convert the files Facebook research gave you the link too into huggingface format and you’re good to go! Alpaca style Loras work too!
I cannot, for the life of me, get this to work with my GPU which is an RTX 3060. I keep getting the error saying that the path to my CUDA libraries cannot be found. I checked to see where the libraries are installed and have added the path to my environment variables, and I have updated my graphics card to the latest update. I also uninstalled and reinstalled the bitsandbytes because it was saying that was an issue. I will post the full error log in the github for help, but if you have any ideas, I would love to hear them.
When I try the 1-click installer, it tells me that it finds the torch 1.13.1, uninstalls it, reinstalls torch, and then tells me that I have torch 2.0.0 but, it must be installing that for me? How can I get the installer to realize that it doesn't need to uninstall torch and reinstall it with the updated version?
The easiest way is to avoid any 1-click installers. I know I do! Pytorch 2 is very new (released today), so you’re best using 1.13 and CUDA 11.7
@@NerdyRodent For sure, I try to do the same thing. I only used the 1-click installer because the anaconda route did not work for me due to CUDA library path issues. Any idea if I can manually uninstall pytorch 2 and reinstall pytorch 1.13.1 to make this work?
@@bigmanong yup, you can still conda / pip install / uninstall whatever you like!
finally now i can build my own personal assistant
gonna make a group chat with me, ai and a bot that runs commands
so ai can look for weather or what is playing on my device play/pause search stuff on youtube etc via the bot
hey maybe i give it access to my terminal 👀