Khoj - Free & Local AI Augmented Document plus Internet Search
HTML-код
- Опубликовано: 1 июл 2024
- Khoj is your AI "second brain". Get answers to your questions, whether they be online or in your own notes. Use local or hosted LLMs. Self-host locally or use our cloud instance. Access from Obsidian, Emacs, Desktop app, Web or Whatsapp. Make agents, do automations and more! RAG made easy :)
Want to support the channel?
/ nerdyrodent
Links:
github.com/khoj-ai/khoj
khoj.dev/
/ khoj-your-second-10502...
== More Links! ==
* Installing Anaconda for MS Windows Beginners - • Anaconda - Python Inst...
* Installing ComfyUI for Beginners - • How to Install ComfyUI...
* ComfyUI Workflows for Beginners - • ComfyUI Workflow Creat...
* Faster Stable Diffusions with the LCM LoRA - • LCM LoRA = Speedy Stab...
* Make an Animated, Talking Avatar - • Create your own animat...
* Make A Consistent Character in ANY pose - • Reposer = Consistent S...
Chapters
0:00 Khoj introduction
10:13 Khoj installation 
Наука
I wanted to let you know how much I enjoy your videos and humor. They are incredibly entertaining and enjoyable to watch. It's amazing that a technical video can make me laugh! Keep up the great work, mate!
Glad you enjoy it!
I've tried some of these tools and also experimented with building simple versions myself. I think for simple "handful of documents" scenario these can work quite well, but they usually only implement basic index search and context injection, so if you ask questions like "count the number of people mentioned" or any question where you need to look at multiple places in the document, they usually do not work well. You would need a pretty elaborate branching workflow and a switching process to choose processing branches to deal with complex queries, and these usually cannot be built without being designed specifically for a dataset. I think the easiest way to get a semi-decent QA system is probably to build one with Flowise, which allows you to compose LLM nodes similar to ComfyUI.
LOL! So refreshing to see a local LLM that'll just admit, "I don't know." 🤣Many of the ones I've tinkered with in the past were more than happy to lie their digital arses off shamelessly & just make up nonsense. Curious to see how this one will handle being inundated with coding docs to assist me & save me time on the RTFM front. Love that you can use other (local?) models alongside it to power it up, too. Very cool, will definitely be up to some mad science with it soon. Thanks, as always, for the great info! 👏
i wonder which LLM out there is best for coding errors. I had 10 faults of 10 queries with all versions of Llama3 , so a 100% failure rate in its answers. if this one can be "specialized" by adding docs that contain correct answers, that would be interesting, if there weren't the ever existing default hallucinations that come with all AI by default.
"The Twirly Grizzler is both ethical and moral, meaning its twisting motion will not cause harm to any rodents." 😂
😉
Thanks Nerdy!! 😊
No problem 😊
Can it read/reference your whole code base and whatever you give it or ia it limited to token, ram, vram, et c.
what a surprise, nerdy rodent play rpg. He's a (classic )nerd!
:shocked rodent emoji:
I have tried a couple of these AI apps that support collections of documents, but the ones I've tried all have a single collection of documents. Are there any that support creating multiple collections of documents and switching between them at will without having to wipe its memory and retrain against each time you want to switch collections, which could be extremely time consuming when you have lots of documents in the collection? The reason I would like to have multiple collections is so that I can have each collection be specialized in a certain topic. For example: 1 collection trained on medieval history documents that I can use for historical research, another collection trained on D&D rule books so I can look up rules, and yet another collection trained on my own fantasy world and all of lore. Keeping them separate, I could select the collection I want it to use for my questions and not have everything all mashed up together. Having collections dedicated to information about specific topics and being able to select which collection(s) you want it to consider when answering questions seems like it would give better results than having everything mashed together in one gigantic collection.
Yes, collections would be a good idea !
Great content. What’s the difference between this and Verba?
Did you give it a pathfinder pdf?
In my own research, I've found that pdfs will have troubles with tables in llms.
There's a few different pdf data extraction methods. But I don't know which one is best. You might want to even do a video on it!
Yup, I just threw the files straight in. I’ve been asking it more about the toughest enemies, weird builds, potential synergies and things like that over reproducing tables. Watch out for the gnome trickster monk 😃 Haven’t done any pdf extraction tests as yet, but if a tool gave options that would be fun…
@@NerdyRodent I've seen a ton of pdf to txt options.
They have some of the same problems as pdfs, but they're way smaller.
(Which isn't too much of an issue for us with our 3090s)
Hmmm. Makes me wonder. If you import a dictionary of any language. For example Chinese or such. Would it be possible to use this to translate texts??!
Not as well as a model trained on your language and the language you are translating. It would have to do it like word by word which it would have a lot of trouble doing normally. I've really wanted a news bot that spoke all languages and went and surfed all the worlds various internets of various languages. It would be cool to get a report every few hours.
Hunyuan DiT = Tencent image generation comparable to SD3, Sora etc....
Maybe it's an interesting topic ?
Hey there my nerdy friend 🤘😉
👋
@@NerdyRodent hope your week is starting off right! 💗 🐁
@@kariannecrysler640 bank holiday is the best start to a week! Hope yours is going great too 😃
@@NerdyRodent rainy and a tad chilly, but perfect for getting my morning glory seeds in the ground by the fence & my daisy seedlings in there permanent homes in the ground. The less harsh conditions & the soft rain water are the best for the plants when transplanting and seed starting. So far everything that is planted is thriving & my zinnia seeds have sprouted from the dampness. All in all a rather good day here too! 😁
Is that you singing at the end??? I really like it...
work with lm studio? i dont understand ollama
It can be installed on Raspberry Pi 4 ? :D
I can’t say how fast it would be, but probably!
@@NerdyRodent I would like to understand this because the possibility of offering "agents" outside the local network is interesting.
can this show transcribe the audio of a youtube video into text to then feed into an llm?
Yup, just drop your transcripts in 😀
@@NerdyRodent but i asked about transcribing due to not all youtube videos having captions.
@@drmarioschannel whisper can make captions from a youtube video. on a Mac when using PINOKIO, (similar to Anaconda i guess), the app WHISPER-WEBUI needs only your youtube link, then after one single click, and a few seconds later, it makes a caption from any youtube video. if its a foreign language it even can translate that youtube video into english captions.
You can use whisper (from open ai) to do that. Download the video and feed it to it
indos and their saffron themes
I love the functionality but damn in 2024 why it have to be so ugly. We have free UI packages literally everywhere