Setup Tutorial on Jan.ai. JAN AI: Run open source LLM on local Windows PC. 100% offline LLM and AI.
HTML-код
- Опубликовано: 2 авг 2024
- This is the BEST YET. With Jan.ai, you can bring AI power to your desktop and AI powered applications. This is new exciting AI product that includes tons of source code for it too. Run LLMs locally. This is a 100% offline ChatGPT alternative.
I will walkthrough details this new exciting AI product and how to set it up on actual computer hardware, how to use it to chat with AI on your private local computer using multiple different LLMs. This is Private GPT.
We use their open source code, MAKE files, CUDA software, Local Large Language models with billions of parameters, and high end video card that handles some of the AI processing. Run Local LLMs on your own hardware.
This is an open source ChatGPT alternative. To be 100% AI offline mode, and and off the grid. This is local AI on a PC. This is local AI on Windows 11. Install fully working GUI for AI Chat application, and similar to OpenAI in many ways.
You get to run your own ChatPGT-like LLM on your Windows PC.
LINKS
---------------------------------------------------------
GUIDES ON JAN.AI
jan.ai/guides/
CUSTOMIZE THE JAN UI
jan.ai/developer/ui/ - awesome UI diagram here!
Troubleshooting JAN AI
jan.ai/guides/troubleshooting/
GPU IS NOT USED
jan.ai/troubleshooting/gpu-no...
MORE INFO ABOUT NVIDIA
github.com/janhq/jan/blob/3bf...
DISCORD GROUP
/ discord
Timecodes:
---------------------------------------------------------
00:00 Intro
02:13 Jan.ai Website
05:45 Developer Docs
06:54 Github repo
08:04 Pre-requisites to Install locally
09:15 Clone the repo, "make dev"
09:40 See it running briefly
10:37 Enable GPU Acceleration
11:15 Parameters and more
12:05 The Hub of LLMs to choose
16:27 Asking it Questions
21:10 Config file
21:45 Begin LLM download from Hub
22:22 cmd line work needed
24:44 the Makefile
25:50 Licensing Info
26:34 Start looking at code blocks
27:40 Customize Source Code
28:25 Many LLMs to choose from !!!
29:50 Kind of like ChatGPT
32:30 Change LLM Models
37:45 Show the running Electron
38:33 CUDA setup
40:36 Review Github, Stats, Activity
43:53 CUDA Samples, VS.NET
44:48 Windows 11 Instructions
46:20 CUDA Toolkit
47:50 Close out 
Наука
Great guide thanks!
thanks for watching, glad it was helpful.
Amazing thank you!
thanks for watching. Glad you liked it. This is amazing stuff happening these days.
lmao i died at "i will send you by email"
yep that was funny. Got to find humor where we can lol
Hey I have a question, do you know how to turn .bin files or pytorch files to a gguf or something compatible for import with Jan? I'm loving the UI obviously coming from terminal ollama but I would like to get my LLM's put onto this and I really don't know how(to be honest, haven't put more then 10 hours into learning and downloading LLM's and as you know, that's mostly downloading.
Nevermind found a tool to convert! Just had to install the python requirements and now its doing its thing in another VM. Thanks dude, your video was super awesome and informative!!!
Glad my video helped you! Hey thats great. I am glad you found an answer. What tool did you find?
Great guide Thank you, does jan have web interface can be accessed and used by many users?? Trying to make it accessible by users in network by URL
:-) Thanks for watching. I will give that a try and see what I can come up with.
Yeah that's gotta be it. Sorry. HAHAHA. I feel dumb for leaving this last comment now though. But you will boost your views so much leaving comments as they are, the likes go up, comments get posted. Algorithm sends more people your way. Totally a win win :)
How do I change the installation location? I have it installed on the C drive, but I need it to be installed on F. Since there is no room on C for large models
Let me see if I can find out, if you haven't already discovered a way to do that.
Hello, I am looking for a way to feed a chatbot daily with emails (I would copy & paste the content of each) I receive from different people, and then be able to ask questions about that content. Does anyone know how I can do this?
What operating system are you using? can you explain in more detail about your configuration and tools you are using?
I keep trying to install CUDA but it won't work. I have a GTX 1660 Super with the most recent Nvidea driver installed. Anything I'm missing?
What kind of error(s) are you getting?
CAn you train the local models like upload your documents?
Thanks for watching, check this link it might help you jan.ai/docs/use-tools
does anyone know how to get the offline api key for jan?
Do you approve your messages? Lol im so confused
Appreciate this vid. Although a suggestion- tone down the music. Gets quite distracting
Thanks for watching!
Which language model can i run in my i5 3570s 4GB Ram without GPU machine?
Hello. Just checking, did you get it up and running on your computer yet?
@@ScottDevOps no
Is Jan uncensored?
Jan AI allows users to adjust their settings so they can change alignment, moderation and censorship levels to their needs on LLM settings.
IN the settings I dont see "enable gpu acc", how come is that?
Hello there.
Have you seen this guide yet:
github.com/janhq/jan/blob/3bfa2773bdee03646e89e38367ef76a7b97a66af/docs/docs/guides/08-troubleshooting/03-gpu-not-used.mdx#L2
A few questions for you: Which Nvidia card are you using and how much RAM does it have? What OS are you using?
Did you install it with the EXE or use the actual github source code like I did in the video?
Some things to consider:
In NVIDIA command line tools, if you don't see the option "Enable GPU Acc" (GPU Acceleration), there could be several reasons:
- Version Difference: The tool or driver version you are using might not support GPU acceleration, or the feature could be named differently.
- GPU Compatibility: Your NVIDIA GPU might not be compatible with the GPU acceleration feature required by the tool.
- Driver Issues: The necessary drivers for GPU acceleration might not be installed or updated on your system.
* Operating System Compatibility:
- The exact operating system you are using might not support the GPU acceleration feature, or there could be compatibility issues.
- Hardware Limitations: Your system's hardware might not meet the requirements for GPU acceleration.
Maybe consider these options if you have not yet:
- Update your NVIDIA drivers and tools to the latest version. Check the tool's documentation for specific instructions or requirements for enabling GPU acceleration.
- Ensure your hardware and operating system are compatible with the feature.
@ScottDevOps GPU is an RTX3070 with 8GB VRAM and 6,144 CUDA cores, GameReady driver is the current latest 546.33 and I installed the nVidia Cuda Toolkit v12.3 (also the currently latest version) CPU is Ryzen 3900 and mem is 2x16GB DDR4, OS is Win11.
All drivers othersie are up to date as well.
And as im writing this and giving it yet another shot, it now runs super fast, compared to earlier where it would be rather slow using the Hermes Neural 7B Q4 model.
Yet in the settings there are still no "GPU accel" option, but oh well, its running super fast, instantly reply when I ask it something and types the answer super fast as well.
So I guess its working really good, but then that begs the question, would it perform even better with GPU accel? X-)
@ScottDevOps also there is this discord for help:
Get help in [Jan Discord] discord.gg/mY69SZaMaC
Just checked compatibility list as well, my 546 gameready driver and my RTX3070 is compatible with Cuda 12.3 it says, so another one to check off the list.
Ill chk the discord and see if I can get further information there, thank you for all your help, its truly and highly appreciated 🙂
Yes, I would think it should perform even faster with GPU support. You can also post an Issue on the github repo github.com/janhq/jan/issues for Jan.AI. The developers of that tool should know why that is happening.
I also noticed many times it was very fast! :-)
PS If you get an answer let us know, Thanks!
Paste no working inside JAN
can you explain a little more what problem you are facing? Are you using Windows 10 or Windows 11, or other OS?