Setting Up a RAG Demo on NVIDIA AI Workbench
HTML-код
- Опубликовано: 2 июл 2024
- Walk through a complete start-to-finish installation of a generative AI retrieval-augmented generation (RAG) system with Lee Bushen, solutions architect at NVIDIA.
You can use this system to create a RAG project on your own computer (no local GPU required) in about 1 hour.
This approach uses NVIDIA AI Workbench to install the system. NVIDIA AI Workbench is an easy-to-use developer toolkit for data science, machine learning, and AI project development. AI Workbench is free, and you can install it in minutes.
For more details, see the resources listed below.
######################
CHAPTERS IN THIS VIDEO
######################
0:00 - Intro and Prerequisites
02:01 - Create GitHub and NGC Accounts
05:39 - AI Workbench Install
09:46 - Clone/Launch the RAG Project
16:49 - How to Re-Enter the API Key
18:35 - Example Prompts
23:11 - Stopping/Restarting the Demo
Project storage and sharing: github.com
Used for API calls to optimized LLM: ngc.nvidia.com
Download NVIDIA AI Workbench: www.nvidia.com/en-us/deep-lea...
Hybrid RAG example project: github.com/NVIDIA/workbench-e...
Sample user data files:
NVIDIA H100 data sheet: resources.nvidia.com/en-us-te...
NVIDIA AI Enterprise Solution Overview:
resources.nvidia.com/en-us-da... 
Наука
This is so far the most complicated RAG system setup I have ever seen. Good luck to anyone who is going to actually set it up and use it.
Its one time setup. I found this video tutorial easy to follow and it worked for me without any hassles.
Excuse me Sir. I have get the Application Error you describe. Im unsure If I have correct listened... You said I shall update BIOS? Isnt it? I dont see any relationship between BIOS and WSL2, as I have run Ubuntu over W11 in this PC before... Is it needed to install WSL nvidia graphic drivers to been able to use CUDA in workbench?
Hi there. If you get an error installing WSL2 you could try first to install it manually using wsl --install. If this works, then you can just run the workbench install again and it should recognise that WSL2 is already installed. If a manual install doesn't work, it's often because Virtualization extentions aren't turned on in your PC BIOS. The name of the setting varies depending on which BIOS you have but look around the CPU settings. I also hear that there may be some conflicts between Hyper-V and WSL2 so try to disable Hyper-V. If you're running Windows itself inside a VM, you might have other problems with WSL2 and nested virtualization. As this is a WSL2 error, I'd search on the web for articles about WSL2 and virtualization. It isn't anything to do with the NVIDIA drivers in the Linux VM. This demo doesn't need those because the inferencing is done by NGC in the cloud, not locally. Thanks for watching and good luck!
Disabling hyper-v didn't solve the issue. WSL2 is installed, and i can check it over powershell or cmd. Nonetheless,when double clic the nvidia desktop link to AI workbench, it says "Application Error" "something happened, please contact support or try restarting the application And installer. The PC is AMD Ryzen 7 5800X 4.2 GHz, 32GB Windows 11 Pro 23H2, RTX 3060Ti. I ve tried on a different PC at the University and I get the same issue
@@carlosrm8091 hard to say what the issue is without looking at the application logs. RUclips won't let me post a link to the Issues forum, but if you search "NVIDIA Forums AI Workbench" you will see some posts that may help.
The open chat cannot launch the webui. the output for chat saying that: stat: cannot statx '/var/host-run/docker.sock': No such file or directory
groupadd: invalid group ID 'docker'
usermod: group 'docker' does not exist
Any idea how to fix this?
Hi! Let me speak to somebody and get back to you to see if it's a known issue.
The Hybrid RAG Project requires Docker as the runtime. It won't work with podman. Which runtime did you select when you installed?
@@tylerwhitehouseus6310 Thanks. I selected podman. I think this is the reason for the error I saw.