Видео

I haven't seen that error before. If I had to guess it would be because you have a multi-line SYSTEM message but it's not wrapped in triple quotes. Example: SYSTEM """Line one Line two Line three '''""

I have tried with OpenWeb UI and ran into the same issues. I'm assuming there is a bug in the OpenWebUI app that is causing the errors.

What does your file look like? Also, on the model files I created I do not use an extension.

What does your model file content look like?

I currently don't have a video covering that yet, but looking to create one in the future.

I don't add an extension to the model file when I create it.

Solved that. There is an 'error message' in the output after editing the systemctl file for the ollama.service that suggests that it didn't take. I forced a copy of the teporary output file to the output file, then followed up with the systemctl daemon-reload, and systemctl restart ollama.service commands (sudo'ed) and it appears to have taken. Now to get openui to point at that. Adding the server to connections under setup seems to have no effect. Ah well. may have to remove and re-install the docker client on my localhost.

thank you for the suggestion. I always read the comments to see what viewers such as yourself would like to see. Hope you found the video helpful.

@@AIDevBytes Very helpful, i was able to setup the hosting for my wife to use on her computer with this. Side note for future topics when you have time and it aligns to your interest. 1. Step by step incorporating stable diffusion into OpenWebUI would be great 2. How to do RAG properly (from the best system in your opinion, to best practices on how to structure your format for better ingestion) 3. How to allow external access securely? I've seen some videos where Ngrok is used, but their walkthrough was confusing. Thanks again.

Thank for the suggestions

Glad you found the video helpful. Yes, check out this video here ruclips.net/video/kDwEIgmqaEE/видео.html

This video is meant to just to show how to get OpenUI up and running. I didn't want to start with anything too complex, since many people who watch the channel are new to many of these tools. I usually create more advanced follow up videos later. Yes, you can change the style with feedback by continuing with a follow on prompt and the output will be altered There is not presently a settings to keep the style consistent. I would personally only use this app for prototyping and then further refine the design in something like VSCode. Also, the model you use is going to dictate the quality of output. I find that using the OpenAI GPT API generates pretty good UI code, but I like to focus on free things on the channel. I may create a video later using Groq to generate the UI.

You could try to change the system prompt to see if it will create python or Java code for you, but it will not render a preview for you. Since Java and Python UIs aren't HTML based natively. This tool is meant to primarily be used developers that by develop who use web frameworks that use HTML and JSX as their native UI output.

You will need to download the Llava model into Ollama. Then set that as your model in OpenUI. I will say that using an image to generate the UI does not give great results based on my testing.

@@AIDevBytes Thank you for your help. I didn't read the documentation very carefully :D I was using the wrong version of the llava model. Everything works. Question is there any model from open source portfolio that can render html from image in GPT-3.5-Turbo quality? For example some model via LM Studio?

Glad you go it working. I had sent another message but deleted it since I realized that would not work because we need a better open-source multi-modal first to generate better HTML

@@AIDevBytes I skipped docker and used directly the frontend application vitejs with a little code modification I was able to connect it to LM Studio (windows) where there is a greater possibility of open source models with Vision Adapter and I achieve much better results than through docker and ollama llava model. The advantage is that with LM Studio I can use one API called Multi Model Session which allows me to achieve better results when generating html code from images without the need to use the paid version of GPT-4o. But need high VRAM of GPU.

Thanks for the info. I will have to try it out.

I will look creating separate videos on those two topics. Thanks for the feedback.

Glad you found the video helpful. I will have to look at getting stable diffusion working correctly on my M3 Mac. It was not the best experience the last time I ran it.

@@AIDevBytes thanks for the reply! - got stable diffusion 2.1 working on my 1max mac, I'm able to generate images with it by it self but stuck having it integrate with WebUI. i can use Dall-E 3 for now np. just hopping for something free to use lol

Check out these vdeos here: Windows Setup - OLLAMA | Want To Run UNCENSORED AI Models on Windows?: ruclips.net/video/E5k_Ilnmehc/видео.html Mac Setup: OLLAMA | Want To Run UNCENSORED AI Models on Mac (M1/M2/M3): ruclips.net/video/03J_Z6FZXT4/видео.html Models can be found here ollama.com/library

Ollama run llama3:instruct will pull and run the latest instruct model

Correct, if you just want to pull the model run -> ollama pull llama:instruct

I'll take a look at lobechat. I have not personally used it, but it sounds interesting.

I have updated the description with Chapters. Jump to the chapter "DALLE-3 Pricing" in the video. I explain pricing there. In the end your cost is going to be determined by the usage of the DALLE-3 API. The more the use the API the more it will cost. Also, note image generation is not tied to tokens like text generation. I am using free models for text generation in this video using Ollama and the Llama model. Videos for setting up Ollama can be found here: Windows Setup ruclips.net/video/E5k_Ilnmehc/видео.html Mac Setup: ruclips.net/video/03J_Z6FZXT4/видео.html

I'll have to take a look at it. I know people seem to use it with Stable Diffusion. I personally don't do a lot with Stable Diffusion because of the hardware requirements that most people need and the clunkiness of the UI.

@@AIDevBytes sdforge was created by the a1111 it works great for low gpu users would be awesome to use in this Ui. I see it only has 3 options. Is there away to add a different image model

Since OpenWeb UI is an open-source project anyone can edit the code. But there is no way to currently add model to the image section. You can add new model for chat. If you check out video 1 in the series I setup OpenWeb UI with Ollama ruclips.net/video/D4H5hMMoZ28/видео.htmlsi=QFr1LzQhONF4mbgp

You bet!

Check out the video description. I have my Mac book specs below.

@@AIDevBytes Awesome thanks. It will help get idea for a RIG Thanks Man Keep it up!...

In this video, I demonstrate local access only. I will explore how to host this on a server and create a follow-up video on the process. Note that hosting on a server will incur costs due to GPU pricing.

@@AIDevBytes I've got homelab and hardware. Just can't seem to figure out the networking side of things for some reason. Tried proxy through Apache but it must be over my head as I can't get it to function correctly.

Gotcha, yeah unfortunately I wouldn't really be able to help there since I don't know your network configuration or topology. You should be able to go into your router settings and setup port forwarding to you homlab server.

Please send all proposals through Up work and someone on the team will reply there. Thanks

@@AIDevBytesBoss, I've run out of connects there.

Sorry the individual that handles openings only accepts proposals through that platform unfortunately.

Feel free to send your Upwork username and we can send you a proposal request. You can email here inquiries@aidevbytes.com Thanks

Glad you liked the video. I'll look into creating a video based on your suggestion.

Looks like you didn't close your double quotes at the end of your template. Simple mistake which can drive you crazy 😁 Let me know if that fixes your issue. EDIT: Also, use triple quotes like this when using multiple lines for your template. TEMPLATE """ Template values goes here """

@@AIDevBytes getting same error with this # Set the base model FROM llama3:latest # Set custom parameter values PARAMETER num_gpu 1 PARAMETER num_thread 6 PARAMETER num_keep 24 PARAMETER stop <|start_header_id|> PARAMETER stop <|end_header_id|> PARAMETER stop <|eot_id|> # Set the model template TEMPLATE """ {{ if .System }}<|start_header_id|>system<|end_header_id|> {{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|> {{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|> """

When I get some free and I'm at my computer again today. I will give it a try to see if I can isolate the problem and let you know.

@@AIDevBytes thanks mate. much appreciated

@@hamzahassan6726 I copied the model file content you had and pasted into a new file and was able to create a new model. I am not quite sure why you are the getting the error: "Error: unexpected EOF". I have not been able to duplicate the error. Also, one thing to call out looks like you are not using the llama3 template from ollama, but that doesn't appear to be causing the issue. I would make sure you are not using rich text format in your model file and ensure that it is plaintext only. if you go to the llama3 model (ollama.com/library/llama3:latest/blobs/8ab4849b038c) the template looks like this: {{ if .System }}<|start_header_id|>system<|end_header_id|> {{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|> {{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|> {{ .Response }}<|eot_id|>

for this architecture you would not need to to retrain the model, since it's using RAG (Retrieval Augmented Generation) to find the answers in the documents. I will say this solution is really for users to run it locally on their computer, since the document store is running in a container on the users computer. This solution would not probably scale well for multiple users across multiple computers. You would want a more enterprise or custom solution if you are going to run this for your business with multiple users on multiple computers.

I don't have to have the M3 Max, but it definitely will help. I say the minimum RAM you will need on Apple Silicon is 16 GB to run the smaller models. With the M3 Max you will be able to run most of the Open-Source models up to about 40 billion parameters, just not the really large parameter models like the 70 billion parameter models. Also, happy you are finding the content useful.

Did you already have docker installed or did you install a new version of docker?

@@AIDevBytes yes I have docker desktop installed, but there is a newer version. Should I update ? Current version: 4.29.0 (145265) New version: 4.30.0 (149282)

I recommend running the latest version to see if that fixes the issue. I am running the newest version of Docker.

@@AIDevBytes it worked thank you

glad that fixed your issue

Can you explain further? Do you mean add you own text to the SYSTEM parameter? Not sure I'm am following your question.

By default the HuggingFace sentence transformer model is used (sentence-transformers/all-MiniLM-L6-v2) for embedding. You can change it to use whatever model you would like by going to your "Document Settings".

This a very simple example, but the purpose would be if you wanted to change multiple parameters as part of the model and use it in another application. Example, you could use the model with something like Open WebUI and then lock users into only using the model you customized with your new parameters.

did not have good gpu,sorry

Yes, I'll create a video of how to run Ollama in a cloud environment. I'll see how to setup Open WebUI also in a cloud environment. Probably two different videos at later dates.

Yes, as long as the notebook is running on a computer with a GPU.

Yes, check out this video ruclips.net/video/03J_Z6FZXT4/видео.htmlsi=rzzfODttC8Qr5m8h

For coding specific models yes codellama. Which parameter count is going to depend on your hardware capabilities. If you have hardware that can run the 70B model I would use that.

@@AIDevBytes which model is good for overall questions? like everyday things etc

For open source models I like llama3 for general purpose use. I also like Mixtral:8x7b if you have the hardware to run it.