That was your video from a year ago that introduced me to the fascinating possibilities of what you can do with today's LLMs and countless tools. Since then, I have created many gadgets to enhance my everyday work. Thank you so much.
Get the Code here: github.com/Leon-Sander/Local-Multimodal-AI-Chat Let me know if you enjoy the Chat App 🔥 If you want to give some support: buymeacoffee.com/leonsanderai Business/Consultation Contact: leonsander.consulting@gmail.com
Really nice interface ! hope one day you will do a more beginner friendly tuto...i install ollama, docker but couldnt pass "docker compose up" instruction without the error " no config file provide" 😅.. i will come back in few weeks, thanks anyway
Did you manage to implement a responsive voice chat mode? I'm currently using Open WebUI for about everything else and LocalAI is also pretty good. But I haven't really found a good working voice assistent yet. The projects that exist only support English and usually can not handle interruptions properly. Though I haven't checked if that has changed in the past few weeks. ;-) Finally there is also this memory issue. There was MemGPT quite a while ago and there are apps here and there that can write summarizations to a database. But I'm not aware of something that actually saves notes and also retrieves them back in context in a smart way. On the other hand, even ChatGPT is not very good at taking notes.
The underlying voice model is whisper AI, you can specify the version you want to use from huggingface. It generally supports multilanguage input. Also I think interruptions should not be a problem. This kind of memory feature is not supported yet.
@@leonsaiagency Input is the easy part, the problem is proper output and responsivity. The amount of mutlilingual fast TTS models is quite limited. Also you want to listen while talking to respond to "STOP" commands and early responses. I tried to modify a project called June on github and had Claude come up with some algorithm to bascially remove output while the AI is speaking from the incoming audio stream. I wasn't happy with the audio output though and it's all too complex to be fixed in in an hour here and then on weekend which is quite frustrating.
@@testales it seems that you’re talking about an advanced interactive voice mode, this is not supported in this repository. The UI is build with streamlit, which gets rerendered for each UI change. Implementing this interactive advanced mode would be very complicated. Also yeah it’s a lot of effort, this repository took me many weeks with almost daily effort.
what about the code in your previous video will it work fine now cause i am follwing it and getting alot of errors also the code in that video is not the same as in the repo (that video is 8 months older)
Generally it should work. The code from the older video is provided under another branch in the same repository: github.com/Leon-Sander/Local-Multimodal-AI-Chat/tree/YTVideoCodeVersion The requirements there contain versions which worked at that time, this might reduce errors.
That was your video from a year ago that introduced me to the fascinating possibilities of what you can do with today's LLMs and countless tools.
Since then, I have created many gadgets to enhance my everyday work.
Thank you so much.
Amazing, that's exactly what was intended with the video.
Get the Code here: github.com/Leon-Sander/Local-Multimodal-AI-Chat
Let me know if you enjoy the Chat App 🔥
If you want to give some support: buymeacoffee.com/leonsanderai
Business/Consultation Contact: leonsander.consulting@gmail.com
Could you make a step by step Tutorial on how you made this? I have seen your previous videos,but I want to know how you made this using Ollama.
There are people on GitHub marking Ollama apps but I have found no tutorials on the subject. :(
Can we consider this RAG system?
Really nice interface ! hope one day you will do a more beginner friendly tuto...i install ollama, docker but couldnt pass "docker compose up" instruction without the error " no config file provide" 😅.. i will come back in few weeks, thanks anyway
Did you manage to implement a responsive voice chat mode? I'm currently using Open WebUI for about everything else and LocalAI is also pretty good. But I haven't really found a good working voice assistent yet. The projects that exist only support English and usually can not handle interruptions properly. Though I haven't checked if that has changed in the past few weeks. ;-) Finally there is also this memory issue. There was MemGPT quite a while ago and there are apps here and there that can write summarizations to a database. But I'm not aware of something that actually saves notes and also retrieves them back in context in a smart way. On the other hand, even ChatGPT is not very good at taking notes.
The underlying voice model is whisper AI, you can specify the version you want to use from huggingface. It generally supports multilanguage input. Also I think interruptions should not be a problem.
This kind of memory feature is not supported yet.
@@leonsaiagency Input is the easy part, the problem is proper output and responsivity. The amount of mutlilingual fast TTS models is quite limited. Also you want to listen while talking to respond to "STOP" commands and early responses. I tried to modify a project called June on github and had Claude come up with some algorithm to bascially remove output while the AI is speaking from the incoming audio stream. I wasn't happy with the audio output though and it's all too complex to be fixed in in an hour here and then on weekend which is quite frustrating.
@@testales it seems that you’re talking about an advanced interactive voice mode, this is not supported in this repository. The UI is build with streamlit, which gets rerendered for each UI change. Implementing this interactive advanced mode would be very complicated.
Also yeah it’s a lot of effort, this repository took me many weeks with almost daily effort.
what about the code in your previous video will it work fine now cause i am follwing it and getting alot of errors also the code in that video is not the same as in the repo
(that video is 8 months older)
Generally it should work. The code from the older video is provided under another branch in the same repository: github.com/Leon-Sander/Local-Multimodal-AI-Chat/tree/YTVideoCodeVersion
The requirements there contain versions which worked at that time, this might reduce errors.
@@leonsaiagency thanks will try again now
@@Danyal_alam also make sure to check github issues, when encountering errors: github.com/Leon-Sander/Local-Multimodal-AI-Chat/issues?q=