hey thank you so much! i appreciate your video and your effort, there isn't many content or people that talks about this im so happy that i found someone who does. i have a youtube channel in Spanish that im trying to start. if you ever want to collaborate let me know. i am going to run 3gb in a laptop using stack. i dont know much about coding and it has taken me a long time to just got to the model. your video is validating and inspires me to continue on thank you so much.
@@anop917 no worries... You can try it, currently gives 5 dollar free credit for sign up... I signed it one year back I got around 25$ free credit... Still 5$ is good enough...
@@anop917 no worries and get don't get worries with these weired requests :) I run a youtube channel so I know it :)... Yes they now give 5$ free credit...
Sure thing - I have a windows PC with 2x 3090 Nvidia cards. This video just uses one. If you are just trying this out and I would use a lambda labs server or one of the other similar services.
@@anop917 Thanks! I have an Ubuntu system which has nvidia 1GB of GPU and hope it will work. I have one more dumb question like, Is it possible to run over the mobile os basically on Android 11 later without any interaction with the server basically set up to the Android app using flutter or native android. Currently checking the possibility of running the llama 3.2 1b over the Android/flutter with RAG
@@muzammildafedar1909I don’t think the current version of llama stack supports android but I can’t imagine that eventually they won’t be supporting edge devices especially for the smaller 1 and 3 B models
Thanks so much for this! First walkthrough I found on youtube! This helped so much-- especially for my ADHD self trying to get through the docs ;)
Glad I could be of help. Here is the short blog article that goes along with that: shwetank-kumar.github.io/posts/llama-stack/
hey thank you so much! i appreciate your video and your effort, there isn't many content or people that talks about this im so happy that i found someone who does. i have a youtube channel in Spanish that im trying to start. if you ever want to collaborate let me know. i am going to run 3gb in a laptop using stack. i dont know much about coding and it has taken me a long time to just got to the model. your video is validating and inspires me to continue on thank you so much.
Thank you and all the best!
@@anop917 you too!! all the best! ty
Thanks
Thank you so much 🙏🏾
Thank you for this, really helpful... Any idea how can we configure with the together api?
Sorry I havent looked into that. Does Together API have a free tier? Happy to look into this. You basically will have to choose them as a provider.
@@anop917 no worries... You can try it, currently gives 5 dollar free credit for sign up... I signed it one year back I got around 25$ free credit...
Still 5$ is good enough...
@@anop917 no worries and get don't get worries with these weired requests :) I run a youtube channel so I know it :)... Yes they now give 5$ free credit...
Very helpful video… curios about your hardware setup can you please share specs ?
Sure thing - I have a windows PC with 2x 3090 Nvidia cards. This video just uses one. If you are just trying this out and I would use a lambda labs server or one of the other similar services.
Nice! Would we be able to run 3.2 1b using llama stack on our local system?
Depends on what your local hardware config is. If you have an Nvidia gpu you will be able to. It does not yet have support for Mac systems
@@anop917 Thanks! I have an Ubuntu system which has nvidia 1GB of GPU and hope it will work. I have one more dumb question like, Is it possible to run over the mobile os basically on Android 11 later without any interaction with the server basically set up to the Android app using flutter or native android. Currently checking the possibility of running the llama 3.2 1b over the Android/flutter with RAG
@@muzammildafedar1909I don’t think the current version of llama stack supports android but I can’t imagine that eventually they won’t be supporting edge devices especially for the smaller 1 and 3 B models