Sir thank you for this informative short video. Quick question can this be done at scale and automated and provided as a service online and using free resources?
Excellent video. Suggesting to optimize the video more in terms of description and keywords so that you end up getting more search traffic. Please keep up the good work.
This seems to be a real game changer for developers, but something feels off. What are caveats of running groq + llama 3 for free? What are limitations? Why is everyone and their grandmother not creating apps with this exact set up right now?
Their is rate limits (I will put them below), so to use it with any volume you would need an enterprise plan. But I still think this is a game changer not just from what groq is doing but in that you now have a top tier model that is open sourced. This allows us to setup our own high performance llm without having to always go through a few megacap tech companies. the limits for llama3-70b-8192 are 30 requests per minute & 6,000 tokens per minute. Also max 14,400 requests per day.
why I have this error? export GROQ_API_KEY="my_api_key" export: The term 'export' is not recognized as a name of a cmdlet, function, script file, or executable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
its likely you are trying to run export in a windows environment. Try set GROQ_API_KEY=my_api_key...google setting windows environment variables if that doesn't work
Thanks , but It can extract data from images ? I tried llava but it gave me a bad result Always say that the image provided doses not have good resolution ? Can you please make a video about that like you did with gpt4 for conversion img to json
personally, I use Groq for more proof of concept type projects, and do not send sensitive data. A key note from Groq: Notwithstanding the restriction on licensing set forth in section 4, Content Available through the Services, User Data shall be owned by you and is not retained or used by Groq other than to perform the Generative AI Services hereunder. Also, groq.com/security/
You make a good point, nothing is truly free. However, I still think what meta is doing by open sourcing a top tier model is a big deal. It allows the entire universe of cloud providers to host llama, not just being stuck with a few big tech companies gatekeeping. Also puts downward pressure on AI API pricing in general.
Hi , is it imp to download the model and do the setup or just calling api in backend is enough to get ai response does meta provide the api service since i dont know python I rely on api only 💀
Great vid! I could manage to make it work, but not with all the code you gave. Either it writes an empty .md or else I erase some of the code and gives me the cv on the output console but it doesn´t create the .md file. This is the error I get: "line 36, in output_file.write(completion.choises[0].message.content) AttributeError: 'Stream' object has no attribute 'choises'" Could you give me some hints?
Hey Thanks for watching. I will post a link to the GitHub with the starter code shortly. Noticed you have a typo in the word choices, should be: completion.choices[0].message.content
At a high level you want to implement a service class that will handle the API calls(the code from this video would go in that class). Then use the service in your controllers or other components. ChatGPT or Claude could walk you through it in some more detail, good luck!
Sir thank you for this informative short video. Quick question can this be done at scale and automated and provided as a service online and using free resources?
Excellent video. Suggesting to optimize the video more in terms of description and keywords so that you end up getting more search traffic. Please keep up the good work.
Thank you, and thanks for the great suggestion. You are right I will add more to the description
Thank You so much sir for providing such rare information , I can now use powerful ml models for free
That's great. Thanks for the video, sir.
This seems to be a real game changer for developers, but something feels off. What are caveats of running groq + llama 3 for free? What are limitations? Why is everyone and their grandmother not creating apps with this exact set up right now?
Their is rate limits (I will put them below), so to use it with any volume you would need an enterprise plan. But I still think this is a game changer not just from what groq is doing but in that you now have a top tier model that is open sourced. This allows us to setup our own high performance llm without having to always go through a few megacap tech companies.
the limits for llama3-70b-8192 are 30 requests per minute & 6,000 tokens per minute. Also max 14,400 requests per day.
Thanks! Super helpful and straight to the point
why I have this error? export GROQ_API_KEY="my_api_key"
export: The term 'export' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
its likely you are trying to run export in a windows environment. Try set GROQ_API_KEY=my_api_key...google setting windows environment variables if that doesn't work
You're in windows. Export command is for linux/macos. Google the ways
Thanks a lot. It was truly helpful
Thanks , but It can extract data from images ?
I tried llava but it gave me a bad result
Always say that the image provided doses not have good resolution ?
Can you please make a video about that like you did with gpt4 for conversion img to json
I don't think you can use this method for AI vision. Working on a video about AI vision with Claude
Super super good!!!!
Thanks glad it was helpful!
Is your data secure if you run it through Groq? i want to use llama on excel data but don’t want to leak information.
personally, I use Groq for more proof of concept type projects, and do not send sensitive data. A key note from Groq:
Notwithstanding the restriction on licensing set forth in section 4, Content Available through the Services, User Data shall be owned by you and is not retained or used by Groq other than to perform the Generative AI Services hereunder.
Also, groq.com/security/
If it's free your prompt content is being saved. 70b costs lots of money to run on a server like this, it's not free!
You make a good point, nothing is truly free. However, I still think what meta is doing by open sourcing a top tier model is a big deal. It allows the entire universe of cloud providers to host llama, not just being stuck with a few big tech companies gatekeeping. Also puts downward pressure on AI API pricing in general.
So what if the prompts are saved, it just makes the model better
@@jonahsatheesh5346data is the new oil. your personal data is actually worth money. you are trading it to use the product.
Agreed. But for learning purpose it's good. For actual enterprise application your company will bear cost of hosting of Api
it says the term export is not recognized can some one help me? untile 1:42 everything was working
Hey sorry for the late reply. My best advice is use cursor AI code editor and put the error you get in the chat. It will likely fix it right away
@@aiunleashed509 ok im currently not home but when i will get home i will write it
so amazing! Thank you so much
Hi , is it imp to download the model and do the setup or just calling api in backend is enough to get ai response does meta provide the api service since i dont know python I rely on api only 💀
No you don't need to download the model. In this video it's run in the cloud with groq(not meta directly).
Great vid! I could manage to make it work, but not with all the code you gave. Either it writes an empty .md or else I erase some of the code and gives me the cv on the output console but it doesn´t create the .md file.
This is the error I get: "line 36, in
output_file.write(completion.choises[0].message.content)
AttributeError: 'Stream' object has no attribute 'choises'"
Could you give me some hints?
Hey Thanks for watching. I will post a link to the GitHub with the starter code shortly. Noticed you have a typo in the word choices, should be: completion.choices[0].message.content
Any idea what the rate limits are? Couldn't find that information on google :/ I only saw the max tokens per request.
Yes the limits for llama3-70b-8192 are 30 requests per minute & 6,000 tokens per minute. Also max 14,400 requests per day.
How much data can be transferred per request? I'm a newbie
@@ianbraganza3065 6000 tokens per minute. A token is about 3 characters of text, so around 18k characters per minute
how to add this api in notion
Hi,
Is there a way for the code from this video to be posted?
Yes, here it is: github.com/AI-Unleashed/GroqResume
Amazing. Thank you. got my sub :)
Thank you, please post source code next time ! :)) I havу tested it, works perfect
Wait what so this is completely free?
Yes! at least for now :-)
i want to use it in spring boot project how to do that
At a high level you want to implement a service class that will handle the API calls(the code from this video would go in that class). Then use the service in your controllers or other components. ChatGPT or Claude could walk you through it in some more detail, good luck!
You're awesome!
Thanks, comments like this keep me going. You're awesome too!
thanks
when i do the 'python project.py' step in terminal it doesnt work. how to fix this?
what error do you get?