How to Create Agent Swarms With the NEW OpenAI Assistants API
HTML-код
- Опубликовано: 16 июн 2024
- Build your own AI dream team and take control with OpenAI's new Assistants API. 🤖✨
📝 Colab: colab.research.google.com/dri...
🚀Custom AI Solutions: www.vrsen.ai/
🤝Need help with your next AI project? calendly.com/vrsen/ai-project...
🐦Twitter: / __vrsen__
Into - 0:00
Building Assistant Swarm - 0:37
Testing Autogen 2.0 - 5:58
Future Improvements - 7:53
Adding New Agents - 8:43
Conclusion - 9:25
About: Learn how to create Agent Swarms using the new OpenAI Assistants API! Watch as we recreate AutoGen from scratch with just a few functions, featuring a stock price chart example. Discover how this highly controllable and customizable system is deployable in production. Stay tuned to find out how to add your own agents and explore the future improvements in this fascinating video tutorial. 
Наука
Its amazing to see youtubers popping up left & right for creating some sort of agent-system by their own.
Thank you for sharing! Very nice approach to the problem. I really like your setup, because unlike others it is easier & more transparent!!
Best wishes, earned a sub.
I just can’t fathom what kind of power these tools hold. I don’t see what is stopping us from creating a virtually infinite amount of online businesses that run with these agents as employees. I mean if their runrates are cents on the hour with a few thousand dollars you could test and validate hundreds of business models through cold outreach and optimise each one iteratively. Are you planning on making such an automation of automation? Since I noticed you spoke about making an open source software that uses agents to create agents, I’m sure this idea has popped up in your mind as well
This is exactly the idea I had in mind when I saw chatdev but it didn't occur to me to do just use the agents to test and validate as many business ideas as possible. I want to create agents that are virtual employees, including the CEO and I would be the business owner and the human in the loop making sure everything is running smoothly.
Who will buy goods if people will lost their jobs because of ai agents?
Working on this project right now. Anyone would be able to create their own agency using agent swarms
so you will die of hunger because an AI managed the grocery data table?@@32zim32
gi
Literally blown away by the value of this!
Dude, this was great. Awesome work, and kudos for both finding and dedicating the time to work on all this. Will be following along closely!
this just worked better than anything!!! Insane, would love to make it able to learn to create additional tools and coordinate the team. The possibilities!!!
You are a genius! Honestly, this is the only channel I need to subscribe....Please continue in this topic.
Thanks so much for the great video explaining the possibility of autonomous AI assistant swarms! I appreciate you taking the time to clearly and concisely walk through how these assistants work together to accomplish tasks. Your step-by-step examples really helped illustrate the collaboration and decision-making processes. Keep up the excellent work and please continue putting out more content on the topic! I would be really interested to learn how to create an assistant that can create other assistants and tools. Stay awesome and always remember that knowledge is power ;)
+1 for expanding on this concept and project. It has so much potential!
This is the best video of showcasing usable AI agents in the production. It’s very detailed and intuitive.
Glad it was helpful!
Literally , 10/10 video and quality.
Everything is on point and the editing is exellent. keep up the great work!
P.S. i hope you actually continue working on the agent swarms idea and project and make it open source cuz its a interesting topic and powerfull tool if done correclty.
Yes, I will!
Awesome video, honestly you deserve more views haha, super concise and high value
Thanks, man!
I keep coming back to this video! Would love to see you build on this with more agents and get you take on ways to manage the level of human via the Assistants API. Perhaps considering retrieval and vectorstore memory? I'm going to try to build this into a Chainlit app. Cheers!
Great video. And Colab. Thank you so much for this. Would love to see this turned into an open source project.
this is amazing! I want to implement a single persistant thread to have multiple agents running multiple aspects of my personal life.. Its only a dream as i dont know how to code but saw some amazing things these days
Absolutely excellent technical explanation - He makes it look easy
🤯🤯🤯🤯🤯
Reminds me of this 🤯 "robots building robots" moment in Star Wars. Applied recursively it has mind-blowing potential!
Subscribed to your channel, you're excellent at simplifying/explaining!
please never stop creating videos! SUPER GOOD!
Of course, I won’t)
Amazing video! I stumbled upon this video, and thought for sure that you'd have more subs. Hope you get the attention you deserve!
First time see ur videos, immediate sub, no hesitation. Very good job Sir
Great video, I am not a programmer but a creative / psychologist type, I have developed the zero metaphor as an evolutionary concept, intially started as an emotion regulation concept, and later turned into a conceptual algorythm. More to the point, evolutionary concepts are problem solving based, where "0" represents problem resolved. This is an integral concentric idea that is both eco friendly and non-aggressive, ensuring balance. I would post more here but is not the place. Thanks again
There is some serious content here! Great Work Arsenii, and thank you for sharing.
Thank you for watching! I appreciate your support
New subscriber! Great content man. Also 100% about scaling small AIs together instead of just one big centralized model.
I agree)
Love this video! Best explanation of swarm of agents!
Glad it was helpful!
New subscriber. Would be awesome to see an open source project from you and a video showcasing the "agent creator agent". Thanks for the work you put in!
Noted!
I really appreciate how you simply explained the entire assistant API workflow. I’m dont know anything about coding but i was still able to follow through!
One question: Can you have a function to get an already created assistant? And then start thread? I dont want to create assistant everytime i start a conversation.
Sorry I’m really new to this and using a low code platform. Thank you
Yes, I should have added this to the notebook. Great suggestion. Thank you
I’m totally excited about agency swarm.
Great video. I have a lot to learn from this, to say the list. Bravo! Excellent! Do more such videos!
Very cool video and well explained. Keep going dude.
Wow! Thanks for explaining this.
Wow! That was a really good, no nonsense video. Very instructive. What do you think of David Shapiros approach to agent swarms? He goes for a more hierarcical approach, where agents create agents.
I greatly respect him and his approach, but I believe for now we should be focusing on more practical ways to use them with some additional guardrails and controls.
Very cool video. Liked and subscribed.
AGI: 1 super-intelligent model controlling thousands of smaller models & tools
Agents are relying on LLMs anyway.
Great video as always. In the tests I did, I noticed that the response is a little slow. Is it something related to the API? Is there any way to make it a little faster?
As always awesome Video :)
I'd really like to see the VrsenGPT OpenSource project with multiple agents 😊
Could you say, that a Thread is basically the chat memory?
And can multiple assistants share one thread? For example, if we'd have a UserProxy, a coder and a reviewer and the user asks the coder to do something which should in turn ask the reviewer for feedback. Is it possible to have the coder-reviewer loop in a different thread which gets deleted later on than the user-coder interaction? Or if you add a uTester to the second thread so that the 3 of them interact via one thread and the user with via another? If yes, how would you chain this together?
pls
Yes, thread is chat memory. I believe they can be shared, but the best way to allow multi-agent conversations is through functions. Otherwise, it will be extremely confusing to determine who is who for the model. I am working on this capbility for the open source version of this project.
@@vrsen Thanks for the answer, I'm looking forward to the project :)
I was just thinking that there may be cases where the ManagerAssistant doesn't need the whole conversation of a Coder and a reviewer, its just interested that a certain file was created and is now available for following task. There maybe a separate thread for the CodeReview process would be smarter than to put everything in one Thread which at some point gets truncated?
would love to see this evolve
I tip my hat to you sir!
Subscribed
great video thanks a lot !!!
This channel is gonna blow up
Phenomenal, thank you 🙏🏽
Glad it was helpful! Have fun!
Huge creative thoughts injection!
Excellent work.
I definitely want to see how an agent can create other agents and tools
Phenomenal work
Great video buddy!
Great fucking content I have been trying this even before assistants came out with autogen but 8K token and memory where an issue. Assistants & 128k context fixed all that and now this is the last piece! TY!
FYI Autogen has added assistant support but it's not too robust yet it's in beta package but it's pretty awesome. I was just planning on waiting for that to be my solution.
Nice, thanks for the info
Wow, you are great. Can your project also helps the repository of David shapiro on Agi?
Very nice video man. Thanks for the content. Any idea how to create an efficient web scrapper and browser automation agent? As webpages' htmls are very long, they can reach context window and use too many tokens
Yeah, that’s true. I think openai uses some form of pagination with web browsing, where the model has a capability to scroll pages
That was a great video thank you. I do have a question what would you propose for building agents that that can access the internet to call API do post and get request bring back data to infer it. I am working on a project that is answer / question where the bot has list of predefined questions I.e (name, age, phone, address,etc) in witch it gathers data from the user instead which I have working very well. I would like to take that basic info to process. it via the APIs Do you think this is feasible with this proposed agent structure as opposed to function calling? thank for ant feedback.
Good Educational Content
FYI there is a failure of direct retrieval with GPT-4 using the new OpenAI Assistant API. GPT tokenizes text and creates its own vector embeddings based on its specific training data. The new terms and sequences may not connect well to the pretrained knowledge in GPT's weight tensors.
There was no semantic similarity between the new API terms and GPT's existing vector space. This is a fundamental issue with retrieval augmentation systems like Rag - external knowledge is not truly integrated into the model's learned weights. Adding more vector stores cannot solve this core problem.
The solution is to have multiple learned "knowledge planes" with trained weight tensors for specific tasks that can be switched in. This is better than just retrieving separate vector representations.
Interesting, I will test this out. Thanks
how can you further add new agents to this one? How would user proxy call them?
this is awesome is it only possible to user openai models for this.? or can you also include local models
Only OpenAI. I would not recommend creating agent swarms with any other models, because no OS models currently support function calling.
Bravo!. im just looking to run open ai assistant by ingesting my docs, pdf, html file bt from your video i to some other ideas. however can u also drop a video for webbased Swarm
im still learning, but can you tell me what kind of use cases (real world ones) I can have with agent swarms?
As I was going through the code, I realized that the class SendMessage accesses two variables initialized in an outside context: "agents_and_threads" as well as "client." I was considering passing them through the __init__ during class initialization, but it seems that the code doesn't instantiate the class:
user_proxy_tools = [SendMessage]
tools=[
{"type": "function", "function": SendMessage.openai_schema},
]
Is there no way then to encapsulate SendMessage and such classes derived from OpenAISchema properly?
I didnt understand anything. but you explained it really well. haha
Lol, I tried
I would love to see how we can create a 'tool maker gpt/assistant' so when there is a task needed it can create and configure an assistant for that job and add it to the chat until its job is completed an no longer needed.
Yes, I am working on setting up an Open Source project. We will work on adding this gradually
did you have to install yfinance ahead of time? because your agent doesn't have access to terminal, hence it wouldn't be able to install it on its own, right?
That’s great.
Question: How can I find there is no temperature for openai api assistants, so how can I make answers of this system more deterministic ?
Unfortunately, there is no way to adjust the temperature. We have yet to see why openai has decided to do so.
Thank you, @vrsen, for the answer. I have a few questions:
- Regarding the use of `custom_tools + files`, is there a way to use custom tools and files together? I encountered this error: 'file_ids are only supported if retrieval or code_interpreter tools are enabled.' In your system it's available, how I can see.
- Concerning communication between agents, suppose the first agent retrieves some data that I need to use in the code_developer agent. One option is to use the main agent for transferring the data, and another is to send the data from the current agent (maybe these are wrong ways). Are there any other stable methods or best practices for doing this?
- Lastly (perhaps), how can I create an agent for various API calls (specifically for APIs that cause GPT to start hallucinating)?
This is pretty cool.
Thanks man!
Would adding more custom agents to this implementation cause the user proxy assistant to call each of those agents in a row, or does it decide if/when one of them needs to be called?
my thinking is you want to prompt the user proxy to use it's judgement to decide which is the appropriate agent to send messages to, depending on the user query. it won't automatically just call every agent in a row
I love it.......
Thanks ❤
Thanks for watching!
Could you maybe give an overview as to wether Autogen ist still a useful tool or wether on should stick to the Assistants API?
Unlike Assistants API, I wouldn't recommend running autogen in production. Assistants allows you to have a lot more control
@@vrsen So would you say that there's nothing I would be able to do with Autogen, that isn't doable with Assistants API? I'm about to deep delve now into it and don't understand the nuances yet
This seems to be the future of agents however how does one prevent malicious behavior of the agent. How can you stop an agent from getting poisoned input and generating code which for example can upload sensitive data to a pastebin ?
nice one :)
Keep it up
Are there no code tools that can do this?
Liked and subscribed
😊
Fire
any chance to integrate code interpreter and vision?
Code interpreter - yes, you can have this out of the box simply by adding "tools": [
{
"type": "code_interpreter"
}
],
Vision API unfortunately does not support tools yet
@@vrsen thank you.. Great work btw I've tested both autogen and your way and I can say your way is much better.
@@vrsen sorry one more question, each type I run the code in my openai assistant tab the user proxy and code assistant are created over and over again. how do we make sure that we use the same assistants?
Can we use local llms?
Soon 😉
Rumble in the (Ro) tunda!
Is there a no code way to do this?
😎🍻
Isn't the exact same thing as auto gpt that has been out for like 8 months?
Can I pay someone to set this up for me
I bet you can)
°• I believe we are meant to be like Jesus in our hearts and not in our flesh. But be careful of AI, for it is just our flesh and that is it. It knows only things of the flesh (our fleshly desires) and cannot comprehend things of the spirit such as peace of heart (which comes from obeying God's Word). Whereas we are a spirit and we have a soul but live in the body (in the flesh). When you go to bed it is your flesh that sleeps but your spirit never sleeps (otherwise you have died physically) that is why you have dreams. More so, true love that endures and last is a thing of the heart (when I say 'heart', I mean 'spirit'). But fake love, pretentious love, love with expectations, love for classic reasons, love for material reasons and love for selfish reasons that is a thing of our flesh. In the beginning God said let us make man in our own image, according to our likeness. Take note, God is Spirit and God is Love. As Love He is the source of it. We also know that God is Omnipotent, for He creates out of nothing and He has no beginning and has no end. That means, our love is but a shadow of God's Love. True love looks around to see who is in need of your help, your smile, your possessions, your money, your strength, your quality time. Love forgives and forgets. Love wants for others what it wants for itself. Take note, true love works in conjunction with other spiritual forces such as patience and faith (in the finished work of our Lord and Savior, Jesus Christ, rather than in what man has done such as science, technology and organizations which won't last forever). To avoid sin and error which leads to the death of our body and also our spirit in hell fire, we should let the Word of God be the standard of our lives not AI. If not, God will let us face AI on our own and it will cast the truth down to the ground, it will be the cause of so much destruction like never seen before, it will deceive many and take many captive in order to enslave them into worshipping it and abiding in lawlessness. We can only destroy ourselves but with God all things are possible. God knows us better because He is our Creater and He knows our beginning and our end. Our prove text is taken from the book of John 5:31-44, 2 Thessalonians 2:1-12, Daniel 7-9, Revelation 13-15, Matthew 24-25 and Luke 21. Let us watch and pray... God bless you as you share this message to others.
Very informative but PLEASE stop every now and then for a breather
If you say or read what you need to say all in a single line it sounds very monotone and lord knows 2023 human brains cannot consume more than 50 seconds of content like this lol
Awesome! Keep going!!🥳🤩🦾
I tried all the questions in the notebook but the assistants could solve only the first question.
Questions.
1. What is today's date?
2. Compare the year-to-date gain for META and TESLA.
3. Plot a chart of their stock price change YTD and save to stock_price_ytd.png.