Quote of the decade “The prompt is the new fundamental unit of knowledge work” Couldn’t agree more. Each video is more useful than the last. Very few channels hit that bar. Incredible stuff. Keep it up. Cheers.
dude, i subscribed. finally a channel tha isnt just " I Can TEll IT TO GEnEratE A mEDium ARTICLE" . instead you provide proper ideas and concepts for multi layer systems design with AI that can be used as stepping stones for larger and more complex problems. Hats off to you sir.
🎯 Key Takeaways for quick navigation: 00:00 *🧠 Introduction to Agent OS Architecture* - Introducing the concept of Agent OS architecture within LLM OS. - Exploring the microarchitecture of Agent OS focusing on LPU, IO, and RAM components. - Highlighting the significance of composing reusable AI agents for immediate and long-term results. 01:06 *🛠️ Components of Agent OS Architecture* - Discussing the components of Agent OS: LPU (Language Processing Unit), IO (Input/Output), and RAM (Random Access Memory). - Explaining the role of each component in building composable and reusable AI agents. - Emphasizing the importance of structuring AI agents for adaptability and longevity. 03:00 *📦 Understanding LPU, IO, and RAM Components* - Delving deeper into the functionality of LPU, IO, and RAM within the Agent OS architecture. - Analyzing the innovation brought by LPU in differentiating AI systems. - Describing the role of IO in enabling communication between AI agents and the external world. 04:08 *💡 Components Breakdown: LPU* - Detailing the components of LPU: Model Provider, Models, and Prompt Chains. - Discussing the significance of prompt chaining and its impact on enhancing AI agent capabilities. - Highlighting the power of language processing units in generating diverse and contextually relevant responses. 07:38 *🔄 Components Breakdown: RAM* - Explaining the importance of RAM in enabling AI agents to operate on internal states and contextual information. - Illustrating how AI agents utilize RAM to update and modify their states for improved performance. - Emphasizing the role of RAM in facilitating autonomous decision-making and problem-solving. 08:36 *🛠️ Components Breakdown: IO* - Discussing the role of IO in facilitating interactions between AI agents and external tools or systems. - Introducing the concept of spyware for monitoring, debugging, and visualizing AI agent activities. - Emphasizing the importance of monitoring and measuring AI agent performance for continual improvement. 10:26 *🔍 Evaluating Architecture Effectiveness* - Assessing the effectiveness of Agent OS architecture in comparison to simplistic AI agent designs. - Analyzing the potential of Agent OS architecture in addressing future challenges and advancements in AI technology. - Discussing the importance of architectural flexibility and adaptability for long-term AI development. 11:09 *🎯 Evaluating Agent OS Capabilities* - Evaluating the capability of Agent OS architecture to perform various tasks outlined by Andrew Karpathy. - Assessing the readiness of Agent OS architecture in achieving self-improvement and customization. - Exploring the potential of Agent OS architecture in enabling seamless communication between AI agents. 13:15 *🧩 Importance of Composability* - Highlighting the critical role of composability in building modular and reusable AI systems. - Discussing the significance of composability in creating agentic workflows and driving greater efficiency. - Emphasizing the iterative nature of AI agent development and the importance of architectural scalability. 16:44 *🧩 Missing Piece: Self-Improvement* - Identifying self-improvement as a crucial missing aspect in Agent OS architecture and AI systems in general. - Exploring the concept of self-improvement within AI agents and its potential implications. - Discussing ongoing experimentation and future prospects for implementing self-improvement mechanisms in AI architectures. Made with HARPA AI
It's crazy the wisdom you drop here. while LinkedIn feels like an AI cesspool no one gets the power of using GPT-4 or any language model as the semantic transistor in a bigger application. Got into a row with people who claim Hallucinations are part of the system and an unfixable problem. I feel like we have too many cooks and no one knows the recipe at all and we're all trying to wing it! So sensible places like this will become invaluable keep up the good work!
Great video! Agreed on self-improvement. One technique that can work to enable self-improve is mentally moving the Prompts and Prompt Chains into RAM. By making them stateful and able to be updated based on user or system feedback, the system becomes capable of more self-improvement.
we can add a long term memory where we save in a database the user preferences, feedbacks etc.., we will need ram, long term memory, entity memory and finally contextual memory that will combine them all when needed
About swapping prompts or prompt chains on models or providers, I suspect that prompts and prompt chains can be highly specific to models or providers so that swapping would destabilize the system in terms of output consistency and quality. Proof: try a specific prompt chain that is engineered to work on ChatGPT 4 on any open source model. The key problem is the models don't have a standard of output (not like APIs with defined signature and schema) which makes them not likely mutually replaceable.
What would you change in Agent X that would improve its outputs? Really just a few knobs we turn to make agent perform differently, AI and ML people know these knobs and turn them often. Reward function is only a "measure" of how much you like or dislike the outputs. Please notice, I think this can thus be solved "agent-i-cally" with a teacher agent and loop between the teacher and the "improving agent". Thus auto-improvment. The architecture you presented already has this ability because it has agents. You will have to specially make this "teacher" agent and I believe there will be big money for the first to write a very good agent that can assist in improving other agents (similar to the invention of transformer, throw back to deep mind paper, attention is all you need...the attention on a reward function that is).
I think David Shapiro is thinking in this direction, too. While not explicitly using a teacher, his open source architecture has a Nexus of collected feedback, that helps agents improve.
Self-improvement IS the missing link! Thank you for a great innovative thinking video! I just cringed on the Spyware labeling though? How about WhereWare? (You know... Where did it go wrong?)
Another great video, really enjoying the content! Wondering, in a development pipeline of something like: Discover ->User Stories -> Jira Ticket (with AC, designs, etc.) -> Development (let's say for a web/mobile hybrid), what would be the 'names' of the agents that might be used? Would it be an agent (or a group of agents) called "Product Manager", one or more called "Front End", one or more called "Back End", one or more called "QA", etc.? I'm just wondering the broad classes and names that you might create as agents in the development process?
when do we get LLMOSaaS? "It's Kubernetes for the OS in an LLM context" - overheard at at coffee shop this morning, which is why I'm checking this video out today. In a few years, an LLM will be our companion. forget pets (something we should actually do) forget spouses. we've got LLMs. It's the functional programming equivalent of LLM development.
Ive been looking into all these autonomous agent system, the browsers are popular because their a lot easier and tractable. but the grail is the computer itself or then a new computer to sit on your old computer that is semantically operated. but its the CLICKING screen elements. even with REaLM from apple. I dont think the click problem is fully solved
It’s called observability. We should not be inventing our own terms for these construction AI, we should be using the ones we already have for other systems.
Love this talk!! You’ve got that grand scheme of thinking - Thinking in big picture and long term. Which is crucial for people in AI for the long haul as it’s not going anywhere, especially those who would like to have their own AGI systems like myself. Would love to join a community if you have intention to start one, together is better :) 🙏🙌🫶
Hey Dan 👋 im a fellow ai architect & engineer, and would love to chat & collab, do you have discord or something similar? about to launch my channel, wish me luck! Thanks for the inspiration and the fire content! All the best! ❤
Quote of the decade “The prompt is the new fundamental unit of knowledge work” Couldn’t agree more. Each video is more useful than the last. Very few channels hit that bar. Incredible stuff. Keep it up. Cheers.
dude, i subscribed. finally a channel tha isnt just " I Can TEll IT TO GEnEratE A mEDium ARTICLE" . instead you provide proper ideas and concepts for multi layer systems design with AI that can be used as stepping stones for larger and more complex problems. Hats off to you sir.
🎯 Key Takeaways for quick navigation:
00:00 *🧠 Introduction to Agent OS Architecture*
- Introducing the concept of Agent OS architecture within LLM OS.
- Exploring the microarchitecture of Agent OS focusing on LPU, IO, and RAM components.
- Highlighting the significance of composing reusable AI agents for immediate and long-term results.
01:06 *🛠️ Components of Agent OS Architecture*
- Discussing the components of Agent OS: LPU (Language Processing Unit), IO (Input/Output), and RAM (Random Access Memory).
- Explaining the role of each component in building composable and reusable AI agents.
- Emphasizing the importance of structuring AI agents for adaptability and longevity.
03:00 *📦 Understanding LPU, IO, and RAM Components*
- Delving deeper into the functionality of LPU, IO, and RAM within the Agent OS architecture.
- Analyzing the innovation brought by LPU in differentiating AI systems.
- Describing the role of IO in enabling communication between AI agents and the external world.
04:08 *💡 Components Breakdown: LPU*
- Detailing the components of LPU: Model Provider, Models, and Prompt Chains.
- Discussing the significance of prompt chaining and its impact on enhancing AI agent capabilities.
- Highlighting the power of language processing units in generating diverse and contextually relevant responses.
07:38 *🔄 Components Breakdown: RAM*
- Explaining the importance of RAM in enabling AI agents to operate on internal states and contextual information.
- Illustrating how AI agents utilize RAM to update and modify their states for improved performance.
- Emphasizing the role of RAM in facilitating autonomous decision-making and problem-solving.
08:36 *🛠️ Components Breakdown: IO*
- Discussing the role of IO in facilitating interactions between AI agents and external tools or systems.
- Introducing the concept of spyware for monitoring, debugging, and visualizing AI agent activities.
- Emphasizing the importance of monitoring and measuring AI agent performance for continual improvement.
10:26 *🔍 Evaluating Architecture Effectiveness*
- Assessing the effectiveness of Agent OS architecture in comparison to simplistic AI agent designs.
- Analyzing the potential of Agent OS architecture in addressing future challenges and advancements in AI technology.
- Discussing the importance of architectural flexibility and adaptability for long-term AI development.
11:09 *🎯 Evaluating Agent OS Capabilities*
- Evaluating the capability of Agent OS architecture to perform various tasks outlined by Andrew Karpathy.
- Assessing the readiness of Agent OS architecture in achieving self-improvement and customization.
- Exploring the potential of Agent OS architecture in enabling seamless communication between AI agents.
13:15 *🧩 Importance of Composability*
- Highlighting the critical role of composability in building modular and reusable AI systems.
- Discussing the significance of composability in creating agentic workflows and driving greater efficiency.
- Emphasizing the iterative nature of AI agent development and the importance of architectural scalability.
16:44 *🧩 Missing Piece: Self-Improvement*
- Identifying self-improvement as a crucial missing aspect in Agent OS architecture and AI systems in general.
- Exploring the concept of self-improvement within AI agents and its potential implications.
- Discussing ongoing experimentation and future prospects for implementing self-improvement mechanisms in AI architectures.
Made with HARPA AI
Love building from the bottom up and this video really puts that foundational Architecture in perspective. Tks
You're far away from the mainstream man ❤💯👏Hats off!!
It's crazy the wisdom you drop here. while LinkedIn feels like an AI cesspool no one gets the power of using GPT-4 or any language model as the semantic transistor in a bigger application. Got into a row with people who claim Hallucinations are part of the system and an unfixable problem. I feel like we have too many cooks and no one knows the recipe at all and we're all trying to wing it!
So sensible places like this will become invaluable keep up the good work!
Great video!
Agreed on self-improvement.
One technique that can work to enable self-improve is mentally moving the Prompts and Prompt Chains into RAM. By making them stateful and able to be updated based on user or system feedback, the system becomes capable of more self-improvement.
we can add a long term memory where we save in a database the user preferences, feedbacks etc.., we will need ram, long term memory, entity memory and finally contextual memory that will combine them all when needed
About swapping prompts or prompt chains on models or providers, I suspect that prompts and prompt chains can be highly specific to models or providers so that swapping would destabilize the system in terms of output consistency and quality. Proof: try a specific prompt chain that is engineered to work on ChatGPT 4 on any open source model. The key problem is the models don't have a standard of output (not like APIs with defined signature and schema) which makes them not likely mutually replaceable.
Look up "Long Net" for the I/O, State, and self improvement parts
Also Infini-attention research on arxiv
What would you change in Agent X that would improve its outputs? Really just a few knobs we turn to make agent perform differently, AI and ML people know these knobs and turn them often. Reward function is only a "measure" of how much you like or dislike the outputs. Please notice, I think this can thus be solved "agent-i-cally" with a teacher agent and loop between the teacher and the "improving agent". Thus auto-improvment. The architecture you presented already has this ability because it has agents. You will have to specially make this "teacher" agent and I believe there will be big money for the first to write a very good agent that can assist in improving other agents (similar to the invention of transformer, throw back to deep mind paper, attention is all you need...the attention on a reward function that is).
I think David Shapiro is thinking in this direction, too. While not explicitly using a teacher, his open source architecture has a Nexus of collected feedback, that helps agents improve.
maybe you can have a meta-prompt chain doing critic and score the result of agents performance and then improve the original prompt automatically.
@@free_thinker4958 langchain cookbook has a meta-prompt notebook that suggests the concept of self improving.
@@free_thinker4958 I think this is what DSPy is trying to do at the prompt level
This is real content. Thanks. Looking forward to seeing how you structure these systems in code.
I like what you’re saying and I believe you will have a successful channel if you deliver on what you said in this video. I subscribed!
Everything on the right side of the arch diagram makes sense, I think the left side needs some more definition or rewording.
thanks for this!
Agent Self improvement is done with reinforcement learning algorithms. Thoughts?
An intelligent memory management system is enough for now i think
Self-improvement IS the missing link! Thank you for a great innovative thinking video!
I just cringed on the Spyware labeling though?
How about WhereWare? (You know... Where did it go wrong?)
Another great video, really enjoying the content! Wondering, in a development pipeline of something like:
Discover ->User Stories -> Jira Ticket (with AC, designs, etc.) -> Development (let's say for a web/mobile hybrid), what would be the 'names' of the agents that might be used? Would it be an agent (or a group of agents) called "Product Manager", one or more called "Front End", one or more called "Back End", one or more called "QA", etc.? I'm just wondering the broad classes and names that you might create as agents in the development process?
The term „Spyware“ is very misleading. Auditing would fit much better.
Auditware…. I like the sound of that. Copyright that 😂
Or just “Monitor“ 🤷
Hell yes. 🎉
One thing we need is an agentic workflow across robots - carry this stuff over into the robot OSes.
Great video. It'd be great to create an open source basic template structure of this architecture to use it as the base for any new projects.
Wow this was fire 🎉
when do we get LLMOSaaS?
"It's Kubernetes for the OS in an LLM context" - overheard at at coffee shop this morning, which is why I'm checking this video out today.
In a few years, an LLM will be our companion.
forget pets (something we should actually do)
forget spouses.
we've got LLMs.
It's the functional programming equivalent of LLM development.
Is there a github repo?
What about hardware by Cerebras Systems and Extropic in comparison to Groq?
Ive been looking into all these autonomous agent system, the browsers are popular because their a lot easier and tractable. but the grail is the computer itself or then a new computer to sit on your old computer that is semantically operated. but its the CLICKING screen elements. even with REaLM from apple. I dont think the click problem is fully solved
make a video where ai can search on multiple sites for answers
Observability. Not spyware, observability. But great video.
100% agree, spyware just sounds silly
It’s called observability. We should not be inventing our own terms for these construction AI, we should be using the ones we already have for other systems.
Lesss goooo!!! 🎉
Confusing terms now that we have a Groq LPU in the zeitgeist
where do you think we can deploy this agents online to serve other users? can we use docker containers as agent microservices?
Yes, docker is recommended, build the backend using a framwork such as crewai and for the front end you can use nextjs or any other front end option
I agree on composability. i guess we don't have any open source project like this yet. right? I was about to dive into crewai.
Dive into crewai, i think it's the most suitable framework especially that there are updates frequently
🔥🔥🔥
Excellent!
besides the good content: the mouse wiggling and shaking hands in the background doesn’t add any benefit.
Either this dude doesn't know what is Micro Architecture, or made a malicious clickbait.
Love this talk!! You’ve got that grand scheme of thinking - Thinking in big picture and long term. Which is crucial for people in AI for the long haul as it’s not going anywhere, especially those who would like to have their own AGI systems like myself. Would love to join a community if you have intention to start one, together is better :) 🙏🙌🫶
Hey Dan 👋 im a fellow ai architect & engineer, and would love to chat & collab, do you have discord or something similar? about to launch my channel, wish me luck!
Thanks for the inspiration and the fire content! All the best! ❤