Surprising Insights from Cutting-Edge Prototypes | Agentic Workflow Series Part 4 of 4
Surprising Insights from Cutting-Edge Prototypes | Agentic Workflow Series Part 4 of 4
Surprising Insights from Cutting-Edge Prototypes | Agentic Workflow Series Part 4 of 4
In the ever-evolving landscape of technology, the implementation of multi-agent systems presents a promising frontier. As part of our ongoing Agentic Workflow Series, we delve into the progress and potential of this innovative design pattern in real-world applications. Whether it’s about tightening operational efficiency, enhancing strategic planning, or pushing the boundaries of what AI can do, understanding how these systems work in practice is crucial. Let’s explore some advanced prototypes and see what they tell us about the future of collaborative AI.
Understanding the Basics of Multi-Agent Systems
Multi-Agent Systems (MAS) involve multiple interacting agents within an environment. These agents can be autonomous or semi-autonomous and are designed to solve problems that are beyond the capabilities of a single agent or monolithic systems. In the realms of AI, these systems promise more dynamic and flexible solutions.
Framework Insights: How They Stack Up
Crew AI: A Platform for Orchestrated Intelligence
Crew AI introduces a robust framework for facilitating a multi-agent collaboration system. Using a platform that mimics human interaction, multiple AI "agents" such as a marketer, technologist, and business development expert come together to evaluate the viability of products. This setup encourages a comprehensive analysis by leveraging diverse perspectives, much like assembling a virtual expert team.
Utility and Cost: Implementing Crew AI involves certain costs, around 40 cents for each agent call, which is a significant consideration given that each call comprises multiple underlying interactions. Despite the costs, the potential for high return on investment through improved decision-making and problem-solving processes presents a compelling case for businesses.
AutoGen Studio 2.0: Enhancing Agent Interaction
AutoGen Studio 2.0 offers predefined models and the capability to create agent groups. This allows for complex tasks to be handled more effectively through enhanced multi-agent interactions. The interface supports group chats, which facilitates better coordination but lacks a feature to view these interactions in real-time, pushing users to rely on terminal logs.
Effectiveness and Limitations: While AutoGen Studio effectively manages multi-agent tasks, the limited visibility into real-time interactions and the potential for miscommunication pose challenges. It scores a 3.5 out of 5, standing out as a useful low-code tool yet requiring improvements for optimal use.
ChatDev: The Quirky Tool with Practical Challenges
ChatDev, developed by the innovative team at Open BMBB, brings a unique and user-friendly interface to the fore. The platform uses role-based AI agents to manage and execute tasks, displayed through a pixelated interface which adds a touch of fun to the workflow.
Practicality and Issues: Despite its appealing interface and concept, ChatDev’s practical application in serious business environments may be limited. Questions about its efficiency and effectiveness remain, particularly with the mysterious end to the showcased video without a clear conclusion on its functionality.
Multi-Agent Frameworks: A Comparison of Centralized and Decentralized Orchestration
As we delve deeper into the potential configurations for MAS, two main types of orchestrations emerge: centralized and decentralized. Centralized systems, like those used by Crew AI and ChatDev, involve a top-down approach where a central unit controls or influences agent interactions. This setup is preferable for scenarios where tight control and coordination are necessary.
On the other hand, decentralized orchestration allows agents to interact based on internal logic without relying on a central coordinator. This approach could potentially lead to more flexible and resilient systems but raises concerns about control and alignment.
Final Thoughts and Future Directions
The exploration of multi-agent systems, particularly through frameworks like Crew AI, AutoGen Studio, and ChatDev, shows promising advancements yet also highlights significant challenges. The balance between control and flexibility, cost and efficiency, and practical implementation needs continuous refinement.
As AI continues to advance rapidly, staying abreast of these developments and understanding their practical applications can provide businesses with critical competitive advantages. The journey of MAS is just beginning, and with each prototype and framework, we step closer to realizing the full potential of collaborative artificial intelligence systems.
[h3]Watch this video for the full details:[/h3]
So today I wanted to finish the series on agentic workflow design patterns. The last one in the series was all about multi agent collaboration.
There are already lots of interesting frameworks. I explored a few of them here. Also I am making lots of improvements to the content creation pipeline. I think from now on I will focus on a few areas on each video.
So topics I kinda, sorta, barely touch on
Centralized and decentralized orchestration
Orchestration and alignment
CrewAI
AutoGen Studio 2.0
ChatDev
Join My Newsletter for Weekly AI Updates? ✅
https://rebrand.ly/kh7q4tv
Need AI Consulting? ✅
https://rebrand.ly/543hphi
Experiment with Top LLMs and Agents? ✅
https://chattree.app/
USE CODE “JakeB” for 25% discount
MY LINKS
👉🏻 Subscribe: https://www.youtube.com/channel/UC2yxl7f69pfgRGGmO_HY5pA?sub_confirmation=1
👉🏻 Twitter: https://twitter.com/jakebatsuuri
👉🏻 Instagram: https://instagram.com/jakebatsuuri/
👉🏻 TikTok: https://www.tiktok.com/@jakebatsuuri
MEDIA & SPONSORSHIP INQUIRIES
https://rebrand.ly/pvumlzb
SOURCES:
https://www.youtube.com/watch?v=4ZqJSfV4818
https://www.youtube.com/watch?v=kJvXT25LkwA
https://arxiv.org/pdf/2308.00352.pdf
https://www.deepwisdom.ai/usecases
https://github.com/OpenBMB/ChatDev/blob/main/wiki.md
TIMESTAMPS:
0:00 Intro
1:00 CrewAI
4:00 AutoGen Studio 2.0
5:24 ChatDev
7:39 Orchestration
9:00 Alignment
9:27 Costs
ABOUT:
My name is Jake Batsuuri, developer who shares interesting AI experiments & products. Email me if you want my help with anything!
#metagpt #aiagents #agents #gpt #autogpt #ai #artificialintelligence #tutorial #stepbystep #openai #llm #largelanguagemodels #largelanguagemodel #chatgpt #gpt4 #machinelearning
[h3]Transcript[/h3]
today we’re going to explore how the multi-agent collaboration design pattern has been working in the field so far I have tried a lot of the early experiments within this design pattern and have Come Away with pretty disappointing results so I’m excited to look into these newer experiments and check out what they’re capable of if you have actually made anything with it please let me know in the comments in a dream world I would ideally make an app with this but I will settle for even a measly little feature because at the end of the day after all the cake and watermelon most people want to know if these babies make money because that would change everything without going off on a tangent if any of these display potential I think I will devote all my time to getting a working prototype any tool that accelerates this kind of work will have huge Returns on investment so let’s find out here are some interesting case studies that show this design pattern possibly working there are three popular Frameworks that allow you to set up multiple agents and orchestrate them chat Dev autogen Studio 2 and crew AI first crew after watching this video I learned that apparently Andre carpy dropped a YouTube bombshell about AI being all fast and no slow referring to system one and system two thinking from Daniel Conan’s book system one operates subconsciously enabling us to make quick automatic decisions system 2 in contrast is slower requiring deliberate and conscious effort to process information basically saying current llms do not possess the capacity for the Deep rational system two type problem solving we Aspire for an AI the video introduces viewers to two methods designed to simulate system 2 reasoning in AI the tree of thought prompting and the utilization of platforms like crew AI for a multi-agent collab system the video presents a practical example where three agents A marketer a technologist and a business development expert are tasked with examining the viability of a product think of it as assembling a team of AI experts like you’re in a low-budget Tech version of Oceans 11 here’s a hot tip that I really liked from this video use data from Custom tools with single source of data like emails and subreddits this will significantly improve the output’s relevance and accuracy here’s something else I learned when using GPT 4 it cost her about 40 cents for each agent call which is not the same as a single API call because agent calls make a lot of other calls in the background another thing she does is use local models using a llama because she wants privacy however she comes across two big problems these models apparently need about 16 GB of RAM for 13 billion parameter models and like 64 GB of RAM for the 70 billion model when she was running even the smallest model on her 16 gigabyte laptop apparently the laptop would just freeze the other thing is because of this constraint the different open source models would give varying amounts of crappy comprehension of the task at hand so essentially not really working the grand reveal of the performant local model it managed to scrape Reddit but the newsletter It produced had less factual accuracy than a horoscope written by a fortune cookie so close yet so far which leads me to think we might just be stuck with API calls to the almighty open AI for the time being and also I thought they didn’t use API calls for training data am I outdated on this and what about anthropic if you know please let me know in the comments lastly the video says the output was slightly better than asking zero shot but if you Beef It Up with tools then maybe it’s worth it my rating four out of five mostly for ease of use next up autogen Studio 2.0 after watching this video my takeaways for this framework were there are predefined models for agents and a capability to create agent groups for complex tasks we will see if these predefined models for agents cover custom workflows I doubt it new features include predefined models for agents and the ability to create agent teams with more than two agents allowing for more complex agent interactions here’s what Auto autogen Studio UI looks like workflows in autogen studio now support group chats enabling coordination among multiple agents for complex tasks autogen Studio does not yet support viewing the back and forth conversation between agents in the UI requiring users to check the terminal for detailed execution logs also having lots of agents in the same chat could increase the potential for errors or miscommunications unless there is a robust mechan ISM for information and task handling so essentially the framework does work it accomplishes the task while the video showcases the use of these models it does not deeply evaluate their effectiveness across different use cases more case studies or examples could bolster the claim overall score 3.5 out of five mostly because it’s a low code tool next up chat Dev after watching this video my takeaways were chat Dev was developed by the folks at open bmbb which to me sounds like w mcmb for some reason this is maybe the cutest tool of the bunch mostly because of this cute interface that gives 100% more quirkiness and 80% less resolution and also for having roles like CEO it’s literally a little AI Studio except the only software company where the CEO is likely to tell you I can’t do that Dave when you ask for a day off after initiating the project chev’s AI agents begin working together with the Project’s progress and the roles of different team members depicted in a replay via a pixelated interface who needs Silicon Valley on HBO when you can install and watch chadev and watch the drama unfold chadev has three key components roles types of Agents phases stages of a task and chat chains procedures involving different agents pH users can also access a chat chain visualizer to see the prompts and roll chains as far as cost goes it uses about 16,000 tokens to build a Flappy Bird app the final code is in the warehouse directory so does it work I don’t know the video cuts off maybe it didn’t very suspicious overall score three out of five mostly for being cute but ultimately I don’t think it’s as practical as the other so on a side note some interesting first use cases that I saw while doing these were for Content writer editor researcher a manager creates subtasks a researcher searches online for each subtask and creates a report an editor reads the reports and provides fixes and a writer takes the reports and fixes from the editor to write an article for business researcher technologist Business Development expert or you could have a client designer programmer code reviewer and product manager honestly there’s so many interesting and cool agent groups and pipelines that you could create really your imagination is your limitation here basically every system I saw today was using some form of central orchestration the chat Dev system also appears to use centralized orchestration as evidenced by the top- down company architecture from CEO to CTO to manager to programmer Etc Crea uses centralized orchestration as well users Define agents and tasks within the crew AI platform and the platform manages the collaboration between the agents the process of agent collaboration is defined within crew Ai and currently only sequential processes are supported where the output of One agent becomes the input for the next agent the user instantiates the crew or team of Agents within the platform including all the agents and tasks and defines the process of their collaboration so it’s like a pipeline essentially however it should be possible to implement more decentralized orchestration with Lang graph agents could be designed to directly pass messages to each other based on their own internal logic rather than relying on a central router supervisors could be more passive aggregating results from sub agents but letting the sub agents decide when to act based on the shared State you could have multiple independent agent graphs communicating with each other without a strict hierarchy note on alignment how can we have control and governance over these systems when you consider the orchestration and Alignment it seems like centralized orchestration would be better because there could be a human in the loop at the center or at the top however with decentralized orchestration alignment could be more problematic to solve as these systems could go off on their own without supervision and do a bunch of stuff how expensive is to run these These are made with the current open AI pricing and under different number of calls assumptions and 1K token input and 1K token output final thoughts I still think this approach is in its infancy but with the way things are developing how fast everything is happening now I wouldn’t be surprised if there’s another breakthrough here very soon personally I want to explore Devon DEA and open Devon in the next video
