Exploring AI Agent Frameworks: crewAI and LangChain as AI Agent Frameworks

We’re peppered with choices in the AI world today—it’s enough to drive some of us in the development and consulting space quite mad! It’s not just the constant release of new and improved LLMs either, but also the development tools we utilize to create applications with them. In this case, we mean AI agents. Choosing the right AI agent framework is crucial for developers and businesses aiming to leverage AI capabilities beyond simple chatbot or RAG applications.

While we at Expected X have championed LangChain, new entrants are constantly adding options that promise extended capabilities, greater abstraction, ease-of-use, or all of the above. This article delves into a comparative analysis of two prominent frameworks: crewAI and LangChain. Each offers unique features and functionalities, making them suitable for various applications across different industries.

What is crewAI?

First, we should note that this article was originally titled with “crewAI vs. LangChain” but crewAI is actually built on top of LangChain so it’s a bit misleading. crewAI is a comprehensive AI agent framework designed to simplify the deployment of AI applications across multiple platforms, which sounds a lot like LangChain’s raison d'être. The difference is that crewAI focuses on agent collaboration and role-playing.

So, let’s break each of these down to get a better understanding of what crewAI is trying to accomplish with its platform:

• Collaboration: While crewAI’s underlying model is designed for modularity in the form of “AI building blocks,” it excels in task delegation between agents. While agents may be assigned a “role” (more on this below) for completing a given task, they are enabled to work together to achieve their goals.

• Role-Playing: Think of this exactly as you would like building a team of humans in your company to perform a given function with a desired output. What types of skill sets would you need for your given application? Social media marketer? Data engineer? crewAI also enables hierarchical agent structures where agents can play the role of “managers” for delegating between agents, planning work execution, and validating agent output.

Core components of crewAI

https://github.com/joaomdmoura/crewAI/blob/main/docs/crewAI-mindmap.png

crewAI documentation is still under development, but from what we’ve seen, there is building interest among the GitHub OS community. crewAI currently has been starred 14.5K times and forked 1.9K times with minor/bugfix releases happening weekly if not daily. As a sidebar: while the OS world has typically dominated the AI/ML software space for years, it’s always wise to monitor the popularity of a given product among the community since updates can become fewer if interest wanes and shifts to the next “shiny new object.” Let’s look more granularly at the components of crewAI:

• Agents: In essence, these are the same as LangChain agents. These are the members of your AI team given a specific role to play as described above.

• Tasks: Tasks are what agents are assigned to perform. In crewAI, tasks can be performed by multiple agents and coordinated between agents by orchestration agents like in the “manager” role described above.

• Tools: How is an agent supposed to complete an assigned task? The tools provided by crewAI (and also those available in LangChain’s toolkit) enable agents with the means to perform functions like web search, RAG, or integrating with GCP, for example.

• Processes: Processes determine the framework for agent task completion. Processes can be sequential in nature (“Perform Task A then perform task B…”) or hierarchical (“Perform Task A, check output, determine quality of output, if output is acceptable, perform Task B, otherwise assign Task A back to agent 1, if agent 2 is available, partition Task A to agents 1 and 2,” and so on…). While not yet available in crewAI, collaborative decision-making between agents is yet another process framework planned in future releases. This looks more like a democratic model that foregoes a “manager” agent and allows all agents to “agree” how to proceed with a process.

• Crews: A crew is just a collection of agents assigned within a process to complete the tasks of your workflow.

• Memory: This is arguable the most significant feature of crewAI or any AI agent framework for that matter. Memory refers to the ability for agents to “learn” and “adapt” to contextual elements of the tasks they are assigned to perform. This is very important in the collaborative and decision-making processes between agents. For example, key entity extraction is a common use case for NLP applications. With crewAI’s memory component, a learned entity will persist through the execution of the process without spending valuable compute on constantly relearning that entity over and over again. Memory in crewAI relies on the use of underlying embedding models such as OpenAI (default) or Vertex AI’s offerings (yay!)

Agent Collaboration

So, how does crewAI work when putting all these components together? This is really the magic ingredient. crewAI manages to orchestrate each agent in a given process on its own abstracting away the need for manual configuration. While a developer can still fine-tune configuration, crewAI is designed to perform this function on its own. Really all one has to do is determine “what are the right agent roles for the task I want to perform” and crewAI does the “managerial” work. Pretty nice!

The Expected X Take…

The future of AI agent frameworks like crewAI (and LangChain) are likely to see increased usage as organizations begin to explore and develop more complex LLM applications. Since LLMs are the underpinning of crewAI, it’s also nice to see that you aren’t locked into paid for LLMs from OpenAI but can also use OS LLMs like those available on Hugging Face.

Ready to take your AI projects to the next level with crewAI? Expected X is the partner you need—contact us today!