Exploring AI Agent Frameworks: What is “Flow Engineering?”
Today, it’s practically impossible not to be distracted constantly, especially when trying to work (maybe reading this blog post is just such a distraction — we secretly hope so!). While there are many of us that proudly declare they are “professional multitaskers,” the research has shown time and time again that remaining focused on a single activity will regularly result in improved outcomes.
Psychologist Mihaly Csikszentmihalyi is attributed to popularizing the concept of “flow” and more importantly the “flow state” where an individual is considered totally focused and immersed in a single activity while undeterred or distracted by stimuli outside the given activity — in other words, this individual is “in the zone,” so to speak. Various recommendations exist for achieving this state, such as:
Balance the activity with your personal skill: Make sure the difficulty of the activity matches your skills or, even better, pushes them slightly beyond your comfort zone.
Set clear goals: Define specific goals to guide your focus and provide a sense of purpose. You can also break the activity into smaller milestones to help you feel a sense of accomplishment.
Reduce distractions: This should be obvious, but for many of us, just one more Instagram post or TikTok swipe is too enticing! To help achieve a flow state, eliminate any distractions in your environment that might be competing for your attention. Background music attuned to the activity at hand can actually help rather than distract (remember hearing how listening to Mozart can “make you smarter?” Yes, something like that).
Be sure the activity you chose is something you enjoy: Pick an activity that you find intrinsically rewarding. If the activity is a component of a larger goal, be sure to regularly envision the result you wish to achieve in the long run
Don't force it and remain stoic: Be comfortable challenging yourself and stretching your boundaries, while simultaneously releasing yourself of negative emotions that may have corrosive effects.
A 10-year longitudinal research study once showed that individuals in a flow state were 500% more productive than those that weren’t. So, what does this all have to do with Expected X? The concept of flow state, when applied to agentic AI application development, can help developers not only envisage the complete system holistically, but can also optimize the system’s overall performance. Let’s take a closer look!
What is Flow Engineering in Agentic AI?
The concept of “flow engineering” isn’t exactly new in the field of system design, and many of us probably utilize it as part of our everyday routines without even knowing it; just think about your morning routine to get ready for the day. We break down a goal (“getting ready for the day”) into a series of tasks/subtasks (get up, take a shower, make coffee, iron clothes) and each of those subtasks utilizes a specialized tool to complete the work (alarm clock, shower, coffee maker, clothes iron). Many of these tasks can be completed concurrently as well (if we start the coffee maker before we shower, the coffee will brew and be ready when we get out).
This, in essence, is the concept behind agentic AI systems: creating an orchestrated graph of specialized agents to automate multifaceted processes and solve challenging problems with minimal human intervention. Flow engineering, therefore, is the design process of agentic AI and it adheres to some of the same tenets we see in the flow state described above.
There are four key considerations in flow engineering for agentic AI design:
Memory: Cache and retrieval of previous information/system states to maintain continuity of task-to-goal flow. Memory is also very important for reducing operating costs when the number of calls to an LLM can be minimized.
Planning: Predicting and creating a series of steps to complete a given task that’s part of the achieving the system’s overall goal. Much of this is handled by correctly designing prompts and prompt templates for a given LLM in the system, such as Chain-of-Thought prompts.
Tools: Providing agents access to the tools they need to complete their task. Tools can be anything from external APIs to custom functions and they can perform almost anything from dynamically writing code, querying database(s), performing web searches, etc.
Reflection: Imbuing agents with the ability to review their work, the work of other agents, and the success or failure of the system’s goal overall. Reflection is sometimes called “self-critique” and involves prompting the AI to analyze its previous outputs and decisions critically. By doing so, the agent can identify mistakes or areas for improvement, which fosters a more methodical approach to problem-solving, akin to human "System 2" thinking, which is characterized by deliberation and analysis. Reflection “closes the loop” and allows systems to “learn” how to improve, which also impacts memory, planning, and tools. This is probably the most “AI” part of agentic AI and flow engineering.
Comparing flow engineering to human flow states exhibits some overlap:
Balance the activity with your personal skill: Utilizing LLMs that excel at an agent’s task (e.g., code creation) rather than one, “general purpose” LLM (e.g., GPT-4o).
Set clear goals: Agentic systems do not function without a goal, but even when given one, if it isn’t clearly defined, the results will be suboptimal. Much of this is controlled by prompts and prompt templates (i.e., prompt engineering).
Reduce distractions: Agents should adhere to the “single-responsibility principle” similar to the SOLID object-oriented design paradigm. Each agent should be designed to be very good at one thing and one thing only.
Flow Engineering Evaluation
Developing agentic AI systems using the tenets of flow engineering requires evaluation, not unlike that found in machine learning development:
How well does our system perform on a given task/goal? This is obviously specific to the given application, but when possible, utilize quantitative metrics. If your system is producing natural language output, there are a multitude of benchmark evaluation data sets available as well.
What are each agent’s strengths and weaknesses? After determining these, the agent’s underlying LLM can be adjusted using hyperparameters (i.e., temperature) or by using a different LLM altogether.
How quickly can our system accomplish its given goal? Again, depending on the application, some systems require results more quickly than others. For example, if you were to build an agentic AI system for use in triaging a patient in a medical situation, you might need output very quickly as opposed to using a system for generating daily reports for your market research business.
What is the cost per execution, and how does that impact the scalability of the system in a production environment? This might be one of the biggest considerations when using flow engineering for agentic AI system design. While the cost of utilizing foundational LLMs has been dropping rapidly lately, agentic systems still can make tens if not hundreds of calls to LLMs over the course of execution. Finding ways to reduce this (such as prompt caching) is absolutely critical during the flow engineering process.
The Expected X Take…
Today, there really doesn’t seem to be any way around developing agentic AI systems without utilizing flow engineering. Thankfully, there are a variety of tools popping up that are making things easier and keeping flow engineering tenets front and center. For example, Microsoft’s Promptflow and ControlFlow are just two examples illustrating how agentic AI (and therefore flow engineering) is receiving serious attention as the new paradigm for software development.
Ready to take your AI projects to the next level with agentic AI? Expected X is the partner you need—contact us today!
