The Next Step in Agent Development: Introducing Control Flow

I recently read the article ‘Agents need control flow, not more prompts’ and deeply resonated with its message. When initially developing LLM-based agents, prompt engineering felt like the entirety of development. However, as systems became more complex and we started dealing with multi-agent runtimes like ZeroClaw, I keenly realized that ‘structure,’ not ‘prompts,’ is the core.

In this article, I’ll share how to introduce Control Flow into your architecture to build scalable agent systems, reducing complex prompts and providing practical code examples.

Problem Definition: Prompt Limitations and ‘Dependency Hell’

It’s challenging to instruct a single agent with a prompt like “Do task A, and if the result is positive, execute B; if negative, execute C, and finally summarize D.” LLMs frequently lose context or ignore logical branches (if/else) and proceed directly.

As discussed in the design proposal for the [ZeroClaw] multi-agent architecture, especially in collaborative environments involving multiple agents, relying solely on prompts leads to the following issues:

Non-determinism: The LLM might choose different paths for the same input.
Debugging Difficulty: It’s hard to pinpoint which part of the prompt is causing errors when they occur.
Lack of Scalability: Adding new steps requires modifying the existing, massive prompt each time.

Solution: Graph-based Control Flow

The solution is to let the LLM decide “What” to do, and delegate “How” to execute it to an external state machine or graph workflow. This allows us to control the data flow and execution order between agents, rather than controlling the agents’ thinking process itself.

Building such a runtime in a stable language like Rust can achieve the high performance and reliability targeted by [ZeroClaw].

Implementation Example: Creating a Simple State Machine Runner with Rust

Without complex frameworks, let’s implement a simple agent workflow using Rust’s Enum. This code defines the steps (Step) an agent needs to perform and structures how the next step is determined based on the result of the previous one.

// 1. Define the agent's state and execution context
#[derive(Debug, Clone)]
struct AgentContext {
    current_data: String,
    step_count: usize,
}

// 2. Define each step of the workflow
enum WorkflowStep {
    Initialize,
    ProcessData,
    ValidateResult, // The next step is determined by the result
    Finalize,
    Complete,
}

// 3. Implement the execution logic for each step
impl WorkflowStep {
    fn execute(&self, ctx: &mut AgentContext) -> WorkflowStep {
        match self {
            WorkflowStep::Initialize => {
                println!("[Step 1] Initializing data...");
                ctx.current_data = "Sample Input Data".to_string();
                ctx.step_count += 1;
                WorkflowStep::ProcessData
            },
            WorkflowStep::ProcessData => {
                println!("[Step 2] Processing data via LLM: {}", ctx.current_data);
                // In a real scenario, this would involve calling an MCP client etc. for LLM inference
                ctx.current_data = format!("Processed: {}", ctx.current_data);
                ctx.step_count += 1;
                WorkflowStep::ValidateResult
            },
            WorkflowStep::ValidateResult => {
                println!("[Step 3] Validating result...");
                // Business logic: e.g., if the result length is less than 10, restart (or handle error)
                if ctx.current_data.len() < 10 {
                    println!("Data is insufficient, returning to initialization step.");
                    WorkflowStep::Initialize // Loop structure
                } else {
                    println!("Validation passed. Moving to finalization step.");
                    WorkflowStep::Finalize
                }
            },
            WorkflowStep::Finalize => {
                println!("[Step 4] Saving final result: {}", ctx.current_data);
                WorkflowStep::Complete
            },
            WorkflowStep::Complete => {
                println!("Workflow complete.");
                WorkflowStep::Complete // Maintain termination state
            }
        }
    }
}

// 4. Main execution loop (Runner)
fn run_agent_workflow() {
    let mut ctx = AgentContext {
        current_data: String::new(),
        step_count: 0,
    };
    
    let mut current_step = WorkflowStep::Initialize;
    
    // Prevent infinite loops with a maximum of 10 iterations (Safety Breaker)
    for _ in 0..10 {
        if matches!(current_step, WorkflowStep::Complete) {
            break;
        }
        current_step = current_step.execute(&mut ctx);
    }
}

fn main() {
    run_agent_workflow();
}

Benefits Gained from the Code

Even with a simple match statement and Enum as in the example above, the benefits are significant.

Clear Flow Visualization: You can understand how the system flows just by looking at the code. This greatly improves maintainability when handling complex requests, like in the [Discord MCP Gateway Architecture].
State Management: Memory state is explicitly managed through the AgentContext struct. Even if the LLM’s context window overflows, the Rust runtime accurately knows the current step.
Error Handling: When a specific step fails, you can easily transition to a Retry logic or a Fallback step at the code level, rather than retrying the prompt.

Conclusion: Agents are Software

We are moving past the era of treating LLMs as ‘magic’ and entering an era where LLMs are viewed as ‘a module.’ The trial and error experienced in projects like [Cloud Monitor] and [Improving LLM Settings] ultimately show that AI must be built upon a robust software architecture.

Instead of writing longer prompts, try building a runtime that clearly defines control flow using Rust or Python. You’ll feel the predictability of your agent’s behavior and the sturdiness of your system improve.

In the next post, we will discuss how to maximize performance by processing multiple LLM calls in parallel on top of this control flow.