IPC on Yarang's Tech Lair

High-Performance IPC Channel Optimization for ZeroClaw Agents: Rust Zero-Copy Strategy

Sun, 31 May 2026 09:01:25 +0900

Bottlenecks in Multi-Agent Systems

While developing the ZeroClaw multi-agent runtime, the biggest performance bottleneck has unequivocally been ‘inter-agent communication’. In our designed architecture, multiple dedicated agents (Workers) communicate with the main Hub to distribute tasks.

The initial design used simple JSON serialization and standard streams. However, as throughput increased, memory copying and serialization overhead began to pose problems. Specifically, latency was detected when relaying token streams of large models like LLMs in real-time, or when transferring large volumes of logs to file system agents.

This post introduces how we significantly improved inter-agent communication performance by leveraging Rust’s powerful features to implement Zero-Copy and Shared Memory.

Problem Diagnosis: Serialization and Copy Overhead

The existing communication method followed this flow:

Data Generation: Agent A creates a data struct.
Serialization: Converts to JSON using serde_json::to_string, etc. (consumes CPU resources).
Transmission: Sends byte streams via IPC channels (sockets, etc.).
Reception and Parsing: Agent B receives bytes and parses them using serde_json::from_str (consumes CPU resources).

In this process, data is copied at least three times between memory spaces. The cost of data on the heap being moved to the stack or buffer reallocations, due to Rust’s safety guarantees, was non-negligible.

Solution: Rust-based Zero-Copy IPC Design

We introduced serde’s zero_copy feature and the bytes::Bytes crate into ZeroClaw’s communication layer to eliminate unnecessary copies.

1. Buffer Management using `Bytes` and `Arc`

Rust’s bytes::Bytes is based on Arc (Atomic Reference Counting). When transferring data ownership, it copies only the pointer and metadata, not the data itself. This allows multiple agents to safely reference data in the same memory region.

use bytes::{Bytes, BytesMut, BufMut};
use serde::{Serialize, Deserialize};

// Message struct to be transmitted between agents
#[derive(Debug, Deserialize, Serialize)]
pub struct AgentMessage {
 pub id: u64,
 pub payload: Bytes, // Stores raw data
}

impl AgentMessage {
 // Directly wraps Bytes read from the network or file
 pub fn from_bytes(id: u64, data: Bytes) -> Self {
 Self { id, payload: data }
 }
}

2. Shared Memory IPC (IPC Channel) Implementation

Beyond simple byte transmission, for high performance, we can also consider using OS-level shared memory. Rust’s ecosystem provides the shared_memory crate for this. However, here we will apply a method to maintain Zero-Copy over the more general tokio::sync::mpsc channel.

use tokio::sync::mpsc;
use std::sync::Arc;

// Agent A (Sender)
pub async fn producer_task(tx: mpsc::Sender<AgentMessage>) {
 let large_data = vec![0u8; 8192]; // Example: 8KB data
 // Convert to BytesMut, then freeze to create immutable Bytes (wrapped in Arc)
 let shared_bytes = BytesMut::from(&large_data[..]).freeze();
 
 let msg = AgentMessage::from_bytes(1, shared_bytes);
 
 // When sending via tx, only the pointer within msg.payload (Bytes) is copied.
 // The actual 8KB data is not copied (Zero-Copy).
 tx.send(msg).await.unwrap();
}

// Agent B (Receiver)
pub async fn consumer_task(mut rx: mpsc::Receiver<AgentMessage>) {
 while let Some(msg) = rx.recv().await {
 // Here, msg.payload is a reference pointing to the original data.
 println!("Received message ID: {}, Payload Len: {}", msg.id, msg.payload.len());
 
 // Can be directly written to disk or sent over the network without further processing.
 // save_to_disk(msg.payload).await;
 }
}

The core of this code is that even when Bytes transfers ownership, it internally shares the heap data via Arc. This means that when tx.send() is called, the 8KB array is not copied; instead, the Arc’s count is incremented, and only the pointer is passed.

Performance Comparison and Measurement

To compare before and after improvements, we conducted benchmarks using Criterion.

Environment: Apple M1 Pro, 16GB RAM
Scenario: Transmitting 1MB of payload 10,000 times

Category	Before Improvement (Vec Clone)	After Improvement (Bytes Zero-Copy)	Performance Improvement
Time Taken	2,450ms	320ms	Approx. 7.6x
Memory Usage	Peak 2.1GB	Stable 150MB	Approx. 14x Reduction

As the data size increases (e.g., for LLM context transfer), the effect of Zero-Copy becomes maximized. The original method caused CPU spikes due to allocation/deallocation, while the improved version showed stable resource usage.

Conclusion: Completing ZeroClaw’s High-Performance Architecture

The Zero-Copy strategy, utilizing Rust’s ownership system with Bytes and Arc, is essential for multi-agent runtimes like ZeroClaw. Beyond simply being ‘fast’, it allows for more efficient use of server resources, enabling the execution of more agents concurrently.

In the future, the ZeroClaw project plans to further abstract this IPC layer and develop a macro that automatically generates optimized communication code simply by using #[derive(AgentMessage)], without the user needing to know the internal Rust implementation.

We hope this experience is helpful for those building high-performance Rust servers, and we recommend applying it to your projects, with actual code examples provided.

Reference Code Repository: ZeroClaw GitHub Repository Related Post: Introducing ZeroClaw - A High-Performance Rust Agent Runtime

[Multi-Agent] File-Based Architecture Design

Fri, 27 Feb 2026 09:30:57 +0900

File-Based Multi-Agent Architecture Design

Status: Design Proposal
Author: multi-agent-architect
Date: 2026-02-24
Type: Alternative Architecture (Process-Per-Agent)

Executive Summary

This document defines a file-based, process-per-agent architecture where each agent is defined as a separate configuration file and runs as an independent process. Agents are invoked on-demand and report results back to a main coordinator agent.

Core Principles

File-Based Agent Definition: Each agent is a .toml file in agents/
Process Isolation: Each agent runs in its own process
On-Demand Invocation: Main agent spawns subprocess agents as needed
Reporting Protocol: Structured result reporting via IPC
Zero-Config Discovery: Auto-discovery of agent definitions

1. Agent Definition File Structure

1.1 Directory Layout

~/.zeroclaw/
├── config.toml # Main configuration
├── agents/ # Agent definitions directory
│ ├── researcher.toml # Research agent
│ ├── coder.toml # Code generation agent
│ ├── tester.toml # Testing agent
│ ├── reviewer.toml # Code review agent
│ └── summarizer.toml # Summarization agent
├── agents.d/ # Optional: additional agent dirs
│ └── custom/
│ └── my_agent.toml
└── workspace/ # Shared workspace

1.2 Agent File Schema

# agents/researcher.toml

# Agent metadata
[agent]
id = "researcher"
name = "Research Agent"
version = "1.0.0"
description = "Conducts research on given topics using web search and knowledge bases"

# Execution configuration
[agent.execution]
# How to run this agent: "subprocess" | "wasm" | "docker"
mode = "subprocess"

# Command to spawn (template variables: {agent_id}, {workspace}, {config_dir})
command = "zeroclaw"
args = [
 "agent",
 "run",
 "--agent-id", "{agent_id}",
 "--config", "{config}/agents/researcher.toml",
 "--workspace", "{workspace}"
]

# Working directory for the subprocess
working_dir = "{workspace}"

# Environment variables for the subprocess
[agent.execution.env]
ZEROCLAW_AGENT_MODE = "worker"
ZEROCLAW_AGENT_ID = "researcher"

# Provider configuration (overrides main config)
[provider]
name = "openrouter"
model = "anthropic/claude-sonnet-4-6"
api_key = null # Inherit from main, or set agent-specific key
temperature = 0.3
max_tokens = 4096

# Tools available to this agent
[[tools]]
name = "web_search"
enabled = true

[[tools]]
name = "web_fetch"
enabled = true

# Tools explicitly denied to this agent
[[tools.deny]]
name = "shell"
reason = "Research agent should not execute shell commands"

# System prompt
[system]
prompt = """
You are a Research Agent. Your role is to:
1. Search for and gather information from credible sources
2. Synthesize findings into structured reports
3. Cite sources and provide references
4. Avoid speculation - stick to verified information
"""

# Memory configuration
[memory]
backend = "shared" # "shared" | "isolated"
category = "research"

# Reporting configuration
[reporting]
mode = "ipc"
format = "json" # "json" | "markdown" | "both"
timeout_seconds = 300

# Retry configuration
[retry]
max_attempts = 3
backoff_ms = 1000

2. Agent Registry

The AgentRegistry discovers and manages agent definitions:

pub struct AgentRegistry {
 agents_dir: PathBuf,
 agents: HashMap<String, AgentDefinition>,
 security: Arc<SecurityPolicy>,
}

impl AgentRegistry {
 pub fn new(agents_dir: PathBuf, security: Arc<SecurityPolicy>) -> Result<Self>;
 pub fn discover(&mut self) -> Result<()>;
 pub fn get(&self, id: &str) -> Option<&AgentDefinition>;
 pub fn list(&self) -> Vec<String>;
}

3. Inter-Process Communication (IPC)

Task Message (Main → Agent)

pub struct AgentTask {
 pub task_id: String,
 pub from_agent: String,
 pub to_agent: String,
 pub prompt: String,
 pub context: HashMap<String, serde_json::Value>,
 pub input: Option<serde_json::Value>,
 pub timestamp: chrono::DateTime<chrono::Utc>,
 pub deadline: Option<chrono::DateTime<chrono::Utc>>,
}

Result Message (Agent → Main)

pub struct AgentResult {
 pub task_id: String,
 pub agent_id: String,
 pub status: AgentStatus,
 pub data: Option<serde_json::Value>,
 pub output: String,
 pub error: Option<String>,
 pub metrics: AgentMetrics,
 pub artifacts: Vec<Artifact>,
 pub timestamp: chrono::DateTime<chrono::Utc>,
}

4. Security Considerations

Execution Mode	Isolation	Use Case
Subprocess	Process-level	Trusted agents
Docker	Container	Untrusted, file operations
Wasm	Memory-only	High security needs

5. CLI Commands

# List available agents
zeroclaw agent list

# Show agent details
zeroclaw agent show <agent_id>

# Run an agent directly (for testing)
zeroclaw agent run --agent-id researcher --prompt "Research X"

# Create a new agent from template
zeroclaw agent create --id my-agent --name "My Agent"

# Validate agent definition
zeroclaw agent validate <agent_id>

6. Success Criteria

Agent definition files validated on load
Subprocess agents spawn and complete tasks
IPC protocol supports tasks >1GB data
Main agent can invoke worker agents via tool
Failed agents don’t crash main agent
Security policy enforced per-agent
CLI commands for agent management

Korean Version: 한국어 버전